Chapter 15. Separating Source and Build Trees: Variant Directories

It is often useful to keep built files completely separate from the source files. Two main benefits are the ability to have different configurations simultaneously without build conflicts, and being version-control friendly.

Consider if you have a project to build an embedded software system for a variety of different controller hardware. The system is able to share a lot of code, so it makes sense to use a common source tree, but certain build options in the source code and header files differ. For a regular in-place build, the build outputs go in the same place as the source code. If you build Controller A first, followed by Controller B, on the Controller B build everything that uses different build options has to be rebuilt since those objects will be different (the build lines, including preprocessor defines, are part of SCons's out-of-date calculation for this reason). If you go back and build for Controller A again, things have to be rebuilt again for the same reason. However, if you can separate the locations of the output files, so each controller has its own location for build outputs, this problem can be avoided.

Having a separated build tree also helps you keep your source tree clean - there is less chance of accidentally checking in build products to version control that were not intended to be checked in. You can add a separated build directory to your version control system's list of items not to track. You can even remove the whole build tree with a single command without risking removing any of the source code.

The key to making this separation work is the ability to do out-of-tree builds: building under a separate root than the sources being built. You set up out of tree builds by establishing what SCons calls a variant directory, a place where you can build a single variant of your software (of course you can define more than one of these if you need to). Since SCons tracks targets by their path, it is able to distinguish build products like build/A/network.obj of the Controller A build from build/B/network.obj of the Controller B build, thus avoiding conflicts.

SCons provides two ways to establish variant directories, one through the SConscript function that we have already seen, and the second through a more flexible VariantDir function.

The variant directory mechanism does support doing multiple builds in one invocation of SCons, but the remainder of this chapter will focus on setting up a single build. You can combine these techniques with ones from the previous chapter and elsewhere in this Guide to set up more complex scenarios.

Note

The VariantDir function used to be called BuildDir, a name which was changed because it turned out to be confusing: the SCons functionality differs from a familiar model of a "build directory" implemented by certain other build systems like GNU Autotools. You might still find references to the old name on the Internet in postings about SCons, but it no longer works.

15.1. Specifying a Variant Directory Tree as Part of an SConscript Call

The most straightforward way to establish a variant directory tree relies on the fact that the usual way to set up a build hierarchy is to have an SConscript file in the source directory. If you pass a variant_dir argument to the SConscript function call:

SConscript('src/SConscript', variant_dir='build')
      

SCons will then build all of the files in the build directory:

% ls src
SConscript  hello.c
% scons -Q
cc -o build/hello.o -c build/hello.c
cc -o build/hello build/hello.o
% ls src
SConscript  hello.c
% ls build
SConscript  hello  hello.c  hello.o

No files were built in src: the object file build/hello.o and the executable file build/hello were built in the build directory, as expected. But notice that even though our hello.c file actually lives in the src directory, SCons has compiled a build/hello.c file to create the object file, and that file is now seen in build.

You can ask SCons to show the dependency tree to illustrate a bit more:

% scons -Q --tree=prune
cc -o build/hello.o -c build/hello.c
cc -o build/hello build/hello.o
+-.
  +-SConstruct
  +-build
  | +-build/SConscript
  | +-build/hello
  | | +-build/hello.o
  | |   +-build/hello.c
  | +-build/hello.c
  | +-[build/hello.o]
  +-src
    +-src/SConscript
    +-src/hello.c

What's happened is that SCons has duplicated the hello.c file from the src directory to the build directory, and built the program from there (it also duplicated SConscript). The next section explains why SCons does this.

The nice thing about the SConscript approach is it is almost invisible to you: this build looks just like an ordinary in-place build except for the extra variant_dir argument in the SConscript call. SCons handles all the path adjustments for the out of tree build directory while it processes that SConscript file.