SCons User Guide 0.98.2 Steven Knight Copyright © 2004, 2005, 2006, 2007 Steven Knight SCons User's Guide Copyright (c) 2004, 2005, 2006, 2007 Steven Knight __________________________________________________________________ Table of Contents [1]Preface 1. [2]SCons Principles 2. [3]A Caveat About This Guide's Completeness 3. [4]Acknowledgements 4. [5]Contact 1. [6]Building and Installing SCons 1.1. [7]Installing Python 1.2. [8]Installing SCons From Pre-Built Packages 1.2.1. [9]Installing SCons on Red Hat (and Other RPM-based) Linux Systems 1.2.2. [10]Installing SCons on Debian Linux Systems 1.2.3. [11]Installing SCons on Windows Systems 1.3. [12]Building and Installing SCons on Any System 1.3.1. [13]Building and Installing Multiple Versions of SCons Side-by-Side 1.3.2. [14]Installing SCons in Other Locations 1.3.3. [15]Building and Installing SCons Without Administrative Privileges 2. [16]Simple Builds 2.1. [17]Building Simple C / C++ Programs 2.2. [18]Building Object Files 2.3. [19]Simple Java Builds 2.4. [20]Cleaning Up After a Build 2.5. [21]The SConstruct File 2.5.1. [22]SConstruct Files Are Python Scripts 2.5.2. [23]SCons Functions Are Order-Independent 2.6. [24]Making the SCons Output Less Verbose 3. [25]Less Simple Things to Do With Builds 3.1. [26]Specifying the Name of the Target (Output) File 3.2. [27]Compiling Multiple Source Files 3.3. [28]Making a list of files with Glob() 3.4. [29]Specifying Single Files Vs. Lists of Files 3.5. [30]Making Lists of Files Easier to Read 3.6. [31]Keyword Arguments 3.7. [32]Compiling Multiple Programs 3.8. [33]Sharing Source Files Between Multiple Programs 4. [34]Building and Linking with Libraries 4.1. [35]Building Libraries 4.1.1. [36]Building Libraries From Source Code or Object Files 4.1.2. [37]Building Static Libraries Explicitly: the StaticLibrary Builder 4.1.3. [38]Building Shared (DLL) Libraries: the SharedLibrary Builder 4.2. [39]Linking with Libraries 4.3. [40]Finding Libraries: the $LIBPATH Construction Variable 5. [41]Node Objects 5.1. [42]Builder Methods Return Lists of Target Nodes 5.2. [43]Explicitly Creating File and Directory Nodes 5.3. [44]Printing Node File Names 5.4. [45]Using a Node's File Name as a String 6. [46]Dependencies 6.1. [47]Deciding When an Input File Has Changed: the Decider Function 6.1.1. [48]Using MD5 Signatures to Decide if a File Has Changed 6.1.2. [49]Using Time Stamps to Decide If a File Has Changed 6.1.3. [50]Deciding If a File Has Changed Using Both MD Signatures and Time Stamps 6.1.4. [51]Writing Your Own Custom Decider Function 6.1.5. [52]Mixing Different Ways of Deciding If a File Has Changed 6.2. [53]Older Functions for Deciding When an Input File Has Changed 6.2.1. [54]The SourceSignatures Function 6.2.2. [55]The TargetSignatures Function 6.3. [56]Implicit Dependencies: The $CPPPATH Construction Variable 6.4. [57]Caching Implicit Dependencies 6.4.1. [58]The --implicit-deps-changed Option 6.4.2. [59]The --implicit-deps-unchanged Option 6.5. [60]Explicit Dependencies: the Depends Function 6.6. [61]Ignoring Dependencies: the Ignore Function 6.7. [62]The AlwaysBuild Function 7. [63]Construction Environments 7.1. [64]Multiple Construction Environments 7.2. [65]Copying Construction Environments 7.3. [66]Fetching Values From a Construction Environment 7.4. [67]Expanding Values From a Construction Environment 7.5. [68]Modifying a Construction Environment 7.5.1. [69]Replacing Values in a Construction Environment 7.5.2. [70]Appending to the End of Values in a Construction Environment 7.5.3. [71]Appending to the Beginning of Values in a Construction Environment 8. [72]Controlling the External Environment Used to Execute Build Commands 8.1. [73]Propagating PATH From the External Environment 9. [74]Controlling a Build From the Command Line 9.1. [75]Not Having to Specify Command-Line Options Each Time: the SCONSFLAGS Environment Variable 9.2. [76]Getting at Command-Line Targets 9.3. [77]Controlling the Default Targets 9.3.1. [78]Getting at the List of Default Targets 9.4. [79]Getting at the List of Build Targets, Regardless of Origin 9.5. [80]Command-Line variable=value Build Options 9.6. [81]Controlling Command-Line Build Options 9.7. [82]Providing Help for Command-Line Build Options 9.8. [83]Reading Build Options From a File 9.9. [84]Canned Build Options 9.9.1. [85]True/False Values: the BoolOption Build Option 9.9.2. [86]Single Value From a List: the EnumOption Build Option 9.9.3. [87]Multiple Values From a List: the ListOption Build Option 9.9.4. [88]Path Names: the PathOption Build Option 9.9.5. [89]Enabled/Disabled Path Names: the PackageOption Build Option 9.10. [90]Adding Multiple Command-Line Build Options at Once 10. [91]Providing Build Help: the Help Function 11. [92]Installing Files in Other Directories: the Install Builder 11.1. [93]Installing Multiple Files in a Directory 11.2. [94]Installing a File Under a Different Name 11.3. [95]Installing Multiple Files Under Different Names 12. [96]Platform-Independent File System Manipulation 12.1. [97]Copying Files or Directories: The Copy Factory 12.2. [98]Deleting Files or Directories: The Delete Factory 12.3. [99]Moving (Renaming) Files or Directories: The Move Factory 12.4. [100]Updating the Modification Time of a File: The Touch Factory 12.5. [101]Creating a Directory: The Mkdir Factory 12.6. [102]Changing File or Directory Permissions: The Chmod Factory 12.7. [103]Executing an action immediately: the Execute Function 13. [104]Preventing Removal of Targets 13.1. [105]Preventing target removal during build: the Precious Function 13.2. [106]Preventing target removal during clean: the NoClean Function 13.3. [107]Removing additional files during clean: the Clean Function 14. [108]Hierarchical Builds 14.1. [109]SConscript Files 14.2. [110]Path Names Are Relative to the SConscript Directory 14.3. [111]Top-Level Path Names in Subsidiary SConscript Files 14.4. [112]Absolute Path Names 14.5. [113]Sharing Environments (and Other Variables) Between SConscript Files 14.5.1. [114]Exporting Variables 14.5.2. [115]Importing Variables 14.5.3. [116]Returning Values From an SConscript File 15. [117]Separating Source and Build Directories 15.1. [118]Specifying a Variant Directory Tree as Part of an SConscript Call 15.2. [119]Why SCons Duplicates Source Files in a Variant Directory Tree 15.3. [120]Telling SCons to Not Duplicate Source Files in the Variant Directory Tree 15.4. [121]The VariantDir Function 15.5. [122]Using VariantDir With an SConscript File 15.6. [123]Using Glob() with VariantDir 16. [124]Variant Builds 17. [125]Writing Your Own Builders 17.1. [126]Writing Builders That Execute External Commands 17.2. [127]Attaching a Builder to a Construction Environment 17.3. [128]Letting SCons Handle The File Suffixes 17.4. [129]Builders That Execute Python Functions 17.5. [130]Builders That Create Actions Using a Generator 17.6. [131]Builders That Modify the Target or Source Lists Using an Emitter 18. [132]Not Writing a Builder: the Command Builder 19. [133]Pseudo-Builders: the AddMethod function 20. [134]Writing Scanners 20.1. [135]A Simple Scanner Example 21. [136]Building From Code Repositories 21.1. [137]The Repository Method 21.2. [138]Finding source files in repositories 21.3. [139]Finding #include files in repositories 21.3.1. [140]Limitations on #include files in repositories 21.4. [141]Finding the SConstruct file in repositories 21.5. [142]Finding derived files in repositories 21.6. [143]Guaranteeing local copies of files 22. [144]Multi-Platform Configuration (Autoconf Functionality) 22.1. [145]Configure Contexts 22.2. [146]Checking for the Existence of Header Files 22.3. [147]Checking for the Availability of a Function 22.4. [148]Checking for the Availability of a Library 22.5. [149]Checking for the Availability of a typedef 22.6. [150]Adding Your Own Custom Checks 22.7. [151]Not Configuring When Cleaning Targets 23. [152]Caching Built Files 23.1. [153]Specifying the Shared Cache Directory 23.2. [154]Keeping Build Output Consistent 23.3. [155]Not Using the Shared Cache for Specific Files 23.4. [156]Disabling the Shared Cache 23.5. [157]Populating a Shared Cache With Already-Built Files 23.6. [158]Minimizing Cache Contention: the --random Option 24. [159]Alias Targets 25. [160]Java Builds 25.1. [161]Building Java Class Files: the Java Builder 25.2. [162]How SCons Handles Java Dependencies 25.3. [163]Building Java Archive (.jar) Files: the Jar Builder 25.4. [164]Building C Header and Stub Files: the JavaH Builder 25.5. [165]Building RMI Stub and Skeleton Class Files: the RMIC Builder 26. [166]Troubleshooting 26.1. [167]Why is That Target Being Rebuilt? the --debug=explain Option 26.2. [168]What's in That Construction Environment? the Dump Method 26.3. [169]What Dependencies Does SCons Know About? the --tree Option 26.4. [170]How is SCons Constructing the Command Lines It Executes? the --debug=presub Option 26.5. [171]Where is SCons Searching for Libraries? the --debug=findlibs Option 26.6. [172]Where is SCons Blowing Up? the --debug=stacktrace Option 26.7. [173]How is SCons Making Its Decisions? the --taskmastertrace Option A. [174]Construction Variables B. [175]Builders C. [176]Tools D. [177]Handling Common Tasks List of Examples D-1. [178]Wildcard globbing to create a list of filenames D-2. [179]Filename extension substitution D-3. [180]Appending a path prefix to a list of filenames D-4. [181]Substituting a path prefix with another one D-5. [182]Filtering a filename list to exclude/retain only a specific set of extensions D-6. [183]The "backtick function": run a shell command and capture the output __________________________________________________________________ Preface Thank you for taking the time to read about SCons. SCons is a next-generation software construction tool, or make tool--that is, a software utility for building software (or other files) and keeping built software up-to-date whenever the underlying input files change. The most distinctive thing about SCons is that its configuration files are actually scripts, written in the Python programming language. This is in contrast to most alternative build tools, which typically invent a new language to configure the build. SCons still has a learning curve, of course, because you have to know what functions to call to set up your build properly, but the underlying syntax used should be familiar to anyone who has ever looked at a Python script. Paradoxically, using Python as the configuration file format makes SCons easier for non-programmers to learn than the cryptic languages of other build tools, which are usually invented by programmers for other programmers. This is in no small part due to the consistency and readability that are built in to Python. It just so happens that making a real, live scripting language the basis for the configuration files makes it a snap for more accomplished programmers to do more complicated things with builds, as necessary. __________________________________________________________________ 1. SCons Principles There are a few overriding principles we try to live up to in designing and implementing SCons: Correctness First and foremost, by default, SCons guarantees a correct build even if it means sacrificing performance a little. We strive to guarantee the build is correct regardless of how the software being built is structured, how it may have been written, or how unusual the tools are that build it. Performance Given that the build is correct, we try to make SCons build software as quickly as possible. In particular, wherever we may have needed to slow down the default SCons behavior to guarantee a correct build, we also try to make it easy to speed up SCons through optimization options that let you trade off guaranteed correctness in all end cases for a speedier build in the usual cases. Convenience SCons tries to do as much for you out of the box as reasonable, including detecting the right tools on your system and using them correctly to build the software. In a nutshell, we try hard to make SCons just "do the right thing" and build software correctly, with a minimum of hassles. __________________________________________________________________ 2. A Caveat About This Guide's Completeness One word of warning as you read through this Guide: Like too much Open Source software out there, the SCons documentation isn't always kept up-to-date with the available features. In other words, there's a lot that SCons can do that isn't yet covered in this User's Guide. (Come to think of it, that also describes a lot of proprietary software, doesn't it?) Although this User's Guide isn't as complete as we'd like it to be, our development process does emphasize making sure that the SCons man page is kept up-to-date with new features. So if you're trying to figure out how to do something that SCons supports but can't find enough (or any) information here, it would be worth your while to look at the man page to see if the information is covered there. And if you do, maybe you'd even consider contributing a section to the User's Guide so the next person looking for that information won't have to go through the same thing...? __________________________________________________________________ 3. Acknowledgements SCons would not exist without a lot of help from a lot of people, many of whom may not even be aware that they helped or served as inspiration. So in no particular order, and at the risk of leaving out someone: First and foremost, SCons owes a tremendous debt to Bob Sidebotham, the original author of the classic Perl-based Cons tool which Bob first released to the world back around 1996. Bob's work on Cons classic provided the underlying architecture and model of specifying a build configuration using a real scripting language. My real-world experience working on Cons informed many of the design decisions in SCons, including the improved parallel build support, making Builder objects easily definable by users, and separating the build engine from the wrapping interface. Greg Wilson was instrumental in getting SCons started as a real project when he initiated the Software Carpentry design competition in February 2000. Without that nudge, marrying the advantages of the Cons classic architecture with the readability of Python might have just stayed no more than a nice idea. The entire SCons team have been absolutely wonderful to work with, and SCons would be nowhere near as useful a tool without the energy, enthusiasm and time people have contributed over the past few years. The "core team" of Chad Austin, Anthony Roach, Charles Crain, Steve Leblanc, Gary Oberbrunner, Greg Spencer and Christoph Wiedemann have been great about reviewing my (and other) changes and catching problems before they get in the code base. Of particular technical note: Anthony's outstanding and innovative work on the tasking engine has given SCons a vastly superior parallel build model; Charles has been the master of the crucial Node infrastructure; Christoph's work on the Configure infrastructure has added crucial Autoconf-like functionality; and Greg has provided excellent support for Microsoft Visual Studio. Special thanks to David Snopek for contributing his underlying "Autoscons" code that formed the basis of Christoph's work with the Configure functionality. David was extremely generous in making this code available to SCons, given that he initially released it under the GPL and SCons is released under a less-restrictive MIT-style license. Thanks to Peter Miller for his splendid change management system, Aegis, which has provided the SCons project with a robust development methodology from day one, and which showed me how you could integrate incremental regression tests into a practical development cycle (years before eXtreme Programming arrived on the scene). And last, thanks to Guido van Rossum for his elegant scripting language, which is the basis not only for the SCons implementation, but for the interface itself. __________________________________________________________________ 4. Contact The best way to contact people involved with SCons, including the author, is through the SCons mailing lists. If you want to ask general questions about how to use SCons send email to users@scons.tigris.org. If you want to contact the SCons development community directly, send email to dev@scons.tigris.org. If you want to receive announcements about SCons, join the low-volume announce@scons.tigris.org mailing list. __________________________________________________________________ Chapter 1. Building and Installing SCons This chapter will take you through the basic steps of installing SCons on your system, and building SCons if you don't have a pre-built package available (or simply prefer the flexibility of building it yourself). Before that, however, this chapter will also describe the basic steps involved in installing Python on your system, in case that is necessary. Fortunately, both SCons and Python are very easy to install on almost any system, and Python already comes installed on many systems. __________________________________________________________________ 1.1. Installing Python Because SCons is written in Python, you must obviously have Python installed on your system to use SCons Before you try to install Python, you should check to see if Python is already available on your system by typing python at your system's command-line prompt. You should see something like the following on a UNIX or Linux system that has Python installed: $ python Python 2.2.2 (#1, Feb 24 2003, 19:13:11) [GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-4)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> ^D And on a Windows system with Python installed: C:\>python Python 2.2.2 (#34, Apr 9 2002, 19:34:33) [MSC 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> ^Z The >>> is the input prompt for the Python interpreter. The ^D and ^Z represent the CTRL-D and CTRL-Z characters that you will need to type to get out of the interpreter before proceeding to installing SCons. If Python is not installed on your system, you will see an error message stating something like "command not found" (on UNIX or Linux) or "'python' is not recognized as an internal or external command, operable progam or batch file" (on Windows). In that case, you need to install Python before you can install SCons. The standard location for information about downloading and installing Python is [184]http://www.python.org/download/. See that page for information about how to download and install Python on your system. __________________________________________________________________ 1.2. Installing SCons From Pre-Built Packages SCons comes pre-packaged for installation on a number of systems, including Linux and Windows systems. You do not need to read this entire section, you should only need to read the section appropriate to the type of system you're running on. __________________________________________________________________ 1.2.1. Installing SCons on Red Hat (and Other RPM-based) Linux Systems SCons comes in RPM (Red Hat Package Manager) format, pre-built and ready to install on Red Hat Linux, Fedora Core, or any other Linux distribution that uses RPM. Your distribution may already have an SCons RPM built specifically for it; many do, including SuSe, Mandrake and Fedora. You can check for the availability of an SCons RPM on your distribution's download servers, or by consulting an RPM search site like [185]http://www.rpmfind.net/ or [186]http://rpm.pbone.net/. If your Linux distribution does not already have a specific SCons RPM file, you can download and install from the generic RPM provided by the SCons project. This will install the SCons script(s) in /usr/bin, and the SCons library modules in /usr/lib/scons. To install from the command line, simply download the appropriate .rpm file, and then run: # rpm -Uvh scons-0.96-1.noarch.rpm Or, you can use a graphical RPM package manager like gnorpm. See your package manager application's documention for specific instructions about how to use it to install a downloaded RPM. __________________________________________________________________ 1.2.2. Installing SCons on Debian Linux Systems Debian Linux systems use a different package management format that also makes it very easy to install SCons. If your system is connected to the Internet, you can install the latest official Debian package by running: # apt-get install scons __________________________________________________________________ 1.2.3. Installing SCons on Windows Systems SCons provides a Windows installer that makes installation extremely easy. Download the scons-0.95.win32.exe file from the SCons download page at [187]http://www.scons.org/download.html. Then all you need to do is execute the file (usually by clicking on its icon in Windows Explorer). These will take you through a small sequence of windows that will install SCons on your system. __________________________________________________________________ 1.3. Building and Installing SCons on Any System If a pre-built SCons package is not available for your system, then you can still easily build and install SCons using the native Python distutils package. The first step is to download either the scons-0.98.2.tar.gz or scons-0.98.2.zip, which are available from the SCons download page at [188]http://www.scons.org/download.html. Unpack the archive you downloaded, using a utility like tar on Linux or UNIX, or WinZip on Windows. This will create a directory called scons-0.98.2, usually in your local directory. Then change your working directory to that directory and install SCons by executing the following commands: # cd scons-0.98.2 # python setup.py install This will build SCons, install the scons script in the default system scripts directory (/usr/local/bin or C:\Python2.2\Scripts), and will install the SCons build engine in an appropriate stand-alone library directory (/usr/local/lib/scons or C:\Python2.2\scons). Because these are system directories, you may need root (on Linux or UNIX) or Administrator (on Windows) privileges to install SCons like this. __________________________________________________________________ 1.3.1. Building and Installing Multiple Versions of SCons Side-by-Side The SCons setup.py script has some extensions that support easy installation of multiple versions of SCons in side-by-side locations. This makes it easier to download and experiment with different versions of SCons before moving your official build process to a new version, for example. To install SCons in a version-specific location, add the --version-lib option when you call setup.py: # python setup.py install --version-lib This will install the SCons build engine in the /usr/lib/scons-0.98.2 or C:\Python2.2\scons-0.98.2 directory, for example. If you use the --version-lib option the first time you install SCons, you do not need to specify it each time you install a new version. The SCons setup.py script will detect the version-specific directory name(s) and assume you want to install all versions in version-specific directories. You can override that assumption in the future by explicitly specifying the --standalone-lib option. __________________________________________________________________ 1.3.2. Installing SCons in Other Locations You can install SCons in locations other than the default by specifying the --prefix= option: # python setup.py install --prefix=/opt/scons This would install the scons script in /opt/scons/bin and the build engine in /opt/scons/lib/scons, Note that you can specify both the --prefix= and the --version-lib options at the same type, in which case setup.py will install the build engine in a version-specific directory relative to the specified prefix. Adding --version-lib to the above example would install the build engine in /opt/scons/lib/scons-0.98.2. __________________________________________________________________ 1.3.3. Building and Installing SCons Without Administrative Privileges If you don't have the right privileges to install SCons in a system location, simply use the --prefix= option to install it in a location of your choosing. For example, to install SCons in appropriate locations relative to the user's $HOME directory, the scons script in $HOME/bin and the build engine in $HOME/lib/scons, simply type: $ python setup.py install --prefix=$HOME You may, of course, specify any other location you prefer, and may use the --version-lib option if you would like to install version-specific directories relative to the specified prefix. __________________________________________________________________ Chapter 2. Simple Builds In this chapter, you will see several examples of very simple build configurations using SCons, which will demonstrate how easy it is to use SCons to build programs from several different programming languages on different types of systems. __________________________________________________________________ 2.1. Building Simple C / C++ Programs Here's the famous "Hello, World!" program in C: int main() { printf("Hello, world!\n"); } And here's how to build it using SCons. Enter the following into a file named SConstruct: Program('hello.c') This minimal configuration file gives SCons two pieces of information: what you want to build (an executable program), and the input file from which you want it built (the hello.c file). [189]Program is a builder_method, a Python call that tells SCons that you want to build an executable program. That's it. Now run the scons command to build the program. On a POSIX-compliant system like Linux or UNIX, you'll see something like: % scons scons: Reading SConscript files ... scons: done reading SConscript files. scons: Building targets ... cc -o hello.o -c hello.c cc -o hello hello.o scons: done building targets. On a Windows system with the Microsoft Visual C++ compiler, you'll see something like: C:\>scons scons: Reading SConscript files ... scons: done reading SConscript files. scons: Building targets ... cl /nologo /c hello.c /Fohello.obj link /nologo /OUT:hello.exe hello.obj scons: done building targets. First, notice that you only need to specify the name of the source file, and that SCons correctly deduces the names of the object and executable files to be built from the base of the source file name. Second, notice that the same input SConstruct file, without any changes, generates the correct output file names on both systems: hello.o and hello on POSIX systems, hello.obj and hello.exe on Windows systems. This is a simple example of how SCons makes it extremely easy to write portable software builds. (Note that we won't provide duplicate side-by-side POSIX and Windows output for all of the examples in this guide; just keep in mind that, unless otherwise specified, any of the examples should work equally well on both types of systems.) __________________________________________________________________ 2.2. Building Object Files The [190]Program builder method is only one of many builder methods that SCons provides to build different types of files. Another is the [191]Object builder method, which tells SCons to build an object file from the specified source file: Object('hello.c') Now when you run the scons command to build the program, it will build just the hello.o object file on a POSIX system: % scons scons: Reading SConscript files ... scons: done reading SConscript files. scons: Building targets ... cc -o hello.o -c hello.c scons: done building targets. And just the hello.obj object file on a Windows system (with the Microsoft Visual C++ compiler): C:\>scons scons: Reading SConscript files ... scons: done reading SConscript files. scons: Building targets ... cl /nologo /c hello.c /Fohello.obj scons: done building targets. __________________________________________________________________ 2.3. Simple Java Builds SCons also makes building with Java extremely easy. Unlike the [192]Program and [193]Object builder methods, however, the [194]Java builder method requires that you specify the name of a destination directory in which you want the class files placed, followed by the source directory in which the .java files live: Java('classes', 'src') If the src directory contains a single hello.java file, then the output from running the scons command would look something like this (on a POSIX system): % scons scons: Reading SConscript files ... scons: done reading SConscript files. scons: Building targets ... javac -d classes -sourcepath src src/hello.java scons: done building targets. We'll cover Java builds in more detail, including building Java archive (.jar) and other types of file, in [195]Chapter 25. __________________________________________________________________ 2.4. Cleaning Up After a Build When using SCons, it is unnecessary to add special commands or target names to clean up after a build. Instead, you simply use the -c or --clean option when you invoke SCons, and SCons removes the appropriate built files. So if we build our example above and then invoke scons -c afterwards, the output on POSIX looks like: % scons scons: Reading SConscript files ... scons: done reading SConscript files. scons: Building targets ... cc -o hello.o -c hello.c cc -o hello hello.o scons: done building targets. % scons -c scons: Reading SConscript files ... scons: done reading SConscript files. scons: Cleaning targets ... Removed hello.o Removed hello scons: done cleaning targets. And the output on Windows looks like: C:\>scons scons: Reading SConscript files ... scons: done reading SConscript files. scons: Building targets ... cl /nologo /c hello.c /Fohello.obj link /nologo /OUT:hello.exe hello.obj scons: done building targets. C:\>scons -c scons: Reading SConscript files ... scons: done reading SConscript files. scons: Cleaning targets ... Removed hello.obj Removed hello.exe scons: done cleaning targets. Notice that SCons changes its output to tell you that it is Cleaning targets ... and done cleaning targets. __________________________________________________________________ 2.5. The SConstruct File If you're used to build systems like Make you've already figured out that the SConstruct file is the SCons equivalent of a Makefile. That is, the SConstruct file is the input file that SCons reads to control the build. __________________________________________________________________ 2.5.1. SConstruct Files Are Python Scripts There is, however, an important difference between an SConstruct file and a Makefile: the SConstruct file is actually a Python script. If you're not already familiar with Python, don't worry. This User's Guide will introduce you step-by-step to the relatively small amount of Python you'll need to know to be able to use SCons effectively. And Python is very easy to learn. One aspect of using Python as the scripting language is that you can put comments in your SConstruct file using Python's commenting convention; that is, everything between a '#' and the end of the line will be ignored: # Arrange to build the "hello" program. Program('hello.c') # "hello.c" is the source file. You'll see throughout the remainder of this Guide that being able to use the power of a real scripting language can greatly simplify the solutions to complex requirements of real-world builds. __________________________________________________________________ 2.5.2. SCons Functions Are Order-Independent One important way in which the SConstruct file is not exactly like a normal Python script, and is more like a Makefile, is that the order in which the SCons functions are called in the SConstruct file does not affect the order in which SCons actually builds the programs and object files you want it to build.[196][1] In other words, when you call the [197]Program builder (or any other builder method), you're not telling SCons to build the program at the instant the builder method is called. Instead, you're telling SCons to build the program that you want, for example, a program built from a file named hello.c, and it's up to SCons to build that program (and any other files) whenever it's necessary. (We'll learn more about how SCons decides when building or rebuilding a file is necessary in [198]Chapter 6, below.) SCons reflects this distinction between calling a builder method like Program> and actually building the program by printing the status messages that indicate when it's "just reading" the SConstruct file, and when it's actually building the target files. This is to make it clear when SCons is executing the Python statements that make up the SConstruct file, and when SCons is actually executing the commands or other actions to build the necessary files. Let's clarify this with an example. Python has a print statement that prints a string of characters to the screen. If we put print statements around our calls to the Program builder method: print "Calling Program('hello.c')" Program('hello.c') print "Calling Program('goodbye.c')" Program('goodbye.c') print "Finished calling Program()" Then when we execute SCons, we see the output from the print statements in between the messages about reading the SConscript files, indicating that that is when the Python statements are being executed: % scons scons: Reading SConscript files ... Calling Program('hello.c') Calling Program('goodbye.c') Finished calling Program() scons: done reading SConscript files. scons: Building targets ... cc -o goodbye.o -c goodbye.c cc -o goodbye goodbye.o cc -o hello.o -c hello.c cc -o hello hello.o scons: done building targets. Notice also that SCons built the goodbye program first, even though the "reading SConscript" output shows that we called Program('hello.c') first in the SConstruct file. __________________________________________________________________ 2.6. Making the SCons Output Less Verbose You've already seen how SCons prints some messages about what it's doing, surrounding the actual commands used to build the software: C:\>scons scons: Reading SConscript files ... scons: done reading SConscript files. scons: Building targets ... cl /nologo /c hello.c /Fohello.obj link /nologo /OUT:hello.exe hello.obj scons: done building targets. These messages emphasize the order in which SCons does its work: all of the configuration files (generically referred to as SConscript files) are read and executed first, and only then are the target files built. Among other benefits, these messages help to distinguish between errors that occur while the configuration files are read, and errors that occur while targets are being built. One drawback, of course, is that these messages clutter the output. Fortunately, they're easily disabled by using the -Q option when invoking SCons: C:\>scons -Q cl /nologo /c hello.c /Fohello.obj link /nologo /OUT:hello.exe hello.obj Because we want this User's Guide to focus on what SCons is actually doing, we're going to use the -Q option to remove these messages from the output of all the remaining examples in this Guide. __________________________________________________________________ Chapter 3. Less Simple Things to Do With Builds In this chapter, you will see several examples of very simple build configurations using SCons, which will demonstrate how easy it is to use SCons to build programs from several different programming languages on different types of systems. __________________________________________________________________ 3.1. Specifying the Name of the Target (Output) File You've seen that when you call the [199]Program builder method, it builds the resulting program with the same base name as the source file. That is, the following call to build an executable program from the hello.c source file will build an executable program named hello on POSIX systems, and an executable program named hello.exe on Windows systems: Program('hello.c') If you want to build a program with a different name than the base of the source file name, you simply put the target file name to the left of the source file name: Program('new_hello', 'hello.c') (SCons requires the target file name first, followed by the source file name, so that the order mimics that of an assignment statement in most programming languages, including Python: "program = source files".) Now SCons will build an executable program named new_hello when run on a POSIX system: % scons -Q cc -o hello.o -c hello.c cc -o new_hello hello.o And SCons will build an executable program named new_hello.exe when run on a Windows system: C:\>scons -Q cl /nologo /c hello.c /Fohello.obj link /nologo /OUT:new_hello.exe hello.obj __________________________________________________________________ 3.2. Compiling Multiple Source Files You've just seen how to configure SCons to compile a program from a single source file. It's more common, of course, that you'll need to build a program from many input source files, not just one. To do this, you need to put the source files in a Python list (enclosed in square brackets), like so: Program(['prog.c', 'file1.c', 'file2.c']) A build of the above example would look like: % scons -Q cc -o file1.o -c file1.c cc -o file2.o -c file2.c cc -o prog.o -c prog.c cc -o prog prog.o file1.o file2.o Notice that SCons deduces the output program name from the first source file specified in the list--that is, because the first source file was prog.c, SCons will name the resulting program prog (or prog.exe on a Windows system). If you want to specify a different program name, then (as we've seen in the previous section) you slide the list of source files over to the right to make room for the output program file name. (SCons puts the output file name to the left of the source file names so that the order mimics that of an assignment statement: "program = source files".) This makes our example: Program('program', ['prog.c', 'file1.c', 'file2.c']) On Linux, a build of this example would look like: % scons -Q cc -o file1.o -c file1.c cc -o file2.o -c file2.c cc -o prog.o -c prog.c cc -o program prog.o file1.o file2.o Or on Windows: C:\>scons -Q cl /nologo /c file1.c /Fofile1.obj cl /nologo /c file2.c /Fofile2.obj cl /nologo /c prog.c /Foprog.obj link /nologo /OUT:program.exe prog.obj file1.obj file2.obj __________________________________________________________________ 3.3. Making a list of files with Glob() You can also use the Glob() function to find all files matching a certain template, using standard the shell pattern matching characters *, ?, and [abc] to match any of a, b, or c. [!abc] is also supported, to match any character except a, b, or c. This makes many multi-source-file builds quite easy: Program('program', Glob('*.c')) The SCons man page has more details on using Glob() with Variant dirs and Repositories and returning strings rather than Nodes. __________________________________________________________________ 3.4. Specifying Single Files Vs. Lists of Files We've now shown you two ways to specify the source for a program, one with a list of files: Program('hello', ['file1.c', 'file2.c']) And one with a single file: Program('hello', 'hello.c') You could actually put a single file name in a list, too, which you might prefer just for the sake of consistency: Program('hello', ['hello.c']) SCons functions will accept a single file name in either form. In fact, internally, SCons treats all input as lists of files, but allows you to omit the square brackets to cut down a little on the typing when there's only a single file name. Important Although SCons functions are forgiving about whether or not you use a string vs. a list for a single file name, Python itself is more strict about treating lists and strings differently. So where SCons allows either a string or list: # The following two calls both work correctly: Program('program1', 'program1.c') Program('program2', ['program2.c']) Trying to do "Python things" that mix strings and lists will cause errors or lead to incorrect results: common_sources = ['file1.c', 'file2.c'] # THE FOLLOWING IS INCORRECT AND GENERATES A PYTHON ERROR # BECAUSE IT TRIES TO ADD A STRING TO A LIST: Program('program1', common_sources + 'program1.c') # The following works correctly, because it's adding two # lists together to make another list. Program('program2', common_sources + ['program2.c']) __________________________________________________________________ 3.5. Making Lists of Files Easier to Read One drawback to the use of a Python list for source files is that each file name must be enclosed in quotes (either single quotes or double quotes). This can get cumbersome and difficult to read when the list of file names is long. Fortunately, SCons and Python provide a number of ways to make sure that the SConstruct file stays easy to read. To make long lists of file names easier to deal with, SCons provides a Split function that takes a quoted list of file names, with the names separated by spaces or other white-space characters, and turns it into a list of separate file names. Using the Split function turns the previous example into: Program('program', Split('main.c file1.c file2.c')) (If you're already familiar with Python, you'll have realized that this is similar to the split() method in the Python standard string module. Unlike the string.split() method, however, the Split function does not require a string as input and will wrap up a single non-string object in a list, or return its argument untouched if it's already a list. This comes in handy as a way to make sure arbitrary values can be passed to SCons functions without having to check the type of the variable by hand.) Putting the call to the Split function inside the Program call can also be a little unwieldy. A more readable alternative is to assign the output from the Split call to a variable name, and then use the variable when calling the Program function: list = Split('main.c file1.c file2.c') Program('program', list) Lastly, the Split function doesn't care how much white space separates the file names in the quoted string. This allows you to create lists of file names that span multiple lines, which often makes for easier editing: list = Split("""main.c file1.c file2.c""") Program('program', list) (Note in this example that we used the Python "triple-quote" syntax, which allows a string to contain multiple lines. The three quotes can be either single or double quotes.) __________________________________________________________________ 3.6. Keyword Arguments SCons also allows you to identify the output file and input source files using Python keyword arguments. The output file is known as the target, and the source file(s) are known (logically enough) as the source. The Python syntax for this is: list = Split('main.c file1.c file2.c') Program(target = 'program', source = list) Because the keywords explicitly identify what each argument is, you can actually reverse the order if you prefer: list = Split('main.c file1.c file2.c') Program(source = list, target = 'program') Whether or not you choose to use keyword arguments to identify the target and source files, and the order in which you specify them when using keywords, are purely personal choices; SCons functions the same regardless. __________________________________________________________________ 3.7. Compiling Multiple Programs In order to compile multiple programs within the same SConstruct file, simply call the Program method multiple times, once for each program you need to build: Program('foo.c') Program('bar', ['bar1.c', 'bar2.c']) SCons would then build the programs as follows: % scons -Q cc -o bar1.o -c bar1.c cc -o bar2.o -c bar2.c cc -o bar bar1.o bar2.o cc -o foo.o -c foo.c cc -o foo foo.o Notice that SCons does not necessarily build the programs in the same order in which you specify them in the SConstruct file. SCons does, however, recognize that the individual object files must be built before the resulting program can be built. We'll discuss this in greater detail in the "Dependencies" section, below. __________________________________________________________________ 3.8. Sharing Source Files Between Multiple Programs It's common to re-use code by sharing source files between multiple programs. One way to do this is to create a library from the common source files, which can then be linked into resulting programs. (Creating libraries is discussed in [200]Chapter 4, below.) A more straightforward, but perhaps less convenient, way to share source files between multiple programs is simply to include the common files in the lists of source files for each program: Program(Split('foo.c common1.c common2.c')) Program('bar', Split('bar1.c bar2.c common1.c common2.c')) SCons recognizes that the object files for the common1.c and common2.c source files each only need to be built once, even though the resulting object files are each linked in to both of the resulting executable programs: % scons -Q cc -o bar1.o -c bar1.c cc -o bar2.o -c bar2.c cc -o common1.o -c common1.c cc -o common2.o -c common2.c cc -o bar bar1.o bar2.o common1.o common2.o cc -o foo.o -c foo.c cc -o foo foo.o common1.o common2.o If two or more programs share a lot of common source files, repeating the common files in the list for each program can be a maintenance problem when you need to change the list of common files. You can simplify this by creating a separate Python list to hold the common file names, and concatenating it with other lists using the Python + operator: common = ['common1.c', 'common2.c'] foo_files = ['foo.c'] + common bar_files = ['bar1.c', 'bar2.c'] + common Program('foo', foo_files) Program('bar', bar_files) This is functionally equivalent to the previous example. __________________________________________________________________ Chapter 4. Building and Linking with Libraries It's often useful to organize large software projects by collecting parts of the software into one or more libraries. SCons makes it easy to create libraries and to use them in the programs. __________________________________________________________________ 4.1. Building Libraries You build your own libraries by specifying [201]Library instead of [202]Program: Library('foo', ['f1.c', 'f2.c', 'f3.c']) SCons uses the appropriate library prefix and suffix for your system. So on POSIX or Linux systems, the above example would build as follows (although ranlib may not be called on all systems): % scons -Q cc -o f1.o -c f1.c cc -o f2.o -c f2.c cc -o f3.o -c f3.c ar rc libfoo.a f1.o f2.o f3.o ranlib libfoo.a On a Windows system, a build of the above example would look like: C:\>scons -Q cl /nologo /c f1.c /Fof1.obj cl /nologo /c f2.c /Fof2.obj cl /nologo /c f3.c /Fof3.obj lib /nologo /OUT:foo.lib f1.obj f2.obj f3.obj The rules for the target name of the library are similar to those for programs: if you don't explicitly specify a target library name, SCons will deduce one from the name of the first source file specified, and SCons will add an appropriate file prefix and suffix if you leave them off. __________________________________________________________________ 4.1.1. Building Libraries From Source Code or Object Files The previous example shows building a library from a list of source files. You can, however, also give the [203]Library call object files, and it will correctly realize In fact, you can arbitrarily mix source code files and object files in the source list: Library('foo', ['f1.c', 'f2.o', 'f3.c', 'f4.o']) And SCons realizes that only the source code files must be compiled into object files before creating the final library: % scons -Q cc -o f1.o -c f1.c cc -o f3.o -c f3.c ar rc libfoo.a f1.o f2.o f3.o f4.o ranlib libfoo.a Of course, in this example, the object files must already exist for the build to succeed. See [204]Chapter 5, below, for information about how you can build object files explicitly and include the built files in a library. __________________________________________________________________ 4.1.2. Building Static Libraries Explicitly: the StaticLibrary Builder The [205]Library function builds a traditional static library. If you want to be explicit about the type of library being built, you can use the synonym [206]StaticLibrary function instead of Library: StaticLibrary('foo', ['f1.c', 'f2.c', 'f3.c']) There is no functional difference between the [207]StaticLibrary and Library functions. __________________________________________________________________ 4.1.3. Building Shared (DLL) Libraries: the SharedLibrary Builder If you want to build a shared library (on POSIX systems) or a DLL file (on Windows systems), you use the [208]SharedLibrary function: SharedLibrary('foo', ['f1.c', 'f2.c', 'f3.c']) The output on POSIX: % scons -Q cc -o f1.os -c f1.c cc -o f2.os -c f2.c cc -o f3.os -c f3.c cc -o libfoo.so -shared f1.os f2.os f3.os And the output on Windows: C:\>scons -Q cl /nologo /c f1.c /Fof1.obj cl /nologo /c f2.c /Fof2.obj cl /nologo /c f3.c /Fof3.obj link /nologo /dll /out:foo.dll /implib:foo.lib f1.obj f2.obj f3.obj RegServerFunc(target, source, env) Notice again that SCons takes care of building the output file correctly, adding the -shared option for a POSIX compilation, and the /dll option on Windows. __________________________________________________________________ 4.2. Linking with Libraries Usually, you build a library because you want to link it with one or more programs. You link libraries with a program by specifying the libraries in the [209]$LIBS construction variable, and by specifying the directory in which the library will be found in the [210]$LIBPATH construction variable: Library('foo', ['f1.c', 'f2.c', 'f3.c']) Program('prog.c', LIBS=['foo', 'bar'], LIBPATH='.') Notice, of course, that you don't need to specify a library prefix (like lib) or suffix (like .a or .lib). SCons uses the correct prefix or suffix for the current system. On a POSIX or Linux system, a build of the above example would look like: % scons -Q cc -o f1.o -c f1.c cc -o f2.o -c f2.c cc -o f3.o -c f3.c ar rc libfoo.a f1.o f2.o f3.o ranlib libfoo.a cc -o prog.o -c prog.c cc -o prog prog.o -L. -lfoo -lbar On a Windows system, a build of the above example would look like: C:\>scons -Q cl /nologo /c f1.c /Fof1.obj cl /nologo /c f2.c /Fof2.obj cl /nologo /c f3.c /Fof3.obj lib /nologo /OUT:foo.lib f1.obj f2.obj f3.obj cl /nologo /c prog.c /Foprog.obj link /nologo /OUT:prog.exe /LIBPATH:. foo.lib bar.lib prog.obj As usual, notice that SCons has taken care of constructing the correct command lines to link with the specified library on each system. Note also that, if you only have a single library to link with, you can specify the library name in single string, instead of a Python list, so that: Program('prog.c', LIBS='foo', LIBPATH='.') is equivalent to: Program('prog.c', LIBS=['foo'], LIBPATH='.') This is similar to the way that SCons handles either a string or a list to specify a single source file. __________________________________________________________________ 4.3. Finding Libraries: the $LIBPATH Construction Variable By default, the linker will only look in certain system-defined directories for libraries. SCons knows how to look for libraries in directories that you specify with the [211]$LIBPATH construction variable. $LIBPATH consists of a list of directory names, like so: Program('prog.c', LIBS = 'm', LIBPATH = ['/usr/lib', '/usr/local/lib']) Using a Python list is preferred because it's portable across systems. Alternatively, you could put all of the directory names in a single string, separated by the system-specific path separator character: a colon on POSIX systems: LIBPATH = '/usr/lib:/usr/local/lib' or a semi-colon on Windows systems: LIBPATH = 'C:\\lib;D:\\lib' (Note that Python requires that the backslash separators in a Windows path name be escaped within strings.) When the linker is executed, SCons will create appropriate flags so that the linker will look for libraries in the same directories as SCons. So on a POSIX or Linux system, a build of the above example would look like: % scons -Q cc -o prog.o -c prog.c cc -o prog prog.o -L/usr/lib -L/usr/local/lib -lm On a Windows system, a build of the above example would look like: C:\>scons -Q cl /nologo /c prog.c /Foprog.obj link /nologo /OUT:prog.exe /LIBPATH:\usr\lib /LIBPATH:\usr\local\lib m.lib prog.obj Note again that SCons has taken care of the system-specific details of creating the right command-line options. __________________________________________________________________ Chapter 5. Node Objects Internally, SCons represents all of the files and directories it knows about as Nodes. These internal objects (not object files) can be used in a variety of ways to make your SConscript files portable and easy to read. __________________________________________________________________ 5.1. Builder Methods Return Lists of Target Nodes All builder methods return a list of Node objects that identify the target file or files that will be built. These returned Nodes can be passed as source files to other builder methods, For example, suppose that we want to build the two object files that make up a program with different options. This would mean calling the [212]Object builder once for each object file, specifying the desired options: Object('hello.c', CCFLAGS='-DHELLO') Object('goodbye.c', CCFLAGS='-DGOODBYE') One way to combine these object files into the resulting program would be to call the [213]Program builder with the names of the object files listed as sources: Object('hello.c', CCFLAGS='-DHELLO') Object('goodbye.c', CCFLAGS='-DGOODBYE') Program(['hello.o', 'goodbye.o']) The problem with listing the names as strings is that our SConstruct file is no longer portable across operating systems. It won't, for example, work on Windows because the object files there would be named hello.obj and goodbye.obj, not hello.o and goodbye.o. A better solution is to assign the lists of targets returned by the calls to the Object builder to variables, which we can then concatenate in our call to the Program builder: hello_list = Object('hello.c', CCFLAGS='-DHELLO') goodbye_list = Object('goodbye.c', CCFLAGS='-DGOODBYE') Program(hello_list + goodbye_list) This makes our SConstruct file portable again, the build output on Linux looking like: % scons -Q cc -o goodbye.o -c -DGOODBYE goodbye.c cc -o hello.o -c -DHELLO hello.c cc -o hello hello.o goodbye.o And on Windows: C:\>scons -Q cl -DGOODBYE /c goodbye.c /Fogoodbye.obj cl -DHELLO /c hello.c /Fohello.obj link /nologo /OUT:hello.exe hello.obj goodbye.obj We'll see examples of using the list of nodes returned by builder methods throughout the rest of this guide. __________________________________________________________________ 5.2. Explicitly Creating File and Directory Nodes It's worth mentioning here that SCons maintains a clear distinction between Nodes that represent files and Nodes that represent directories. SCons supports File and Dir functions that, repectively, return a file or directory Node: hello_c = File('hello.c') Program(hello_c) classes = Dir('classes') Java(classes, 'src') Normally, you don't need to call File or Dir directly, because calling a builder method automatically treats strings as the names of files or directories, and translates them into the Node objects for you. The File and Dir functions can come in handy in situations where you need to explicitly instruct SCons about the type of Node being passed to a builder or other function, or unambiguously refer to a specific file in a directory tree. There are also times when you may need to refer to an entry in a file system without knowing in advance whether it's a file or a directory. For those situations, SCons also supports an Entry function, which returns a Node that can represent either a file or a directory. xyzzy = Entry('xyzzy') The returned xyzzy Node will be turned into a file or directory Node the first time it is used by a builder method or other function that requires one vs. the other. __________________________________________________________________ 5.3. Printing Node File Names One of the most common things you can do with a Node is use it to print the file name that the node represents. For example, the following SConstruct file: hello_c = File('hello.c') Program(hello_c) classes = Dir('classes') Java(classes, 'src') object_list = Object('hello.c') program_list = Program(object_list) print "The object file is:", object_list[0] print "The program file is:", program_list[0] Would print the following file names on a POSIX system: % scons -Q The object file is: hello.o The program file is: hello cc -o hello.o -c hello.c cc -o hello hello.o And the following file names on a Windows system: C:\>scons -Q The object file is: hello.obj The program file is: hello.exe cl /nologo /c hello.c /Fohello.obj link /nologo /OUT:hello.exe hello.obj __________________________________________________________________ 5.4. Using a Node's File Name as a String Printing a Node's name as described in the previous section works because the string representation of a Node is the name of the file. If you want to do something other than print the name of the file, you can fetch it by using the builtin Python str function. For example, if you want to use the Python os.path.exists to figure out whether a file exists while the SConstruct file is being read and executed, you can fetch the string as follows: import os.path program_list = Program('hello.c') program_name = str(program_list[0]) if not os.path.exists(program_name): print program_name, "does not exist!" Which executes as follows on a POSIX system: % scons -Q hello does not exist! cc -o hello.o -c hello.c cc -o hello hello.o __________________________________________________________________ Chapter 6. Dependencies So far we've seen how SCons handles one-time builds. But one of the main functions of a build tool like SCons is to rebuild only the necessary things when source files change--or, put another way, SCons should not waste time rebuilding things that have already been built. You can see this at work simply by re-invoking SCons after building our simple hello example: % scons -Q cc -o hello.o -c hello.c cc -o hello hello.o % scons -Q scons: `.' is up to date. The second time it is executed, SCons realizes that the hello program is up-to-date with respect to the current hello.c source file, and avoids rebuilding it. You can see this more clearly by naming the hello program explicitly on the command line: % scons -Q hello cc -o hello.o -c hello.c cc -o hello hello.o % scons -Q hello scons: `hello' is up to date. Note that SCons reports "...is up to date" only for target files named explicitly on the command line, to avoid cluttering the output. __________________________________________________________________ 6.1. Deciding When an Input File Has Changed: the Decider Function Another aspect of avoiding unnecessary rebuilds is the fundamental build tool behavior of rebuilding things when an input file changes, so that the built software is up to date. By default, SCons keeps track of this through an MD5 signature, or checksum, of the contents of each file, although you can easily configure SCons to use the modification times (or time stamps) instead. You can even specify your own Python function for deciding if an input file has changed. __________________________________________________________________ 6.1.1. Using MD5 Signatures to Decide if a File Has Changed By default, SCons keeps track of whether a file has changed based on an MD5 checksum of the file's contents, not the file's modification time. This means that you may be surprised by the default SCons behavior if you are used to the Make convention of forcing a rebuild by updating the file's modification time (using the touch command, for example): % scons -Q hello cc -o hello.o -c hello.c cc -o hello hello.o % touch hello.c % scons -Q hello scons: `hello' is up to date. Even though the file's modification time has changed, SCons realizes that the contents of the hello.c file have not changed, and therefore that the hello program need not be rebuilt. This avoids unnecessary rebuilds when, for example, someone rewrites the contents of a file without making a change. But if the contents of the file really do change, then SCons detects the change and rebuilds the program as required: % scons -Q hello cc -o hello.o -c hello.c cc -o hello hello.o % edit hello.c [CHANGE THE CONTENTS OF hello.c] % scons -Q hello cc -o hello.o -c hello.c cc -o hello hello.o Note that you can, if you wish, specify this default behavior (MD5 signatures) explicitly using the Decider function as follows: Program('hello.c') Decider('MD5') You can also use the string 'content' as a synonym for 'MD5' when calling the Decider function. __________________________________________________________________ 6.1.1.1. Ramifications of Using MD5 Signatures Using MD5 Signatures to decide if an input file has changed has one surprising benefit: if a source file has been changed in such a way that the contents of the rebuilt target file(s) will be exactly the same as the last time the file was built, then any "downstream" target files that depend on the rebuilt-but-not-changed target file actually need not be rebuilt. So if, for example, a user were to only change a comment in a hello.c file, then the rebuilt hello.o file would be exactly the same as the one previously built (assuming the compiler doesn't put any build-specific information in the object file). SCons would then realize that it would not need to rebuild the hello program as follows: % scons -Q hello cc -o hello.o -c hello.c cc -o hello hello.o % edit hello.c [CHANGE A COMMENT IN hello.c] % scons -Q hello cc -o hello.o -c hello.c scons: `hello' is up to date. In essence, SCons "short-circuits" any dependent builds when it realizes that a target file has been rebuilt to exactly the same file as the last build. This does take some extra processing time to read the contents of the target (hello.o) file, but often saves time when the rebuild that was avoided would have been time-consuming and expensive. __________________________________________________________________ 6.1.2. Using Time Stamps to Decide If a File Has Changed If you prefer, you can configure SCons to use the modification time of a file, not the file contents, when deciding if a target needs to be rebuilt. SCons gives you two ways to use time stamps to decide if an input file has changed since the last time a target has been built. The most familiar way to use time stamps is the way Make does: that is, have SCons decide and target must be rebuilt if if a source file's modification time is newer than the target file. To do this, call the Decider function as follows: Program('hello.c') Decider('timestamp-newer') This makes SCons act like Make when a file's modification time is updated (using the touch command, for example): % scons -Q hello cc -o hello.o -c hello.c cc -o hello hello.o % touch hello.c % scons -Q hello cc -o hello.o -c hello.c cc -o hello hello.o And, in fact, because this behavior is the same as the behavior of Make, you can also use the string 'make' as a synonym for 'timestamp-newer' when calling the Decider function: Program('hello.c') Decider('make') One drawback to using times stamps exactly like Make is that if an input file's modification time suddenly becomes older than a target file, the target file will not be rebuilt. This can happen if an old copy of a source file is restored from a backup archive, for example. The contents of the restored file will likely be different than they were the last time a dependent target was built, but the target won't be rebuilt because the modification time of the source file is not newer than the target. Because SCons actually stores information about the source files' time stamps whenever a target is built, it can handle this situation by checking for an exact match of the source file time stamp, instead of just whether or not the source file is newer than the target file. To do this, specify the argument 'timestamp-match' when calling the Decider function: Program('hello.c') Decider('timestamp-match') When configured this way, SCons will rebuild a target whenever a source file's modification time has changed. So if we use the touch -t option to change the modification time of hello.c to an old date (January 1, 1989), SCons will still rebuild the target file: % scons -Q hello cc -o hello.o -c hello.c cc -o hello hello.o % touch -t 198901010000 hello.c % scons -Q hello cc -o hello.o -c hello.c scons: `hello' is up to date. In general, the only reason to prefer timestamp-newer instead of timestamp-match, would be if you have some specific reason to require this Make-like behavior of not rebuilding a target when an otherwise-modified source file is older. __________________________________________________________________ 6.1.3. Deciding If a File Has Changed Using Both MD Signatures and Time Stamps As a performance enhancement, SCons provides a way to use MD5 checksums of file contents but to only read the contents whenever the file's timestamp has changed. To do this, call the Decider function with 'MD5-timestamp' argument as follows: Program('hello.c') Decider('MD5-timestamp') So configured, SCons will still behave like it does when using Decider('MD5'): % scons -Q hello cc -o hello.o -c hello.c cc -o hello hello.o % touch hello.c % scons -Q hello scons: `hello' is up to date. % edit hello.c [CHANGE THE CONTENTS OF hello.c] % scons -Q hello scons: `hello' is up to date. However, the second call to SCons in the above output, when the build is up-to-date, will have been performed by simply looking at the modification time of the hello.c file, not by opening it and performing an MD5 checksum calcuation on its contents. This can significantly speed up many up-to-date builds. The only drawback to using Decider('MD5-timestamp') is that SCons will not rebuild a target file if a source file was modified within one second of the last time SCons built the file. While most developers are programming, this isn't a problem in practice, since it's unlikely that someone will have built and then thought quickly enought to make a substantive change to a source file within one second. Certain build scripts or continuous integration tools may, however, rely on the ability to applying changes to files automatically and then rebuild as quickly as possible, in which case use of Decider('MD5-timestamp') may not be appropriate. __________________________________________________________________ 6.1.4. Writing Your Own Custom Decider Function The different string values that we've passed to the Decider function are essentially used by SCons to pick one of several specific internal functions that implement various ways of deciding if a dependency (usually a source file) has changed since a target file has been built. As it turns out, you can also supply your own function to decide if a dependency has changed. For example, suppose we have an input file that contains a lot of data, in some specific regular format, that is used to rebuild a lot of different target files, but each target file really only depends on one particular section of the input file. We'd like to have each target file depend on only its section of the input file. However, since the input file may contain a lot of data, we only want to open the input file if its timestamp has changed. This could done with a custom Decider function that might look something like this: Program('hello.c') def decide_if_changed(dependency, target, prev_ni): if self.get_timestamp() != prev_ni.timestamp: dep = str(dependency) tgt = str(target) if specific_part_of_file_has_changed(dep, tgt): return True return False Decider(decide_if_changed) Note that in the function definition, the dependency (input file) is the first argument, and then the target. Both of these are passed to the functions as SCons Node objects, which we convert to strings using the Python str(). The third argument, prev_ni, is an object that holds the signature or timestamp information that was recorded about the dependency the last time the target was built. Note that ignoring some of the arguments in your custom Decider function is a perfectly normal thing to do, if they don't impact the way you want to decide if the dependency file has changed. __________________________________________________________________ 6.1.5. Mixing Different Ways of Deciding If a File Has Changed The previous examples have all demonstrated calling the global Decider function to configure all dependency decisions that SCons makes. Sometimes, however, you want to be able to configure different decision-making for different targets. When that's necessary, you can use the env.Decider method to affect only the configuration decisions for targets built with a specific construction environment. For example, if we arbitrarily want to build one program using MD5 checkums and another use file modification times from the same source we might configure it this way: env1 = Environment(CPPPATH = ['.']) env2 = env1.Clone() env2.Decider('timestamp-match') env1.Program('prog-MD5', 'program1.c') env2.Program('prog-timestamp', 'program2.c') If both of the programs include the same inc.h file, then updating the modification time of inc.h (using the touch command) will cause only prog-timestamp to be rebuilt: % scons -Q cc -o program1.o -c -I. program1.c cc -o prog-MD5 program1.o cc -o program2.o -c -I. program2.c cc -o prog-timestamp program2.o % touch inc.h % scons -Q cc -o program2.o -c -I. program2.c cc -o prog-timestamp program2.o __________________________________________________________________ 6.2. Older Functions for Deciding When an Input File Has Changed SCons still supports two functions that used to be the primary methods for configuring the decision about whether or not an input file has changed. Although they're not officially deprecated yet, their use is discouraged, mainly because they rely on a somewhat confusing distinction between how source files and target files are handled. These functions are documented here mainly in case you encounter them in existing SConscript files. __________________________________________________________________ 6.2.1. The SourceSignatures Function The SourceSignatures function is fairly straightforward, and supports two different argument values to configure whether source file changes should be decided using MD5 signatures: Program('hello.c') SourceSignatures('MD5') Or using time stamps: Program('hello.c') SourceSignatures('timestamp') These are roughly equivalent to specifying Decider('MD5') or Decider('timestamp-match'), respectively, although it only affects how SCons makes decisions about dependencies on source files--that is, files that are not built from any other files. __________________________________________________________________ 6.2.2. The TargetSignatures Function The TargetSignatures function specifies how SCons decides when a target file has changed when it is used as a dependency of (input to) another target--that is, the TargetSignatures function configures how the signatures of "intermediate" target files are used when deciding if a "downstream" target file must be rebuilt. [214][2] The TargetSignatures function supports the same 'MD5' and 'timestamp' argument values that are supported by the SourceSignatures, with the same meanings, but applied to target files. That is, in the example: Program('hello.c') TargetSignatures('MD5') The MD5 checksum of the hello.o target file will be used to decide if it has changed since the last time the "downstream" hello target file was built. And in the example: Program('hello.c') TargetSignatures('timestamp') The modification time of the hello.o target file will be used to decide if it has changed since the last time the "downstream" hello target file was built. The TargetSignatures function supports two additional argument values: 'source' and 'build'. The 'source' argument specifies that decisions involving whether target files have changed since a previous build should use the same behavior for the decisions configured for source files (using the SourceSignatures function). So in the example: Program('hello.c') TargetSignatures('source') SourceSignatures('timestamp') All files, both targets and sources, will use modification times when deciding if an input file has changed since the last time a target was built. Lastly, the 'build' argument specifies that SCons should examine the build status of a target file and always rebuild a "downstream" target if the target file was itself rebuilt, without re-examining the contents or timestamp of the newly-built target file. If the target file was not rebuilt during this scons invocation, then the target file will be examined the same way as configured by the SourceSignature call to decide if it has changed. This mimics the behavior of build signatures in earlier versions of SCons. A build signature re-combined signatures of all the input files that went into making the target file, so that the target file itself did not need to have its contents read to compute an MD5 signature. This can improve performance for some configurations, but is generally not as effective as using Decider('MD5-timestamp'). __________________________________________________________________ 6.3. Implicit Dependencies: The $CPPPATH Construction Variable Now suppose that our "Hello, World!" program actually has an #include line to include the hello.h file in the compilation: #include int main() { printf("Hello, %s!\n", string); } And, for completeness, the hello.h file looks like this: #define string "world" In this case, we want SCons to recognize that, if the contents of the hello.h file change, the hello program must be recompiled. To do this, we need to modify the SConstruct file like so: Program('hello.c', CPPPATH = '.') The [215]$CPPPATH value tells SCons to look in the current directory ('.') for any files included by C source files (.c or .h files). With this assignment in the SConstruct file: % scons -Q hello cc -o hello.o -c -I. hello.c cc -o hello hello.o % scons -Q hello scons: `hello' is up to date. % edit hello.h [CHANGE THE CONTENTS OF hello.h] % scons -Q hello cc -o hello.o -c -I. hello.c cc -o hello hello.o First, notice that SCons added the -I. argument from the $CPPPATH variable so that the compilation would find the hello.h file in the local directory. Second, realize that SCons knows that the hello program must be rebuilt because it scans the contents of the hello.c file for the #include lines that indicate another file is being included in the compilation. SCons records these as implicit dependencies of the target file, Consequently, when the hello.h file changes, SCons realizes that the hello.c file includes it, and rebuilds the resulting hello program that depends on both the hello.c and hello.h files. Like the [216]$LIBPATH variable, the $CPPPATH variable may be a list of directories, or a string separated by the system-specific path separation character (':' on POSIX/Linux, ';' on Windows). Either way, SCons creates the right command-line options so that the following example: Program('hello.c', CPPPATH = ['include', '/home/project/inc']) Will look like this on POSIX or Linux: % scons -Q hello cc -o hello.o -c -Iinclude -I/home/project/inc hello.c cc -o hello hello.o And like this on Windows: C:\>scons -Q hello.exe cl /nologo /Iinclude /I\home\project\inc /c hello.c /Fohello.obj link /nologo /OUT:hello.exe hello.obj __________________________________________________________________ 6.4. Caching Implicit Dependencies Scanning each file for #include lines does take some extra processing time. When you're doing a full build of a large system, the scanning time is usually a very small percentage of the overall time spent on the build. You're most likely to notice the scanning time, however, when you rebuild all or part of a large system: SCons will likely take some extra time to "think about" what must be built before it issues the first build command (or decides that everything is up to date and nothing must be rebuilt). In practice, having SCons scan files saves time relative to the amount of potential time lost to tracking down subtle problems introduced by incorrect dependencies. Nevertheless, the "waiting time" while SCons scans files can annoy individual developers waiting for their builds to finish. Consequently, SCons lets you cache the implicit dependencies that its scanners find, for use by later builds. You can do this by specifying the --implicit-cache option on the command line: % scons -Q --implicit-cache hello cc -o hello.o -c hello.c cc -o hello hello.o % scons -Q hello scons: `hello' is up to date. If you don't want to specify --implicit-cache on the command line each time, you can make it the default behavior for your build by setting the implicit_cache option in an SConscript file: SetOption('implicit_cache', 1) SCons does not cache implicit dependencies like this by default because the --implicit-cache causes SCons to simply use the implicit dependencies stored during the last run, without any checking for whether or not those dependencies are still correct. Specifically, this means --implicit-cache instructs SCons to not rebuild "correctly" in the following cases: * When --implicit-cache is used, SCons will ignore any changes that may have been made to search paths (like $CPPPATH or $LIBPATH,). This can lead to SCons not rebuilding a file if a change to $CPPPATH would normally cause a different, same-named file from a different directory to be used. * When --implicit-cache is used, SCons will not detect if a same-named file has been added to a directory that is earlier in the search path than the directory in which the file was found last time. __________________________________________________________________ 6.4.1. The --implicit-deps-changed Option When using cached implicit dependencies, sometimes you want to "start fresh" and have SCons re-scan the files for which it previously cached the dependencies. For example, if you have recently installed a new version of external code that you use for compilation, the external header files will have changed and the previously-cached implicit dependencies will be out of date. You can update them by running SCons with the --implicit-deps-changed option: % scons -Q --implicit-deps-changed hello cc -o hello.o -c hello.c cc -o hello hello.o % scons -Q hello scons: `hello' is up to date. In this case, SCons will re-scan all of the implicit dependencies and cache updated copies of the information. __________________________________________________________________ 6.4.2. The --implicit-deps-unchanged Option By default when caching dependencies, SCons notices when a file has been modified and re-scans the file for any updated implicit dependency information. Sometimes, however, you may want to force SCons to use the cached implicit dependencies, even if the source files changed. This can speed up a build for example, when you have changed your source files but know that you haven't changed any #include lines. In this case, you can use the --implicit-deps-unchanged option: % scons -Q --implicit-deps-unchanged hello cc -o hello.o -c hello.c cc -o hello hello.o % scons -Q hello scons: `hello' is up to date. In this case, SCons will assume that the cached implicit dependencies are correct and will not bother to re-scan changed files. For typical builds after small, incremental changes to source files, the savings may not be very big, but sometimes every bit of improved performance counts. __________________________________________________________________ 6.5. Explicit Dependencies: the Depends Function Sometimes a file depends on another file that is not detected by an SCons scanner. For this situation, SCons allows you to specific explicitly that one file depends on another file, and must be rebuilt whenever that file changes. This is specified using the Depends method: hello = Program('hello.c') Depends(hello, 'other_file') % scons -Q hello cc -c hello.c -o hello.o cc -o hello hello.o % scons -Q hello scons: `hello' is up to date. % edit other_file [CHANGE THE CONTENTS OF other_file] % scons -Q hello cc -c hello.c -o hello.o cc -o hello hello.o __________________________________________________________________ 6.6. Ignoring Dependencies: the Ignore Function Sometimes it makes sense to not rebuild a program, even if a dependency file changes. In this case, you would tell SCons specifically to ignore a dependency as follows: hello = Program('hello.c') Ignore(hello, 'hello.h') % scons -Q hello cc -c -o hello.o hello.c cc -o hello hello.o % scons -Q hello scons: `hello' is up to date. % edit hello.h [CHANGE THE CONTENTS OF hello.h] % scons -Q hello scons: `hello' is up to date. Now, the above example is a little contrived, because it's hard to imagine a real-world situation where you wouldn't want to rebuild hello if the hello.h file changed. A more realistic example might be if the hello program is being built in a directory that is shared between multiple systems that have different copies of the stdio.h include file. In that case, SCons would notice the differences between the different systems' copies of stdio.h and would rebuild hello each time you change systems. You could avoid these rebuilds as follows: hello = Program('hello.c') Ignore(hello, '/usr/include/stdio.h') __________________________________________________________________ 6.7. The AlwaysBuild Function How SCons handles dependencies can also be affected by the AlwaysBuild method. When a file is passed to the AlwaysBuild method, like so: hello = Program('hello.c') AlwaysBuild(hello) Then the specified target file (hello in our example) will always be considered out-of-date and rebuilt whenever that target file is evaluated while walking the dependency graph: % scons -Q cc -o hello.o -c hello.c cc -o hello hello.o % scons -Q cc -o hello hello.o The AlwaysBuild function has a somewhat misleading name, because it does not actually mean the target file will be rebuilt every single time SCons is invoked. Instead, it means that the target will, in fact, be rebuilt whenever the target file is encountered while evaluating the targets specified on the command line (and their dependencies). So specifying some other target on the command line, a target that does not itself depend on the AlwaysBuild target, will still be rebuilt only if it's out-of-date with respect to its dependencies: % scons -Q cc -o hello.o -c hello.c cc -o hello hello.o % scons -Q hello.o scons: `hello.o' is up to date. __________________________________________________________________ Chapter 7. Construction Environments It is rare that all of the software in a large, complicated system needs to be built the same way. For example, different source files may need different options enabled on the command line, or different executable programs need to be linked with different libraries. SCons accommodates these different build requirements by allowing you to create and configure multiple construction environments that control how the software is built. Technically, a construction environment is an object that has a number of associated construction variables, each with a name and a value. (A construction environment also has an attached set of Builder methods, about which we'll learn more later.) A construction environment is created by the Environment method: env = Environment() By default, SCons initializes every new construction environment with a set of construction variables based on the tools that it finds on your system, plus the default set of builder methods necessary for using those tools. The construction variables are initialized with values describing the C compiler, the Fortran compiler, the linker, etc., as well as the command lines to invoke them. When you initialize a construction environment you can set the values of the environment's construction variables to control how a program is built. For example: env = Environment(CC = 'gcc', CCFLAGS = '-O2') env.Program('foo.c') The construction environment in this example is still initialized with the same default construction variable values, except that the user has explicitly specified use of the GNU C compiler gcc, and further specifies that the -O2 (optimization level two) flag should be used when compiling the object file. In other words, the explicit initializations of [217]$CC and [218]$CCFLAGS override the default values in the newly-created construction environment. So a run from this example would look like: % scons -Q gcc -o foo.o -c -O2 foo.c gcc -o foo foo.o __________________________________________________________________ 7.1. Multiple Construction Environments The real advantage of construction environments is that you can create as many different construction environments as you need, each tailored to a different way to build some piece of software or other file. If, for example, we need to build one program with the -O2 flag and another with the -g (debug) flag, we would do this like so: opt = Environment(CCFLAGS = '-O2') dbg = Environment(CCFLAGS = '-g') opt.Program('foo', 'foo.c') dbg.Program('bar', 'bar.c') % scons -Q cc -o bar.o -c -g bar.c cc -o bar bar.o cc -o foo.o -c -O2 foo.c cc -o foo foo.o We can even use multiple construction environments to build multiple versions of a single program. If you do this by simply trying to use the [219]Program builder with both environments, though, like this: opt = Environment(CCFLAGS = '-O2') dbg = Environment(CCFLAGS = '-g') opt.Program('foo', 'foo.c') dbg.Program('foo', 'foo.c') Then SCons generates the following error: % scons -Q scons: *** Two environments with different actions were specified for the same target: foo.o File "/home/my/project/SConstruct", line 6, in This is because the two Program calls have each implicitly told SCons to generate an object file named foo.o, one with a [220]$CCFLAGS value of -O2 and one with a [221]$CCFLAGS value of -g. SCons can't just decide that one of them should take precedence over the other, so it generates the error. To avoid this problem, we must explicitly specify that each environment compile foo.c to a separately-named object file using the [222]Object builder, like so: opt = Environment(CCFLAGS = '-O2') dbg = Environment(CCFLAGS = '-g') o = opt.Object('foo-opt', 'foo.c') opt.Program(o) d = dbg.Object('foo-dbg', 'foo.c') dbg.Program(d) Notice that each call to the Object builder returns a value, an internal SCons object that represents the object file that will be built. We then use that object as input to the Program builder. This avoids having to specify explicitly the object file name in multiple places, and makes for a compact, readable SConstruct file. Our SCons output then looks like: % scons -Q cc -o foo-dbg.o -c -g foo.c cc -o foo-dbg foo-dbg.o cc -o foo-opt.o -c -O2 foo.c cc -o foo-opt foo-opt.o __________________________________________________________________ 7.2. Copying Construction Environments Sometimes you want more than one construction environment to share the same values for one or more variables. Rather than always having to repeat all of the common variables when you create each construction environment, you can use the Clone method to create a copy of a construction environment. Like the Environment call that creates a construction environment, the Clone method takes construction variable assignments, which will override the values in the copied construction environment. For example, suppose we want to use gcc to create three versions of a program, one optimized, one debug, and one with neither. We could do this by creating a "base" construction environment that sets [223]$CC to gcc, and then creating two copies, one which sets [224]$CCFLAGS for optimization and the other which sets $CCFLAGS for debugging: env = Environment(CC = 'gcc') opt = env.Clone(CCFLAGS = '-O2') dbg = env.Clone(CCFLAGS = '-g') env.Program('foo', 'foo.c') o = opt.Object('foo-opt', 'foo.c') opt.Program(o) d = dbg.Object('foo-dbg', 'foo.c') dbg.Program(d) Then our output would look like: % scons -Q gcc -o foo.o -c foo.c gcc -o foo foo.o gcc -o foo-dbg.o -c -g foo.c gcc -o foo-dbg foo-dbg.o gcc -o foo-opt.o -c -O2 foo.c gcc -o foo-opt foo-opt.o __________________________________________________________________ 7.3. Fetching Values From a Construction Environment You can fetch individual construction variables using the normal syntax for accessing individual named items in a Python dictionary: env = Environment() print "CC is:", env['CC'] This example SConstruct file doesn't build anything, but because it's actually a Python script, it will print the value of [225]$CC for us: % scons -Q CC is: cc scons: `.' is up to date. A construction environment, however, is actually an object with associated methods, etc. If you want to have direct access to only the dictionary of construction variables, you can fetch this using the Dictionary method: env = Environment(FOO = 'foo', BAR = 'bar') dict = env.Dictionary() for key in ['OBJSUFFIX', 'LIBSUFFIX', 'PROGSUFFIX']: print "key = %s, value = %s" % (key, dict[key]) This SConstruct file will print the specified dictionary items for us on POSIX systems as follows: % scons -Q key = OBJSUFFIX, value = .o key = LIBSUFFIX, value = .a key = PROGSUFFIX, value = scons: `.' is up to date. And on Windows: C:\>scons -Q key = OBJSUFFIX, value = .obj key = LIBSUFFIX, value = .lib key = PROGSUFFIX, value = .exe scons: `.' is up to date. If you want to loop through and print the values of all of the construction variables in a construction environment, the Python code to do that in sorted order might look something like: env = Environment() dict = env.Dictionary() keys = dict.keys() keys.sort() for key in keys: print "construction variable = '%s', value = '%s'" % (key, dict[key]) __________________________________________________________________ 7.4. Expanding Values From a Construction Environment Another way to get information from a construction environment. is to use the subst method on a string containing $-expansions of construction variable names. As a simple example, the example from the previous section that used env['CC'] to fetch the value of [226]$CC could also be written as: env = Environment() print "CC is:", env.subst('$CC') The real advantage of using subst to expand strings is that construction variables in the result get re-expanded until there are no expansions left in the string. So a simple fetch of a value like [227]$CCCOM: env = Environment(CCFLAGS = '-DFOO') print "CCCOM is:", env['CCCOM'] Will print the unexpanded value of $CCCOM, showing us the construction variables that still need to be expanded: % scons -Q CCCOM is: $CC $CCFLAGS $CPPFLAGS $_CPPDEFFLAGS $_CPPINCFLAGS -c -o $TARGET $SOURCES scons: `.' is up to date. Calling the subst method on $CCOM, however: env = Environment(CCFLAGS = '-DFOO') print "CCCOM is:", env.subst('$CCCOM') Will recursively expand all of the $-prefixed construction variables, showing us the final output: % scons -Q CCCOM is: gcc -DFOO -c -o scons: `.' is up to date. (Note that because we're not expanding this in the context of building something there are no target or source files for [228]$TARGET and [229]$SOURCES to expand.) __________________________________________________________________ 7.5. Modifying a Construction Environment SCons provides various methods that support modifying existing values in a construction environment. __________________________________________________________________ 7.5.1. Replacing Values in a Construction Environment You can replace existing construction variable values using the Replace method: env = Environment(CCFLAGS = '-DDEFINE1') env.Replace(CCFLAGS = '-DDEFINE2') env.Program('foo.c') The replacing value (-DDEFINE2 in the above example) completely replaces the value in the construction environment: % scons -Q cc -o foo.o -c -DDEFINE2 foo.c cc -o foo foo.o You can safely call Replace for construction variables that don't exist in the construction environment: env = Environment() env.Replace(NEW_VARIABLE = 'xyzzy') print "NEW_VARIABLE =", env['NEW_VARIABLE'] In this case, the construction variable simply gets added to the construction environment: % scons -Q NEW_VARIABLE = xyzzy scons: `.' is up to date. Because the variables aren't expanded until the construction environment is actually used to build the targets, and because SCons function and method calls are order-independent, the last replacement "wins" and is used to build all targets, regardless of the order in which the calls to Replace() are interspersed with calls to builder methods: env = Environment(CCFLAGS = '-DDEFINE1') print "CCFLAGS =", env['CCFLAGS'] env.Program('foo.c') env.Replace(CCFLAGS = '-DDEFINE2') print "CCFLAGS =", env['CCFLAGS'] env.Program('bar.c') The timing of when the replacement actually occurs relative to when the targets get built becomes apparent if we run scons without the -Q option: % scons scons: Reading SConscript files ... CCFLAGS = -DDEFINE1 CCFLAGS = -DDEFINE2 scons: done reading SConscript files. scons: Building targets ... cc -o bar.o -c -DDEFINE2 bar.c cc -o bar bar.o cc -o foo.o -c -DDEFINE2 foo.c cc -o foo foo.o scons: done building targets. Because the replacement occurs while the SConscript files are being read, the [230]$CCFLAGS variable has already been set to -DDEFINE2 by the time the foo.o target is built, even though the call to the Replace method does not occur until later in the SConscript file. __________________________________________________________________ 7.5.2. Appending to the End of Values in a Construction Environment You can append a value to an existing construction variable using the Append method: env = Environment(CCFLAGS = '-DMY_VALUE') env.Append(CCFLAGS = ' -DLAST') env.Program('foo.c') SCons then supplies both the -DMY_VALUE and -DLAST flags when compiling the object file: % scons -Q cc -o foo.o -c -DMY_VALUE -DLAST foo.c cc -o foo foo.o If the construction variable doesn't already exist, the Append method will create it: env = Environment() env.Append(NEW_VARIABLE = 'added') print "NEW_VARIABLE =", env['NEW_VARIABLE'] Which yields: % scons -Q NEW_VARIABLE = added scons: `.' is up to date. __________________________________________________________________ 7.5.3. Appending to the Beginning of Values in a Construction Environment You can append a value to the beginning of an existing construction variable using the Prepend method: env = Environment(CCFLAGS = '-DMY_VALUE') env.Prepend(CCFLAGS = '-DFIRST ') env.Program('foo.c') SCons then supplies both the -DFIRST and -DMY_VALUE flags when compiling the object file: % scons -Q cc -o foo.o -c -DFIRST -DMY_VALUE foo.c cc -o foo foo.o If the construction variable doesn't already exist, the Prepend method will create it: env = Environment() env.Prepend(NEW_VARIABLE = 'added') print "NEW_VARIABLE =", env['NEW_VARIABLE'] Which yields: % scons -Q NEW_VARIABLE = added scons: `.' is up to date. __________________________________________________________________ Chapter 8. Controlling the External Environment Used to Execute Build Commands When SCons builds a target file, it does not execute the commands with the same external environment that you used to execute SCons. Instead, it uses the dictionary stored in the [231]$ENV construction variable as the external environment for executing commands. The most important ramification of this behavior is that the PATH environment variable, which controls where the operating system will look for commands and utilities, is not the same as in the external environment from which you called SCons. This means that SCons will not, by default, necessarily find all of the tools that you can execute from the command line. The default value of the PATH environment variable on a POSIX system is /usr/local/bin:/bin:/usr/bin. The default value of the PATH environment variable on a Windows system comes from the Windows registry value for the command interpreter. If you want to execute any commands--compilers, linkers, etc.--that are not in these default locations, you need to set the PATH value in the $ENV dictionary in your construction environment. The simplest way to do this is to initialize explicitly the value when you create the construction environment; this is one way to do that: path = ['/usr/local/bin', '/bin', '/usr/bin'] env = Environment(ENV = {'PATH' : path}) Assign a dictionary to the $ENV construction variable in this way completely resets the external environment so that the only variable that will be set when external commands are executed will be the PATH value. If you want to use the rest of the values in $ENV and only set the value of PATH, the most straightforward way is probably: env['ENV']['PATH'] = ['/usr/local/bin', '/bin', '/usr/bin'] Note that SCons does allow you to define the directories in the PATH in a string, separated by the pathname-separator character for your system (':' on POSIX systems, ';' on Windows): env['ENV']['PATH'] = '/usr/local/bin:/bin:/usr/bin' But doing so makes your SConscript file less portable, (although in this case that may not be a huge concern since the directories you list are likley system-specific, anyway). __________________________________________________________________ 8.1. Propagating PATH From the External Environment You may want to propagate the external PATH to the execution environment for commands. You do this by initializing the PATH variable with the PATH value from the os.environ dictionary, which is Python's way of letting you get at the external environment: import os env = Environment(ENV = {'PATH' : os.environ['PATH']}) Alternatively, you may find it easier to just propagate the entire external environment to the execution environment for commands. This is simpler to code than explicity selecting the PATH value: import os env = Environment(ENV = os.environ) Either of these will guarantee that SCons will be able to execute any command that you can execute from the command line. The drawback is that the build can behave differently if it's run by people with different PATH values in their environment--for example, if both the /bin and /usr/local/bin directories have different cc commands, then which one will be used to compile programs will depend on which directory is listed first in the user's PATH variable. __________________________________________________________________ Chapter 9. Controlling a Build From the Command Line SCons provides a number of ways that allow the writer of the SConscript files to give users a great deal of control over how to run the builds. __________________________________________________________________ 9.1. Not Having to Specify Command-Line Options Each Time: the SCONSFLAGS Environment Variable Users may find themselves supplying the same command-line options every time they run SCons. For example, a user might find that it saves time to specify a value of -j 2 to run the builds in parallel. To avoid having to type -j 2 by hand every time, you can set the external environment variable SCONSFLAGS to a string containing command-line options that you want SCons to use. If, for example, you're using a POSIX shell that's compatible with the Bourne shell, and you always want SCons to use the -Q option, you can set the SCONSFLAGS environment as follows: % scons scons: Reading SConscript files ... scons: done reading SConscript files. scons: Building targets ... ... [build output] ... scons: done building targets. % export SCONSFLAGS="-Q" % scons ... [build output] ... Users of csh-style shells on POSIX systems can set the SCONSFLAGS environment as follows: $ setenv SCONSFLAGS "-Q" Windows users may typically want to set the SCONSFLAGS in the appropriate tab of the System Properties window. __________________________________________________________________ 9.2. Getting at Command-Line Targets SCons supports a COMMAND_LINE_TARGETS variable that lets you get at the list of targets that the user specified on the command line. You can use the targets to manipulate the build in any way you wish. As a simple example, suppose that you want to print a reminder to the user whenever a specific program is built. You can do this by checking for the target in the COMMAND_LINE_TARGETS list: if 'bar' in COMMAND_LINE_TARGETS: print "Don't forget to copy `bar' to the archive!" Default(Program('foo.c')) Program('bar.c') Then, running SCons with the default target works as it always does, but explicity specifying the bar target on the command line generates the warning message: % scons -Q cc -o foo.o -c foo.c cc -o foo foo.o % scons -Q bar Don't forget to copy `bar' to the archive! cc -o bar.o -c bar.c cc -o bar bar.o Another practical use for the COMMAND_LINE_TARGETS variable might be to speed up a build by only reading certain subsidiary SConscript files if a specific target is requested. __________________________________________________________________ 9.3. Controlling the Default Targets One of the most basic things you can control is which targets SCons will build by default--that is, when there are no targets specified on the command line. As mentioned previously, SCons will normally build every target in or below the current directory by default--that is, when you don't explicitly specify one or more targets on the command line. Sometimes, however, you may want to specify explicitly that only certain programs, or programs in certain directories, should be built by default. You do this with the Default function: env = Environment() hello = env.Program('hello.c') env.Program('goodbye.c') Default(hello) This SConstruct file knows how to build two programs, hello and goodbye, but only builds the hello program by default: % scons -Q cc -o hello.o -c hello.c cc -o hello hello.o % scons -Q scons: `hello' is up to date. % scons -Q goodbye cc -o goodbye.o -c goodbye.c cc -o goodbye goodbye.o Note that, even when you use the Default function in your SConstruct file, you can still explicitly specify the current directory (.) on the command line to tell SCons to build everything in (or below) the current directory: % scons -Q . cc -o goodbye.o -c goodbye.c cc -o goodbye goodbye.o cc -o hello.o -c hello.c cc -o hello hello.o You can also call the Default function more than once, in which case each call adds to the list of targets to be built by default: env = Environment() prog1 = env.Program('prog1.c') Default(prog1) prog2 = env.Program('prog2.c') prog3 = env.Program('prog3.c') Default(prog3) Or you can specify more than one target in a single call to the Default function: env = Environment() prog1 = env.Program('prog1.c') prog2 = env.Program('prog2.c') prog3 = env.Program('prog3.c') Default(prog1, prog3) Either of these last two examples will build only the prog1 and prog3 programs by default: % scons -Q cc -o prog1.o -c prog1.c cc -o prog1 prog1.o cc -o prog3.o -c prog3.c cc -o prog3 prog3.o % scons -Q . cc -o prog2.o -c prog2.c cc -o prog2 prog2.o You can list a directory as an argument to Default: env = Environment() env.Program(['prog1/main.c', 'prog1/foo.c']) env.Program(['prog2/main.c', 'prog2/bar.c']) Default('prog1') In which case only the target(s) in that directory will be built by default: % scons -Q cc -o prog1/foo.o -c prog1/foo.c cc -o prog1/main.o -c prog1/main.c cc -o prog1/main prog1/main.o prog1/foo.o % scons -Q scons: `prog1' is up to date. % scons -Q . cc -o prog2/bar.o -c prog2/bar.c cc -o prog2/main.o -c prog2/main.c cc -o prog2/main prog2/main.o prog2/bar.o Lastly, if for some reason you don't want any targets built by default, you can use the Python None variable: env = Environment() prog1 = env.Program('prog1.c') prog2 = env.Program('prog2.c') Default(None) Which would produce build output like: % scons -Q scons: *** No targets specified and no Default() targets found. Stop. % scons -Q . cc -o prog1.o -c prog1.c cc -o prog1 prog1.o cc -o prog2.o -c prog2.c cc -o prog2 prog2.o __________________________________________________________________ 9.3.1. Getting at the List of Default Targets SCons supports a DEFAULT_TARGETS variable that lets you get at the current list of default targets. The DEFAULT_TARGETS variable has two important differences from the COMMAND_LINE_TARGETS variable. First, the DEFAULT_TARGETS variable is a list of internal SCons nodes, so you need to convert the list elements to strings if you want to print them or look for a specific target name. Fortunately, you can do this easily by using the Python map function to run the list through str: prog1 = Program('prog1.c') Default(prog1) print "DEFAULT_TARGETS is", map(str, DEFAULT_TARGETS) (Keep in mind that all of the manipulation of the DEFAULT_TARGETS list takes place during the first phase when SCons is reading up the SConscript files, which is obvious if we leave off the -Q flag when we run SCons:) % scons scons: Reading SConscript files ... DEFAULT_TARGETS is ['prog1'] scons: done reading SConscript files. scons: Building targets ... cc -o prog1.o -c prog1.c cc -o prog1 prog1.o scons: done building targets. Second, the contents of the DEFAULT_TARGETS list change in response to calls to the Default: function, as you can see from the following SConstruct file: prog1 = Program('prog1.c') Default(prog1) print "DEFAULT_TARGETS is now", map(str, DEFAULT_TARGETS) prog2 = Program('prog2.c') Default(prog2) print "DEFAULT_TARGETS is now", map(str, DEFAULT_TARGETS) Which yields the output: % scons scons: Reading SConscript files ... DEFAULT_TARGETS is now ['prog1'] DEFAULT_TARGETS is now ['prog1', 'prog2'] scons: done reading SConscript files. scons: Building targets ... cc -o prog1.o -c prog1.c cc -o prog1 prog1.o cc -o prog2.o -c prog2.c cc -o prog2 prog2.o scons: done building targets. In practice, this simply means that you need to pay attention to the order in which you call the Default function and refer to the DEFAULT_TARGETS list, to make sure that you don't examine the list before you've added the default targets you expect to find in it. __________________________________________________________________ 9.4. Getting at the List of Build Targets, Regardless of Origin We've already been introduced to the COMMAND_LINE_TARGETS variable, which contains a list of targets specified on the command line, and the DEFAULT_TARGETS variable, which contains a list of targets specified via calls to the Default method or function. Sometimes, however, you want a list of whatever targets SCons will try to build, regardless of whether the targets came from the command line or a Default call. You could code this up by hand, as follows: if COMMAND_LINE_TARGETS: targets = COMMAND_LINE_TARGETS else: targets = DEFAULT_TARGETS SCons, however, provides a convenient BUILD_TARGETS variable that eliminates the need for this by-hand manipulation. Essentially, the BUILD_TARGETS variable contains a list of the command-line targets, if any were specified, and if no command-line targets were specified, it contains a list of the targets specified via the Default method or function. Because BUILD_TARGETS may contain a list of SCons nodes, you must convert the list elements to strings if you want to print them or look for a specific target name, just like the DEFAULT_TARGETS list: prog1 = Program('prog1.c') Program('prog2.c') Default(prog1) print "BUILD_TARGETS is", map(str, BUILD_TARGETS) Notice how the value of BUILD_TARGETS changes depending on whether a target is specified on the command line: % scons -Q BUILD_TARGETS is ['prog1'] cc -o prog1.o -c prog1.c cc -o prog1 prog1.o % scons -Q prog2 BUILD_TARGETS is ['prog2'] cc -o prog2.o -c prog2.c cc -o prog2 prog2.o % scons -Q -c . BUILD_TARGETS is ['.'] Removed prog1.o Removed prog1 Removed prog2.o Removed prog2 __________________________________________________________________ 9.5. Command-Line variable=value Build Options You may want to control various aspects of your build by allowing the user to specify variable=value values on the command line. For example, suppose you want users to be able to build a debug version of a program by running SCons as follows: % scons -Q debug=1 SCons provides an ARGUMENTS dictionary that stores all of the variable=value assignments from the command line. This allows you to modify aspects of your build in response to specifications on the command line. (Note that unless you want to require that users always specify an option, you probably want to use the Python ARGUMENTS.get() function, which allows you to specify a default value to be used if there is no specification on the command line.) The following code sets the [232]$CCFLAGS construction variable in response to the debug flag being set in the ARGUMENTS dictionary: env = Environment() debug = ARGUMENTS.get('debug', 0) if int(debug): env.Append(CCFLAGS = '-g') env.Program('prog.c') This results in the -g compiler option being used when debug=1 is used on the command line: % scons -Q debug=0 cc -o prog.o -c prog.c cc -o prog prog.o % scons -Q debug=0 scons: `.' is up to date. % scons -Q debug=1 cc -o prog.o -c -g prog.c cc -o prog prog.o % scons -Q debug=1 scons: `.' is up to date. Notice that SCons keeps track of the last values used to build the object files, and as a result correctly rebuilds the object and executable files only when the value of the debug argument has changed. __________________________________________________________________ 9.6. Controlling Command-Line Build Options Being able to use a command-line build option like debug=1 is handy, but it can be a chore to write specific Python code to recognize each such option and apply the values to a construction variable. To help with this, SCons supports a class to define such build options easily, and a mechanism to apply the build options to a construction environment. This allows you to control how the build options affect construction environments. For example, suppose that you want users to set a RELEASE construction variable on the command line whenever the time comes to build a program for release, and that the value of this variable should be added to the command line with the appropriate -D option (or other command line option) to pass the value to the C compiler. Here's how you might do that by setting the appropriate value in a dictionary for the [233]$CPPDEFINES construction variable: opts = Options() opts.Add('RELEASE', 'Set to 1 to build for release', 0) env = Environment(options = opts, CPPDEFINES={'RELEASE_BUILD' : '${RELEASE}'}) env.Program(['foo.c', 'bar.c']) This SConstruct file first creates an Options object (the opts = Options() call), and then uses the object's Add method to indicate that the RELEASE option can be set on the command line, and that it's default value will be 0 (the third argument to the Add method). The second argument is a line of help text; we'll learn how to use it in the next section. We then pass the created Options object as an options keyword argument to the Environment call used to create the construction environment. This then allows a user to set the RELEASE build option on the command line and have the variable show up in the command line used to build each object from a C source file: % scons -Q RELEASE=1 cc -o bar.o -c -DRELEASE_BUILD=1 bar.c cc -o foo.o -c -DRELEASE_BUILD=1 foo.c cc -o foo foo.o bar.o __________________________________________________________________ 9.7. Providing Help for Command-Line Build Options To make command-line build options most useful, you ideally want to provide some help text that will describe the available options when the user runs scons -h. You could write this text by hand, but SCons provides an easier way. Options objects support a GenerateHelpText method that will, as its name indicates, generate text that describes the various options that have been added to it. You then pass the output from this method to the Help function: opts = Options('custom.py') opts.Add('RELEASE', 'Set to 1 to build for release', 0) env = Environment(options = opts) Help(opts.GenerateHelpText(env)) SCons will now display some useful text when the -h option is used: % scons -Q -h RELEASE: Set to 1 to build for release default: 0 actual: 0 Use scons -H for help about command-line options. Notice that the help output shows the default value, and the current actual value of the build option. __________________________________________________________________ 9.8. Reading Build Options From a File Being able to use a command-line build option like debug=1 is handy, but it can be a chore to write specific Python code to recognize each such option and apply the values to a construction variable. To help with this, SCons supports a class to define such build options easily and to read build option values from a file. This allows you to control how the build options affect construction environments. The way you do this is by specifying a file name when you call Options, like custom.py in the following example: opts = Options('custom.py') opts.Add('RELEASE', 'Set to 1 to build for release', 0) env = Environment(options = opts, CPPDEFINES={'RELEASE_BUILD' : '${RELEASE}'}) env.Program(['foo.c', 'bar.c']) Help(opts.GenerateHelpText(env)) This then allows us to control the RELEASE variable by setting it in the custom.py file: RELEASE = 1 Note that this file is actually executed like a Python script. Now when we run SCons: % scons -Q cc -o bar.o -c -DRELEASE_BUILD=1 bar.c cc -o foo.o -c -DRELEASE_BUILD=1 foo.c cc -o foo foo.o bar.o And if we change the contents of custom.py to: RELEASE = 0 The object files are rebuilt appropriately with the new option: % scons -Q cc -o bar.o -c -DRELEASE_BUILD=0 bar.c cc -o foo.o -c -DRELEASE_BUILD=0 foo.c cc -o foo foo.o bar.o __________________________________________________________________ 9.9. Canned Build Options SCons provides a number of functions that provide ready-made behaviors for various types of command-line build options. __________________________________________________________________ 9.9.1. True/False Values: the BoolOption Build Option It's often handy to be able to specify an option that controls a simple Boolean variable with a true or false value. It would be even more handy to accomodate users who have different preferences for how to represent true or false values. The BoolOption function makes it easy to accomodate a variety of common values that represent true or false. The BoolOption function takes three arguments: the name of the build option, the default value of the build option, and the help string for the option. It then returns appropriate information for passing to the Add method of an Options object, like so: opts = Options('custom.py') opts.Add(BoolOption('RELEASE', 'Set to build for release', 0)) env = Environment(options = opts, CPPDEFINES={'RELEASE_BUILD' : '${RELEASE}'}) env.Program('foo.c') With this build option, the RELEASE variable can now be enabled by setting it to the value yes or t: % scons -Q RELEASE=yes foo.o cc -o foo.o -c -DRELEASE_BUILD=True foo.c % scons -Q RELEASE=t foo.o cc -o foo.o -c -DRELEASE_BUILD=True foo.c Other values that equate to true include y, 1, on and all. Conversely, RELEASE may now be given a false value by setting it to no or f: % scons -Q RELEASE=no foo.o cc -o foo.o -c -DRELEASE_BUILD=False foo.c % scons -Q RELEASE=f foo.o cc -o foo.o -c -DRELEASE_BUILD=False foo.c Other values that equate to false include n, 0, off and none. Lastly, if a user tries to specify any other value, SCons supplies an appropriate error message: % scons -Q RELEASE=bad_value foo.o scons: *** Error converting option: RELEASE Invalid value for boolean option: bad_value File "/home/my/project/SConstruct", line 4, in __________________________________________________________________ 9.9.2. Single Value From a List: the EnumOption Build Option Suppose that we want a user to be able to set a COLOR option that selects a background color to be displayed by an application, but that we want to restrict the choices to a specific set of allowed colors. This can be set up quite easily using the EnumOption, which takes a list of allowed_values in addition to the variable name, default value, and help text arguments: opts = Options('custom.py') opts.Add(EnumOption('COLOR', 'Set background color', 'red', allowed_values=('red', 'green', 'blue'))) env = Environment(options = opts, CPPDEFINES={'COLOR' : '"${COLOR}"'}) env.Program('foo.c') The user can now explicity set the COLOR build option to any of the specified allowed values: % scons -Q COLOR=red foo.o cc -o foo.o -c -DCOLOR="red" foo.c % scons -Q COLOR=blue foo.o cc -o foo.o -c -DCOLOR="blue" foo.c % scons -Q COLOR=green foo.o cc -o foo.o -c -DCOLOR="green" foo.c But, almost more importantly, an attempt to set COLOR to a value that's not in the list generates an error message: % scons -Q COLOR=magenta foo.o scons: *** Invalid value for option COLOR: magenta File "/home/my/project/SConstruct", line 5, in The EnumOption function also supports a way to map alternate names to allowed values. Suppose, for example, that we want to allow the user to use the word navy as a synonym