INTRO
This aims to be a complete, usable, and fairly non-trivial SCons example for C/C++.
I set out with the goal of making a simple to use SCons-based build set up that would automatically build Release and Debug variants into separate directories. Additionally I sometimes like to have my build consolidate all the libs, executables, and/or headers into given directories after building. For instance creating, a source_tree/Lib dir and a source_tree/Include directory for the libs and public header files for utility library projects.
And when you build multiple variations of a lib, some Debug some Release, you often want to give them different names, like 'foo.lib' vs 'foo_dbg.lib' (or 'libfoo.a' vs 'libfoo_dbg.a'). The scripts below handle all that, while providing a few other niceties.
This is still a work in progress, but already it does a lot of things I find useful.
Naturally, you want your SConscript files to be as simple as possible and basically have them 'include' all your standard rules for building. I considered doing that via the Python execfile() function, and storing the default rules in a separate file at the build root, but in the end it seemed like I might as well just Export/Import a build object with all the smarts in methods. Then the global build object can just be defined in the main SConstruct. Originally I thought it would be good to put all the source and target data etc into a big dictionary structure in the SConscript files and pass that to my processing methods, but upon consideration it became clear that what I was really accomplishing with all that was decorating, or 'wrapping', the default Program() and Library() builder methods. The WrapperFunctions Wiki page describes exactly how to do this in a much more straightforward way. Part of the benefit of a wrappers approach is that to the SConscript writer, the syntax looks pretty much like the basic SConscript examples in the SCons User's Guide.
So basically the result is that the SConscript just calls some things that look pretty similar to Program() and Library() calls (specifically, I've named them dProgram() and dLibrary()), but all the extra things happen that I described above. For example
Complete examples of the SConstruct files are included below.
OVERVIEW
So what's good about it:
- The SConscripts are relatively simple
- Easy to write initially
- Easier to edit by non-SCons experts
- All the logic is in the SConstruct, so only need to modify SConstruct to change whole build behavior.
- Variant build products are put in separate sub-directories (build/Debug, build/Release)
Using the convention of debug=1 to specify a debug build
Just two variants right now, Debug and Release
- Each sub-SConscript can define multiple libs and exes to build and install, and also specify headers to install.
Build products and sources are described using same syntax as the built-in builders, like Program()
New functions include dProgram(),dLibrary(),dSubdirs(), and dHeaders(). The latter installs headers.
- Lists of source files can include glob patterns
Glob patterns search the source dir not the build dir. The latter is SCons' not-very-useful default.
Glob patterns relative to the root using the '#' character (e.g. #subdir/*.cpp) also work properly.
- Optional install step for libs apps and headers.
- Not a system install, but copying elsewhere in the build tree
- E.g., allows copying all libs to a #lib directory
- E.g., allows copying all exes to a #bin directory
- E.g., allows copying all include files to a #include directory
Optional decoration of lib names based on build variant. E.g. tack on a '_dbg' to the end of library names in the debug variant.
What needs work:
The mechanism for linking with external libraries. (ParseConfig?)
- Proper settings of compiler flags for more compilers.
- External specification of site-specific info like standard include dirs.
- Overall better separation of config data and instructions in the SConstruct file. (Currently there are many things you have to delve into the guts of the SConstruct to change, like the names of the install directories for libs, programs, and headers.)
Other possible negatives:
- Too C/C++ specific.
- Recent Python required (I use a fair number of list comprehensions. And being pretty new to Python, I may be using other 'new' Python stuff without realizing it, because it's all new to me.)
THE CODE
Here are the files.
An example SConscript for a dir with some libraries:
There's only one lib there, but you can of course make multiple dLibrary calls. Also note that not all the sources for this lib are located in the directory with this SConscript. That's fine. The dHeaders call is used to install the headers listed in the '#Include' directory of the tree. Note that it can take a single string that include a combination of globs, build relative globs, and non-globs. Any non-list argument is automatically Split() and anything that looks like a glob is globbed on (and globbed properly, in the source directory, not in the build directory as is the default SCons behavior)
An example SConscript for a dir with several binaries:
The above example uses a loop, but you could certainly also just write each one, one-by-one, if you're afraid of Python and find that more intuitive.
Finally, the top-level Root SConscript just calls subdirs:
And now the main SConstruct file:
1 # The goal here is to be able to build libs and programs using just
2 # the list of sources.
3 # I wish to 'wrap' or 'decorate' the base scons behavior in
4 # a few ways:
5 # * default variant builds ('Debug'/'Release')
6 # * automatic build dirs (which are variant-sensitive)
7 # - with work-arounds to make *everything* go in the build dir
8 # * automatic no-fuss globs (that *also* work properly for build dirs)
9 # * default platform-specific build env vars (also variant-sensitive)
10 # (i.e. some extra compiler specific CCFLAGS and such)
11 # * automatic 'installation' of libs, program, and headers into specific
12 # directories (e.g. '#lib', '#bin', and '#include')
13 import sys
14
15 ############################################
16 # Site-specific setup
17
18 # Hmm... this should go somewhere else. SCustomize??
19 # This is my own site-specific win32 setup...
20 if sys.platform == 'win32':
21 stdinc = [r'c:\usr\include']
22 stdlibinc = [r'c:\usr\lib']
23 else:
24 stdinc = []
25 stdlibinc = []
26
27 ############################################
28 # Generic boiler plate
29 #-------------------------------------------
30 import os
31 import os.path
32
33 opts = Options('SCustomize')
34 opts.Add('debug', 'Build with debugging symbols', 0)
35 opts.Add('CC', 'Set C compiler')
36
37 env = Environment(options=opts)
38 Help(opts.GenerateHelpText(env))
39
40 debug = env.get('debug',0)
41 build_base = 'build'
42
43 if debug:
44 env.Append(CPPDEFINES = ['DEBUG', '_DEBUG'])
45 variant = 'Debug'
46 else:
47 env.Append(CPPDEFINES = ['NDEBUG'])
48 variant = 'Release'
49
50 ############################################
51 # PLATFORM SPECIFIC CONFIGS
52 ############################################
53 #-------------- win32 MSVC -----------------
54 if env['CC'] == 'cl':
55 def freeMSVCHack(env, vclibs):
56 # SCons automatically finds full versions of msvc via the registry, so
57 # if it can't find 'cl', it may be because we're trying to use the
58 # free version
59 def isMicrosoftSDKDir(dir):
60 return os.path.exists(os.path.join(dir, 'Include', 'Windows.h')) and os.path.exists(os.path.join(dir, 'Lib', 'WinMM.lib'))
61
62 def findMicrosoftSDK():
63 import SCons.Platform.win32
64 import SCons.Util
65 import re
66 if not SCons.Util.can_read_reg:
67 return None
68 HLM = SCons.Util.HKEY_LOCAL_MACHINE
69 K = r'Software\Microsoft\.NETFramework\AssemblyFolders\PSDK Assemblies'
70 try:
71 k = SCons.Util.RegOpenKeyEx(HLM, K)
72 p = SCons.Util.RegQueryValueEx(k,'')[0]
73 # this should have \include at the end, so chop that off
74 p = re.sub(r'(?i)\\+Include\\*$','',p)
75 if isMicrosoftSDKDir(p): return p
76 except SCons.Util.RegError:
77 pass
78
79 K = r'SOFTWARE\Microsoft\MicrosoftSDK\InstalledSDKs'
80 try:
81 k = SCons.Util.RegOpenKeyEx(HLM, K)
82 i=0
83 while 1:
84 p = SCons.Util.RegEnumKey(k,i)
85 i+=1
86 subk = SCons.Util.RegOpenKeyEx(k, p)
87 try:
88 p = SCons.Util.RegQueryValueEx(subk,'Install Dir')[0]
89 # trim trailing backslashes
90 p = re.sub(r'\\*$','',p)
91 if isMicrosoftSDKDir(p): return p
92 except SCons.Util.RegError:
93 pass
94 except SCons.Util.RegError:
95 pass
96
97 return None
98
99 # End of local defs. Actual freeMSVCHack begins here
100 if not env['MSVS'].get('VCINSTALLDIR'):
101 if os.environ.get('VCToolkitInstallDir'):
102 vcdir=os.environ['VCToolkitInstallDir']
103 env.PrependENVPath('INCLUDE', os.path.join(vcdir, 'Include'))
104 env.PrependENVPath('LIB', os.path.join(vcdir, 'Lib'))
105 env.PrependENVPath('PATH', os.path.join(vcdir, 'Bin'))
106 env['MSVS']['VERSION'] = '7.1'
107 env['MSVS']['VERSIONS'] = ['7.1']
108 if not env['MSVS'].get('PLATFORMSDKDIR'):
109 sdkdir = findMicrosoftSDK()
110 if sdkdir:
111 env.PrependENVPath('INCLUDE', os.path.join(sdkdir, 'Include'))
112 env.PrependENVPath('LIB', os.path.join(sdkdir, 'Lib'))
113 env.PrependENVPath('PATH', os.path.join(sdkdir, 'Bin'))
114 env['MSVS']['PLATFORMSDKDIR']=sdkdir
115 # FREE MSVC7 only allows
116 # /ML(libc) /MT(libcmt) or /MLd(libcd)
117 # Full IDE versions also have
118 # /MD(msvcrtd) /MTd(libcmtd) and /MDd(msvcrtd)
119 # So if you want to debug with the freever, the only option is
120 # the single-threaded lib, /MLd
121 vclibs['Debug']='/MLd'
122 vclibs['Release']='/MT'
123
124 # MSVC SETUP
125 # MDd is for multithreaded debug dll CRT (msvcrtd)
126 # MD is for multithreaded dll CRT (msvcrt)
127 # These are just my preferences
128 vclibs = {'Debug':'/MDd','Release':'/MD'}
129 freeMSVCHack(env, vclibs)
130
131 env.Append(CCFLAGS=[vclibs[variant]])
132 if debug:
133 env.Append(CCFLAGS=Split('/Zi /Fd${TARGET}.pdb'))
134 env.Append(LINKFLAGS = ['/DEBUG'])
135 # env.Clean('.', '${TARGET}.pdb')
136 # Need to clean .pdbs somehow! The above line doesn't work!
137 else:
138 env.Append(CCFLAGS=Split('/Og /Ot /Ob1 /Op /G6'))
139
140 env.Append(CCFLAGS=Split('/EHsc /J /W3 /Gd'))
141 env.Append(CPPDEFINES=Split('WIN32 _WINDOWS'))
142
143 #-------------- gcc-like (default) ---------
144 else: # generic posix-like
145 if debug:
146 env.Append(CPPFLAGS = ['-g'])
147 else:
148 env.Append(CPPFLAGS = ['-O3'])
149 #-------------------------------------------
150
151
152 # Put all the little .sconsign files into one big file.
153 # (Does this slow down parallel builds?)
154 # Need to create the build dir before we put the signatures db in there
155 fullbuildpath = Dir(build_base).abspath
156 if not os.path.exists(fullbuildpath): os.makedirs(fullbuildpath)
157 import dbhash
158 env.SConsignFile(os.path.join(build_base, 'sconsignatures'), dbhash)
159
160
161 # Make a singleton global object for keeping track of all the extra data
162 # and methods that are being added
163 class Globals:
164 def __init__(self):
165 self.env = env
166 self.stdinc = stdinc
167 self.stdlibinc = stdlibinc
168 self.variant = variant
169 self.build_base = os.path.join(build_base, variant)
170 self.libname_decorators = { 'Debug' : '_dbg' }
171 self.appname_decorators = { 'Debug' : '_dbg' }
172 self.incinstdir = '#include'
173 self.libinstdir = '#lib'
174 self.appinstdir = '#bin'
175 self.objcache = {}
176
177 def Glob(self, pat):
178 ## GLOB IN THE REAL SOURCE DIRECTORY (NOT BUILD DIR)
179 import glob
180 prevdir = os.getcwd();
181 if pat[0] != '#':
182 os.chdir(self.env.Dir('.').srcnode().abspath)
183 ret = glob.glob(pat)
184 else:
185 pat = pat[1:]
186 base = os.path.dirname(pat)
187 searchdir = self.env.Dir('#').srcnode().abspath
188 os.chdir(searchdir)
189 ret = ['#'+x for x in glob.glob(pat)]
190 os.chdir(prevdir)
191 return ret
192
193 def GlobExpand(self, list):
194 ## look for pattern-like things and glob on those
195 ret = []
196 for item in list:
197 if item.find('*') or item.find('?') or item.find('['):
198 ret += self.Glob(item)
199 else:
200 ret += [item]
201 return ret
202
203 def IsALocalLib(self, lib):
204 # This is rather heuristic determining if a lib is local or not
205 return lib.find('/') or lib.find(os.sep) or lib[0]=='#'
206
207 def MyHeaderMethod(self, env, source, **dict):
208 if type(source)==type(''): source = Split(source)
209 source = self.GlobExpand(source)
210 nodes = []
211 if hasattr(self,'incinstdir') and self.incinstdir:
212 for i in source:
213 nodes.append( env.Install(self.incinstdir, i) )
214 return nodes
215
216 def MyLibraryMethod(self, env, **dict):
217 relincs = dict.get('CPPPATH',[])
218 dict['CPPPATH'] = ['.'] + relincs + self.stdinc
219
220 # These shenanigans are necessary to get SCons to build non-local
221 # sources in the BuildDir instead of their own local directories
222 target = dict.pop('target')
223 source = dict.pop('source')
224 if type(source)==type(''): source = Split(source)
225 allsrc = []
226 for x in source:
227 objbase = os.path.basename(x)
228 if self.objcache.get(objbase):
229 #print 'Reusing node', objbase
230 #NOTE: We should check that defines etc are all the same!!
231 allsrc += self.objcache[objbase]
232 else:
233 onode = env.SharedObject(
234 os.path.splitext(objbase)[0], x,
235 **dict)
236 allsrc += onode
237 self.objcache[objbase]=onode
238
239 targpath = '#' + os.path.join(self.build_dir, target)
240 # decorate libname with e.g. '_dbg'
241 if hasattr(self,'libname_decorators'):
242 targpath += self.libname_decorators.get(self.variant,'')
243
244 dict['source'] = allsrc
245 dict['target'] = targpath
246 node = env.Library(**dict)
247 if hasattr(self,'libinstdir') and self.libinstdir:
248 env.Install(self.libinstdir, node)
249 return node
250
251 def MyProgramMethod(self, env, **dict):
252 # Enhance CPPPATH,LIBPATH
253 dict['CPPPATH'] = ['.'] + dict.get('CPPPATH',[]) + self.stdinc
254 dict['LIBPATH'] = dict.get('LIBPATH',[]) + self.stdlibinc
255
256 # These shenanigans are necessary to get SCons to build non-local
257 # sources in the BuildDir instead of their own local directories
258 target = dict.pop('target')
259 source = dict.pop('source',[])
260 if type(source)==type(''): source = Split(source)
261 allsrc = []
262 for x in source:
263 objbase = os.path.basename(x)
264 if self.objcache.get(objbase):
265 print 'Reusing node', objbase
266 allsrc += self.objcache[objbase]
267 else:
268 onode = self.env.SharedObject(
269 os.path.splitext(objbase)[0], x,
270 **dict)
271 allsrc += onode
272 self.objcache[objbase]=onode
273
274 targpath = '#' + os.path.join(self.build_dir, target)
275 # decorate app name with e.g. '_dbg'
276 if hasattr(self,'appname_decorators'):
277 deco = self.appname_decorators.get(self.variant,'')
278 targpath += deco
279 # decorate local lib names with e.g. '_dbg'
280 if hasattr(self,'libname_decorators'):
281 deco = self.libname_decorators.get(self.variant,'')
282 # decorate source lib names with e.g. '_dbg'
283 LIBS = []
284 for l in dict.pop('LIBS',[]):
285 if self.IsALocalLib(l):
286 #print "Decorating lib", l, '->', l+deco
287 LIBS += [l+deco]
288 else:
289 LIBS += [l]
290
291 dict['target']=targpath
292 dict['source']=allsrc
293 dict['LIBS']=LIBS
294 node = env.Program(**dict)
295
296 if hasattr(self,'appinstdir') and self.appinstdir:
297 self.env.Install(self.appinstdir, node)
298
299 return node
300
301 def MySubdirsMethod(self, env, subdirs, **dict):
302 # Build sub-directories
303 if type(subdirs)==type(''): subdirs=Split(subdirs)
304 for d in subdirs:
305 savedir = self.build_dir
306 self.build_dir = os.path.join(self.build_dir, d)
307 env.SConscript(os.path.join(d, 'SConscript'),
308 exports=['G','env'])
309 self.build_dir = savedir
310
311
312 G = Globals()
313
314 # Wrap the methods of G into method objects, and then add them
315 # as methods to Environment.
316 # See http://www.scons.org/cgi-bin/wiki/WrapperFunctions
317 def MyLibraryMethod(env, **dict):
318 G.MyLibraryMethod(env, **dict)
319 def MyHeaderMethod(env, source, **dict):
320 G.MyHeaderMethod(env, source, **dict)
321 def MyProgramMethod(env, **dict):
322 G.MyProgramMethod(env, **dict)
323 def MySubdirsMethod(env, subdirs, **dict):
324 G.MySubdirsMethod(env, subdirs, **dict)
325 from SCons.Script.SConscript import SConsEnvironment # just do this once
326 SConsEnvironment.dLibrary = MyLibraryMethod
327 SConsEnvironment.dProgram = MyProgramMethod
328 SConsEnvironment.dHeaders = MyHeaderMethod
329 SConsEnvironment.dSubdirs = MySubdirsMethod
330
331 G.build_dir = G.build_base
332
333 # Call the SConscript in the top-level directory
334 env.SConscript('SConscript',build_dir=G.build_dir,exports=['env','G'])
Note that a big chunk of that code has to do with trying to find out if the free VC++ toolkit is installed. SCons doesn't currently (ver 0.96.90) detect this even if it is the only compiler available. I just happened to be using the free toolkit while I waited for my actual copy of Visual Studio to arrive, when I started playing with SCons. Consequently I spent a lot of my first day or so with SCons struggling to figure out why on earth the simplest compile with VC required so much code to get the environment right. Well since I struggled through it, I'm including the results of that struggle here in the form of the freeMSVCHack() function above. Hopefully something similar will be built into future releases of SCons.
That's it for now. This is still a work in progress as of 2/18/05, JST. I'll keep updating this page as I make progress. Feedback from the experts is much appreciated.
One last thing, why did I name all my functions like dProgram or dLibrary? What's with the d?
The answer is just I wanted to use names similar to the existing builders I was wrapping, and d seemed like a decent prefix. Maybe it stands for different or decorated or decent. Or maybe dorky 
Bill Baxter
ACKNOWLEDGEMENTS
Thanks to Steven Knight, Dobes Vandermeer, Gary Oberbrunner and the other folks on the SCons users mailing list for helpful suggestions, pointers, and advice. And of course a big thanks to Steven Knight and the other developers of SCons.