Please note:The SCons wiki is now restored from the attack in March 2013. All old passwords have been invalidated. Please reset your password if you have an account. If you note missing pages, please report them to webmaster@scons.org. Also, new account creation is currently disabled due to an ongoing spam flood (2013/08/27).

Using Microsoft Visual C++ with SCons is hard. We struggled for a long time to get decent build speed in a "debug" configuration.

It turns out that incremental linking is the only way to get fast "debug" builds. However, we found problems at every step along the way to get this feature working within SCons.

The easiest and most elegant way we could find to make this the default behavior was to replace the Program and SharedLibrary builders with custom ones that do extra steps.

Here's what we did:

   1 envDebug = '... your global environment ...'.Clone()
   2 #------------------------------------------------------------
   3 # Link with debugging informations   
   4 # 
   5 envDebug.AppendUnique(LINKFLAGS=['/DEBUG'])
   6 #------------------------------------------------------------
   7 # Dynamically link with the debugging CRT
   8 # 
   9 envDebug.AppendUnique(CCFLAGS=['/MDd'])
  10 #------------------------------------------------------------
  11 # Disable optimizations
  12 # 
  13 envDebug.AppendUnique(CCFLAGS=['/Od'])
  14 #------------------------------------------------------------
  15 # Produce one .PDB file per .OBJ when compiling, then merge them when linking.
  16 # Doing this enables parallel builds to work properly (the -j parameter).
  17 # See: http://www.scons.org/doc/HTML/scons-man.html section CCPDBFLAGS
  18 # 
  19 envDebug['CCPDBFLAGS'] = '/Zi /Fd${TARGET}.pdb'
  20 envDebug['PDB']='${TARGET.base}.pdb'
  21 #------------------------------------------------------------
  22 # Link incrementally. Produces larger (and possibly corrupted files,
  23 # but takes less time to build. Not recommended for final build.
  24 # 
  25 envDebug.AppendUnique(LINKFLAGS=['/INCREMENTAL'])
  26 #------------------------------------------------------------
  27 # Hacks to allow incremental linking on Microsoft Visual C++.
  28 # 
  29 # 1 - Don't generate a .manifest file, we're doing that below
  30 # 
  31 envDebug.AppendUnique(LINKFLAGS=['/MANIFEST:NO'])
  32 # 
  33 # 2 - Override the SharedLibrary builder to add some extra steps
  34 # 
  35 def SharedLibraryIncrementallyLinked(env, library, sources, **args):
  36    # 
  37    # Hack 1: Embed a manifest while linking
  38    # 
  39    # Since we can't embed the .manifest file *AFTER* linking, because it
  40    # would modify the binary and prevent subsequent incremental linking,
  41    # we must embed it *WHILE* linking. We achieve that by generating a
  42    # manifest file from a dummy program created with the current
  43    # environment. This manifest file is then added to a resource, which
  44    # is compiled into an object file and linked into the final binary.
  45    # 
  46    subBuild = env.Clone()
  47    subBuild['WINDOWS_INSERT_MANIFEST'] = True
  48    subBuild.AppendUnique(LINKFLAGS='/MANIFEST')
  49    subBuild.AppendUnique(LINKFLAGS='/INCREMENTAL:NO')
  50    if '/MANIFEST:NO' in subBuild['LINKFLAGS']:
  51       subBuild['LINKFLAGS'].remove('/MANIFEST:NO')
  52    if '/INCREMENTAL' in subBuild['LINKFLAGS']:
  53       subBuild['LINKFLAGS'].remove('/INCREMENTAL')
  54    if '/DEBUG' in subBuild['LINKFLAGS']:
  55       subBuild['LINKFLAGS'].remove('/DEBUG')
  56    del subBuild['PDB']
  57    def createDummySourceFile(env, target, source):
  58       file(target[0].abspath, 'w').write("int main(int, char **) { return 0; }\n")
  59    dummyName = library[0] + '_dummy_for_manifest'
  60    dummySourceFile = subBuild.Command(dummyName + '.cpp', None, createDummySourceFile)
  61    dummyProgram = subBuild.ProgramOriginal(dummyName + '.exe', dummySourceFile)
  62    dummyManifest = dummyProgram[1]
  63    def createManifestResourceFile(env, target, source):
  64       file(target[0].abspath, 'w').write('2 24 "%s"' % source[0].abspath.replace('\\','\\\\'))
  65    manifestResourceFile = subBuild.Command(dummyName + '.rc', dummyManifest, createManifestResourceFile)
  66    manifestResource = subBuild.RES(dummyName + '.res', manifestResourceFile)
  67    # 
  68    # Hack 2: Precious binary
  69    # 
  70    # By default, SCons will delete the files before re-building them.
  71    # This prevents incremental linking from working because it relies
  72    # on timestamps. Therefore, we must prevent SCons from deleting the
  73    # the build products. This is achieved by making them as "Precious".
  74    # 
  75    library = env.SharedLibraryOriginal(library, [sources, manifestResource], **args)
  76    env.Precious(library)
  77    return library
  78 envDebug['BUILDERS']['SharedLibraryOriginal'] = envDebug['BUILDERS']['SharedLibrary']
  79 envDebug['BUILDERS']['SharedLibrary'] = SharedLibraryIncrementallyLinked
  80 # 
  81 # 3 - Override the Program builder to add some extra steps
  82 # 
  83 def ProgramIncrementallyLinked(env, program, sources, **args):
  84    # 
  85    # Hack 1: Embed a manifest while linking
  86    # 
  87    # Since we can't embed the .manifest file *AFTER* linking, because it
  88    # would modify the binary and prevent subsequent incremental linking,
  89    # we must embed it *WHILE* linking. We achieve that by generating a
  90    # manifest file from a dummy program created with the current
  91    # environment. This manifest file is then added to a resource, which
  92    # is compiled into an object file and linked into the final binary.
  93    # 
  94    subBuild = env.Clone()
  95    subBuild['WINDOWS_INSERT_MANIFEST'] = True
  96    subBuild.AppendUnique(LINKFLAGS='/MANIFEST')
  97    subBuild.AppendUnique(LINKFLAGS='/INCREMENTAL:NO')
  98    if '/MANIFEST:NO' in subBuild['LINKFLAGS']:
  99       subBuild['LINKFLAGS'].remove('/MANIFEST:NO')
 100    if '/INCREMENTAL' in subBuild['LINKFLAGS']:
 101       subBuild['LINKFLAGS'].remove('/INCREMENTAL')
 102    if '/DEBUG' in subBuild['LINKFLAGS']:
 103       subBuild['LINKFLAGS'].remove('/DEBUG')
 104    del subBuild['CCPDBFLAGS']
 105    del subBuild['PDB']
 106    def createDummySourceFile(env, target, source):
 107       file(target[0].abspath, 'w').write("int main(int, char **) { return 0; }\n")
 108    dummyName = program[0] + '_dummy_for_manifest'
 109    dummySourceFile = subBuild.Command(dummyName + '.cpp', None, createDummySourceFile)
 110    dummyProgram = subBuild.ProgramOriginal(dummyName + '.exe', dummySourceFile)
 111    dummyManifest = dummyProgram[1]
 112    def createManifestResourceFile(env, target, source):
 113       file(target[0].abspath, 'w').write('1 24 "%s"' % source[0].abspath.replace('\\','\\\\'))
 114    manifestResourceFile = subBuild.Command(dummyName + '.rc', dummyManifest, createManifestResourceFile)
 115    manifestResource = subBuild.RES(dummyName + '.res', manifestResourceFile)
 116    # 
 117    # Hack 2: Precious binary
 118    # 
 119    # By default, SCons will delete the files before re-building them.
 120    # This prevents incremental linking from working because it relies
 121    # on timestamps. Therefore, we must prevent SCons from deleting the
 122    # the build products. This is achieved by making them as "Precious".
 123    # 
 124    program = env.ProgramOriginal(program, [sources, manifestResource], **args)
 125    env.Precious(program)
 126    return program
 127 envDebug['BUILDERS']['ProgramOriginal'] = envDebug['BUILDERS']['Program']
 128 envDebug['BUILDERS']['Program'] = ProgramIncrementallyLinked

Then you use the builders as usual and you'll get incremental linking:

   1 env = envDebug
   2 env.Program('MyProgram', 'MyProgram.cpp')
   3 env.SharedLibrary('MyLibrary', 'MyLibrary.cpp')

MsvcIncrementalLinking (last edited 2011-02-16 05:24:41 by FrancisBolduc)