Please note:The SCons wiki is now restored from the attack in March 2013. All old passwords have been invalidated. Please reset your password if you have an account. If you note missing pages, please report them to webmaster@scons.org. Also, new account creation is currently disabled due to an ongoing spam flood (2013/08/27).
Differences between revisions 12 and 13
Revision 12 as of 2013-08-27 19:23:20
Size: 342
Editor: WayneWinc
Comment:
Revision 13 as of 2013-08-28 20:43:58
Size: 6548
Comment: Spam
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
I am Earle from M�. I am learning to play the Pedal Steel Guitar. Other hobbies are Games Club - Dungeons and Dragons, Monopoly, Etc..<<BR>>
<<BR>>
Feel free to surf to my website - [[http://www.christianteenagers.net/blog/46777/shy-plus-randy-women-lick-every-other-individuals%E2%80%99-twats-preceding-shoving-/|sexual opportunities]]
= Using a Source Generator to Add Targets Dynamically =

Here's what we do to add targets dynamically, from a list which isn't known in advance.

The scenario in our case is you have a source generator 'srcgen' which is built by scons from sources in the usual way. It puts out a bunch of other sources whose names can't be predicted in advance. In our case it also puts out a src-list.text file. Call the generated sources srcA, srcB, etc.

Now you want to (say) compile each of those srcX into binX and include those in your final product.

In our SConscript we do this:

{{{
#!python
srcgen = env.Program('srcgen', ...)
srclist = env.Command('src-list.txt', srcgen,
                      '$SOURCE $SRCGEN_ARGS > $TARGET')
dummy = env.ScanSrcs('#dummy-target', srclist, SCANSRCS_FUNC=add_target)
env.AlwaysBuild(dummy)
Alias('TopLevelAlias', dummy)
}}}

Now when scons goes to build !TopLevelAlias, the dependencies are set so it builds srcgen, runs it to produce the srclist (and the other sources), and then runs !ScanSrcs on the srclist. The interesting thing is that the !ScanSrcs function will be called ''during the build phase'' after its dependencies are up-to-date, rather than while the SConscript is being read. It will call Program() or !SharedLibrary() or whatever builders we want, which will update the dependency graph while scons is building. Sounds scary, but it works.

We made our !ScanSrcs parameterized so it could be reused. Note that it's practically all plain python, no SCons calls at all except adding the Builder and checking for construction variables.

{{{
#!python
    def scansrcs(target, source, env):
        """Scans through the list in the file 'source', calling
        env['SCANSRCS_FUNC'](env,line) on each line of that file.
        Calls env['SCANSRCS_PREFUNC'](env,source) once before scanning.
        """
        if SCons.Util.is_List(source):
            source=source[0]
        if 'SCANSRCS_FUNC' not in env:
            raise Error, "You must define SCANSRCS_FUNC as a scons env function."
        # Call the pre-func
        if 'SCANSRCS_PREFUNC' in env:
            env['SCANSRCS_PREFUNC'](env, source.path)

        try:
            f=open(source.path, 'r')
        except:
            print "scansrcs: Can't open source list file '%s' in %s" % \
                  (source.path,os.getcwd())
            raise
        # Scan through the lines
        for line in f:
            src = line.strip()
            # print "Found " + src
            try:
                env['SCANSRCS_FUNC'](env, src)
            except:
                print "SCANSRCS func raised exception:"
                raise
        f.close()
    # This is a funky builder, because it never creates its target.
    # Should always be called with a fake target name.
    env.Append(BUILDERS = {'ScanSrcs' : Builder(action=scansrcs)})
}}}

And finally, here's the function ''add_target'', called from !ScanSrcs, that does the real work:

{{{
#!python
    def add_target(env, source):
        """Add scons commands to build a new target from a scanned src generator list file."""
        # Build the new targets in the build dir:
        target = '#'+join(env['BUILDDIR'], basename(re.sub(r'\.c$', env.get('PROGSUFFIX',''), source)))
        tgt = env.Program(target, source)
        env.Depends(tgt, '#dummy-target')
        p = env.Install(env['INSTALLDIR'], tgt)
        Alias('TopLevelAlias', p)
}}}

Note that it adds new Nodes for the program target and the installed version of it, and also adds that to the !TopLevelAlias. This way as the scanner runs scons knows that the new target doesn't exist or is out of date, so it marks the !TopLevelAlias out of date; then scons comes around again to look at !TopLevelAlias and discovers it now needs to build 'binA', which depends on 'srcA' and so it builds it and installs it as we want it to.

The above has been cut down from our real version, but I hope it shows how it can be done, and that it's not really that hard.

----

Here is a version of the code above that you can copy & paste, ready to edit. I added some imports and altered some small stuff that was too related to the code's origins. Now this example works as soon as you change the file names to match those in your project.

{{{
#!python
import types
import os.path
import re

env = Environment(BUILDDIR = "bin",INSTALLDIR = "installdir")

def scansrcs(target, source, env):
        """Scans through the list in the file 'source', calling
        env['SCANSRCS_FUNC'](env,line) on each line of that file.
        Calls env['SCANSRCS_PREFUNC'](env,source) once before scanning.
        """
        if type(source) is list:
            source=source[0]
        if 'SCANSRCS_FUNC' not in env:
            raise Error, "You must define SCANSRCS_FUNC as a scons env function."
        # Call the pre-func
        if 'SCANSRCS_PREFUNC' in env:
            env['SCANSRCS_PREFUNC'](env, source.path)

        try:
            f=open(source.path, 'r')
        except:
            print "scansrcs: Can't open source list file '%s' in %s" % \
                  (source.path,os.getcwd())
            raise
        # Scan through the lines
        for line in f:
            src = line.strip()
            # print "Found " + src
            try:
                env['SCANSRCS_FUNC'](env, src)
            except:
                print "SCANSRCS func raised exception:"
                raise
        f.close()


# This is a funky builder, because it never creates its target.
# Should always be called with a fake target name.
env.Append(BUILDERS = {'ScanSrcs' : Builder(action=scansrcs)})


def add_target(env, source):
        """Add scons commands to build a new target from a scanned src generator list file."""
        # Build the new targets in the build dir:
        target = '#' + os.path.join(env['BUILDDIR'], os.path.basename(re.sub(r'\.c$',\
                                    env.get('PROGSUFFIX',''),source)))
        tgt = env.Program(target, source)
        env.Depends(tgt, '#dummy-target')
        p = env.Install(env['INSTALLDIR'],tgt)
        Alias('TopLevelAlias', p)


srcgen = env.Program('srcgen', 'srcgen.c')
srclist = env.Command('src-list.txt', srcgen,
                      './$SOURCE $SRCGEN_ARGS > $TARGET')
dummy = env.ScanSrcs('#dummy-target', srclist, SCANSRCS_FUNC=add_target)
env.AlwaysBuild(dummy)
Alias('TopLevelAlias', dummy)
}}}

Using a Source Generator to Add Targets Dynamically

Here's what we do to add targets dynamically, from a list which isn't known in advance.

The scenario in our case is you have a source generator 'srcgen' which is built by scons from sources in the usual way. It puts out a bunch of other sources whose names can't be predicted in advance. In our case it also puts out a src-list.text file. Call the generated sources srcA, srcB, etc.

Now you want to (say) compile each of those srcX into binX and include those in your final product.

In our SConscript we do this:

   1 srcgen = env.Program('srcgen', ...)
   2 srclist = env.Command('src-list.txt', srcgen, 
   3                       '$SOURCE $SRCGEN_ARGS > $TARGET')
   4 dummy = env.ScanSrcs('#dummy-target', srclist, SCANSRCS_FUNC=add_target)
   5 env.AlwaysBuild(dummy)
   6 Alias('TopLevelAlias', dummy)

Now when scons goes to build TopLevelAlias, the dependencies are set so it builds srcgen, runs it to produce the srclist (and the other sources), and then runs ScanSrcs on the srclist. The interesting thing is that the ScanSrcs function will be called during the build phase after its dependencies are up-to-date, rather than while the SConscript is being read. It will call Program() or SharedLibrary() or whatever builders we want, which will update the dependency graph while scons is building. Sounds scary, but it works.

We made our ScanSrcs parameterized so it could be reused. Note that it's practically all plain python, no SCons calls at all except adding the Builder and checking for construction variables.

   1     def scansrcs(target, source, env):
   2         """Scans through the list in the file 'source', calling
   3         env['SCANSRCS_FUNC'](env,line) on each line of that file.
   4         Calls env['SCANSRCS_PREFUNC'](env,source) once before scanning. 
   5         """
   6         if SCons.Util.is_List(source):
   7             source=source[0]
   8         if 'SCANSRCS_FUNC' not in env:
   9             raise Error, "You must define SCANSRCS_FUNC as a scons env function."
  10         # Call the pre-func
  11         if 'SCANSRCS_PREFUNC' in env:
  12             env['SCANSRCS_PREFUNC'](env, source.path)
  13 
  14         try:
  15             f=open(source.path, 'r')
  16         except:
  17             print "scansrcs: Can't open source list file '%s' in %s" % \
  18                   (source.path,os.getcwd())
  19             raise
  20         # Scan through the lines
  21         for line in f:
  22             src = line.strip()
  23             # print "Found " + src
  24             try:
  25                 env['SCANSRCS_FUNC'](env, src)
  26             except:
  27                 print "SCANSRCS func raised exception:"
  28                 raise
  29         f.close()
  30     # This is a funky builder, because it never creates its target.
  31     # Should always be called with a fake target name.
  32     env.Append(BUILDERS = {'ScanSrcs' : Builder(action=scansrcs)})

And finally, here's the function add_target, called from ScanSrcs, that does the real work:

   1     def add_target(env, source):
   2         """Add scons commands to build a new target from a scanned src generator list file."""
   3         # Build the new targets in the build dir:
   4         target = '#'+join(env['BUILDDIR'], basename(re.sub(r'\.c$', env.get('PROGSUFFIX',''), source)))
   5         tgt = env.Program(target, source)
   6         env.Depends(tgt, '#dummy-target')
   7         p = env.Install(env['INSTALLDIR'], tgt)
   8         Alias('TopLevelAlias', p)

Note that it adds new Nodes for the program target and the installed version of it, and also adds that to the TopLevelAlias. This way as the scanner runs scons knows that the new target doesn't exist or is out of date, so it marks the TopLevelAlias out of date; then scons comes around again to look at TopLevelAlias and discovers it now needs to build 'binA', which depends on 'srcA' and so it builds it and installs it as we want it to.

The above has been cut down from our real version, but I hope it shows how it can be done, and that it's not really that hard.


Here is a version of the code above that you can copy & paste, ready to edit. I added some imports and altered some small stuff that was too related to the code's origins. Now this example works as soon as you change the file names to match those in your project.

   1 import types
   2 import os.path
   3 import re
   4 
   5 env = Environment(BUILDDIR = "bin",INSTALLDIR = "installdir")
   6 
   7 def scansrcs(target, source, env):
   8         """Scans through the list in the file 'source', calling
   9         env['SCANSRCS_FUNC'](env,line) on each line of that file.
  10         Calls env['SCANSRCS_PREFUNC'](env,source) once before scanning.
  11         """
  12         if type(source) is list:
  13             source=source[0]
  14         if 'SCANSRCS_FUNC' not in env:
  15             raise Error, "You must define SCANSRCS_FUNC as a scons env function."
  16         # Call the pre-func
  17         if 'SCANSRCS_PREFUNC' in env:
  18             env['SCANSRCS_PREFUNC'](env, source.path)
  19 
  20         try:
  21             f=open(source.path, 'r')
  22         except:
  23             print "scansrcs: Can't open source list file '%s' in %s" % \
  24                   (source.path,os.getcwd())
  25             raise
  26         # Scan through the lines
  27         for line in f:
  28             src = line.strip()
  29             # print "Found " + src
  30             try:
  31                 env['SCANSRCS_FUNC'](env, src)
  32             except:
  33                 print "SCANSRCS func raised exception:"
  34                 raise
  35         f.close()
  36 
  37 
  38 # This is a funky builder, because it never creates its target.
  39 # Should always be called with a fake target name.
  40 env.Append(BUILDERS = {'ScanSrcs' : Builder(action=scansrcs)})
  41 
  42 
  43 def add_target(env, source):
  44         """Add scons commands to build a new target from a scanned src generator list file."""
  45         # Build the new targets in the build dir:
  46         target = '#' + os.path.join(env['BUILDDIR'], os.path.basename(re.sub(r'\.c$',\
  47                                     env.get('PROGSUFFIX',''),source)))
  48         tgt = env.Program(target, source)
  49         env.Depends(tgt, '#dummy-target')
  50         p = env.Install(env['INSTALLDIR'],tgt)
  51         Alias('TopLevelAlias', p)
  52 
  53 
  54 srcgen = env.Program('srcgen', 'srcgen.c')
  55 srclist = env.Command('src-list.txt', srcgen,
  56                       './$SOURCE $SRCGEN_ARGS > $TARGET')
  57 dummy = env.ScanSrcs('#dummy-target', srclist, SCANSRCS_FUNC=add_target)
  58 env.AlwaysBuild(dummy)
  59 Alias('TopLevelAlias', dummy)

DynamicSourceGenerator (last edited 2013-08-28 20:43:58 by WilliamDeegan)