This page presents material that is no longer needed but is being kept for historical purposes. As of version 0.98, SCons has a Glob function built-in.


This page shows you how to glob (search files in dir) based on scons Nodes rather than files. This way you get all the files that will be built as well as the ones that already exist in the filesystem.

As of this writing, none of these functions recurse into subdirs, but that's not hard either. Just check each node returned by all_children() and if it's a Dir node, recurse. Here's a trivial one:

   1 def print_all_nodes(dirnode, level=0):
   2     """Print all the scons nodes that are children of this node, recursively."""
   3     if type(dirnode)==type(''):
   4         dirnode=Dir(dirnode)
   5     dt = type(Dir('.'))
   6     for f in dirnode.all_children():
   7         if type(f) == dt:
   8             print "%s%s: .............."%(level * ' ', str(f))
   9             print_dir(f, level+2)
  10         print "%s%s"%(level * ' ', str(f))


NOTE: this is the best version available. The others below are mostly of historical interest.

   1 import fnmatch
   2 import os
   3 def Glob( includes = Split( '*' ), excludes = None, dir = '.'):
   4    """Similar to glob.glob, except globs SCons nodes, and thus sees
   5    generated files and files from build directories.  Basically, it sees
   6    anything SCons knows about.  A key subtlety is that since this function
   7    operates on generated nodes as well as source nodes on the filesystem,
   8    it needs to be called after builders that generate files you want to
   9    include.
  10    It will return both Dir entries and File entries
  11    """
  12    def fn_filter(node):
  13       fn = os.path.basename(str(node))
  14       match = False
  15       for include in includes:
  16          if fnmatch.fnmatchcase( fn, include ):
  17             match = True
  18             break
  19       if match and excludes is not None:
  20          for exclude in excludes:
  21             if fnmatch.fnmatchcase( fn, exclude ):
  22                match = False
  23                break
  24       return match
  25    def filter_nodes(where):
  26        children = filter(fn_filter, where.all_children(scan=0))
  27        nodes = []
  28        for f in children:
  29            nodes.append(gen_node(f))
  30        return nodes
  31    def gen_node(n):
  32        """Checks first to see if the node is a file or a dir, then
  33        creates the appropriate node. [code seems redundant, if the node
  34        is a node, then shouldn't it just be left as is?
  35        """
  36        if type(n) in (type(''), type(u'')):
  37            path = n
  38        else:
  39            path = n.abspath
  40        if os.path.isdir(path):
  41            return Dir(n)
  42        else:
  43            return File(n)
  44    here = Dir(dir)
  45    nodes = filter_nodes(here)
  46    node_srcs = [n.srcnode() for n in nodes]
  47    src = here.srcnode()
  48    if src is not here:
  49        for s in filter_nodes(src):
  50            if s not in node_srcs:
  51                # Probably need to check if this node is a directory
  52                nodes.append(gen_node(os.path.join(dir,os.path.basename(str(s)))))
  53    return nodes

You can either include this function directly in your SConscript, or place in an external python script to include in any SConscript it's needed. SCons CVS has a better way to do it, but 0.96.1 requires the use of SConscript( 'external_script.py' ) and Import( ) / Export( )

I don't take much credit for this. Most credit belongs to John Meinel from the SCons mailinglist.

If you want to glob for generated files or for source files in a BuildDir, copy the following function into your project and call Glob instead of glob.glob.

   1 def Glob(match):
   2     """Similar to glob.glob, except globs SCons nodes, and thus sees
   3     generated files and files from build directories.  Basically, it sees
   4     anything SCons knows about.  A key subtlety is that since this function
   5     operates on generated nodes as well as source nodes on the filesystem,
   6     it needs to be called after builders that generate files you want to
   7     include."""
   8     def fn_filter(node):
   9         fn = str(node)
  10         return fnmatch.fnmatch(os.path.basename(fn), match)
  11     here = Dir('.')
  12     children = here.all_children()
  13     nodes = map(File, filter(fn_filter, children))
  14     node_srcs = [n.srcnode() for n in nodes]
  15     src = here.srcnode()
  16     if src is not here:
  17         src_children = map(File, filter(fn_filter, src.all_children()))
  18         for s in src_children:
  19             if s not in node_srcs:
  20                 nodes.append(File(os.path.basename(str(s))))
  21     return nodes

Eventually this should go into SCons proper with test cases and documentation.

Note: Glob currently does not support usages such as Glob('src/*.cpp'). Eventually it should split the match string into directories and use the fnmatch module to glob on subdirectories as well.

(Notes by chenlee@ustc.edu ) Below is the grab function used in my building script:

   1 def Glob( pattern = '*.*', dir = '.' ):
   2     import os, fnmatch
   3     files = []
   4     for file in os.listdir( Dir(dir).srcnode().abspath ):
   5         if fnmatch.fnmatch(file, pattern) :
   6             files.append( os.path.join( dir, file ) )
   7     return files

Here's another version that has includes and excludes:

   1   def Matches(file, includes, excludes):
   2       match = False
   3       for pattern in includes:
   4          if fnmatch.fnmatchcase(file, pattern):
   5              match = True
   6              break
   7       if match and excludes is not None:
   8           for pattern in excludes:
   9               if fnmatch.fnmatchcase(file, pattern):
  10                   match = False
  11                   break
  12       return match
  13   def AddDir(src_dir, includes, excludes=None):
  14       files = []
  15       for file in os.listdir(Dir(src_dir).srcnode().abspath):
  16           fqpath = src_dir + '/' + file
  17           if os.path.isdir(fqpath):  continue   #don't include subdirs
  18           if Matches(file, includes, excludes):
  19              files.append(file)
  20       return files
  21 used like so:
  22    AddDir("c:/mystuff", "*.cpp", excludes=['test*.cpp', '.svn'])

And yet another version: SGlob('*.cpp'), SGlob('Test/*.cpp')

   1 def SGlob(pattern):
   2         path = GetBuildPath('SConscript').replace('SConscript', '')
   3         result = []
   4         for i in glob.glob(path + pattern):
   5                 result.append(i.replace(path, ''))
   6         return result

Note: This Version is kind a dirty, but it searches the sourcedirectory, which makes it usable until there is an offical 'the-SCons-way' Glob.



BuildDirGlob (last edited 2010-01-20 09:46:03 by s235-200)