Please note:The SCons wiki is now restored from the attack in March 2013. All old passwords have been invalidated. Please reset your password if you have an account. If you note missing pages, please report them to webmaster@scons.org. Also, new account creation is currently disabled due to an ongoing spam flood (2013/08/27).

Note: this is just a rough draft - I'll email the users list after I clean it up.... fix:

I've cobbled up a quick and dirty lil Source Code module that uses 'wget' to retrieve files from the net. Not finished at the moment, but it might be helpful to others. It's based off the ?BitKeeper tool. You can throw it in your Tools dir, and edit Tools to load it automatically or just do what I did and add:

      env.Tool('WGet', toolpath=['./path/to/your/modules/dir'])

To your SConstruct file. I've been testing it with a lil wrapper that munges download dir and path name together, but according to Gary O. on the mailing list, using a Value() node would eliminate the need for this. As for usage, In the SConscript files I just have:

  GetSourceFile(
    'http://url/to/download/file.tar.gz',
    'path/to/download/to',
    env)

GetSourceFile wrapper:

def GetSourceFile(url, output_dir, env):
  '''
  Create a node for a file retrieved via wget.
  This node can be used as a dependency in other rules.
  Hmm - how do we create the node ??
  '''
  # Nab everything after the last slash as the filename
  basefilename = str(url)[str( url).rfind(os.sep) + 1:]
  munged = output_dir + os.sep + basefilename + os.sep + url
  env.SourceCode( munged, env.WGet() )
  return None $

Here is the WGet.py file:

import os.path
import sys
import popen2
import SCons.Action
import SCons.Builder
import SCons.Util


def generate(env):
  """
  Add a Builder factory function and construction variables for
   WGET'ing to an Environment.
  """

  def WGetDownload(target, source, env):
    ret = []
    for x in target:
      url1 = str(x)
      if 'http:' in url1:
        div = url1.find('http:')
        outn = url1[ :div - 1]
        url = url1[div:]
        url = url.replace('http:/','http://')
      elif 'ftp:' in url1:
        div = url1.find('ftp:')
        outn = url1[ :div - 1]
        url = url1[div:]
        url = url.replace('ftp:/','ftp://')
      basename = str(url)[str( url).rfind(os.sep) + 1:]
      if not os.path.exists( outn):
        cmd = popen2.Popen4('wget -O %s %s' % (outn,url) )
        rc = cmd.wait()
        if rc != 0 :
          print "wget %s returned: %s" % (url, rc)
          print "%s" % cmd.fromchild.read()
          sys.exit()
    return None

  def WGetFactory(env=env):
    """ """
    act = SCons.Action.Action(WGetDownload)
    return SCons.Builder.Builder(action = act, env = env)

  env.WGet = WGetFactory
  env['WGET'] = 'wget'
  env['WGETFLAGS'] = SCons.Util.CLVar('')
  env['WGETCOM'] = '$WGET $WGETFLAGS $TARGET'

def exists(env):
  return env.Detect('wget')

WgetSourceCode (last edited 2010-03-13 11:19:17 by s235-148)