Please note:The SCons wiki is in read-only mode due to ongoing spam/DoS issues. Also, new account creation is currently disabled. We are looking into alternative wiki hosts.
Differences between revisions 7 and 8
Revision 7 as of 2013-08-27 12:03:30
Size: 403
Editor: HoustonQ7
Comment:
Revision 8 as of 2013-08-27 13:07:29
Size: 5633
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
Hello and welcome. My name is Marna Calzada. For years I've been living in Florida plus I have everything that I want here. What me plus my family love is martial arts and I've been doing it for very a while. Taking care of animals is where my main income comes from. I'm not wise at webdesign however, you could wish To check my website: http://www.help123computer.com/?q=blog/dating-european-girls-2 == Goal ==

The goal is to get SCons to track dependencies and the need to build RPMs all in one SConstruct instead of relying on external SConscripts. This is due to the fact that the build of the repository is a simple matter of globbing a set of *.spec files. The creation process is as follows:
{{{
    .spec ==> .src.rpm ==> .rpm
}}}

Since SCons doesn't support this out-of-the-box, we need to create some custom Builders and setup to get this working.

To see the full unedited source, you can view and comment directly within here: /FullScript


== .spec ==> .src.rpm ==

To create a source rpm out of a spec file we need to know a few things:
  1. The sources that go to create a source rpm.
  2. The target that is produced.

=== SourceRPM Sources ===

A source rpm is really just a compressed CPIO package with some extra header. The sources that comprise of a source RPM include the .spec file, files mentioned in "Source:" tags, and files mentioned in "Patch:" tags. These tags are found in the specfile, so we build a scanner to get this information. In my situation, some of the files are tarballs and are not checked into SVN. Therefore, I use an env.Repository() to point to a cache_dir where I download the appropriate tarballs into.

''I do this because BuildDir() does not really duplicate anything generated from this Scanner. I have no clue why.''

The code looks like this:

{{{#!python
def specfile_scan(node, env, path, *args):
    sources = env.current.sources
    srccache = os.path.join(env.cache_dir, os.path.dirname(str(node)))
    # mkdir -p
    try:
        os.mkdir(srccache)
    except OSError:
        pass
    for k,v in sources.items():
        if v.startswith('http') or v.startswith('ftp'):
            urlgrab(v, os.path.join(srccache, k))
        else:
            shutil.copy(os.path.join(os.path.dirname(str(node)), k),srccache)
    return sources.keys()

specscan = Scanner(function = specfile_scan,
                   skeys = ['.spec'])

...

env.cache_dir = '/data/pyvault-build/sources'
env.Repository(env.cache_dir)
}}}

env.current is a small Class that contains information parsed out of the specfile by an external utility. This is done outside of any Builder() step. ''But this part of my SConstruct is probably messed up.''


=== SourceRPM Target ===

The target is pretty easy to identify. Building of a source rpm is always of the format: "name-version-release.src.rpm", which is readily available in the env.current Class that was created before the Builder is executed:

''I had to prepend the build_dir (which I define explicitly) to get things to work. Not sure what's wrong here.''

{{{#!python
def srpm_targets(target, source, env):
    targets = env.current.srcrpm
    targets = [os.path.join(env['build_dir'], t) for t in targets]
    return targets, source
}}}

=== SourceRPM Builder ===

We wrap this all up by creating a custom builder based on the Target. The Scanner part is AutoMagically called when SCons encounters a .spec file.

{{{#!python
macros = """--define='_sourcedir %_topdir/sources/%name' \
            --define='_srcrpmdir %_topdir/srpms' """
srpmbld = Builder(action = "rpmbuild %s --nodeps -bs $SOURCE" % macros,
                  suffix = '.src.rpm',
                  src_suffix = '.spec',
                  emitter = srpm_targets,
                  single_source = True,
          )
}}}

== .src.rpm ==> .rpm ==

Setting this up is pretty cake-like since there's only one source. But, there are multiple targets. The difficulty is taken care of prior to invoking the Builder() stuff by using an external utility as discussed above.

=== RPM Source ===

Always whatever is supplied to the custom builder. Should end in .src.rpm.

=== RPM Target ===

Computed by using an external utility, then wrapped:

{{{#!python
def rpm_targets(target, source, env):
    targets = [os.path.join(env['build_dir'], t) for t in env.current.packages]
    return env.current.packages, source
}}}

''Again: needed to prepend build_dir. Dunno why...''

=== RPM Builder ===

{{{#!python
# this could be "rpmbuild --rebuild", or mach. This is what I am working with now.
rpmbld = Builder(action = "mock --no-clean --resultdir=$build_dir -r $chroot $SOURCE",
                 suffix = '.rpm',
                 src_suffix = '.src.rpm',
                 emitter = rpm_targets,
                 single_source = True,
                 src_builder = srpmbld,
}}}

I put in `src_builder` because I wanted to see if making this a MultiStageBuilder would make any difference. ''XXX: So far it has not made a difference.''

== Failed Experiment ==

So far, with my script, I've only had SCons exhibit the following behavior, either:
  1. Always executing the .spec => .src.rpm => .rpm build chain, or
  2. Never executing it.

There's one thing I tried to see if it would help. I created a `MakeSpec` builder that bridged the multistage into a .spec ==> .rpm step:

{{{#!python

def buildit(target, source, env):
    env.MakeSRPM(source[0])
    env.MakeRPM(os.path.join(env['build_dir'],env.current.srcrpm[0]))

specbld = Builder(action = buildit,
                 suffix = '.rpm',
                 src_suffix = '.spec',
                 emitter = rpm_targets,
                 single_source = True,
                )
}}}

But, it complained loudly since I just reused the `rpm_targets` found in the `MakeRPM` builder. So, I need to bail on this and use the original global function called buildit() that wraps multiple builders...

Goal

The goal is to get SCons to track dependencies and the need to build RPMs all in one SConstruct instead of relying on external SConscripts. This is due to the fact that the build of the repository is a simple matter of globbing a set of *.spec files. The creation process is as follows:

    .spec ==> .src.rpm ==> .rpm

Since SCons doesn't support this out-of-the-box, we need to create some custom Builders and setup to get this working.

To see the full unedited source, you can view and comment directly within here: /FullScript

.spec ==> .src.rpm

To create a source rpm out of a spec file we need to know a few things:

  1. The sources that go to create a source rpm.
  2. The target that is produced.

SourceRPM Sources

A source rpm is really just a compressed CPIO package with some extra header. The sources that comprise of a source RPM include the .spec file, files mentioned in "Source:" tags, and files mentioned in "Patch:" tags. These tags are found in the specfile, so we build a scanner to get this information. In my situation, some of the files are tarballs and are not checked into SVN. Therefore, I use an env.Repository() to point to a cache_dir where I download the appropriate tarballs into.

I do this because BuildDir() does not really duplicate anything generated from this Scanner. I have no clue why.

The code looks like this:

   1 def specfile_scan(node, env, path, *args):
   2     sources = env.current.sources
   3     srccache = os.path.join(env.cache_dir, os.path.dirname(str(node)))
   4     # mkdir -p
   5     try:
   6         os.mkdir(srccache)
   7     except OSError:
   8         pass
   9     for k,v in sources.items():
  10         if v.startswith('http') or v.startswith('ftp'):
  11             urlgrab(v, os.path.join(srccache, k))
  12         else:
  13             shutil.copy(os.path.join(os.path.dirname(str(node)), k),srccache)
  14     return sources.keys()
  15 
  16 specscan = Scanner(function = specfile_scan,
  17                    skeys = ['.spec'])
  18 
  19 ...
  20 
  21 env.cache_dir = '/data/pyvault-build/sources'
  22 env.Repository(env.cache_dir)

env.current is a small Class that contains information parsed out of the specfile by an external utility. This is done outside of any Builder() step. But this part of my SConstruct is probably messed up.

SourceRPM Target

The target is pretty easy to identify. Building of a source rpm is always of the format: "name-version-release.src.rpm", which is readily available in the env.current Class that was created before the Builder is executed:

I had to prepend the build_dir (which I define explicitly) to get things to work. Not sure what's wrong here.

   1 def srpm_targets(target, source, env):
   2     targets = env.current.srcrpm
   3     targets = [os.path.join(env['build_dir'], t) for t in targets]
   4     return targets, source

SourceRPM Builder

We wrap this all up by creating a custom builder based on the Target. The Scanner part is AutoMagically called when SCons encounters a .spec file.

   1 macros = """--define='_sourcedir %_topdir/sources/%name' \
   2             --define='_srcrpmdir %_topdir/srpms' """
   3 srpmbld = Builder(action = "rpmbuild %s --nodeps -bs $SOURCE" % macros,
   4                   suffix = '.src.rpm',
   5                   src_suffix = '.spec',
   6                   emitter = srpm_targets,
   7                   single_source = True,
   8           )

.src.rpm ==> .rpm

Setting this up is pretty cake-like since there's only one source. But, there are multiple targets. The difficulty is taken care of prior to invoking the Builder() stuff by using an external utility as discussed above.

RPM Source

Always whatever is supplied to the custom builder. Should end in .src.rpm.

RPM Target

Computed by using an external utility, then wrapped:

   1 def rpm_targets(target, source, env):
   2     targets = [os.path.join(env['build_dir'], t) for t in env.current.packages]
   3     return env.current.packages, source

Again: needed to prepend build_dir. Dunno why...

RPM Builder

   1 # this could be "rpmbuild --rebuild", or mach. This is what I am working with now.
   2 rpmbld = Builder(action = "mock --no-clean --resultdir=$build_dir -r $chroot $SOURCE",
   3                  suffix = '.rpm',
   4                  src_suffix = '.src.rpm',
   5                  emitter = rpm_targets,
   6                  single_source = True,
   7                  src_builder = srpmbld,

I put in src_builder because I wanted to see if making this a MultiStageBuilder would make any difference. XXX: So far it has not made a difference.

Failed Experiment

So far, with my script, I've only had SCons exhibit the following behavior, either:

  1. Always executing the .spec => .src.rpm => .rpm build chain, or

  2. Never executing it.

There's one thing I tried to see if it would help. I created a MakeSpec builder that bridged the multistage into a .spec ==> .rpm step:

   1 def buildit(target, source, env):
   2     env.MakeSRPM(source[0])
   3     env.MakeRPM(os.path.join(env['build_dir'],env.current.srcrpm[0]))
   4 
   5 specbld = Builder(action = buildit,
   6                  suffix = '.rpm',
   7                  src_suffix = '.spec',
   8                  emitter = rpm_targets,
   9                  single_source = True,
  10                 )

But, it complained loudly since I just reused the rpm_targets found in the MakeRPM builder. So, I need to bail on this and use the original global function called buildit() that wraps multiple builders...

RpmHoncho (last edited 2013-08-27 13:07:29 by GaryOberbrunner)