Please note:The SCons wiki is in read-only mode due to ongoing spam/DoS issues. Also, new account creation is currently disabled. We are looking into alternative wiki hosts.

Introduction

This wiki page provides step-by-step instructions for preparing and releasing SCons. (NEEDS UPDATING TO WORK WITH MERCURIAL/hg)

Simplified 2012+ Mercurial-based Release Procedure (EXPERIMENTAL)

Simplified Release Procedure for 2012 and later

The procedure has been used for all releases since 2.2.0.

Prepare Binaries and Doc

  • validate and update all documentation files by calling:
    python bin/docs-update-generated.py
    python bin/docs-validate.py
    python bin/docs-create-example-outputs.py

    , then check all diffs for the files in docs/generated/*.

  • Commit the auto-generated doc changes to current branch ("Regenerated docs for X.Y.Z release.")
  • update CHANGES.txt (should already be up to date)
  • update Announce.txt (not for checkpoints): add section for this release, important user-visible changes only. This is really long since it also has old releases. Is it useful?
  • update Release.txt: this gets its content *replaced* for each release. New functionality, deprecated functionality, changed functionality, and fixes. Get this from CHANGES.txt. Add new contributors to list at end.
  • edit debian/changelog. Be careful of formatting here, it gets machine-parsed.

  • NOTE: I think Announce and Release are backwards; Release should be complete release notes for all versions (latest at top) and Announce should be a short blurb of just this release. FIXME!
  • Commit this to the current branch (normally default branch in hg, unless this release is coming off a branch).
  • update ReleaseConfig and run python bin/update-release-info.py release (this modifies CHANGES, Release and Announce -- that's why you should commit the above first.)

  • build packages and doc: python bootstrap.py >& build-XYZ.log (good idea to save build logfile somewhere)

  • test them: python runtest.py -a (Q: aren't there special tests to test the unpacked installers?)

You should now have the following in build/dist:

  scons-$VERSION-1.noarch.rpm
  scons-$VERSION-1.src.rpm
  scons-$VERSION.linux-x86_64.tar.gz
  scons-$VERSION.linux-x86_64.zip
  scons-$VERSION.tar.gz
  scons-$VERSION.win32.exe
  scons-$VERSION.zip
  scons-doc-$VERSION.tar.gz
  scons-local-$VERSION.tar.gz
  scons-local-$VERSION.zip
  scons-src-$VERSION.tar.gz
  scons-src-$VERSION.zip

The .linux-x86_64 ones are not needed and may be deleted; the others all get uploaded to SF.

  • You have to rename the scons-$VERSION.win32.exe to scons-$VERSION-setup.exe; the build SConstruct should be fixed to do this. (Note that the upload script requires this.)

Tag Release in Mercurial

  • commit the changes made by update-release-info.py onto a release branch:

   hg branch rel_<NAME>
   hg commit (message: final auto updates for x.y.z release)
   hg tag <NAME> (e.g. 2.2.0)

Upload Software and Doc

  • There is now a shell script to do this: bin/upload-release-files.sh X.Y.Z mysfusername as long as SourceForge and scons.org have your ssh pub key and you're using SSH Agent Forwarding.

  • It uploads all the packages to SF, uploads the doc to scons.org, unpacks it, and updates the doc symlinks.
  • You may still have to tell SF that the new release dirs exist in its File Manager (it's a bit buggy).

Prepare Announcement and announce to all

  • Use Announce.txt and/or Release.txt as blurb
  • Update scons.org. Much of the hard work is already done by the script. You just have to manually edit these files in public_html/production:

includes/versions.php

update $latestrelease, update $docversions[] and $apiversions[] list

index.php

add an announcement for the home page
remove any out-of-date announcements

news-raw.xhtml

add an announcement to the list (duplicate it from what you just added to index.php)

  • Commit the above changes to hg and push.
  • Update Sourceforge:
    • set default downloads for each win/linux/mac etc. appropriately, using the "info" link on the right of each download.
  • Update Tigris:
    • check out tigris website: svn checkout http://scons.tigris.org/svn/scons/ scons-tigris --username USERNAME

    • Then edit the trunk/www/project-highlights and trunk/www/roadmap.html pages and svn commit. That will make them live.
    • Manually add a new Announcement: log in to the site, then upper left Announcements, then in there Add New Announcement.
    • Add version to issue tracker: Issue Tracker, Configuration, Add/Edit, Add new version.
  • Announce to scons-users and scons-dev
  • Others?

After Release

  • On default branch, copy all the changes from the release branch back to the default branch. (XXX: then why do we have the release branch??)
    • hg diff -r default:X.Y.Z | patch -p1

  • On default branch, run python bin/update-release-info.py post to go back to develop mode.

  • Commit those changes after review

There is more detail on some of the steps here at http://www.scons.org/wiki/ReleaseHOWTO/TipBetaBody although that is still based on the old svn system.

It's OK to do the release-branch creation, commit and tag at the very end, just in case something goes wrong and packages need to be rebuilt.

TODO:

  • do we need both Announce.txt and RELEASE.txt? Let's optimize for what we really need.
  • research above FIXMEs
  • integrate useful parts of http://www.scons.org/wiki/ReleaseHOWTO/TipBetaBody into this page, while keeping it short and to the point.

  • make list of dependencies needed to produce release (e.g. for ubuntu: all doc tools, man2html, rpm)

Tip Releases

Alpha Release

You can also look at a page with just this flow.

An alpha release is published from the trunk. It can be done at any point, even from a read-only tree, as there's no requirement for committing trunk when the process is completed.

An alpha release has a very short lifetime; in general, it exists long enough to do some testing and then it is thrown away. These steps are suggestive, not prescriptive; individual cases may choose among the steps (and even add new ones) to perform their tasks.

For example, the standard BuildBot run skips the steps after the regression tests, while the packaging run skips the regression tests and instead does the tests of the candidate packages. In both cases, the alpha release is immediately discarded.

Another example might be automatically publishing the packages every day; to keep the run-time reasonable, the testing steps would be skipped.

BLAH BLAH BLAH

Assumed Setup

These initializations are the conventions that will be used in the rest of this flow.

The location of the SCons SVN archive:

  $ export HG=https://bitbucket.org/scons/scons

The version string, which should look something like this:

BLAH BLAH BLAH

  $ VERSION=3.2.0-alpha.yyyymmdd

Check Out Trunk

From within your base directory, execute this command:

  $ svn co $SVN/trunk

Update Release Values

From within the trunk directory:

$ python bin/update-release-info.py release

The ReleaseConfig file is where the "official" version number ($VERSION), the Python version floors, and other information about the release is recorded. This command takes the information in ReleaseConfig and inserts it in the necessary files.

Run Regression Tests (Optional)

Since an alpha package is intended to be transient, only useful for a short time, running the regression tests may not be necessary.

To run the tests, simply execute this command:

  $ python runtest.py -a

Build Candidate Packages

Provided you have all of the necessary utilities installed, this should be a simple matter of:

  $ rm -rf build
  $ python bootstrap.py

Test Candidate Packages (Optional)

Since an alpha package is intended to be transient, only useful for a short time, testing the packages may not be necessary.

The build step above not only builds the packages but also unpacks all of them into subdirectories so that you can run the test suite against the packaged goods. This is intended to catch packaging problems such as not adding a new module to the packaging MANIFEST list(s). The runtest.py script supports a -p option and arguments that run the SCons tests against the different unpacked directories.

Edit the script to include the tests you want to include. If you want to be complete, test all of the packages.

To be quicker but still reasonably thorough, test tar-gz and zip, one each of local- and src- (probably do -tar-gz for one and -zip for the other), and rpm.

For a quick-n-dirty test, just test tar-gz or zip, and maybe rpm. Since all of the different packages use the same lists as input, it's pretty unlikely that the tests will pass on one package and fail for another.

Make Available

TO BE ADDED LATER; if you have some good words for this topic, please FIXME and put them in.

Beta and Candidate Releases

You can also look at a page with just this flow.

These instructions use outdated workflow based on svnmerge tool.

This flow takes the available changesets in trunk and brings checkpoint up to date. It assumes that trunk and checkpoint already exist and have been set up correctly.

The trunk is the authoritative source for everything that goes in a release; only under exceptional circumstances are changes made to other branches. The general rule is that a change is made to trunk and brought over to checkpoint.

This flow assumes that any needed changesets are already in trunk; if not, you will need to commit a suitable changeset following your usual development practices.

If you want a checklist you can mark off as you go, cut-n-paste these contents.

Assumed Setup

These initializations are the conventions that will be used in the rest of this flow.

The location of the SCons SVN archive:

  $ export HG=https://bitbucket.org/scons/scons

The version string, which should look something like this:

  $ export RELEASE=3.2.0.beta.yyyymmdd

Send Starting Message

FIXME Send blurb out to release mailing list

Check Out Trunk

From within your base directory, execute this command:

  $ svn co $SVN/trunk

Check Out Checkpoint

From within your base directory, execute this command:

  $ svn co $SVN/checkpoint

Determine Available Changesets

FIXME Use svnmerge avail --log instead of dry run. Use svnmerge to find out what changesets are available to be merged:

  $ cd checkpoint
  $ svnmerge avail --log >../trunk/log.file
  $ cd ../trunk

Edit Configuration Files

The commit.txt file created in the previous step has the commit comments for all the applied changesets. In theory, all of them have been included in the trunk files src/CHANGES.txt, stc/RELEASE.txt, and src/Announce.txt, but in practice, some of the information has not been included, so you will have to add it. At the same time, you want to prepare a log.file that will be used as the commit comments.

The log.file and src/CHANGES.txt files

Both log.file and src/CHANGES.txt use similar formats. Assuming commit.txt has three log entries, three from John Doe on two issues and two by Jim Smith on a single issue, log.file would look like this:

Rebased to trunk revision 54321:

From John Doe:
 - One item that was done by John Doe.
 - Another item that was done by John Doe.

From Jim Smith:
 - Item done by Jim Smith.

In theory, the first section of src/CHANGES.TXT should already have the corresponding information. If it does, steal it for log.file. If it doesn't, add it to src/CHANGES.txt. In the end, the items in src/CHANGES.txt for John Doe and Jim Smith should have this information:

  From John Doe:

    - One item that was done by John Doe with details and examples.

    - Another item that was done by John Doe with details and examples.

  From Jim Smith:

    - Item done by Jim Smith with details and examples.

Verify the src/RELEASE.txt file

In theory, the entries in src/RELEASE.txt should already summarize src/CHANGES.txt except that issues are summarized by topic and the names of contributors are collected in a section at the bottom. In practice, not all of the entries have been transcribed.

Since this is an announcement, try to edit down the src/CHANGES.txt descriptions so that, ideally, each item fits on a single line. In general, get rid of examples and explanations. The idea is to pique the reader's interest, not provide full documentation. Still, use your judgment and feel free to leave in an explanation if it would be too confusing to shorten it.

Try to reorganize the items so that, within each section, all of the Visual Studio items are listed together, all of the Java items are together, etc. More "important" (or visible) items should appear towards the top of each section.

It's all right just to delete the description of something that's too difficult to explain and too minor to make much of a difference.

Whether or not a given item belongs in "NEW FUNCTIONALITY" vs. "CHANGED/ENHANCED" vs. "FIXED" vs. "IMPROVEMENT" is a judgment call.

In the final "Thanks to" paragraph, at a minimum list everyone in the src/CHANGES.txt file who contributed to the release, plus anyone else who (in your judgment) merits a mention even if they didn't contribute an actual patch.

The src/Announce.txt file

The src/Announce.txt file is the announcement used for production releases. In theory, it summarizes src/CHANGES.txt, but like src/CHANGES.txt, entries may not have been transcribed. Rather than trying to do the summarization all at once in a candidate release, it's better to do it incrementally while the changes to src/CHANGES.txt and src/RELEASE.txt are fresh in your mind.

Under the topic "scheduled for next release" add anything that's expected to be a significant change. To get a feel for what's in play, consult the tracker for issues scheduled for the next next release. The quickest way of getting that list is to go to the BugParty wiki page and click on the "how we're doing" link. On that report, click on the summary link at the bottom of the column for the next release. Use the future tense for these items.

Under the next topic, add any major items that aren't already included. Use your judgment as to what to add, but if it's not something the user needs to know, don't include it, so things like performance improvements, code cleanup, typos, and minor enhancements don't make the cut.

FIXME expand this section

Confirm Users' Guide is OK

CAUTION: Only try this step if you have the toolchain for building the documentation. The packages needed are discussed in DeveloperGuide/Documentation. It's quite a list.

The following steps verify that the user guide sources (some of which are generated from other files) are up to date.

FIXME (This will probably be reworked to be part of the build step.)

  $ python bin/scons-doc.py --diff
  $ python bin/scons-doc.py --update

Commit Trunk

This step is needed only if any trunk files were modified. If nothing was changed, skip to the next step step.

From within the trunk directory:

Verify that version_tuple in ReleaseConfig has the correct release number and a release level of 'alpha'.

  $ svn commit -F log.file
  Sending        some/file
  Sending        some/other/file
  Transmitting file data ..
  Committed revision 54321.
  $

Merge Trunk into Checkpoint

To merge the changes from the trunk into the checkpoint branch, run this command from within the checkpoint directory:

  $ svnmerge merge -b -S $SVN/trunk -f commit.txt

Resolve Conflicts

FIXME This segment is overkill for anyone who actually uses SVN. Cut it or drop it.

A conflict occurs when SVN tries to apply changes from the archive that overlap with other changes made locally. These are rare but can happen when cherry-picking applies changes out of order. In general, the correct approach is to use the text from the archive rather than the local version, but use your judgment when making the changes as there will be some cases where the original text (or even a merger of the two texts) is the correct choice as SVN is not always accurate when marking the conflicted regions.

Finding conflicts

To determine if there're conflicts, run this command:

  $ svn status
  ?      test/example.py.r4941
  ?      test/example.py.mine
  ?      test/example.py.r4649
  C      test/example.py
  ?      src/engine/SCons/CoreFile.py.r4941
  ?      src/engine/SCons/CoreFile.py.mine
  ?      src/engine/SCons/CoreFile.py.r4649
  C      src/engine/SCons/CoreFile.py
  $

Lines that begin with "C" identify a file that's conflicted. There are also a number of related files provided by SVN that can be used in the process of resolution, identified by lines that begin with "?". This is a simple resolution, so we won't use them.

Removing Conflicts

To remove the conflicts from a file, follow these steps:

  • Locate a conflict marker in the text. The center of each conflicted region is marked with "=======" so searching for a line with only that on it will get you to the right place.

  • The start of the region is marked with "<<<<<<< .mine" and contains the local text. Delete this text as well as the start and center markers.

  • The end of the region is marked with something that looks like ">>>>>>> .r54321" and contains the text from the SVN archive. Delete the end marker.

  • Repeat until there are no more conflict markers.

To see how your updated file compares with the archive, run this command:

  $ svn diff test/example.py

If the changes aren't satisfactory, repeat the editing step.

Once the editing is done, use this command to tell SVN that the conflict has been resolved:

  $ svn resolved test/example.py

As a side-effect, resolving the conflict will remove the related files provided by SVN.

Repeat this section for each conflicted file.

Update Release Files

Verify that the version_tuple in ReleaseConfig says 'beta' or 'candidate' as desired.

From within the checkpoint directory:

$ python bin/update-release-info.py release

The ReleaseConfig file is where the "official" version number ($VERSION), the Python version floors, and other information about the release is recorded. This command takes the information in ReleaseConfig and inserts it in the necessary files.

Commit Checkpoint

FIXME

  $ svn commit -m"Update checkpoint branch for $VERSION; see CHANGES.txt for details."

Build Candidate Packages

Provided you have all of the necessary utilities installed, this should be a simple matter of:

  $ rm -rf build bootstrap
  $ python bootstrap.py

While you're waiting for the candidate packages to be built, open another shell window and proceed with preparing the test script. Make sure that $SVN and $VERSION are set up as shell variables as described above.

Test

Prepare test script

The script to run the tests is in the source in bin/fill_me_in. Make a copy and edit it according to the instructions below.

  # report skipped tests as successful
  export TESTCOMMON_PASS_SKIPS=yes
  set -e       # stop on error
  #
  python runtest.py -a
  python runtest.py -a -p tar-gz
  python runtest.py -a -p zip
  python runtest.py -a -p local-tar-gz
  python runtest.py -a -p local-zip
  python runtest.py -a -p src-tar-gz
  python runtest.py -a -p src-zip
  python runtest.py -a -p rpm

The regression suite exercises a wide range of functionality. There are around a thousand tests in the suite, so each test run can take a while.

Regression tests of source tree

Normally, the SCons BuildBot monitors this branch, so it probably started running regression tests as soon as you checked in the changes above. If your machine is covered by one of the BuildBot machines (particularly if your machine is one of the BuildBot machines), you can remove the first runtest in the script.

Regression tests of candidate packages

The build step above not only builds the packages but also unpacks all of them into subdirectories so that you can run the test suite against the packaged goods. This is intended to catch packaging problems such as not adding a new module to the packaging MANIFEST list(s). The runtest.py script supports a -p option and arguments that run the SCons tests against the different unpacked directories.

Edit the script to include the tests you want to include. If you want to be complete, test all of the packages.

To be quicker but still reasonably thorough, test tar-gz and zip, one each of local- and src- (probably do -tar-gz for one and -zip for the other), and rpm.

For a quick-n-dirty test, just test tar-gz or zip, and maybe rpm. Since all of the different packages use the same lists as input, it's pretty unlikely that the tests will pass on one package and fail for another.

Once the script is prepared, save it and return to the window creating the packages.

Check the Build

The build creates the packages in the build/dist subdirectory. If everything built correctly, you should see files that look like the following:

  scons-$VERSION-1.noarch.rpm
  scons-$VERSION-1.src.rpm
  scons-$VERSION.linux-x86_64.tar.gz
  scons-$VERSION.linux-x86_64.zip
  scons-$VERSION.tar.gz
  scons-$VERSION.win32.exe
  scons-$VERSION.zip
  scons-doc-$VERSION.tar.gz
  scons-local-$VERSION.tar.gz
  scons-local-$VERSION.zip
  scons-src-$VERSION.tar.gz
  scons-src-$VERSION.zip

Note that the linux-x86_64 strings in some of the file names above may be different, depending on your local system architecture. That doesn't matter; those files don't go public. Go ahead and remove them.

Run Test Script

FIXME

If the build succeeded, run the test script:

  $ sh path/to/copy-of-script

The tests can run a very long time. While you're waiting, go back to your other shell window and prepare what you can on the assumption that the tests will succeed.

Download SCons Home

From within your base directory, run these commands:

  $ svn co $SVN/scons.org
  $ cd scons.org

The scons.org checkout is pretty large. If you don't want to wait, open another shell window and proceed. Make sure the new shell is set up with the SVN and VERSION shell variables as described above.

Prepare Blurb

Prepare the blurb by starting with a copy of build/scons/RELEASE.txt. It should be pretty much good-to-go, but look over it and see if there are any changes that should be made to make it more suitable as a blurb rather than release notes in a distribution. In general, use your own judgment as to what to keep and what to cut.

FIXME HTML copy of blurb by inserting <br/><br/><br/> instead of double blank lines, <br/><br/> instead of single blank lines, and <br/> in front of items in a list. Not perfect, but more readable.

FIXME Source of beta and final blurbs are different.

Prepare Tigris Announcement

Note that these files must be checked in to the trunk for them to show up on our tigris.org project pages; you can not update and commit these files from the directory in which you built the packages.

From within the www subdirectory of your trunk directory, edit these files:

project_highlights.html

Add a short highlight announcement that will appear at the top of all of the scons.tigris.org pages. Trim the list to remove any entries that are now obsolete.

roadmap.html

Update the text at the top to reflect that this release is now the latest available release.

Prepare scons.org

If you opened another window to do the prep above, move back to the window downloading scons.org; it should be done by now. If you didn't, change back to the scons.org subdirectory of your base directory.

Web site

Make the following changes to the following files: FIXME The destination names aren't CHANGES.txt or RELEASE.txt and the explanation needs to be tweaked. And the source for RELEASE.txt could be either RELEASE.txt or Announce.txt.

CHANGES.txt

copy the new file from build/scons/CHANGES.txt in your packaging directory

RELEASE.txt

copy the new file from build/scons/RELEASE.txt in your packaging directory

includes/versions.php

update $latestrelease to the value $VERSION FIXME
add the new version number to the $docversions[] list
add the new version number to the $apiversions[] list

index.php

add an announcement for the home page
remove any out-of-date announcements

news-raw.xhtml

add an announcement to the list (duplicate it from what you just added to index.php)

Install documentation

To unpack the release documentation into where it can be used by the web pages, just run this script:

FIXME When this script is checked in, it will allow arguments on the command line
  $ sh bin/FillMeIn [ $VERSION [ $SVN ] ]
or something similar. In the meantime, copy the script and run it.

  mkdir doc/$VERSION
  (cd doc/$VERSION && tar zxf build/dist/scons-doc-$RELEASE.tar.gz)
  svn add doc/$VERSION
  (cd doc && rm -f latest && ln -s $VERSION latest)
  case $VERSION in *.0.final.*)
      # This is a 'final' release on the main branch
      (cd doc && rm -f production && ln -s $VERSION production)
  esac

FIXME If any of the candidate packages fail, fix the problem in trunk following your usual development practices and go and do another dry run.

Tag the Release

From within the checkpoint directory:

  $ svn cp . $SVN/tags/$VERSION

Archive Candidate Packages

Update SourceForge

Upload packages

Verify that you have SVN and VERSION in your shell environment as described in the setup section above, then run the bin/scp-sourceforge script:

  $ sh bin/scp-sourceforge sf_username 

Mark packages with release notes

Navigate to the SourceForge File Manager page for SCons. If you're reading this from a printed copy rather than the wiki page and can't click on the link, follow these instructions:

  • Log in to your SourceForge account

  • Click on Develop just under Forge in the SourceForge banner.

  • Under My Projects, click on Develop next to the SCons project.

  • Click on Project Admin -> File Manager

Open the scons, scons-src, and scons-local directories. Within each one, open the $VERSION folder. In the scons/$VERSION directory, select the RELEASE.txt file and mark mark it as the release notes in the popup that appears, then save.

If this is a final release of the production branch, select scons-$VERSION.win32.exe and mark it as the default selection for Windows, then select scons-$VERSION.tar.gz and mark it as the default for all other OSes.

In turn, select each of the files (other than the release notes file itself, which is automatically set) in all three folders (and the folders themselves) and specify that RELEASE.txt is to be the release note for that file. Because of the way the page refreshes after clicking Save, it seems to be easier to start at the bottom and work your way up.

FIXME: TEST THIS: You can do multiple releases quickly by opening the File Manager page multiple times in a separate tabs, but if you do, make sure to wait for the page to reload completely in one tab before clicking Save in another. Trying to update multiple releases at once doesn't work, presumably because the session can only handle one update at a time.

Hide previous releases

  • For a checkpoint release, hide all checkpoints in the same series that are older than this checkpoint.
  • For a minor or micro release, hide all the checkpoint releases leading up to this release.
  • For a major release, hide all the checkpoint releases leading up this release AND hide all major and minor releases older than this release.

For each release name you wish to hide, you have to do the following in scons, scons-local, and scons-src:

  • Click on the gear icon to the left of the package name and select Cut from the popup.

  • Click on the gear icon to the left of old checkpoints and select Paste from the popup.

Test downloading

Navigate to the SCons download page. If you're reading this from a printed copy rather than the wiki page and can't click on the link, follow these instructions:

  • FIXME Add instructions if can't click.

For each of the scons, scons-src, and scons-local directories:

  • Open the directory.
  • Open the $VERSION folder within the directory.
  • For each package in the folder, click on the link to download it.
  • Verify that the files arrived and look reasonable.

Add news item

Navigate to the Project News page. If you're reading this from a printed copy rather than the wiki page and can't click on the link, follow these instructions:

  • FIXME These aren't complete.

  • Go to the Project Admin -> Feature Settings page

  • In Project News, click Submit.

Once you get to the Project News page:

  • Fill in the Subject: box with something like "Release $VERSION now available"

  • Cut-and-paste the blurb you prepared above into the Details: box

  • Click Submit

Update scons.org

Update the scons.org Web Site

Apply changes to web site

Commit the changes you prepared above:

  $ svn commit -m"Changes to publish the SCons $VERSION release"

Now you have to go update the site:

  $ ssh -l scons manam.pair.com
  $ cd public_html
  $ cp -al production new  # hard-link the whole tree; fast.
  $ (cd new && svn up)     # should be fast here too
  $ rm -rf previous
  $ mv production previous && mv new production
  $ exit

Now point your browser to the SCons home page. If anything is amiss, fix it, commit the necessary changes, and update the site.

Test downloading

  • Navigate to the SCons download page.

  • Verify that the opening paragraph makes sense.
  • In the right-hand sidebar, find the download section corresponding to $VERSION.
  • For each file in the section, click on the link to download it.
  • Verify that the files arrived and look reasonable.

Update Tigris.org

Update project pages

Send the changes you prepared above to Tigris:

From within the www subdirectory of your trunk directory:

  $ svn commit -m"Update project web pages for $VERSION"

Point your browser at the roadmap page; if anything's amiss, fix it and commit again.

Add news item

  • Log in to your tigris.org account

  • Click Announcements in the left-hand nav bar

  • Click Add new announcement

  • Double-check the date (it's probably already set)
  • Fill in the Headline box

  • Fill in the Body box with the HTML blurb

  • Click Add new announcement

Add release name to issue tracker

  • Click Issue Tracker on the left-hand nav bar

  • Click Configuration options

  • Click Add/edit components

  • Under scons, to the far right of Add ..., click Version

  • At the bottom of the list, to the right of "Add a new version", click Add

  • Fill in the Version: box with the release ($VERSION)

  • Click the Add button

Announce

Send blurb to SCons mailing lists

Send the blurb to each of these mailing lists individually with a title something like "SCons release $VERSION is now available".

Cherry-picking into Checkpoint

You can also look at a page with just this flow.

These instructions use outdated workflow based on svnmerge tool.

Cherry-picking is the act of applying only selected changesets to a current branch, rather than taking all possible changesets to bring the branch up-to-date.

In our case, trunk is the authoritative source for everything that goes in a release; only under exceptional circumstances are changes made to checkpoint or trunk. The general rule is that a change is made to trunk and brought over to checkpoint.

This flow assumes that the needed changesets are already in trunk; if not, you will need to commit a suitable changeset to trunk following your usual development practices prior to using this flow.

Assumed Setup

These initializations are the conventions that will be used in the rest of this flow.

The location of the SCons SVN archive:

  $ export HG=https://bitbucket.org/scons/scons

The version string, which should look something like this:

  $ VERSION=3.2.0.beta.yyyymmdd

Check Out Checkpoint

From within your base directory, execute this command:

  $ svn co $SVN/checkpoint

Determine Revision(s)

You can review the changes available on trunk to be integrated into checkpoint:

  $ svnmerge.py avail --log -S $SVN/trunk

You can look at the diff that was applied by this revision:

  $ svnmerge.py avail --diff -S $SVN/trunk -r54321

Use this information to determine the revision(s) to be applied.

Merge Revision(s) from Trunk

After selecting the revision(s), merge them into the checkpoint branch.

From within the checkpoint directory:

  $ svnmerge.py merge -r54321,54123,53412 -b -S $SVN/trunk -f commit.txt

Resolve Conflicts

FIXME This segment is overkill for anyone who actually uses SVN. Cut it or drop it.

A conflict occurs when SVN tries to apply changes from the archive that overlap with other changes made locally. These are rare but can happen when cherry-picking applies changes out of order. In general, the correct approach is to use the text from the archive rather than the local version, but use your judgment when making the changes as there will be some cases where the original text (or even a merger of the two texts) is the correct choice as SVN is not always accurate when marking the conflicted regions.

Finding conflicts

To determine if there're conflicts, run this command:

  $ svn status
  ?      test/example.py.r4941
  ?      test/example.py.mine
  ?      test/example.py.r4649
  C      test/example.py
  ?      src/engine/SCons/CoreFile.py.r4941
  ?      src/engine/SCons/CoreFile.py.mine
  ?      src/engine/SCons/CoreFile.py.r4649
  C      src/engine/SCons/CoreFile.py
  $

Lines that begin with "C" identify a file that's conflicted. There are also a number of related files provided by SVN that can be used in the process of resolution, identified by lines that begin with "?". This is a simple resolution, so we won't use them.

Removing Conflicts

To remove the conflicts from a file, follow these steps:

  • Locate a conflict marker in the text. The center of each conflicted region is marked with "=======" so searching for a line with only that on it will get you to the right place.

  • The start of the region is marked with "<<<<<<< .mine" and contains the local text. Delete this text as well as the start and center markers.

  • The end of the region is marked with something that looks like ">>>>>>> .r54321" and contains the text from the SVN archive. Delete the end marker.

  • Repeat until there are no more conflict markers.

To see how your updated file compares with the archive, run this command:

  $ svn diff test/example.py

If the changes aren't satisfactory, repeat the editing step.

Once the editing is done, use this command to tell SVN that the conflict has been resolved:

  $ svn resolved test/example.py

As a side-effect, resolving the conflict will remove the related files provided by SVN.

Repeat this section for each conflicted file.

Update Release Values

Verify that release_tuple in ReleaseConfig has the correct release and says 'beta' or 'candidate' as appropriate, then run the update-release-info command:

$ python bin/update-release-info.py release

The ReleaseConfig file is where the "official" version number ($VERSION), the Python version floors, and other information about the release is recorded. This command takes the information in ReleaseConfig and inserts it in the necessary files.

Commit Modified Checkpoint

Prepare a log.file that describes the revisions you pulled over. You can use the commit.txt returned by the merge step to help prepare the log file. Don't just reuse commit.txt as it tends to be very verbose and hard to read.

Here's an example of a reasonable log.file:

Cherry-picked revisions from trunk because blah blah blah.

From John Doe:
 - One item that was done by John Doe.
 - Another item that was done by John Doe.

From Jim Smith:
 - Item done by Jim Smith.

Use your judgment. If all the revision(s) from John Doe were documentation, and Jim Smith fixed an important bug (and was the actual reason for the cherry-picking), the log message might be like this:

Jim Smith fixed the yadda yadda by blah blah, and the issue was
deemed important enough to be brought over into the checkpoint
branch on a expedited schedule toward a micro release.

Also picked up documentation updates from John Doe as they have
no possibility of destabilizing the release.

Then commit the result to SVN:

  $ svn commit -F log.file
  Sending        some/file
  Sending        some/other/file
  Transmitting file data ..
  Committed revision 54321.
  $

Build Candidate Packages

Provided you have all of the necessary utilities installed, this should be a simple matter of:

  $ rm -rf build bootstrap
  $ python bootstrap.py

While you're waiting for the candidate packages to be built, open another shell window and proceed with preparing the test script. Make sure that $SVN and $VERSION are set up as shell variables as described above.

Test

Prepare test script

The script to run the tests is in the source in bin/fill_me_in. Make a copy and edit it according to the instructions below.

  # report skipped tests as successful
  export TESTCOMMON_PASS_SKIPS=yes
  set -e       # stop on error
  #
  python runtest.py -a
  python runtest.py -a -p tar-gz
  python runtest.py -a -p zip
  python runtest.py -a -p local-tar-gz
  python runtest.py -a -p local-zip
  python runtest.py -a -p src-tar-gz
  python runtest.py -a -p src-zip
  python runtest.py -a -p rpm

The regression suite exercises a wide range of functionality. There are around a thousand tests in the suite, so each test run can take a while.

Regression tests of source tree

Normally, the SCons BuildBot monitors this branch, so it probably started running regression tests as soon as you checked in the changes above. If your machine is covered by one of the BuildBot machines (particularly if your machine is one of the BuildBot machines), you can remove the first runtest in the script.

Regression tests of candidate packages

The build step above not only builds the packages but also unpacks all of them into subdirectories so that you can run the test suite against the packaged goods. This is intended to catch packaging problems such as not adding a new module to the packaging MANIFEST list(s). The runtest.py script supports a -p option and arguments that run the SCons tests against the different unpacked directories.

Edit the script to include the tests you want to include. If you want to be complete, test all of the packages.

To be quicker but still reasonably thorough, test tar-gz and zip, one each of local- and src- (probably do -tar-gz for one and -zip for the other), and rpm.

For a quick-n-dirty test, just test tar-gz or zip, and maybe rpm. Since all of the different packages use the same lists as input, it's pretty unlikely that the tests will pass on one package and fail for another.

Once the script is prepared, save it and return to the window creating the packages.

Check the Build

The build creates the packages in the build/dist subdirectory. If everything built correctly, you should see files that look like the following:

  scons-$VERSION-1.noarch.rpm
  scons-$VERSION-1.src.rpm
  scons-$VERSION.linux-x86_64.tar.gz
  scons-$VERSION.linux-x86_64.zip
  scons-$VERSION.tar.gz
  scons-$VERSION.win32.exe
  scons-$VERSION.zip
  scons-doc-$VERSION.tar.gz
  scons-local-$VERSION.tar.gz
  scons-local-$VERSION.zip
  scons-src-$VERSION.tar.gz
  scons-src-$VERSION.zip

Note that the linux-x86_64 strings in some of the file names above may be different, depending on your local system architecture. That doesn't matter; those files don't go public. Go ahead and remove them.

Run Test Script

FIXME

If the build succeeded, run the test script:

  $ sh path/to/copy-of-script

Determine What and Where to Announce

The sequel describes all the steps for a full-fledged beta or candidate release. This could be overkill. For example, if the change is small and has no impact on the way SCons operates (such as a fix to a regression test so that a production release may proceed), there's no reason to publish the checkpoint at all, as publishing the final release will be sufficient.

There's usually a reason for the choice of changesets to be cherry-picked, and it's important to someone. At a minimum, those interested parties should be notified. Other forms of publishing are optional, so use your judgment as to which, if any, of the following steps should be done.

The tests can run a very long time. While you're waiting, go back to your other shell window and prepare what you can on the assumption that the tests will succeed.

Download SCons Home

From within your base directory, run these commands:

  $ svn co $SVN/scons.org
  $ cd scons.org

The scons.org checkout is pretty large. If you don't want to wait, open another shell window and proceed. Make sure the new shell is set up with the SVN and VERSION shell variables as described above.

Prepare Blurb

Prepare the blurb by starting with a copy of build/scons/RELEASE.txt. It should be pretty much good-to-go, but look over it and see if there are any changes that should be made to make it more suitable as a blurb rather than release notes in a distribution. In general, use your own judgment as to what to keep and what to cut.

FIXME HTML copy of blurb by inserting <br/><br/><br/> instead of double blank lines, <br/><br/> instead of single blank lines, and <br/> in front of items in a list. Not perfect, but more readable.

FIXME Source of beta and final blurbs are different.

Prepare Tigris Announcement

Note that these files must be checked in to the trunk for them to show up on our tigris.org project pages; you can not update and commit these files from the directory in which you built the packages.

From within the www subdirectory of your trunk directory, edit these files:

project_highlights.html

Add a short highlight announcement that will appear at the top of all of the scons.tigris.org pages. Trim the list to remove any entries that are now obsolete.

roadmap.html

Update the text at the top to reflect that this release is now the latest available release.

Prepare scons.org

If you opened another window to do the prep above, move back to the window downloading scons.org; it should be done by now. If you didn't, change back to the scons.org subdirectory of your base directory.

Web site

Make the following changes to the following files: FIXME The destination names aren't CHANGES.txt or RELEASE.txt and the explanation needs to be tweaked. And the source for RELEASE.txt could be either RELEASE.txt or Announce.txt.

CHANGES.txt

copy the new file from build/scons/CHANGES.txt in your packaging directory

RELEASE.txt

copy the new file from build/scons/RELEASE.txt in your packaging directory

includes/versions.php

update $latestrelease to the value $VERSION FIXME
add the new version number to the $docversions[] list
add the new version number to the $apiversions[] list

index.php

add an announcement for the home page
remove any out-of-date announcements

news-raw.xhtml

add an announcement to the list (duplicate it from what you just added to index.php)

Install documentation

To unpack the release documentation into where it can be used by the web pages, just run this script:

FIXME When this script is checked in, it will allow arguments on the command line
  $ sh bin/FillMeIn [ $VERSION [ $SVN ] ]
or something similar. In the meantime, copy the script and run it.

  mkdir doc/$VERSION
  (cd doc/$VERSION && tar zxf build/dist/scons-doc-$RELEASE.tar.gz)
  svn add doc/$VERSION
  (cd doc && rm -f latest && ln -s $VERSION latest)
  case $VERSION in *.0.final.*)
      # This is a 'final' release on the main branch
      (cd doc && rm -f production && ln -s $VERSION production)
  esac

FIXME If any of the candidate packages fail, fix the problem in trunk following your usual development practices and go and do another dry run.

Confirm Tests Succeeded

If the BuildBot display and the test run(s) are OK, then proceed below to publishing the release. If not, fix the problem in trunk following your usual development practices and go back and start again.

Tag the Checkpoint

Verify that you have SVN and VERSION in your shell environment as described in the setup section above, then run this command:

  $ svn cp . $SVN/tags/$VERSION -m"Tagging $VERSION"

Archive Candidate Packages

Verify that you have SVN and VERSION in your shell environment as described in the setup section above, then run the bin/FillMeIn script:

  set -e   # stop on error
  rm -rf ARCHIVE
  svn co --depth=files $SVN/ARCHIVE
  mkdir ARCHIVE/scons-$VERSION
  cp build/dist/* ARCHIVE/scons-$VERSION
  svn add ARCHIVE/scons-$VERSION
  svn commit -m"Saving copy of $VERSION in archive" ARCHIVE
  rm -rf ARCHIVE

Update SourceForge

Upload packages

Verify that you have SVN and VERSION in your shell environment as described in the setup section above, then run the bin/scp-sourceforge script:

  $ sh bin/scp-sourceforge sf_username 

Mark packages with release notes

Navigate to the SourceForge File Manager page for SCons. If you're reading this from a printed copy rather than the wiki page and can't click on the link, follow these instructions:

  • Log in to your SourceForge account

  • Click on Develop just under Forge in the SourceForge banner.

  • Under My Projects, click on Develop next to the SCons project.

  • Click on Project Admin -> File Manager

Open the scons, scons-src, and scons-local directories. Within each one, open the $VERSION folder. In the scons/$VERSION directory, select the RELEASE.txt file and mark mark it as the release notes in the popup that appears, then save.

If this is a final release of the production branch, select scons-$VERSION.win32.exe and mark it as the default selection for Windows, then select scons-$VERSION.tar.gz and mark it as the default for all other OSes.

In turn, select each of the files (other than the release notes file itself, which is automatically set) in all three folders (and the folders themselves) and specify that RELEASE.txt is to be the release note for that file. Because of the way the page refreshes after clicking Save, it seems to be easier to start at the bottom and work your way up.

FIXME: TEST THIS: You can do multiple releases quickly by opening the File Manager page multiple times in a separate tabs, but if you do, make sure to wait for the page to reload completely in one tab before clicking Save in another. Trying to update multiple releases at once doesn't work, presumably because the session can only handle one update at a time.

Hide previous releases

  • For a checkpoint release, hide all checkpoints in the same series that are older than this checkpoint.
  • For a minor or micro release, hide all the checkpoint releases leading up to this release.
  • For a major release, hide all the checkpoint releases leading up this release AND hide all major and minor releases older than this release.

For each release name you wish to hide, you have to do the following in scons, scons-local, and scons-src:

  • Click on the gear icon to the left of the package name and select Cut from the popup.

  • Click on the gear icon to the left of old checkpoints and select Paste from the popup.

Test downloading

Navigate to the SCons download page. If you're reading this from a printed copy rather than the wiki page and can't click on the link, follow these instructions:

  • FIXME Add instructions if can't click.

For each of the scons, scons-src, and scons-local directories:

  • Open the directory.
  • Open the $VERSION folder within the directory.
  • For each package in the folder, click on the link to download it.
  • Verify that the files arrived and look reasonable.

Add news item

Navigate to the Project News page. If you're reading this from a printed copy rather than the wiki page and can't click on the link, follow these instructions:

  • FIXME These aren't complete.

  • Go to the Project Admin -> Feature Settings page

  • In Project News, click Submit.

Once you get to the Project News page:

  • Fill in the Subject: box with something like "Release $VERSION now available"

  • Cut-and-paste the blurb you prepared above into the Details: box

  • Click Submit

Update Scons.Org

Update Tigris.Org

Announce

Send blurb to SCons mailing lists

Send the blurb to each of these mailing lists individually with a title something like "SCons release $VERSION is now available".

Final Release

You can also look at a page with just this flow.

These instructions use outdated workflow based on svnmerge tool.

This flow takes the available changesets in checkpoint and brings release up to date. It assumes that checkpoint and release already exist and have been set up correctly.

The trunk is the authoritative source for everything that goes in a release; only under exceptional circumstances are changes made to other branches. The general rule is that a change is made to trunk, brought over to checkpoint and then merged into release.

This flow assumes that any needed changesets are already in checkpoint; if not, you will need to commit a suitable changeset to trunk following your usual development practices, then cherry-pick the changes into checkpoint before starting this flow.

If you want a checklist you can mark off as you go, cut-n-paste these contents.

Assumed Setup

These initializations are the conventions that will be used in the rest of this flow.

The location of the SCons SVN archive:

  $ export HG=https://bitbucket.org/scons/scons

The version string, which should look something like this:

  $ VERSION=3.2.0.final.0

Check Out Release

From within your base directory, execute this command:

  $ svn co $SVN/release

Merge Checkpoint into Release

  $ cd release
  $ svnmerge merge -b -S $SVN/checkpoint -f commit.txt

Resolve Conflicts

FIXME This segment is overkill for anyone who actually uses SVN. Cut it or drop it.

A conflict occurs when SVN tries to apply changes from the archive that overlap with other changes made locally. These are rare but can happen when cherry-picking applies changes out of order. In general, the correct approach is to use the text from the archive rather than the local version, but use your judgment when making the changes as there will be some cases where the original text (or even a merger of the two texts) is the correct choice as SVN is not always accurate when marking the conflicted regions.

Finding conflicts

To determine if there're conflicts, run this command:

  $ svn status
  ?      test/example.py.r4941
  ?      test/example.py.mine
  ?      test/example.py.r4649
  C      test/example.py
  ?      src/engine/SCons/CoreFile.py.r4941
  ?      src/engine/SCons/CoreFile.py.mine
  ?      src/engine/SCons/CoreFile.py.r4649
  C      src/engine/SCons/CoreFile.py
  $

Lines that begin with "C" identify a file that's conflicted. There are also a number of related files provided by SVN that can be used in the process of resolution, identified by lines that begin with "?". This is a simple resolution, so we won't use them.

Removing Conflicts

To remove the conflicts from a file, follow these steps:

  • Locate a conflict marker in the text. The center of each conflicted region is marked with "=======" so searching for a line with only that on it will get you to the right place.

  • The start of the region is marked with "<<<<<<< .mine" and contains the local text. Delete this text as well as the start and center markers.

  • The end of the region is marked with something that looks like ">>>>>>> .r54321" and contains the text from the SVN archive. Delete the end marker.

  • Repeat until there are no more conflict markers.

To see how your updated file compares with the archive, run this command:

  $ svn diff test/example.py

If the changes aren't satisfactory, repeat the editing step.

Once the editing is done, use this command to tell SVN that the conflict has been resolved:

  $ svn resolved test/example.py

As a side-effect, resolving the conflict will remove the related files provided by SVN.

Repeat this section for each conflicted file.

Update Release Values

Verify that 'release_level in ReleaseConfig is being set to 'final'`.

$ python bin/update-release-info.py release

The ReleaseConfig file is where the "official" version number ($VERSION), the Python version floors, and other information about the release is recorded. This command takes the information in ReleaseConfig and inserts it in the necessary files.

Commit Release

Look at the log messages in commit.txt and create log.file that hits the highlights of the changes that have been incorporated. Much of the information should already be in src/Announce.txt, so it mostly involves copy-n-past from there. Cut it heavily to one or two lines per feature; the major use of this log is people who are scanning to see where a feature was added, and they will appreciate clarity and brevity. This is an example of what log.file might be like:

Rebased the release branch to r54321 of the checkpoint branch.

  -- The blah was updated to do foo

  -- Feature bar was added by John Doe

  -- Various typos

Commit the changes using the log message:

  $ svn commit -F log.file
  Sending        some/file
  Sending        some/other/file
  Transmitting file data ..
  Committed revision 54321.
  $

Build Candidate Packages

Provided you have all of the necessary utilities installed, this should be a simple matter of:

  $ rm -rf build
  $ python bootstrap.py

Test the Candidate Packages

This script is kept in the distribution as bin/xxxxxxxxx:

  $ set -e
  $ python runtest.py -a

Make a copy of it and edit it based on the description below.

Regression Test

The first line of the script runs the regression test. The SCons BuildBot monitors this branch and probably started running regression tests as soon as you checked in the branch. If so, delete the first line of the script as the BuildBot is more comprehensive that running the test on a single machine. You will need to keep an eye on the BuildBot display until all the slaves have finished.

Package Tests

The build step above not only builds the packages but also unpacks all of them into subdirectories so that you can run the test suite against the packaged goods. This is intended to catch packaging problems such as not adding a new module to the packaging MANIFEST list(s). The runtest.py script supports a -p option and arguments that run the SCons tests against the different unpacked directories.

Edit the script to include the tests you want to include. If you want to be complete, test all of the packages.

To be quicker but still reasonably thorough, test tar-gz and zip, one each of local- and src- (probably do -tar-gz for one and -zip for the other), and rpm.

For a quick-n-dirty test, just test tar-gz or zip, and maybe rpm. Since all of the different packages use the same lists as input, it's pretty unlikely that the tests will pass on one package and fail for another.

If any of the tests fail, fix the problem in trunk following your usual development practices, cherry-pick the changes into the checkpoint branch, and go back to merge the changeset(s) into the release branch.

Tag the Release

Verify that you have SVN and VERSION in your shell environment as described in the setup section above, then run this command:

  $ svn cp . $SVN/tags/$VERSION -m"Tagging $VERSION"

Archive Candidate Packages

Verify that you have SVN and VERSION in your shell environment as described in the setup section above, then run the bin/FillMeIn script:

  set -e   # stop on error
  rm -rf ARCHIVE
  svn co --depth=files $SVN/ARCHIVE
  mkdir ARCHIVE/scons-$VERSION
  cp build/dist/* ARCHIVE/scons-$VERSION
  svn add ARCHIVE/scons-$VERSION
  svn commit -m"Saving copy of $VERSION in archive" ARCHIVE
  rm -rf ARCHIVE

Update SourceForge

Upload packages

Verify that you have SVN and VERSION in your shell environment as described in the setup section above, then run the bin/scp-sourceforge script:

  $ sh bin/scp-sourceforge sf_username 

Mark packages with release notes

Navigate to the SourceForge File Manager page for SCons. If you're reading this from a printed copy rather than the wiki page and can't click on the link, follow these instructions:

  • Log in to your SourceForge account

  • Click on Develop just under Forge in the SourceForge banner.

  • Under My Projects, click on Develop next to the SCons project.

  • Click on Project Admin -> File Manager

Open the scons, scons-src, and scons-local directories. Within each one, open the $VERSION folder. In the scons/$VERSION directory, select the RELEASE.txt file and mark mark it as the release notes in the popup that appears, then save.

If this is a final release of the production branch, select scons-$VERSION.win32.exe and mark it as the default selection for Windows, then select scons-$VERSION.tar.gz and mark it as the default for all other OSes.

In turn, select each of the files (other than the release notes file itself, which is automatically set) in all three folders (and the folders themselves) and specify that RELEASE.txt is to be the release note for that file. Because of the way the page refreshes after clicking Save, it seems to be easier to start at the bottom and work your way up.

FIXME: TEST THIS: You can do multiple releases quickly by opening the File Manager page multiple times in a separate tabs, but if you do, make sure to wait for the page to reload completely in one tab before clicking Save in another. Trying to update multiple releases at once doesn't work, presumably because the session can only handle one update at a time.

Hide previous releases

  • For a checkpoint release, hide all checkpoints in the same series that are older than this checkpoint.
  • For a minor or micro release, hide all the checkpoint releases leading up to this release.
  • For a major release, hide all the checkpoint releases leading up this release AND hide all major and minor releases older than this release.

For each release name you wish to hide, you have to do the following in scons, scons-local, and scons-src:

  • Click on the gear icon to the left of the package name and select Cut from the popup.

  • Click on the gear icon to the left of old checkpoints and select Paste from the popup.

Test downloading

Navigate to the SCons download page. If you're reading this from a printed copy rather than the wiki page and can't click on the link, follow these instructions:

  • FIXME Add instructions if can't click.

For each of the scons, scons-src, and scons-local directories:

  • Open the directory.
  • Open the $VERSION folder within the directory.
  • For each package in the folder, click on the link to download it.
  • Verify that the files arrived and look reasonable.

Add news item

Navigate to the Project News page. If you're reading this from a printed copy rather than the wiki page and can't click on the link, follow these instructions:

  • FIXME These aren't complete.

  • Go to the Project Admin -> Feature Settings page

  • In Project News, click Submit.

Once you get to the Project News page:

  • Fill in the Subject: box with something like "Release $VERSION now available"

  • Cut-and-paste the blurb you prepared above into the Details: box

  • Click Submit

Update scons.org

Update the scons.org Web Site

Apply changes to web site

Commit the changes you prepared above:

  $ svn commit -m"Changes to publish the SCons $VERSION release"

Now you have to go update the site:

  $ ssh -l scons manam.pair.com
  $ cd public_html
  $ cp -al production new  # hard-link the whole tree; fast.
  $ (cd new && svn up)     # should be fast here too
  $ rm -rf previous
  $ mv production previous && mv new production
  $ exit

Now point your browser to the SCons home page. If anything is amiss, fix it, commit the necessary changes, and update the site.

Test downloading

  • Navigate to the SCons download page.

  • Verify that the opening paragraph makes sense.
  • In the right-hand sidebar, find the download section corresponding to $VERSION.
  • For each file in the section, click on the link to download it.
  • Verify that the files arrived and look reasonable.

Update Tigris.org

Update project pages

Send the changes you prepared above to Tigris:

From within the www subdirectory of your trunk directory:

  $ svn commit -m"Update project web pages for $VERSION"

Point your browser at the roadmap page; if anything's amiss, fix it and commit again.

Add news item

  • Log in to your tigris.org account

  • Click Announcements in the left-hand nav bar

  • Click Add new announcement

  • Double-check the date (it's probably already set)
  • Fill in the Headline box

  • Fill in the Body box with the HTML blurb

  • Click Add new announcement

Add release name to issue tracker

  • Click Issue Tracker on the left-hand nav bar

  • Click Configuration options

  • Click Add/edit components

  • Under scons, to the far right of Add ..., click Version

  • At the bottom of the list, to the right of "Add a new version", click Add

  • Fill in the Version: box with the release ($VERSION)

  • Click the Add button

Announce

Prepare a blurb by starting with a copy of src/Announce.txt. Edit the file and follow these guidelines:

  • If the "scheduled for the next release" section has no content, cut it.
  • Keep the first "changes since release" section (which should be since the last production release).
  • Cut all of the subsequent "changes since release" sections.
  • Leave the copyright notice at the bottom.

In general, use your own judgment as to what to keep and what to cut.

Post Blurb on FreshMeat

HOW? If you know how to do this, UPDATE THIS INFORMATION.

Send Blurb to Other Interested Parties

  • Debian maintainer?
  • Fedora maintainer?
  • Ubuntu maintainer?
  • RedHat maintainer?

OTHERS? Are there other individuals we should notify? Or other mailing lists? If you know of any, UPDATE THIS INFORMATION.

See http://en.wikipedia.org/wiki/Linux_distributions and http://en.wikipedia.org/wiki/DistroWatch for candidates to be contacted.

Send blurb to SCons mailing lists

Send the blurb to each of these mailing lists individually with a title something like "SCons release $VERSION is now available".

Update Trunk to Next Version

Initialize files for next version

From within your base directory, execute this command:

  $ svn co $SVN/trunk

From within the trunk directory, edit ReleaseConfig and change the assignment of release_level to 'post'.

$ python bin/update-release-info.py release

The ReleaseConfig file is where the "official" version number ($VERSION), the Python version floors, and other information about the release is recorded. This command takes the information in ReleaseConfig and inserts it in the necessary files.

A release level of 'post' causes update-release-info.py to update ReleaseConfig itself to the next minor version and then tweak some other files to set them up for the the beginning of a new version.

Verify files are correct

Edit files ... FIXME

Commit changes for next version

Create log.file ... FIXME

  $ svn commit -F log.file
  Sending        some/file
  Sending        some/other/file
  Transmitting file data ..
  Committed revision 54321.
  $

Finalize

Go celebrate; you're all done.

Non-tip Releases

Setup

These instructions use outdated workflow based on svnmerge tool.

This flow sets up a branch for a potential series of point releases.

FIXME:

  • steal comments and examples from BranchAndMerge

  • add some setup for ReleaseConfig

  • InitSVN ...

Initialize...
  $ export SVN=http://scons.tigris.org/svn/scons
  $ export POINT=3.2.x

Create point branch...
  $ cd my_working_directory
  $ svn co $SVN/trunk
  $ svn cp trunk $SVN/branches/$POINT
  $ svn commit -m'Added branches/$POINT based on trunk' trunk
I don't know why that last line is needed...

Set svnmerge base...
  $ svn co $SVN/branches/$POINT working
  $ (cd working && svnmerge init ../trunk)
  $ (cd trunk && svnmerge init ../working)
  $ (cd working && svn commit -m'Set svnmerge base for $POINT')
  $ (cd trunk && svn commit -m'Set svnmerge base for trunk')

You can actually do both the "svnmerge init" and "svn commit" on one branch (in one directory) and then do both on the other branch, but doing it this way makes both of them end up with the same base revision number in the svnmerge property, which is nice and symmetric.

Alpha Release

You can also look at a page with just this flow.

An alpha release is a quick-n-dirty way of getting out initial releases for testing. It can be done at any point, even from a read-only tree, as there's no requirement for committing the branch when the process is completed.

If you want a checklist you can mark off as you go, cut-n-paste these contents.

Assumed Setup

These initializations are the conventions that will be used in the rest of this flow.

The location of the SCons SVN archive:

  $ export HG=https://bitbucket.org/scons/scons

The version string, which should look something like this:

  $ VERSION=3.2.1.alpha.yyyymmdd

Check Out Branch

From within your base directory, execute this command:

  $ svn co $SVN/branches/3.2.x working
  $ cd working

Update Release Values

Verify that the release_level variable in ReleaseConfig is being set to 'alpha'.

$ python bin/update-release-info.py release

The ReleaseConfig file is where the "official" version number ($VERSION), the Python version floors, and other information about the release is recorded. This command takes the information in ReleaseConfig and inserts it in the necessary files.

Run Regression Tests (Optional)

Since an alpha package is intended to be transient, only useful for a short time, running the regression tests may not be necessary.

To run the tests, simply execute this command:

  $ python runtest.py -a

Build Candidate Packages

Provided you have all of the necessary utilities installed, this should be a simple matter of:

  $ rm -rf build
  $ python bootstrap.py

Test Candidate Packages (Optional)

Since an alpha package is intended to be transient, only useful for a short time, testing the packages may not be necessary.

The build step above not only builds the packages but also unpacks all of them into subdirectories so that you can run the test suite against the packaged goods. This is intended to catch packaging problems such as not adding a new module to the packaging MANIFEST list(s). The runtest.py script supports a -p option and arguments that run the SCons tests against the different unpacked directories.

Edit the script to include the tests you want to include. If you want to be complete, test all of the packages.

To be quicker but still reasonably thorough, test tar-gz and zip, one each of local- and src- (probably do -tar-gz for one and -zip for the other), and rpm.

For a quick-n-dirty test, just test tar-gz or zip, and maybe rpm. Since all of the different packages use the same lists as input, it's pretty unlikely that the tests will pass on one package and fail for another.

Make Available

TO BE ADDED LATER; if you have some good words for this topic, please edit ReleaseHOWTO/NonTipAlphaBody and put them in.

Beta and Candidate Releases

You can also look at a page with just this flow.

A beta release is prepared and published when the branch has stabilized enough that it's worth exposing to a wider audience.

If you want a checklist you can mark off as you go, cut-n-paste these contents.

Assumed Setup

These initializations are the conventions that will be used in the rest of this flow.

The location of the SCons SVN archive:

  $ export HG=https://bitbucket.org/scons/scons

The version string, which should look something like this:

  $ VERSION=3.2.1.beta.yyyymmdd

Check Out Branch

From within your base directory, execute this command:

  $ svn co $SVN/branches/3.2.x working
  $ cd working

Edit Configuration Files

FIXME

Update Release Values

Verify that the release_level variable in ReleaseConfig is being set to 'beta'.

$ python bin/update-release-info.py release

The ReleaseConfig file is where the "official" version number ($VERSION), the Python version floors, and other information about the release is recorded. This command takes the information in ReleaseConfig and inserts it in the necessary files.

Confirm Users' Guide is OK

CAUTION: Only try this step if you have the toolchain for building the documentation. The packages needed are discussed in DeveloperGuide/Documentation. It's quite a list.

The following steps verify that the user guide sources (some of which are generated from other files) are up to date.

FIXME (This will probably be reworked to be part of the build step.)

  $ python bin/scons-doc.py --diff
  $ python bin/scons-doc.py --update

Commit Branch

Create log.file ... FIXME

  $ svn commit -F log.file
  Sending        some/file
  Sending        some/other/file
  Transmitting file data ..
  Committed revision 54321.
  $

Run Regression Tests

The SCons BuildBot will usually be monitoring this branch, so all you have to do is wait until the BuildBot display for the commit from the previous step shows that all the regression tests have been run. On the other hand, if you're feeling bored, you can run the regression tests while you're waiting... ;-)

To run the tests, simply execute this command:

  $ python runtest.py -a

If the regression tests fail, fix the problem in the branch following your usual development practices and start the flow again.

Build Candidate Packages

Provided you have all of the necessary utilities installed, this should be a simple matter of:

  $ rm -rf build
  $ python bootstrap.py

Test Candidate Packages

The build step above not only builds the packages but also unpacks all of them into subdirectories so that you can run the test suite against the packaged goods. This is intended to catch packaging problems such as not adding a new module to the packaging MANIFEST list(s). The runtest.py script supports a -p option and arguments that run the SCons tests against the different unpacked directories.

Edit the script to include the tests you want to include. If you want to be complete, test all of the packages.

To be quicker but still reasonably thorough, test tar-gz and zip, one each of local- and src- (probably do -tar-gz for one and -zip for the other), and rpm.

For a quick-n-dirty test, just test tar-gz or zip, and maybe rpm. Since all of the different packages use the same lists as input, it's pretty unlikely that the tests will pass on one package and fail for another.

If any of the candidate packages fail, fix the problem in the branch following your usual development practices and start the flow again.

Tag the Release

Verify that you have SVN and VERSION in your shell environment as described in the setup section above, then run this command:

  $ svn cp . $SVN/tags/$VERSION -m"Tagging $VERSION"

Archive Candidate Packages

Verify that you have SVN and VERSION in your shell environment as described in the setup section above, then run the bin/FillMeIn script:

  set -e   # stop on error
  rm -rf ARCHIVE
  svn co --depth=files $SVN/ARCHIVE
  mkdir ARCHIVE/scons-$VERSION
  cp build/dist/* ARCHIVE/scons-$VERSION
  svn add ARCHIVE/scons-$VERSION
  svn commit -m"Saving copy of $VERSION in archive" ARCHIVE
  rm -rf ARCHIVE

Update SourceForge

Upload packages

Verify that you have SVN and VERSION in your shell environment as described in the setup section above, then run the bin/scp-sourceforge script:

  $ sh bin/scp-sourceforge sf_username 

Mark packages with release notes

Navigate to the SourceForge File Manager page for SCons. If you're reading this from a printed copy rather than the wiki page and can't click on the link, follow these instructions:

  • Log in to your SourceForge account

  • Click on Develop just under Forge in the SourceForge banner.

  • Under My Projects, click on Develop next to the SCons project.

  • Click on Project Admin -> File Manager

Open the scons, scons-src, and scons-local directories. Within each one, open the $VERSION folder. In the scons/$VERSION directory, select the RELEASE.txt file and mark mark it as the release notes in the popup that appears, then save.

If this is a final release of the production branch, select scons-$VERSION.win32.exe and mark it as the default selection for Windows, then select scons-$VERSION.tar.gz and mark it as the default for all other OSes.

In turn, select each of the files (other than the release notes file itself, which is automatically set) in all three folders (and the folders themselves) and specify that RELEASE.txt is to be the release note for that file. Because of the way the page refreshes after clicking Save, it seems to be easier to start at the bottom and work your way up.

FIXME: TEST THIS: You can do multiple releases quickly by opening the File Manager page multiple times in a separate tabs, but if you do, make sure to wait for the page to reload completely in one tab before clicking Save in another. Trying to update multiple releases at once doesn't work, presumably because the session can only handle one update at a time.

Hide previous releases

  • For a checkpoint release, hide all checkpoints in the same series that are older than this checkpoint.
  • For a minor or micro release, hide all the checkpoint releases leading up to this release.
  • For a major release, hide all the checkpoint releases leading up this release AND hide all major and minor releases older than this release.

For each release name you wish to hide, you have to do the following in scons, scons-local, and scons-src:

  • Click on the gear icon to the left of the package name and select Cut from the popup.

  • Click on the gear icon to the left of old checkpoints and select Paste from the popup.

Test downloading

Navigate to the SCons download page. If you're reading this from a printed copy rather than the wiki page and can't click on the link, follow these instructions:

  • FIXME Add instructions if can't click.

For each of the scons, scons-src, and scons-local directories:

  • Open the directory.
  • Open the $VERSION folder within the directory.
  • For each package in the folder, click on the link to download it.
  • Verify that the files arrived and look reasonable.

Add news item

Navigate to the Project News page. If you're reading this from a printed copy rather than the wiki page and can't click on the link, follow these instructions:

  • FIXME These aren't complete.

  • Go to the Project Admin -> Feature Settings page

  • In Project News, click Submit.

Once you get to the Project News page:

  • Fill in the Subject: box with something like "Release $VERSION now available"

  • Cut-and-paste the blurb you prepared above into the Details: box

  • Click Submit

Update scons.org

Update the scons.org Web Site

Apply changes to web site

Commit the changes you prepared above:

  $ svn commit -m"Changes to publish the SCons $VERSION release"

Now you have to go update the site:

  $ ssh -l scons manam.pair.com
  $ cd public_html
  $ cp -al production new  # hard-link the whole tree; fast.
  $ (cd new && svn up)     # should be fast here too
  $ rm -rf previous
  $ mv production previous && mv new production
  $ exit

Now point your browser to the SCons home page. If anything is amiss, fix it, commit the necessary changes, and update the site.

Test downloading

  • Navigate to the SCons download page.

  • Verify that the opening paragraph makes sense.
  • In the right-hand sidebar, find the download section corresponding to $VERSION.
  • For each file in the section, click on the link to download it.
  • Verify that the files arrived and look reasonable.

Update Tigris.org

Update project pages

Send the changes you prepared above to Tigris:

From within the www subdirectory of your trunk directory:

  $ svn commit -m"Update project web pages for $VERSION"

Point your browser at the roadmap page; if anything's amiss, fix it and commit again.

Add news item

  • Log in to your tigris.org account

  • Click Announcements in the left-hand nav bar

  • Click Add new announcement

  • Double-check the date (it's probably already set)
  • Fill in the Headline box

  • Fill in the Body box with the HTML blurb

  • Click Add new announcement

Add release name to issue tracker

  • Click Issue Tracker on the left-hand nav bar

  • Click Configuration options

  • Click Add/edit components

  • Under scons, to the far right of Add ..., click Version

  • At the bottom of the list, to the right of "Add a new version", click Add

  • Fill in the Version: box with the release ($VERSION)

  • Click the Add button

Announce

Send blurb to SCons mailing lists

Send the blurb to each of these mailing lists individually with a title something like "SCons release $VERSION is now available".

Final Release

You can also look at a page with just this flow.

A final release is prepared and published when FIXME

If you want a checklist you can mark off as you go, cut-n-paste these contents.

Assumed Setup

These initializations are the conventions that will be used in the rest of this flow.

The location of the SCons SVN archive:

  $ export HG=https://bitbucket.org/scons/scons

The version string, which should look something like this:

  $ VERSION=3.2.1.final.0

Check Out Branch

From within your base directory, execute this command:

  $ svn co $SVN/branches/3.2.x working
  $ cd working

Edit Configuration Files

FIXME

Update Release Values

Verify that the release_level variable in ReleaseConfig is being set to 'final'.

$ python bin/update-release-info.py release

The ReleaseConfig file is where the "official" version number ($VERSION), the Python version floors, and other information about the release is recorded. This command takes the information in ReleaseConfig and inserts it in the necessary files.

Confirm Users' Guide is OK

CAUTION: Only try this step if you have the toolchain for building the documentation. The packages needed are discussed in DeveloperGuide/Documentation. It's quite a list.

The following steps verify that the user guide sources (some of which are generated from other files) are up to date.

FIXME (This will probably be reworked to be part of the build step.)

  $ python bin/scons-doc.py --diff
  $ python bin/scons-doc.py --update

Commit Branch

Create log.file ... FIXME

  $ svn commit -F log.file
  Sending        some/file
  Sending        some/other/file
  Transmitting file data ..
  Committed revision 54321.
  $

Run Regression Tests

The SCons BuildBot will usually be monitoring this branch, so all you have to do is wait until the BuildBot display for the commit from the previous step shows that all the regression tests have been run. On the other hand, if you're feeling bored, you can run the regression tests while you're waiting... ;-)

To run the tests, simply execute this command:

  $ python runtest.py -a

If the regression tests fail, fix the problem in the branch following your usual development practices and start the flow again.

Build Candidate Packages

Provided you have all of the necessary utilities installed, this should be a simple matter of:

  $ rm -rf build
  $ python bootstrap.py

Test Candidate Packages

The build step above not only builds the packages but also unpacks all of them into subdirectories so that you can run the test suite against the packaged goods. This is intended to catch packaging problems such as not adding a new module to the packaging MANIFEST list(s). The runtest.py script supports a -p option and arguments that run the SCons tests against the different unpacked directories.

Edit the script to include the tests you want to include. If you want to be complete, test all of the packages.

To be quicker but still reasonably thorough, test tar-gz and zip, one each of local- and src- (probably do -tar-gz for one and -zip for the other), and rpm.

For a quick-n-dirty test, just test tar-gz or zip, and maybe rpm. Since all of the different packages use the same lists as input, it's pretty unlikely that the tests will pass on one package and fail for another.

If any of the candidate packages fail, fix the problem in the branch following your usual development practices and start the flow again.

Tag the Release

Verify that you have SVN and VERSION in your shell environment as described in the setup section above, then run this command:

  $ svn cp . $SVN/tags/$VERSION -m"Tagging $VERSION"

Archive Candidate Packages

Verify that you have SVN and VERSION in your shell environment as described in the setup section above, then run the bin/FillMeIn script:

  set -e   # stop on error
  rm -rf ARCHIVE
  svn co --depth=files $SVN/ARCHIVE
  mkdir ARCHIVE/scons-$VERSION
  cp build/dist/* ARCHIVE/scons-$VERSION
  svn add ARCHIVE/scons-$VERSION
  svn commit -m"Saving copy of $VERSION in archive" ARCHIVE
  rm -rf ARCHIVE

Update SourceForge

Upload packages

Verify that you have SVN and VERSION in your shell environment as described in the setup section above, then run the bin/scp-sourceforge script:

  $ sh bin/scp-sourceforge sf_username 

Mark packages with release notes

Navigate to the SourceForge File Manager page for SCons. If you're reading this from a printed copy rather than the wiki page and can't click on the link, follow these instructions:

  • Log in to your SourceForge account

  • Click on Develop just under Forge in the SourceForge banner.

  • Under My Projects, click on Develop next to the SCons project.

  • Click on Project Admin -> File Manager

Open the scons, scons-src, and scons-local directories. Within each one, open the $VERSION folder. In the scons/$VERSION directory, select the RELEASE.txt file and mark mark it as the release notes in the popup that appears, then save.

If this is a final release of the production branch, select scons-$VERSION.win32.exe and mark it as the default selection for Windows, then select scons-$VERSION.tar.gz and mark it as the default for all other OSes.

In turn, select each of the files (other than the release notes file itself, which is automatically set) in all three folders (and the folders themselves) and specify that RELEASE.txt is to be the release note for that file. Because of the way the page refreshes after clicking Save, it seems to be easier to start at the bottom and work your way up.

FIXME: TEST THIS: You can do multiple releases quickly by opening the File Manager page multiple times in a separate tabs, but if you do, make sure to wait for the page to reload completely in one tab before clicking Save in another. Trying to update multiple releases at once doesn't work, presumably because the session can only handle one update at a time.

Hide previous releases

  • For a checkpoint release, hide all checkpoints in the same series that are older than this checkpoint.
  • For a minor or micro release, hide all the checkpoint releases leading up to this release.
  • For a major release, hide all the checkpoint releases leading up this release AND hide all major and minor releases older than this release.

For each release name you wish to hide, you have to do the following in scons, scons-local, and scons-src:

  • Click on the gear icon to the left of the package name and select Cut from the popup.

  • Click on the gear icon to the left of old checkpoints and select Paste from the popup.

Test downloading

Navigate to the SCons download page. If you're reading this from a printed copy rather than the wiki page and can't click on the link, follow these instructions:

  • FIXME Add instructions if can't click.

For each of the scons, scons-src, and scons-local directories:

  • Open the directory.
  • Open the $VERSION folder within the directory.
  • For each package in the folder, click on the link to download it.
  • Verify that the files arrived and look reasonable.

Add news item

Navigate to the Project News page. If you're reading this from a printed copy rather than the wiki page and can't click on the link, follow these instructions:

  • FIXME These aren't complete.

  • Go to the Project Admin -> Feature Settings page

  • In Project News, click Submit.

Once you get to the Project News page:

  • Fill in the Subject: box with something like "Release $VERSION now available"

  • Cut-and-paste the blurb you prepared above into the Details: box

  • Click Submit

Update scons.org

Update the scons.org Web Site

Apply changes to web site

Commit the changes you prepared above:

  $ svn commit -m"Changes to publish the SCons $VERSION release"

Now you have to go update the site:

  $ ssh -l scons manam.pair.com
  $ cd public_html
  $ cp -al production new  # hard-link the whole tree; fast.
  $ (cd new && svn up)     # should be fast here too
  $ rm -rf previous
  $ mv production previous && mv new production
  $ exit

Now point your browser to the SCons home page. If anything is amiss, fix it, commit the necessary changes, and update the site.

Test downloading

  • Navigate to the SCons download page.

  • Verify that the opening paragraph makes sense.
  • In the right-hand sidebar, find the download section corresponding to $VERSION.
  • For each file in the section, click on the link to download it.
  • Verify that the files arrived and look reasonable.

Update Tigris.org

Update project pages

Send the changes you prepared above to Tigris:

From within the www subdirectory of your trunk directory:

  $ svn commit -m"Update project web pages for $VERSION"

Point your browser at the roadmap page; if anything's amiss, fix it and commit again.

Add news item

  • Log in to your tigris.org account

  • Click Announcements in the left-hand nav bar

  • Click Add new announcement

  • Double-check the date (it's probably already set)
  • Fill in the Headline box

  • Fill in the Body box with the HTML blurb

  • Click Add new announcement

Add release name to issue tracker

  • Click Issue Tracker on the left-hand nav bar

  • Click Configuration options

  • Click Add/edit components

  • Under scons, to the far right of Add ..., click Version

  • At the bottom of the list, to the right of "Add a new version", click Add

  • Fill in the Version: box with the release ($VERSION)

  • Click the Add button

Announce

Prepare a blurb by starting with a copy of src/Announce.txt. Edit the file and follow these guidelines:

  • If the "scheduled for the next release" section has no content, cut it.
  • Keep the first "changes since release" section (which should be since the last production release).
  • Cut all of the subsequent "changes since release" sections.
  • Leave the copyright notice at the bottom.

In general, use your own judgment as to what to keep and what to cut.

Post Blurb on FreshMeat

HOW? If you know how to do this, UPDATE THIS INFORMATION.

Send Blurb to Other Interested Parties

  • Debian maintainer?
  • Fedora maintainer?
  • Ubuntu maintainer?
  • RedHat maintainer?

OTHERS? Are there other individuals we should notify? Or other mailing lists? If you know of any, UPDATE THIS INFORMATION.

See http://en.wikipedia.org/wiki/Linux_distributions and http://en.wikipedia.org/wiki/DistroWatch for candidates to be contacted.

Send blurb to SCons mailing lists

Send the blurb to each of these mailing lists individually with a title something like "SCons release $VERSION is now available".

ReleaseHOWTO (last edited 2012-08-07 02:44:39 by GaryOberbrunner)