Differences between revisions 35 and 36
Revision 35 as of 2009-09-11 11:57:52
Size: 4339
Revision 36 as of 2013-12-28 14:12:33
Size: 0
Editor: DirkBaechle
Comment: contents got merged into DeveloperGuide/TestingMethodology
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
= About this page =
SCons has an extensive test suite.

The official guidelines for new functionality say that a new feature should be accompanied by a test that fails on old code.

This page has some tips for running, writing and debugging the tests in the external test suite.

(These are the scripts in the {{{test/}}} subdirectory, not the unit test modules that live with the source.)

See also DeveloperGuide/TestingMethodology for more info on writing, debugging, and using the test suite. Some of that is redundant with this page and should be cleaned up. It has good info on how to write a test. This page is currently more about debugging tests; I think this page should be merged into that one, but don't have time right now.

== Running the tests ==
Run the tests using runtest.py like this:

{{{python runtest.py -a}}} for all the tests, or {{{python runtest.py test/aTest.py}}} for a specific test.

runtest executes the test scripts, and cleans up after them.

== Debugging failing tests ==
For failing tests or bugs in the SConstruct files that are generated by the test scripts, you may want to look at the files that are being generated.

To do so, set one of these environment variables: {{{PRESERVE, PRESERVE_FAIL, PRESERVE_PASS, PRESERVE_NO_RESULT}}}. {{{PRESERVE}}} is equivalent to all being set, while each other one only preserves files if a condition is met (test failure, passing, etc)

The temporary files will not be deleted, so you can check them out and try them manually. Be sure you set them to some value (any will do):

% setenv PRESERVE yes
Then, when you run runtest.py, it will print out the location of the temporary files used in the script.

% python runtest.py test/SharedLibrary.py
/sw/bin/python /Users/mike/Code/SCons/scons-cvs/test/SharedLibrary.py
/sw/bin/python "/Users/mike/Code/SCons/scons-cvs/src/script/scons.py" .
/sw/bin/python "/Users/mike/Code/SCons/scons-cvs/src/script/scons.py" -f SConstructFoo
/sw/bin/python "/Users/mike/Code/SCons/scons-cvs/src/script/scons.py" -f SConstructFoo
/sw/bin/python "/Users/mike/Code/SCons/scons-cvs/src/script/scons.py" -f SConstructFoo2
Preserved directory /private/tmp/tmpXiLuky
Preserved directory /private/tmp/tmpXiLuky
If you are curious what exact command lines are being run, you can alter the test script to add this call:

{{{test.verbose_set(1)}}}, a method defined in TestCmd, causes the test script to print out the exact commands that the test is running.

They will usually not be that useful unless you are also preserving the files as above.

== Debugging test scripts ==
For errors in the test script itself, runtest.py has a -d switch that starts the script in the debugger. This helps avoid pain caused by setting up PYTHONPATH by hand when debugging. See also DebuggingScons for how to run SCons in the debugger without the test infrastructure (e.g. when debugging a SConstruct/SConscript).

From Kevin Quick on the dev list:

When using -d, runtest.py enters pdb right away, which is why it doesn't really know where you are. I typically use it as follows:

If I'm having a problem with test/foo.py failing. I figure out what line is failing, or correspondingly look for the test.run() call that I want to see things at; I do this in my editor and I have the editor tell me what line this is (NNN).

  $ python runtest.py test/foo.py
  (pdb) b NNN
  (pdb) c
  <runs until reaching breakpoint at line NNN>
From there I can do all the standard pdb stuff like examining variables, stepping, etc.

The pdb really only works for the test script; the actual test run is done in a subprocess; the SCons engine is running in that subprocess and so it's separate from the pdb. Honestly, for debugging SCons engine stuff, I usually just insert print statements and re-run things because that's usually fastest; other folks may have pdb recommendations for the engine itself.

With versions of scons following 0.96.1, an internal exception during scons execution will print a one-line message about the exception only. To get more information about where the exception occurred, goto the {{{test.run(...)}}} line that failed and add " {{{--debug=stacktrace}}} " to the arguments for the test.