SCons - a software construction tool
-Welcome to the SCons development tree. The real purpose of this tree is
-to package SCons for production distribution in a variety of formats,
+Welcome to the SCons development tree. The real purpose of this tree
+is to package SCons for production distribution in a variety of formats,
not just to hack SCons code.
-To that extent, the normal development cycle (enforced by Aegis) is not
-to test the code directly, but to package SCons, unpack the package,
-"install" SCons in a test subdirectory, and then to run the tests
-against the unpacked and installed software. This helps eliminate
-problems caused by, for example, failure to update the list of files to
-be packaged.
+If all you want to do is install and run SCons, it will be easier for you
+to download and install the scons-{version}.tar.gz or scons-{version}.zip
+package rather than to work with the packaging logic in this tree.
-Note that if all you want to do is install and run SCons, it
-will probably be easier for you to download and install the
-scons-{version}.tar.gz or scons-{version}.zip package rather than to
-work with the packaging logic in this tree.
+To the extent that this tree is about building SCons packages, the *full*
+development cycle is not just to test the code directly, but to package
+SCons, unpack the package, "install" SCons in a test subdirectory,
+and then to run the tests against the unpacked and installed software.
+This helps eliminate problems caused by, for example, failure to update
+the list of files to be packaged.
+
+For just working on making an individual change to the SCons source,
+however, you don't actually need to build or install SCons; you
+*can* actually edit and execute SCons in-place. See the following
+sections below for more information:
+
+ MAKING CHANGES
+ How to edit and execute SCons in-place.
+
+ DEBUGGING
+ Tips for debugging problems in SCons.
+
+ TESTING
+ How to use the automated regression tests.
+
+ DEVELOPMENT WORKFLOW
+ An example of how to put the edit/execute/test pieces
+ together in a reasonable development workflow.
LATEST VERSION
-- (Optional.) Install from a pre-packaged SCons package that
does not require distutils:
- Red Hat Linux scons-0.96.noarch.rpm
+ Red Hat Linux scons-1.3.0.noarch.rpm
- Debian GNU/Linux scons_0.96_all.deb
- (or use apt-get)
+ Debian GNU/Linux use apt-get to get the official package
- Windows scons-0.96.win32.exe
+ Windows scons-1.3.0.win32.exe
-- (Recommended.) Download the latest distutils package from the
following URL:
INSTALLATION
============
+ NOTE: You don't need to build SCons packages or install SCons if
+ you just want to work on developing a patch. See the sections
+ about MAKING CHANGES and TESTING below if you just want to submit
+ a bug fix or some new functionality. See the sections below about
+ BUILDING PACKAGES and TESTING PACKAGES if your enhancement involves
+ changing the way in which SCons is packaged and/or installed on an
+ end-user system.
+
Assuming your system satisfies the installation requirements in the
previous section, install SCons from this package by first populating
the build/scons/ subdirectory. (For an easier way to install SCons,
$ scons build/scons
-If you don't have SCons version 0.96 or later already installed on your
-system, you can use SCons itself to populate the build/scons/ directory
-with a little more typing. You must first set the SCONS_LIB_DIR
-environment variable to the local src/engine subdirectory, and then
-execute the local src/script/scons.py script to populate the build/scons/
-subdirectory. You would do this as follows on a Linux or UNIX system
-(using sh or a derivative like bash or ksh):
+You can also use this version of SCons to populate its own build directory
+by using a supplied bootstrap.py script:
+
+ $ python bootstrap.py build/scons
+
+The bootstrap.py keeps the src/ subdirectory free of compiled Python
+(*.pyc or *.pyo) files by copying the necessary SCons files to a local
+bootstrap/ subdirectory and executing it from there.
+
+You can also execute the local SCons directly from the src/ subdirectory
+by first setting the SCONS_LIB_DIR environment variable to the local
+src/engine subdirectory, and then executing the local src/script/scons.py
+script to populate the build/scons/ subdirectory. You would do this as
+follows on a Linux or UNIX system (using sh or a derivative like bash
+or ksh):
$ export SCONS_LIB_DIR=`pwd`/src/engine
$ python src/script/scons.py build/scons
Or as follows on Windows:
C:\scons\>set SCONS_LIB_DIR=%cd%\src\engine
- C:\scons\>python src/script/scons.py build/scons
+ C:\scons\>python src\script\scons.py build/scons
-Either command will populate the build/scons/ directory with the necessary
-files and directory structure to use the Python-standard setup script
-as follows on Linux or UNIX:
+Any of the above commands will populate the build/scons/ directory with
+the necessary files and directory structure to use the Python-standard
+setup script as follows on Linux or UNIX:
# cd build/scons
# python setup.py install
By default, the above commands will do the following:
- -- Install the version-numbered "scons-0.96" and "sconsign-0.96"
+ -- Install the version-numbered "scons-1.3.0" and "sconsign-1.3.0"
scripts in the default system script directory (/usr/bin or
C:\Python*\Scripts, for example). This can be disabled by
specifying the "--no-version-script" option on the command
for example). This can be disabled by specifying the
"--no-scons-script" option on the command line, which is useful
if you want to install and experiment with a new version before
- making it the default on your system. On UNIX or Linux systems,
- you can have the "scons" and "sconsign" scripts be hard links or
- symbolic links to the "scons-0.96" and "sconsign-0.96" scripts
- by specifying the "--hardlink-scons" or "--symlink-scons"
- options on the command line.
+ making it the default on your system.
- -- Install "scons-0.96.bat" and "scons.bat" wrapper scripts in the
+ On UNIX or Linux systems, you can have the "scons" and "sconsign"
+ scripts be hard links or symbolic links to the "scons-1.3.0" and
+ "sconsign-1.3.0" scripts by specifying the "--hardlink-scons" or
+ "--symlink-scons" options on the command line.
+
+ -- Install "scons-1.3.0.bat" and "scons.bat" wrapper scripts in the
Python prefix directory on Windows (C:\Python*, for example).
This can be disabled by specifying the "--no-install-bat" option
- on the command line. On UNIX or Linux systems, the
- "--install-bat" option may be specified to have "scons-0.96.bat"
- and "scons.bat" files installed in the default system script
- directory, which is useful if you want to install SCons in a
- shared file system directory that can be used to execute SCons
- from both UNIX/Linux and Windows systems.
+ on the command line.
+
+ On UNIX or Linux systems, the "--install-bat" option may be
+ specified to have "scons-1.3.0.bat" and "scons.bat" files installed
+ in the default system script directory, which is useful if you
+ want to install SCons in a shared file system directory that can
+ be used to execute SCons from both UNIX/Linux and Windows systems.
-- Install the SCons build engine (a Python module) in an
appropriate version-numbered SCons library directory
- (/usr/lib/scons-0.96 or C:\Python*\scons-0.96, for example).
+ (/usr/lib/scons-1.3.0 or C:\Python*\scons-1.3.0, for example).
See below for more options related to installing the build
engine library.
library in $HOME/lib/scons, for example.
+MAKING CHANGES
+==============
+
+Because SCons is implemented in a scripting language, you don't need to
+build it in order to make changes and test them.
+
+Virtually all of the SCons functionality exists in the "build engine,"
+the src/engine/SCons subdirectory hierarchy that contains all of the
+modules that make up SCons. The src/script/scons.py wrapper script exists
+mainly to find the appropriate build engine library and then execute it.
+
+In order to make your own changes locally and test them by hand, simply
+edit modules in the local src/engine/SCons subdirectory tree and either
+use the local bootstrap.py script:
+
+ $ python bootstrap.py [arguments]
+
+Or set the SCONS_LIB_DIR to point to the src/engine/SCons directory and
+then execute the src/script/scons.py script. Here is one way you can
+set up environment variables to do this on a UNIX or Linux system:
+
+ $ setenv MYSCONS=`pwd`/src
+ $ setenv SCONS_LIB_DIR=$MYSCONS/engine
+ $ python $MYSCONS/script/scons.py [arguments]
+
+Or on Windows:
+
+ C:\scons>set MYSCONS=%cd%\src
+ C:\scons>set SCONS_LIB_DIR=%MYSCONS%\engine
+ C:\scons>python %MYSCONS%\script\scons.py [arguments]
+
+You can use the -C option to have SCons change directory to another
+location where you already have a build configuration set up (for example,
+if the SCons configuration for your project seems to be blocked by
+an SCons bug, and you want to see if a patch you make actually fixes
+that bug):
+
+ $ python bootstrap.py -C /some/other/location [arguments]
+
+Lastly, if you want to be able to just execute your modified version
+of SCons from the command line, you can make it executable and add its
+directory to your $PATH like so:
+
+ $ chmod 755 src/script/scons.py
+ $ export PATH=$PATH:`pwd`/src/script
+
+You should then be able to run this version of SCons by just typing
+"scons.py" at your UNIX or Linux command line.
+
+Note that the regular SCons development process makes heavy use of
+automated testing. See the TESTING and DEVELOPMENT WORKFLOW sections
+below for more information about the automated regression tests and how
+they can be used in a development cycle to validate that your changes
+don't break existing functionality.
+
+
+DEBUGGING
+=========
+
+Python comes with a good interactive debugger. When debugging changes
+by hand (i.e., when not using the automated tests), you can invoke SCons
+under control of the Python debugger by specifying the --debug=pdb option:
+
+ $ scons --debug=pdb [arguments]
+ > /home/knight/SCons/src/engine/SCons/Script/Main.py(927)_main()
+ -> default_warnings = [ SCons.Warnings.CorruptSConsignWarning,
+ (Pdb)
+
+Once in the debugger, you can set breakpoints at lines in files in the
+build engine modules by providing the path name of the file relative to
+the src/engine subdirectory (that is, including the SCons/ as the first
+directory component):
+
+ (Pdb) b SCons/Tool/msvc.py:158
+
+The debugger also supports single stepping, stepping into functions,
+printing variables, etc.
+
+Trying to debug problems found by running the automated tests (see the
+TESTING section, below) is more difficult, because the test automation
+harness re-invokes SCons and captures output. Consequently, there isn't an
+easy way to invoke the Python debugger in a useful way on any particular
+SCons call within a test script.
+
+The most effective technique for debugging problems that occur during an
+automated test is to use the good old tried-and-true technique of adding
+statements to print tracing information. But note that you can't just use
+"print" statement, or even "sys.stdout.write()," because those change the
+SCons output, and the automated tests usually look for matches of specific
+output strings to decide if a given SCons invocations passes the test.
+
+To deal with this, SCons supports a Trace() function that (by default)
+will print messages to your console screen ("/dev/tty" on UNIX or Linux,
+"con" on Windows). By adding Trace() calls to the SCons source code:
+
+ def sample_method(self, value):
+ from SCons.Debug import Trace
+ Trace('called sample_method(%s, %s)\n' % (self, value))
+
+You can then run automated tests that print any arbitrary information
+you wish about what's going on inside SCons, without interfering with
+the test automation.
+
+The Trace() function can also redirect its output to a file, rather than
+the screen:
+
+ def sample_method(self, value):
+ from SCons.Debug import Trace
+ Trace('called sample_method(%s, %s)\n' % (self, value),
+ file='trace.out')
+
+Where the Trace() function sends its output is stateful: once you use the
+"file=" argument, all subsequent calls to Trace() send their output to
+the same file, until another call with a "file=" argument is reached.
+
+
TESTING
=======
$ python runtest.py test/option-j.py test/Program.py
-Alternatively, the runtest.py script takes a -a option that searches
-the tree for all of the tests and runs them:
+You also use the -f option to execute just the tests listed in a specified
+text file:
+
+ $ cat testlist.txt
+ test/option-j.py
+ test/Program.py
+ $ python runtest.py -f testlist.txt
+
+One test must be listed per line, and any lines that begin with '#'
+will be ignored (allowing you, for example, to comment out tests that
+are currently passing and then uncomment all of the tests in the file
+for a final validation run).
+
+The runtest.py script also takes a -a option that searches the tree for
+all of the tests and runs them:
$ python runtest.py -a
PACKAGES" section below.
+DEVELOPMENT WORKFLOW
+====================
+
+ CAVEAT: The point of this section isn't to describe one dogmatic
+ workflow. Just running the test suite can be time-consuming, and
+ getting a patch to pass all of the tests can be more so. If you're
+ genuinely blocked, it may make more sense to submit a patch with
+ a note about which tests still fail, and how. Someone else may be
+ able to take your "initial draft" and figure out how to improve it
+ to fix the rest of the tests. So there's plenty of room for use of
+ good judgement.
+
+The various techniques described in the above sections can be combined
+to create simple and effective workflows that allow you to validate
+that patches you submit to SCons don't break existing functionality and
+have adequate testing, thereby increasing the speed with which they can
+be integrated.
+
+For example, suppose your project's SCons configuration is blocked by
+an SCons bug, and you decide you want to fix it and submit the patch.
+Here's one possible way to go about doing that (using UNIX/Linux as the
+development platform, Windows users can translate as appropriate)):
+
+ -- Change to the top of your checked-out SCons tree and set
+ $SCONS_LIB_DIR to point to its build engine:
+
+ $ setenv SCONS_LIB_DIR=`pwd`/src
+
+ -- Confirm that the bug still exists in this version of SCons
+ by using the -C option to run the broken build:
+
+ $ python script/scons.py -C /home/me/broken_project .
+
+ -- Fix the bug in SCons by editing appropriate module files
+ underneath src/engine/SCons.
+
+ -- Confirm that you've fixed the bug affecting your project:
+
+ $ python script/scons.py -C /home/me/broken_project .
+
+ -- Test to see if your fix had any unintended side effects
+ that break existing functionality:
+
+ $ python runtest.py -a
+
+ Be patient, there are more than 700 test scripts in the
+ whole suite.
+
+ If any test scripts fail, they will be listed in a summary at
+ the end of the run. Some test scripts may also report NO RESULT
+ because (for example) your local system is the wrong type or
+ doesn't have some installed utilities necessary to run the
+ script. In general, you can ignore the NO RESULT list.
+
+ -- Cut-and-paste the list of failed tests into a file:
+
+ $ cat > failed.txt
+ test/failed-test-1.py
+ test/failed-test-2.py
+ test/failed-test-3.py
+ ^D
+ $
+
+ -- Now debug the test failures and fix them, either by changing
+ SCons, or by making necessary changes to the tests (if, for
+ example, you have a strong reason to change functionality, or
+ if you find that the bug really is in the test script itself).
+ After each change, use the runtest.py -f option to examine the
+ effects of the change on the subset of tests that originally
+ failed:
+
+ $ [edit]
+ $ python runtest.py -f failed.txt
+
+ Repeat this until all of the tests that originally failed
+ now pass.
+
+ -- Now you need to go back and validate that any changes you
+ made while getting the tests to pass didn't break the fix
+ you originally put in, and didn't introduce any *additional*
+ unintended side effects that broke other tests:
+
+ $ python script/scons.py -C /home/me/broken_project .
+ $ python runtest.py -a
+
+ If you find any newly-broken tests, add them to your "failed.txt"
+ file and go back to the previous step.
+
+Of course, the above is only one suggested workflow. In practice, there
+is a lot of room for judgment and experience to make things go quicker.
+For example, if you're making a change to just the Java support, you
+might start looking for regressions by just running the test/Java/*.py
+tests instead of running all of "runtest.py -a".
+
+
BUILDING PACKAGES
=================
-We use SCons (version 0.96 or later) to build its own packages. If you
+We use SCons (version 0.96.93 later) to build its own packages. If you
already have an appropriate version of SCons installed on your system,
you can build everything by simply running it:
$ scons
-If you don't have SCons version 0.96 or later already installed on your
-system, you can build this version of SCons with itself with a little
-more typing:
+If you don't have SCons version 0.96.93 later already installed on your
+system, you can use the supplied bootstrap.py script:
+
+ $ python bootstrap.py build/scons
- $ SCONS_LIB_DIR=`pwd`/src/engine python src/script/scons.py
+The bootstrap.py keeps the src/ subdirectory free of compiled Python
+(*.pyc or *.pyo) files by copying the necessary SCons files to a local
+bootstrap/ subdirectory and executing it from there.
+
+You can also build this version of SCons by hand with a little more
+typing. On UNIX or Linux (using sh or a derivative like bash or ksh):
+
+ $ export SCONS_LIB_DIR=`pwd`/src/engine
+ $ python src/script/scons.py
+
+On Windows:
+
+ C:\scons\>set SCONS_LIB_DIR=%cd%\src\engine
+ C:\scons\>python src/script/scons.py
Depending on the utilities installed on your system, any or all of the
following packages will be built:
- build/dist/scons-0.96-1.noarch.rpm
- build/dist/scons-0.96-1.src.rpm
- build/dist/scons-0.96.linux-i686.tar.gz
- build/dist/scons-0.96.tar.gz
- build/dist/scons-0.96.win32.exe
- build/dist/scons-0.96.zip
- build/dist/scons-doc-0.96.tar.gz
- build/dist/scons-local-0.96.tar.gz
- build/dist/scons-local-0.96.zip
- build/dist/scons-src-0.96.tar.gz
- build/dist/scons-src-0.96.zip
- build/dist/scons_0.96-1_all.deb
+ build/dist/scons-1.3.0-1.noarch.rpm
+ build/dist/scons-1.3.0-1.src.rpm
+ build/dist/scons-1.3.0.linux-i686.tar.gz
+ build/dist/scons-1.3.0.tar.gz
+ build/dist/scons-1.3.0.win32.exe
+ build/dist/scons-1.3.0.zip
+ build/dist/scons-doc-1.3.0.tar.gz
+ build/dist/scons-local-1.3.0.tar.gz
+ build/dist/scons-local-1.3.0.zip
+ build/dist/scons-src-1.3.0.tar.gz
+ build/dist/scons-src-1.3.0.zip
+ build/dist/scons_1.3.0-1_all.deb
The SConstruct file is supposed to be smart enough to avoid trying to
build packages for which you don't have the proper utilities installed.
Not guaranteed to be up-to-date (but better than nothing):
+bench/
+ A subdirectory for benchmarking scripts, used to perform timing
+ tests to decide what specific idioms are most efficient for
+ various parts of the code base. We check these in so they're
+ available in case we have to revisit any of these decisions in
+ the future.
+
bin/
Miscellaneous utilities used in SCons development. Right now,
some of the stuff here includes:
of lines in each
-- a script for synchronizing the Aegis tree to SourceForge
-- a prototype script for capturing sample SCons output
- in sgml files
+ in xml files
-- a script that can profile and time a packaging build of
SCons itself
-- a copy of xml_export, which can retrieve project data
from SourceForge
+ -- scripts and a Python module for translating the SCons
+ home-brew XML documentation tags into DocBook and
+ man page format
bootstrap.py
A build script for use with Aegis. This collects a current copy
SCons documentation. A variety of things here, in various
stages of (in)completeness.
-etc/
- A subdirectory for miscellaneous things that we need. Right
- now, it has copies of Python modules that we use for testing,
- and which we don't want to force people to have to install on
- their own just to help out with SCons development.
-
gentoo/
Stuff to generate files for Gentoo Linux.
the licensing terms are for SCons itself, not any other
package that includes SCons.
+QMTest/
+ The Python modules we use for testing, some generic modules
+ originating elsewhere and some specific to SCons.
+
README
What you're looking at right now.
REPORTING BUGS
==============
-Please report bugs by following the "Tracker - Bugs" link on the SCons
-project page and filling out the form:
+Please report bugs by following the detailed instructions on our Bug
+Submission page:
- http://sourceforge.net/projects/scons/
+ http://scons.tigris.org/bug-submission.html
-You can also send mail to the SCons developers mailing list:
+You can also send mail to the SCons developers' mailing list:
- scons-devel@lists.sourceforge.net
+ dev@scons.tigris.org
-But please make sure that you also submit a bug report to the project
-page bug tracker, because bug reports in email can sometimes get lost
-in the general flood of messages.
+But even if you send email to the mailing list please make sure that you
+ALSO submit a bug report to the project page bug tracker, because bug
+reports in email often get overlooked in the general flood of messages.
MAILING LISTS
With plenty of help from the SCons Development team:
Chad Austin
Charles Crain
+ Bill Deegan
Steve Leblanc
+ Greg Noel
Gary Oberbrunner
Anthony Roach
Greg Spencer
projects / scons.git / blobdiff