-# Copyright (c) 2001, 2002 Steven Knight
+# __COPYRIGHT__
SCons - a software construction tool
-Welcome to the SCons development tree. The real purpose of this tree is
-to package SCons for production distribution in a variety of formats,
+Welcome to the SCons development tree. The real purpose of this tree
+is to package SCons for production distribution in a variety of formats,
not just to hack SCons code.
-To that extent, the normal development cycle (enforced by Aegis) is not
-to test the code directly, but to package SCons, unpack the package,
-"install" SCons in a test subdirectory, and then to run the tests
-against the unpacked and installed software. This helps eliminate
-problems caused by, for example, failure to update the list of files to
-be packaged.
+If all you want to do is install and run SCons, it will be easier for you
+to download and install the scons-{version}.tar.gz or scons-{version}.zip
+package rather than to work with the packaging logic in this tree.
-Note that if all you want to do is install and run SCons, it
-will probably be easier for you to download and install the
-scons-{version}.tar.gz or scons-{version}.zip package rather than to
-work with the packaging logic in this tree.
+To the extent that this tree is about building SCons packages, the *full*
+development cycle is not just to test the code directly, but to package
+SCons, unpack the package, "install" SCons in a test subdirectory,
+and then to run the tests against the unpacked and installed software.
+This helps eliminate problems caused by, for example, failure to update
+the list of files to be packaged.
+
+For just working on making an individual change to the SCons source,
+however, you don't actually need to build or install SCons; you
+*can* actually edit and execute SCons in-place. See the following
+sections below for more information:
+
+ MAKING CHANGES
+ How to edit and execute SCons in-place.
+
+ DEBUGGING
+ Tips for debugging problems in SCons.
+
+ TESTING
+ How to use the automated regression tests.
+
+ DEVELOPMENT WORKFLOW
+ An example of how to put the edit/execute/test pieces
+ together in a reasonable development workflow.
LATEST VERSION
You may, of course, override these default values by appropriate
configuration of Environment construction variables.
+By default, SCons knows how to search for available programming tools
+on various systems--see the SCons man page for details. You may,
+of course, override the default SCons choices made by appropriate
+configuration of Environment construction variables.
+
INSTALLATION REQUIREMENTS
=========================
Python 1.5.2, you may not have distutils installed. If you are running
Python version 1.6 or later, you should be fine.
-NOTE TO RED HAT USERS: All Red Hat Linux versions as late as 7.3 (at
-least) still ship Python 1.5.2 as the default, so you probably do *not*
-have distutils installed, unless you have already done so manually.
+NOTE TO RED HAT USERS: Red Hat shipped Python 1.5.2 as the default all
+the way up to Red Hat Linux 7.3, so you probably do *not* have distutils
+installed, unless you have already done so manually or are running Red
+Hat 8.0 or later.
In this case, your options are:
-- (Optional.) Install from a pre-packaged SCons package that
does not require distutils:
- Red Hat Linux scons-0.09-1.noarch.rpm
+ Red Hat Linux scons-1.3.0.noarch.rpm
- Debian GNU/Linux scons_0.09-1_all.deb
- (or use apt-get)
+ Debian GNU/Linux use apt-get to get the official package
- Windows scons-0.09.win32.exe
+ Windows scons-1.3.0.win32.exe
-- (Recommended.) Download the latest distutils package from the
following URL:
INSTALLATION
============
-Assuming your system satisfies the installation requirements in
-the previous section, install SCons from this package by first
-populating the build/scons/ directory. (For an easier way to
-install SCons, without having to populate this directory, use the
-scons-{version}.tar.gz or scons-{version}.zip package.)
+ NOTE: You don't need to build SCons packages or install SCons if
+ you just want to work on developing a patch. See the sections
+ about MAKING CHANGES and TESTING below if you just want to submit
+ a bug fix or some new functionality. See the sections below about
+ BUILDING PACKAGES and TESTING PACKAGES if your enhancement involves
+ changing the way in which SCons is packaged and/or installed on an
+ end-user system.
+
+Assuming your system satisfies the installation requirements in the
+previous section, install SCons from this package by first populating
+the build/scons/ subdirectory. (For an easier way to install SCons,
+without having to populate this directory, use the scons-{version}.tar.gz
+or scons-{version}.zip package.)
If you already have an appropriate version of SCons installed on your
system, populate the build/scons/ directory by running:
$ scons build/scons
-If you don't have SCons version 0.08 or later already installed on your
-system, you can use SCons itself to populate the build/scons/ directory
-with a little more typing:
+You can also use this version of SCons to populate its own build directory
+by using a supplied bootstrap.py script:
+
+ $ python bootstrap.py build/scons
+
+The bootstrap.py keeps the src/ subdirectory free of compiled Python
+(*.pyc or *.pyo) files by copying the necessary SCons files to a local
+bootstrap/ subdirectory and executing it from there.
+
+You can also execute the local SCons directly from the src/ subdirectory
+by first setting the SCONS_LIB_DIR environment variable to the local
+src/engine subdirectory, and then executing the local src/script/scons.py
+script to populate the build/scons/ subdirectory. You would do this as
+follows on a Linux or UNIX system (using sh or a derivative like bash
+or ksh):
+
+ $ export SCONS_LIB_DIR=`pwd`/src/engine
+ $ python src/script/scons.py build/scons
- $ SCONS_LIB_DIR=`pwd`/src/engine python src/script/scons.py build/scons
+Or as follows on Windows:
+
+ C:\scons\>set SCONS_LIB_DIR=%cd%\src\engine
+ C:\scons\>python src\script\scons.py build/scons
-Either command will populate the build/scons/ directory with the
-necessary files and directory structure to use the Python-standard setup
-script as follows:
+Any of the above commands will populate the build/scons/ directory with
+the necessary files and directory structure to use the Python-standard
+setup script as follows on Linux or UNIX:
# cd build/scons
# python setup.py install
-This will install the scons script in the default system script
-directory (/usr/bin or C:\Python*\Scripts, for example) and the build
-engine in an appropriate SCons library directory (/usr/lib/scons or
-C:\Python*\SCons, for example).
+And on Windows:
+
+ C:\scons\>cd build\scons
+ C:\scons\build\scons>python setup.py install
+
+By default, the above commands will do the following:
+
+ -- Install the version-numbered "scons-1.3.0" and "sconsign-1.3.0"
+ scripts in the default system script directory (/usr/bin or
+ C:\Python*\Scripts, for example). This can be disabled by
+ specifying the "--no-version-script" option on the command
+ line.
+
+ -- Install scripts named "scons" and "sconsign" scripts in the
+ default system script directory (/usr/bin or C:\Python*\Scripts,
+ for example). This can be disabled by specifying the
+ "--no-scons-script" option on the command line, which is useful
+ if you want to install and experiment with a new version before
+ making it the default on your system.
+
+ On UNIX or Linux systems, you can have the "scons" and "sconsign"
+ scripts be hard links or symbolic links to the "scons-1.3.0" and
+ "sconsign-1.3.0" scripts by specifying the "--hardlink-scons" or
+ "--symlink-scons" options on the command line.
+
+ -- Install "scons-1.3.0.bat" and "scons.bat" wrapper scripts in the
+ Python prefix directory on Windows (C:\Python*, for example).
+ This can be disabled by specifying the "--no-install-bat" option
+ on the command line.
+
+ On UNIX or Linux systems, the "--install-bat" option may be
+ specified to have "scons-1.3.0.bat" and "scons.bat" files installed
+ in the default system script directory, which is useful if you
+ want to install SCons in a shared file system directory that can
+ be used to execute SCons from both UNIX/Linux and Windows systems.
+
+ -- Install the SCons build engine (a Python module) in an
+ appropriate version-numbered SCons library directory
+ (/usr/lib/scons-1.3.0 or C:\Python*\scons-1.3.0, for example).
+ See below for more options related to installing the build
+ engine library.
+
+ -- Install the troff-format man pages in an appropriate directory
+ on UNIX or Linux systems (/usr/share/man/man1 or /usr/man/man1,
+ for example). This can be disabled by specifying the
+ "--no-install-man" option on the command line. The man pages
+ can be installed on Windows systems by specifying the
+ "--install-man" option on the command line.
+
+Note that, by default, SCons does not install its build engine library
+in the standard Python library directories. If you want to be able to
+use the SCons library modules (the build engine) in other Python
+scripts, specify the "--standard-lib" option on the command line, as
+follows:
+
+ # python setup.py install --standard-lib
+
+This will install the build engine in the standard Python library
+directory (/usr/lib/python*/site-packages or
+C:\Python*\Lib\site-packages).
+
+Alternatively, you can have SCons install its build engine library in a
+hard-coded standalone library directory, instead of the default
+version-numbered directory, by specifying the "--standalone-lib" option
+on the command line, as follows:
+
+ # python setup.py install --standalone-lib
+
+This is usually not recommended, however.
+
+Note that, to install SCons in any of the above system directories,
+you should have system installation privileges (that is, "root" or
+"Administrator") when running the setup.py script. If you don't have
+system installation privileges, you can use the --prefix option to
+specify an alternate installation location, such as your home directory:
-You should have system installation privileges (that is, "root" on POSIX
-or "Administrator" on Windows) when running the setup.py script to
-install SCons in the default system directories.
+ $ python setup.py install --prefix=$HOME
-If you don't have system installation privileges, you can use the
---prefix option to specify an alternate installation location, such as
-your home directory:
+This will install SCons in the appropriate locations relative to
+$HOME--that is, the scons script itself $HOME/bin and the associated
+library in $HOME/lib/scons, for example.
- $ cd build/scons
- $ python setup.py install --prefix=$HOME
-This will install the scons script itself in $HOME/bin and the
-associated library in $HOME/lib/scons
+MAKING CHANGES
+==============
+
+Because SCons is implemented in a scripting language, you don't need to
+build it in order to make changes and test them.
+
+Virtually all of the SCons functionality exists in the "build engine,"
+the src/engine/SCons subdirectory hierarchy that contains all of the
+modules that make up SCons. The src/script/scons.py wrapper script exists
+mainly to find the appropriate build engine library and then execute it.
+
+In order to make your own changes locally and test them by hand, simply
+edit modules in the local src/engine/SCons subdirectory tree and either
+use the local bootstrap.py script:
+
+ $ python bootstrap.py [arguments]
+
+Or set the SCONS_LIB_DIR to point to the src/engine/SCons directory and
+then execute the src/script/scons.py script. Here is one way you can
+set up environment variables to do this on a UNIX or Linux system:
+
+ $ setenv MYSCONS=`pwd`/src
+ $ setenv SCONS_LIB_DIR=$MYSCONS/engine
+ $ python $MYSCONS/script/scons.py [arguments]
+
+Or on Windows:
+
+ C:\scons>set MYSCONS=%cd%\src
+ C:\scons>set SCONS_LIB_DIR=%MYSCONS%\engine
+ C:\scons>python %MYSCONS%\script\scons.py [arguments]
+
+You can use the -C option to have SCons change directory to another
+location where you already have a build configuration set up (for example,
+if the SCons configuration for your project seems to be blocked by
+an SCons bug, and you want to see if a patch you make actually fixes
+that bug):
+
+ $ python bootstrap.py -C /some/other/location [arguments]
+
+Lastly, if you want to be able to just execute your modified version
+of SCons from the command line, you can make it executable and add its
+directory to your $PATH like so:
+
+ $ chmod 755 src/script/scons.py
+ $ export PATH=$PATH:`pwd`/src/script
+
+You should then be able to run this version of SCons by just typing
+"scons.py" at your UNIX or Linux command line.
+
+Note that the regular SCons development process makes heavy use of
+automated testing. See the TESTING and DEVELOPMENT WORKFLOW sections
+below for more information about the automated regression tests and how
+they can be used in a development cycle to validate that your changes
+don't break existing functionality.
+
+
+DEBUGGING
+=========
+
+Python comes with a good interactive debugger. When debugging changes
+by hand (i.e., when not using the automated tests), you can invoke SCons
+under control of the Python debugger by specifying the --debug=pdb option:
+
+ $ scons --debug=pdb [arguments]
+ > /home/knight/SCons/src/engine/SCons/Script/Main.py(927)_main()
+ -> default_warnings = [ SCons.Warnings.CorruptSConsignWarning,
+ (Pdb)
+
+Once in the debugger, you can set breakpoints at lines in files in the
+build engine modules by providing the path name of the file relative to
+the src/engine subdirectory (that is, including the SCons/ as the first
+directory component):
+
+ (Pdb) b SCons/Tool/msvc.py:158
+
+The debugger also supports single stepping, stepping into functions,
+printing variables, etc.
+
+Trying to debug problems found by running the automated tests (see the
+TESTING section, below) is more difficult, because the test automation
+harness re-invokes SCons and captures output. Consequently, there isn't an
+easy way to invoke the Python debugger in a useful way on any particular
+SCons call within a test script.
+
+The most effective technique for debugging problems that occur during an
+automated test is to use the good old tried-and-true technique of adding
+statements to print tracing information. But note that you can't just use
+"print" statement, or even "sys.stdout.write()," because those change the
+SCons output, and the automated tests usually look for matches of specific
+output strings to decide if a given SCons invocations passes the test.
+
+To deal with this, SCons supports a Trace() function that (by default)
+will print messages to your console screen ("/dev/tty" on UNIX or Linux,
+"con" on Windows). By adding Trace() calls to the SCons source code:
+
+ def sample_method(self, value):
+ from SCons.Debug import Trace
+ Trace('called sample_method(%s, %s)\n' % (self, value))
+
+You can then run automated tests that print any arbitrary information
+you wish about what's going on inside SCons, without interfering with
+the test automation.
+
+The Trace() function can also redirect its output to a file, rather than
+the screen:
+
+ def sample_method(self, value):
+ from SCons.Debug import Trace
+ Trace('called sample_method(%s, %s)\n' % (self, value),
+ file='trace.out')
+
+Where the Trace() function sends its output is stateful: once you use the
+"file=" argument, all subsequent calls to Trace() send their output to
+the same file, until another call with a "file=" argument is reached.
TESTING
$ python runtest.py test/option-j.py test/Program.py
-Alternatively, the runtest.py script takes a -a option that searches
-the tree for all of the tests and runs them:
+You also use the -f option to execute just the tests listed in a specified
+text file:
+
+ $ cat testlist.txt
+ test/option-j.py
+ test/Program.py
+ $ python runtest.py -f testlist.txt
+
+One test must be listed per line, and any lines that begin with '#'
+will be ignored (allowing you, for example, to comment out tests that
+are currently passing and then uncomment all of the tests in the file
+for a final validation run).
+
+The runtest.py script also takes a -a option that searches the tree for
+all of the tests and runs them:
$ python runtest.py -a
PACKAGES" section below.
+DEVELOPMENT WORKFLOW
+====================
+
+ CAVEAT: The point of this section isn't to describe one dogmatic
+ workflow. Just running the test suite can be time-consuming, and
+ getting a patch to pass all of the tests can be more so. If you're
+ genuinely blocked, it may make more sense to submit a patch with
+ a note about which tests still fail, and how. Someone else may be
+ able to take your "initial draft" and figure out how to improve it
+ to fix the rest of the tests. So there's plenty of room for use of
+ good judgement.
+
+The various techniques described in the above sections can be combined
+to create simple and effective workflows that allow you to validate
+that patches you submit to SCons don't break existing functionality and
+have adequate testing, thereby increasing the speed with which they can
+be integrated.
+
+For example, suppose your project's SCons configuration is blocked by
+an SCons bug, and you decide you want to fix it and submit the patch.
+Here's one possible way to go about doing that (using UNIX/Linux as the
+development platform, Windows users can translate as appropriate)):
+
+ -- Change to the top of your checked-out SCons tree and set
+ $SCONS_LIB_DIR to point to its build engine:
+
+ $ setenv SCONS_LIB_DIR=`pwd`/src
+
+ -- Confirm that the bug still exists in this version of SCons
+ by using the -C option to run the broken build:
+
+ $ python script/scons.py -C /home/me/broken_project .
+
+ -- Fix the bug in SCons by editing appropriate module files
+ underneath src/engine/SCons.
+
+ -- Confirm that you've fixed the bug affecting your project:
+
+ $ python script/scons.py -C /home/me/broken_project .
+
+ -- Test to see if your fix had any unintended side effects
+ that break existing functionality:
+
+ $ python runtest.py -a
+
+ Be patient, there are more than 700 test scripts in the
+ whole suite.
+
+ If any test scripts fail, they will be listed in a summary at
+ the end of the run. Some test scripts may also report NO RESULT
+ because (for example) your local system is the wrong type or
+ doesn't have some installed utilities necessary to run the
+ script. In general, you can ignore the NO RESULT list.
+
+ -- Cut-and-paste the list of failed tests into a file:
+
+ $ cat > failed.txt
+ test/failed-test-1.py
+ test/failed-test-2.py
+ test/failed-test-3.py
+ ^D
+ $
+
+ -- Now debug the test failures and fix them, either by changing
+ SCons, or by making necessary changes to the tests (if, for
+ example, you have a strong reason to change functionality, or
+ if you find that the bug really is in the test script itself).
+ After each change, use the runtest.py -f option to examine the
+ effects of the change on the subset of tests that originally
+ failed:
+
+ $ [edit]
+ $ python runtest.py -f failed.txt
+
+ Repeat this until all of the tests that originally failed
+ now pass.
+
+ -- Now you need to go back and validate that any changes you
+ made while getting the tests to pass didn't break the fix
+ you originally put in, and didn't introduce any *additional*
+ unintended side effects that broke other tests:
+
+ $ python script/scons.py -C /home/me/broken_project .
+ $ python runtest.py -a
+
+ If you find any newly-broken tests, add them to your "failed.txt"
+ file and go back to the previous step.
+
+Of course, the above is only one suggested workflow. In practice, there
+is a lot of room for judgment and experience to make things go quicker.
+For example, if you're making a change to just the Java support, you
+might start looking for regressions by just running the test/Java/*.py
+tests instead of running all of "runtest.py -a".
+
+
BUILDING PACKAGES
=================
-We now use SCons (version 0.08 or later) to build its own packages.
-If you already have an appropriate version of SCons installed on your
-system, you can build everything by simply running it:
+We use SCons (version 0.96.93 later) to build its own packages. If you
+already have an appropriate version of SCons installed on your system,
+you can build everything by simply running it:
$ scons
-If you don't have SCons version 0.08 or later already installed on your
-system, you can build this version of SCons with itself with a little
-more typing:
+If you don't have SCons version 0.96.93 later already installed on your
+system, you can use the supplied bootstrap.py script:
+
+ $ python bootstrap.py build/scons
+
+The bootstrap.py keeps the src/ subdirectory free of compiled Python
+(*.pyc or *.pyo) files by copying the necessary SCons files to a local
+bootstrap/ subdirectory and executing it from there.
+
+You can also build this version of SCons by hand with a little more
+typing. On UNIX or Linux (using sh or a derivative like bash or ksh):
+
+ $ export SCONS_LIB_DIR=`pwd`/src/engine
+ $ python src/script/scons.py
- $ SCONS_LIB_DIR=`pwd`/src/engine python src/script/scons.py
+On Windows:
+
+ C:\scons\>set SCONS_LIB_DIR=%cd%\src\engine
+ C:\scons\>python src/script/scons.py
Depending on the utilities installed on your system, any or all of the
following packages will be built:
- build/dist/scons-0.09-1.noarch.rpm
- build/dist/scons-0.09-1.src.rpm
- build/dist/scons-0.09.linux-i686.tar.gz
- build/dist/scons-0.09.tar.gz
- build/dist/scons-0.09.win32.exe
- build/dist/scons-0.09.zip
- build/dist/scons-doc-0.09.tar.gz
- build/dist/scons-local-0.09.tar.gz
- build/dist/scons-local-0.09.zip
- build/dist/scons-src-0.09.tar.gz
- build/dist/scons-src-0.09.zip
- build/dist/scons_0.09-1_all.deb
+ build/dist/scons-1.3.0-1.noarch.rpm
+ build/dist/scons-1.3.0-1.src.rpm
+ build/dist/scons-1.3.0.linux-i686.tar.gz
+ build/dist/scons-1.3.0.tar.gz
+ build/dist/scons-1.3.0.win32.exe
+ build/dist/scons-1.3.0.zip
+ build/dist/scons-doc-1.3.0.tar.gz
+ build/dist/scons-local-1.3.0.tar.gz
+ build/dist/scons-local-1.3.0.zip
+ build/dist/scons-src-1.3.0.tar.gz
+ build/dist/scons-src-1.3.0.zip
+ build/dist/scons_1.3.0-1_all.deb
The SConstruct file is supposed to be smart enough to avoid trying to
build packages for which you don't have the proper utilities installed.
Not guaranteed to be up-to-date (but better than nothing):
+bench/
+ A subdirectory for benchmarking scripts, used to perform timing
+ tests to decide what specific idioms are most efficient for
+ various parts of the code base. We check these in so they're
+ available in case we have to revisit any of these decisions in
+ the future.
+
bin/
Miscellaneous utilities used in SCons development. Right now,
- there's a copy of the script we use to translate an Aegis change
- into a CVS checkin.
+ some of the stuff here includes:
+
+ -- a copy of the script we use to translate an Aegis change
+ into a CVS checkin
+ -- a script that runs pychecker on our source tree
+ -- a script that counts source and test files and numbers
+ of lines in each
+ -- a script for synchronizing the Aegis tree to SourceForge
+ -- a prototype script for capturing sample SCons output
+ in xml files
+ -- a script that can profile and time a packaging build of
+ SCons itself
+ -- a copy of xml_export, which can retrieve project data
+ from SourceForge
+ -- scripts and a Python module for translating the SCons
+ home-brew XML documentation tags into DocBook and
+ man page format
+
+bootstrap.py
+ A build script for use with Aegis. This collects a current copy
+ of SCons from the Aegis baseline directories in a bootstrap/
+ subdirectory, and then executes SCons with the supplied
+ command-line arguments.
build/
This doesn't exist yet if you're looking at a vanilla source
SCons documentation. A variety of things here, in various
stages of (in)completeness.
-etc/
- A subdirectory for miscellaneous things that we need. Right
- now, it has copies of Python modules that we use for testing,
- and which we don't want to force people to have to install on
- their own just to help out with SCons development.
+gentoo/
+ Stuff to generate files for Gentoo Linux.
HOWTO/
Documentation of SCons administrative procedures (making a
the licensing terms are for SCons itself, not any other
package that includes SCons.
+QMTest/
+ The Python modules we use for testing, some generic modules
+ originating elsewhere and some specific to SCons.
+
README
What you're looking at right now.
REPORTING BUGS
==============
-You can report bugs either by following the "Tracker - Bugs" link
-on the SCons project page:
+Please report bugs by following the detailed instructions on our Bug
+Submission page:
+
+ http://scons.tigris.org/bug-submission.html
- http://sourceforge.net/projects/scons/
+You can also send mail to the SCons developers' mailing list:
-or by sending mail to the SCons developers mailing list:
+ dev@scons.tigris.org
- scons-devel@lists.sourceforge.net
+But even if you send email to the mailing list please make sure that you
+ALSO submit a bug report to the project page bug tracker, because bug
+reports in email often get overlooked in the general flood of messages.
MAILING LISTS
=============
-A mailing list for developers of SCons is available. You may send
-questions or comments to the list at:
+An active mailing list for developers of SCons is available. You may
+send questions or comments to the list at:
- scons-devel@lists.sourceforge.net
+ dev@scons.tigris.org
-You may request a subscription to the scons-devel mailing list at:
+You may request a subscription to the developer's mailing list by sending
+email to:
- http://lists.sourceforge.net/lists/listinfo/scons-devel
+ dev-subscribe@scons.tigris.org
-Subscription to the list is by approval. In practice, no one is being
-refused list membership right now, but we reserve the right to limit
+Subscription to the developer's mailing list is by approval. In practice,
+no one is refused list membership, but we reserve the right to limit
membership in the future and/or weed out lurkers.
+There is also a low-volume mailing list available for announcements
+about SCons. Subscribe by sending email to:
+
+ announce-subscribe@scons.tigris.org
+
+There are other mailing lists available for SCons users, for notification
+of SCons code changes, and for notification of updated bug reports and
+project documents. Please see our mailing lists page for details.
+
+
+DONATIONS
+=========
+
+If you find SCons helpful, please consider making a donation (of cash,
+software, or hardware) to support continued work on the project.
+Information is available at:
+
+ http://www.scons.org/donate.html
+
FOR MORE INFORMATION
====================
With plenty of help from the SCons Development team:
Chad Austin
Charles Crain
+ Bill Deegan
Steve Leblanc
+ Greg Noel
+ Gary Oberbrunner
Anthony Roach
- Terrel Shumway
+ Greg Spencer
+ Christoph Wiedemann