From: stevenknight Date: Tue, 25 Jul 2006 02:30:45 +0000 (+0000) Subject: Merged revisions 1441-1539 via svnmerge from X-Git-Url: http://git.tremily.us/?a=commitdiff_plain;h=bba939c88f349473b755ad9e981adb58e373b3ad;p=scons.git Merged revisions 1441-1539 via svnmerge from http://scons.tigris.org/svn/scons/branches/core ........ r1441 | stevenknight | 2006-04-22 23:06:53 -0400 (Sat, 22 Apr 2006) | 1 line 0.96.D397 - The scons command, branch 0.96.91. ........ r1442 | stevenknight | 2006-04-27 00:45:12 -0400 (Thu, 27 Apr 2006) | 1 line 0.96.D398 - The scons command, branch 0.96.92. ........ r1443 | stevenknight | 2006-04-27 00:49:25 -0400 (Thu, 27 Apr 2006) | 1 line 0.96.D399 - Taskmaster clean-ups in anticipation of refactoring speedups. ........ r1450 | stevenknight | 2006-05-02 00:04:55 -0400 (Tue, 02 May 2006) | 1 line 0.96.D400 - Fix VC+++ 2005 Express detection. (Atul Varma) Fix parsing Intel C compiler Li ........ r1451 | stevenknight | 2006-05-02 01:14:24 -0400 (Tue, 02 May 2006) | 1 line 0.96.D401 - Enhance ParseConfig() to understand -arch and -isysroot options. (Gary Oberbrun ........ r1458 | stevenknight | 2006-05-02 23:21:04 -0400 (Tue, 02 May 2006) | 1 line 0.96.D402 - Make strfunction handling consistent. (David Gruener) ........ r1459 | stevenknight | 2006-05-02 23:37:08 -0400 (Tue, 02 May 2006) | 1 line 0.96.D403 - Comment out the test of CVS checkout from the old tigris.org repository. ........ r1460 | stevenknight | 2006-05-03 23:47:54 -0400 (Wed, 03 May 2006) | 1 line 0.96.D404 - Preserve white space in display Action string. (David Gruener) ........ r1461 | stevenknight | 2006-05-04 09:16:15 -0400 (Thu, 04 May 2006) | 1 line 0.96.D405 - Add MergeFlags() and AddFlags() methods. (Greg Noel) Support recognizing compi ........ r1462 | stevenknight | 2006-05-04 23:46:53 -0400 (Thu, 04 May 2006) | 1 line 0.96.D406 - Fix stack trace when ParseFlags has a null string. ........ r1464 | stevenknight | 2006-05-05 17:21:27 -0400 (Fri, 05 May 2006) | 1 line 0.96.D408 - Fix the string displayed by InstallAs() when called through the default construc ........ r1465 | stevenknight | 2006-05-05 18:30:28 -0400 (Fri, 05 May 2006) | 1 line 0.96.D409 - Fix test/ParseConfig.py, broken in the previous checkin by ParseFlags() changes. ........ r1466 | stevenknight | 2006-05-05 20:42:35 -0400 (Fri, 05 May 2006) | 1 line 0.96.D407 - Avoid recursive calls to main() in SConf test programs. (Karol Pietrzak) ........ r1467 | stevenknight | 2006-05-06 00:27:21 -0400 (Sat, 06 May 2006) | 1 line 0.96.D410 - Catch errors from commands that ParseConfig() calls. (John Pye) ........ r1468 | stevenknight | 2006-05-06 10:55:38 -0400 (Sat, 06 May 2006) | 1 line 0.96.D411 - Significant taskmaster speedup by using reference counts, not list manipulation. ........ r1469 | stevenknight | 2006-05-06 18:38:02 -0400 (Sat, 06 May 2006) | 1 line 0.96.D413 - TeX improvements. ........ r1471 | stevenknight | 2006-05-07 09:07:58 -0400 (Sun, 07 May 2006) | 2 lines Delete properties interfering with clean .jpg checkout. ........ r1472 | stevenknight | 2006-05-07 09:23:54 -0400 (Sun, 07 May 2006) | 1 line 0.96.D412 - Windows portability fixes for two tests and ParseConfig() execution. ........ r1473 | stevenknight | 2006-05-07 09:30:11 -0400 (Sun, 07 May 2006) | 1 line 0.96.D414 - Various man page and documentation updates. ........ r1474 | stevenknight | 2006-05-07 23:53:12 -0400 (Sun, 07 May 2006) | 1 line 0.96.D415 - Initial infrastructure for executing tests under QMTest. (Stefan Seefeld) ........ r1476 | stevenknight | 2006-05-09 00:03:47 -0400 (Tue, 09 May 2006) | 1 line 0.96.D416 - Fix QMTest infrastructure to avoid listing directories with no tests and to find ........ r1477 | stevenknight | 2006-05-16 06:47:51 -0400 (Tue, 16 May 2006) | 1 line 0.96.D417 - Fix Alias turning Entries into Nodes or Dirs too soon. ........ r1478 | stevenknight | 2006-05-17 08:32:58 -0400 (Wed, 17 May 2006) | 1 line 0.96.D418 - Next QMTest changes (including fixing copyrights). ........ r1479 | stevenknight | 2006-05-18 05:07:06 -0400 (Thu, 18 May 2006) | 1 line 0.96.D419 - Fix DVIPDF tests after recent changes. ........ r1497 | stevenknight | 2006-05-23 08:47:01 -0400 (Tue, 23 May 2006) | 1 line 0.96.D420 - Better error message when trying to build a file from an unknown sufix. (Gary O ........ r1498 | stevenknight | 2006-05-23 09:38:52 -0400 (Tue, 23 May 2006) | 1 line 0.96.D421 - Suppress duplicate entries in latest TeX patch. (Joel B. Mohler) ........ r1499 | stevenknight | 2006-05-23 22:00:06 -0400 (Tue, 23 May 2006) | 1 line 0.96.D422 - Add tests for tuple variable expansion. (Gary Oberbrunner) ........ r1515 | stevenknight | 2006-06-12 06:44:24 -0400 (Mon, 12 Jun 2006) | 1 line 0.96.D423 - More QMTest work: start giving runtest.py its own tests, more functionality for ........ r1517 | stevenknight | 2006-06-21 07:34:30 -0400 (Wed, 21 Jun 2006) | 1 line 0.96.D424 - Move test/Configure.py and test/Options.py to avoid confusion with similarly-nam ........ r1518 | stevenknight | 2006-06-21 12:40:37 -0400 (Wed, 21 Jun 2006) | 1 line 0.96.D425 - Change the QMTest infrastructure to use File naming, not Python. Rename tests w ........ r1533 | stevenknight | 2006-07-23 20:10:08 -0400 (Sun, 23 Jul 2006) | 1 line 0.96.D426 - Fix ramifications of changing when Node disambiguation happens. ........ r1535 | stevenknight | 2006-07-24 06:40:43 -0400 (Mon, 24 Jul 2006) | 3 lines Initialized merge tracking via "svnmerge" with revisions "1-1534" from http://scons.tigris.org/svn/scons/trunk ........ r1536 | stevenknight | 2006-07-24 21:45:40 -0400 (Mon, 24 Jul 2006) | 2 lines Remove svnmerge-integrated property to start over. ........ r1538 | stevenknight | 2006-07-24 21:51:32 -0400 (Mon, 24 Jul 2006) | 3 lines Initialized merge tracking via "svnmerge" with revisions "1-1440" from http://scons.tigris.org/svn/scons/trunk ........ git-svn-id: http://scons.tigris.org/svn/scons/trunk@1540 fdb21ef1-2011-0410-befe-b5e4ea1792b1 --- diff --git a/HOWTO/subrelease.txt b/HOWTO/subrelease.txt index d102130e..fb799e69 100644 --- a/HOWTO/subrelease.txt +++ b/HOWTO/subrelease.txt @@ -36,6 +36,9 @@ Things to do to release a new X.Y.Z version of SCons: aecp rpm/scons.spec.in vi rpm/scons.spec.in + aecp src/setup.py + vi src/setup.py + aecp src/test_setup.py vi src/test_setup.py diff --git a/etc/.aeignore b/QMTest/.aeignore similarity index 100% rename from etc/.aeignore rename to QMTest/.aeignore diff --git a/etc/SConscript b/QMTest/SConscript similarity index 89% rename from etc/SConscript rename to QMTest/SConscript index e68663d7..e016dc4d 100644 --- a/etc/SConscript +++ b/QMTest/SConscript @@ -30,8 +30,12 @@ import os.path Import('env') files = [ + 'classes.qmc', + 'configuration', + 'scons_tdb.py', 'TestCmd.py', 'TestCommon.py', + 'TestRuntest.py', 'TestSCons.py', 'unittest.py', ] @@ -43,9 +47,9 @@ def copy(target, source, env): for file in files: # Guarantee that real copies of these files always exist in - # build/etc. If there's a symlink there, then this is an Aegis + # build/QMTest. If there's a symlink there, then this is an Aegis # build and we blow them away now so that they'll get "built" later. - p = os.path.join('build', 'etc', file) + p = os.path.join('build', 'QMTest', file) if os.path.islink(p): os.unlink(p) sp = '#' + p diff --git a/etc/TestCmd.py b/QMTest/TestCmd.py similarity index 100% rename from etc/TestCmd.py rename to QMTest/TestCmd.py diff --git a/etc/TestCommon.py b/QMTest/TestCommon.py similarity index 100% rename from etc/TestCommon.py rename to QMTest/TestCommon.py diff --git a/QMTest/TestRuntest.py b/QMTest/TestRuntest.py new file mode 100644 index 00000000..6fd423a9 --- /dev/null +++ b/QMTest/TestRuntest.py @@ -0,0 +1,137 @@ +""" +TestRuntest.py: a testing framework for the runtest.py command used to +invoke SCons tests. + +A TestRuntest environment object is created via the usual invocation: + + test = TestRuntest() + +TestRuntest is a subclass of TestCommon, which is in turn is a subclass +of TestCmd), and hence has available all of the methods and attributes +from those classes, as well as any overridden or additional methods or +attributes defined in this subclass. +""" + +# __COPYRIGHT__ + +__revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" + +import os +import os.path +import string +import shutil +import sys + +from TestCommon import * +from TestCommon import __all__ + +__all__.extend([ 'TestRuntest', + 'python', + ]) + +python = python_executable + + +failing_test_template = """\ +import sys +sys.stdout.write('FAILING TEST STDOUT\\n') +sys.stderr.write('FAILING TEST STDERR\\n') +sys.exit(1) +""" + +no_result_test_template = """\ +import sys +sys.stdout.write('NO RESULT TEST STDOUT\\n') +sys.stderr.write('NO RESULT TEST STDERR\\n') +sys.exit(2) +""" + +passing_test_template = """\ +import sys +sys.stdout.write('PASSING TEST STDOUT\\n') +sys.stderr.write('PASSING TEST STDERR\\n') +sys.exit(0) +""" + +class TestRuntest(TestCommon): + """Class for testing the runtest.py script. + + This provides a common place for initializing Runtest tests, + eliminating the need to begin every test with the same repeated + initializations. + """ + + def __init__(self, **kw): + """Initialize a Runtest testing object. + + If they're not overridden by keyword arguments, this + initializes the object with the following default values: + + program = 'runtest.py' + interpreter = ['python', '-tt'] + match = match_exact + workdir = '' + + The workdir value means that, by default, a temporary + workspace directory is created for a TestRuntest environment. + The superclass TestCommon.__init__() will change directory (chdir) + to the workspace directory, so an explicit "chdir = '.'" on all + of the run() method calls is not necessary. This initialization + also copies the runtest.py and QMTest/ subdirectory tree to the + temporary directory, duplicating how this test infrastructure + appears in a normal workspace. + """ + set_workpath_runtest = None + if not kw.has_key('program'): + kw['program'] = 'runtest.py' + set_workpath_runtest = 1 + if not kw.has_key('interpreter'): + kw['interpreter'] = [python, '-tt'] + if not kw.has_key('match'): + kw['match'] = match_exact + if not kw.has_key('workdir'): + kw['workdir'] = '' + orig_cwd = os.getcwd() + apply(TestCommon.__init__, [self], kw) + + things_to_copy = [ + 'runtest.py', + 'QMTest', + ] + + dirs = [] + + spe = os.environ.get('SCONS_SOURCE_PATH_EXECUTABLE', orig_cwd) + for d in string.split(spe, os.pathsep): + dirs.append(os.path.join(d, 'build')) + dirs.append(d) + + for thing in things_to_copy: + for dir in dirs: + t = os.path.join(dir, thing) + if os.path.exists(t): + if os.path.isdir(t): + copy_func = shutil.copytree + else: + copy_func = shutil.copyfile + copy_func(t, self.workpath(thing)) + break + + if set_workpath_runtest: + self.program_set(self.workpath('runtest.py')) + + for key in os.environ.keys(): + if key[:5] == 'AEGIS': + os.environ[key] = '' + + os.environ['PYTHONPATH'] = '' + os.environ['SCONS_SOURCE_PATH_EXECUTABLE'] = '' + + def write_failing_test(self, name): + self.write(name, failing_test_template) + + def write_no_result_test(self, name): + self.write(name, no_result_test_template) + + def write_passing_test(self, name): + self.write(name, passing_test_template) diff --git a/etc/TestSCons.py b/QMTest/TestSCons.py similarity index 98% rename from etc/TestSCons.py rename to QMTest/TestSCons.py index 49e8da6d..0904d315 100644 --- a/etc/TestSCons.py +++ b/QMTest/TestSCons.py @@ -123,6 +123,13 @@ class TestSCons(TestCommon): so an explicit "chdir = '.'" on all of the run() method calls is not necessary. """ + self.orig_cwd = os.getcwd() + try: + script_dir = os.environ['SCONS_SCRIPT_DIR'] + except KeyError: + pass + else: + os.chdir(script_dir) if not kw.has_key('program'): kw['program'] = os.environ.get('SCONS') if not kw['program']: diff --git a/QMTest/classes.qmc b/QMTest/classes.qmc new file mode 100644 index 00000000..73e3df3a --- /dev/null +++ b/QMTest/classes.qmc @@ -0,0 +1,11 @@ + + + + + + + + + diff --git a/QMTest/configuration b/QMTest/configuration new file mode 100644 index 00000000..db648aef --- /dev/null +++ b/QMTest/configuration @@ -0,0 +1,6 @@ + + + + . + + diff --git a/QMTest/scons_tdb.py b/QMTest/scons_tdb.py new file mode 100644 index 00000000..145c2a7a --- /dev/null +++ b/QMTest/scons_tdb.py @@ -0,0 +1,520 @@ +#!/usr/bin/env python +# +# __COPYRIGHT__ +# +# Permission is hereby granted, free of charge, to any person obtaining +# a copy of this software and associated documentation files (the +# "Software"), to deal in the Software without restriction, including +# without limitation the rights to use, copy, modify, merge, publish, +# distribute, sublicense, and/or sell copies of the Software, and to +# permit persons to whom the Software is furnished to do so, subject to +# the following conditions: +# +# The above copyright notice and this permission notice shall be included +# in all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY +# KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE +# WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE +# LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION +# WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. +# + +__revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" + +""" +QMTest classes to support SCons' testing and Aegis-inspired workflow. + +Thanks to Stefan Seefeld for the initial code. +""" + +######################################################################## +# Imports +######################################################################## + +import qm +import qm.common +import qm.test.base +from qm.fields import * +from qm.executable import * +from qm.test import database +from qm.test import test +from qm.test import resource +from qm.test import suite +from qm.test.result import Result +from qm.test.file_result_stream import FileResultStream +from qm.test.classes.text_result_stream import TextResultStream +from qm.test.directory_suite import DirectorySuite +from qm.extension import get_extension_class_name, get_class_arguments_as_dictionary +import os, dircache + +if sys.platform == 'win32': + console = 'con' +else: + console = '/dev/tty' + +def Trace(msg): + open(console, 'w').write(msg) + +# QMTest 2.3 hard-codes how it captures the beginning and end time by +# calling the qm.common.format_time_iso() function, which canonicalizes +# the time stamp in one-second granularity ISO format. In order to get +# sub-second granularity, as well as to use the more precise time.clock() +# function on Windows, we must replace that function with our own. + +orig_format_time_iso = qm.common.format_time_iso + +if sys.platform == 'win32': + time_func = time.clock +else: + time_func = time.time + +def my_format_time(time_secs=None): + return str(time_func()) + +qm.common.format_time_iso = my_format_time + +######################################################################## +# Classes +######################################################################## + +def get_explicit_arguments(e): + """This function can be removed once QMTest 2.4 is out.""" + + # Get all of the arguments. + arguments = get_class_arguments_as_dictionary(e.__class__) + # Determine which subset of the 'arguments' have been set + # explicitly. + explicit_arguments = {} + for name, field in arguments.items(): + # Do not record computed fields. + if field.IsComputed(): + continue + if e.__dict__.has_key(name): + explicit_arguments[name] = e.__dict__[name] + + return explicit_arguments + + +def check_exit_status(result, prefix, desc, status): + """This function can be removed once QMTest 2.4 is out.""" + + if sys.platform == "win32" or os.WIFEXITED(status): + # Obtain the exit code. + if sys.platform == "win32": + exit_code = status + else: + exit_code = os.WEXITSTATUS(status) + # If the exit code is non-zero, the test fails. + if exit_code != 0: + result.Fail("%s failed with exit code %d." % (desc, exit_code)) + # Record the exit code in the result. + result[prefix + "exit_code"] = str(exit_code) + return False + + elif os.WIFSIGNALED(status): + # Obtain the signal number. + signal = os.WTERMSIG(status) + # If the program gets a fatal signal, the test fails . + result.Fail("%s received fatal signal %d." % (desc, signal)) + result[prefix + "signal"] = str(signal) + return False + else: + # A process should only be able to stop by exiting, or + # by being terminated with a signal. + assert None + + return True + +# XXX I'd like to annotate the overall test run with the following +# information about the Python version, SCons version, and environment. +# Not sure how to do that yet; ask Stefan. +# +# sys_keys = ['byteorder', 'exec_prefix', 'executable', 'maxint', 'maxunicode', 'platform', 'prefix', 'version', 'version_info'] + +# " <%s>" % tag +# " %s" % module.__version__ +# " %s" % module.__build__ +# " %s" % module.__buildsys__ +# " %s" % module.__date__ +# " %s" % module.__developer__ +# " " % tag + +# " " +# print_version_info("script", scons) +# print_version_info("engine", SCons) +# " " + +# environ_keys = [ +# 'PATH', +# 'SCONSFLAGS', +# 'SCONS_LIB_DIR', +# 'PYTHON_ROOT', +# 'QTDIR', +# +# 'COMSPEC', +# 'INTEL_LICENSE_FILE', +# 'INCLUDE', +# 'LIB', +# 'MSDEVDIR', +# 'OS', +# 'PATHEXT', +# 'SYSTEMROOT', +# 'TEMP', +# 'TMP', +# 'USERNAME', +# 'VXDOMNTOOLS', +# 'WINDIR', +# 'XYZZY' +# +# 'ENV', +# 'HOME', +# 'LANG', +# 'LANGUAGE', +# 'LOGNAME', +# 'MACHINE', +# 'OLDPWD', +# 'PWD', +# 'OPSYS', +# 'SHELL', +# 'TMPDIR', +# 'USER', +# ] + +class AegisStream(TextResultStream): + def __init__(self, *args, **kw): + super(AegisStream, self).__init__(*args, **kw) + self._num_tests = 0 + self._outcomes = {} + self._outcome_counts = {} + for outcome in AegisTest.aegis_outcomes: + self._outcome_counts[outcome] = 0 + self.format = "full" + def _percent(self, outcome): + return 100. * self._outcome_counts[outcome] / self._num_tests + def _aegis_no_result(self, result): + outcome = result.GetOutcome() + return (outcome == Result.FAIL and result.get('Test.exit_code') == '2') + def _DisplayText(self, text): + # qm.common.html_to_text() uses htmllib, which sticks an extra + # '\n' on the front of the text. Strip it and only display + # the text if there's anything to display. + text = qm.common.html_to_text(text) + if text[0] == '\n': + text = text[1:] + if text: + lines = text.splitlines() + if lines[-1] == '': + lines = lines[:-1] + self.file.write(' ' + '\n '.join(lines) + '\n\n') + def _DisplayResult(self, result, format): + test_id = result.GetId() + kind = result.GetKind() + if self._aegis_no_result(result): + outcome = "NO_RESULT" + else: + outcome = result.GetOutcome() + self._WriteOutcome(test_id, kind, outcome) + self.file.write('\n') + def _DisplayAnnotations(self, result): + try: + self._DisplayText(result["Test.stdout"]) + except KeyError: + pass + try: + self._DisplayText(result["Test.stderr"]) + except KeyError: + pass + if result["Test.print_time"] != "0": + start = float(result['qmtest.start_time']) + end = float(result['qmtest.end_time']) + fmt = " Total execution time: %.1f seconds\n\n" + self.file.write(fmt % (end - start)) + +class AegisChangeStream(AegisStream): + def WriteResult(self, result): + test_id = result.GetId() + if self._aegis_no_result(result): + outcome = AegisTest.NO_RESULT + else: + outcome = result.GetOutcome() + self._num_tests += 1 + self._outcome_counts[outcome] += 1 + super(AegisStream, self).WriteResult(result) + def _SummarizeTestStats(self): + self.file.write("\n") + self._DisplayHeading("STATISTICS") + if self._num_tests != 0: + # We'd like to use the _FormatStatistics() method to do + # this, but it's wrapped around the list in Result.outcomes, + # so it's simpler to just do it ourselves. + print " %6d tests total\n" % self._num_tests + for outcome in AegisTest.aegis_outcomes: + if self._outcome_counts[outcome] != 0: + print " %6d (%3.0f%%) tests %s" % ( + self._outcome_counts[outcome], + self._percent(outcome), + outcome + ) + +class AegisBaselineStream(AegisStream): + def WriteResult(self, result): + test_id = result.GetId() + if self._aegis_no_result(result): + outcome = AegisTest.NO_RESULT + self.expected_outcomes[test_id] = Result.PASS + self._outcome_counts[outcome] += 1 + else: + self.expected_outcomes[test_id] = Result.FAIL + outcome = result.GetOutcome() + if outcome != Result.Fail: + self._outcome_counts[outcome] += 1 + self._num_tests += 1 + super(AegisStream, self).WriteResult(result) + def _SummarizeRelativeTestStats(self): + self.file.write("\n") + self._DisplayHeading("STATISTICS") + if self._num_tests != 0: + # We'd like to use the _FormatStatistics() method to do + # this, but it's wrapped around the list in Result.outcomes, + # so it's simpler to just do it ourselves. + if self._outcome_counts[AegisTest.FAIL]: + print " %6d (%3.0f%%) tests as expected" % ( + self._outcome_counts[AegisTest.FAIL], + self._percent(AegisTest.FAIL), + ) + non_fail_outcomes = list(AegisTest.aegis_outcomes[:]) + non_fail_outcomes.remove(AegisTest.FAIL) + for outcome in non_fail_outcomes: + if self._outcome_counts[outcome] != 0: + print " %6d (%3.0f%%) tests unexpected %s" % ( + self._outcome_counts[outcome], + self._percent(outcome), + outcome, + ) + +class AegisBatchStream(FileResultStream): + arguments = [ + qm.fields.TextField( + name = "results_file", + title = "Aegis Results File", + description = """ + """, + verbatim = "true", + default_value = "aegis-results.txt", + ), + ] + def __init__(self, arguments): + self.filename = arguments['results_file'] + super(AegisBatchStream, self).__init__(arguments) + self._outcomes = {} + def WriteResult(self, result): + test_id = result.GetId() + kind = result.GetKind() + outcome = result.GetOutcome() + exit_status = '0' + if outcome == Result.FAIL: + exit_status = result.get('Test.exit_code') + self._outcomes[test_id] = exit_status + def Summarize(self): + self.file.write('test_result = [\n') + for file_name, exit_status in self._outcomes.items(): + self.file.write(' { file_name = "%s";\n' % file_name) + self.file.write(' exit_status = %s; },\n' % exit_status) + self.file.write('];\n') + +class AegisTest(test.Test): + PASS = "PASS" + FAIL = "FAIL" + NO_RESULT = "NO_RESULT" + ERROR = "ERROR" + UNTESTED = "UNTESTED" + + aegis_outcomes = ( + PASS, FAIL, NO_RESULT, ERROR, UNTESTED, + ) + """Aegis test outcomes.""" + +class Test(AegisTest): + """Simple test that runs a python script and checks the status + to determine whether the test passes.""" + + script = TextField(title="Script to test") + topdir = TextField(title="Top source directory") + + def Run(self, context, result): + """Run the test. The test passes if the command exits with status=0, + and fails otherwise. The program output is logged, but not validated.""" + + command = RedirectedExecutable() + args = [context.get('python', 'python'), self.script] + status = command.Run(args, os.environ) + result["Test.print_time"] = context.get('print_time', '0') + if not check_exit_status(result, 'Test.', self.script, status): + # In case of failure record exit code, stdout, and stderr. + result.Fail("Non-zero exit_code.") + result["Test.stdout"] = result.Quote(command.stdout) + result["Test.stderr"] = result.Quote(command.stderr) + + +class Database(database.Database): + """Scons test database. + * The 'src' and 'test' directories are explicit suites. + * Their subdirectories are implicit suites. + * All files under 'src/' ending with 'Tests.py' contain tests. + * All files under 'test/' with extension '.py' contain tests. + * Right now there is only a single test class, which simply runs + the specified python interpreter on the given script. To be refined...""" + + srcdir = TextField(title = "Source Directory", + description = "The root of the test suite's source tree.") + _is_generic_database = True + + def is_a_test_under_test(path, t): + return os.path.splitext(t)[1] == '.py' \ + and os.path.isfile(os.path.join(path, t)) + + def is_a_test_under_src(path, t): + return t[-8:] == 'Tests.py' \ + and os.path.isfile(os.path.join(path, t)) + + is_a_test = { + 'src' : is_a_test_under_src, + 'test' : is_a_test_under_test, + } + + exclude_subdirs = { + '.svn' : 1, + 'CVS' : 1, + } + + def is_a_test_subdir(path, subdir): + if exclude_subdirs.get(subdir): + return None + return os.path.isdir(os.path.join(path, subdir)) + + def __init__(self, path, arguments): + + self.label_class = "file_label.FileLabel" + self.modifiable = "false" + # Initialize the base class. + super(Database, self).__init__(path, arguments) + + + def GetRoot(self): + + return self.srcdir + + + def GetSubdirectories(self, directory): + + components = self.GetLabelComponents(directory) + path = os.path.join(self.GetRoot(), *components) + if directory: + dirs = [d for d in dircache.listdir(path) + if os.path.isdir(os.path.join(path, d))] + else: + dirs = self.is_a_test.keys() + + dirs.sort() + return dirs + + + def GetIds(self, kind, directory = "", scan_subdirs = 1): + + components = self.GetLabelComponents(directory) + path = os.path.join(self.GetRoot(), *components) + + if kind == database.Database.TEST: + + if not components: + return [] + + ids = [self.JoinLabels(directory, t) + for t in dircache.listdir(path) + if self.is_a_test[components[0]](path, t)] + + elif kind == Database.RESOURCE: + return [] # no resources yet + + else: # SUITE + + if directory: + ids = [self.JoinLabels(directory, d) + for d in dircache.listdir(path) + if os.path.isdir(os.path.join(path, d))] + else: + ids = self.is_a_test.keys() + + if scan_subdirs: + for d in dircache.listdir(path): + if (os.path.isdir(d)): + ids.extend(self.GetIds(kind, + self.JoinLabels(directory, d), + True)) + + return ids + + + def GetExtension(self, id): + + if not id: + return DirectorySuite(self, id) + + components = self.GetLabelComponents(id) + path = os.path.join(self.GetRoot(), *components) + + if os.path.isdir(path): # a directory + return DirectorySuite(self, id) + + elif os.path.isfile(path): # a test + + arguments = {} + arguments['script'] = path + arguments['topdir'] = self.GetRoot() + + return Test(arguments, qmtest_id = id, qmtest_database = self) + + else: # nothing else to offer + + return None + + + def GetTest(self, test_id): + """This method can be removed once QMTest 2.4 is out.""" + + t = self.GetExtension(test_id) + if isinstance(t, test.Test): + return database.TestDescriptor(self, + test_id, + get_extension_class_name(t.__class__), + get_explicit_arguments(t)) + + raise database.NoSuchTestError(test_id) + + def GetSuite(self, suite_id): + """This method can be removed once QMTest 2.4 is out.""" + + if suite_id == "": + return DirectorySuite(self, "") + + s = self.GetExtension(suite_id) + if isinstance(s, suite.Suite): + return s + + raise database.NoSuchSuiteError(suite_id) + + + def GetResource(self, resource_id): + """This method can be removed once QMTest 2.4 is out.""" + + r = self.GetExtension(resource_id) + if isinstance(r, resource.Resource): + return ResourceDescriptor(self, + resource_id, + get_extension_class_name(r.__class__), + get_explicit_arguments(r)) + + raise database.NoSuchResourceError(resource_id) diff --git a/etc/unittest.py b/QMTest/unittest.py similarity index 100% rename from etc/unittest.py rename to QMTest/unittest.py diff --git a/README b/README index d4d27575..c0c013e0 100644 --- a/README +++ b/README @@ -23,7 +23,7 @@ however, you don't actually need to build or install SCons; you sections below for more information: MAKING CHANGES - How to edit and executing SCons in-place. + How to edit and execute SCons in-place. DEBUGGING Tips for debugging problems in SCons. @@ -83,12 +83,12 @@ In this case, your options are: -- (Optional.) Install from a pre-packaged SCons package that does not require distutils: - Red Hat Linux scons-0.96.noarch.rpm + Red Hat Linux scons-0.96.92.noarch.rpm - Debian GNU/Linux scons_0.96_all.deb + Debian GNU/Linux scons_0.96.92_all.deb (or use apt-get) - Windows scons-0.96.win32.exe + Windows scons-0.96.92.win32.exe -- (Recommended.) Download the latest distutils package from the following URL: @@ -152,7 +152,7 @@ And on Windows: By default, the above commands will do the following: - -- Install the version-numbered "scons-0.96" and "sconsign-0.96" + -- Install the version-numbered "scons-0.96.92" and "sconsign-0.96.92" scripts in the default system script directory (/usr/bin or C:\Python*\Scripts, for example). This can be disabled by specifying the "--no-version-script" option on the command @@ -165,15 +165,15 @@ By default, the above commands will do the following: if you want to install and experiment with a new version before making it the default on your system. On UNIX or Linux systems, you can have the "scons" and "sconsign" scripts be hard links or - symbolic links to the "scons-0.96" and "sconsign-0.96" scripts + symbolic links to the "scons-0.96.92" and "sconsign-0.96.92" scripts by specifying the "--hardlink-scons" or "--symlink-scons" options on the command line. - -- Install "scons-0.96.bat" and "scons.bat" wrapper scripts in the + -- Install "scons-0.96.92.bat" and "scons.bat" wrapper scripts in the Python prefix directory on Windows (C:\Python*, for example). This can be disabled by specifying the "--no-install-bat" option on the command line. On UNIX or Linux systems, the - "--install-bat" option may be specified to have "scons-0.96.bat" + "--install-bat" option may be specified to have "scons-0.96.92.bat" and "scons.bat" files installed in the default system script directory, which is useful if you want to install SCons in a shared file system directory that can be used to execute SCons @@ -181,7 +181,7 @@ By default, the above commands will do the following: -- Install the SCons build engine (a Python module) in an appropriate version-numbered SCons library directory - (/usr/lib/scons-0.96 or C:\Python*\scons-0.96, for example). + (/usr/lib/scons-0.96.92 or C:\Python*\scons-0.96.92, for example). See below for more options related to installing the build engine library. @@ -505,18 +505,18 @@ On Windows: Depending on the utilities installed on your system, any or all of the following packages will be built: - build/dist/scons-0.96-1.noarch.rpm - build/dist/scons-0.96-1.src.rpm - build/dist/scons-0.96.linux-i686.tar.gz - build/dist/scons-0.96.tar.gz - build/dist/scons-0.96.win32.exe - build/dist/scons-0.96.zip - build/dist/scons-doc-0.96.tar.gz - build/dist/scons-local-0.96.tar.gz - build/dist/scons-local-0.96.zip - build/dist/scons-src-0.96.tar.gz - build/dist/scons-src-0.96.zip - build/dist/scons_0.96-1_all.deb + build/dist/scons-0.96.92-1.noarch.rpm + build/dist/scons-0.96.92-1.src.rpm + build/dist/scons-0.96.92.linux-i686.tar.gz + build/dist/scons-0.96.92.tar.gz + build/dist/scons-0.96.92.win32.exe + build/dist/scons-0.96.92.zip + build/dist/scons-doc-0.96.92.tar.gz + build/dist/scons-local-0.96.92.tar.gz + build/dist/scons-local-0.96.92.zip + build/dist/scons-src-0.96.92.tar.gz + build/dist/scons-src-0.96.92.zip + build/dist/scons_0.96.92-1_all.deb The SConstruct file is supposed to be smart enough to avoid trying to build packages for which you don't have the proper utilities installed. diff --git a/SConstruct b/SConstruct index 067e7573..49cc6ff3 100644 --- a/SConstruct +++ b/SConstruct @@ -40,7 +40,7 @@ import sys import time project = 'scons' -default_version = '0.96' +default_version = '0.96.92' copyright = "Copyright (c) %s The SCons Foundation" % copyright_years Default('.') @@ -152,6 +152,17 @@ for key in ['AEGIS_PROJECT', 'LOGNAME', 'PYTHONPATH']: cwd_build = os.path.join(os.getcwd(), "build") +packaging_flavors = [ + 'deb', + 'rpm', + 'tar-gz', + 'src-tar-gz', + 'local-tar-gz', + 'zip', + 'src-zip', + 'local-zip', +] + test_deb_dir = os.path.join(cwd_build, "test-deb") test_rpm_dir = os.path.join(cwd_build, "test-rpm") test_tar_gz_dir = os.path.join(cwd_build, "test-tar-gz") @@ -949,7 +960,30 @@ for p in [ scons ]: # Export('env') -SConscript('etc/SConscript') +SConscript('QMTest/SConscript') + +# +# +# +files = [ + 'runtest.py', +] + +def copy(target, source, env): + t = str(target[0]) + s = str(source[0]) + open(t, 'wb').write(open(s, 'rb').read()) + +for file in files: + # Guarantee that real copies of these files always exist in + # build/. If there's a symlink there, then this is an Aegis + # build and we blow them away now so that they'll get "built" later. + p = os.path.join('build', file) + if os.path.islink(p): + os.unlink(p) + sp = '#' + p + env.Command(sp, file, copy) + Local(sp) # # Documentation. @@ -1136,3 +1170,6 @@ if change: 'setup.py'), ], ENV = ENV) + +for pf in packaging_flavors: + Alias(pf, ['build/test-'+pf, 'build/QMTest', 'build/runtest.py']) diff --git a/config b/config index 0e83aad3..e540484f 100644 --- a/config +++ b/config @@ -258,9 +258,9 @@ diff_command = * is set appropriately during a baseline test. So we just use the * proper aesub variable to comment out the expanded $spe. */ -test_command = "python1.5 ${Source runtest.py Absolute} -p tar-gz -t -v ${SUBSTitute '\\.[CD][0-9]+$' '' ${VERsion}} -q ${File_Name}"; +test_command = "python1.5 ${Source runtest.py Absolute} -p tar-gz -t -v ${SUBSTitute '\\.[CD][0-9]+$' '' ${VERsion}} -q --sp ${Search_Path} --spe ${Search_Path_Executable} ${File_Name}"; -batch_test_command = "python1.5 ${Source runtest.py Absolute} -p tar-gz -t -v ${SUBSTitute '\\.[CD][0-9]+$' '' ${VERsion}} -o ${Output} --aegis ${File_Names} ${COMment $spe}"; +batch_test_command = "python1.5 ${Source runtest.py Absolute} -p tar-gz -t -v ${SUBSTitute '\\.[CD][0-9]+$' '' ${VERsion}} -o ${Output} --aegis --sp ${Search_Path} --spe ${Search_Path_Executable} ${File_Names}"; new_test_filename = "test/CHANGETHIS.py"; diff --git a/debian/changelog b/debian/changelog index d6a9cc69..f205e257 100644 --- a/debian/changelog +++ b/debian/changelog @@ -1,3 +1,10 @@ +scons (0.96-92) unstable; urgency=low + + * Pre-release of eighth beta release. + + -- Steven Knight Mon, 10 Apr 2006 21:08:22 -0400 + + scons (0.96-1) unstable; urgency=low * Seventh beta release. diff --git a/doc/SConscript b/doc/SConscript index de8e21e5..6162f939 100644 --- a/doc/SConscript +++ b/doc/SConscript @@ -382,10 +382,10 @@ for m in man_page_list: orig_env.SCons_revision(os.path.join(build, 'man', m), os.path.join('man', m)) -man_intermediate_files = ['builders.man', 'tools.man', 'variables.man'] +man_i_files = ['builders.man', 'tools.man', 'variables.man'] man_intermediate_files = map(lambda x: os.path.join(build, 'man', x), - man_intermediate_files) + man_i_files) cmd = "python $SCONS_PROC_PY --man -b ${TARGETS[0]} -t ${TARGETS[1]} -v ${TARGETS[2]} $( $SOURCES $)" man_intermediate_files = env.Command(man_intermediate_files, @@ -417,7 +417,18 @@ for man_1 in man_page_list: if man2html: html = os.path.join(build, 'HTML' , '%s-man.html' % man) - cmds = [ "( cd ${SOURCES.dir} && man2html ${SOURCES.dir} ) > $TARGET" ] + def strip_to_first_html_tag(target, source, env): + t = str(target[0]) + contents = open(t).read() + contents = contents[string.find(contents, ''):] + open(t, 'w').write(contents) + return 0 + + cmds = [ + "( cd build/doc/man && cp %s .. )" % string.join(man_i_files), + "( cd ${SOURCE.dir} && man2html ${SOURCE.file} ) > $TARGET", + Action(strip_to_first_html_tag), + ] if tidy: cmds.append("tidy -m -q $TARGET || true") b = env.Command(html, man_1, cmds) diff --git a/doc/man/scons.1 b/doc/man/scons.1 index 45d7a5b2..49f9a305 100644 --- a/doc/man/scons.1 +++ b/doc/man/scons.1 @@ -2182,6 +2182,32 @@ env.Command('baz.out', 'baz.in', rename ]) .EE +Note that the +.BR Command () +function will usually assume, by default, +that the specified targets and/or sources are Files, +if no other part of the configuration +identifies what type of entry it is. +If necessary, you can explicitly specify +that targets or source nodes should +be treated as directoriese +by using the +.BR Dir () +or +.BR env.Dir () +functions: + +.ES +env.Command('ddd.list', Dir('ddd'), 'ls -l $SOURCE > $TARGET') + +env['DISTDIR'] = 'destination/directory' +env.Command(env.Dir('$DISTDIR')), None, make_distdir) +.EE + +(Also note that SCons will usually +automatically create any directory necessary to hold a target file, +so you normally don't need to create directories by hand.) + '\""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" .TP .RI Configure( env ", [" custom_tests ", " conf_dir ", " log_file ", " config_h ]) @@ -2780,6 +2806,56 @@ even if an already up-to-date copy exists in a repository. Returns a list of the target Node or Nodes. +'\""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" +.TP +.RI env.MergeFlags( arg ", [" unique ]) +Merges the specified +.I arg +values to the construction envrionment's construction variables. +If the +.I arg +argument is not a dictionary, +it is converted to one by calling +.B env.ParseFlags() +on the argument +before the values are merged. +Note that +.I arg +must be a single value, +so multiple strings must +be passed in as a list, +not as separate arguments to +.BR env.MergeFlags (). + +By default, +duplicate values are eliminated; +you can, however, specify +.B unique=0 +to allow duplicate +values to be added. +When eliminating duplicate values, +any construction variables that end with +the string +.B PATH +keep the left-most unique value. +All other construction variables keep +the right-most unique value. + +Examples: + +.ES +# Add an optimization flag to $CCFLAGS. +env.MergeFlags('-O3') + +# Combine the flags returned from running pkg-config with an optimization +# flag and merge the result into the construction variables. +env.MergeFlags(['!pkg-config gtk+-2.0 --cflags', '-O3']) + +env.MergeFlags(['-O3', + '!pkg-config gtk+-2.0 --cflags --libs', + '!pkg-config libpng12 --cflags --libs']) +.EE + '\""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" .TP .RI NoClean( target ", ...)" @@ -2836,7 +2912,9 @@ to modify the environment as specified by the output of .I command . The default .I function -expects the output of a typical +is +.BR env.MergeFlags (), +which expects the output of a typical .I *-config command (for example, .BR gtk-config ) @@ -2850,42 +2928,13 @@ you can specify to allow duplicate values to be added. -By default, -.BR -L , -.BR -l , -.BR -Wa , -.BR -Wl , -.BR -Wp , -.B -I -and other options, -are add to the -.BR LIBPATH , -.BR LIBS , -.BR ASFLAGS , -.BR LINKFLAGS , -.BR CPPFLAGS , -.B CPPPATH -and -.B CCFLAGS -construction variables, -respectively. -A returned -.B -pthread -option gets added to both the -.B CCFLAGS -and -.B LINKFLAGS -variables. -A returned -.B -framework -option gets added to the -.B LINKFLAGS -variable. -Any other strings not associated with options -are assumed to be the names of libraries -and added to the -.B LIBS -construction variable. +Interpreted options +and the construction variables they affect +are as specified for the +.BR env.ParseFlags () +method (which thie method calls). +See that method's description, below, +for a table of options and construction variables. '\""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" .TP @@ -2944,6 +2993,73 @@ file which calls the .B ParseDepends function. +'\""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" +.TP +.RI env.ParseFlags( flags ", ...)" +Parses one or more strings containing +typical command-line flags for GCC tool chains +and returns a dictionary with the flag values +separated into the appropriate SCons construction variables. +This is intended as a companion to the +.BR env.MergeFlags () +method, but allows for the values in the returned dictionary +to be modified, if necessary, +before merging them into the construction environment. +(Note that +.BR env.MergeFlags () +will call this method if its argument is not a dictionary, +so it is usually not necessary to call +.BR env.ParseFlags () +directly unless you want to manipulate the values.) + +If the first character in any string is +an exclamation mark (!), +the rest of the string is executed as a command, +and the output from the command is +parsed as GCC tool chain command-line flags +and added to the resulting dictionary. + +Flag values are translated accordig to the prefix found, +and added to the following construction variables: + +.ES +-arch CCFLAGS, LINKFLAGS +-D CPPDEFINES +-framework FRAMEWORKS +-frameworkdir= FRAMEWORKPATH +-include CCFLAGS +-isysroot CCFLAGS, LINKFLAGS +-I CPPPATH +-l LIBS +-L LIBPATH +-mno-cygwin CCFLAGS, LINKFLAGS +-mwindows LINKFLAGS +-pthread CCFLAGS, LINKFLAGS +-Wa, ASFLAGS, CCFLAGS +-Wl,-rpath= RPATH +-Wl,-R, RPATH +-Wl,-R RPATH +-Wl, LINKFLAGS +-Wp, CPPFLAGS +- CCFLAGS ++ CCFLAGS, LINKFLAGS +.EE + +Any other strings not associated with options +are assumed to be the names of libraries +and added to the +.B LIBS +construction variable. + +Examples (all of which produce the same result): + +.ES +dict = env.ParseFlags('-O2 -Dfoo -Dbar=1') +dict = env.ParseFlags('-O2', '-Dfoo', '-Dbar=1') +dict = env.ParseFlags(['-O2', '-Dfoo -Dbar=1']) +dict = env.ParseFlags('-O2', '!echo -Dfoo -Dbar=1') +.EE + '\""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" .TP env.Perforce() @@ -4425,10 +4541,13 @@ or a list of library names, in which case each library in the list will be checked for .IR symbol . -The default +If .I symbol -is "main", -which just check if +is not set or is +.BR None , +then +.BR Configure.CheckLib () +just checks if you can link against the specified .IR library . The optional @@ -4442,7 +4561,6 @@ the default is "C". The default value for .I autoadd is 1. -It is assumed, that the C-language is used. This method returns 1 on success and 0 on error. .TP @@ -4468,7 +4586,13 @@ header line being checked for. .I language may be one of 'C','c','CXX','cxx','C++' and 'c++'. .I call -can be any valid expression (with a trailing ';'). The default is 'main();'. +can be any valid expression (with a trailing ';'). +If +.I call +is not set, +the default simply checks that you +can link against the specified +.IR library . .I autoadd specifies whether to add the library to the environment (only if the check succeeds). This method returns 1 on success and 0 on error. @@ -4705,13 +4829,30 @@ the construction variable will be added to the construction environment. .I validator is called to validate the value of the variable, and should take three -arguments: key, value, and environment +arguments: key, value, and environment. +The recommended way to handle an invalid value is +to raise an exception (see example below). .I converter is called to convert the value before putting it in the environment, and -should take a single argument: value. Example: +should take a single argument: value. +The +.I converter +must return a value, +which will be converted into a string +before being validated by the +.I validator +(if any) +and then added to the environment. + +Examples: .ES opts.Add('CC', 'The C compiler') + +def validate_color(key, val, env): + if not val in ['red', 'blue', 'yellow']: + raise "Invalid color value '%s'" % val +opts.Add('COLOR', validator=valid_color) .EE .TP @@ -5472,6 +5613,48 @@ and .I action arguments must not both be used for the same Builder. +.IP source_ext_match +When the specified +.I action +argument is a dictionary, +the default behavior when a builder is passed +multiple source files is to make sure that the +extensions of all the source files match. +If it is legal for this builder to be +called with a list of source files with different extensions, +this check can be suppressed by setting +.B source_ext_match +to +.B None +or some other non-true value. +When +.B source_ext_match +is disable, +.B scons +will use the suffix of the first specified +source file to select the appropriate action from the +.I action +dictionary. + +In the following example, +the setting of +.B source_ext_match +prevents +.B scons +from exiting with an error +due to the mismatched suffixes of +.B foo.in +and +.BR foo.extra . + +.ES +b = Builder(action={'.in' : 'build $SOURCES > $TARGET'}, + source_ext_match = None) + +env = Environment(BUILDERS = {'MyBuild':b}) +env.MyBuild('foo.out', ['foo.in', 'foo.extra']) +.EE + .IP env A construction environment that can be used to fetch source code using this Builder. @@ -5535,6 +5718,21 @@ env = Environment(BUILDERS = {'MyBuild' : b}) env.MyBuild('sub/dir/foo.out', 'sub/dir/foo.in') .EE +.B WARNING: +Python only keeps one current directory +location for all of the threads. +This means that use of the +.B chdir +argument +will +.I not +work with the SCons +.B -j +option, +because individual worker threads spawned +by SCons interfere with each other +when they start changing directory. + .RE Any additional keyword arguments supplied when a Builder object is created @@ -5694,9 +5892,18 @@ a = Action(build_it) If the action argument is not one of the above, None is returned. +.PP The second, optional argument -is a Python function that returns +is used to define the output which is printed +when the Action is actually performed. +In the absence of this parameter, or if it's an +empty string, a default output depending on the type of the action +is used. For example, a command-line action will print +the executed command. The argument is either a python function +or a string. + +In the first case, it's a function that returns a string to be printed to describe the action being executed. Like a function to build a file, this function takes three arguments: @@ -5713,6 +5920,13 @@ and .B source arguments may be lists of Node objects if there is more than one target file or source file. + +In the second case, you provide the string itself. +The string typically contains variables, notably +$TARGET(S) and $SOURCE(S), or consists of just a single +variable, which is optionally defined somewhere else. +SCons itself heavily uses the latter variant. + Examples: .ES @@ -5724,10 +5938,15 @@ def string_it(target, source, env): return "building '%s' from '%s'" % (target[0], source[0]) # Use a positional argument. -a = Action(build_it, string_it) +f = Action(build_it, string_it) +s = Action(build_it, "building '$TARGET' from '$SOURCE'") # Alternatively, use a keyword argument. -a = Action(build_it, strfunction=string_it) +f = Action(build_it, strfunction=string_it) +s = Action(build_it, cmdstr="building '$TARGET' from '$SOURCE'") + +# You can provide a configurable variable. +l = Action(build_it, '$STRINGIT') .EE The third, also optional argument @@ -5748,16 +5967,12 @@ def build_it(target, source, env): open(target[0], 'w').write(env['XXX']) return 0 -def string_it(target, source): - return "building '%s' from '%s'" % (target[0], source[0]) - # Use positional arguments. -a = Action(build_it, string_it, ['XXX']) +a = Action(build_it, '$STRINGIT', ['XXX']) # Alternatively, use a keyword argument. a = Action(build_it, varlist=['XXX']) .EE -.PP The .BR Action () @@ -5890,7 +6105,7 @@ that env = Environment(TMPBUILD = '/tmp/builddir') env.Command('foo.out', 'foo.in', [Mkdir('$TMPBUILD'), - Copy('$TMPBUILD', '${SOURCE.dir}') + Copy('$TMPBUILD', '${SOURCE.dir}'), "cd $TMPBUILD && make", Delete('$TMPBUILD')]) .EE @@ -6301,6 +6516,13 @@ Use to fetch the name of the file, and .B node.get_contents() to fetch contents of the file. +Note that the file is +.I not +guaranteed to exist before the scanner is called, +so the scanner function should check that +if there's any chance that the scanned file +might not exist +(for example, if it's built from other files). The .B env @@ -6348,10 +6570,14 @@ then it will be expanded into a list by the current environment. .IP path_function -A Python function that takes -two or three arguments: -a construction environment, directory Node, -and optional argument supplied +A Python function that takes four or five arguments: +a construction environment, +a Node for the directory containing +the SConscript file in which +the first target was defined, +a list of target nodes, +a list of source nodes, +and an optional argument supplied when the scanner was created. The .B path_function diff --git a/rpm/scons.spec.in b/rpm/scons.spec.in index dc6ec03e..92330d58 100644 --- a/rpm/scons.spec.in +++ b/rpm/scons.spec.in @@ -1,5 +1,5 @@ %define name scons -%define version 0.96 +%define version 0.96.92 %define release 1 Summary: an Open Source software construction tool diff --git a/runtest.py b/runtest.py index e4042d99..7f5cbc3c 100644 --- a/runtest.py +++ b/runtest.py @@ -8,7 +8,7 @@ # directories to test the SCons modules. # # By default, it directly uses the modules in the local tree: -# ./src/ (source files we ship) and ./etc/ (other modules we don't). +# ./src/ (source files we ship) and ./QMTest/ (other modules we don't). # # HOWEVER, now that SCons has Repository support, we don't have # Aegis copy all of the files into the local tree. So if you're @@ -62,6 +62,10 @@ # command line it will execute before # executing it. This suppresses that print. # +# --sp The Aegis search path. +# +# --spe The Aegis executable search path. +# # -t Print the execution time of each test. # # -X The scons "script" is an executable; don't @@ -93,6 +97,7 @@ import sys import time all = 0 +baseline = 0 debug = '' execute_tests = 1 format = None @@ -104,14 +109,13 @@ print_passed_summary = None scons = None scons_exec = None outputfile = None +qmtest = None testlistfile = None version = '' -print_time = lambda fmt, time: None - -if os.name == 'java': - python = os.path.join(sys.prefix, 'jython') -else: - python = sys.executable +print_times = None +python = None +sp = None +spe = None cwd = os.getcwd() @@ -127,6 +131,7 @@ Usage: runtest.py [OPTIONS] [TEST ...] Options: -a, --all Run all tests. --aegis Print results in Aegis format. + -b BASE, --baseline BASE Run test scripts against baseline BASE. -d, --debug Run test scripts under the Python debugger. -f FILE, --file FILE Run tests in specified FILE. -h, --help Print this message and exit. @@ -145,7 +150,10 @@ Options: tar-gz .tar.gz distribution zip .zip distribution --passed Summarize which tests passed. + --qmtest Run using the QMTest harness. -q, --quiet Don't print the test being executed. + --sp PATH The Aegis search path. + --spe PATH The Aegis executable search path. -t, --time Print test execution time. -v version Specify the SCons version. --verbose=LEVEL Set verbose level: 1 = print executed commands, @@ -156,17 +164,20 @@ Options: --xml Print results in SCons XML format. """ -opts, args = getopt.getopt(sys.argv[1:], "adf:hlno:P:p:qv:Xx:t", - ['all', 'aegis', +opts, args = getopt.getopt(sys.argv[1:], "ab:df:hlno:P:p:qv:Xx:t", + ['all', 'aegis', 'baseline=', 'debug', 'file=', 'help', 'list', 'no-exec', 'output=', - 'package=', 'passed', 'python=', 'quiet', + 'package=', 'passed', 'python=', + 'qmtest', 'quiet', 'spe=', 'version=', 'exec=', 'time', 'verbose=', 'xml']) for o, a in opts: if o in ['-a', '--all']: all = 1 + elif o in ['-b', '--baseline']: + baseline = a elif o in ['-d', '--debug']: debug = os.path.join(lib_dir, "pdb.py") elif o in ['-f', '--file']: @@ -190,10 +201,16 @@ for o, a in opts: print_passed_summary = 1 elif o in ['-P', '--python']: python = a + elif o in ['--qmtest']: + qmtest = 1 elif o in ['-q', '--quiet']: printcommand = 0 + elif o in ['--sp']: + sp = string.split(a, os.pathsep) + elif o in ['--spe']: + spe = string.split(a, os.pathsep) elif o in ['-t', '--time']: - print_time = lambda fmt, time: sys.stdout.write(fmt % time) + print_times = 1 elif o in ['--verbose']: os.environ['TESTCMD_VERBOSE'] = a elif o in ['-v', '--version']: @@ -219,14 +236,21 @@ def whereis(file): aegis = whereis('aegis') -sp = [] +if format == '--aegis' and aegis: + change = os.popen("aesub '$c' 2>/dev/null", "r").read() + if change: + if sp is None: + paths = os.popen("aesub '$sp' 2>/dev/null", "r").read()[:-1] + sp = string.split(paths, os.pathsep) + if spe is None: + spe = os.popen("aesub '$spe' 2>/dev/null", "r").read()[:-1] + spe = string.split(spe, os.pathsep) + else: + aegis = None -if aegis: - paths = os.popen("aesub '$sp' 2>/dev/null", "r").read()[:-1] - sp.extend(string.split(paths, os.pathsep)) - spe = os.popen("aesub '$spe' 2>/dev/null", "r").read()[:-1] - spe = string.split(spe, os.pathsep) -else: +if sp is None: + sp = [] +if spe is None: spe = [] sp.append(cwd) @@ -330,7 +354,9 @@ format_class = { } Test = format_class[format] -if args: +if qmtest: + pass +elif args: if spe: for a in args: if os.path.isabs(a): @@ -376,7 +402,7 @@ elif all: tdict[t] = Test(t) os.path.walk('test', find_py, 0) - if aegis: + if format == '--aegis' and aegis: cmd = "aegis -list -unf pf 2>/dev/null" for line in os.popen(cmd, "r").readlines(): a = string.split(line) @@ -469,9 +495,43 @@ else: # spe = map(lambda x: os.path.join(x, 'src', 'engine'), spe) # ld = string.join(spe, os.pathsep) - scons_script_dir = sd or os.path.join(cwd, 'src', 'script') + if not baseline or baseline == '.': + base = cwd + elif baseline == '-': + # Tentative code for fetching information directly from the + # QMTest context file. + # + #import qm.common + #import qm.test.context + #qm.rc.Load("test") + #context = qm.test.context.Context() + #context.Read('context') + + url = None + svn_info = os.popen("svn info 2>&1", "r").read() + match = re.search('URL: (.*)', svn_info) + if match: + url = match.group(1) + if not url: + sys.stderr.write('runtest.py: could not find a URL:\n') + sys.stderr.write(svn_info) + sys.exit(1) + import tempfile + base = tempfile.mkdtemp(prefix='runtest-tmp-') + + command = 'cd %s && svn co -q %s' % (base, url) + + base = os.path.join(base, os.path.split(url)[1]) + if printcommand: + print command + if execute_tests: + os.system(command) + else: + base = baseline - scons_lib_dir = ld or os.path.join(cwd, 'src', 'engine') + scons_script_dir = sd or os.path.join(base, 'src', 'script') + + scons_lib_dir = ld or os.path.join(base, 'src', 'engine') pythonpath_dir = scons_lib_dir @@ -496,9 +556,16 @@ os.environ['SCONS_VERSION'] = version old_pythonpath = os.environ.get('PYTHONPATH') pythonpaths = [ pythonpath_dir ] -for p in sp: - pythonpaths.append(os.path.join(p, 'build', 'etc')) - pythonpaths.append(os.path.join(p, 'etc')) + +for dir in sp: + if format == '--aegis': + q = os.path.join(dir, 'build', 'QMTest') + else: + q = os.path.join(dir, 'QMTest') + pythonpaths.append(q) + +os.environ['SCONS_SOURCE_PATH_EXECUTABLE'] = string.join(spe, os.pathsep) + os.environ['PYTHONPATH'] = string.join(pythonpaths, os.pathsep) if old_pythonpath: @@ -506,10 +573,49 @@ if old_pythonpath: os.pathsep + \ old_pythonpath -try: - os.chdir(scons_script_dir) -except OSError: - pass +if qmtest: + if baseline: + result_stream = 'AegisBaselineStream' + qmr_file = 'baseline.qmr' + else: + result_stream = 'AegisChangeStream' + qmr_file = 'results.qmr' + #qmtest = r'D:\Applications\python23\scripts\qmtest.py' + qmtest = 'qmtest.py' + qmtest_args = [ + qmtest, + 'run', + '--output %s' % qmr_file, + '--format none', + '--result-stream=scons_tdb.%s' % result_stream, + ] + + if python: + qmtest_args.append('--context python=%s' % python) + + if print_times: + qmtest_args.append('--context print_time=1') + + if outputfile: + #rs = '--result-stream=scons_tdb.AegisBatchStream(results_file=\\"%s\\")' % outputfile + rs = '\'--result-stream=scons_tdb.AegisBatchStream(results_file="%s")\'' % outputfile + qmtest_args.append(rs) + + os.environ['SCONS'] = os.path.join(cwd, 'src', 'script', 'scons.py') + + cmd = string.join(qmtest_args + args, ' ') + if printcommand: + sys.stdout.write(cmd + '\n') + sys.stdout.flush() + status = 0 + if execute_tests: + status = os.system(cmd) + sys.exit(status) + +#try: +# os.chdir(scons_script_dir) +#except OSError: +# pass class Unbuffered: def __init__(self, file): @@ -527,9 +633,27 @@ if list_only: sys.stdout.write(t.abspath + "\n") sys.exit(0) +# +if not python: + if os.name == 'java': + python = os.path.join(sys.prefix, 'jython') + else: + python = sys.executable + # time.clock() is the suggested interface for doing benchmarking timings, # but time.time() does a better job on Linux systems, so let that be # the non-Windows default. + +if print_times: + print_time_func = lambda fmt, time: sys.stdout.write(fmt % time) +else: + print_time_func = lambda fmt, time: None + +if sys.platform == 'win32': + time_func = time.clock +else: + time_func = time.time + if sys.platform == 'win32': time_func = time.clock else: @@ -548,10 +672,10 @@ for t in tests: if execute_tests: t.execute() t.test_time = time_func() - test_start_time - print_time("Test execution time: %.1f seconds\n", t.test_time) + print_time_func("Test execution time: %.1f seconds\n", t.test_time) if len(tests) > 0: tests[0].total_time = time_func() - total_start_time - print_time("Total execution time for all tests: %.1f seconds\n", tests[0].total_time) + print_time_func("Total execution time for all tests: %.1f seconds\n", tests[0].total_time) passed = filter(lambda t: t.status == 0, tests) fail = filter(lambda t: t.status == 1, tests) diff --git a/src/CHANGES.txt b/src/CHANGES.txt index 0df16969..b3eb4848 100644 --- a/src/CHANGES.txt +++ b/src/CHANGES.txt @@ -10,6 +10,101 @@ RELEASE 0.97 - XXX + From Ken Boortz: + + - Enhance ParseConfig() to recognize options that begin with '+'. + + From Christopher Drexler: + + - Make SCons aware bibtex must be called if any \include files + cause creation of a bibliography. + + - Make SCons aware that "\bilbiography" in TeX source files means + that related .bbl and .blg bibliography files will be created. + (NOTE: This still needs to search for the string in \include files.) + + From David Gruener: + + - Fix inconsistent handling of Action strfunction arguments. + + - Preserve white space in display Action strfunction strings. + + From Steven Knight: + + - Speed up the Taskmaster significantly by avoiding unnecessary + re-scans of Nodes to find out if there's work to be done, having it + track the currently-executed top-level target directly and not + through its presence on the target list, and eliminating some other + minor list(s), method(s) and manipulation. + + - Fix the expansion of $TARGET and $SOURCE in the expansion of + $INSTALLSTR displayed for non-environment calls to InstallAs(). + + - Fix the ability to have an Alias() call refer to a directory + name that's not identified as a directory until later. + + - Enhance runtest.py with an option to use QMTest as the harness. + This will become the default behavior as we add more functionality + to the QMTest side. + + From Sanjoy Mahajan: + + - Change use of $SOURCES to $SOURCE in all TeX-related Tool modules. + + From Joel B. Mohler: + + - Make SCons aware that "\makeindex" in TeX source files means that + related .ilg, .ind and .idx index files will be created. + (NOTE: This still needs to search for the string in \include files.) + + - Prevent scanning the TeX .aux file for additional files from + trying to remove it twice when the -c option is used. + + From Greg Noel: + + - Add an env.ParseFlags() method that provides separate logic for + parsing GNU tool chain flags into a dictionary. + + - Add an env.MergeFlags() method to apply an arbitrary dictionary + of flags to a construction environment's variables. + + From Gary Oberbrunner: + + - Fix parsing tripartite Intel C compiler version numbers on Linux. + + - Extend the ParseConfig() function to recognize -arch and + -isysroot options. + + - Have the error message list the known suffixes when a Builder call + can't build a source file with an unknown suffix. + + From Karol Pietrzak: + + - Avoid recursive calls to main() in the program snippet used by the + SConf subsystem to test linking against libraries. This changes the + default behavior of CheckLib() and CheckLibWithHeader() to print + "Checking for C library foo..." instead of "Checking for main() + in C library foo...". + + From John Pye: + + - Throw an exception if a command called by ParseConfig() or + ParseFlags() returns an error. + + From Stefan Seefeld: + + - Initial infrastructure for running SCons tests under QMTest. + + From Atul Varma: + + - Fix detection of Visual C++ Express Edition. + + + +RELEASE 0.96.92 - Mon, 10 Apr 2006 21:08:22 -0400 + + NOTE: This was a pre-release of 0.97 for testing purposes. + From Anonymous: - Fix the intelc.py Tool module to not throw an exception if the diff --git a/src/README.txt b/src/README.txt index c821e941..a8969004 100644 --- a/src/README.txt +++ b/src/README.txt @@ -88,8 +88,7 @@ provided Python-standard setup script as follows: By default, the above command will do the following: - -- Install the version-numbered "scons-__VERSION__" and - "sconsign-__VERSION__" + -- Install the version-numbered "scons-__VERSION__" and "sconsign-__VERSION__" scripts in the default system script directory (/usr/bin or C:\Python*\Scripts, for example). This can be disabled by specifying the "--no-version-script" option on the command @@ -263,8 +262,12 @@ With plenty of help from the SCons Development team: Chad Austin Charles Crain Steve Leblanc + Baptiste Lepilleur + Elliot Murphy Gary Oberbrunner Anthony Roach + Greg Noel + Kevin Quick Greg Spencer Christoph Wiedemann diff --git a/src/RELEASE.txt b/src/RELEASE.txt index aefe6ea6..b5c03744 100644 --- a/src/RELEASE.txt +++ b/src/RELEASE.txt @@ -20,7 +20,7 @@ more effectively, please sign up for the scons-users mailing list at: -RELEASE 0.96.92 - XXX +RELEASE 0.96.92 - Mon, 10 Apr 2006 21:08:22 -0400 This is a pre-release for testing the eighth beta release of SCons. Please consult the CHANGES.txt file for a list of specific changes diff --git a/src/engine/SCons/Action.py b/src/engine/SCons/Action.py index 4014dea9..4576164a 100644 --- a/src/engine/SCons/Action.py +++ b/src/engine/SCons/Action.py @@ -372,6 +372,16 @@ class CommandAction(_ActionAction): # Environment.subst_list() for substituting environment # variables. if __debug__: logInstanceCreation(self, 'Action.CommandAction') + + if not cmdstr is None: + if callable(cmdstr): + args = (cmdstr,)+args + elif not SCons.Util.is_String(cmdstr): + raise SCons.Errors.UserError(\ + 'Invalid command display variable type. ' \ + 'You must either pass a string or a callback which ' \ + 'accepts (target, source, env) as parameters.') + apply(_ActionAction.__init__, (self,)+args, kw) if SCons.Util.is_List(cmd): if filter(SCons.Util.is_List, cmd): @@ -405,7 +415,7 @@ class CommandAction(_ActionAction): def strfunction(self, target, source, env): if not self.cmdstr is None: - c = env.subst(self.cmdstr, 0, target, source) + c = env.subst(self.cmdstr, SCons.Subst.SUBST_RAW, target, source) if c: return c cmd_list, ignore, silent = self.process(target, source, env) @@ -492,6 +502,7 @@ class CommandGeneratorAction(ActionBase): def __init__(self, generator, *args, **kw): if __debug__: logInstanceCreation(self, 'Action.CommandGeneratorAction') self.generator = generator + self.gen_args = args self.gen_kw = kw def _generate(self, target, source, env, for_signature): @@ -501,7 +512,7 @@ class CommandGeneratorAction(ActionBase): target = [target] ret = self.generator(target=target, source=source, env=env, for_signature=for_signature) - gen_cmd = apply(Action, (ret,), self.gen_kw) + gen_cmd = apply(Action, (ret,)+self.gen_args, self.gen_kw) if not gen_cmd: raise SCons.Errors.UserError("Object returned from command generator: %s cannot be used to create an Action." % repr(ret)) return gen_cmd @@ -559,6 +570,7 @@ class LazyAction(CommandGeneratorAction, CommandAction): if __debug__: logInstanceCreation(self, 'Action.LazyAction') apply(CommandAction.__init__, (self, '$'+var)+args, kw) self.var = SCons.Util.to_String(var) + self.gen_args = args self.gen_kw = kw def get_parent_class(self, env): @@ -570,7 +582,7 @@ class LazyAction(CommandGeneratorAction, CommandAction): def _generate_cache(self, env): """__cacheable__""" c = env.get(self.var, '') - gen_cmd = apply(Action, (c,), self.gen_kw) + gen_cmd = apply(Action, (c,)+self.gen_args, self.gen_kw) if not gen_cmd: raise SCons.Errors.UserError("$%s value %s cannot be used to create an Action." % (self.var, repr(c))) return gen_cmd @@ -599,11 +611,22 @@ if not SCons.Memoize.has_metaclass: class FunctionAction(_ActionAction): """Class for Python function actions.""" - def __init__(self, execfunction, *args, **kw): + def __init__(self, execfunction, cmdstr=_null, *args, **kw): if __debug__: logInstanceCreation(self, 'Action.FunctionAction') + + if not cmdstr is _null: + if callable(cmdstr): + args = (cmdstr,)+args + elif not (cmdstr is None or SCons.Util.is_String(cmdstr)): + raise SCons.Errors.UserError(\ + 'Invalid function display variable type. ' \ + 'You must either pass a string or a callback which ' \ + 'accepts (target, source, env) as parameters.') + self.execfunction = execfunction apply(_ActionAction.__init__, (self,)+args, kw) self.varlist = kw.get('varlist', []) + self.cmdstr = cmdstr def function_name(self): try: @@ -615,6 +638,12 @@ class FunctionAction(_ActionAction): return "unknown_python_function" def strfunction(self, target, source, env): + if self.cmdstr is None: + return None + if not self.cmdstr is _null: + c = env.subst(self.cmdstr, SCons.Subst.SUBST_RAW, target, source) + if c: + return c def array(a): def quote(s): return '"' + str(s) + '"' diff --git a/src/engine/SCons/ActionTests.py b/src/engine/SCons/ActionTests.py index 63e86de7..1085586e 100644 --- a/src/engine/SCons/ActionTests.py +++ b/src/engine/SCons/ActionTests.py @@ -688,6 +688,18 @@ class CommandActionTestCase(unittest.TestCase): assert a.cmd_list == [ "abra" ], a.cmd_list assert a.cmdstr == "cadabra", a.cmdstr + def test_bad_cmdstr(self): + """Test handling of bad CommandAction(cmdstr) arguments + """ + try: + a = SCons.Action.CommandAction('foo', []) + except SCons.Errors.UserError, e: + s = str(e) + m = 'Invalid command display variable' + assert string.find(s, m) != -1, 'Unexpected string: %s' % s + else: + raise "did not catch expected UserError" + def test___str__(self): """Test fetching the pre-substitution string for command Actions """ @@ -760,7 +772,7 @@ class CommandActionTestCase(unittest.TestCase): act = SCons.Action.CommandAction('xyzzy $TARGET $SOURCE', 'cmdstr - $SOURCE - $TARGET -') s = act.strfunction([], [], env) - assert s == 'cmdstr - - -', s + assert s == 'cmdstr - - -', s s = act.strfunction([t1], [s1], env) assert s == 'cmdstr - s1 - t1 -', s s = act.strfunction([t1, t2], [s1, s2], env) @@ -777,7 +789,7 @@ class CommandActionTestCase(unittest.TestCase): act = SCons.Action.CommandAction('xyzzy $TARGETS $SOURCES', 'cmdstr = $SOURCES = $TARGETS =') s = act.strfunction([], [], env) - assert s == 'cmdstr = = =', s + assert s == 'cmdstr = = =', s s = act.strfunction([t1], [s1], env) assert s == 'cmdstr = s1 = t1 =', s s = act.strfunction([t1, t2], [s1, s2], env) @@ -793,6 +805,16 @@ class CommandActionTestCase(unittest.TestCase): s = act.strfunction([t1, t2], [s1, s2], env) assert s == 'xyzzy t1 s1 t1 t2 s1 s2', s + act = SCons.Action.CommandAction('xyzzy $TARGETS $SOURCES', + 'cmdstr\t$TARGETS\n$SOURCES ') + + s = act.strfunction([], [], env) + assert s == 'cmdstr\t\n ', s + s = act.strfunction([t1], [s1], env) + assert s == 'cmdstr\tt1\ns1 ', s + s = act.strfunction([t1, t2], [s1, s2], env) + assert s == 'cmdstr\tt1 t2\ns1 s2 ', s + def sf(target, source, env): return "sf was called" act = SCons.Action.CommandAction('foo', strfunction=sf) @@ -1337,6 +1359,20 @@ class FunctionActionTestCase(unittest.TestCase): assert a.execfunction == func2, a.execfunction assert a.strfunction == func3, a.strfunction + def test_cmdstr_bad(self): + """Test handling of bad FunctionAction(cmdstr) arguments + """ + def func(): + pass + try: + a = SCons.Action.FunctionAction(func, []) + except SCons.Errors.UserError, e: + s = str(e) + m = 'Invalid function display variable' + assert string.find(s, m) != -1, 'Unexpected string: %s' % s + else: + raise "did not catch expected UserError" + def test___str__(self): """Test the __str__() method for function Actions """ @@ -1470,6 +1506,24 @@ class FunctionActionTestCase(unittest.TestCase): c = a.get_contents(target=[], source=[], env=Environment()) assert c in matches, repr(c) + def test_strfunction(self): + """Test the FunctionAction.strfunction() method + """ + def func(): + pass + + a = SCons.Action.FunctionAction(func) + s = a.strfunction(target=[], source=[], env=Environment()) + assert s == 'func([], [])', s + + a = SCons.Action.FunctionAction(func, None) + s = a.strfunction(target=[], source=[], env=Environment()) + assert s is None, s + + a = SCons.Action.FunctionAction(func, 'function') + s = a.strfunction(target=[], source=[], env=Environment()) + assert s == 'function', s + class ListActionTestCase(unittest.TestCase): def test___init__(self): diff --git a/src/engine/SCons/Builder.py b/src/engine/SCons/Builder.py index ae24f43f..16f11915 100644 --- a/src/engine/SCons/Builder.py +++ b/src/engine/SCons/Builder.py @@ -143,6 +143,10 @@ class DictCmdGenerator(SCons.Util.Selector): to return the proper action based on the file suffix of the source file.""" + def __init__(self, dict=None, source_ext_match=1): + SCons.Util.Selector.__init__(self, dict) + self.source_ext_match = source_ext_match + def src_suffixes(self): return self.keys() @@ -155,12 +159,15 @@ class DictCmdGenerator(SCons.Util.Selector): if not source: return [] - ext = None - for src in map(str, source): - my_ext = SCons.Util.splitext(src)[1] - if ext and my_ext != ext: - raise UserError("While building `%s' from `%s': Cannot build multiple sources with different extensions: %s, %s" % (repr(map(str, target)), src, ext, my_ext)) - ext = my_ext + if self.source_ext_match: + ext = None + for src in map(str, source): + my_ext = SCons.Util.splitext(src)[1] + if ext and my_ext != ext: + raise UserError("While building `%s' from `%s': Cannot build multiple sources with different extensions: %s, %s" % (repr(map(str, target)), src, ext, my_ext)) + ext = my_ext + else: + ext = SCons.Util.splitext(str(source[0]))[1] if not ext: raise UserError("While building `%s': Cannot deduce file extension from source files: %s" % (repr(map(str, target)), repr(map(str, source)))) @@ -170,7 +177,8 @@ class DictCmdGenerator(SCons.Util.Selector): except KeyError, e: raise UserError("Ambiguous suffixes after environment substitution: %s == %s == %s" % (e[0], e[1], e[2])) if ret is None: - raise UserError("While building `%s': Don't know how to build a file with suffix `%s'." % (repr(map(str, target)), ext)) + raise UserError("While building `%s' from `%s': Don't know how to build from a source file with suffix `%s'. Expected a suffix in this list: %s." % \ + (repr(map(str, target)), repr(map(str, source)), ext, repr(self.keys()))) return ret class CallableSelector(SCons.Util.Selector): @@ -249,8 +257,11 @@ def Builder(**kw): kw['action'] = SCons.Action.CommandGeneratorAction(kw['generator']) del kw['generator'] elif kw.has_key('action'): + source_ext_match = kw.get('source_ext_match', 1) + if kw.has_key('source_ext_match'): + del kw['source_ext_match'] if SCons.Util.is_Dict(kw['action']): - composite = DictCmdGenerator(kw['action']) + composite = DictCmdGenerator(kw['action'], source_ext_match) kw['action'] = SCons.Action.CommandGeneratorAction(composite) kw['src_suffix'] = composite.src_suffixes() else: @@ -538,9 +549,6 @@ class BuilderBase: tlist = env.arg2nodes(target, target_factory) slist = env.arg2nodes(source, source_factory) - tlist = map(lambda n: n.disambiguate(), tlist) - slist = map(lambda n: n.disambiguate(), slist) - return tlist, slist def _execute(self, env, target, source, overwarn={}, executor_kw={}): diff --git a/src/engine/SCons/BuilderTests.py b/src/engine/SCons/BuilderTests.py index c80f00c6..c5b428c5 100644 --- a/src/engine/SCons/BuilderTests.py +++ b/src/engine/SCons/BuilderTests.py @@ -820,145 +820,6 @@ class BuilderTestCase(unittest.TestCase): assert s == ['test_wrap.c'], s s = map(str, tgt.sources[0].sources[0].sources) assert s == ['test.i'], s - - def test_CompositeBuilder(self): - """Testing CompositeBuilder class.""" - def func_action(target, source, env): - return 0 - - env = Environment(BAR_SUFFIX = '.BAR2', FOO_SUFFIX = '.FOO2') - builder = SCons.Builder.Builder(action={ '.foo' : func_action, - '.bar' : func_action, - '$BAR_SUFFIX' : func_action, - '$FOO_SUFFIX' : func_action }) - - tgt = builder(env, source=[]) - assert tgt == [], tgt - - assert isinstance(builder, SCons.Builder.CompositeBuilder) - assert isinstance(builder.action, SCons.Action.CommandGeneratorAction) - - tgt = builder(env, target='test1', source='test1.foo')[0] - assert isinstance(tgt.builder, SCons.Builder.BuilderBase) - assert tgt.builder.action is builder.action - - tgt = builder(env, target='test2', source='test1.bar')[0] - assert isinstance(tgt.builder, SCons.Builder.BuilderBase) - assert tgt.builder.action is builder.action - - flag = 0 - tgt = builder(env, target='test3', source=['test2.bar', 'test1.foo'])[0] - try: - tgt.build() - except SCons.Errors.UserError, e: - flag = 1 - assert flag, "UserError should be thrown when we build targets with files of different suffixes." - match = str(e) == "While building `['test3']' from `test1.foo': Cannot build multiple sources with different extensions: .bar, .foo" - assert match, e - - tgt = builder(env, target='test4', source=['test4.BAR2'])[0] - assert isinstance(tgt.builder, SCons.Builder.BuilderBase) - try: - tgt.build() - flag = 1 - except SCons.Errors.UserError, e: - print e - flag = 0 - assert flag, "It should be possible to define actions in composite builders using variables." - env['FOO_SUFFIX'] = '.BAR2' - builder.add_action('$NEW_SUFFIX', func_action) - flag = 0 - tgt = builder(env, target='test5', source=['test5.BAR2'])[0] - try: - tgt.build() - except SCons.Errors.UserError: - flag = 1 - assert flag, "UserError should be thrown when we build targets with ambigous suffixes." - del env.d['FOO_SUFFIX'] - del env.d['BAR_SUFFIX'] - - foo_bld = SCons.Builder.Builder(action = 'a-foo', - src_suffix = '.ina', - suffix = '.foo') - assert isinstance(foo_bld, SCons.Builder.BuilderBase) - builder = SCons.Builder.Builder(action = { '.foo' : 'foo', - '.bar' : 'bar' }, - src_builder = foo_bld) - assert isinstance(builder, SCons.Builder.CompositeBuilder) - assert isinstance(builder.action, SCons.Action.CommandGeneratorAction) - - tgt = builder(env, target='t1', source='t1a.ina t1b.ina')[0] - assert isinstance(tgt.builder, SCons.Builder.BuilderBase) - - tgt = builder(env, target='t2', source='t2a.foo t2b.ina')[0] - assert isinstance(tgt.builder, SCons.Builder.MultiStepBuilder), tgt.builder.__dict__ - - bar_bld = SCons.Builder.Builder(action = 'a-bar', - src_suffix = '.inb', - suffix = '.bar') - assert isinstance(bar_bld, SCons.Builder.BuilderBase) - builder = SCons.Builder.Builder(action = { '.foo' : 'foo'}, - src_builder = [foo_bld, bar_bld]) - assert isinstance(builder, SCons.Builder.CompositeBuilder) - assert isinstance(builder.action, SCons.Action.CommandGeneratorAction) - - builder.add_action('.bar', 'bar') - - tgt = builder(env, target='t3-foo', source='t3a.foo t3b.ina')[0] - assert isinstance(tgt.builder, SCons.Builder.MultiStepBuilder) - - tgt = builder(env, target='t3-bar', source='t3a.bar t3b.inb')[0] - assert isinstance(tgt.builder, SCons.Builder.MultiStepBuilder) - - flag = 0 - tgt = builder(env, target='t5', source=['test5a.foo', 'test5b.inb'])[0] - try: - tgt.build() - except SCons.Errors.UserError, e: - flag = 1 - assert flag, "UserError should be thrown when we build targets with files of different suffixes." - match = str(e) == "While building `['t5']' from `test5b.bar': Cannot build multiple sources with different extensions: .foo, .bar" - assert match, e - - flag = 0 - tgt = builder(env, target='t6', source=['test6a.bar', 'test6b.ina'])[0] - try: - tgt.build() - except SCons.Errors.UserError, e: - flag = 1 - assert flag, "UserError should be thrown when we build targets with files of different suffixes." - match = str(e) == "While building `['t6']' from `test6b.foo': Cannot build multiple sources with different extensions: .bar, .foo" - assert match, e - - flag = 0 - tgt = builder(env, target='t4', source=['test4a.ina', 'test4b.inb'])[0] - try: - tgt.build() - except SCons.Errors.UserError, e: - flag = 1 - assert flag, "UserError should be thrown when we build targets with files of different suffixes." - match = str(e) == "While building `['t4']' from `test4b.bar': Cannot build multiple sources with different extensions: .foo, .bar" - assert match, e - - flag = 0 - tgt = builder(env, target='t7', source=['test7'])[0] - try: - tgt.build() - except SCons.Errors.UserError, e: - flag = 1 - assert flag, "UserError should be thrown when we build targets with files of different suffixes." - match = str(e) == "While building `['t7']': Cannot deduce file extension from source files: ['test7']" - assert match, e - - flag = 0 - tgt = builder(env, target='t8', source=['test8.unknown'])[0] - try: - tgt.build() - except SCons.Errors.UserError, e: - flag = 1 - assert flag, "UserError should be thrown when we build targets with files of different suffixes." - match = str(e) == "While building `['t8']': Don't know how to build a file with suffix `.unknown'." - assert match, e def test_target_scanner(self): """Testing ability to set target and source scanners through a builder.""" @@ -1554,7 +1415,191 @@ class BuilderTestCase(unittest.TestCase): tgt = b4(env, target = 'moo', source='cow') assert tgt[0].builder.get_name(env) == 'bldr4' +class CompositeBuilderTestCase(unittest.TestCase): + + def setUp(self): + def func_action(target, source, env): + return 0 + + builder = SCons.Builder.Builder(action={ '.foo' : func_action, + '.bar' : func_action}) + + self.func_action = func_action + self.builder = builder + + def test___init__(self): + """Test CompositeBuilder creation""" + env = Environment() + builder = SCons.Builder.Builder(action={}) + + tgt = builder(env, source=[]) + assert tgt == [], tgt + + assert isinstance(builder, SCons.Builder.CompositeBuilder) + assert isinstance(builder.action, SCons.Action.CommandGeneratorAction) + + def test_target_action(self): + """Test CompositeBuilder setting of target builder actions""" + env = Environment() + builder = self.builder + + tgt = builder(env, target='test1', source='test1.foo')[0] + assert isinstance(tgt.builder, SCons.Builder.BuilderBase) + assert tgt.builder.action is builder.action + + tgt = builder(env, target='test2', source='test1.bar')[0] + assert isinstance(tgt.builder, SCons.Builder.BuilderBase) + assert tgt.builder.action is builder.action + + def test_multiple_suffix_error(self): + """Test the CompositeBuilder multiple-source-suffix error""" + env = Environment() + builder = self.builder + + flag = 0 + tgt = builder(env, target='test3', source=['test2.bar', 'test1.foo'])[0] + try: + tgt.build() + except SCons.Errors.UserError, e: + flag = 1 + assert flag, "UserError should be thrown when we build targets with files of different suffixes." + expect = "While building `['test3']' from `test1.foo': Cannot build multiple sources with different extensions: .bar, .foo" + assert str(e) == expect, e + + def test_source_ext_match(self): + """Test the CompositeBuilder source_ext_match argument""" + env = Environment() + func_action = self.func_action + builder = SCons.Builder.Builder(action={ '.foo' : func_action, + '.bar' : func_action}, + source_ext_match = None) + + tgt = builder(env, target='test3', source=['test2.bar', 'test1.foo'])[0] + tgt.build() + + def test_suffix_variable(self): + """Test CompositeBuilder defining action suffixes through a variable""" + env = Environment(BAR_SUFFIX = '.BAR2', FOO_SUFFIX = '.FOO2') + func_action = self.func_action + builder = SCons.Builder.Builder(action={ '.foo' : func_action, + '.bar' : func_action, + '$BAR_SUFFIX' : func_action, + '$FOO_SUFFIX' : func_action }) + + tgt = builder(env, target='test4', source=['test4.BAR2'])[0] + assert isinstance(tgt.builder, SCons.Builder.BuilderBase) + try: + tgt.build() + flag = 1 + except SCons.Errors.UserError, e: + print e + flag = 0 + assert flag, "It should be possible to define actions in composite builders using variables." + env['FOO_SUFFIX'] = '.BAR2' + builder.add_action('$NEW_SUFFIX', func_action) + flag = 0 + tgt = builder(env, target='test5', source=['test5.BAR2'])[0] + try: + tgt.build() + except SCons.Errors.UserError: + flag = 1 + assert flag, "UserError should be thrown when we build targets with ambigous suffixes." + + def test_src_builder(self): + """Test CompositeBuilder's use of a src_builder""" + env = Environment() + + foo_bld = SCons.Builder.Builder(action = 'a-foo', + src_suffix = '.ina', + suffix = '.foo') + assert isinstance(foo_bld, SCons.Builder.BuilderBase) + builder = SCons.Builder.Builder(action = { '.foo' : 'foo', + '.bar' : 'bar' }, + src_builder = foo_bld) + assert isinstance(builder, SCons.Builder.CompositeBuilder) + assert isinstance(builder.action, SCons.Action.CommandGeneratorAction) + + tgt = builder(env, target='t1', source='t1a.ina t1b.ina')[0] + assert isinstance(tgt.builder, SCons.Builder.BuilderBase) + + tgt = builder(env, target='t2', source='t2a.foo t2b.ina')[0] + assert isinstance(tgt.builder, SCons.Builder.MultiStepBuilder), tgt.builder.__dict__ + + bar_bld = SCons.Builder.Builder(action = 'a-bar', + src_suffix = '.inb', + suffix = '.bar') + assert isinstance(bar_bld, SCons.Builder.BuilderBase) + builder = SCons.Builder.Builder(action = { '.foo' : 'foo'}, + src_builder = [foo_bld, bar_bld]) + assert isinstance(builder, SCons.Builder.CompositeBuilder) + assert isinstance(builder.action, SCons.Action.CommandGeneratorAction) + + builder.add_action('.bar', 'bar') + + tgt = builder(env, target='t3-foo', source='t3a.foo t3b.ina')[0] + assert isinstance(tgt.builder, SCons.Builder.MultiStepBuilder) + + tgt = builder(env, target='t3-bar', source='t3a.bar t3b.inb')[0] + assert isinstance(tgt.builder, SCons.Builder.MultiStepBuilder) + + flag = 0 + tgt = builder(env, target='t5', source=['test5a.foo', 'test5b.inb'])[0] + try: + tgt.build() + except SCons.Errors.UserError, e: + flag = 1 + assert flag, "UserError should be thrown when we build targets with files of different suffixes." + expect = "While building `['t5']' from `test5b.bar': Cannot build multiple sources with different extensions: .foo, .bar" + assert str(e) == expect, e + + flag = 0 + tgt = builder(env, target='t6', source=['test6a.bar', 'test6b.ina'])[0] + try: + tgt.build() + except SCons.Errors.UserError, e: + flag = 1 + assert flag, "UserError should be thrown when we build targets with files of different suffixes." + expect = "While building `['t6']' from `test6b.foo': Cannot build multiple sources with different extensions: .bar, .foo" + assert str(e) == expect, e + + flag = 0 + tgt = builder(env, target='t4', source=['test4a.ina', 'test4b.inb'])[0] + try: + tgt.build() + except SCons.Errors.UserError, e: + flag = 1 + assert flag, "UserError should be thrown when we build targets with files of different suffixes." + expect = "While building `['t4']' from `test4b.bar': Cannot build multiple sources with different extensions: .foo, .bar" + assert str(e) == expect, e + + flag = 0 + tgt = builder(env, target='t7', source=['test7'])[0] + try: + tgt.build() + except SCons.Errors.UserError, e: + flag = 1 + assert flag, "UserError should be thrown when we build targets with files of different suffixes." + expect = "While building `['t7']': Cannot deduce file extension from source files: ['test7']" + assert str(e) == expect, e + + flag = 0 + tgt = builder(env, target='t8', source=['test8.unknown'])[0] + try: + tgt.build() + except SCons.Errors.UserError, e: + flag = 1 + assert flag, "UserError should be thrown when we build a target with an unknown suffix." + expect = "While building `['t8']' from `['test8.unknown']': Don't know how to build from a source file with suffix `.unknown'. Expected a suffix in this list: ['.foo', '.bar']." + assert str(e) == expect, e + if __name__ == "__main__": - suite = unittest.makeSuite(BuilderTestCase, 'test_') + suite = unittest.TestSuite() + tclasses = [ +# BuilderTestCase, + CompositeBuilderTestCase + ] + for tclass in tclasses: + names = unittest.getTestCaseNames(tclass, 'test_') + suite.addTests(map(tclass, names)) if not unittest.TextTestRunner().run(suite).wasSuccessful(): sys.exit(1) diff --git a/src/engine/SCons/Conftest.py b/src/engine/SCons/Conftest.py index 4d178bd8..ddb1a992 100644 --- a/src/engine/SCons/Conftest.py +++ b/src/engine/SCons/Conftest.py @@ -319,7 +319,7 @@ int main() { return ret -def CheckLib(context, libs, func_name, header = None, +def CheckLib(context, libs, func_name = None, header = None, extra_libs = None, call = None, language = None, autoadd = 1): """ Configure check for a C or C++ libraries "libs". Searches through @@ -333,13 +333,15 @@ def CheckLib(context, libs, func_name, header = None, depends on. Optional "call" replaces the call to "func_name" in the test code. It must consist of complete C statements, including a trailing ";". - There must either be a "func_name" or a "call" argument (or both). + Both "func_name" and "call" arguments are optional, and in that case, just + linking against the libs is tested. "language" should be "C" or "C++" and is used to select the compiler. Default is "C". Note that this uses the current value of compiler and linker flags, make sure $CFLAGS, $CPPFLAGS and $LIBS are set correctly. Returns an empty string for success, an error message for failure. """ + from SCons.Debug import Trace # Include "confdefs.h" first, so that the header can use HAVE_HEADER_H. if context.headerfilename: includetext = '#include "%s"' % context.headerfilename @@ -353,32 +355,36 @@ def CheckLib(context, libs, func_name, header = None, %s""" % (includetext, header) # Add a function declaration if needed. - if func_name and func_name != "main" and not header: - text = text + """ + if func_name and func_name != "main": + if not header: + text = text + """ #ifdef __cplusplus extern "C" #endif char %s(); """ % func_name - # The actual test code. - if not call: - call = "%s();" % func_name + # The actual test code. + if not call: + call = "%s();" % func_name + + # if no function to test, leave main() blank text = text + """ int main() { %s return 0; } -""" % call - - i = string.find(call, "\n") - if i > 0: - calltext = call[:i] + ".." - elif call[-1] == ';': - calltext = call[:-1] - else: - calltext = call +""" % (call or "") + + if call: + i = string.find(call, "\n") + if i > 0: + calltext = call[:i] + ".." + elif call[-1] == ';': + calltext = call[:-1] + else: + calltext = call for lib_name in libs: @@ -387,8 +393,15 @@ return 0; context.Display("Cannot check for library %s: %s\n" % (lib_name, msg)) return msg - context.Display("Checking for %s in %s library %s... " - % (calltext, lang, lib_name)) + # if a function was specified to run in main(), say it + if call: + context.Display("Checking for %s in %s library %s... " + % (calltext, lang, lib_name)) + # otherwise, just say the name of library and language + else: + context.Display("Checking for %s library %s... " + % (lang, lib_name)) + if lib_name: l = [ lib_name ] if extra_libs: diff --git a/src/engine/SCons/Environment.py b/src/engine/SCons/Environment.py index 691098d0..d76f71d5 100644 --- a/src/engine/SCons/Environment.py +++ b/src/engine/SCons/Environment.py @@ -38,6 +38,7 @@ __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" import copy import os import os.path +import popen2 import string from UserDict import UserDict @@ -83,7 +84,7 @@ def installFunc(target, source, env): return install(target[0].path, source[0].path, env) def installString(target, source, env): - return env.subst(env['INSTALLSTR'], 0, target, source) + return env.subst_target_source(env['INSTALLSTR'], 0, target, source) installAction = SCons.Action.Action(installFunc, installString) @@ -435,6 +436,34 @@ class SubstitutionEnvironment: subst_target_source = subst + def backtick(self, command): + try: + popen2.Popen3 + except AttributeError: + (tochild, fromchild, childerr) = os.popen3(self.subst(command)) + tochild.close() + err = childerr.read() + out = fromchild.read() + fromchild.close() + status = childerr.close() + else: + p = popen2.Popen3(command, 1) + p.tochild.close() + out = p.fromchild.read() + err = p.childerr.read() + status = p.wait() + if err: + import sys + sys.stderr.write(err) + if status: + try: + if os.WIFEXITED(status): + status = os.WEXITSTATUS(status) + except AttributeError: + pass + raise OSError("'%s' exited %s" % (command, status)) + return out + def Override(self, overrides): """ Produce a modified environment whose variables are overriden by @@ -458,6 +487,196 @@ class SubstitutionEnvironment: else: return self + def ParseFlags(self, *flags): + """ + Parse the set of flags and return a dict with the flags placed + in the appropriate entry. The flags are treated as a typical + set of command-line flags for a GNU-like toolchain and used to + populate the entries in the dict immediately below. If one of + the flag strings begins with a bang (exclamation mark), it is + assumed to be a command and the rest of the string is executed; + the result of that evaluation is then added to the dict. + """ + dict = { + 'ASFLAGS' : [], + 'CCFLAGS' : [], + 'CPPDEFINES' : [], + 'CPPFLAGS' : [], + 'CPPPATH' : [], + 'FRAMEWORKPATH' : [], + 'FRAMEWORKS' : [], + 'LIBPATH' : [], + 'LIBS' : [], + 'LINKFLAGS' : [], + 'RPATH' : [], + } + + # The use of the "me" parameter to provide our own name for + # recursion is an egregious hack to support Python 2.1 and before. + def do_parse(arg, me, self = self, dict = dict): + # if arg is a sequence, recurse with each element + if not arg: + return + + if not SCons.Util.is_String(arg): + for t in arg: me(t, me) + return + + # if arg is a command, execute it + if arg[0] == '!': + arg = self.backtick(arg[1:]) + + # utility function to deal with -D option + def append_define(name, dict = dict): + t = string.split(name, '=') + if len(t) == 1: + dict['CPPDEFINES'].append(name) + else: + dict['CPPDEFINES'].append([t[0], string.join(t[1:], '=')]) + + # Loop through the flags and add them to the appropriate option. + # This tries to strike a balance between checking for all possible + # flags and keeping the logic to a finite size, so it doesn't + # check for some that don't occur often. It particular, if the + # flag is not known to occur in a config script and there's a way + # of passing the flag to the right place (by wrapping it in a -W + # flag, for example) we don't check for it. Note that most + # preprocessor options are not handled, since unhandled options + # are placed in CCFLAGS, so unless the preprocessor is invoked + # separately, these flags will still get to the preprocessor. + # Other options not currently handled: + # -iqoutedir (preprocessor search path) + # -u symbol (linker undefined symbol) + # -s (linker strip files) + # -static* (linker static binding) + # -shared* (linker dynamic binding) + # -symbolic (linker global binding) + # -R dir (deprecated linker rpath) + # IBM compilers may also accept -qframeworkdir=foo + + params = string.split(arg) + append_next_arg_to = None # for multi-word args + for arg in params: + if append_next_arg_to: + if append_next_arg_to == 'CPPDEFINES': + append_define(arg) + elif append_next_arg_to == '-include': + t = ('-include', self.fs.File(arg)) + dict['CCFLAGS'].append(t) + elif append_next_arg_to == '-isysroot': + t = ('-isysroot', arg) + dict['CCFLAGS'].append(t) + dict['LINKFLAGS'].append(t) + elif append_next_arg_to == '-arch': + t = ('-arch', arg) + dict['CCFLAGS'].append(t) + dict['LINKFLAGS'].append(t) + else: + dict[append_next_arg_to].append(arg) + append_next_arg_to = None + elif not arg[0] in ['-', '+']: + dict['LIBS'].append(self.fs.File(arg)) + elif arg[:2] == '-L': + if arg[2:]: + dict['LIBPATH'].append(arg[2:]) + else: + append_next_arg_to = 'LIBPATH' + elif arg[:2] == '-l': + if arg[2:]: + dict['LIBS'].append(arg[2:]) + else: + append_next_arg_to = 'LIBS' + elif arg[:2] == '-I': + if arg[2:]: + dict['CPPPATH'].append(arg[2:]) + else: + append_next_arg_to = 'CPPPATH' + elif arg[:4] == '-Wa,': + dict['ASFLAGS'].append(arg[4:]) + dict['CCFLAGS'].append(arg) + elif arg[:4] == '-Wl,': + if arg[:11] == '-Wl,-rpath=': + dict['RPATH'].append(arg[11:]) + elif arg[:7] == '-Wl,-R,': + dict['RPATH'].append(arg[7:]) + elif arg[:6] == '-Wl,-R': + dict['RPATH'].append(arg[6:]) + else: + dict['LINKFLAGS'].append(arg) + elif arg[:4] == '-Wp,': + dict['CPPFLAGS'].append(arg) + elif arg[:2] == '-D': + if arg[2:]: + append_define(arg[2:]) + else: + appencd_next_arg_to = 'CPPDEFINES' + elif arg == '-framework': + append_next_arg_to = 'FRAMEWORKS' + elif arg[:14] == '-frameworkdir=': + dict['FRAMEWORKPATH'].append(arg[14:]) + elif arg[:2] == '-F': + if arg[2:]: + dict['FRAMEWORKPATH'].append(arg[2:]) + else: + append_next_arg_to = 'FRAMEWORKPATH' + elif arg == '-mno-cygwin': + dict['CCFLAGS'].append(arg) + dict['LINKFLAGS'].append(arg) + elif arg == '-mwindows': + dict['LINKFLAGS'].append(arg) + elif arg == '-pthread': + dict['CCFLAGS'].append(arg) + dict['LINKFLAGS'].append(arg) + elif arg[0] == '+': + dict['CCFLAGS'].append(arg) + dict['LINKFLAGS'].append(arg) + elif arg in ['-include', '-isysroot', '-arch']: + append_next_arg_to = arg + else: + dict['CCFLAGS'].append(arg) + + for arg in flags: + do_parse(arg, do_parse) + return dict + + def MergeFlags(self, args, unique=1): + """ + Merge the dict in args into the construction variables. If args + is not a dict, it is converted into a dict using ParseFlags. + If unique is not set, the flags are appended rather than merged. + """ + + if not SCons.Util.is_Dict(args): + args = self.ParseFlags(args) + if not unique: + apply(self.Append, (), args) + return self + for key, value in args.items(): + if value == '': + continue + try: + orig = self[key] + except KeyError: + orig = value + else: + if len(orig) == 0: orig = [] + elif not SCons.Util.is_List(orig): orig = [orig] + orig = orig + value + t = [] + if key[-4:] == 'PATH': + ### keep left-most occurence + for v in orig: + if v not in t: + t.append(v) + else: + ### keep right-most occurence + orig.reverse() + for v in orig: + if v not in t: + t.insert(0, v) + self[key] = t + return self + class Base(SubstitutionEnvironment): """Base class for "real" construction Environments. These are the primary objects used to communicate dependency and construction @@ -838,82 +1057,22 @@ class Base(SubstitutionEnvironment): def ParseConfig(self, command, function=None, unique=1): """ Use the specified function to parse the output of the command - in order to modify the current environment. The 'command' can + in order to modify the current environment. The 'command' can be a string or a list of strings representing a command and - it's arguments. 'Function' is an optional argument that takes - the environment and the output of the command. If no function is - specified, the output will be treated as the output of a typical - 'X-config' command (i.e. gtk-config) and used to append to the - ASFLAGS, CCFLAGS, CPPFLAGS, CPPPATH, LIBPATH, LIBS, LINKFLAGS - and CCFLAGS variables. + its arguments. 'Function' is an optional argument that takes + the environment, the output of the command, and the unique flag. + If no function is specified, MergeFlags, which treats the output + as the result of a typical 'X-config' command (i.e. gtk-config), + will merge the output into the appropriate variables. """ - - # the default parse function - def parse_conf(env, output, fs=self.fs, unique=unique): - dict = { - 'ASFLAGS' : [], - 'CCFLAGS' : [], - 'CPPFLAGS' : [], - 'CPPPATH' : [], - 'LIBPATH' : [], - 'LIBS' : [], - 'LINKFLAGS' : [], - } - - params = string.split(output) - append_next_arg_to='' # for multi-word args - for arg in params: - if append_next_arg_to: - dict[append_next_arg_to].append(arg) - append_next_arg_to = '' - elif arg[0] != '-': - dict['LIBS'].append(fs.File(arg)) - elif arg[:2] == '-L': - if arg[2:]: - dict['LIBPATH'].append(arg[2:]) - else: - append_next_arg_to = 'LIBPATH' - elif arg[:2] == '-l': - if arg[2:]: - dict['LIBS'].append(arg[2:]) - else: - append_next_arg_to = 'LIBS' - elif arg[:2] == '-I': - if arg[2:]: - dict['CPPPATH'].append(arg[2:]) - else: - append_next_arg_to = 'CPPPATH' - elif arg[:4] == '-Wa,': - dict['ASFLAGS'].append(arg) - elif arg[:4] == '-Wl,': - dict['LINKFLAGS'].append(arg) - elif arg[:4] == '-Wp,': - dict['CPPFLAGS'].append(arg) - elif arg == '-framework': - dict['LINKFLAGS'].append(arg) - append_next_arg_to='LINKFLAGS' - elif arg == '-mno-cygwin': - dict['CCFLAGS'].append(arg) - dict['LINKFLAGS'].append(arg) - elif arg == '-mwindows': - dict['LINKFLAGS'].append(arg) - elif arg == '-pthread': - dict['CCFLAGS'].append(arg) - dict['LINKFLAGS'].append(arg) - else: - dict['CCFLAGS'].append(arg) - if unique: - appender = env.AppendUnique - else: - appender = env.Append - apply(appender, (), dict) - if function is None: + def parse_conf(env, cmd, unique=unique): + return env.MergeFlags(cmd, unique) function = parse_conf - if type(command) is type([]): + if SCons.Util.is_List(command): command = string.join(command) command = self.subst(command) - return function(self, os.popen(command).read()) + return function(self, self.backtick(command)) def ParseDepends(self, filename, must_exist=None, only_one=0): """ @@ -1137,7 +1296,11 @@ class Base(SubstitutionEnvironment): ####################################################################### def Action(self, *args, **kw): - nargs = self.subst(args) + def subst_string(a, self=self): + if SCons.Util.is_String(a): + a = self.subst(a) + return a + nargs = map(subst_string, args) nkw = self.subst_kw(kw) return apply(SCons.Action.Action, nargs, nkw) diff --git a/src/engine/SCons/EnvironmentTests.py b/src/engine/SCons/EnvironmentTests.py index cc04451d..c56f1f52 100644 --- a/src/engine/SCons/EnvironmentTests.py +++ b/src/engine/SCons/EnvironmentTests.py @@ -25,6 +25,7 @@ __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" import os import string +import StringIO import sys import TestCmd import unittest @@ -200,13 +201,13 @@ class SubstitutionTestCase(unittest.TestCase): assert env1 == env2 def test___getitem__(self): - """Test deleting a variable from a SubstitutionEnvironment + """Test fetching a variable from a SubstitutionEnvironment """ env = SubstitutionEnvironment(XXX = 'x') assert env['XXX'] == 'x', env['XXX'] def test___setitem__(self): - """Test deleting a variable from a SubstitutionEnvironment + """Test setting a variable in a SubstitutionEnvironment """ env1 = SubstitutionEnvironment(XXX = 'x') env2 = SubstitutionEnvironment(XXX = 'x', YYY = 'y') @@ -387,6 +388,16 @@ class SubstitutionTestCase(unittest.TestCase): mystr = env.subst("$AAA ${AAA}A ${AAA}B $BBB") assert mystr == "c cA cB c", mystr + # Lists: + env = SubstitutionEnvironment(AAA = ['a', 'aa', 'aaa']) + mystr = env.subst("$AAA") + assert mystr == "a aa aaa", mystr + + # Tuples: + env = SubstitutionEnvironment(AAA = ('a', 'aa', 'aaa')) + mystr = env.subst("$AAA") + assert mystr == "a aa aaa", mystr + t1 = DummyNode('t1') t2 = DummyNode('t2') s1 = DummyNode('s1') @@ -562,6 +573,56 @@ class SubstitutionTestCase(unittest.TestCase): mystr = env.subst_target_source("$AAA ${AAA}A $BBBB $BBB") assert mystr == "a aA b", mystr + def test_backtick(self): + """Test the backtick() method for capturing command output""" + env = SubstitutionEnvironment() + + test = TestCmd.TestCmd(workdir = '') + test.write('stdout.py', """\ +import sys +sys.stdout.write('this came from stdout.py\\n') +sys.exit(0) +""") + test.write('stderr.py', """\ +import sys +sys.stderr.write('this came from stderr.py\\n') +sys.exit(0) +""") + test.write('fail.py', """\ +import sys +sys.exit(1) +""") + + save_stderr = sys.stderr + + try: + cmd = '%s %s' % (sys.executable, test.workpath('stdout.py')) + output = env.backtick(cmd) + + assert output == 'this came from stdout.py\n', output + + sys.stderr = StringIO.StringIO() + + cmd = '%s %s' % (sys.executable, test.workpath('stderr.py')) + output = env.backtick(cmd) + errout = sys.stderr.getvalue() + + assert output == '', output + assert errout == 'this came from stderr.py\n', errout + + sys.stderr = StringIO.StringIO() + + cmd = '%s %s' % (sys.executable, test.workpath('fail.py')) + try: + env.backtick(cmd) + except OSError, e: + assert str(e) == "'%s' exited 1" % cmd, str(e) + else: + self.fail("did not catch expected OSError") + + finally: + sys.stderr = save_stderr + def test_Override(self): "Test overriding construction variables" env = SubstitutionEnvironment(ONE=1, TWO=2, THREE=3, FOUR=4) @@ -587,6 +648,88 @@ class SubstitutionTestCase(unittest.TestCase): assert env2['ONE'] == "won", env2['ONE'] assert env['ONE'] == 1, env['ONE'] + def test_ParseFlags(self): + """Test the ParseFlags() method + """ + env = SubstitutionEnvironment() + + empty = { + 'ASFLAGS' : [], + 'CCFLAGS' : [], + 'CPPDEFINES' : [], + 'CPPFLAGS' : [], + 'CPPPATH' : [], + 'FRAMEWORKPATH' : [], + 'FRAMEWORKS' : [], + 'LIBPATH' : [], + 'LIBS' : [], + 'LINKFLAGS' : [], + 'RPATH' : [], + } + + d = env.ParseFlags(None) + assert d == empty, d + + d = env.ParseFlags('') + assert d == empty, d + + d = env.ParseFlags([]) + assert d == empty, d + + s = "-I/usr/include/fum -I bar -X\n" + \ + "-L/usr/fax -L foo -lxxx -l yyy " + \ + "-Wa,-as -Wl,-link " + \ + "-Wl,-rpath=rpath1 " + \ + "-Wl,-R,rpath2 " + \ + "-Wl,-Rrpath3 " + \ + "-Wp,-cpp " + \ + "-framework Carbon " + \ + "-frameworkdir=fwd1 " + \ + "-Ffwd2 " + \ + "-F fwd3 " + \ + "-pthread " + \ + "-mno-cygwin -mwindows " + \ + "-arch i386 -isysroot /tmp +DD64 " + \ + "-DFOO -DBAR=value" + + d = env.ParseFlags(s) + + assert d['ASFLAGS'] == ['-as'], d['ASFLAGS'] + assert d['CCFLAGS'] == ['-X', '-Wa,-as', + '-pthread', '-mno-cygwin', + ('-arch', 'i386'), ('-isysroot', '/tmp'), + '+DD64'], d['CCFLAGS'] + assert d['CPPDEFINES'] == ['FOO', ['BAR', 'value']], d['CPPDEFINES'] + assert d['CPPFLAGS'] == ['-Wp,-cpp'], d['CPPFLAGS'] + assert d['CPPPATH'] == ['/usr/include/fum', 'bar'], d['CPPPATH'] + assert d['FRAMEWORKPATH'] == ['fwd1', 'fwd2', 'fwd3'], d['FRAMEWORKPATH'] + assert d['FRAMEWORKS'] == ['Carbon'], d['FRAMEWORKS'] + assert d['LIBPATH'] == ['/usr/fax', 'foo'], d['LIBPATH'] + assert d['LIBS'] == ['xxx', 'yyy'], d['LIBS'] + assert d['LINKFLAGS'] == ['-Wl,-link', '-pthread', + '-mno-cygwin', '-mwindows', + ('-arch', 'i386'), + ('-isysroot', '/tmp'), + '+DD64'], d['LINKFLAGS'] + assert d['RPATH'] == ['rpath1', 'rpath2', 'rpath3'], d['RPATH'] + + + def test_MergeFlags(self): + """Test the MergeFlags() method + """ + env = SubstitutionEnvironment() + env.MergeFlags('') + assert env['CCFLAGS'] == [], env['CCFLAGS'] + env.MergeFlags('-X') + assert env['CCFLAGS'] == ['-X'], env['CCFLAGS'] + env.MergeFlags('-X') + assert env['CCFLAGS'] == ['-X'], env['CCFLAGS'] + + env = SubstitutionEnvironment() + env.MergeFlags({'A':['aaa'], 'B':['bbb']}) + assert env['A'] == ['aaa'], env['A'] + assert env['B'] == ['bbb'], env['B'] + class BaseTestCase(unittest.TestCase,TestEnvironmentFixture): @@ -1476,52 +1619,74 @@ def generate(env): def test_ParseConfig(self): """Test the ParseConfig() method""" - env = self.TestEnvironment(ASFLAGS='assembler', - COMMAND='command', + env = self.TestEnvironment(COMMAND='command', + ASFLAGS='assembler', + CCFLAGS=[''], + CPPDEFINES=[], CPPFLAGS=[''], CPPPATH='string', + FRAMEWORKPATH=[], + FRAMEWORKS=[], LIBPATH=['list'], LIBS='', LINKFLAGS=[''], - CCFLAGS=['']) - orig_popen = os.popen - class my_popen: + RPATH=[]) + + orig_backtick = env.backtick + class my_backtick: def __init__(self, save_command, output): self.save_command = save_command self.output = output def __call__(self, command): self.save_command.append(command) - class fake_file: - def __init__(self, output): - self.output = output - def read(self): - return self.output - return fake_file(self.output) + return self.output + try: save_command = [] - os.popen = my_popen(save_command, + env.backtick = my_backtick(save_command, "-I/usr/include/fum -I bar -X\n" + \ "-L/usr/fax -L foo -lxxx -l yyy " + \ - "-Wa,-as -Wl,-link -Wp,-cpp abc " + \ - "-pthread -framework Carbon " + \ - "-mno-cygwin -mwindows") + "-Wa,-as -Wl,-link " + \ + "-Wl,-rpath=rpath1 " + \ + "-Wl,-R,rpath2 " + \ + "-Wl,-Rrpath3 " + \ + "-Wp,-cpp abc " + \ + "-framework Carbon " + \ + "-frameworkdir=fwd1 " + \ + "-Ffwd2 " + \ + "-F fwd3 " + \ + "-pthread " + \ + "-mno-cygwin -mwindows " + \ + "-arch i386 -isysroot /tmp +DD64 " + \ + "-DFOO -DBAR=value") env.ParseConfig("fake $COMMAND") assert save_command == ['fake command'], save_command - assert env['ASFLAGS'] == ['assembler', '-Wa,-as'], env['ASFLAGS'] - assert env['CPPPATH'] == ['string', '/usr/include/fum', 'bar'], env['CPPPATH'] + assert env['ASFLAGS'] == ['assembler', '-as'], env['ASFLAGS'] + assert env['CCFLAGS'] == ['', '-X', '-Wa,-as', + '-pthread', '-mno-cygwin', + ('-arch', 'i386'), ('-isysroot', '/tmp'), + '+DD64'], env['CCFLAGS'] + assert env['CPPDEFINES'] == ['FOO', ['BAR', 'value']], env['CPPDEFINES'] assert env['CPPFLAGS'] == ['', '-Wp,-cpp'], env['CPPFLAGS'] + assert env['CPPPATH'] == ['string', '/usr/include/fum', 'bar'], env['CPPPATH'] + assert env['FRAMEWORKPATH'] == ['fwd1', 'fwd2', 'fwd3'], env['FRAMEWORKPATH'] + assert env['FRAMEWORKS'] == ['Carbon'], env['FRAMEWORKS'] assert env['LIBPATH'] == ['list', '/usr/fax', 'foo'], env['LIBPATH'] assert env['LIBS'] == ['xxx', 'yyy', env.File('abc')], env['LIBS'] - assert env['LINKFLAGS'] == ['', '-Wl,-link', '-pthread', '-framework', 'Carbon', '-mno-cygwin', '-mwindows'], env['LINKFLAGS'] - assert env['CCFLAGS'] == ['', '-X', '-pthread', '-mno-cygwin'], env['CCFLAGS'] - - os.popen = my_popen([], "-Ibar") + assert env['LINKFLAGS'] == ['', '-Wl,-link', '-pthread', + '-mno-cygwin', '-mwindows', + ('-arch', 'i386'), + ('-isysroot', '/tmp'), + '+DD64'], env['LINKFLAGS'] + assert env['RPATH'] == ['rpath1', 'rpath2', 'rpath3'], env['RPATH'] + + env.backtick = my_backtick([], "-Ibar") env.ParseConfig("fake2") assert env['CPPPATH'] == ['string', '/usr/include/fum', 'bar'], env['CPPPATH'] env.ParseConfig("fake2", unique=0) assert env['CPPPATH'] == ['string', '/usr/include/fum', 'bar', 'bar'], env['CPPPATH'] finally: - os.popen = orig_popen + env.backtick = orig_backtick def test_ParseDepends(self): """Test the ParseDepends() method""" @@ -1982,21 +2147,25 @@ def generate(env): a = env.Action('foo') assert a, a - assert a.__class__ is SCons.Action.CommandAction, a + assert a.__class__ is SCons.Action.CommandAction, a.__class__ a = env.Action('$FOO') assert a, a - assert a.__class__ is SCons.Action.LazyAction, a + assert a.__class__ is SCons.Action.CommandAction, a.__class__ + + a = env.Action('$$FOO') + assert a, a + assert a.__class__ is SCons.Action.LazyAction, a.__class__ a = env.Action(['$FOO', 'foo']) assert a, a - assert a.__class__ is SCons.Action.ListAction, a + assert a.__class__ is SCons.Action.ListAction, a.__class__ def func(arg): pass a = env.Action(func) assert a, a - assert a.__class__ is SCons.Action.FunctionAction, a + assert a.__class__ is SCons.Action.FunctionAction, a.__class__ def test_AddPostAction(self): """Test the AddPostAction() method""" @@ -2815,12 +2984,12 @@ def generate(env): SOURCE = 'source', TARGET = 'target', INIT = 'init') + bad_msg = '%s is not reserved, but got omitted; see Environment.construction_var_name_ok' added.append('INIT') for x in reserved: assert not env.has_key(x), env[x] for x in added: - assert env.has_key(x), \ - '%s is not reserved, but got omitted; see Environment.construction_var_name_ok'%x + assert env.has_key(x), bad_msg % x env.Append(TARGETS = 'targets', SOURCES = 'sources', @@ -2831,8 +3000,7 @@ def generate(env): for x in reserved: assert not env.has_key(x), env[x] for x in added: - assert env.has_key(x), \ - '%s is not reserved, but got omitted; see Environment.construction_var_name_ok'%x + assert env.has_key(x), bad_msg % x env.AppendUnique(TARGETS = 'targets', SOURCES = 'sources', @@ -2843,8 +3011,7 @@ def generate(env): for x in reserved: assert not env.has_key(x), env[x] for x in added: - assert env.has_key(x), \ - '%s is not reserved, but got omitted; see Environment.construction_var_name_ok'%x + assert env.has_key(x), bad_msg % x env.Prepend(TARGETS = 'targets', SOURCES = 'sources', @@ -2855,8 +3022,7 @@ def generate(env): for x in reserved: assert not env.has_key(x), env[x] for x in added: - assert env.has_key(x), \ - '%s is not reserved, but got omitted; see Environment.construction_var_name_ok'%x + assert env.has_key(x), bad_msg % x env.Prepend(TARGETS = 'targets', SOURCES = 'sources', @@ -2867,8 +3033,7 @@ def generate(env): for x in reserved: assert not env.has_key(x), env[x] for x in added: - assert env.has_key(x), \ - '%s is not reserved, but got omitted; see Environment.construction_var_name_ok'%x + assert env.has_key(x), bad_msg % x env.Replace(TARGETS = 'targets', SOURCES = 'sources', @@ -2879,8 +3044,7 @@ def generate(env): for x in reserved: assert not env.has_key(x), env[x] for x in added: - assert env.has_key(x), \ - '%s is not reserved, but got omitted; see Environment.construction_var_name_ok'%x + assert env.has_key(x), bad_msg % x copy = env.Copy(TARGETS = 'targets', SOURCES = 'sources', @@ -2890,8 +3054,7 @@ def generate(env): for x in reserved: assert not copy.has_key(x), env[x] for x in added + ['COPY']: - assert copy.has_key(x), \ - '%s is not reserved, but got omitted; see Environment.construction_var_name_ok'%x + assert copy.has_key(x), bad_msg % x over = env.Override({'TARGETS' : 'targets', 'SOURCES' : 'sources', @@ -2901,8 +3064,7 @@ def generate(env): for x in reserved: assert not over.has_key(x), over[x] for x in added + ['OVERRIDE']: - assert over.has_key(x), \ - '%s is not reserved, but got omitted; see Environment.construction_var_name_ok'%x + assert over.has_key(x), bad_msg % x diff --git a/src/engine/SCons/Executor.py b/src/engine/SCons/Executor.py index d95fd813..ffc1ba38 100644 --- a/src/engine/SCons/Executor.py +++ b/src/engine/SCons/Executor.py @@ -191,6 +191,8 @@ class Executor: This essentially short-circuits an N*M scan of the sources for each individual target, which is a hell of a lot more efficient. """ + map(lambda N: N.disambiguate(), node_list) + env = self.get_build_env() select_specific_scanner = lambda t: (t[0], t[1].select(t[0])) remove_null_scanners = lambda t: not t[1] is None diff --git a/src/engine/SCons/ExecutorTests.py b/src/engine/SCons/ExecutorTests.py index 44086d7c..8aedb760 100644 --- a/src/engine/SCons/ExecutorTests.py +++ b/src/engine/SCons/ExecutorTests.py @@ -88,6 +88,8 @@ class MyNode: return self.missing_val def calc_signature(self, calc): return 'cs-'+calc+'-'+self.name + def disambiguate(self): + return self class MyScanner: def __init__(self, prefix): diff --git a/src/engine/SCons/Job.py b/src/engine/SCons/Job.py index 60eb8615..cb8a77ad 100644 --- a/src/engine/SCons/Job.py +++ b/src/engine/SCons/Job.py @@ -89,8 +89,7 @@ class Serial: that needs to be executed, or None if there are no more tasks. The taskmaster's executed() method will be called for each task when it is successfully executed or failed() will be called if it failed to - execute (e.g. execute() raised an exception). The taskmaster's - is_blocked() method will not be called. """ + execute (e.g. execute() raised an exception).""" self.taskmaster = taskmaster @@ -193,20 +192,17 @@ else: def __init__(self, taskmaster, num): """Create a new parallel job given a taskmaster. - The taskmaster's next_task() method should return the next task - that needs to be executed, or None if there are no more tasks. The - taskmaster's executed() method will be called for each task when it - is successfully executed or failed() will be called if the task - failed to execute (i.e. execute() raised an exception). The - taskmaster's is_blocked() method should return true iff there are - more tasks, but they can't be executed until one or more other - tasks have been executed. next_task() will be called iff - is_blocked() returned false. - - Note: calls to taskmaster are serialized, but calls to execute() on - distinct tasks are not serialized, because that is the whole point - of parallel jobs: they can execute multiple tasks - simultaneously. """ + The taskmaster's next_task() method should return the next + task that needs to be executed, or None if there are no more + tasks. The taskmaster's executed() method will be called + for each task when it is successfully executed or failed() + will be called if the task failed to execute (i.e. execute() + raised an exception). + + Note: calls to taskmaster are serialized, but calls to + execute() on distinct tasks are not serialized, because + that is the whole point of parallel jobs: they can execute + multiple tasks simultaneously. """ self.taskmaster = taskmaster self.tp = ThreadPool(num) diff --git a/src/engine/SCons/JobTests.py b/src/engine/SCons/JobTests.py index 2ace9c31..d2c019f2 100644 --- a/src/engine/SCons/JobTests.py +++ b/src/engine/SCons/JobTests.py @@ -61,7 +61,7 @@ class Task: self.taskmaster = taskmaster self.was_executed = 0 self.was_prepared = 0 - + def prepare(self): self.was_prepared = 1 @@ -71,7 +71,7 @@ class Task: def execute(self): self.taskmaster.test_case.failUnless(self.was_prepared, "the task wasn't prepared") - + self.taskmaster.guard.acquire() self.taskmaster.begin_list.append(self.i) self.taskmaster.guard.release() @@ -119,7 +119,7 @@ class ExceptionTask: def prepare(self): self.was_prepared = 1 - + def execute(self): raise "exception" @@ -188,14 +188,6 @@ class Taskmaster: def all_tasks_are_postprocessed(self): return self.num_postprocessed == self.num_tasks - def is_blocked(self): - if self.stop or self.all_tasks_are_executed(): - return 0 - if self.all_tasks_are_iterated(): - return 1 - # simulate blocking tasks - return self.num_iterated - self.num_executed >= max(num_jobs/2, 2) - def tasks_were_serial(self): "analyze the task order to see if they were serial" serial = 1 # assume the tasks were serial @@ -233,7 +225,7 @@ class ParallelTestCase(unittest.TestCase): self.failUnless(taskmaster.all_tasks_are_postprocessed(), "all the tests were not postprocessed") self.failIf(taskmaster.num_failed, - "some task(s) failed to execute") + "some task(s) failed to execute") # Verify that parallel jobs will pull all of the completed tasks # out of the queue at once, instead of one by one. We do this by @@ -298,7 +290,7 @@ class SerialTestCase(unittest.TestCase): self.failUnless(taskmaster.all_tasks_are_postprocessed(), "all the tests were not postprocessed") self.failIf(taskmaster.num_failed, - "some task(s) failed to execute") + "some task(s) failed to execute") class NoParallelTestCase(unittest.TestCase): def runTest(self): @@ -310,7 +302,7 @@ class NoParallelTestCase(unittest.TestCase): try: taskmaster = Taskmaster(num_tasks, self, RandomTask) jobs = SCons.Job.Jobs(2, taskmaster) - self.failUnless(jobs.num_jobs == 1, + self.failUnless(jobs.num_jobs == 1, "unexpected number of jobs %d" % jobs.num_jobs) jobs.run() self.failUnless(taskmaster.tasks_were_serial(), @@ -322,7 +314,7 @@ class NoParallelTestCase(unittest.TestCase): self.failUnless(taskmaster.all_tasks_are_postprocessed(), "all the tests were not postprocessed") self.failIf(taskmaster.num_failed, - "some task(s) failed to execute") + "some task(s) failed to execute") finally: SCons.Job.Parallel = save_Parallel @@ -357,7 +349,7 @@ class ParallelExceptionTestCase(unittest.TestCase): self.failUnless(taskmaster.num_iterated >= 1, "one or more task should have been iterated") self.failUnless(taskmaster.num_failed >= 1, - "one or more tasks should have failed") + "one or more tasks should have failed") self.failUnless(taskmaster.num_postprocessed >= 1, "one or more tasks should have been postprocessed") @@ -385,7 +377,7 @@ class slowgoodnode (goodnode): # by this test. time.sleep(0.15) goodnode.prepare(self) - + class badnode (goodnode): def __init__(self): goodnode.__init__(self) @@ -436,7 +428,7 @@ class _SConsTaskTest(unittest.TestCase): # Exceptions thrown by tasks are not actually propagated to # this level, but are instead stored in the Taskmaster. - + jobs.run() # Now figure out if tests proceeded correctly. The first test diff --git a/src/engine/SCons/Node/FS.py b/src/engine/SCons/Node/FS.py index 5e01526b..ce5bcc0c 100644 --- a/src/engine/SCons/Node/FS.py +++ b/src/engine/SCons/Node/FS.py @@ -53,8 +53,6 @@ import SCons.Subst import SCons.Util import SCons.Warnings -from SCons.Debug import Trace - # The max_drift value: by default, use a cached signature value for # any file that's been untouched for more than two days. default_max_drift = 2*24*60*60 @@ -685,6 +683,10 @@ class Base(SCons.Node.Node): def target_from_source(self, prefix, suffix, splitext=SCons.Util.splitext): return self.dir.Entry(prefix + splitext(self.name)[0] + suffix) + def RDirs(self, pathlist): + """Search for a list of directories in the Repository list.""" + return self.fs.Rfindalldirs(pathlist, self.cwd) + class Entry(Base): """This is the class for generic Node.FS entries--that is, things that could be a File or a Dir, but we're just not sure yet. @@ -697,7 +699,7 @@ class Entry(Base): pass def disambiguate(self): - if self.isdir(): + if self.isdir() or self.srcnode().isdir(): self.__class__ = Dir self._morph() else: @@ -1703,10 +1705,6 @@ class File(Base): the SConscript directory of this file.""" return self.fs.File(name, self.cwd) - def RDirs(self, pathlist): - """Search for a list of directories in the Repository list.""" - return self.fs.Rfindalldirs(pathlist, self.cwd) - #def generate_build_dict(self): # """Return an appropriate dictionary of values for building # this File.""" @@ -1778,7 +1776,7 @@ class File(Base): __cacheable__""" if not scanner: return [] - return scanner(self, env, path) + return map(lambda N: N.disambiguate(), scanner(self, env, path)) def _createDir(self): # ensure that the directories for this node are diff --git a/src/engine/SCons/Node/FSTests.py b/src/engine/SCons/Node/FSTests.py index f8512c92..98e08a9f 100644 --- a/src/engine/SCons/Node/FSTests.py +++ b/src/engine/SCons/Node/FSTests.py @@ -2736,6 +2736,20 @@ class disambiguateTestCase(unittest.TestCase): f = efile.disambiguate() assert f.__class__ is fff.__class__, f.__class__ + test.subdir('src') + test.subdir(['src', 'edir']) + test.write(['src', 'efile'], "src/efile\n") + + fs.BuildDir(test.workpath('build'), test.workpath('src')) + + build_edir = fs.Entry(test.workpath('build/edir')) + d = build_edir.disambiguate() + assert d.__class__ is ddd.__class__, d.__class__ + + build_efile = fs.Entry(test.workpath('build/efile')) + f = build_efile.disambiguate() + assert f.__class__ is fff.__class__, f.__class__ + class postprocessTestCase(unittest.TestCase): def runTest(self): """Test calling the postprocess() method.""" diff --git a/src/engine/SCons/Node/NodeTests.py b/src/engine/SCons/Node/NodeTests.py index 3949abb4..50de2b0a 100644 --- a/src/engine/SCons/Node/NodeTests.py +++ b/src/engine/SCons/Node/NodeTests.py @@ -1271,7 +1271,6 @@ class NodeTestCase(unittest.TestCase): n.includes = 'testincludes' n.found_include = {'testkey':'testvalue'} n.implicit = 'testimplicit' - n.waiting_parents = ['foo', 'bar'] x = MyExecutor() n.set_executor(x) @@ -1282,7 +1281,6 @@ class NodeTestCase(unittest.TestCase): assert n.includes is None, n.includes assert n.found_includes == {}, n.found_includes assert n.implicit is None, n.implicit - assert n.waiting_parents == [], n.waiting_parents assert x.cleaned_up def test_get_subst_proxy(self): @@ -1306,15 +1304,18 @@ class NodeTestCase(unittest.TestCase): def test_postprocess(self): """Test calling the base Node postprocess() method""" n = SCons.Node.Node() + n.waiting_parents = {'foo':1, 'bar':1} + n.postprocess() + assert n.waiting_parents == {}, n.waiting_parents def test_add_to_waiting_parents(self): """Test the add_to_waiting_parents() method""" n1 = SCons.Node.Node() n2 = SCons.Node.Node() - assert n1.waiting_parents == [], n1.waiting_parents + assert n1.waiting_parents == {}, n1.waiting_parents n1.add_to_waiting_parents(n2) - assert n1.waiting_parents == [n2], n1.waiting_parents + assert n1.waiting_parents == {n2:1}, n1.waiting_parents def test_call_for_all_waiting_parents(self): """Test the call_for_all_waiting_parents() method""" diff --git a/src/engine/SCons/Node/__init__.py b/src/engine/SCons/Node/__init__.py index 6f87a1a6..bda3a482 100644 --- a/src/engine/SCons/Node/__init__.py +++ b/src/engine/SCons/Node/__init__.py @@ -68,7 +68,6 @@ executing = 2 up_to_date = 3 executed = 4 failed = 5 -stack = 6 # nodes that are in the current Taskmaster execution stack StateString = { 0 : "0", @@ -77,7 +76,6 @@ StateString = { 3 : "up_to_date", 4 : "executed", 5 : "failed", - 6 : "stack", } # controls whether implicit dependencies are cached: @@ -193,7 +191,9 @@ class Node: self.ignore = [] # dependencies to ignore self.ignore_dict = {} self.implicit = None # implicit (scanned) dependencies (None means not scanned yet) - self.waiting_parents = [] + self.waiting_parents = {} + self.waiting_s_e = {} + self.ref_count = 0 self.wkids = None # Kids yet to walk, when it's an array self.env = None @@ -208,7 +208,7 @@ class Node: self.side_effects = [] # the side effects of building this target self.pre_actions = [] self.post_actions = [] - self.linked = 0 # is this node linked to the build directory? + self.linked = 0 # is this node linked to the build directory? # Let the interface in which the build engine is embedded # annotate this Node with its own info (like a description of @@ -281,7 +281,7 @@ class Node: Returns true iff the node was successfully retrieved. """ return 0 - + def build(self, **kw): """Actually build the node. @@ -301,10 +301,10 @@ class Node: # Clear the implicit dependency caches of any Nodes # waiting for this Node to be built. - for parent in self.waiting_parents: + for parent in self.waiting_parents.keys(): parent.implicit = None parent.del_binfo() - + try: new = self.binfo except AttributeError: @@ -316,7 +316,7 @@ class Node: # Reset this Node's cached state since it was just built and # various state has changed. self.clear() - + if new: # It had build info, so it should be stored in the signature # cache. However, if the build info included a content @@ -328,18 +328,22 @@ class Node: self.binfo = new self.store_info(self.binfo) + def add_to_waiting_s_e(self, node): + self.waiting_s_e[node] = 1 + def add_to_waiting_parents(self, node): - self.waiting_parents.append(node) + self.waiting_parents[node] = 1 def call_for_all_waiting_parents(self, func): func(self) - for parent in self.waiting_parents: + for parent in self.waiting_parents.keys(): parent.call_for_all_waiting_parents(func) def postprocess(self): """Clean up anything we don't need to hang onto after we've been built.""" self.executor_cleanup() + self.waiting_parents = {} def clear(self): """Completely clear a Node of all its cached state (so that it @@ -357,8 +361,6 @@ class Node: self.found_includes = {} self.implicit = None - self.waiting_parents = [] - def visited(self): """Called just after this node has been visited without requiring a build..""" @@ -757,7 +759,7 @@ class Node: """Does this node exists?""" # All node exist by default: return 1 - + def rexists(self): """Does this node exist locally or in a repositiory?""" # There are no repositories by default: diff --git a/src/engine/SCons/SConf.py b/src/engine/SCons/SConf.py index 6dfb8434..5c20f264 100644 --- a/src/engine/SCons/SConf.py +++ b/src/engine/SCons/SConf.py @@ -246,8 +246,7 @@ class SConfBuildTask(SCons.Taskmaster.Task): cachable = 1 for t in self.targets: bi = t.get_stored_info() - c_bi = isinstance(bi, SConfBuildInfo) - if c_bi: + if isinstance(bi, SConfBuildInfo): if cache_mode == CACHE: t.set_state(SCons.Node.up_to_date) else: @@ -480,7 +479,6 @@ class SConf: result = self.BuildNodes(nodesToBeBuilt) finally: - # Restor the SPAWN value to the environment. self.env['SPAWN'] = save_spawn _ac_build_counter = _ac_build_counter + 1 @@ -851,11 +849,11 @@ def CheckLib(context, library = None, symbol = "main", # Bram: Can only include one header and can't use #ifdef HAVE_HEADER_H. def CheckLibWithHeader(context, libs, header, language, - call = "main();", autoadd = 1): + call = None, autoadd = 1): # ToDo: accept path for library. Support system header files. """ Another (more sophisticated) test for a library. - Checks, if library and header is available for language (maybe 'C' + Checks, if library and header is available for language (may be 'C' or 'CXX'). Call maybe be a valid expression _with_ a trailing ';'. As in CheckLib, we support library=None, to test if the call compiles without extra link flags. @@ -868,7 +866,7 @@ def CheckLibWithHeader(context, libs, header, language, if not SCons.Util.is_List(libs): libs = [libs] - res = SCons.Conftest.CheckLib(context, libs, "main", prog_prefix, + res = SCons.Conftest.CheckLib(context, libs, None, prog_prefix, call = call, language = language, autoadd = autoadd) context.did_show_result = 1 return not res diff --git a/src/engine/SCons/SConfTests.py b/src/engine/SCons/SConfTests.py index 85b46905..0acbee3d 100644 --- a/src/engine/SCons/SConfTests.py +++ b/src/engine/SCons/SConfTests.py @@ -162,7 +162,8 @@ class SConfTestCase(unittest.TestCase): def __init__(self, name): self.name = name self.state = None - self.side_effects = [] + self.waiting_parents = {} + self.side_effects = {} self.builder = None def disambiguate(self): return self diff --git a/src/engine/SCons/Scanner/__init__.py b/src/engine/SCons/Scanner/__init__.py index ce9ae18e..1fd77e52 100644 --- a/src/engine/SCons/Scanner/__init__.py +++ b/src/engine/SCons/Scanner/__init__.py @@ -125,12 +125,14 @@ class Base: which scanner should be used for a given Node. In the case of File nodes, for example, the 'skeys' would be file suffixes. - 'path_function' - a function that takes one to three arguments - (a construction environment, optional directory, and optional - argument for this instance) and returns a tuple of the - directories that can be searched for implicit dependency files. - May also return a callable() which is called with no args and - returns the tuple (supporting Bindable class). + 'path_function' - a function that takes four or five arguments + (a construction environment, Node for the directory containing + the SConscript file that defined the primary target, list of + target nodes, list of source nodes, and optional argument for + this instance) and returns a tuple of the directories that can + be searched for implicit dependency files. May also return a + callable() which is called with no args and returns the tuple + (supporting Bindable class). 'node_class' - the class of Nodes which this scan will return. If node_class is None, then this scanner will not enforce any diff --git a/src/engine/SCons/Subst.py b/src/engine/SCons/Subst.py index 56522a13..eff8f971 100644 --- a/src/engine/SCons/Subst.py +++ b/src/engine/SCons/Subst.py @@ -37,7 +37,7 @@ import UserList import SCons.Errors -from SCons.Util import is_String, is_List +from SCons.Util import is_String, is_List, is_Tuple # Indexed by the SUBST_* constants below. _strconv = [SCons.Util.to_String, @@ -170,7 +170,7 @@ class NLWrapper: list = self.list if list is None: list = [] - elif not is_List(list): + elif not is_List(list) and not is_Tuple(list): list = [list] # The map(self.func) call is what actually turns # a list into appropriate proxies. @@ -411,7 +411,7 @@ def scons_subst(strSubst, env, mode=SUBST_RAW, target=None, source=None, gvars={ var = string.split(key, '.')[0] lv[var] = '' return self.substitute(s, lv) - elif is_List(s): + elif is_List(s) or is_Tuple(s): def func(l, conv=self.conv, substitute=self.substitute, lvars=lvars): return conv(substitute(l, lvars)) r = map(func, s) @@ -618,7 +618,7 @@ def scons_subst_list(strSubst, env, mode=SUBST_RAW, target=None, source=None, gv lv[var] = '' self.substitute(s, lv, 0) self.this_word() - elif is_List(s): + elif is_List(s) or is_Tuple(s): for a in s: self.substitute(a, lvars, 1) self.next_word() @@ -808,18 +808,18 @@ def scons_subst_once(strSubst, env, key): a = match.group(1) if a in matchlist: a = val - if is_List(a): + if is_List(a) or is_Tuple(a): return string.join(map(str, a)) else: return str(a) - if is_List(strSubst): + if is_List(strSubst) or is_Tuple(strSubst): result = [] for arg in strSubst: if is_String(arg): if arg in matchlist: arg = val - if is_List(arg): + if is_List(arg) or is_Tuple(arg): result.extend(arg) else: result.append(arg) diff --git a/src/engine/SCons/SubstTests.py b/src/engine/SCons/SubstTests.py index 8a56f674..4de3348f 100644 --- a/src/engine/SCons/SubstTests.py +++ b/src/engine/SCons/SubstTests.py @@ -184,6 +184,8 @@ class SubstTestCase(unittest.TestCase): 'S' : 'x y', 'LS' : ['x y'], 'L' : ['x', 'y'], + 'TS' : ('x y'), + 'T' : ('x', 'y'), 'CS' : cs, 'CL' : cl, @@ -302,9 +304,13 @@ class SubstTestCase(unittest.TestCase): '$S', 'x y', '$LS', 'x y', '$L', 'x y', + '$TS', 'x y', + '$T', 'x y', '$S z', 'x y z', '$LS z', 'x y z', '$L z', 'x y z', + '$TS z', 'x y z', + '$T z', 'x y z', #cs, 'cs', #cl, 'cl', '$CS', 'cs', diff --git a/src/engine/SCons/Taskmaster.py b/src/engine/SCons/Taskmaster.py index ecd6b07a..7cdecf3b 100644 --- a/src/engine/SCons/Taskmaster.py +++ b/src/engine/SCons/Taskmaster.py @@ -1,9 +1,3 @@ -"""SCons.Taskmaster - -Generic Taskmaster. - -""" - # # __COPYRIGHT__ # @@ -27,6 +21,32 @@ Generic Taskmaster. # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. # +__doc__ = """ +Generic Taskmaster module for the SCons build engine. + +This module contains the primary interface(s) between a wrapping user +interface and the SCons build engine. There are two key classes here: + + Taskmaster + This is the main engine for walking the dependency graph and + calling things to decide what does or doesn't need to be built. + + Task + This is the base class for allowing a wrapping interface to + decide what does or doesn't actually need to be done. The + intention is for a wrapping interface to subclass this as + appropriate for different types of behavior it may need. + + The canonical example is the SCons native Python interface, + which has Task subclasses that handle its specific behavior, + like printing "`foo' is up to date" when a top-level target + doesn't need to be built, and handling the -c option by removing + targets as its "build" action. + + The Taskmaster instantiates a Task object for each (set of) + target(s) that it decides need to be evaluated and/or built. +""" + __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" import string @@ -36,6 +56,8 @@ import traceback import SCons.Node import SCons.Errors +StateString = SCons.Node.StateString + # A subsystem for recording stats about how different Nodes are handled by # the main Taskmaster loop. There's no external control here (no need for # a --debug= option); enable it by changing the value of CollectStats. @@ -43,12 +65,22 @@ import SCons.Errors CollectStats = None class Stats: + """ + A simple class for holding statistics about the disposition of a + Node by the Taskmaster. If we're collecting statistics, each Node + processed by the Taskmaster gets one of these attached, in which + the Taskmaster records its decision each time it processes the Node. + (Ideally, that's just once per Node.) + """ def __init__(self): + """ + Instantiates a Taskmaster.Stats object, initializing all + appropriate counters to zero. + """ self.considered = 0 self.already_handled = 0 self.problem = 0 self.child_failed = 0 - self.not_started = 0 self.not_built = 0 self.side_effects = 0 self.build = 0 @@ -59,7 +91,6 @@ fmt = "%(considered)3d "\ "%(already_handled)3d " \ "%(problem)3d " \ "%(child_failed)3d " \ - "%(not_started)3d " \ "%(not_built)3d " \ "%(side_effects)3d " \ "%(build)3d " @@ -118,6 +149,11 @@ class Task: for s in t.side_effects: s.prepare() + def get_target(self): + """Fetch the target being built or updated by this task. + """ + return self.node + def execute(self): """Called to execute the task. @@ -146,11 +182,6 @@ class Task: raise SCons.Errors.TaskmasterException(self.targets[0], sys.exc_info()) - def get_target(self): - """Fetch the target being built or updated by this task. - """ - return self.node - def executed(self): """Called when the task has been successfully executed. @@ -161,8 +192,6 @@ class Task: back on the pending list.""" for t in self.targets: if t.get_state() == SCons.Node.executing: - for side_effect in t.side_effects: - side_effect.set_state(SCons.Node.no_state) t.set_state(SCons.Node.executed) t.built() else: @@ -179,15 +208,13 @@ class Task: for t in self.targets: t.set_state(SCons.Node.failed) self.tm.failed(self.node) - next_top = self.tm.next_top_level_candidate() self.tm.stop() - if next_top: - # We're stopping because of a build failure, but give the - # calling Task class a chance to postprocess() the top-level - # target under which the build failure occurred. - self.targets = [next_top] - self.top = 1 + # We're stopping because of a build failure, but give the + # calling Task class a chance to postprocess() the top-level + # target under which the build failure occurred. + self.targets = [self.tm.current_top] + self.top = 1 def fail_continue(self): """Explicit continue-the-build failure. @@ -211,9 +238,9 @@ class Task: """ self.out_of_date = self.targets[:] for t in self.targets: + t.disambiguate().set_state(SCons.Node.executing) for s in t.side_effects: - s.set_state(SCons.Node.executing) - t.set_state(SCons.Node.executing) + s.set_state(SCons.Node.pending) def make_ready_current(self): """Mark all targets in a task ready for execution if any target @@ -227,14 +254,32 @@ class Task: t.set_state(SCons.Node.up_to_date) else: self.out_of_date.append(t) - for s in t.side_effects: - s.set_state(SCons.Node.executing) t.set_state(SCons.Node.executing) + for s in t.side_effects: + s.set_state(SCons.Node.pending) make_ready = make_ready_current def postprocess(self): """Post process a task after it's been executed.""" + parents = {} + for t in self.targets: + for p in t.waiting_parents.keys(): + parents[p] = parents.get(p, 0) + 1 + for t in self.targets: + for s in t.side_effects: + if s.get_state() == SCons.Node.pending: + s.set_state(SCons.Node.no_state) + for p in s.waiting_parents.keys(): + if not parents.has_key(p): + parents[p] = 1 + for p in s.waiting_s_e.keys(): + if p.ref_count == 0: + self.tm.candidates.append(p) + for p, subtract in parents.items(): + p.ref_count = p.ref_count - subtract + if p.ref_count == 0: + self.tm.candidates.append(p) for t in self.targets: t.postprocess() @@ -265,6 +310,17 @@ def order(dependencies): return dependencies +def find_cycle(stack): + if stack[0] == stack[-1]: + return stack + for n in stack[-1].waiting_parents.keys(): + stack.append(n) + if find_cycle(stack): + return stack + stack.pop() + return None + + class Taskmaster: """A generic Taskmaster for handling a bunch of targets. @@ -273,25 +329,36 @@ class Taskmaster: """ def __init__(self, targets=[], tasker=Task, order=order, trace=None): - self.targets = targets # top level targets - self.candidates = targets[:] # nodes that might be ready to be executed - self.candidates.reverse() - self.executing = [] # nodes that are currently executing - self.pending = [] # nodes that depend on a currently executing node + self.top_targets = targets[:] + self.top_targets.reverse() + self.candidates = [] self.tasker = tasker self.ready = None # the next task that is ready to be executed self.order = order self.message = None self.trace = trace + self.next_candidate = self.find_next_candidate - # See if we can alter the target list to find any - # corresponding targets in linked build directories - for node in self.targets: - alt, message = node.alter_targets() - if alt: - self.message = message - self.candidates.extend(self.order(alt)) - continue + def find_next_candidate(self): + try: + return self.candidates.pop() + except IndexError: + pass + try: + node = self.top_targets.pop() + except IndexError: + return None + self.current_top = node + alt, message = node.alter_targets() + if alt: + self.message = message + self.candidates.append(node) + self.candidates.extend(self.order(alt)) + node = self.candidates.pop() + return node + + def no_next_candidate(self): + return None def _find_next_ready_node(self): """Find the next node that is ready to be built""" @@ -303,8 +370,13 @@ class Taskmaster: T = self.trace - while self.candidates: - node = self.candidates.pop().disambiguate() + while 1: + node = self.next_candidate() + if node is None: + self.ready = None + break + + node = node.disambiguate() state = node.get_state() if CollectStats: @@ -318,14 +390,14 @@ class Taskmaster: if T: T.write('Taskmaster: %s:' % repr(str(node))) - # Skip this node if it has already been handled: - if not state in [ SCons.Node.no_state, SCons.Node.stack ]: + # Skip this node if it has already been evaluated: + if state > SCons.Node.pending: if S: S.already_handled = S.already_handled + 1 - if T: T.write(' already handled\n') + if T: T.write(' already handled (%s)\n' % StateString[state]) continue # Mark this node as being on the execution stack: - node.set_state(SCons.Node.stack) + node.set_state(SCons.Node.pending) try: children = node.children() @@ -349,10 +421,11 @@ class Taskmaster: if S: S.problem = S.problem + 1 if T: T.write(' exception\n') break - else: + + if T and children: c = map(str, children) c.sort() - if T: T.write(' children:\n %s\n ' % c) + T.write(' children:\n %s\n ' % c) childinfo = map(lambda N: (N.get_state(), N.is_derived() or N.is_pseudo_derived(), @@ -374,19 +447,14 @@ class Taskmaster: continue # Detect dependency cycles: - cycle = filter(lambda I: I[0] == SCons.Node.stack, childinfo) - if cycle: - # The node we popped from the candidate stack is part of - # the cycle we detected, so put it back before generating - # the message to report. - self.candidates.append(node) - nodes = filter(lambda N: N.get_state() == SCons.Node.stack, - self.candidates) + \ - map(lambda I: I[2], cycle) - nodes.reverse() - desc = "Dependency cycle: " + string.join(map(str, nodes), " -> ") - if T: T.write(' dependency cycle\n') - raise SCons.Errors.UserError, desc + pending_nodes = filter(lambda I: I[0] == SCons.Node.pending, childinfo) + if pending_nodes: + for p in pending_nodes: + cycle = find_cycle([p[2], node]) + if cycle: + desc = "Dependency cycle: " + string.join(map(str, cycle), " -> ") + if T: T.write(' dependency cycle\n') + raise SCons.Errors.UserError, desc # Select all of the dependencies that are derived targets # (that is, children who have builders or are side effects). @@ -403,6 +471,7 @@ class Taskmaster: # list will get cleared and we'll re-scan the newly-built # file(s) for updated implicit dependencies. map(lambda n, P=node: n.add_to_waiting_parents(P), not_started) + node.ref_count = len(not_started) # Now we add these derived targets to the candidates # list so they can be examined and built. We have to @@ -418,6 +487,7 @@ class Taskmaster: self.candidates.append(node) not_started.reverse() self.candidates.extend(self.order(not_started)) + if S: S.not_started = S.not_started + 1 if T: c = map(str, not_started) @@ -436,11 +506,8 @@ class Taskmaster: # dependency list will get cleared and we'll re-scan the # newly-built file(s) for updated implicit dependencies. map(lambda n, P=node: n.add_to_waiting_parents(P), not_built) + node.ref_count = len(not_built) - # And add this node to the "pending" list, so it can get - # put back on the candidates list when appropriate. - self.pending.append(node) - node.set_state(SCons.Node.pending) if S: S.not_built = S.not_built + 1 if T: c = map(str, not_built) @@ -455,8 +522,7 @@ class Taskmaster: node.side_effects, 0) if side_effects: - self.pending.append(node) - node.set_state(SCons.Node.pending) + map(lambda n, P=node: n.add_to_waiting_s_e(P), side_effects) if S: S.side_effects = S.side_effects + 1 if T: c = map(str, side_effects) @@ -468,7 +534,7 @@ class Taskmaster: # this node is ready to be built. self.ready = node if S: S.build = S.build + 1 - if T: T.write(' evaluating\n') + if T: T.write(' evaluating %s\n' % node) break def next_task(self): @@ -485,10 +551,8 @@ class Taskmaster: tlist = node.builder.targets(node) except AttributeError: tlist = [node] - self.executing.extend(tlist) - self.executing.extend(node.side_effects) - task = self.tasker(self, tlist, node in self.targets, node) + task = self.tasker(self, tlist, node is self.current_top, node) try: task.make_ready() except KeyboardInterrupt: @@ -508,53 +572,19 @@ class Taskmaster: return task - def is_blocked(self): - self._find_next_ready_node() - - return not self.ready and (self.pending or self.executing) - - def next_top_level_candidate(self): - candidates = self.candidates[:] - candidates.reverse() - for c in candidates: - if c in self.targets: - return c - return None - def stop(self): """Stop the current build completely.""" - self.candidates = [] + self.next_candidate = self.no_next_candidate self.ready = None - self.pending = [] def failed(self, node): - try: - tlist = node.builder.targets(node) - except AttributeError: - tlist = [node] - for t in tlist: - self.executing.remove(t) - for side_effect in node.side_effects: - self.executing.remove(side_effect) + pass def executed(self, node): try: tlist = node.builder.targets(node) except AttributeError: tlist = [node] - for t in tlist: - self.executing.remove(t) - for side_effect in node.side_effects: - self.executing.remove(side_effect) - - # move the current pending nodes to the candidates list: - # (they may not all be ready to build, but _find_next_ready_node() - # will figure out which ones are really ready) - for node in self.pending: - node.set_state(SCons.Node.no_state) - self.pending.reverse() - self.candidates.extend(self.pending) - self.pending = [] def exception_raise(self, exception): exc = exception[:] diff --git a/src/engine/SCons/TaskmasterTests.py b/src/engine/SCons/TaskmasterTests.py index 7e24d2b7..8d71d71f 100644 --- a/src/engine/SCons/TaskmasterTests.py +++ b/src/engine/SCons/TaskmasterTests.py @@ -54,7 +54,8 @@ class Node: self.csig = None self.state = SCons.Node.no_state self.prepared = None - self.waiting_parents = [] + self.waiting_parents = {} + self.waiting_s_e = {} self.side_effect = 0 self.side_effects = [] self.alttargets = [] @@ -109,13 +110,13 @@ class Node: def scanner_key(self): return self.name - + def add_to_waiting_parents(self, node): - self.waiting_parents.append(node) - + self.waiting_parents[node] = 1 + def call_for_all_waiting_parents(self, func): func(self) - for parent in self.waiting_parents: + for parent in self.waiting_parents.keys(): parent.call_for_all_waiting_parents(func) def get_state(self): @@ -186,7 +187,7 @@ class TaskmasterTestCase(unittest.TestCase): n1 = Node("n1") n2 = Node("n2") n3 = Node("n3", [n1, n2]) - + tm = SCons.Taskmaster.Taskmaster([n3]) t = tm.next_task() @@ -194,18 +195,21 @@ class TaskmasterTestCase(unittest.TestCase): t.execute() assert built_text == "n1 built", built_text t.executed() + t.postprocess() t = tm.next_task() t.prepare() t.execute() assert built_text == "n2 built", built_text t.executed() + t.postprocess() t = tm.next_task() t.prepare() t.execute() assert built_text == "n3 built", built_text t.executed() + t.postprocess() assert tm.next_task() == None @@ -236,18 +240,21 @@ class TaskmasterTestCase(unittest.TestCase): t.execute() assert built_text == "n1 up-to-date", built_text t.executed() + t.postprocess() t = tm.next_task() t.prepare() t.execute() assert built_text == "n2 up-to-date", built_text t.executed() + t.postprocess() t = tm.next_task() t.prepare() t.execute() assert built_text == "n3 up-to-date top", built_text t.executed() + t.postprocess() assert tm.next_task() == None @@ -259,41 +266,34 @@ class TaskmasterTestCase(unittest.TestCase): n5 = Node("n5", [n3, n4]) tm = SCons.Taskmaster.Taskmaster([n5]) - assert not tm.is_blocked() - t1 = tm.next_task() assert t1.get_target() == n1 - assert not tm.is_blocked() - + t2 = tm.next_task() assert t2.get_target() == n2 - assert not tm.is_blocked() t4 = tm.next_task() assert t4.get_target() == n4 - assert tm.is_blocked() t4.executed() - assert tm.is_blocked() - + t4.postprocess() + t1.executed() - assert tm.is_blocked() + t1.postprocess() t2.executed() - assert not tm.is_blocked() + t2.postprocess() t3 = tm.next_task() assert t3.get_target() == n3 - assert tm.is_blocked() t3.executed() - assert not tm.is_blocked() + t3.postprocess() t5 = tm.next_task() assert t5.get_target() == n5, t5.get_target() - assert tm.is_blocked() # still executing t5 t5.executed() - assert not tm.is_blocked() + t5.postprocess() assert tm.next_task() == None - + n4 = Node("n4") n4.set_state(SCons.Node.executed) tm = SCons.Taskmaster.Taskmaster([n4]) @@ -303,9 +303,8 @@ class TaskmasterTestCase(unittest.TestCase): n2 = Node("n2", [n1]) tm = SCons.Taskmaster.Taskmaster([n2,n2]) t = tm.next_task() - assert tm.is_blocked() t.executed() - assert not tm.is_blocked() + t.postprocess() t = tm.next_task() assert tm.next_task() == None @@ -318,14 +317,17 @@ class TaskmasterTestCase(unittest.TestCase): target = t.get_target() assert target == n1, target t.executed() + t.postprocess() t = tm.next_task() target = t.get_target() assert target == n2, target t.executed() + t.postprocess() t = tm.next_task() target = t.get_target() assert target == n3, target t.executed() + t.postprocess() assert tm.next_task() == None n1 = Node("n1") @@ -339,15 +341,19 @@ class TaskmasterTestCase(unittest.TestCase): t = tm.next_task() assert t.get_target() == n1 t.executed() + t.postprocess() t = tm.next_task() assert t.get_target() == n2 t.executed() + t.postprocess() t = tm.next_task() assert t.get_target() == n3 t.executed() + t.postprocess() t = tm.next_task() assert t.get_target() == n4 t.executed() + t.postprocess() assert tm.next_task() == None assert scan_called == 4, scan_called @@ -368,28 +374,26 @@ class TaskmasterTestCase(unittest.TestCase): tm = SCons.Taskmaster.Taskmaster([n1,n2,n3,n4,n5]) t = tm.next_task() assert t.get_target() == n1 - assert n4.state == SCons.Node.executing - assert tm.is_blocked() + assert n4.state == SCons.Node.pending, n4.state t.executed() - assert not tm.is_blocked() + t.postprocess() t = tm.next_task() assert t.get_target() == n2 - assert tm.is_blocked() t.executed() + t.postprocess() t = tm.next_task() assert t.get_target() == n3 - assert tm.is_blocked() t.executed() + t.postprocess() t = tm.next_task() assert t.get_target() == n4 - assert tm.is_blocked() t.executed() + t.postprocess() t = tm.next_task() assert t.get_target() == n5 - assert tm.is_blocked() # still executing n5 assert not tm.next_task() t.executed() - assert not tm.is_blocked() + t.postprocess() n1 = Node("n1") n2 = Node("n2") @@ -402,31 +406,40 @@ class TaskmasterTestCase(unittest.TestCase): t = tm.next_task() assert t.get_target() == n3, t.get_target() t.executed() + t.postprocess() t = tm.next_task() assert t.get_target() == n2, t.get_target() t.executed() + t.postprocess() t = tm.next_task() assert t.get_target() == n1, t.get_target() t.executed() + t.postprocess() t = tm.next_task() assert t.get_target() == n4, t.get_target() t.executed() + t.postprocess() n5 = Node("n5") n6 = Node("n6") n7 = Node("n7") n6.alttargets = [n7] + tm = SCons.Taskmaster.Taskmaster([n5]) t = tm.next_task() assert t.get_target() == n5 t.executed() + t.postprocess() + tm = SCons.Taskmaster.Taskmaster([n6]) t = tm.next_task() assert t.get_target() == n7 t.executed() + t.postprocess() t = tm.next_task() assert t.get_target() == n6 t.executed() + t.postprocess() n1 = Node("n1") n2 = Node("n2", [n1]) @@ -441,6 +454,7 @@ class TaskmasterTestCase(unittest.TestCase): tm = SCons.Taskmaster.Taskmaster([n1]) t = tm.next_task() t.executed() + t.postprocess() s = n1.get_state() assert s == SCons.Node.up_to_date, s @@ -590,56 +604,19 @@ class TaskmasterTestCase(unittest.TestCase): def test_cycle_detection(self): """Test detecting dependency cycles - """ n1 = Node("n1") n2 = Node("n2", [n1]) n3 = Node("n3", [n2]) n1.kids = [n3] + tm = SCons.Taskmaster.Taskmaster([n3]) try: - tm = SCons.Taskmaster.Taskmaster([n3]) t = tm.next_task() except SCons.Errors.UserError, e: assert str(e) == "Dependency cycle: n3 -> n1 -> n2 -> n3", str(e) else: - assert 0 - - def test_is_blocked(self): - """Test whether a task is blocked - - Both default and overridden in a subclass. - """ - tm = SCons.Taskmaster.Taskmaster() - assert not tm.is_blocked() - - class MyTM(SCons.Taskmaster.Taskmaster): - def _find_next_ready_node(self): - self.ready = 1 - tm = MyTM() - assert not tm.is_blocked() - - class MyTM(SCons.Taskmaster.Taskmaster): - def _find_next_ready_node(self): - self.ready = None - self.pending = [] - self.executing = [] - tm = MyTM() - assert not tm.is_blocked() - - class MyTM(SCons.Taskmaster.Taskmaster): - def _find_next_ready_node(self): - self.ready = None - self.pending = [1] - tm = MyTM() - assert tm.is_blocked() - - class MyTM(SCons.Taskmaster.Taskmaster): - def _find_next_ready_node(self): - self.ready = None - self.executing = [1] - tm = MyTM() - assert tm.is_blocked() + assert 'Did not catch expected UserError' def test_next_top_level_candidate(self): """Test the next_top_level_candidate() method @@ -650,9 +627,9 @@ class TaskmasterTestCase(unittest.TestCase): tm = SCons.Taskmaster.Taskmaster([n3]) t = tm.next_task() - assert tm.executing == [n1], tm.executing + assert t.targets == [n1], t.targets t.fail_stop() - assert t.targets == [n3], t.targets + assert t.targets == [n3], map(str, t.targets) assert t.top == 1, t.top def test_stop(self): @@ -665,13 +642,14 @@ class TaskmasterTestCase(unittest.TestCase): n1 = Node("n1") n2 = Node("n2") n3 = Node("n3", [n1, n2]) - + tm = SCons.Taskmaster.Taskmaster([n3]) t = tm.next_task() t.prepare() t.execute() assert built_text == "n1 built", built_text t.executed() + t.postprocess() assert built_text == "n1 built really", built_text tm.stop() @@ -702,9 +680,9 @@ class TaskmasterTestCase(unittest.TestCase): n1 = Node("n1") tm = SCons.Taskmaster.Taskmaster([n1]) t = tm.next_task() - assert tm.executing == [n1], tm.executing + assert t.targets == [n1], map(str, t.targets) tm.failed(n1) - assert tm.executing == [], tm.executing + assert t.targets == [n1], map(str, t.targets) def test_executed(self): """Test when a task has been executed @@ -759,7 +737,6 @@ class TaskmasterTestCase(unittest.TestCase): def test_prepare(self): """Test preparation of multiple Nodes for a task - """ n1 = Node("n1") n2 = Node("n2") @@ -818,7 +795,7 @@ class TaskmasterTestCase(unittest.TestCase): n6.side_effects = [ n8 ] n7.side_effects = [ n9, n10 ] - + tm = SCons.Taskmaster.Taskmaster([n6, n7]) t = tm.next_task() # More bogus reaching in and setting the targets. @@ -832,7 +809,6 @@ class TaskmasterTestCase(unittest.TestCase): def test_execute(self): """Test executing a task - """ global built_text global cache_text @@ -994,7 +970,7 @@ class TaskmasterTestCase(unittest.TestCase): t.exception_set(("exception 4", "XYZZY")) def fw_exc(exc): raise 'exception_forwarded', exc - tm.exception_raise = fw_exc + tm.exception_raise = fw_exc try: t.exception_raise() except: @@ -1007,7 +983,6 @@ class TaskmasterTestCase(unittest.TestCase): def test_postprocess(self): """Test postprocessing targets to give them a chance to clean up - """ n1 = Node("n1") tm = SCons.Taskmaster.Taskmaster([n1]) @@ -1053,17 +1028,13 @@ class TaskmasterTestCase(unittest.TestCase): value = trace.getvalue() expect = """\ -Taskmaster: 'n1': children: - [] - evaluating -Taskmaster: 'n1': already handled +Taskmaster: 'n1': evaluating n1 +Taskmaster: 'n1': already handled (executed) Taskmaster: 'n3': children: ['n1', 'n2'] waiting on unstarted children: ['n2'] -Taskmaster: 'n2': children: - [] - evaluating +Taskmaster: 'n2': evaluating n2 Taskmaster: 'n3': children: ['n1', 'n2'] waiting on unfinished children: diff --git a/src/engine/SCons/Tool/dvi.py b/src/engine/SCons/Tool/dvi.py index 1be710c1..fce98509 100644 --- a/src/engine/SCons/Tool/dvi.py +++ b/src/engine/SCons/Tool/dvi.py @@ -47,7 +47,8 @@ def generate(env): DVIBuilder = SCons.Builder.Builder(action = {}, source_scanner = SCons.Tool.LaTeXScanner, suffix = '.dvi', - emitter = {}) + emitter = {}, + source_ext_match = None) env['BUILDERS']['DVI'] = DVIBuilder diff --git a/src/engine/SCons/Tool/dvipdf.py b/src/engine/SCons/Tool/dvipdf.py index bade8ff2..51dfae18 100644 --- a/src/engine/SCons/Tool/dvipdf.py +++ b/src/engine/SCons/Tool/dvipdf.py @@ -66,7 +66,7 @@ def generate(env): env['DVIPDF'] = 'dvipdf' env['DVIPDFFLAGS'] = SCons.Util.CLVar('') - env['DVIPDFCOM'] = '$DVIPDF $DVIPDFFLAGS $SOURCES $TARGET' + env['DVIPDFCOM'] = '$DVIPDF $DVIPDFFLAGS $SOURCE $TARGET' # Deprecated synonym. env['PDFCOM'] = ['$DVIPDFCOM'] diff --git a/src/engine/SCons/Tool/intelc.py b/src/engine/SCons/Tool/intelc.py index b247a384..e668bf0c 100644 --- a/src/engine/SCons/Tool/intelc.py +++ b/src/engine/SCons/Tool/intelc.py @@ -71,12 +71,18 @@ def linux_ver_normalize(vstr): is greater than 60 it's an old-style number and otherwise new-style. Always returns an old-style float like 80 or 90 for compatibility with Windows. Shades of Y2K!""" - f = float(vstr) - if is_windows: - return f + # Check for version number like 9.1.026: return 91.026 + m = re.match(r'([0-9]+)\.([0-9]+)\.([0-9]+)', vstr) + if m: + vmaj,vmin,build = m.groups() + return float(vmaj) * 10 + float(vmin) + float(build) / 1000.; else: - if f < 60: return f * 10.0 - else: return f + f = float(vstr) + if is_windows: + return f + else: + if f < 60: return f * 10.0 + else: return f def check_abi(abi): """Check for valid ABI (application binary interface) name, diff --git a/src/engine/SCons/Tool/latex.py b/src/engine/SCons/Tool/latex.py index 5bd21d9f..72371b3a 100644 --- a/src/engine/SCons/Tool/latex.py +++ b/src/engine/SCons/Tool/latex.py @@ -64,7 +64,7 @@ def generate(env): env['LATEX'] = 'latex' env['LATEXFLAGS'] = SCons.Util.CLVar('') - env['LATEXCOM'] = '$LATEX $LATEXFLAGS $SOURCES' + env['LATEXCOM'] = '$LATEX $LATEXFLAGS $SOURCE' env['LATEXRETRIES'] = 3 def exists(env): diff --git a/src/engine/SCons/Tool/msvc.py b/src/engine/SCons/Tool/msvc.py index 80c5896c..86cde784 100644 --- a/src/engine/SCons/Tool/msvc.py +++ b/src/engine/SCons/Tool/msvc.py @@ -296,7 +296,7 @@ def _get_msvc8_path(path, version, platform, suite): "Unable to retrieve the %s path from MS VC++."%path # collect some useful information for later expansions... - paths = SCons.Tool.msvs.get_msvs_install_dirs(version) + paths = SCons.Tool.msvs.get_msvs_install_dirs(version, suite) # expand the directory path variables that we support. If there # is a variable we don't support, then replace that entry with @@ -474,7 +474,7 @@ def _get_msvc8_default_paths(env, version, suite, use_mfc_dirs): lib_paths = [] include_paths = [] try: - paths = SCons.Tool.msvs.get_msvs_install_dirs(version) + paths = SCons.Tool.msvs.get_msvs_install_dirs(version, suite) MVSdir = paths['VSINSTALLDIR'] except (KeyError, SCons.Util.RegError, SCons.Errors.InternalError): if os.environ.has_key('VSCOMNTOOLS'): diff --git a/src/engine/SCons/Tool/msvs.py b/src/engine/SCons/Tool/msvs.py index e17dcfd8..e8aaf834 100644 --- a/src/engine/SCons/Tool/msvs.py +++ b/src/engine/SCons/Tool/msvs.py @@ -1295,7 +1295,7 @@ def get_visualstudio8_suites(): except SCons.Util.RegError: pass - # Detect Expression edition + # Detect Express edition try: idk = SCons.Util.RegOpenKeyEx(SCons.Util.HKEY_LOCAL_MACHINE, r'Software\Microsoft\VCExpress\8.0') @@ -1316,7 +1316,7 @@ def is_msvs_installed(): except (SCons.Util.RegError, SCons.Errors.InternalError): return 0 -def get_msvs_install_dirs(version = None): +def get_msvs_install_dirs(version = None, vs8suite = None): """ Get installed locations for various msvc-related products, like the .NET SDK and the Platform SDK. @@ -1336,9 +1336,14 @@ def get_msvs_install_dirs(version = None): K = 'Software\\Microsoft\\VisualStudio\\' + str(version_num) if (version_num >= 8.0): - try: - SCons.Util.RegOpenKeyEx(SCons.Util.HKEY_LOCAL_MACHINE, K ) - except SCons.Util.RegError: + if vs8suite == None: + # We've been given no guidance about which Visual Studio 8 + # suite to use, so attempt to autodetect. + suites = get_visualstudio8_suites() + if suites: + vs8suite = suites[0] + + if vs8suite == 'EXPRESS': K = 'Software\\Microsoft\\VCExpress\\' + str(version_num) # vc++ install dir diff --git a/src/engine/SCons/Tool/msvs.xml b/src/engine/SCons/Tool/msvs.xml index 56180b00..c16beaaa 100644 --- a/src/engine/SCons/Tool/msvs.xml +++ b/src/engine/SCons/Tool/msvs.xml @@ -131,17 +131,17 @@ barlocalincs = ['StdAfx.h'] barresources = ['bar.rc','resource.h'] barmisc = ['bar_readme.txt'] -dll = local.SharedLibrary(target = 'bar.dll', - source = barsrcs) - -local.MSVSProject(target = 'Bar' + env['MSVSPROJECTSUFFIX'], - srcs = barsrcs, - incs = barincs, - localincs = barlocalincs, - resources = barresources, - misc = barmisc, - buildtarget = dll, - variant = 'Release') +dll = env.SharedLibrary(target = 'bar.dll', + source = barsrcs) + +env.MSVSProject(target = 'Bar' + env['MSVSPROJECTSUFFIX'], + srcs = barsrcs, + incs = barincs, + localincs = barlocalincs, + resources = barresources, + misc = barmisc, + buildtarget = dll, + variant = 'Release') @@ -193,9 +193,9 @@ not the source files used to build the solution file. Example Usage: -local.MSVSSolution(target = 'Bar' + env['MSVSSOLUTIONSUFFIX'], - projects = ['bar' + env['MSVSPROJECTSUFFIX']], - variant = 'Release') +env.MSVSSolution(target = 'Bar' + env['MSVSSOLUTIONSUFFIX'], + projects = ['bar' + env['MSVSPROJECTSUFFIX']], + variant = 'Release') diff --git a/src/engine/SCons/Tool/pdf.py b/src/engine/SCons/Tool/pdf.py index b4bfc17b..0f6468b0 100644 --- a/src/engine/SCons/Tool/pdf.py +++ b/src/engine/SCons/Tool/pdf.py @@ -44,7 +44,8 @@ def generate(env): source_scanner = SCons.Tool.LaTeXScanner, prefix = '$PDFPREFIX', suffix = '$PDFSUFFIX', - emitter = {}) + emitter = {}, + source_ext_match = None) env['BUILDERS']['PDF'] = PDFBuilder env['PDFPREFIX'] = '' diff --git a/src/engine/SCons/Tool/pdftex.py b/src/engine/SCons/Tool/pdftex.py index 075315d8..ddf5a231 100644 --- a/src/engine/SCons/Tool/pdftex.py +++ b/src/engine/SCons/Tool/pdftex.py @@ -87,12 +87,12 @@ def generate(env): # Duplicate from latex.py. If latex.py goes away, then this is still OK. env['PDFLATEX'] = 'pdflatex' env['PDFLATEXFLAGS'] = SCons.Util.CLVar('') - env['PDFLATEXCOM'] = '$PDFLATEX $PDFLATEXFLAGS $SOURCES' + env['PDFLATEXCOM'] = '$PDFLATEX $PDFLATEXFLAGS $SOURCE' env['LATEXRETRIES'] = 3 env['BIBTEX'] = 'bibtex' env['BIBTEXFLAGS'] = SCons.Util.CLVar('') - env['BIBTEXCOM'] = '$BIBTEX $BIBTEXFLAGS $SOURCES' + env['BIBTEXCOM'] = '$BIBTEX $BIBTEXFLAGS ${SOURCE.base}' def exists(env): return env.Detect('pdftex') diff --git a/src/engine/SCons/Tool/tex.py b/src/engine/SCons/Tool/tex.py index 1ca40f20..d6139588 100644 --- a/src/engine/SCons/Tool/tex.py +++ b/src/engine/SCons/Tool/tex.py @@ -42,6 +42,10 @@ import SCons.Node import SCons.Node.FS import SCons.Util +warning_rerun_re = re.compile("^LaTeX Warning:.*Rerun", re.MULTILINE) +undefined_references_re = re.compile("^LaTeX Warning:.*undefined references", re.MULTILINE) +openout_aux_re = re.compile(r"\\openout.*`(.*\.aux)'") + # An Action sufficient to build any generic tex file. TeXAction = None @@ -62,28 +66,36 @@ def InternalLaTeXAuxAction(XXXLaTeXAction, target = None, source= None, env=None basename, ext = SCons.Util.splitext(str(target[0])) # Run LaTeX once to generate a new aux file. - XXXLaTeXAction(target,source,env) + XXXLaTeXAction(target, source, env) # Decide if various things need to be run, or run again. We check # for the existence of files before opening them--even ones like the # aux file that TeX always creates--to make it possible to write tests # with stubs that don't necessarily generate all of the same files. + # Read the log file to find all .aux files + logfilename = basename + '.log' + auxfiles = [] + if os.path.exists(logfilename): + content = open(logfilename, "rb").read() + auxfiles = openout_aux_re.findall(content) + # Now decide if bibtex will need to be run. - auxfilename = basename + '.aux' - if os.path.exists(auxfilename): - content = open(auxfilename, "rb").read() - if string.find(content, "bibdata") != -1: - bibfile = env.fs.File(basename) - BibTeXAction(None,bibfile,env) + for auxfilename in auxfiles: + if os.path.exists(auxfilename): + content = open(auxfilename, "rb").read() + if string.find(content, "bibdata") != -1: + bibfile = env.fs.File(basename) + BibTeXAction(None, bibfile, env) + break # Now decide if makeindex will need to be run. idxfilename = basename + '.idx' if os.path.exists(idxfilename): idxfile = env.fs.File(basename) # TODO: if ( idxfile has changed) ... - MakeIndexAction(None,idxfile,env) - LaTeXAction(target,source,env) + MakeIndexAction(None, idxfile, env) + XXXLaTeXAction(target, source, env) # Now decide if latex needs to be run yet again. logfilename = basename + '.log' @@ -91,9 +103,10 @@ def InternalLaTeXAuxAction(XXXLaTeXAction, target = None, source= None, env=None if not os.path.exists(logfilename): break content = open(logfilename, "rb").read() - if not re.search("^LaTeX Warning:.*Rerun",content,re.MULTILINE) and not re.search("^LaTeX Warning:.*undefined references",content,re.MULTILINE): + if not warning_rerun_re.search(content) and \ + not undefined_references_re.search(content): break - XXXLaTeXAction(target,source,env) + XXXLaTeXAction(target, source, env) return 0 def LaTeXAuxAction(target = None, source= None, env=None): @@ -123,6 +136,23 @@ def tex_emitter(target, source, env): base = SCons.Util.splitext(str(source[0]))[0] target.append(base + '.aux') target.append(base + '.log') + for f in source: + content = f.get_contents() + if string.find(content, r'\makeindex') != -1: + target.append(base + '.ilg') + target.append(base + '.ind') + target.append(base + '.idx') + if string.find(content, r'\bibliography') != -1: + target.append(base + '.bbl') + target.append(base + '.blg') + + # read log file to get all .aux files + logfilename = base + '.log' + if os.path.exists(logfilename): + content = open(logfilename, "rb").read() + aux_files = openout_aux_re.findall(content) + target.extend(filter(lambda f, b=base+'.aux': f != b, aux_files)) + return (target, source) TeXLaTeXAction = None @@ -149,7 +179,7 @@ def generate(env): # Define an action to run MakeIndex on a file. global MakeIndexAction if MakeIndexAction is None: - MakeIndexAction = SCons.Action.Action("$MAKEINDEXCOM", "$MAKEINDEXOMSTR") + MakeIndexAction = SCons.Action.Action("$MAKEINDEXCOM", "$MAKEINDEXCOMSTR") global TeXLaTeXAction if TeXLaTeXAction is None: @@ -174,7 +204,7 @@ def generate(env): env['BIBTEX'] = 'bibtex' env['BIBTEXFLAGS'] = SCons.Util.CLVar('') - env['BIBTEXCOM'] = '$BIBTEX $BIBTEXFLAGS $SOURCE' + env['BIBTEXCOM'] = '$BIBTEX $BIBTEXFLAGS ${SOURCE.base}' env['MAKEINDEX'] = 'makeindex' env['MAKEINDEXFLAGS'] = SCons.Util.CLVar('') diff --git a/src/engine/SCons/Util.py b/src/engine/SCons/Util.py index 71126d26..d673d22a 100644 --- a/src/engine/SCons/Util.py +++ b/src/engine/SCons/Util.py @@ -48,6 +48,7 @@ DictType = types.DictType InstanceType = types.InstanceType ListType = types.ListType StringType = types.StringType +TupleType = types.TupleType try: from UserString import UserString @@ -424,6 +425,10 @@ def is_List(obj): return t is ListType \ or (t is InstanceType and isinstance(obj, UserList)) +def is_Tuple(obj): + t = type(obj) + return t is TupleType + if hasattr(types, 'UnicodeType'): def is_String(obj): t = type(obj) @@ -439,7 +444,7 @@ else: def is_Scalar(e): - return is_String(e) or not is_List(e) + return is_String(e) or (not is_List(e) and not is_Tuple(e)) def flatten(sequence, scalarp=is_Scalar, result=None): if result is None: @@ -575,7 +580,7 @@ if sys.platform == 'win32': if string.lower(ext) == string.lower(file[-len(ext):]): pathext = [''] break - if not is_List(reject): + if not is_List(reject) and not is_Tuple(reject): reject = [reject] for dir in path: f = os.path.join(dir, file) @@ -605,7 +610,7 @@ elif os.name == 'os2': if string.lower(ext) == string.lower(file[-len(ext):]): pathext = [''] break - if not is_List(reject): + if not is_List(reject) and not is_Tuple(reject): reject = [reject] for dir in path: f = os.path.join(dir, file) @@ -629,7 +634,7 @@ else: return None if is_String(path): path = string.split(path, os.pathsep) - if not is_List(reject): + if not is_List(reject) and not is_Tuple(reject): reject = [reject] for d in path: f = os.path.join(d, file) @@ -668,11 +673,11 @@ def PrependPath(oldpath, newpath, sep = os.pathsep): orig = oldpath is_list = 1 paths = orig - if not is_List(orig): + if not is_List(orig) and not is_Tuple(orig): paths = string.split(paths, sep) is_list = 0 - if is_List(newpath): + if is_List(newpath) or is_Tuple(newpath): newpaths = newpath else: newpaths = string.split(newpath, sep) @@ -711,11 +716,11 @@ def AppendPath(oldpath, newpath, sep = os.pathsep): orig = oldpath is_list = 1 paths = orig - if not is_List(orig): + if not is_List(orig) and not is_Tuple(orig): paths = string.split(paths, sep) is_list = 0 - if is_List(newpath): + if is_List(newpath) or is_Tuple(newpath): newpaths = newpath else: newpaths = string.split(newpath, sep) @@ -753,7 +758,7 @@ else: display = DisplayEngine() def Split(arg): - if is_List(arg): + if is_List(arg) or is_Tuple(arg): return arg elif is_String(arg): return string.split(arg) diff --git a/src/engine/SCons/UtilTests.py b/src/engine/SCons/UtilTests.py index aa93db1a..e2916621 100644 --- a/src/engine/SCons/UtilTests.py +++ b/src/engine/SCons/UtilTests.py @@ -188,6 +188,7 @@ class UtilTestCase(unittest.TestCase): assert is_Dict({}) assert is_Dict(UserDict()) assert not is_Dict([]) + assert not is_Dict(()) assert not is_Dict("") if hasattr(types, 'UnicodeType'): exec "assert not is_Dict(u'')" @@ -196,6 +197,7 @@ class UtilTestCase(unittest.TestCase): assert is_List([]) import UserList assert is_List(UserList.UserList()) + assert not is_List(()) assert not is_List({}) assert not is_List("") if hasattr(types, 'UnicodeType'): @@ -213,6 +215,15 @@ class UtilTestCase(unittest.TestCase): assert is_String(UserString.UserString('')) assert not is_String({}) assert not is_String([]) + assert not is_String(()) + + def test_is_Tuple(self): + assert is_Tuple(()) + assert not is_Tuple([]) + assert not is_Tuple({}) + assert not is_Tuple("") + if hasattr(types, 'UnicodeType'): + exec "assert not is_Tuple(u'')" def test_to_String(self): """Test the to_String() method.""" diff --git a/src/setup.py b/src/setup.py index 2eed585a..68b46ab0 100644 --- a/src/setup.py +++ b/src/setup.py @@ -29,7 +29,7 @@ import stat import string import sys -Version = "0.96" +Version = "0.96.92" (head, tail) = os.path.split(sys.argv[0]) diff --git a/src/test_copyrights.py b/src/test_copyrights.py index 44dc8fa9..195649bf 100644 --- a/src/test_copyrights.py +++ b/src/test_copyrights.py @@ -46,84 +46,93 @@ try: except KeyError: cwd = os.getcwd() +build_scons = os.path.join(cwd, 'build', 'scons') +build_local = os.path.join(cwd, 'build', 'scons-local', 'scons-local-0.96.92') +build_src = os.path.join(cwd, 'build', 'scons-src') + class Collect: expression = re.compile('Copyright.*The SCons Foundation') - def __init__(self, remove_list): + def __init__(self, directory, remove_list): self.copyright = [] self.no_copyright = [] - self.remove_list = remove_list + self.remove = {} + for r in remove_list: + self.remove[os.path.join(directory, r)] = 1 def visit(collect, dirname, names): - for r in collect.remove_list: - try: - names.remove(r) - except ValueError: - pass - for name in map(lambda n, d=dirname: os.path.join(d, n), names): - if not os.path.isfile(name): - continue - if collect.expression.search(open(name, 'r').read()): - collect.copyright.append(name) - else: - collect.no_copyright.append(name) - -remove_list = [ + make_path_tuple = lambda n, d=dirname: (n, os.path.join(d, n)) + for name, path in map(make_path_tuple, names): + if collect.remove.get(path): + names.remove(name) + elif os.path.isfile(path): + if collect.expression.search(open(path, 'r').read()): + collect.copyright.append(path) + else: + collect.no_copyright.append(path) + +# Map each directory to search (dictionary keys) to a list of its +# subsidiary files and directories to exclude from copyright checks. +check = { + build_scons : [ 'build', - 'debian', 'dist', - 'Optik', - 'dblite.py', - 'Conftest.py', + 'engine/SCons/Conftest.py', + 'engine/SCons/dblite.py', + 'engine/SCons/Optik', 'MANIFEST', 'os_spawnv_fix.diff', 'setup.cfg', - 'SCons-win32-install-1.jpg', - 'SCons-win32-install-2.jpg', - 'SCons-win32-install-3.jpg', - 'SCons-win32-install-4.jpg', -] - -src_remove_list = [ + ], + build_local : [ + 'SCons/Conftest.py', + 'SCons/dblite.py', + 'SCons/Optik', + ], + build_src : [ 'bin', - 'cons.pl', - 'design', - 'python10', - 'reference', - 'etc', - 'gentoo', 'config', - 'MANIFEST.in', - 'MANIFEST-xml.in', -] - -# XXX Remove '*-stamp' when we get rid of those. -scons = Collect(remove_list + ['build-stamp', 'configure-stamp']) -# XXX Remove '.sconsign' when we start using SConsignFile() for SCons builds. -local = Collect(remove_list + ['.sconsign']) -src = Collect(remove_list + src_remove_list) - -build_scons = os.path.join(cwd, 'build', 'scons') -build_local = os.path.join(cwd, 'build', 'scons-local') -build_src = os.path.join(cwd, 'build', 'scons-src') - + 'debian', + 'doc/design', + 'doc/MANIFEST', + 'doc/python10', + 'doc/reference', + 'doc/man/MANIFEST', + 'doc/user/cons.pl', + 'doc/user/MANIFEST', + 'doc/user/SCons-win32-install-1.jpg', + 'doc/user/SCons-win32-install-2.jpg', + 'doc/user/SCons-win32-install-3.jpg', + 'doc/user/SCons-win32-install-4.jpg', + 'gentoo', + 'QMTest/classes.qmc', + 'QMTest/configuration', + 'QMTest/TestCmd.py', + 'QMTest/TestCommon.py', + 'QMTest/unittest.py', + 'src/os_spawnv_fix.diff', + 'src/MANIFEST.in', + 'src/setup.cfg', + 'src/engine/MANIFEST.in', + 'src/engine/MANIFEST-xml.in', + 'src/engine/setup.cfg', + 'src/engine/SCons/Conftest.py', + 'src/engine/SCons/dblite.py', + 'src/engine/SCons/Optik', + 'src/script/MANIFEST.in', + 'src/script/setup.cfg', + ], +} + +no_copyright = [] no_result = [] -if os.path.exists(build_scons): - os.path.walk(build_scons, visit, scons) -else: - no_result.append(build_scons) - -if os.path.exists(build_local): - os.path.walk(build_local, visit, local) -else: - no_result.append(build_local) - -if os.path.exists(build_src): - os.path.walk(build_src, visit, src) -else: - no_result.append(build_src) - -no_copyright = scons.no_copyright + local.no_copyright + src.no_copyright +for directory, remove_list in check.items(): + if os.path.exists(directory): + c = Collect(directory, remove_list) + os.path.walk(directory, visit, c) + no_copyright.extend(c.no_copyright) + else: + no_result.append(directory) if no_copyright: print "Found the following files with no copyrights:" @@ -135,5 +144,4 @@ if no_result: print "\t" + string.join(no_result, "\n\t") test.no_result(1) -# All done. test.pass_test() diff --git a/src/test_setup.py b/src/test_setup.py index b49dde42..d6128972 100644 --- a/src/test_setup.py +++ b/src/test_setup.py @@ -44,7 +44,7 @@ except NameError: WindowsError = OSError # version = os.environ['SCONS_VERSION'] #except KeyError: # version = '__VERSION__' -version = '0.96' +version = '0.96.92' scons_version = 'scons-%s' % version diff --git a/test/Alias/Dir-order.py b/test/Alias/Dir-order.py new file mode 100644 index 00000000..8f1388d7 --- /dev/null +++ b/test/Alias/Dir-order.py @@ -0,0 +1,43 @@ +#!/usr/bin/env python +# +# __COPYRIGHT__ +# +# Permission is hereby granted, free of charge, to any person obtaining +# a copy of this software and associated documentation files (the +# "Software"), to deal in the Software without restriction, including +# without limitation the rights to use, copy, modify, merge, publish, +# distribute, sublicense, and/or sell copies of the Software, and to +# permit persons to whom the Software is furnished to do so, subject to +# the following conditions: +# +# The above copyright notice and this permission notice shall be included +# in all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY +# KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE +# WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE +# LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION +# WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. +# + +__revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" + +""" +Validate that calling Dir() for a string after we've used it as an +Alias() expansion works. +""" + +import TestSCons + +test = TestSCons.TestSCons() + +test.write('SConstruct', """ +Alias('afoo', 'foo') +f = Dir('foo') +""") + +test.run(arguments = '.') + +test.pass_test() diff --git a/test/CVS.py b/test/CVS.py index 79257df9..3002ee76 100644 --- a/test/CVS.py +++ b/test/CVS.py @@ -276,23 +276,23 @@ test.must_match(['work3', 'all'], mode='r') # Test CVS checkouts from a remote server (Tigris.org). -test.subdir(['work4']) - -test.write(['work4', 'SConstruct'], """\ -import os -env = Environment(ENV = { 'PATH' : os.environ['PATH'] }) -# We used to use the SourceForge server, but SourceForge has restrictions -# that make them deny access on occasion. Leave the incantation here -# in case we need to use it again some day. -#cvs = env.CVS(':pserver:anonymous@cvs.sourceforge.net:/cvsroot/scons') -cvs = env.CVS(':pserver:anoncvs@cvs.tigris.org:/cvs') -env.SourceCode('.', cvs) -env.Install('install', 'scons/SConstruct') -""") - -test.run(chdir = 'work4', arguments = '.') - -test.must_exist(test.workpath('work4', 'install', 'SConstruct')) +#test.subdir(['work4']) +# +#test.write(['work4', 'SConstruct'], """\ +#import os +#env = Environment(ENV = { 'PATH' : os.environ['PATH'] }) +## We used to use the SourceForge server, but SourceForge has restrictions +## that make them deny access on occasion. Leave the incantation here +## in case we need to use it again some day. +##cvs = env.CVS(':pserver:anonymous@cvs.sourceforge.net:/cvsroot/scons') +#cvs = env.CVS(':pserver:anoncvs@cvs.tigris.org:/cvs') +#env.SourceCode('.', cvs) +#env.Install('install', 'scons/SConstruct') +#""") +# +#test.run(chdir = 'work4', arguments = '.') +# +#test.must_exist(test.workpath('work4', 'install', 'SConstruct')) test.pass_test() diff --git a/test/Configure.py b/test/Configure/Configure.py similarity index 94% rename from test/Configure.py rename to test/Configure/Configure.py index 2512907b..41c858b1 100644 --- a/test/Configure.py +++ b/test/Configure/Configure.py @@ -224,10 +224,10 @@ if not (r1 and r2 and r3 and r4 and r5 and r6): """ % (lib,lib)) test.run(chdir=work_dir) - checkLogAndStdout(["Checking for main() in C library %s... " % lib, - "Checking for main() in C library None... ", - "Checking for main() in C library %s... " % lib, - "Checking for main() in C library None... ", + checkLogAndStdout(["Checking for C library %s... " % lib, + "Checking for C library None... ", + "Checking for C library %s... " % lib, + "Checking for C library None... ", "Checking for C header file math.h... ", "Checking for C++ header file vector... "], ["yes"]*6, @@ -238,10 +238,10 @@ if not (r1 and r2 and r3 and r4 and r5 and r6): test.run(chdir=work_dir) - checkLogAndStdout(["Checking for main() in C library %s... " % lib, - "Checking for main() in C library None... ", - "Checking for main() in C library %s... " % lib, - "Checking for main() in C library None... ", + checkLogAndStdout(["Checking for C library %s... " % lib, + "Checking for C library None... ", + "Checking for C library %s... " % lib, + "Checking for C library None... ", "Checking for C header file math.h... ", "Checking for C++ header file vector... "], ["yes"]*6, @@ -253,10 +253,10 @@ if not (r1 and r2 and r3 and r4 and r5 and r6): # same should be true for TargetSignatures('content') test.run(chdir=work_dir, arguments='target_signatures_content=1 --config=force') - checkLogAndStdout(["Checking for main() in C library %s... " % lib, - "Checking for main() in C library None... ", - "Checking for main() in C library %s... " % lib, - "Checking for main() in C library None... ", + checkLogAndStdout(["Checking for C library %s... " % lib, + "Checking for C library None... ", + "Checking for C library %s... " % lib, + "Checking for C library None... ", "Checking for C header file math.h... ", "Checking for C++ header file vector... "], ["yes"]*6, @@ -266,10 +266,10 @@ if not (r1 and r2 and r3 and r4 and r5 and r6): test, "config.log", ".sconf_temp", "SConstruct") test.run(chdir=work_dir, arguments='target_signatures_content=1') - checkLogAndStdout(["Checking for main() in C library %s... " % lib, - "Checking for main() in C library None... ", - "Checking for main() in C library %s... " % lib, - "Checking for main() in C library None... ", + checkLogAndStdout(["Checking for C library %s... " % lib, + "Checking for C library None... ", + "Checking for C library %s... " % lib, + "Checking for C library None... ", "Checking for C header file math.h... ", "Checking for C++ header file vector... "], ["yes"]*6, @@ -299,7 +299,7 @@ if not (not r1 and not r2): test.run(chdir=work_dir) checkLogAndStdout(["Checking for C header file no_std_c_header.h... ", - "Checking for main() in C library no_c_library_SAFFDG... "], + "Checking for C library no_c_library_SAFFDG... "], ["no"]*2, [[((".c", NCR), (_obj, NCF))], [((".c", NCR), (_obj, NCR), (_exe, NCF))]], @@ -307,7 +307,7 @@ if not (not r1 and not r2): test.run(chdir=work_dir) checkLogAndStdout(["Checking for C header file no_std_c_header.h... ", - "Checking for main() in C library no_c_library_SAFFDG... "], + "Checking for C library no_c_library_SAFFDG... "], ["no"]*2, [[((".c", CR), (_obj, CF))], [((".c", CR), (_obj, CR), (_exe, CF))]], @@ -316,7 +316,7 @@ if not (not r1 and not r2): # 1.3 same should be true for TargetSignatures('content') test.run(chdir=work_dir, arguments='--config=force target_signatures_content=1') checkLogAndStdout(["Checking for C header file no_std_c_header.h... ", - "Checking for main() in C library no_c_library_SAFFDG... "], + "Checking for C library no_c_library_SAFFDG... "], ["no"]*2, [[((".c", NCR), (_obj, NCF))], [((".c", NCR), (_obj, NCR), (_exe, NCF))]], @@ -324,7 +324,7 @@ if not (not r1 and not r2): test.run(chdir=work_dir, arguments='target_signatures_content=1') checkLogAndStdout(["Checking for C header file no_std_c_header.h... ", - "Checking for main() in C library no_c_library_SAFFDG... "], + "Checking for C library no_c_library_SAFFDG... "], ["no"]*2, [[((".c", CR), (_obj, CF))], [((".c", CR), (_obj, CR), (_exe, CF))]], @@ -698,8 +698,8 @@ File "SConstruct", line 6, in ? """ % os.path.join(".sconf_temp", "conftest_0.c")) test.run(chdir=work_dir) - checkLogAndStdout( ["Checking for main() in C library %s... " % lib, - "Checking for main() in C library hopefullynolib... "], + checkLogAndStdout( ["Checking for C library %s... " % lib, + "Checking for C library hopefullynolib... "], ["yes", "no"], [[((".c", NCR), (_obj, NCR))], [((".c", NCR), (_obj, NCF))]], @@ -707,8 +707,8 @@ File "SConstruct", line 6, in ? oldLog = test.read(test.workpath(work_dir, 'config.log')) test.run(chdir=work_dir, arguments='-n') - checkLogAndStdout( ["Checking for main() in C library %s... " % lib, - "Checking for main() in C library hopefullynolib... "], + checkLogAndStdout( ["Checking for C library %s... " % lib, + "Checking for C library hopefullynolib... "], ["yes", "no"], [[((".c", CR), (_obj, CR))], [((".c", CR), (_obj, CF))]], @@ -842,9 +842,9 @@ Checking for C++ header file vector... yes Checking for C++ header file hopefullynocxx-header.h... no Checking for sin() in C library %(lib)s... yes Checking for sin() in C library hopefullynolib... no -Checking for main() in C library %(lib)s... yes -Checking for main() in C library %(lib)s... no -Checking for main() in C library hopefullynolib2... no +Checking for C library %(lib)s... yes +Checking for C library %(lib)s... no +Checking for C library hopefullynolib2... no """ % {'lib' : lib} expected_build_str = """\ diff --git a/test/DVIPDF/DVIPDF.py b/test/DVIPDF/DVIPDF.py index a140c3ab..8db57850 100644 --- a/test/DVIPDF/DVIPDF.py +++ b/test/DVIPDF/DVIPDF.py @@ -129,13 +129,6 @@ foo.PDF(target = 'xxx.pdf', source = 'xxx.tex') tex = r""" This is the %s TeX file. \end -""" - - latex = r""" -\documentclass{letter} -\begin{document} -This is the %s LaTeX file. -\end{document} """ test.write('foo.tex', tex % 'foo.tex') diff --git a/test/DVIPDF/makeindex.py b/test/DVIPDF/makeindex.py new file mode 100644 index 00000000..cfa235fe --- /dev/null +++ b/test/DVIPDF/makeindex.py @@ -0,0 +1,71 @@ +#!/usr/bin/env python +# +# __COPYRIGHT__ +# +# Permission is hereby granted, free of charge, to any person obtaining +# a copy of this software and associated documentation files (the +# "Software"), to deal in the Software without restriction, including +# without limitation the rights to use, copy, modify, merge, publish, +# distribute, sublicense, and/or sell copies of the Software, and to +# permit persons to whom the Software is furnished to do so, subject to +# the following conditions: +# +# The above copyright notice and this permission notice shall be included +# in all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY +# KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE +# WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE +# LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION +# WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. +# + +__revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" + +import TestSCons + +test = TestSCons.TestSCons() + + + +dvipdf = test.where_is('dvipdf') +tex = test.where_is('tex') + +if not dvipdf or not tex: + test.skip_test('Could not find dvipdf or text; skipping test(s).\n') + + + +test.write('SConstruct', """ +import os +env = Environment(ENV = { 'PATH' : os.environ['PATH'] }) +dvipdf = env.Dictionary('DVIPDF') +env.PDF(target = 'foo.pdf', + source = env.DVI(target = 'foo.dvi', source = 'foo.tex')) +""") + +test.write('foo.tex', r""" +\documentclass{article} +\usepackage{makeidx} +\makeindex + +\begin{document} +\section{Test 1} +I would like to \index{index} this. + +\section{test 2} +I'll index \index{this} as well. + +\printindex +\end{document} +""") + +test.run(arguments = 'foo.pdf', stderr = None) + +test.must_exist(test.workpath('foo.pdf')) + + + +test.pass_test() diff --git a/test/Install/InstallAs.py b/test/Install/InstallAs.py index c4e88f68..4fe4bd0b 100644 --- a/test/Install/InstallAs.py +++ b/test/Install/InstallAs.py @@ -56,7 +56,19 @@ test.write('file1.in', "file1.in\n") test.write('file2.in', "file2.in\n") test.write(['subdir', 'file3.in'], "subdir/file3.in\n") -test.run(arguments = '.') +install_file1_out = os.path.join('install', 'file1.out') +install_file2_out = os.path.join('install', 'file2.out') +install_file3_out = os.path.join('install', 'file3.out') + +subdir_file3_in = os.path.join('subdir', 'file3.in') + +expect = test.wrap_stdout("""\ +Install file: "file1.in" as "%(install_file1_out)s" +Install file: "file2.in" as "%(install_file2_out)s" +Install file: "%(subdir_file3_in)s" as "%(install_file3_out)s" +""" % locals()) + +test.run(arguments = '.', stdout=expect) test.fail_test(test.read(install_file1_out) != "file1.in\n") test.fail_test(test.read(install_file2_out) != "file2.in\n") diff --git a/test/Options.py b/test/Options/Options.py similarity index 100% rename from test/Options.py rename to test/Options/Options.py diff --git a/test/ParseConfig.py b/test/ParseConfig.py index 1ab27c4f..55678fea 100644 --- a/test/ParseConfig.py +++ b/test/ParseConfig.py @@ -36,12 +36,13 @@ test = TestSCons.TestSCons() test_config1 = test.workpath('test-config1') test_config2 = test.workpath('test-config2') +test_config3 = test.workpath('test-config3') # 'abc' is supposed to be a static lib; it is included in LIBS as a # File node. # It used to be returned as the 'static_libs' output of ParseConfig. test.write(test_config1, """\ -print "-I/usr/include/fum -Ibar -X" +print "-I/usr/include/fum -Ibar -X -arch i386" print "-L/usr/fax -Lfoo -lxxx abc" """) @@ -49,6 +50,11 @@ test.write(test_config2, """\ print "-L foo -L lib_dir" """) +# This is like what wxWidgets does on OSX w/ Universal Binaries +test.write(test_config3, """\ +print "-L foo -L lib_dir -isysroot /tmp -arch ppc -arch i386" +""") + test.write('SConstruct', """ env = Environment(CPPPATH = [], LIBPATH = [], LIBS = [], CCFLAGS = '') env.ParseConfig([r"%(python)s", r"%(test_config1)s", "--libs --cflags"]) @@ -70,15 +76,34 @@ print map(lambda x: str(x), env['LIBS']) print env['CCFLAGS'] """ % locals()) +test.write('SConstruct3', """ +env = Environment(CPPPATH = [], LIBPATH = [], LIBS = [], CCFLAGS = '', + PYTHON = '%(python)s') +env.ParseConfig(r"$PYTHON %(test_config3)s --libs --cflags") +print env['CPPPATH'] +print env['LIBPATH'] +print map(lambda x: str(x), env['LIBS']) +print env['CCFLAGS'] +""" % locals()) + good_stdout = test.wrap_stdout(read_str = """\ ['/usr/include/fum', 'bar'] ['/usr/fax', 'foo', 'lib_dir'] ['xxx', 'abc'] -['-X'] +['-X', ('-arch', 'i386')] +""", build_str = "scons: `.' is up to date.\n") + +stdout3 = test.wrap_stdout(read_str = """\ +[] +['foo', 'lib_dir'] +[] +[('-isysroot', '/tmp'), ('-arch', 'ppc'), ('-arch', 'i386')] """, build_str = "scons: `.' is up to date.\n") test.run(arguments = ".", stdout = good_stdout) test.run(arguments = "-f SConstruct2 .", stdout = good_stdout) +test.run(arguments = "-f SConstruct3 .", stdout = stdout3) + test.pass_test() diff --git a/test/README b/test/README index cc27ffe1..7c88b39a 100644 --- a/test/README +++ b/test/README @@ -3,11 +3,10 @@ __COPYRIGHT__ This directory contains our end-to-end SCons tests. They are all meant to be run essentially standalone, with the exception -of the TestSCons.py module (and the TestCmd.py module it imports) that -is imported by each test. These modules are in the etc/ subdirectory, -and PYTHONPATH needs to be set up correctly so that the test script can -find them, and so that the SCons script itself can find the build engine -modules. +of the TestSCons.py module and the other modules imported by the various +tests. These modules are in the QMTest/ subdirectory, and PYTHONPATH +needs to be set up correctly so that the test scripts can find them, +and so that the SCons script itself can find the build engine modules. There is a wrapper script, runtest.py, that takes care of this for you, so the canonical invocation of a test is: @@ -17,9 +16,7 @@ so the canonical invocation of a test is: There is also a "runtest.py -a" option that will search the tree for all tests, and execute them. -Many of these tests have code for features that are not yet supported, -but which will be part of SCons IN THE CURRENT PLANNED RELEASE. These -are either: +Some of these tests have code for features that are not yet supported. commented out with a "#XXX" at the beginning of the line; @@ -27,13 +24,6 @@ are either: early call to test.pass_test(), which has a "#XXX" comment at the end of the line -The upshot is that you should be able to: - - egrep -l '#XXX' test/*.py - -and see a list of the tests we still have to make work in order to get -the next release out the door. - If you're trying to implement one of these features, DO NOT BLINDLY ASSUME THAT THE NEW CODE IN THE TEST IS CORRECT. It may have problems that have gone undiscovered due to the fact that the code testing the @@ -50,17 +40,9 @@ semblance of uniformity, here are the naming conventions for tests: keep this description reasonably short - Feature-01.py additional tests of specified - Feature-02.py feature - Feature-03.py - Feature-x.py test of specified feature using option x - Feature-x-01.py additional tests of specified - Feature-x-02.py feature using option x - Feature-x-03.py - -- Command line option tests take the form: option-x.py lower-case single-letter option diff --git a/test/TEX/bibliography.py b/test/TEX/bibliography.py new file mode 100644 index 00000000..6949a319 --- /dev/null +++ b/test/TEX/bibliography.py @@ -0,0 +1,210 @@ +#!/usr/bin/env python +# +# __COPYRIGHT__ +# +# Permission is hereby granted, free of charge, to any person obtaining +# a copy of this software and associated documentation files (the +# "Software"), to deal in the Software without restriction, including +# without limitation the rights to use, copy, modify, merge, publish, +# distribute, sublicense, and/or sell copies of the Software, and to +# permit persons to whom the Software is furnished to do so, subject to +# the following conditions: +# +# The above copyright notice and this permission notice shall be included +# in all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY +# KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE +# WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE +# LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION +# WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. +# + +__revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" + +""" +Validate that use of \bibliography in TeX source files causes SCons to +be aware of the necessary created bibliography files. + +Test configuration contributed by Christopher Drexler. +""" + +import string + +import TestSCons + +test = TestSCons.TestSCons() + +dvips = test.where_is('dvips') +bibtex = test.where_is('bibtex') + +if not dvips or not bibtex: + test.skip_test("Could not find dvips or bibtex; skipping test(s).\n") + +test.write('SConstruct', """\ +env = Environment(tools = ['tex', 'latex', 'dvips']) +env.PostScript('simple', 'simple.tex') +""") + +test.write('simple.tex', r""" +\documentclass[12pt]{book} + +\begin{document} + +\chapter{Chapter 1}\label{c:c1} + +Test test.\cite{Aloimonos88:AV} + +\section{Section 1}\label{s:c1s1} +Test test. + +\section{Section 2}\label{s:c1s2} +Test test. + +\chapter{Chapter 2}\label{c:c2} + +Test test.\cite{Ayache86:HAN} + +\section{Section 1}\label{s:c2s1} +Test test. + +\section{Section 2}\label{s:c2s2} +Test test. + +\bibliographystyle{plain} +\bibliography{simple} +\end{document} +""") + +test.write('simple.bib', r""" +@Article{Aloimonos88:AV, + Author = {Aloimonos,~J and Weiss,~I. and Bandyopadyay,~A.}, + Title = {Active Vision}, + Journal = ijcv, + Volume = 2, + Number = 3, + Pages = {333--356}, + year = 1988, +} + +@Article{Ayache86:HAN, + Author = {Ayache, N. and Faugeras, O. D.}, + Title = {HYPER: A new approach for the recognition and + positioning of 2D objects}, + Journal = pami, + Volume = 8, + Number = 1, + Pages = {44-54}, + year = 1986, +} +""") + +test.run(arguments = '.', stderr=None) + +test.must_exist(test.workpath('simple.aux')) +test.must_exist(test.workpath('simple.bbl')) +test.must_exist(test.workpath('simple.blg')) + +test.run(arguments = '-c .') + +x = "Could not remove 'simple.aux': No such file or directory" +test.fail_test(string.find(test.stdout(), x) != -1) + +test.must_not_exist(test.workpath('simple.aux')) +test.must_not_exist(test.workpath('simple.bbl')) +test.must_not_exist(test.workpath('simple.blg')) + +test.pass_test() + + + +# FUTURE: + +test.write('SConstruct', """\ +env = Environment(tools = ['tex', 'latex', 'dvips']) +env.PostScript('d00', 'd00.tex') +""") + +test.write('d00.tex', r""" +\documentclass[12pt]{book} + +\begin{document} +\include{d-toc} + +\include{d01} +\include{d02} +\include{d03} + +\include{d-lit} +\end{document} +""") + +test.write('d01.tex', r""" +\chapter{Chapter 1}\label{c:c1} + +Test test.\cite{Aloimonos88:AV} + +\section{Section 1}\label{s:c1s1} +Test test. + +\section{Section 2}\label{s:c1s2} +Test test. + +\section{Section 3}\label{s:c1s3} +Test test. + +\section{Section 4}\label{s:c1s4} +Test test. +""") + +test.write('d02.tex', r""" +\chapter{Chapter 2}\label{c:c2} + +Test test.\cite{Ayache86:HAN} + +\section{Section 1}\label{s:c2s1} +Test test. + +\section{Section 2}\label{s:c2s2} +Test test. + +\section{Section 3}\label{s:c2s3} +Test test. + +\section{Section 4}\label{s:c2s4} +Test test. +""") + +test.write('d03.tex', r""" +\chapter{Chapter 3}\label{c:c3} + +Test test. + +\section{Section 1}\label{s:c3s1} +Test test. + +\section{Section 2}\label{s:c3s2} +Test test. + +\section{Section 3}\label{s:c3s3} +Test test. + +\section{Section 4}\label{s:c3s4} +Test test. +""") + +test.write('d-lit.tex', r""" +\bibliographystyle{plain} +\bibliography{d00} +""") + +test.write('d-toc.tex', r""" +\tableofcontents +\clearpage +\listoffigures +\clearpage +\listoftables +\cleardoublepage +""") diff --git a/test/TEX/makeindex.py b/test/TEX/makeindex.py new file mode 100644 index 00000000..8f992843 --- /dev/null +++ b/test/TEX/makeindex.py @@ -0,0 +1,104 @@ +#!/usr/bin/env python +# +# __COPYRIGHT__ +# +# Permission is hereby granted, free of charge, to any person obtaining +# a copy of this software and associated documentation files (the +# "Software"), to deal in the Software without restriction, including +# without limitation the rights to use, copy, modify, merge, publish, +# distribute, sublicense, and/or sell copies of the Software, and to +# permit persons to whom the Software is furnished to do so, subject to +# the following conditions: +# +# The above copyright notice and this permission notice shall be included +# in all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY +# KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE +# WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE +# LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION +# WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. +# + +__revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" + +""" +Validate that use of \makeindex in TeX source files causes SCons to be +aware of the necessary created index files. + +Test configuration courtesy Joel B. Mohler. +""" + +import string + +import TestSCons + +test = TestSCons.TestSCons() + +pdflatex = test.where_is('pdflatex') +makeindex = test.where_is('makeindex') + +if not pdflatex or not makeindex: + test.skip_test("Could not find pdflatex or makeindex; skipping test(s).\n") + +test.write('SConstruct', """\ +PDF( "no_index.tex" ) +PDF( "simple.tex" ) +""") + +test.write('no_index.tex', r""" +\documentclass{article} + +\begin{document} +Nothing to see here, move along! +\end{document} +""") + +test.write('simple.tex', r""" +\documentclass{article} +\usepackage{makeidx} +\makeindex + +\begin{document} +\section{Test 1} +I would like to \index{index} this. + +\section{test 2} +I'll index \index{this} as well. + +\printindex +\end{document} +""") + +# Bleah, makeindex seems to print a bunch of diagnostic stuff on stderr, +# so we have to ignore it. +test.run(arguments = '.', stderr=None) + +test.must_exist(test.workpath('simple.aux')) +test.must_exist(test.workpath('simple.idx')) +test.must_exist(test.workpath('simple.ilg')) +test.must_exist(test.workpath('simple.ind')) + +test.must_exist(test.workpath('no_index.aux')) + +test.must_not_exist(test.workpath('no_index.idx')) +test.must_not_exist(test.workpath('no_index.ilg')) +test.must_not_exist(test.workpath('no_index.ind')) + +test.run(arguments = '-c .') + +x = "Could not remove 'no_index.aux': No such file or directory" +test.fail_test(string.find(test.stdout(), x) != -1) +x = "Could not remove 'simple.aux': No such file or directory" +test.fail_test(string.find(test.stdout(), x) != -1) + +test.must_not_exist(test.workpath('simple.aux')) +test.must_not_exist(test.workpath('simple.idx')) +test.must_not_exist(test.workpath('simple.ilg')) +test.must_not_exist(test.workpath('simple.ind')) + +test.must_not_exist(test.workpath('no_index.aux')) + +test.pass_test() diff --git a/test/_CPPINCFLAGS.py b/test/_CPPINCFLAGS.py new file mode 100644 index 00000000..b2f78262 --- /dev/null +++ b/test/_CPPINCFLAGS.py @@ -0,0 +1,55 @@ +#!/usr/bin/env python +# +# __COPYRIGHT__ +# +# Permission is hereby granted, free of charge, to any person obtaining +# a copy of this software and associated documentation files (the +# "Software"), to deal in the Software without restriction, including +# without limitation the rights to use, copy, modify, merge, publish, +# distribute, sublicense, and/or sell copies of the Software, and to +# permit persons to whom the Software is furnished to do so, subject to +# the following conditions: +# +# The above copyright notice and this permission notice shall be included +# in all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY +# KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE +# WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE +# LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION +# WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. +# + +__revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" + +""" +Test that we can expand $_CPPINCFLAGS correctly regardless of whether +the target is an entry, a directory, or a file. (Internally, this tests +that RDirs() is available to be called for each Node.FS type.) +""" + +import TestSCons + +test = TestSCons.TestSCons() + +test.write('SConstruct', """\ +env=Environment(CPPPATH=['tmp'], INCPREFIX='-I') +d=Entry('foo.d') +e=Entry('foo.e') +f=File('foo.f') +print env.subst('$_CPPINCFLAGS', target=e, source=f) +print env.subst('$_CPPINCFLAGS', target=d, source=f) +print env.subst('$_CPPINCFLAGS', target=f, source=d) +""") + +expect = """\ +-Itmp +-Itmp +-Itmp +""" + +test.run(arguments = '-Q -q', stdout = expect) + +test.pass_test() diff --git a/test/dependency-cycle.py b/test/dependency-cycle.py index 1486c763..8d1275f2 100644 --- a/test/dependency-cycle.py +++ b/test/dependency-cycle.py @@ -48,7 +48,7 @@ f1(void) """) test.run(arguments = ".", stderr=r""" -scons: \*\*\* Dependency cycle: .*foo1.* -> .*foo3.* -> .*foo2.* -> .*foo1.* -> \. +scons: \*\*\* Dependency cycle: .*foo1.* -> .*foo3.* -> .*foo2.* -> .*foo1.* .* """, status=2) diff --git a/test/option/taskmastertrace.py b/test/option/taskmastertrace.py index b1b8717d..30facedd 100644 --- a/test/option/taskmastertrace.py +++ b/test/option/taskmastertrace.py @@ -47,15 +47,16 @@ Taskmaster: '.': children: ['file.mid', 'file.out'] Taskmaster: 'file.mid': children: ['file.in'] - evaluating + evaluating file.mid Copy("file.mid", "file.in") Taskmaster: 'file.out': children: ['file.mid'] - evaluating + evaluating file.out Copy("file.out", "file.mid") Taskmaster: '.': children: ['SConstruct', 'file.in', 'file.mid', 'file.out'] - evaluating + evaluating . +Taskmaster: '.': already handled (executed) """) test.run(arguments='--taskmastertrace=- .', stdout=expect_stdout) @@ -80,13 +81,14 @@ Taskmaster: '.': children: ['file.mid', 'file.out'] Taskmaster: 'file.mid': children: ['file.in'] - evaluating + evaluating file.mid Taskmaster: 'file.out': children: ['file.mid'] - evaluating + evaluating file.out Taskmaster: '.': children: ['SConstruct', 'file.in', 'file.mid', 'file.out'] - evaluating + evaluating . +Taskmaster: '.': already handled (executed) """ test.must_match('trace.out', expect_trace) diff --git a/test/runtest/baseline/combined.py b/test/runtest/baseline/combined.py new file mode 100644 index 00000000..6382cb18 --- /dev/null +++ b/test/runtest/baseline/combined.py @@ -0,0 +1,81 @@ + +#!/usr/bin/env python +# +# __COPYRIGHT__ +# +# Permission is hereby granted, free of charge, to any person obtaining +# a copy of this software and associated documentation files (the +# "Software"), to deal in the Software without restriction, including +# without limitation the rights to use, copy, modify, merge, publish, +# distribute, sublicense, and/or sell copies of the Software, and to +# permit persons to whom the Software is furnished to do so, subject to +# the following conditions: +# +# The above copyright notice and this permission notice shall be included +# in all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY +# KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE +# WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE +# LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION +# WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. +# + +__revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" + +""" +Test a combination of a passing test, failing test, and no-result +test with no argument on the command line. +""" + +import TestRuntest + +test = TestRuntest.TestRuntest() + +test.subdir('test') + +test.write_failing_test(['test', 'fail.py']) + +test.write_no_result_test(['test', 'no_result.py']) + +test.write_passing_test(['test', 'pass.py']) + +# NOTE: The "test/fail.py : FAIL" and "test/pass.py : PASS" lines both +# have spaces at the end. + +expect = r"""qmtest.py run --output baseline.qmr --format none --result-stream=scons_tdb.AegisBaselineStream test +--- TEST RESULTS ------------------------------------------------------------- + + test/fail.py : FAIL + + FAILING TEST STDOUT + + FAILING TEST STDERR + + test/no_result.py : NO_RESULT + + NO RESULT TEST STDOUT + + NO RESULT TEST STDERR + + test/pass.py : PASS + +--- TESTS WITH UNEXPECTED OUTCOMES ------------------------------------------- + + test/no_result.py : NO_RESULT + + test/pass.py : PASS + + +--- STATISTICS --------------------------------------------------------------- + + 1 ( 33%) tests as expected + 1 ( 33%) tests unexpected PASS + 1 ( 33%) tests unexpected NO_RESULT +""" + +test.run(arguments = '--qmtest -b . test', stdout = expect) + +test.pass_test() diff --git a/test/runtest/baseline/fail.py b/test/runtest/baseline/fail.py new file mode 100644 index 00000000..b650a8c3 --- /dev/null +++ b/test/runtest/baseline/fail.py @@ -0,0 +1,62 @@ +#!/usr/bin/env python +# +# __COPYRIGHT__ +# +# Permission is hereby granted, free of charge, to any person obtaining +# a copy of this software and associated documentation files (the +# "Software"), to deal in the Software without restriction, including +# without limitation the rights to use, copy, modify, merge, publish, +# distribute, sublicense, and/or sell copies of the Software, and to +# permit persons to whom the Software is furnished to do so, subject to +# the following conditions: +# +# The above copyright notice and this permission notice shall be included +# in all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY +# KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE +# WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE +# LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION +# WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. +# + +__revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" + +""" +Test how we handle a failing test specified on the command line. +""" + +import TestRuntest + +test = TestRuntest.TestRuntest() + +test.subdir('test') + +test.write_failing_test(['test', 'fail.py']) + +# NOTE: The "test/fail.py : FAIL" line has spaces at the end. + +expect = r"""qmtest.py run --output baseline.qmr --format none --result-stream=scons_tdb.AegisBaselineStream test/fail.py +--- TEST RESULTS ------------------------------------------------------------- + + test/fail.py : FAIL + + FAILING TEST STDOUT + + FAILING TEST STDERR + +--- TESTS WITH UNEXPECTED OUTCOMES ------------------------------------------- + + None. + + +--- STATISTICS --------------------------------------------------------------- + + 1 (100%) tests as expected +""" + +test.run(arguments = '--qmtest -b . test/fail.py', stdout = expect) + +test.pass_test() diff --git a/test/runtest/baseline/no_result.py b/test/runtest/baseline/no_result.py new file mode 100644 index 00000000..dc2586d9 --- /dev/null +++ b/test/runtest/baseline/no_result.py @@ -0,0 +1,60 @@ +#!/usr/bin/env python +# +# __COPYRIGHT__ +# +# Permission is hereby granted, free of charge, to any person obtaining +# a copy of this software and associated documentation files (the +# "Software"), to deal in the Software without restriction, including +# without limitation the rights to use, copy, modify, merge, publish, +# distribute, sublicense, and/or sell copies of the Software, and to +# permit persons to whom the Software is furnished to do so, subject to +# the following conditions: +# +# The above copyright notice and this permission notice shall be included +# in all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY +# KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE +# WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE +# LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION +# WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. +# + +__revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" + +""" +Test how we handle a no-results test specified on the command line. +""" + +import TestRuntest + +test = TestRuntest.TestRuntest() + +test.subdir('test') + +test.write_no_result_test(['test', 'no_result.py']) + +expect = r"""qmtest.py run --output baseline.qmr --format none --result-stream=scons_tdb.AegisBaselineStream test/no_result.py +--- TEST RESULTS ------------------------------------------------------------- + + test/no_result.py : NO_RESULT + + NO RESULT TEST STDOUT + + NO RESULT TEST STDERR + +--- TESTS WITH UNEXPECTED OUTCOMES ------------------------------------------- + + test/no_result.py : NO_RESULT + + +--- STATISTICS --------------------------------------------------------------- + + 1 (100%) tests unexpected NO_RESULT +""" + +test.run(arguments = '--qmtest -b . test/no_result.py', stdout = expect) + +test.pass_test() diff --git a/test/runtest/baseline/pass.py b/test/runtest/baseline/pass.py new file mode 100644 index 00000000..b32ecbe4 --- /dev/null +++ b/test/runtest/baseline/pass.py @@ -0,0 +1,58 @@ +#!/usr/bin/env python +# +# __COPYRIGHT__ +# +# Permission is hereby granted, free of charge, to any person obtaining +# a copy of this software and associated documentation files (the +# "Software"), to deal in the Software without restriction, including +# without limitation the rights to use, copy, modify, merge, publish, +# distribute, sublicense, and/or sell copies of the Software, and to +# permit persons to whom the Software is furnished to do so, subject to +# the following conditions: +# +# The above copyright notice and this permission notice shall be included +# in all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY +# KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE +# WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE +# LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION +# WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. +# + +__revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" + +""" +Test how we handle a passing test specified on the command line. +""" + +import TestRuntest + +test = TestRuntest.TestRuntest() + +test.subdir('test') + +test.write_passing_test(['test', 'pass.py']) + +# NOTE: The "test/pass.py : PASS" line has spaces at the end. + +expect = r"""qmtest.py run --output baseline.qmr --format none --result-stream=scons_tdb.AegisBaselineStream test/pass.py +--- TEST RESULTS ------------------------------------------------------------- + + test/pass.py : PASS + +--- TESTS WITH UNEXPECTED OUTCOMES ------------------------------------------- + + test/pass.py : PASS + + +--- STATISTICS --------------------------------------------------------------- + + 1 (100%) tests unexpected PASS +""" + +test.run(arguments = '--qmtest -b . test/pass.py', stdout = expect) + +test.pass_test() diff --git a/test/runtest/print_time.py b/test/runtest/print_time.py new file mode 100644 index 00000000..1d86baad --- /dev/null +++ b/test/runtest/print_time.py @@ -0,0 +1,89 @@ +#!/usr/bin/env python +# +# __COPYRIGHT__ +# +# Permission is hereby granted, free of charge, to any person obtaining +# a copy of this software and associated documentation files (the +# "Software"), to deal in the Software without restriction, including +# without limitation the rights to use, copy, modify, merge, publish, +# distribute, sublicense, and/or sell copies of the Software, and to +# permit persons to whom the Software is furnished to do so, subject to +# the following conditions: +# +# The above copyright notice and this permission notice shall be included +# in all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY +# KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE +# WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE +# LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION +# WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. +# + +__revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" + +""" +Test a combination of a passing test, failing test, and no-result +test with no argument on the command line. +""" + +import TestCmd +import TestRuntest + +test = TestRuntest.TestRuntest(match = TestCmd.match_re) + +test.subdir('test') + +test.write_failing_test(['test', 'fail.py']) + +test.write_no_result_test(['test', 'no_result.py']) + +test.write_passing_test(['test', 'pass.py']) + +# NOTE: The "test/fail.py : FAIL" and "test/pass.py : PASS" lines both +# have spaces at the end. + +expect = r"""qmtest.py run --output results.qmr --format none --result-stream=scons_tdb.AegisChangeStream --context print_time=1 test +--- TEST RESULTS ------------------------------------------------------------- + + test/fail.py : FAIL + + FAILING TEST STDOUT + + FAILING TEST STDERR + + Total execution time: \d+\.\d+ seconds + + test/no_result.py : NO_RESULT + + NO RESULT TEST STDOUT + + NO RESULT TEST STDERR + + Total execution time: \d+\.\d+ seconds + + test/pass.py : PASS + + Total execution time: \d+\.\d+ seconds + +--- TESTS THAT DID NOT PASS -------------------------------------------------- + + test/fail.py : FAIL + + test/no_result.py : NO_RESULT + + +--- STATISTICS --------------------------------------------------------------- + + 3 tests total + + 1 \( 33%\) tests PASS + 1 \( 33%\) tests FAIL + 1 \( 33%\) tests NO_RESULT +""" + +test.run(arguments = '--qmtest -t test', stdout = expect) + +test.pass_test() diff --git a/test/runtest/python.py b/test/runtest/python.py new file mode 100644 index 00000000..76eec7dc --- /dev/null +++ b/test/runtest/python.py @@ -0,0 +1,80 @@ +#!/usr/bin/env python +# +# __COPYRIGHT__ +# +# Permission is hereby granted, free of charge, to any person obtaining +# a copy of this software and associated documentation files (the +# "Software"), to deal in the Software without restriction, including +# without limitation the rights to use, copy, modify, merge, publish, +# distribute, sublicense, and/or sell copies of the Software, and to +# permit persons to whom the Software is furnished to do so, subject to +# the following conditions: +# +# The above copyright notice and this permission notice shall be included +# in all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY +# KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE +# WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE +# LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION +# WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. +# + +__revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" + +""" +Test that the -P option lets us specify a Python version to use. +""" + +import TestRuntest + +test = TestRuntest.TestRuntest() + +mypython_py = test.workpath('mypython.py') +mypython_out = test.workpath('mypython.out') + +test.subdir('test') + +test.write_passing_test(['test', 'pass.py']) + +test.write(mypython_py, """\ +#!/usr/bin/env python +import os +import sys +import string +open(r'%s', 'a').write(string.join(sys.argv) + '\\n') +os.system(string.join([sys.executable] + sys.argv[1:])) +""" % mypython_out) + +test.chmod(mypython_py, 0755) + +# NOTE: The "test/fail.py : FAIL" and "test/pass.py : PASS" lines both +# have spaces at the end. + +expect = r"""qmtest.py run --output results.qmr --format none --result-stream=scons_tdb.AegisChangeStream --context python=%(mypython_py)s test +--- TEST RESULTS ------------------------------------------------------------- + + test/pass.py : PASS + +--- TESTS THAT DID NOT PASS -------------------------------------------------- + + None. + + +--- STATISTICS --------------------------------------------------------------- + + 1 tests total + + 1 (100%%) tests PASS +""" % locals() + +test.run(arguments = '--qmtest -P %s test' % mypython_py, + stdout = expect) + +test.must_match(mypython_out, """\ +%s ./test/pass.py +""" % mypython_py) + +test.pass_test() diff --git a/test/runtest/simple/combined.py b/test/runtest/simple/combined.py new file mode 100644 index 00000000..1640d94d --- /dev/null +++ b/test/runtest/simple/combined.py @@ -0,0 +1,83 @@ + +#!/usr/bin/env python +# +# __COPYRIGHT__ +# +# Permission is hereby granted, free of charge, to any person obtaining +# a copy of this software and associated documentation files (the +# "Software"), to deal in the Software without restriction, including +# without limitation the rights to use, copy, modify, merge, publish, +# distribute, sublicense, and/or sell copies of the Software, and to +# permit persons to whom the Software is furnished to do so, subject to +# the following conditions: +# +# The above copyright notice and this permission notice shall be included +# in all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY +# KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE +# WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE +# LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION +# WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. +# + +__revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" + +""" +Test a combination of a passing test, failing test, and no-result +test with no argument on the command line. +""" + +import TestRuntest + +test = TestRuntest.TestRuntest() + +test.subdir('test') + +test.write_failing_test(['test', 'fail.py']) + +test.write_no_result_test(['test', 'no_result.py']) + +test.write_passing_test(['test', 'pass.py']) + +# NOTE: The "test/fail.py : FAIL" and "test/pass.py : PASS" lines both +# have spaces at the end. + +expect = r"""qmtest.py run --output results.qmr --format none --result-stream=scons_tdb.AegisChangeStream test +--- TEST RESULTS ------------------------------------------------------------- + + test/fail.py : FAIL + + FAILING TEST STDOUT + + FAILING TEST STDERR + + test/no_result.py : NO_RESULT + + NO RESULT TEST STDOUT + + NO RESULT TEST STDERR + + test/pass.py : PASS + +--- TESTS THAT DID NOT PASS -------------------------------------------------- + + test/fail.py : FAIL + + test/no_result.py : NO_RESULT + + +--- STATISTICS --------------------------------------------------------------- + + 3 tests total + + 1 ( 33%) tests PASS + 1 ( 33%) tests FAIL + 1 ( 33%) tests NO_RESULT +""" + +test.run(arguments = '--qmtest test', stdout = expect) + +test.pass_test() diff --git a/test/runtest/simple/fail.py b/test/runtest/simple/fail.py new file mode 100644 index 00000000..ba2cc97a --- /dev/null +++ b/test/runtest/simple/fail.py @@ -0,0 +1,64 @@ +#!/usr/bin/env python +# +# __COPYRIGHT__ +# +# Permission is hereby granted, free of charge, to any person obtaining +# a copy of this software and associated documentation files (the +# "Software"), to deal in the Software without restriction, including +# without limitation the rights to use, copy, modify, merge, publish, +# distribute, sublicense, and/or sell copies of the Software, and to +# permit persons to whom the Software is furnished to do so, subject to +# the following conditions: +# +# The above copyright notice and this permission notice shall be included +# in all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY +# KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE +# WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE +# LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION +# WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. +# + +__revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" + +""" +Test how we handle a failing test specified on the command line. +""" + +import TestRuntest + +test = TestRuntest.TestRuntest() + +test.subdir('test') + +test.write_failing_test(['test', 'fail.py']) + +# NOTE: The "test/fail.py : FAIL" line has spaces at the end. + +expect = r"""qmtest.py run --output results.qmr --format none --result-stream=scons_tdb.AegisChangeStream test/fail.py +--- TEST RESULTS ------------------------------------------------------------- + + test/fail.py : FAIL + + FAILING TEST STDOUT + + FAILING TEST STDERR + +--- TESTS THAT DID NOT PASS -------------------------------------------------- + + test/fail.py : FAIL + + +--- STATISTICS --------------------------------------------------------------- + + 1 tests total + + 1 (100%) tests FAIL +""" + +test.run(arguments = '--qmtest test/fail.py', stdout = expect) + +test.pass_test() diff --git a/test/runtest/simple/no_result.py b/test/runtest/simple/no_result.py new file mode 100644 index 00000000..7c9687e4 --- /dev/null +++ b/test/runtest/simple/no_result.py @@ -0,0 +1,62 @@ +#!/usr/bin/env python +# +# __COPYRIGHT__ +# +# Permission is hereby granted, free of charge, to any person obtaining +# a copy of this software and associated documentation files (the +# "Software"), to deal in the Software without restriction, including +# without limitation the rights to use, copy, modify, merge, publish, +# distribute, sublicense, and/or sell copies of the Software, and to +# permit persons to whom the Software is furnished to do so, subject to +# the following conditions: +# +# The above copyright notice and this permission notice shall be included +# in all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY +# KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE +# WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE +# LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION +# WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. +# + +__revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" + +""" +Test how we handle a no-results test specified on the command line. +""" + +import TestRuntest + +test = TestRuntest.TestRuntest() + +test.subdir('test') + +test.write_no_result_test(['test', 'no_result.py']) + +expect = r"""qmtest.py run --output results.qmr --format none --result-stream=scons_tdb.AegisChangeStream test/no_result.py +--- TEST RESULTS ------------------------------------------------------------- + + test/no_result.py : NO_RESULT + + NO RESULT TEST STDOUT + + NO RESULT TEST STDERR + +--- TESTS THAT DID NOT PASS -------------------------------------------------- + + test/no_result.py : NO_RESULT + + +--- STATISTICS --------------------------------------------------------------- + + 1 tests total + + 1 (100%) tests NO_RESULT +""" + +test.run(arguments = '--qmtest test/no_result.py', stdout = expect) + +test.pass_test() diff --git a/test/runtest/simple/pass.py b/test/runtest/simple/pass.py new file mode 100644 index 00000000..8dfc9964 --- /dev/null +++ b/test/runtest/simple/pass.py @@ -0,0 +1,60 @@ +#!/usr/bin/env python +# +# __COPYRIGHT__ +# +# Permission is hereby granted, free of charge, to any person obtaining +# a copy of this software and associated documentation files (the +# "Software"), to deal in the Software without restriction, including +# without limitation the rights to use, copy, modify, merge, publish, +# distribute, sublicense, and/or sell copies of the Software, and to +# permit persons to whom the Software is furnished to do so, subject to +# the following conditions: +# +# The above copyright notice and this permission notice shall be included +# in all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY +# KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE +# WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE +# LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION +# WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. +# + +__revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" + +""" +Test how we handle a passing test specified on the command line. +""" + +import TestRuntest + +test = TestRuntest.TestRuntest() + +test.subdir('test') + +test.write_passing_test(['test', 'pass.py']) + +# NOTE: The "test/pass.py : PASS" line has spaces at the end. + +expect = r"""qmtest.py run --output results.qmr --format none --result-stream=scons_tdb.AegisChangeStream test/pass.py +--- TEST RESULTS ------------------------------------------------------------- + + test/pass.py : PASS + +--- TESTS THAT DID NOT PASS -------------------------------------------------- + + None. + + +--- STATISTICS --------------------------------------------------------------- + + 1 tests total + + 1 (100%) tests PASS +""" + +test.run(arguments = '--qmtest test/pass.py', stdout = expect) + +test.pass_test() diff --git a/test/runtest/src.py b/test/runtest/src.py new file mode 100644 index 00000000..2f723b45 --- /dev/null +++ b/test/runtest/src.py @@ -0,0 +1,71 @@ +#!/usr/bin/env python +# +# __COPYRIGHT__ +# +# Permission is hereby granted, free of charge, to any person obtaining +# a copy of this software and associated documentation files (the +# "Software"), to deal in the Software without restriction, including +# without limitation the rights to use, copy, modify, merge, publish, +# distribute, sublicense, and/or sell copies of the Software, and to +# permit persons to whom the Software is furnished to do so, subject to +# the following conditions: +# +# The above copyright notice and this permission notice shall be included +# in all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY +# KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE +# WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE +# LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION +# WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. +# + +__revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" + +""" +Verify that we find tests under the src/ tree only if they end +with *Tests.py. +""" + +import TestRuntest + +test = TestRuntest.TestRuntest(verbose=1) + +test.subdir(['src'], + ['src', 'suite']) + +test.write_passing_test(['src', 'pass.py']) + +test.write_passing_test(['src', 'passTests.py']) + +test.write_passing_test(['src', 'suite', 'pass.py']) + +test.write_passing_test(['src', 'suite', 'passTests.py']) + +# NOTE: The "test/fail.py : FAIL" and "test/pass.py : PASS" lines both +# have spaces at the end. + +expect = r"""qmtest.py run --output results.qmr --format none --result-stream=scons_tdb.AegisChangeStream src +--- TEST RESULTS ------------------------------------------------------------- + + src/passTests.py : PASS + + src/suite/passTests.py : PASS + +--- TESTS THAT DID NOT PASS -------------------------------------------------- + + None. + + +--- STATISTICS --------------------------------------------------------------- + + 2 tests total + + 2 (100%) tests PASS +""" + +test.run(arguments = '--qmtest src', stdout = expect) + +test.pass_test() diff --git a/test/scan-once.py b/test/scan-once.py index c86eaa49..044e2603 100644 --- a/test/scan-once.py +++ b/test/scan-once.py @@ -275,7 +275,9 @@ for k in fromdict.keys(): # When $TARGET is None, so $TARGET.attributes would throw an # exception. f = fromdict[k] - if SCons.Util.is_String(f) and string.find(f, "TARGET") == -1: + if SCons.Util.is_String(f) and \ + string.find(f, "TARGET") == -1 and \ + string.find(f, "SOURCE") == -1: todict[k] = env.subst(f) todict["CFLAGS"] = fromdict["CPPFLAGS"] + " " + \ string.join(map(lambda x: "-I" + x, env["CPPPATH"])) + " " + \ diff --git a/test/sconsign/script.py b/test/sconsign/script.py index 86f62928..9e0caa50 100644 --- a/test/sconsign/script.py +++ b/test/sconsign/script.py @@ -31,15 +31,41 @@ import time import TestCmd import TestSCons +class MyTestSCons(TestSCons.TestSCons): + # subclass with a method for running the sconsign script + def __init__(self, *args, **kw): + try: + script_dir = os.environ['SCONS_SCRIPT_DIR'] + except KeyError: + pass + else: + os.chdir(script_dir) + self.script_dir = os.getcwd() + apply(TestSCons.TestSCons.__init__, (self,)+args, kw) + self.my_kw = { + 'interpreter' : TestSCons.python, + } + def script_path(self, script): + return os.path.join(self.script_dir, script) + def set_sconsign(self, sconsign): + self.my_kw['program'] = sconsign + def run_sconsign(self, *args, **kw): + kw.update(self.my_kw) + return apply(self.run, args, kw) + +test = MyTestSCons(match = TestCmd.match_re) + # Check for the sconsign script before we instantiate TestSCons(), # because that will change directory on us. -if os.path.exists('sconsign.py'): +if os.path.exists(test.script_path('sconsign.py')): sconsign = 'sconsign.py' -elif os.path.exists('sconsign'): +elif os.path.exists(test.script_path('sconsign')): sconsign = 'sconsign' else: print "Can find neither 'sconsign.py' nor 'sconsign' scripts." - test.no_result(1) + test.no_result() + +test.set_sconsign(sconsign) def sort_match(test, lines, expect): lines = string.split(lines, '\n') @@ -53,23 +79,6 @@ def re_sep(*args): -class MyTestSCons(TestSCons.TestSCons): - # subclass with a method for running the sconsign script - def __init__(self, *args, **kw): - apply(TestSCons.TestSCons.__init__, (self,)+args, kw) - self.my_kw = { - 'interpreter' : TestSCons.python, - 'program' : sconsign, - } - def run_sconsign(self, *args, **kw): - kw.update(self.my_kw) - return apply(self.run, args, kw) - -test = MyTestSCons(match = TestCmd.match_re) - - - - test.subdir('work1', ['work1', 'sub1'], ['work1', 'sub2'], 'work2', ['work2', 'sub1'], ['work2', 'sub2']) diff --git a/test/strfunction.py b/test/strfunction.py index ef8c483d..a0b23447 100644 --- a/test/strfunction.py +++ b/test/strfunction.py @@ -45,48 +45,69 @@ def strfunction(target, source, env): t = str(target[0]) s = str(source[0]) return "Building %%s from %%s" %% (t, s) + def func(target, source, env): t = str(target[0]) s = str(source[0]) open(t, 'w').write(open(s, 'r').read()) -funcaction = Action(func, strfunction=strfunction) -cmd = r"%s cat.py $SOURCE $TARGET" -cmdaction = Action(cmd, strfunction=strfunction) -list = [ r"%s cat.py $SOURCE .temp", r"%s cat.py .temp $TARGET" ] +func1action = Action(func, strfunction) +func2action = Action(func, strfunction=strfunction) + +cmd = r"%(python)s cat.py $SOURCE $TARGET" +cmd1action = Action(cmd, strfunction) +cmd2action = Action(cmd, strfunction=strfunction) + +list = [ r"%(python)s cat.py $SOURCE .temp", + r"%(python)s cat.py .temp $TARGET" ] listaction = Action(list, strfunction=strfunction) + lazy = '$LAZY' -lazyaction = Action(lazy, strfunction=strfunction) +lazy1action = Action(lazy, strfunction) +lazy2action = Action(lazy, strfunction=strfunction) + +targetaction = Action(func, '$TARGET') + dict = { '.cmd' : cmd, - '.cmdstr' : cmdaction, + '.cmdstr' : cmd2action, '.func' : func, - '.funcstr' : funcaction, + '.funcstr' : func2action, '.list' : list, '.liststr' : listaction, '.lazy' : lazy, - '.lazystr' : lazyaction, + '.lazystr' : lazy2action, } + env = Environment(BUILDERS = { 'Cmd' : Builder(action=cmd), - 'CmdStr' : Builder(action=cmdaction), + 'Cmd1Str' : Builder(action=cmd1action), + 'Cmd2Str' : Builder(action=cmd2action), 'Func' : Builder(action=func), - 'FuncStr' : Builder(action=funcaction), + 'Func1Str' : Builder(action=func1action), + 'Func2Str' : Builder(action=func2action), 'Lazy' : Builder(action=lazy), - 'LazyStr' : Builder(action=lazyaction), + 'Lazy1Str' : Builder(action=lazy1action), + 'Lazy2Str' : Builder(action=lazy2action), 'List' : Builder(action=list), 'ListStr' : Builder(action=listaction), + 'Target' : Builder(action=targetaction), 'Dict' : Builder(action=dict), }, - LAZY = r"%s cat.py $SOURCE $TARGET") + LAZY = r"%(python)s cat.py $SOURCE $TARGET") + env.Cmd('cmd.out', 'cmd.in') -env.CmdStr('cmdstr.out', 'cmdstr.in') +env.Cmd1Str('cmd1str.out', 'cmdstr.in') +env.Cmd2Str('cmd2str.out', 'cmdstr.in') env.Func('func.out', 'func.in') -env.FuncStr('funcstr.out', 'funcstr.in') +env.Func1Str('func1str.out', 'funcstr.in') +env.Func2Str('func2str.out', 'funcstr.in') env.Lazy('lazy.out', 'lazy.in') -env.LazyStr('lazystr.out', 'lazystr.in') +env.Lazy1Str('lazy1str.out', 'lazystr.in') +env.Lazy2Str('lazy2str.out', 'lazystr.in') env.List('list.out', 'list.in') env.ListStr('liststr.out', 'liststr.in') +env.Target('target.out', 'target.in') env.Dict('dict1.out', 'dict1.cmd') env.Dict('dict2.out', 'dict2.cmdstr') @@ -96,7 +117,7 @@ env.Dict('dict5.out', 'dict5.lazy') env.Dict('dict6.out', 'dict6.lazystr') env.Dict('dict7.out', 'dict7.list') env.Dict('dict8.out', 'dict8.liststr') -""" % (python, python, python, python)) +""" % locals()) test.write('func.in', "func.in\n") test.write('funcstr.in', "funcstr.in\n") @@ -106,6 +127,7 @@ test.write('lazy.in', "lazy.in\n") test.write('lazystr.in', "lazystr.in\n") test.write('list.in', "list.in\n") test.write('liststr.in', "liststr.in\n") +test.write('target.in', "target.in\n") test.write('dict1.cmd', "dict1.cmd\n") test.write('dict2.cmdstr', "dict2.cmdstr\n") @@ -116,27 +138,33 @@ test.write('dict6.lazystr', "dict6.lazystr\n") test.write('dict7.list', "dict7.list\n") test.write('dict8.liststr', "dict8.liststr\n") -test.run(arguments = '.', stdout=test.wrap_stdout("""\ -%s cat.py cmd.in cmd.out -Building cmdstr.out from cmdstr.in -%s cat.py dict1.cmd dict1.out +expect = test.wrap_stdout("""\ +%(python)s cat.py cmd.in cmd.out +Building cmd1str.out from cmdstr.in +Building cmd2str.out from cmdstr.in +%(python)s cat.py dict1.cmd dict1.out Building dict2.out from dict2.cmdstr func(["dict3.out"], ["dict3.func"]) Building dict4.out from dict4.funcstr -%s cat.py dict5.lazy dict5.out +%(python)s cat.py dict5.lazy dict5.out Building dict6.out from dict6.lazystr -%s cat.py dict7.list .temp -%s cat.py .temp dict7.out +%(python)s cat.py dict7.list .temp +%(python)s cat.py .temp dict7.out Building dict8.out from dict8.liststr Building dict8.out from dict8.liststr func(["func.out"], ["func.in"]) -Building funcstr.out from funcstr.in -%s cat.py lazy.in lazy.out -Building lazystr.out from lazystr.in -%s cat.py list.in .temp -%s cat.py .temp list.out +Building func1str.out from funcstr.in +Building func2str.out from funcstr.in +%(python)s cat.py lazy.in lazy.out +Building lazy1str.out from lazystr.in +Building lazy2str.out from lazystr.in +%(python)s cat.py list.in .temp +%(python)s cat.py .temp list.out Building liststr.out from liststr.in Building liststr.out from liststr.in -""") % (python, python, python, python, python, python, python, python)) +target.out +""" % locals()) + +test.run(arguments = '.', stdout=expect) test.pass_test()