3 Copyright (c) 2001, 2002, 2003 Steven Knight
5 Permission is hereby granted, free of charge, to any person obtaining
6 a copy of this software and associated documentation files (the
7 "Software"), to deal in the Software without restriction, including
8 without limitation the rights to use, copy, modify, merge, publish,
9 distribute, sublicense, and/or sell copies of the Software, and to
10 permit persons to whom the Software is furnished to do so, subject to
11 the following conditions:
13 The above copyright notice and this permission notice shall be included
14 in all copies or substantial portions of the Software.
16 THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY
17 KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
18 WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
19 NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
20 LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
21 OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
22 WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
26 <section id="sect-architecture">
27 <title>Architecture</title>
31 The heart of &SCons; is its <emphasis>Build Engine</emphasis>.
32 The &SCons; Build Engine is a Python module
33 that manages dependencies between
35 such as files or database records.
36 The Build Engine is designed to
38 and easily embeddable in any
39 software system that needs dependency
40 analysis between updatable objects.
46 The key parts of the Build Engine architecture
47 are captured in the following quasi-UML diagram:
52 Including this figure makes our PDF build blow up.
54 is left over from the Software Carpentry contest
55 and is therefore old, out-of-date, and needs to be redone anyway.
56 This is where it will go, anyway...
60 YARG! THIS MAKES THE PDF BUILD BLOW UP. HELP!
62 <title>&SCons; Architecture</title>
63 <graphic fileref="engine.jpg">
69 The point of &SCons; is to manage
70 dependencies between arbitrary external objects.
71 Consequently, the Build Engine does not restrict or specify
72 the nature of the external objects it manages,
73 but instead relies on subclass of the &Node;
74 class to interact with the external system or systems
75 (file systems, database management systems)
76 that maintain the objects being examined or updated.
82 The Build Engine presents to the software system in
84 a Python API for specifying source (input) and target (output) objects,
85 rules for building/updating objects,
86 rules for scanning objects for dependencies, etc.
88 the Build Engine is completely
89 interface-independent,
90 and can be encapsulated by any other software
91 that supports embedded Python.
97 Software that chooses to use the Build Engine
98 for dependency management
100 through <emphasis>Construction Environments</emphasis>.
101 A Construction Environment consists
102 of a dictionary of environment variables,
103 and one or more associated
105 and &Builder; objects.
106 The Python API is used to
107 form these associations.
113 A &Scanner; object specifies
114 how to examine a type of source object
115 (C source file, database record)
116 for dependency information.
117 A &Scanner; object may use
118 variables from the associated
119 Construction Environment
120 to modify how it scans an object:
121 specifying a search path for included files,
122 which field in a database record to consult,
129 A &Builder; object specifies
130 how to update a type of target object:
131 executable program, object file, database field, etc.
132 Like a &Scanner; object,
133 a &Builder; object may use
134 variables from the associated
135 Construction Environment
136 to modify how it builds an object:
137 specifying flags to a compiler,
138 using a different update function,
145 &Scanner; and &Builder; objects will return one or more
146 &Node; objects that represent external objects.
147 &Node; objects are the means by which the
148 Build Engine tracks dependencies:
149 A &Node; may represent a source (input) object that
150 should already exist,
151 or a target (output) object which may be built,
153 The &Node; class is sub-classed to
154 represent external objects of specific type:
155 files, directories, database fields or records, etc.
156 Because dependency information, however,
157 is tracked by the top-level &Node; methods and attributes,
158 dependencies can exist
159 between nodes representing different external object types.
161 building a file could be made
162 dependent on the value of a given
163 field in a database record,
164 or a database table could depend
165 on the contents of an external file.
171 The Build Engine uses a &Job; class (not displayed)
172 to manage the actual work of updating external target objects:
173 spawning commands to build files,
174 submitting the necessary commands to update a database record,
176 The &Job; class has sub-classes
177 to handle differences between spawning
178 jobs in parallel and serially.
184 The Build Engine also uses a
185 &Signature; class (not displayed)
186 to maintain information about whether
187 an external object is up-to-date.
188 Target objects with out-of-date signatures
189 are updated using the appropriate
197 Details on the composition, methods,
198 and attributes of these classes
199 are available in the A HREF="internals.html" Internals /A page.
208 <section id="sect-engine">
209 <title>Build Engine</title>
213 More detailed discussion of some of the
214 Build Engine's characteristics:
219 <title>Python API</title>
223 The Build Engine can be embedded in any other software
224 that supports embedding Python:
226 in a wrapper script that
227 interprets classic <filename>Makefile</filename> syntax,
228 or in any other software that
229 can translate its dependency representation
230 into the appropriate calls to the Build Engine API.
231 <!--<xref linkend="chap-native">--> describes in detail
232 the specification for a "Native Python" interface
233 that will drive the &SCons; implementation effort.
240 <title>Single-image execution</title>
244 When building/updating the objects,
245 the Build Engine operates as a single executable
246 with a complete Directed Acyclic Graph (DAG)
247 of the dependencies in the entire build tree.
248 This is in stark contrast to the
249 commonplace recursive use of Make
250 to handle hierarchical directory-tree builds.
257 <title>Dependency analysis</title>
261 Dependency analysis is carried out via digital signatures
262 (a.k.a. "fingerprints").
263 Contents of object are examined and reduced
264 to a number that can be stored and compared to
265 see if the object has changed.
266 Additionally, &SCons; uses the same
267 signature technique on the command-lines that
268 are executed to update an object.
269 If the command-line has changed since the last time,
270 then the object must be rebuilt.
277 <title>Customized output</title>
281 The output of Build Engine is customizable
282 through user-defined functions.
283 This could be used to print additional desired
284 information about what &SCons; is doing,
285 or tailor output to a specific build analyzer,
293 <title>Build failures</title>
297 &SCons; detects build failures via the exit status from the tools
298 used to build the target files. By default, a failed exit status
299 (non-zero on UNIX systems) terminates the build with an appropriate
300 error message. An appropriate class from the Python library will
301 interpret build-tool failures via an OS-independent API.
307 If multiple tasks are executing in a parallel build, and one tool
308 returns failure, &SCons; will not initiate any further build tasks,
309 but allow the other build tasks to complete before terminating.
315 A <option>-k</option> command-line option may be used to ignore
316 errors and continue building other targets. In no case will a target
317 that depends on a failed build be rebuilt.
327 <section id="sect-interfaces">
328 <title>Interfaces</title>
332 As previously described,
333 the &SCons; Build Engine
334 is interface-independent above its Python API,
335 and can be embedded in any software system
336 that can translate its dependency requirements
337 into the necessary Python calls.
343 The "main" &SCons; interface
344 for implementation purposes,
345 uses Python scripts as configuration files.
346 Because this exposes the Build Engine's Python API to the user,
347 it is current called the "Native Python" interface.
353 This section will also discuss
354 how &SCons; will function in the context
355 of two other interfaces:
356 the &Makefile; interface of the classic &Make; utility,
357 and a hypothetical graphical user interface (GUI).
362 <title>Native Python interface</title>
366 The Native Python interface is intended to be the primary interface
367 by which users will know &SCons;--that is,
368 it is the interface they will use
369 if they actually type &SCons; at a command-line prompt.
375 In the Native Python interface, &SCons; configuration files are simply
376 Python scripts that directly invoke methods from the Build Engine's
377 Python API to specify target files to be built, rules for building
378 the target files, and dependencies. Additional methods, specific to
379 this interface, are added to handle functionality that is specific to
380 the Native Python interface: reading a subsidiary configuration file;
381 copying target files to an installation directory; etc.
387 Because configuration files are Python scripts, Python flow control
388 can be used to provide very flexible manipulation of objects and
389 dependencies. For example, a function could be used to invoke a common
390 set of methods on a file, and called iteratively over an array of
397 As an additional advantage, syntax errors in &SCons; Native Python
398 configuration files will be caught by the Python parser. Target-building
399 does not begin until after all configuration files are read, so a syntax
400 error will not cause a build to fail half-way.
407 <title>Makefile interface</title>
411 An alternate &SCons; interface would provide backwards
412 compatibility with the classic &Make utility.
413 This would be done by embedding the &SCons; Build Engine
414 in a Python script that can translate existing
415 &Makefile;s into the underlying calls to the
416 Build Engine's Python API
417 for building and tracking dependencies.
418 Here are approaches to solving some of the issues
419 that arise from marrying these two pieces:
427 &Makefile; suffix rules can be translated
428 into an appropriate &Builder; object
429 with suffix maps from the Construction Environment.
435 Long lists of static dependences
436 appended to a &Makefile; by
437 various <command>"make depend"</command> schemes
440 the more accurate dependency information
441 provided by &Scanner; objects.
447 Recursive invocations of &Make;
448 can be avoided by reading up
449 the subsidiary &Makefile; instead.
457 Lest this seem like too outlandish an undertaking,
458 there is a working example of this approach:
459 Gary Holt's &Makepp; utility
460 is a Perl script that provides
461 admirably complete parsing of complicated &Makefile;s
462 around an internal build engine inspired,
463 in part, by the classic <application>Cons</application> utility.
470 <title>Graphical interfaces</title>
474 The &SCons; Build Engine
475 is designed from the ground up to be embedded
476 into multiple interfaces.
477 Consequently, embedding the dependency capabilities
478 of &SCons; into graphical interface
479 would be a matter of mapping the
480 GUI's dependency representation
481 (either implicit or explicit)
482 into corresponding calls to the Python API
483 of the &SCons; Build Engine.
489 Note, however, that this proposal leaves the problem of
490 designed a good graphical interface
491 for representing software build dependencies
492 to people with actual GUI design experience...