3 Builder object subsystem.
5 A Builder object is a callable that encapsulates information about how
6 to execute actions to create a target Node (file) from source Nodes
7 (files), and how to create those dependencies for tracking.
9 The main entry point here is the Builder() factory method. This provides
10 a procedural interface that creates the right underlying Builder object
11 based on the keyword arguments supplied and the types of the arguments.
13 The goal is for this external interface to be simple enough that the
14 vast majority of users can create new Builders as necessary to support
15 building new types of files in their configurations, without having to
16 dive any deeper into this subsystem.
18 The base class here is BuilderBase. This is a concrete base class which
19 does, in fact, represent most Builder objects that we (or users) create.
21 There is (at present) one subclasses:
25 This is a Builder that knows how to "chain" Builders so that
26 users can specify a source file that requires multiple steps
27 to turn into a target file. A canonical example is building a
28 program from yacc input file, which requires invoking a builder
29 to turn the .y into a .c, the .c into a .o, and the .o into an
32 There is also two proxies that look like Builders:
36 This proxies for a Builder with an action that is actually a
37 dictionary that knows how to map file suffixes to a specific
38 action. This is so that we can invoke different actions
39 (compilers, compile options) for different flavors of source
44 This proxies for a Builder *invocation* where the target
45 is a list of files, not a single file.
47 Builders and their proxies have the following public interface methods
48 used by other modules:
51 THE public interface. Calling a Builder object (with the
52 use of internal helper methods) sets up the target and source
53 dependencies, appropriate mapping to a specific action, and the
54 environment manipulation necessary for overridden construction
55 variable. This also takes care of warning about possible mistakes
59 Returns the list of targets for a specific builder instance.
62 Adds an emitter for a specific file suffix, used by some Tool
63 modules to specify that (for example) a yacc invocation on a .y
64 can create a .h *and* a .c file.
67 Adds an action for a specific file suffix, heavily used by
68 Tool modules to add their specific action(s) for turning
69 a source file into an object file to the global static
70 and shared object file Builders.
72 There are the following methods for internal use within this module:
75 The internal method that handles the heavily lifting when a
76 Builder is called. This is used so that the __call__() methods
77 can set up warning about possible mistakes in keyword-argument
78 overrides, and *then* execute all of the steps necessary so that
79 the warnings only occur once.
82 Returns the Builder's name within a specific Environment,
83 primarily used to try to return helpful information in error
91 Miscellaneous stuff for handling the prefix and suffix
92 manipulation we use in turning source file names into target
100 # Permission is hereby granted, free of charge, to any person obtaining
101 # a copy of this software and associated documentation files (the
102 # "Software"), to deal in the Software without restriction, including
103 # without limitation the rights to use, copy, modify, merge, publish,
104 # distribute, sublicense, and/or sell copies of the Software, and to
105 # permit persons to whom the Software is furnished to do so, subject to
106 # the following conditions:
108 # The above copyright notice and this permission notice shall be included
109 # in all copies or substantial portions of the Software.
111 # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY
112 # KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
113 # WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
114 # NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
115 # LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
116 # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
117 # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
120 __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__"
127 from SCons.Debug import logInstanceCreation
128 from SCons.Errors import InternalError, UserError
129 import SCons.Executor
132 import SCons.Warnings
139 class DictCmdGenerator(SCons.Util.Selector):
140 """This is a callable class that can be used as a
141 command generator function. It holds on to a dictionary
142 mapping file suffixes to Actions. It uses that dictionary
143 to return the proper action based on the file suffix of
146 def src_suffixes(self):
149 def add_action(self, suffix, action):
150 """Add a suffix-action pair to the mapping.
152 self[suffix] = action
154 def __call__(self, target, source, env, for_signature):
156 for src in map(str, source):
157 my_ext = SCons.Util.splitext(src)[1]
158 if ext and my_ext != ext:
159 raise UserError("While building `%s' from `%s': Cannot build multiple sources with different extensions: %s, %s" % (repr(map(str, target)), src, ext, my_ext))
163 raise UserError("While building `%s': Cannot deduce file extension from source files: %s" % (repr(map(str, target)), repr(map(str, source))))
166 ret = SCons.Util.Selector.__call__(self, env, source)
168 raise UserError("Ambiguous suffixes after environment substitution: %s == %s == %s" % (e[0], e[1], e[2]))
170 raise UserError("While building `%s': Don't know how to build a file with suffix `%s'." % (repr(map(str, target)), ext))
173 class CallableSelector(SCons.Util.Selector):
174 """A callable dictionary that will, in turn, call the value it
176 def __call__(self, env, source):
177 value = SCons.Util.Selector.__call__(self, env, source)
179 value = value(env, source)
182 class DictEmitter(SCons.Util.Selector):
183 """A callable dictionary that maps file suffixes to emitters.
184 When called, it finds the right emitter in its dictionary for the
185 suffix of the first source file, and calls that emitter to get the
186 right lists of targets and sources to return. If there's no emitter
187 for the suffix in its dictionary, the original target and source are
190 def __call__(self, target, source, env):
191 emitter = SCons.Util.Selector.__call__(self, env, source)
193 target, source = emitter(target, source, env)
194 return (target, source)
196 class ListEmitter(UserList.UserList):
197 """A callable list of emitters that calls each in sequence,
198 returning the result.
200 def __call__(self, target, source, env):
202 target, source = e(target, source, env)
203 return (target, source)
205 # These are a common errors when calling a Builder;
206 # they are similar to the 'target' and 'source' keyword args to builders,
207 # so we issue warnings when we see them. The warnings can, of course,
209 misleading_keywords = {
210 'targets' : 'target',
211 'sources' : 'source',
214 class OverrideWarner(UserDict.UserDict):
215 """A class for warning about keyword arguments that we use as
216 overrides in a Builder call.
218 This class exists to handle the fact that a single MultiStepBuilder
219 call can actually invoke multiple builders as a result of a single
220 user-level Builder call. This class only emits the warnings once,
221 no matter how many Builders are invoked.
223 def __init__(self, dict):
224 UserDict.UserDict.__init__(self, dict)
225 self.already_warned = None
227 if self.already_warned:
229 for k in self.keys():
231 alt = misleading_keywords[k]
235 SCons.Warnings.warn(SCons.Warnings.MisleadingKeywordsWarning,
236 "Did you mean to use `%s' instead of `%s'?" % (alt, k))
237 self.already_warned = 1
240 """A factory for builder objects."""
242 if kw.has_key('generator'):
243 if kw.has_key('action'):
244 raise UserError, "You must not specify both an action and a generator."
245 kw['action'] = SCons.Action.CommandGenerator(kw['generator'])
247 elif kw.has_key('action') and SCons.Util.is_Dict(kw['action']):
248 composite = DictCmdGenerator(kw['action'])
249 kw['action'] = SCons.Action.CommandGenerator(composite)
250 kw['src_suffix'] = composite.src_suffixes()
252 if kw.has_key('emitter'):
253 emitter = kw['emitter']
254 if SCons.Util.is_String(emitter):
255 # This allows users to pass in an Environment
256 # variable reference (like "$FOO") as an emitter.
257 # We will look in that Environment variable for
258 # a callable to use as the actual emitter.
259 var = SCons.Util.get_environment_var(emitter)
261 raise UserError, "Supplied emitter '%s' does not appear to refer to an Environment variable" % emitter
262 kw['emitter'] = EmitterProxy(var)
263 elif SCons.Util.is_Dict(emitter):
264 kw['emitter'] = DictEmitter(emitter)
265 elif SCons.Util.is_List(emitter):
266 kw['emitter'] = ListEmitter(emitter)
268 if kw.has_key('src_builder'):
269 ret = apply(MultiStepBuilder, (), kw)
271 ret = apply(BuilderBase, (), kw)
273 if not composite is None:
274 ret = CompositeBuilder(ret, composite)
278 def _init_nodes(builder, env, overrides, tlist, slist):
279 """Initialize lists of target and source nodes with all of
280 the proper Builder information.
283 # First, figure out if there are any errors in the way the targets
287 raise UserError, "Multiple ways to build the same target were specified for: %s" % str(t)
290 t_contents = t.builder.action.get_contents(tlist, slist, t.env)
291 contents = t.builder.action.get_contents(tlist, slist, env)
293 if t_contents == contents:
294 SCons.Warnings.warn(SCons.Warnings.DuplicateEnvironmentWarning,
295 "Two different environments were specified for target %s,\n\tbut they appear to have the same action: %s"%(str(t), t.builder.action.strfunction(tlist, slist, t.env)))
298 raise UserError, "Two environments with different actions were specified for the same target: %s"%str(t)
300 elif t.overrides != overrides:
301 raise UserError, "Two different sets of overrides were specified for the same target: %s"%str(t)
303 elif builder.scanner and t.target_scanner and builder.scanner != t.target_scanner:
304 raise UserError, "Two different scanners were specified for the same target: %s"%str(t)
307 if t.builder != builder:
308 if isinstance(t.builder, ListBuilder) and isinstance(builder, ListBuilder) and t.builder.builder == builder.builder:
309 raise UserError, "Two different target sets have a target in common: %s"%str(t)
311 raise UserError, "Two different builders (%s and %s) were specified for the same target: %s"%(t.builder.get_name(env), builder.get_name(env), str(t))
312 elif t.sources != slist:
313 raise UserError, "Multiple ways to build the same target were specified for: %s" % str(t)
315 # The targets are fine, so find or make the appropriate Executor to
316 # build this particular list of targets from this particular list of
321 executor = tlist[0].get_executor(create = 0)
322 except AttributeError:
325 executor.add_sources(slist)
327 executor = SCons.Executor.Executor(builder.action,
329 [builder.overrides, overrides],
333 # Now set up the relevant information in the target Nodes themselves.
335 t.overrides = overrides
336 t.cwd = SCons.Node.FS.default_fs.getcwd()
337 t.builder_set(builder)
340 t.set_executor(executor)
342 t.target_scanner = builder.scanner
343 if not t.source_scanner:
344 t.source_scanner = env.get_scanner(t.scanner_key())
346 # Last, add scanners from the Environment to the source Nodes.
348 if not s.source_scanner:
349 s.source_scanner = env.get_scanner(s.scanner_key())
352 """This is a callable class that can act as a
353 Builder emitter. It holds on to a string that
354 is a key into an Environment dictionary, and will
355 look there at actual build time to see if it holds
356 a callable. If so, we will call that as the actual
358 def __init__(self, var):
359 self.var = SCons.Util.to_String(var)
361 def __call__(self, target, source, env):
364 # Recursively substitute the variable.
365 # We can't use env.subst() because it deals only
366 # in strings. Maybe we should change that?
367 while SCons.Util.is_String(emitter) and env.has_key(emitter):
368 emitter = env[emitter]
369 if callable(emitter):
370 target, source = emitter(target, source, env)
371 elif SCons.Util.is_List(emitter):
373 target, source = e(target, source, env)
375 return (target, source)
378 def __cmp__(self, other):
379 return cmp(self.var, other.var)
382 """Base class for Builders, objects that create output
383 nodes (files) from input nodes (files).
386 def __init__(self, action = None,
390 node_factory = SCons.Node.FS.default_fs.File,
391 target_factory = None,
392 source_factory = None,
398 if __debug__: logInstanceCreation(self, 'BuilderBase')
399 self.action = SCons.Action.Action(action)
401 if SCons.Util.is_Dict(prefix):
402 prefix = CallableSelector(prefix)
404 if SCons.Util.is_Dict(suffix):
405 suffix = CallableSelector(suffix)
408 if overrides.has_key('overrides'):
409 SCons.Warnings.warn(SCons.Warnings.DeprecatedWarning,
410 "The \"overrides\" keyword to Builder() creation has been deprecated;\n" +\
411 "\tspecify the items as keyword arguments to the Builder() call instead.")
412 overrides.update(overrides['overrides'])
413 del overrides['overrides']
414 self.overrides = overrides
416 self.set_src_suffix(src_suffix)
418 self.target_factory = target_factory or node_factory
419 self.source_factory = source_factory or node_factory
420 self.scanner = scanner
422 self.emitter = emitter
424 def __nonzero__(self):
425 raise InternalError, "Do not test for the Node.builder attribute directly; use Node.has_builder() instead"
427 def get_name(self, env):
428 """Attempts to get the name of the Builder.
430 Look at the BUILDERS variable of env, expecting it to be a
431 dictionary containing this Builder, and return the key of the
435 index = env['BUILDERS'].values().index(self)
436 return env['BUILDERS'].keys()[index]
437 except (AttributeError, KeyError, ValueError):
438 return str(self.__class__)
440 def __cmp__(self, other):
441 return cmp(self.__dict__, other.__dict__)
443 def splitext(self, path):
444 return SCons.Util.splitext(path)
446 def _create_nodes(self, env, overwarn, target = None, source = None):
447 """Create and return lists of target and source nodes.
449 def _adjustixes(files, pre, suf):
453 if not SCons.Util.is_List(files):
457 if SCons.Util.is_String(f):
458 f = SCons.Util.adjustixes(f, pre, suf)
464 env = env.Override(overwarn.data)
466 src_suf = self.get_src_suffix(env)
468 source = _adjustixes(source, None, src_suf)
469 slist = env.arg2nodes(source, self.source_factory)
471 pre = self.get_prefix(env, slist)
472 suf = self.get_suffix(env, slist)
476 t_from_s = slist[0].target_from_source
477 except AttributeError:
478 raise UserError("Do not know how to create a target from source `%s'" % slist[0])
479 tlist = [ t_from_s(pre, suf, self.splitext) ]
481 target = _adjustixes(target, pre, suf)
482 tlist = env.arg2nodes(target, self.target_factory)
485 # The emitter is going to do str(node), but because we're
486 # being called *from* a builder invocation, the new targets
487 # don't yet have a builder set on them and will look like
488 # source files. Fool the emitter's str() calls by setting
489 # up a temporary builder on the new targets.
492 if not t.is_derived():
494 new_targets.append(t)
496 target, source = self.emitter(target=tlist, source=slist, env=env)
498 # Now delete the temporary builders that we attached to any
499 # new targets, so that _init_nodes() doesn't do weird stuff
500 # to them because it thinks they already have builders.
501 for t in new_targets:
502 if t.builder is self:
503 # Only delete the temporary builder if the emitter
504 # didn't change it on us.
507 # Have to call arg2nodes yet again, since it is legal for
508 # emitters to spit out strings as well as Node instances.
509 slist = env.arg2nodes(source, self.source_factory)
510 tlist = env.arg2nodes(target, self.target_factory)
514 def _execute(self, env, target = None, source = _null, overwarn={}):
518 tlist, slist = self._create_nodes(env, overwarn, target, source)
524 builder = ListBuilder(self, env, tlist)
526 _init_nodes(builder, env, overwarn.data, tlist, slist)
530 def __call__(self, env, target = None, source = _null, **kw):
531 return self._execute(env, target, source, OverrideWarner(kw))
533 def adjust_suffix(self, suff):
534 if suff and not suff[0] in [ '.', '_', '$' ]:
538 def get_prefix(self, env, sources=[]):
541 prefix = prefix(env, sources)
542 return env.subst(prefix)
544 def get_suffix(self, env, sources=[]):
547 suffix = suffix(env, sources)
549 suffix = self.adjust_suffix(suffix)
550 return env.subst(suffix)
552 def src_suffixes(self, env):
553 return map(lambda x, s=self, e=env: e.subst(s.adjust_suffix(x)),
556 def set_src_suffix(self, src_suffix):
559 elif not SCons.Util.is_List(src_suffix):
560 src_suffix = [ src_suffix ]
561 self.src_suffix = src_suffix
563 def get_src_suffix(self, env):
564 """Get the first src_suffix in the list of src_suffixes."""
565 ret = self.src_suffixes(env)
570 def targets(self, node):
571 """Return the list of targets for this builder instance.
573 For most normal builders, this is just the supplied node.
577 def add_emitter(self, suffix, emitter):
578 """Add a suffix-emitter mapping to this Builder.
580 This assumes that emitter has been initialized with an
581 appropriate dictionary type, and will throw a TypeError if
582 not, so the caller is responsible for knowing that this is an
583 appropriate method to call for the Builder in question.
585 self.emitter[suffix] = emitter
587 class ListBuilder(SCons.Util.Proxy):
588 """A Proxy to support building an array of targets (for example,
589 foo.o and foo.h from foo.y) from a single Action execution.
592 def __init__(self, builder, env, tlist):
593 if __debug__: logInstanceCreation(self)
594 SCons.Util.Proxy.__init__(self, builder)
595 self.builder = builder
596 self.scanner = builder.scanner
599 self.multi = builder.multi
601 def targets(self, node):
602 """Return the list of targets for this builder instance.
606 def __cmp__(self, other):
607 return cmp(self.__dict__, other.__dict__)
609 def get_name(self, env):
610 """Attempts to get the name of the Builder."""
612 return "ListBuilder(%s)" % self.builder.get_name(env)
614 class MultiStepBuilder(BuilderBase):
615 """This is a builder subclass that can build targets in
616 multiple steps. The src_builder parameter to the constructor
617 accepts a builder that is called to build sources supplied to
618 this builder. The targets of that first build then become
619 the sources of this builder.
621 If this builder has a src_suffix supplied, then the src_builder
622 builder is NOT invoked if the suffix of a source file matches
625 def __init__(self, src_builder,
630 node_factory = SCons.Node.FS.default_fs.File,
631 target_factory = None,
632 source_factory = None,
635 if __debug__: logInstanceCreation(self)
636 BuilderBase.__init__(self, action, prefix, suffix, src_suffix,
637 node_factory, target_factory, source_factory,
639 if not SCons.Util.is_List(src_builder):
640 src_builder = [ src_builder ]
641 self.src_builder = src_builder
643 self.cached_src_suffixes = {} # source suffixes keyed on id(env)
645 def _execute(self, env, target = None, source = _null, overwarn={}):
650 slist = env.arg2nodes(source, self.source_factory)
654 sdict = self.sdict[id(env)]
657 self.sdict[id(env)] = sdict
658 for bld in self.src_builder:
659 if SCons.Util.is_String(bld):
661 bld = env['BUILDERS'][bld]
664 for suf in bld.src_suffixes(env):
667 src_suffixes = self.src_suffixes(env)
671 get_suffix = snode.get_suffix
672 except AttributeError:
673 ext = self.splitext(str(snode))
677 subsidiary_builder = sdict[ext]
679 final_sources.append(snode)
681 tgt = subsidiary_builder._execute(env, None, snode, overwarn)
682 # Only supply the builder with sources it is capable
684 if SCons.Util.is_List(tgt):
685 tgt = filter(lambda x, self=self, suf=src_suffixes:
686 self.splitext(SCons.Util.to_String(x))[1] in suf,
688 if not SCons.Util.is_List(tgt):
689 final_sources.append(tgt)
691 final_sources.extend(tgt)
693 return BuilderBase._execute(self, env, target, final_sources, overwarn)
695 def get_src_builders(self, env):
696 """Return all the src_builders for this Builder.
698 This is essentially a recursive descent of the src_builder "tree."
701 for bld in self.src_builder:
702 if SCons.Util.is_String(bld):
703 # All Environments should have a BUILDERS
704 # variable, so no need to check for it.
706 bld = env['BUILDERS'][bld]
712 def src_suffixes(self, env):
713 """Return a list of the src_suffix attributes for all
714 src_builders of this Builder.
717 return self.cached_src_suffixes[id(env)]
719 suffixes = BuilderBase.src_suffixes(self, env)
720 for builder in self.get_src_builders(env):
721 suffixes.extend(builder.src_suffixes(env))
722 self.cached_src_suffixes[id(env)] = suffixes
725 class CompositeBuilder(SCons.Util.Proxy):
726 """A Builder Proxy whose main purpose is to always have
727 a DictCmdGenerator as its action, and to provide access
728 to the DictCmdGenerator's add_action() method.
731 def __init__(self, builder, cmdgen):
732 if __debug__: logInstanceCreation(self)
733 SCons.Util.Proxy.__init__(self, builder)
735 # cmdgen should always be an instance of DictCmdGenerator.
737 self.builder = builder
739 def add_action(self, suffix, action):
740 self.cmdgen.add_action(suffix, action)
741 self.set_src_suffix(self.cmdgen.src_suffixes())
743 def __cmp__(self, other):
744 return cmp(self.__dict__, other.__dict__)