3 Builder object subsystem.
5 A Builder object is a callable that encapsulates information about how
6 to execute actions to create a target Node (file) from source Nodes
7 (files), and how to create those dependencies for tracking.
9 The main entry point here is the Builder() factory method. This provides
10 a procedural interface that creates the right underlying Builder object
11 based on the keyword arguments supplied and the types of the arguments.
13 The goal is for this external interface to be simple enough that the
14 vast majority of users can create new Builders as necessary to support
15 building new types of files in their configurations, without having to
16 dive any deeper into this subsystem.
18 The base class here is BuilderBase. This is a concrete base class which
19 does, in fact, represent most Builder objects that we (or users) create.
21 There is (at present) one subclasses:
25 This is a Builder that knows how to "chain" Builders so that
26 users can specify a source file that requires multiple steps
27 to turn into a target file. A canonical example is building a
28 program from yacc input file, which requires invoking a builder
29 to turn the .y into a .c, the .c into a .o, and the .o into an
32 There is also two proxies that look like Builders:
36 This proxies for a Builder with an action that is actually a
37 dictionary that knows how to map file suffixes to a specific
38 action. This is so that we can invoke different actions
39 (compilers, compile options) for different flavors of source
44 This proxies for a Builder *invocation* where the target
45 is a list of files, not a single file.
47 Builders and their proxies have the following public interface methods
48 used by other modules:
51 THE public interface. Calling a Builder object (with the
52 use of internal helper methods) sets up the target and source
53 dependencies, appropriate mapping to a specific action, and the
54 environment manipulation necessary for overridden construction
55 variable. This also takes care of warning about possible mistakes
59 Returns the list of targets for a specific builder instance.
62 Adds an emitter for a specific file suffix, used by some Tool
63 modules to specify that (for example) a yacc invocation on a .y
64 can create a .h *and* a .c file.
67 Adds an action for a specific file suffix, heavily used by
68 Tool modules to add their specific action(s) for turning
69 a source file into an object file to the global static
70 and shared object file Builders.
72 There are the following methods for internal use within this module:
75 The internal method that handles the heavily lifting when a
76 Builder is called. This is used so that the __call__() methods
77 can set up warning about possible mistakes in keyword-argument
78 overrides, and *then* execute all of the steps necessary so that
79 the warnings only occur once.
82 Returns the Builder's name within a specific Environment,
83 primarily used to try to return helpful information in error
91 Miscellaneous stuff for handling the prefix and suffix
92 manipulation we use in turning source file names into target
100 # Permission is hereby granted, free of charge, to any person obtaining
101 # a copy of this software and associated documentation files (the
102 # "Software"), to deal in the Software without restriction, including
103 # without limitation the rights to use, copy, modify, merge, publish,
104 # distribute, sublicense, and/or sell copies of the Software, and to
105 # permit persons to whom the Software is furnished to do so, subject to
106 # the following conditions:
108 # The above copyright notice and this permission notice shall be included
109 # in all copies or substantial portions of the Software.
111 # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY
112 # KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
113 # WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
114 # NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
115 # LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
116 # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
117 # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
120 __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__"
126 from SCons.Debug import logInstanceCreation
127 from SCons.Errors import InternalError, UserError
128 import SCons.Executor
131 import SCons.Warnings
138 class DictCmdGenerator(SCons.Util.Selector):
139 """This is a callable class that can be used as a
140 command generator function. It holds on to a dictionary
141 mapping file suffixes to Actions. It uses that dictionary
142 to return the proper action based on the file suffix of
145 def src_suffixes(self):
148 def add_action(self, suffix, action):
149 """Add a suffix-action pair to the mapping.
151 self[suffix] = action
153 def __call__(self, target, source, env, for_signature):
155 for src in map(str, source):
156 my_ext = SCons.Util.splitext(src)[1]
157 if ext and my_ext != ext:
158 raise UserError("While building `%s' from `%s': Cannot build multiple sources with different extensions: %s, %s" % (repr(map(str, target)), src, ext, my_ext))
162 raise UserError("While building `%s': Cannot deduce file extension from source files: %s" % (repr(map(str, target)), repr(map(str, source))))
165 ret = SCons.Util.Selector.__call__(self, env, source)
167 raise UserError("Ambiguous suffixes after environment substitution: %s == %s == %s" % (e[0], e[1], e[2]))
169 raise UserError("While building `%s': Don't know how to build a file with suffix `%s'." % (repr(map(str, target)), ext))
172 class CallableSelector(SCons.Util.Selector):
173 """A callable dictionary that will, in turn, call the value it
175 def __call__(self, env, source):
176 value = SCons.Util.Selector.__call__(self, env, source)
178 value = value(env, source)
181 class DictEmitter(SCons.Util.Selector):
182 """A callable dictionary that maps file suffixes to emitters.
183 When called, it finds the right emitter in its dictionary for the
184 suffix of the first source file, and calls that emitter to get the
185 right lists of targets and sources to return. If there's no emitter
186 for the suffix in its dictionary, the original target and source are
189 def __call__(self, target, source, env):
190 emitter = SCons.Util.Selector.__call__(self, env, source)
192 target, source = emitter(target, source, env)
193 return (target, source)
195 class ListEmitter(UserList.UserList):
196 """A callable list of emitters that calls each in sequence,
197 returning the result.
199 def __call__(self, target, source, env):
201 target, source = e(target, source, env)
202 return (target, source)
204 # These are a common errors when calling a Builder;
205 # they are similar to the 'target' and 'source' keyword args to builders,
206 # so we issue warnings when we see them. The warnings can, of course,
208 misleading_keywords = {
209 'targets' : 'target',
210 'sources' : 'source',
213 class OverrideWarner(UserDict.UserDict):
214 """A class for warning about keyword arguments that we use as
215 overrides in a Builder call.
217 This class exists to handle the fact that a single MultiStepBuilder
218 call can actually invoke multiple builders as a result of a single
219 user-level Builder call. This class only emits the warnings once,
220 no matter how many Builders are invoked.
222 def __init__(self, dict):
223 UserDict.UserDict.__init__(self, dict)
224 self.already_warned = None
226 if self.already_warned:
228 for k in self.keys():
230 alt = misleading_keywords[k]
234 SCons.Warnings.warn(SCons.Warnings.MisleadingKeywordsWarning,
235 "Did you mean to use `%s' instead of `%s'?" % (alt, k))
236 self.already_warned = 1
239 """A factory for builder objects."""
241 if kw.has_key('generator'):
242 if kw.has_key('action'):
243 raise UserError, "You must not specify both an action and a generator."
244 kw['action'] = SCons.Action.CommandGenerator(kw['generator'])
246 elif kw.has_key('action') and SCons.Util.is_Dict(kw['action']):
247 composite = DictCmdGenerator(kw['action'])
248 kw['action'] = SCons.Action.CommandGenerator(composite)
249 kw['src_suffix'] = composite.src_suffixes()
251 if kw.has_key('emitter'):
252 emitter = kw['emitter']
253 if SCons.Util.is_String(emitter):
254 # This allows users to pass in an Environment
255 # variable reference (like "$FOO") as an emitter.
256 # We will look in that Environment variable for
257 # a callable to use as the actual emitter.
258 var = SCons.Util.get_environment_var(emitter)
260 raise UserError, "Supplied emitter '%s' does not appear to refer to an Environment variable" % emitter
261 kw['emitter'] = EmitterProxy(var)
262 elif SCons.Util.is_Dict(emitter):
263 kw['emitter'] = DictEmitter(emitter)
264 elif SCons.Util.is_List(emitter):
265 kw['emitter'] = ListEmitter(emitter)
267 if kw.has_key('src_builder'):
268 ret = apply(MultiStepBuilder, (), kw)
270 ret = apply(BuilderBase, (), kw)
272 if not composite is None:
273 ret = CompositeBuilder(ret, composite)
277 def _init_nodes(builder, env, overrides, executor_kw, tlist, slist):
278 """Initialize lists of target and source nodes with all of
279 the proper Builder information.
282 # First, figure out if there are any errors in the way the targets
286 raise UserError, "Multiple ways to build the same target were specified for: %s" % str(t)
287 if t.has_explicit_builder():
288 if not t.env is None and not t.env is env:
289 t_contents = t.builder.action.get_contents(tlist, slist, t.env)
290 contents = t.builder.action.get_contents(tlist, slist, env)
292 if t_contents == contents:
293 SCons.Warnings.warn(SCons.Warnings.DuplicateEnvironmentWarning,
294 "Two different environments were specified for target %s,\n\tbut they appear to have the same action: %s"%(str(t), t.builder.action.genstring(tlist, slist, t.env)))
297 raise UserError, "Two environments with different actions were specified for the same target: %s"%str(t)
299 elif t.overrides != overrides:
300 raise UserError, "Two different sets of overrides were specified for the same target: %s"%str(t)
302 elif builder.target_scanner and t.target_scanner and builder.target_scanner != t.target_scanner:
303 raise UserError, "Two different scanners were specified for the same target: %s"%str(t)
306 if t.builder != builder:
307 if isinstance(t.builder, ListBuilder) and isinstance(builder, ListBuilder) and t.builder.builder == builder.builder:
308 raise UserError, "Two different target sets have a target in common: %s"%str(t)
310 raise UserError, "Two different builders (%s and %s) were specified for the same target: %s"%(t.builder.get_name(env), builder.get_name(env), str(t))
311 elif isinstance(t.builder, ListBuilder) ^ isinstance(builder, ListBuilder):
312 raise UserError, "Cannot build same target `%s' as singular and list"%str(t)
313 elif t.sources != slist:
314 raise UserError, "Multiple ways to build the same target were specified for: %s (from %s and from %s)" % (str(t), map(str,t.sources), map(str,slist))
316 if builder.single_source:
318 raise UserError, "More than one source given for single-source builder: targets=%s sources=%s" % (map(str,tlist), map(str,slist))
320 # The targets are fine, so find or make the appropriate Executor to
321 # build this particular list of targets from this particular list of
326 executor = tlist[0].get_executor(create = 0)
327 except AttributeError:
330 executor.add_sources(slist)
332 if not builder.action:
333 raise UserError, "Builder %s must have an action to build %s."%(builder.get_name(env or builder.env), map(str,tlist))
334 executor = SCons.Executor.Executor(builder.action,
336 [builder.overrides, overrides],
341 # Now set up the relevant information in the target Nodes themselves.
343 t.overrides = overrides
344 t.cwd = SCons.Node.FS.default_fs.getcwd()
345 t.builder_set(builder)
348 t.set_executor(executor)
349 if builder.target_scanner:
350 t.target_scanner = builder.target_scanner
351 if t.source_scanner is None:
352 t.source_scanner = builder.source_scanner
354 # Add backup source scanners from the environment to the source
355 # nodes. This may not be necessary if the node will have a real
356 # source scanner added later (which is why these are the "backup"
357 # source scanners, not the real ones), but because source nodes may
358 # be used multiple times for different targets, it ends up being
359 # more efficient to do this calculation once here, as opposed to
360 # delaying it until later when we potentially have to calculate it
361 # over and over and over.
363 if s.source_scanner is None and s.backup_source_scanner is None:
364 s.backup_source_scanner = env.get_scanner(s.scanner_key())
367 """This is a callable class that can act as a
368 Builder emitter. It holds on to a string that
369 is a key into an Environment dictionary, and will
370 look there at actual build time to see if it holds
371 a callable. If so, we will call that as the actual
373 def __init__(self, var):
374 self.var = SCons.Util.to_String(var)
376 def __call__(self, target, source, env):
379 # Recursively substitute the variable.
380 # We can't use env.subst() because it deals only
381 # in strings. Maybe we should change that?
382 while SCons.Util.is_String(emitter) and env.has_key(emitter):
383 emitter = env[emitter]
384 if callable(emitter):
385 target, source = emitter(target, source, env)
386 elif SCons.Util.is_List(emitter):
388 target, source = e(target, source, env)
390 return (target, source)
393 def __cmp__(self, other):
394 return cmp(self.var, other.var)
397 """Base class for Builders, objects that create output
398 nodes (files) from input nodes (files).
401 def __init__(self, action = None,
405 target_factory = SCons.Node.FS.default_fs.File,
406 source_factory = SCons.Node.FS.default_fs.File,
407 target_scanner = None,
408 source_scanner = None,
417 if __debug__: logInstanceCreation(self, 'BuilderBase')
418 self.action = SCons.Action.Action(action)
420 if SCons.Util.is_Dict(prefix):
421 prefix = CallableSelector(prefix)
423 if SCons.Util.is_Dict(suffix):
424 suffix = CallableSelector(suffix)
427 self.single_source = single_source
428 if overrides.has_key('overrides'):
429 SCons.Warnings.warn(SCons.Warnings.DeprecatedWarning,
430 "The \"overrides\" keyword to Builder() creation has been deprecated;\n" +\
431 "\tspecify the items as keyword arguments to the Builder() call instead.")
432 overrides.update(overrides['overrides'])
433 del overrides['overrides']
434 if overrides.has_key('scanner'):
435 SCons.Warnings.warn(SCons.Warnings.DeprecatedWarning,
436 "The \"scanner\" keyword to Builder() creation has been deprecated;\n"
437 "\tuse: source_scanner or target_scanner as appropriate.")
438 del overrides['scanner']
439 self.overrides = overrides
441 self.set_src_suffix(src_suffix)
443 self.target_factory = target_factory
444 self.source_factory = source_factory
445 self.target_scanner = target_scanner
446 self.source_scanner = source_scanner
448 self.emitter = emitter
450 # Optional Builder name should only be used for Builders
451 # that don't get attached to construction environments.
454 self.executor_kw = {}
455 if not chdir is _null:
456 self.executor_kw['chdir'] = chdir
457 self.is_explicit = is_explicit
459 def __nonzero__(self):
460 raise InternalError, "Do not test for the Node.builder attribute directly; use Node.has_builder() instead"
462 def get_name(self, env):
463 """Attempts to get the name of the Builder.
465 Look at the BUILDERS variable of env, expecting it to be a
466 dictionary containing this Builder, and return the key of the
467 dictionary. If there's no key, then return a directly-configured
468 name (if there is one) or the name of the class (by default)."""
471 index = env['BUILDERS'].values().index(self)
472 return env['BUILDERS'].keys()[index]
473 except (AttributeError, KeyError, ValueError):
476 except AttributeError:
477 return str(self.__class__)
479 def __cmp__(self, other):
480 return cmp(self.__dict__, other.__dict__)
482 def splitext(self, path, env=None):
486 matchsuf = filter(lambda S,path=path: path[-len(S):] == S,
487 self.src_suffixes(env))
489 suf = max(map(None, map(len, matchsuf), matchsuf))[1]
490 return [path[:-len(suf)], path[-len(suf):]]
491 return SCons.Util.splitext(path)
493 def _create_nodes(self, env, overwarn, target = None, source = None):
494 """Create and return lists of target and source nodes.
496 def _adjustixes(files, pre, suf):
500 if not SCons.Util.is_List(files):
504 if SCons.Util.is_String(f):
505 f = SCons.Util.adjustixes(f, pre, suf)
511 env = env.Override(overwarn.data)
513 src_suf = self.get_src_suffix(env)
515 source = _adjustixes(source, None, src_suf)
516 slist = env.arg2nodes(source, self.source_factory)
518 pre = self.get_prefix(env, slist)
519 suf = self.get_suffix(env, slist)
523 t_from_s = slist[0].target_from_source
524 except AttributeError:
525 raise UserError("Do not know how to create a target from source `%s'" % slist[0])
526 splitext = lambda S,self=self,env=env: self.splitext(S,env)
527 tlist = [ t_from_s(pre, suf, splitext) ]
529 target = _adjustixes(target, pre, suf)
530 tlist = env.arg2nodes(target, self.target_factory)
533 # The emitter is going to do str(node), but because we're
534 # being called *from* a builder invocation, the new targets
535 # don't yet have a builder set on them and will look like
536 # source files. Fool the emitter's str() calls by setting
537 # up a temporary builder on the new targets.
540 if not t.is_derived():
542 new_targets.append(t)
544 target, source = self.emitter(target=tlist, source=slist, env=env)
546 # Now delete the temporary builders that we attached to any
547 # new targets, so that _init_nodes() doesn't do weird stuff
548 # to them because it thinks they already have builders.
549 for t in new_targets:
550 if t.builder is self:
551 # Only delete the temporary builder if the emitter
552 # didn't change it on us.
555 # Have to call arg2nodes yet again, since it is legal for
556 # emitters to spit out strings as well as Node instances.
557 slist = env.arg2nodes(source, self.source_factory)
558 tlist = env.arg2nodes(target, self.target_factory)
562 def _execute(self, env, target=None, source=_null, overwarn={}, executor_kw={}):
567 if(self.single_source and
568 SCons.Util.is_List(source) and
572 if target is None: target = [None]*len(source)
573 for k in range(len(source)):
574 t = self._execute(env, target[k], source[k], overwarn)
575 if SCons.Util.is_List(t):
581 tlist, slist = self._create_nodes(env, overwarn, target, source)
586 builder = ListBuilder(self, env, tlist)
587 _init_nodes(builder, env, overwarn.data, executor_kw, tlist, slist)
591 def __call__(self, env, target=None, source=_null, chdir=_null, **kw):
593 ekw = self.executor_kw
595 ekw = self.executor_kw.copy()
597 return self._execute(env, target, source, OverrideWarner(kw), ekw)
599 def adjust_suffix(self, suff):
600 if suff and not suff[0] in [ '.', '_', '$' ]:
604 def get_prefix(self, env, sources=[]):
607 prefix = prefix(env, sources)
608 return env.subst(prefix)
610 def get_suffix(self, env, sources=[]):
613 suffix = suffix(env, sources)
615 suffix = self.adjust_suffix(suffix)
616 return env.subst(suffix)
618 def src_suffixes(self, env):
619 return map(lambda x, s=self, e=env: e.subst(s.adjust_suffix(x)),
622 def set_src_suffix(self, src_suffix):
625 elif not SCons.Util.is_List(src_suffix):
626 src_suffix = [ src_suffix ]
627 self.src_suffix = src_suffix
629 def get_src_suffix(self, env):
630 """Get the first src_suffix in the list of src_suffixes."""
631 ret = self.src_suffixes(env)
636 def targets(self, node):
637 """Return the list of targets for this builder instance.
639 For most normal builders, this is just the supplied node.
643 def add_emitter(self, suffix, emitter):
644 """Add a suffix-emitter mapping to this Builder.
646 This assumes that emitter has been initialized with an
647 appropriate dictionary type, and will throw a TypeError if
648 not, so the caller is responsible for knowing that this is an
649 appropriate method to call for the Builder in question.
651 self.emitter[suffix] = emitter
653 class ListBuilder(SCons.Util.Proxy):
654 """A Proxy to support building an array of targets (for example,
655 foo.o and foo.h from foo.y) from a single Action execution.
658 def __init__(self, builder, env, tlist):
659 if __debug__: logInstanceCreation(self)
660 SCons.Util.Proxy.__init__(self, builder)
661 self.builder = builder
662 self.target_scanner = builder.target_scanner
663 self.source_scanner = builder.source_scanner
666 self.multi = builder.multi
667 self.single_source = builder.single_source
669 def targets(self, node):
670 """Return the list of targets for this builder instance.
674 def get_name(self, env):
675 """Attempts to get the name of the Builder."""
677 return "ListBuilder(%s)" % self.builder.get_name(env)
679 class MultiStepBuilder(BuilderBase):
680 """This is a builder subclass that can build targets in
681 multiple steps. The src_builder parameter to the constructor
682 accepts a builder that is called to build sources supplied to
683 this builder. The targets of that first build then become
684 the sources of this builder.
686 If this builder has a src_suffix supplied, then the src_builder
687 builder is NOT invoked if the suffix of a source file matches
690 def __init__(self, src_builder,
695 target_factory = SCons.Node.FS.default_fs.File,
696 source_factory = SCons.Node.FS.default_fs.File,
697 target_scanner = None,
698 source_scanner = None,
701 if __debug__: logInstanceCreation(self)
702 BuilderBase.__init__(self, action, prefix, suffix, src_suffix,
703 target_factory, source_factory,
704 target_scanner, source_scanner, emitter,
705 single_source = single_source)
706 if not SCons.Util.is_List(src_builder):
707 src_builder = [ src_builder ]
708 self.src_builder = src_builder
710 self.cached_src_suffixes = {} # source suffixes keyed on id(env)
712 def _execute(self, env, target = None, source = _null, overwarn={}, executor_kw={}):
717 slist = env.arg2nodes(source, self.source_factory)
721 sdict = self.sdict[id(env)]
724 self.sdict[id(env)] = sdict
725 for bld in self.src_builder:
726 if SCons.Util.is_String(bld):
728 bld = env['BUILDERS'][bld]
731 for suf in bld.src_suffixes(env):
734 src_suffixes = self.src_suffixes(env)
737 for srcsuf in src_suffixes:
738 if str(snode)[-len(srcsuf):] == srcsuf and sdict.has_key(srcsuf):
739 tgt = sdict[srcsuf]._execute(env, None, snode, overwarn)
740 # If the subsidiary Builder returned more than one target,
741 # then filter out any sources that this Builder isn't
742 # capable of building.
744 tgt = filter(lambda x, self=self, suf=src_suffixes, e=env:
745 self.splitext(SCons.Util.to_String(x),e)[1] in suf,
747 final_sources.extend(tgt)
751 final_sources.append(snode)
753 return BuilderBase._execute(self, env, target, final_sources, overwarn)
755 def get_src_builders(self, env):
756 """Return all the src_builders for this Builder.
758 This is essentially a recursive descent of the src_builder "tree."
761 for bld in self.src_builder:
762 if SCons.Util.is_String(bld):
763 # All Environments should have a BUILDERS
764 # variable, so no need to check for it.
766 bld = env['BUILDERS'][bld]
772 def src_suffixes(self, env):
773 """Return a list of the src_suffix attributes for all
774 src_builders of this Builder.
777 return self.cached_src_suffixes[id(env)]
779 suffixes = BuilderBase.src_suffixes(self, env)
780 for builder in self.get_src_builders(env):
781 suffixes.extend(builder.src_suffixes(env))
782 self.cached_src_suffixes[id(env)] = suffixes
785 class CompositeBuilder(SCons.Util.Proxy):
786 """A Builder Proxy whose main purpose is to always have
787 a DictCmdGenerator as its action, and to provide access
788 to the DictCmdGenerator's add_action() method.
791 def __init__(self, builder, cmdgen):
792 if __debug__: logInstanceCreation(self)
793 SCons.Util.Proxy.__init__(self, builder)
795 # cmdgen should always be an instance of DictCmdGenerator.
797 self.builder = builder
799 def add_action(self, suffix, action):
800 self.cmdgen.add_action(suffix, action)
801 self.set_src_suffix(self.cmdgen.src_suffixes())