3 Builder object subsystem.
5 A Builder object is a callable that encapsulates information about how
6 to execute actions to create a target Node (file) from source Nodes
7 (files), and how to create those dependencies for tracking.
9 The main entry point here is the Builder() factory method. This provides
10 a procedural interface that creates the right underlying Builder object
11 based on the keyword arguments supplied and the types of the arguments.
13 The goal is for this external interface to be simple enough that the
14 vast majority of users can create new Builders as necessary to support
15 building new types of files in their configurations, without having to
16 dive any deeper into this subsystem.
18 The base class here is BuilderBase. This is a concrete base class which
19 does, in fact, represent most Builder objects that we (or users) create.
21 There is (at present) one subclasses:
25 This is a Builder that knows how to "chain" Builders so that
26 users can specify a source file that requires multiple steps
27 to turn into a target file. A canonical example is building a
28 program from yacc input file, which requires invoking a builder
29 to turn the .y into a .c, the .c into a .o, and the .o into an
32 There is also two proxies that look like Builders:
36 This proxies for a Builder with an action that is actually a
37 dictionary that knows how to map file suffixes to a specific
38 action. This is so that we can invoke different actions
39 (compilers, compile options) for different flavors of source
44 This proxies for a Builder *invocation* where the target
45 is a list of files, not a single file.
47 Builders and their proxies have the following public interface methods
48 used by other modules:
51 THE public interface. Calling a Builder object (with the
52 use of internal helper methods) sets up the target and source
53 dependencies, appropriate mapping to a specific action, and the
54 environment manipulation necessary for overridden construction
55 variable. This also takes care of warning about possible mistakes
59 Returns the list of targets for a specific builder instance.
62 Adds an emitter for a specific file suffix, used by some Tool
63 modules to specify that (for example) a yacc invocation on a .y
64 can create a .h *and* a .c file.
67 Adds an action for a specific file suffix, heavily used by
68 Tool modules to add their specific action(s) for turning
69 a source file into an object file to the global static
70 and shared object file Builders.
72 There are the following methods for internal use within this module:
75 The internal method that handles the heavily lifting when a
76 Builder is called. This is used so that the __call__() methods
77 can set up warning about possible mistakes in keyword-argument
78 overrides, and *then* execute all of the steps necessary so that
79 the warnings only occur once.
82 Returns the Builder's name within a specific Environment,
83 primarily used to try to return helpful information in error
91 Miscellaneous stuff for handling the prefix and suffix
92 manipulation we use in turning source file names into target
100 # Permission is hereby granted, free of charge, to any person obtaining
101 # a copy of this software and associated documentation files (the
102 # "Software"), to deal in the Software without restriction, including
103 # without limitation the rights to use, copy, modify, merge, publish,
104 # distribute, sublicense, and/or sell copies of the Software, and to
105 # permit persons to whom the Software is furnished to do so, subject to
106 # the following conditions:
108 # The above copyright notice and this permission notice shall be included
109 # in all copies or substantial portions of the Software.
111 # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY
112 # KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
113 # WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
114 # NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
115 # LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
116 # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
117 # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
120 __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__"
126 from SCons.Debug import logInstanceCreation
127 from SCons.Errors import InternalError, UserError
128 import SCons.Executor
131 import SCons.Warnings
138 class DictCmdGenerator(SCons.Util.Selector):
139 """This is a callable class that can be used as a
140 command generator function. It holds on to a dictionary
141 mapping file suffixes to Actions. It uses that dictionary
142 to return the proper action based on the file suffix of
145 def src_suffixes(self):
148 def add_action(self, suffix, action):
149 """Add a suffix-action pair to the mapping.
151 self[suffix] = action
153 def __call__(self, target, source, env, for_signature):
155 for src in map(str, source):
156 my_ext = SCons.Util.splitext(src)[1]
157 if ext and my_ext != ext:
158 raise UserError("While building `%s' from `%s': Cannot build multiple sources with different extensions: %s, %s" % (repr(map(str, target)), src, ext, my_ext))
162 raise UserError("While building `%s': Cannot deduce file extension from source files: %s" % (repr(map(str, target)), repr(map(str, source))))
165 ret = SCons.Util.Selector.__call__(self, env, source)
167 raise UserError("Ambiguous suffixes after environment substitution: %s == %s == %s" % (e[0], e[1], e[2]))
169 raise UserError("While building `%s': Don't know how to build a file with suffix `%s'." % (repr(map(str, target)), ext))
172 class CallableSelector(SCons.Util.Selector):
173 """A callable dictionary that will, in turn, call the value it
175 def __call__(self, env, source):
176 value = SCons.Util.Selector.__call__(self, env, source)
178 value = value(env, source)
181 class DictEmitter(SCons.Util.Selector):
182 """A callable dictionary that maps file suffixes to emitters.
183 When called, it finds the right emitter in its dictionary for the
184 suffix of the first source file, and calls that emitter to get the
185 right lists of targets and sources to return. If there's no emitter
186 for the suffix in its dictionary, the original target and source are
189 def __call__(self, target, source, env):
190 emitter = SCons.Util.Selector.__call__(self, env, source)
192 target, source = emitter(target, source, env)
193 return (target, source)
195 class ListEmitter(UserList.UserList):
196 """A callable list of emitters that calls each in sequence,
197 returning the result.
199 def __call__(self, target, source, env):
201 target, source = e(target, source, env)
202 return (target, source)
204 # These are a common errors when calling a Builder;
205 # they are similar to the 'target' and 'source' keyword args to builders,
206 # so we issue warnings when we see them. The warnings can, of course,
208 misleading_keywords = {
209 'targets' : 'target',
210 'sources' : 'source',
213 class OverrideWarner(UserDict.UserDict):
214 """A class for warning about keyword arguments that we use as
215 overrides in a Builder call.
217 This class exists to handle the fact that a single MultiStepBuilder
218 call can actually invoke multiple builders as a result of a single
219 user-level Builder call. This class only emits the warnings once,
220 no matter how many Builders are invoked.
222 def __init__(self, dict):
223 UserDict.UserDict.__init__(self, dict)
224 self.already_warned = None
226 if self.already_warned:
228 for k in self.keys():
230 alt = misleading_keywords[k]
234 SCons.Warnings.warn(SCons.Warnings.MisleadingKeywordsWarning,
235 "Did you mean to use `%s' instead of `%s'?" % (alt, k))
236 self.already_warned = 1
239 """A factory for builder objects."""
241 if kw.has_key('generator'):
242 if kw.has_key('action'):
243 raise UserError, "You must not specify both an action and a generator."
244 kw['action'] = SCons.Action.CommandGenerator(kw['generator'])
246 elif kw.has_key('action') and SCons.Util.is_Dict(kw['action']):
247 composite = DictCmdGenerator(kw['action'])
248 kw['action'] = SCons.Action.CommandGenerator(composite)
249 kw['src_suffix'] = composite.src_suffixes()
251 if kw.has_key('emitter'):
252 emitter = kw['emitter']
253 if SCons.Util.is_String(emitter):
254 # This allows users to pass in an Environment
255 # variable reference (like "$FOO") as an emitter.
256 # We will look in that Environment variable for
257 # a callable to use as the actual emitter.
258 var = SCons.Util.get_environment_var(emitter)
260 raise UserError, "Supplied emitter '%s' does not appear to refer to an Environment variable" % emitter
261 kw['emitter'] = EmitterProxy(var)
262 elif SCons.Util.is_Dict(emitter):
263 kw['emitter'] = DictEmitter(emitter)
264 elif SCons.Util.is_List(emitter):
265 kw['emitter'] = ListEmitter(emitter)
267 if kw.has_key('src_builder'):
268 ret = apply(MultiStepBuilder, (), kw)
270 ret = apply(BuilderBase, (), kw)
272 if not composite is None:
273 ret = CompositeBuilder(ret, composite)
277 def _init_nodes(builder, env, overrides, tlist, slist):
278 """Initialize lists of target and source nodes with all of
279 the proper Builder information.
282 # First, figure out if there are any errors in the way the targets
286 raise UserError, "Multiple ways to build the same target were specified for: %s" % str(t)
288 if not t.env is None and not t.env is env:
289 t_contents = t.builder.action.get_contents(tlist, slist, t.env)
290 contents = t.builder.action.get_contents(tlist, slist, env)
292 if t_contents == contents:
293 SCons.Warnings.warn(SCons.Warnings.DuplicateEnvironmentWarning,
294 "Two different environments were specified for target %s,\n\tbut they appear to have the same action: %s"%(str(t), t.builder.action.strfunction(tlist, slist, t.env)))
297 raise UserError, "Two environments with different actions were specified for the same target: %s"%str(t)
299 elif t.overrides != overrides:
300 raise UserError, "Two different sets of overrides were specified for the same target: %s"%str(t)
302 elif builder.target_scanner and t.target_scanner and builder.target_scanner != t.target_scanner:
303 raise UserError, "Two different scanners were specified for the same target: %s"%str(t)
306 if t.builder != builder:
307 if isinstance(t.builder, ListBuilder) and isinstance(builder, ListBuilder) and t.builder.builder == builder.builder:
308 raise UserError, "Two different target sets have a target in common: %s"%str(t)
310 raise UserError, "Two different builders (%s and %s) were specified for the same target: %s"%(t.builder.get_name(env), builder.get_name(env), str(t))
311 elif isinstance(t.builder, ListBuilder) ^ isinstance(builder, ListBuilder):
312 raise UserError, "Cannot build same target `%s' as singular and list"%str(t)
313 elif t.sources != slist:
314 raise UserError, "Multiple ways to build the same target were specified for: %s (from %s and from %s)" % (str(t), map(str,t.sources), map(str,slist))
316 if builder.single_source:
318 raise UserError, "More than one source given for single-source builder: targets=%s sources=%s" % (map(str,tlist), map(str,slist))
320 # The targets are fine, so find or make the appropriate Executor to
321 # build this particular list of targets from this particular list of
326 executor = tlist[0].get_executor(create = 0)
327 except AttributeError:
330 executor.add_sources(slist)
332 executor = SCons.Executor.Executor(builder.action,
334 [builder.overrides, overrides],
338 # Now set up the relevant information in the target Nodes themselves.
340 t.overrides = overrides
341 t.cwd = SCons.Node.FS.default_fs.getcwd()
342 t.builder_set(builder)
345 t.set_executor(executor)
346 if builder.target_scanner:
347 t.target_scanner = builder.target_scanner
348 if t.source_scanner is None:
349 t.source_scanner = builder.source_scanner
351 # Add backup source scanners from the environment to the source
352 # nodes. This may not be necessary if the node will have a real
353 # source scanner added later (which is why these are the "backup"
354 # source scanners, not the real ones), but because source nodes may
355 # be used multiple times for different targets, it ends up being
356 # more efficient to do this calculation once here, as opposed to
357 # delaying it until later when we potentially have to calculate it
358 # over and over and over.
360 if s.source_scanner is None and s.backup_source_scanner is None:
361 s.backup_source_scanner = env.get_scanner(s.scanner_key())
364 """This is a callable class that can act as a
365 Builder emitter. It holds on to a string that
366 is a key into an Environment dictionary, and will
367 look there at actual build time to see if it holds
368 a callable. If so, we will call that as the actual
370 def __init__(self, var):
371 self.var = SCons.Util.to_String(var)
373 def __call__(self, target, source, env):
376 # Recursively substitute the variable.
377 # We can't use env.subst() because it deals only
378 # in strings. Maybe we should change that?
379 while SCons.Util.is_String(emitter) and env.has_key(emitter):
380 emitter = env[emitter]
381 if callable(emitter):
382 target, source = emitter(target, source, env)
383 elif SCons.Util.is_List(emitter):
385 target, source = e(target, source, env)
387 return (target, source)
390 def __cmp__(self, other):
391 return cmp(self.var, other.var)
394 """Base class for Builders, objects that create output
395 nodes (files) from input nodes (files).
398 def __init__(self, action = None,
402 target_factory = SCons.Node.FS.default_fs.File,
403 source_factory = SCons.Node.FS.default_fs.File,
404 target_scanner = None,
405 source_scanner = None,
411 if __debug__: logInstanceCreation(self, 'BuilderBase')
412 self.action = SCons.Action.Action(action)
414 if SCons.Util.is_Dict(prefix):
415 prefix = CallableSelector(prefix)
417 if SCons.Util.is_Dict(suffix):
418 suffix = CallableSelector(suffix)
421 self.single_source = single_source
422 if overrides.has_key('overrides'):
423 SCons.Warnings.warn(SCons.Warnings.DeprecatedWarning,
424 "The \"overrides\" keyword to Builder() creation has been deprecated;\n" +\
425 "\tspecify the items as keyword arguments to the Builder() call instead.")
426 overrides.update(overrides['overrides'])
427 del overrides['overrides']
428 self.overrides = overrides
430 self.set_src_suffix(src_suffix)
432 self.target_factory = target_factory
433 self.source_factory = source_factory
434 self.target_scanner = target_scanner
435 self.source_scanner = source_scanner
437 self.emitter = emitter
439 def __nonzero__(self):
440 raise InternalError, "Do not test for the Node.builder attribute directly; use Node.has_builder() instead"
442 def get_name(self, env):
443 """Attempts to get the name of the Builder.
445 Look at the BUILDERS variable of env, expecting it to be a
446 dictionary containing this Builder, and return the key of the
450 index = env['BUILDERS'].values().index(self)
451 return env['BUILDERS'].keys()[index]
452 except (AttributeError, KeyError, ValueError):
453 return str(self.__class__)
455 def __cmp__(self, other):
456 return cmp(self.__dict__, other.__dict__)
458 def splitext(self, path):
459 return SCons.Util.splitext(path)
461 def _create_nodes(self, env, overwarn, target = None, source = None):
462 """Create and return lists of target and source nodes.
464 def _adjustixes(files, pre, suf):
468 if not SCons.Util.is_List(files):
472 if SCons.Util.is_String(f):
473 f = SCons.Util.adjustixes(f, pre, suf)
479 env = env.Override(overwarn.data)
481 src_suf = self.get_src_suffix(env)
483 source = _adjustixes(source, None, src_suf)
484 slist = env.arg2nodes(source, self.source_factory)
486 pre = self.get_prefix(env, slist)
487 suf = self.get_suffix(env, slist)
491 t_from_s = slist[0].target_from_source
492 except AttributeError:
493 raise UserError("Do not know how to create a target from source `%s'" % slist[0])
494 tlist = [ t_from_s(pre, suf, self.splitext) ]
496 target = _adjustixes(target, pre, suf)
497 tlist = env.arg2nodes(target, self.target_factory)
500 # The emitter is going to do str(node), but because we're
501 # being called *from* a builder invocation, the new targets
502 # don't yet have a builder set on them and will look like
503 # source files. Fool the emitter's str() calls by setting
504 # up a temporary builder on the new targets.
507 if not t.is_derived():
509 new_targets.append(t)
511 target, source = self.emitter(target=tlist, source=slist, env=env)
513 # Now delete the temporary builders that we attached to any
514 # new targets, so that _init_nodes() doesn't do weird stuff
515 # to them because it thinks they already have builders.
516 for t in new_targets:
517 if t.builder is self:
518 # Only delete the temporary builder if the emitter
519 # didn't change it on us.
522 # Have to call arg2nodes yet again, since it is legal for
523 # emitters to spit out strings as well as Node instances.
524 slist = env.arg2nodes(source, self.source_factory)
525 tlist = env.arg2nodes(target, self.target_factory)
529 def _execute(self, env, target = None, source = _null, overwarn={}):
534 if(self.single_source and
535 SCons.Util.is_List(source) and
539 if target is None: target = [None]*len(source)
540 for k in range(len(source)):
541 t = self._execute(env, target[k], source[k], overwarn)
542 if SCons.Util.is_List(t):
548 tlist, slist = self._create_nodes(env, overwarn, target, source)
553 builder = ListBuilder(self, env, tlist)
554 _init_nodes(builder, env, overwarn.data, tlist, slist)
558 def __call__(self, env, target = None, source = _null, **kw):
559 return self._execute(env, target, source, OverrideWarner(kw))
561 def adjust_suffix(self, suff):
562 if suff and not suff[0] in [ '.', '_', '$' ]:
566 def get_prefix(self, env, sources=[]):
569 prefix = prefix(env, sources)
570 return env.subst(prefix)
572 def get_suffix(self, env, sources=[]):
575 suffix = suffix(env, sources)
577 suffix = self.adjust_suffix(suffix)
578 return env.subst(suffix)
580 def src_suffixes(self, env):
581 return map(lambda x, s=self, e=env: e.subst(s.adjust_suffix(x)),
584 def set_src_suffix(self, src_suffix):
587 elif not SCons.Util.is_List(src_suffix):
588 src_suffix = [ src_suffix ]
589 self.src_suffix = src_suffix
591 def get_src_suffix(self, env):
592 """Get the first src_suffix in the list of src_suffixes."""
593 ret = self.src_suffixes(env)
598 def targets(self, node):
599 """Return the list of targets for this builder instance.
601 For most normal builders, this is just the supplied node.
605 def add_emitter(self, suffix, emitter):
606 """Add a suffix-emitter mapping to this Builder.
608 This assumes that emitter has been initialized with an
609 appropriate dictionary type, and will throw a TypeError if
610 not, so the caller is responsible for knowing that this is an
611 appropriate method to call for the Builder in question.
613 self.emitter[suffix] = emitter
615 class ListBuilder(SCons.Util.Proxy):
616 """A Proxy to support building an array of targets (for example,
617 foo.o and foo.h from foo.y) from a single Action execution.
620 def __init__(self, builder, env, tlist):
621 if __debug__: logInstanceCreation(self)
622 SCons.Util.Proxy.__init__(self, builder)
623 self.builder = builder
624 self.target_scanner = builder.target_scanner
625 self.source_scanner = builder.source_scanner
628 self.multi = builder.multi
629 self.single_source = builder.single_source
631 def targets(self, node):
632 """Return the list of targets for this builder instance.
636 def get_name(self, env):
637 """Attempts to get the name of the Builder."""
639 return "ListBuilder(%s)" % self.builder.get_name(env)
641 class MultiStepBuilder(BuilderBase):
642 """This is a builder subclass that can build targets in
643 multiple steps. The src_builder parameter to the constructor
644 accepts a builder that is called to build sources supplied to
645 this builder. The targets of that first build then become
646 the sources of this builder.
648 If this builder has a src_suffix supplied, then the src_builder
649 builder is NOT invoked if the suffix of a source file matches
652 def __init__(self, src_builder,
657 target_factory = SCons.Node.FS.default_fs.File,
658 source_factory = SCons.Node.FS.default_fs.File,
659 target_scanner = None,
660 source_scanner = None,
663 if __debug__: logInstanceCreation(self)
664 BuilderBase.__init__(self, action, prefix, suffix, src_suffix,
665 target_factory, source_factory,
666 target_scanner, source_scanner, emitter,
667 single_source = single_source)
668 if not SCons.Util.is_List(src_builder):
669 src_builder = [ src_builder ]
670 self.src_builder = src_builder
672 self.cached_src_suffixes = {} # source suffixes keyed on id(env)
674 def _execute(self, env, target = None, source = _null, overwarn={}):
679 slist = env.arg2nodes(source, self.source_factory)
683 sdict = self.sdict[id(env)]
686 self.sdict[id(env)] = sdict
687 for bld in self.src_builder:
688 if SCons.Util.is_String(bld):
690 bld = env['BUILDERS'][bld]
693 for suf in bld.src_suffixes(env):
696 src_suffixes = self.src_suffixes(env)
700 get_suffix = snode.get_suffix
701 except AttributeError:
702 ext = self.splitext(str(snode))
706 subsidiary_builder = sdict[ext]
708 final_sources.append(snode)
710 tgt = subsidiary_builder._execute(env, None, snode, overwarn)
711 # If the subsidiary Builder returned more than one target,
712 # then filter out any sources that this Builder isn't
713 # capable of building.
715 tgt = filter(lambda x, self=self, suf=src_suffixes:
716 self.splitext(SCons.Util.to_String(x))[1] in suf,
718 final_sources.extend(tgt)
720 return BuilderBase._execute(self, env, target, final_sources, overwarn)
722 def get_src_builders(self, env):
723 """Return all the src_builders for this Builder.
725 This is essentially a recursive descent of the src_builder "tree."
728 for bld in self.src_builder:
729 if SCons.Util.is_String(bld):
730 # All Environments should have a BUILDERS
731 # variable, so no need to check for it.
733 bld = env['BUILDERS'][bld]
739 def src_suffixes(self, env):
740 """Return a list of the src_suffix attributes for all
741 src_builders of this Builder.
744 return self.cached_src_suffixes[id(env)]
746 suffixes = BuilderBase.src_suffixes(self, env)
747 for builder in self.get_src_builders(env):
748 suffixes.extend(builder.src_suffixes(env))
749 self.cached_src_suffixes[id(env)] = suffixes
752 class CompositeBuilder(SCons.Util.Proxy):
753 """A Builder Proxy whose main purpose is to always have
754 a DictCmdGenerator as its action, and to provide access
755 to the DictCmdGenerator's add_action() method.
758 def __init__(self, builder, cmdgen):
759 if __debug__: logInstanceCreation(self)
760 SCons.Util.Proxy.__init__(self, builder)
762 # cmdgen should always be an instance of DictCmdGenerator.
764 self.builder = builder
766 def add_action(self, suffix, action):
767 self.cmdgen.add_action(suffix, action)
768 self.set_src_suffix(self.cmdgen.src_suffixes())