3 Builder object subsystem.
5 A Builder object is a callable that encapsulates information about how
6 to execute actions to create a target Node (file) from source Nodes
7 (files), and how to create those dependencies for tracking.
9 The main entry point here is the Builder() factory method. This provides
10 a procedural interface that creates the right underlying Builder object
11 based on the keyword arguments supplied and the types of the arguments.
13 The goal is for this external interface to be simple enough that the
14 vast majority of users can create new Builders as necessary to support
15 building new types of files in their configurations, without having to
16 dive any deeper into this subsystem.
18 The base class here is BuilderBase. This is a concrete base class which
19 does, in fact, represent most Builder objects that we (or users) create.
21 There is (at present) one subclasses:
25 This is a Builder that knows how to "chain" Builders so that
26 users can specify a source file that requires multiple steps
27 to turn into a target file. A canonical example is building a
28 program from yacc input file, which requires invoking a builder
29 to turn the .y into a .c, the .c into a .o, and the .o into an
32 There is also two proxies that look like Builders:
36 This proxies for a Builder with an action that is actually a
37 dictionary that knows how to map file suffixes to a specific
38 action. This is so that we can invoke different actions
39 (compilers, compile options) for different flavors of source
44 This proxies for a Builder *invocation* where the target
45 is a list of files, not a single file.
47 Builders and their proxies have the following public interface methods
48 used by other modules:
51 THE public interface. Calling a Builder object (with the
52 use of internal helper methods) sets up the target and source
53 dependencies, appropriate mapping to a specific action, and the
54 environment manipulation necessary for overridden construction
55 variable. This also takes care of warning about possible mistakes
59 Returns the list of targets for a specific builder instance.
62 Adds an emitter for a specific file suffix, used by some Tool
63 modules to specify that (for example) a yacc invocation on a .y
64 can create a .h *and* a .c file.
67 Adds an action for a specific file suffix, heavily used by
68 Tool modules to add their specific action(s) for turning
69 a source file into an object file to the global static
70 and shared object file Builders.
72 There are the following methods for internal use within this module:
75 The internal method that handles the heavily lifting when a
76 Builder is called. This is used so that the __call__() methods
77 can set up warning about possible mistakes in keyword-argument
78 overrides, and *then* execute all of the steps necessary so that
79 the warnings only occur once.
82 Returns the Builder's name within a specific Environment,
83 primarily used to try to return helpful information in error
91 Miscellaneous stuff for handling the prefix and suffix
92 manipulation we use in turning source file names into target
100 # Permission is hereby granted, free of charge, to any person obtaining
101 # a copy of this software and associated documentation files (the
102 # "Software"), to deal in the Software without restriction, including
103 # without limitation the rights to use, copy, modify, merge, publish,
104 # distribute, sublicense, and/or sell copies of the Software, and to
105 # permit persons to whom the Software is furnished to do so, subject to
106 # the following conditions:
108 # The above copyright notice and this permission notice shall be included
109 # in all copies or substantial portions of the Software.
111 # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY
112 # KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
113 # WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
114 # NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
115 # LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
116 # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
117 # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
120 __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__"
126 from SCons.Debug import logInstanceCreation
127 from SCons.Errors import InternalError, UserError
128 import SCons.Executor
131 import SCons.Warnings
138 class DictCmdGenerator(SCons.Util.Selector):
139 """This is a callable class that can be used as a
140 command generator function. It holds on to a dictionary
141 mapping file suffixes to Actions. It uses that dictionary
142 to return the proper action based on the file suffix of
145 def src_suffixes(self):
148 def add_action(self, suffix, action):
149 """Add a suffix-action pair to the mapping.
151 self[suffix] = action
153 def __call__(self, target, source, env, for_signature):
155 for src in map(str, source):
156 my_ext = SCons.Util.splitext(src)[1]
157 if ext and my_ext != ext:
158 raise UserError("While building `%s' from `%s': Cannot build multiple sources with different extensions: %s, %s" % (repr(map(str, target)), src, ext, my_ext))
162 raise UserError("While building `%s': Cannot deduce file extension from source files: %s" % (repr(map(str, target)), repr(map(str, source))))
165 ret = SCons.Util.Selector.__call__(self, env, source)
167 raise UserError("Ambiguous suffixes after environment substitution: %s == %s == %s" % (e[0], e[1], e[2]))
169 raise UserError("While building `%s': Don't know how to build a file with suffix `%s'." % (repr(map(str, target)), ext))
172 class CallableSelector(SCons.Util.Selector):
173 """A callable dictionary that will, in turn, call the value it
175 def __call__(self, env, source):
176 value = SCons.Util.Selector.__call__(self, env, source)
178 value = value(env, source)
181 class DictEmitter(SCons.Util.Selector):
182 """A callable dictionary that maps file suffixes to emitters.
183 When called, it finds the right emitter in its dictionary for the
184 suffix of the first source file, and calls that emitter to get the
185 right lists of targets and sources to return. If there's no emitter
186 for the suffix in its dictionary, the original target and source are
189 def __call__(self, target, source, env):
190 emitter = SCons.Util.Selector.__call__(self, env, source)
192 target, source = emitter(target, source, env)
193 return (target, source)
195 class ListEmitter(UserList.UserList):
196 """A callable list of emitters that calls each in sequence,
197 returning the result.
199 def __call__(self, target, source, env):
201 target, source = e(target, source, env)
202 return (target, source)
204 # These are a common errors when calling a Builder;
205 # they are similar to the 'target' and 'source' keyword args to builders,
206 # so we issue warnings when we see them. The warnings can, of course,
208 misleading_keywords = {
209 'targets' : 'target',
210 'sources' : 'source',
213 class OverrideWarner(UserDict.UserDict):
214 """A class for warning about keyword arguments that we use as
215 overrides in a Builder call.
217 This class exists to handle the fact that a single MultiStepBuilder
218 call can actually invoke multiple builders as a result of a single
219 user-level Builder call. This class only emits the warnings once,
220 no matter how many Builders are invoked.
222 def __init__(self, dict):
223 UserDict.UserDict.__init__(self, dict)
224 self.already_warned = None
226 if self.already_warned:
228 for k in self.keys():
230 alt = misleading_keywords[k]
234 SCons.Warnings.warn(SCons.Warnings.MisleadingKeywordsWarning,
235 "Did you mean to use `%s' instead of `%s'?" % (alt, k))
236 self.already_warned = 1
239 """A factory for builder objects."""
241 if kw.has_key('generator'):
242 if kw.has_key('action'):
243 raise UserError, "You must not specify both an action and a generator."
244 kw['action'] = SCons.Action.CommandGenerator(kw['generator'])
246 elif kw.has_key('action') and SCons.Util.is_Dict(kw['action']):
247 composite = DictCmdGenerator(kw['action'])
248 kw['action'] = SCons.Action.CommandGenerator(composite)
249 kw['src_suffix'] = composite.src_suffixes()
251 if kw.has_key('emitter'):
252 emitter = kw['emitter']
253 if SCons.Util.is_String(emitter):
254 # This allows users to pass in an Environment
255 # variable reference (like "$FOO") as an emitter.
256 # We will look in that Environment variable for
257 # a callable to use as the actual emitter.
258 var = SCons.Util.get_environment_var(emitter)
260 raise UserError, "Supplied emitter '%s' does not appear to refer to an Environment variable" % emitter
261 kw['emitter'] = EmitterProxy(var)
262 elif SCons.Util.is_Dict(emitter):
263 kw['emitter'] = DictEmitter(emitter)
264 elif SCons.Util.is_List(emitter):
265 kw['emitter'] = ListEmitter(emitter)
267 if kw.has_key('src_builder'):
268 ret = apply(MultiStepBuilder, (), kw)
270 ret = apply(BuilderBase, (), kw)
272 if not composite is None:
273 ret = CompositeBuilder(ret, composite)
277 def _init_nodes(builder, env, overrides, executor_kw, tlist, slist):
278 """Initialize lists of target and source nodes with all of
279 the proper Builder information.
282 # First, figure out if there are any errors in the way the targets
286 raise UserError, "Multiple ways to build the same target were specified for: %s" % str(t)
288 if not t.env is None and not t.env is env:
289 t_contents = t.builder.action.get_contents(tlist, slist, t.env)
290 contents = t.builder.action.get_contents(tlist, slist, env)
292 if t_contents == contents:
293 SCons.Warnings.warn(SCons.Warnings.DuplicateEnvironmentWarning,
294 "Two different environments were specified for target %s,\n\tbut they appear to have the same action: %s"%(str(t), t.builder.action.strfunction(tlist, slist, t.env)))
297 raise UserError, "Two environments with different actions were specified for the same target: %s"%str(t)
299 elif t.overrides != overrides:
300 raise UserError, "Two different sets of overrides were specified for the same target: %s"%str(t)
303 if t.builder != builder:
304 if isinstance(t.builder, ListBuilder) and isinstance(builder, ListBuilder) and t.builder.builder == builder.builder:
305 raise UserError, "Two different target sets have a target in common: %s"%str(t)
307 raise UserError, "Two different builders (%s and %s) were specified for the same target: %s"%(t.builder.get_name(env), builder.get_name(env), str(t))
308 elif isinstance(t.builder, ListBuilder) ^ isinstance(builder, ListBuilder):
309 raise UserError, "Cannot build same target `%s' as singular and list"%str(t)
310 elif t.sources != slist:
311 raise UserError, "Multiple ways to build the same target were specified for: %s (from %s and from %s)" % (str(t), map(str,t.sources), map(str,slist))
313 if builder.single_source:
315 raise UserError, "More than one source given for single-source builder: targets=%s sources=%s" % (map(str,tlist), map(str,slist))
317 # The targets are fine, so find or make the appropriate Executor to
318 # build this particular list of targets from this particular list of
323 executor = tlist[0].get_executor(create = 0)
324 except AttributeError:
327 executor.add_sources(slist)
329 if not builder.action:
330 raise UserError, "Builder %s must have an action to build %s."%(builder.get_name(env or builder.env), map(str,tlist))
331 executor = SCons.Executor.Executor(builder.action,
333 [builder.overrides, overrides],
338 # Now set up the relevant information in the target Nodes themselves.
340 t.overrides = overrides
341 t.cwd = SCons.Node.FS.default_fs.getcwd()
342 t.builder_set(builder)
345 t.set_executor(executor)
348 """This is a callable class that can act as a
349 Builder emitter. It holds on to a string that
350 is a key into an Environment dictionary, and will
351 look there at actual build time to see if it holds
352 a callable. If so, we will call that as the actual
354 def __init__(self, var):
355 self.var = SCons.Util.to_String(var)
357 def __call__(self, target, source, env):
360 # Recursively substitute the variable.
361 # We can't use env.subst() because it deals only
362 # in strings. Maybe we should change that?
363 while SCons.Util.is_String(emitter) and env.has_key(emitter):
364 emitter = env[emitter]
365 if callable(emitter):
366 target, source = emitter(target, source, env)
367 elif SCons.Util.is_List(emitter):
369 target, source = e(target, source, env)
371 return (target, source)
374 def __cmp__(self, other):
375 return cmp(self.var, other.var)
378 """Base class for Builders, objects that create output
379 nodes (files) from input nodes (files).
382 def __init__(self, action = None,
386 target_factory = SCons.Node.FS.default_fs.File,
387 source_factory = SCons.Node.FS.default_fs.File,
388 target_scanner = None,
389 source_scanner = None,
397 if __debug__: logInstanceCreation(self, 'BuilderBase')
398 self.action = SCons.Action.Action(action)
400 if SCons.Util.is_Dict(prefix):
401 prefix = CallableSelector(prefix)
403 if SCons.Util.is_Dict(suffix):
404 suffix = CallableSelector(suffix)
407 self.single_source = single_source
408 if overrides.has_key('overrides'):
409 SCons.Warnings.warn(SCons.Warnings.DeprecatedWarning,
410 "The \"overrides\" keyword to Builder() creation has been deprecated;\n" +\
411 "\tspecify the items as keyword arguments to the Builder() call instead.")
412 overrides.update(overrides['overrides'])
413 del overrides['overrides']
414 if overrides.has_key('scanner'):
415 SCons.Warnings.warn(SCons.Warnings.DeprecatedWarning,
416 "The \"scanner\" keyword to Builder() creation has been deprecated;\n"
417 "\tuse: source_scanner or target_scanner as appropriate.")
418 del overrides['scanner']
419 self.overrides = overrides
421 self.set_src_suffix(src_suffix)
423 self.target_factory = target_factory
424 self.source_factory = source_factory
425 self.target_scanner = target_scanner
426 self.source_scanner = source_scanner
428 self.emitter = emitter
430 # Optional Builder name should only be used for Builders
431 # that don't get attached to construction environments.
434 self.executor_kw = {}
435 if not chdir is _null:
436 self.executor_kw['chdir'] = chdir
438 def __nonzero__(self):
439 raise InternalError, "Do not test for the Node.builder attribute directly; use Node.has_builder() instead"
441 def get_name(self, env):
442 """Attempts to get the name of the Builder.
444 Look at the BUILDERS variable of env, expecting it to be a
445 dictionary containing this Builder, and return the key of the
446 dictionary. If there's no key, then return a directly-configured
447 name (if there is one) or the name of the class (by default)."""
450 index = env['BUILDERS'].values().index(self)
451 return env['BUILDERS'].keys()[index]
452 except (AttributeError, KeyError, ValueError):
455 except AttributeError:
456 return str(self.__class__)
458 def __cmp__(self, other):
459 return cmp(self.__dict__, other.__dict__)
461 def splitext(self, path, env=None):
465 matchsuf = filter(lambda S,path=path: path[-len(S):] == S,
466 self.src_suffixes(env))
468 suf = max(map(None, map(len, matchsuf), matchsuf))[1]
469 return [path[:-len(suf)], path[-len(suf):]]
470 return SCons.Util.splitext(path)
472 def _create_nodes(self, env, overwarn, target = None, source = None):
473 """Create and return lists of target and source nodes.
475 def _adjustixes(files, pre, suf):
479 if not SCons.Util.is_List(files):
483 if SCons.Util.is_String(f):
484 f = SCons.Util.adjustixes(f, pre, suf)
490 env = env.Override(overwarn.data)
492 src_suf = self.get_src_suffix(env)
494 source = _adjustixes(source, None, src_suf)
495 slist = env.arg2nodes(source, self.source_factory)
497 pre = self.get_prefix(env, slist)
498 suf = self.get_suffix(env, slist)
502 t_from_s = slist[0].target_from_source
503 except AttributeError:
504 raise UserError("Do not know how to create a target from source `%s'" % slist[0])
505 splitext = lambda S,self=self,env=env: self.splitext(S,env)
506 tlist = [ t_from_s(pre, suf, splitext) ]
508 target = _adjustixes(target, pre, suf)
509 tlist = env.arg2nodes(target, self.target_factory)
512 # The emitter is going to do str(node), but because we're
513 # being called *from* a builder invocation, the new targets
514 # don't yet have a builder set on them and will look like
515 # source files. Fool the emitter's str() calls by setting
516 # up a temporary builder on the new targets.
519 if not t.is_derived():
521 new_targets.append(t)
523 target, source = self.emitter(target=tlist, source=slist, env=env)
525 # Now delete the temporary builders that we attached to any
526 # new targets, so that _init_nodes() doesn't do weird stuff
527 # to them because it thinks they already have builders.
528 for t in new_targets:
529 if t.builder is self:
530 # Only delete the temporary builder if the emitter
531 # didn't change it on us.
534 # Have to call arg2nodes yet again, since it is legal for
535 # emitters to spit out strings as well as Node instances.
536 slist = env.arg2nodes(source, self.source_factory)
537 tlist = env.arg2nodes(target, self.target_factory)
541 def _execute(self, env, target=None, source=_null, overwarn={}, executor_kw={}):
546 if(self.single_source and
547 SCons.Util.is_List(source) and
551 if target is None: target = [None]*len(source)
552 for k in range(len(source)):
553 t = self._execute(env, target[k], source[k], overwarn)
554 if SCons.Util.is_List(t):
560 tlist, slist = self._create_nodes(env, overwarn, target, source)
565 builder = ListBuilder(self, env, tlist)
566 _init_nodes(builder, env, overwarn.data, executor_kw, tlist, slist)
570 def __call__(self, env, target=None, source=_null, chdir=_null, **kw):
572 ekw = self.executor_kw
574 ekw = self.executor_kw.copy()
576 return self._execute(env, target, source, OverrideWarner(kw), ekw)
578 def adjust_suffix(self, suff):
579 if suff and not suff[0] in [ '.', '_', '$' ]:
583 def get_prefix(self, env, sources=[]):
586 prefix = prefix(env, sources)
587 return env.subst(prefix)
589 def get_suffix(self, env, sources=[]):
592 suffix = suffix(env, sources)
594 suffix = self.adjust_suffix(suffix)
595 return env.subst(suffix)
597 def src_suffixes(self, env):
598 return map(lambda x, s=self, e=env: e.subst(s.adjust_suffix(x)),
601 def set_src_suffix(self, src_suffix):
604 elif not SCons.Util.is_List(src_suffix):
605 src_suffix = [ src_suffix ]
606 self.src_suffix = src_suffix
608 def get_src_suffix(self, env):
609 """Get the first src_suffix in the list of src_suffixes."""
610 ret = self.src_suffixes(env)
615 def targets(self, node):
616 """Return the list of targets for this builder instance.
618 For most normal builders, this is just the supplied node.
622 def add_emitter(self, suffix, emitter):
623 """Add a suffix-emitter mapping to this Builder.
625 This assumes that emitter has been initialized with an
626 appropriate dictionary type, and will throw a TypeError if
627 not, so the caller is responsible for knowing that this is an
628 appropriate method to call for the Builder in question.
630 self.emitter[suffix] = emitter
632 class ListBuilder(SCons.Util.Proxy):
633 """A Proxy to support building an array of targets (for example,
634 foo.o and foo.h from foo.y) from a single Action execution.
637 def __init__(self, builder, env, tlist):
638 if __debug__: logInstanceCreation(self)
639 SCons.Util.Proxy.__init__(self, builder)
640 self.builder = builder
641 self.target_scanner = builder.target_scanner
642 self.source_scanner = builder.source_scanner
645 self.multi = builder.multi
646 self.single_source = builder.single_source
648 def targets(self, node):
649 """Return the list of targets for this builder instance.
653 def get_name(self, env):
654 """Attempts to get the name of the Builder."""
656 return "ListBuilder(%s)" % self.builder.get_name(env)
658 class MultiStepBuilder(BuilderBase):
659 """This is a builder subclass that can build targets in
660 multiple steps. The src_builder parameter to the constructor
661 accepts a builder that is called to build sources supplied to
662 this builder. The targets of that first build then become
663 the sources of this builder.
665 If this builder has a src_suffix supplied, then the src_builder
666 builder is NOT invoked if the suffix of a source file matches
669 def __init__(self, src_builder,
674 target_factory = SCons.Node.FS.default_fs.File,
675 source_factory = SCons.Node.FS.default_fs.File,
676 target_scanner = None,
677 source_scanner = None,
680 if __debug__: logInstanceCreation(self)
681 BuilderBase.__init__(self, action, prefix, suffix, src_suffix,
682 target_factory, source_factory,
683 target_scanner, source_scanner, emitter,
684 single_source = single_source)
685 if not SCons.Util.is_List(src_builder):
686 src_builder = [ src_builder ]
687 self.src_builder = src_builder
689 self.cached_src_suffixes = {} # source suffixes keyed on id(env)
691 def _execute(self, env, target = None, source = _null, overwarn={}, executor_kw={}):
696 slist = env.arg2nodes(source, self.source_factory)
700 sdict = self.sdict[id(env)]
703 self.sdict[id(env)] = sdict
704 for bld in self.src_builder:
705 if SCons.Util.is_String(bld):
707 bld = env['BUILDERS'][bld]
710 for suf in bld.src_suffixes(env):
713 src_suffixes = self.src_suffixes(env)
716 for srcsuf in src_suffixes:
717 if str(snode)[-len(srcsuf):] == srcsuf and sdict.has_key(srcsuf):
718 tgt = sdict[srcsuf]._execute(env, None, snode, overwarn)
719 # If the subsidiary Builder returned more than one target,
720 # then filter out any sources that this Builder isn't
721 # capable of building.
723 tgt = filter(lambda x, self=self, suf=src_suffixes, e=env:
724 self.splitext(SCons.Util.to_String(x),e)[1] in suf,
726 final_sources.extend(tgt)
730 final_sources.append(snode)
732 return BuilderBase._execute(self, env, target, final_sources, overwarn)
734 def get_src_builders(self, env):
735 """Return all the src_builders for this Builder.
737 This is essentially a recursive descent of the src_builder "tree."
740 for bld in self.src_builder:
741 if SCons.Util.is_String(bld):
742 # All Environments should have a BUILDERS
743 # variable, so no need to check for it.
745 bld = env['BUILDERS'][bld]
751 def src_suffixes(self, env):
752 """Return a list of the src_suffix attributes for all
753 src_builders of this Builder.
756 return self.cached_src_suffixes[id(env)]
758 suffixes = BuilderBase.src_suffixes(self, env)
759 for builder in self.get_src_builders(env):
760 suffixes.extend(builder.src_suffixes(env))
761 self.cached_src_suffixes[id(env)] = suffixes
764 class CompositeBuilder(SCons.Util.Proxy):
765 """A Builder Proxy whose main purpose is to always have
766 a DictCmdGenerator as its action, and to provide access
767 to the DictCmdGenerator's add_action() method.
770 def __init__(self, builder, cmdgen):
771 if __debug__: logInstanceCreation(self)
772 SCons.Util.Proxy.__init__(self, builder)
774 # cmdgen should always be an instance of DictCmdGenerator.
776 self.builder = builder
778 def add_action(self, suffix, action):
779 self.cmdgen.add_action(suffix, action)
780 self.set_src_suffix(self.cmdgen.src_suffixes())