5 Permission is hereby granted, free of charge, to any person obtaining
6 a copy of this software and associated documentation files (the
7 "Software"), to deal in the Software without restriction, including
8 without limitation the rights to use, copy, modify, merge, publish,
9 distribute, sublicense, and/or sell copies of the Software, and to
10 permit persons to whom the Software is furnished to do so, subject to
11 the following conditions:
13 The above copyright notice and this permission notice shall be included
14 in all copies or substantial portions of the Software.
16 THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY
17 KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
18 WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
19 NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
20 LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
21 OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
22 WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
28 On multi-developer software projects,
29 you can sometimes speed up every developer's builds a lot by
30 allowing them to share the derived files that they build.
31 &SCons; makes this easy, as well as reliable.
36 <title>Specifying the Shared Cache Directory</title>
40 To enable sharing of derived files,
41 use the &CacheDir; function
42 in any &SConscript; file:
47 CacheDir('/usr/local/build_cache')
52 Note that the directory you specify must already exist
53 and be readable and writable by all developers
54 who will be sharing derived files.
55 It should also be in some central location
56 that all builds will be able to access.
57 In environments where developers are using separate systems
58 (like individual workstations) for builds,
59 this directory would typically be
60 on a shared or NFS-mounted file system.
67 When a build has a &CacheDir; specified,
68 every time a file is built,
69 it is stored in the shared cache directory
70 along with its MD5 build signature.
73 Actually, the MD5 signature is used as the name of the file
74 in the shared cache directory in which the contents are stored.
78 before an action is invoked to build a file,
79 &SCons; will check the shared cache directory
80 to see if a file with the exact same build
81 signature already exists.
82 If so, the derived file will not be built locally,
83 but will be copied into the local build directory
84 from the shared cache directory,
90 % <userinput>scons -Q</userinput>
91 cc -o hello.o -c hello.c
93 % <userinput>scons -Q -c</userinput>
96 % <userinput>scons -Q</userinput>
97 Retrieved `hello.o' from cache
98 Retrieved `hello' from cache
103 Note that the &CacheDir; feature still calculates
104 MD5 build sigantures for the shared cache file names
105 even if you configure &SCons; to use timestamps
106 to decide if files are up to date.
107 (See the <xref linkend="chap-depends"></xref>
108 chapter for information about the &Decider; function.)
109 Consequently, using &CacheDir; may reduce or eliminate any
110 potential performance improvements
111 from using timestamps for up-to-date decisions.
118 <title>Keeping Build Output Consistent</title>
122 One potential drawback to using a shared cache
123 is that the output printed by &SCons;
124 can be inconsistent from invocation to invocation,
125 because any given file may be rebuilt one time
126 and retrieved from the shared cache the next time.
127 This can make analyzing build output more difficult,
128 especially for automated scripts that
129 expect consistent output each time.
135 If, however, you use the <literal>--cache-show</literal> option,
136 &SCons; will print the command line that it
137 <emphasis>would</emphasis> have executed
139 even when it is retrieving the file from the shared cache.
140 This makes the build output consistent
141 every time the build is run:
146 % <userinput>scons -Q</userinput>
147 cc -o hello.o -c hello.c
149 % <userinput>scons -Q -c</userinput>
152 % <userinput>scons -Q --cache-show</userinput>
153 cc -o hello.o -c hello.c
159 The trade-off, of course, is that you no longer
160 know whether or not &SCons;
161 has retrieved a derived file from cache
162 or has rebuilt it locally.
169 <title>Not Using the Shared Cache for Specific Files</title>
173 You may want to disable caching for certain
174 specific files in your configuration.
175 For example, if you only want to put
176 executable files in a central cache,
177 but not the intermediate object files,
178 you can use the &NoCache;
179 function to specify that the
180 object files should not be cached:
186 obj = env.Object('hello.c')
187 env.Program('hello.c')
194 Then when you run &scons; after cleaning
196 it will recompile the object file locally
197 (since it doesn't exist in the shared cache directory),
198 but still realize that the shared cache directory
199 contains an up-to-date executable program
200 that can be retrieved instead of re-linking:
206 <scons_output example="ex1">
207 <scons_output_command>scons -Q</scons_output_command>
208 <scons_output_command>scons -Q -c</scons_output_command>
209 <scons_output_command>scons -Q</scons_output_command>
215 % <userinput>scons -Q</userinput>
216 cc -o hello.o -c hello.c
218 % <userinput>scons -Q -c</userinput>
221 % <userinput>scons -Q</userinput>
222 cc -o hello.o -c hello.c
223 Retrieved `hello' from cache
229 <title>Disabling the Shared Cache</title>
233 Retrieving an already-built file
234 from the shared cache
235 is usually a significant time-savings
236 over rebuilding the file,
237 but how much of a savings
238 (or even whether it saves time at all)
239 can depend a great deal on your
240 system or network configuration.
241 For example, retrieving cached files
242 from a busy server over a busy network
243 might end up being slower than
244 rebuilding the files locally.
250 In these cases, you can specify
251 the <literal>--cache-disable</literal>
252 command-line option to tell &SCons;
253 to not retrieve already-built files from the
254 shared cache directory:
259 % <userinput>scons -Q</userinput>
260 cc -o hello.o -c hello.c
262 % <userinput>scons -Q -c</userinput>
265 % <userinput>scons -Q</userinput>
266 Retrieved `hello.o' from cache
267 Retrieved `hello' from cache
268 % <userinput>scons -Q -c</userinput>
271 % <userinput>scons -Q --cache-disable</userinput>
272 cc -o hello.o -c hello.c
279 <title>Populating a Shared Cache With Already-Built Files</title>
283 Sometimes, you may have one or more derived files
284 already built in your local build tree
285 that you wish to make available to other people doing builds.
286 For example, you may find it more effective to perform
287 integration builds with the cache disabled
288 (per the previous section)
289 and only populate the shared cache directory
290 with the built files after the integration build
291 has completed successfully.
292 This way, the cache will only get filled up
293 with derived files that are part of a complete, successful build
294 not with files that might be later overwritten
295 while you debug integration problems.
301 In this case, you can use the
302 the <literal>--cache-force</literal> option
303 to tell &SCons; to put all derived files in the cache,
304 even if the files already exist in your local tree
305 from having been built by a previous invocation:
310 % <userinput>scons -Q --cache-disable</userinput>
311 cc -o hello.o -c hello.c
313 % <userinput>scons -Q -c</userinput>
316 % <userinput>scons -Q --cache-disable</userinput>
317 cc -o hello.o -c hello.c
319 % <userinput>scons -Q --cache-force</userinput>
320 scons: `.' is up to date.
321 % <userinput>scons -Q</userinput>
322 scons: `.' is up to date.
327 Notice how the above sample run
328 demonstrates that the <literal>--cache-disable</literal>
329 option avoids putting the built
330 <filename>hello.o</filename>
332 <filename>hello</filename> files in the cache,
333 but after using the <literal>--cache-force</literal> option,
334 the files have been put in the cache
335 for the next invocation to retrieve.
342 <title>Minimizing Cache Contention: the <literal>--random</literal> Option</title>
346 If you allow multiple builds to update the
347 shared cache directory simultaneously,
348 two builds that occur at the same time
349 can sometimes start "racing"
350 with one another to build the same files
353 you are linking multiple files into an executable program:
359 ['f1.c', 'f2.c', 'f3.c', 'f4.c', 'f5.c'])
364 &SCons; will normally build the input object files
365 on which the program depends in their normal, sorted order:
370 % <userinput>scons -Q</userinput>
376 cc -o prog f1.o f2.o f3.o f4.o f5.o
381 But if two such builds take place simultaneously,
382 they may each look in the cache at nearly the same
383 time and both decide that <filename>f1.o</filename>
384 must be rebuilt and pushed into the shared cache directory,
385 then both decide that <filename>f2.o</filename>
386 must be rebuilt (and pushed into the shared cache directory),
387 then both decide that <filename>f3.o</filename>
389 This won't cause any actual build problems--both
391 generate correct output files,
392 and populate the cache--but
393 it does represent wasted effort.
399 To alleviate such contention for the cache,
400 you can use the <literal>--random</literal> command-line option
401 to tell &SCons; to build dependencies
408 The following <screen> output was generated by this:
410 <scons_output example="ex-random">
411 <scons_output_command>scons -Q - -random</scons_output_command>
414 We captured it directly here to guarantee a "random" order,
415 guarding against the potential for - -random to happen
416 to return things in the original sorted order.
421 % <userinput>scons -Q --random</userinput>
427 cc -o prog f1.o f2.o f3.o f4.o f5.o
432 Multiple builds using the <literal>--random</literal> option
433 will usually build their dependencies in different,
435 which minimizes the chances for a lot of
436 contention for same-named files
437 in the shared cache directory.
438 Multiple simultaneous builds might still race to try to build
439 the same target file on occasion,
440 but long sequences of inefficient contention
448 the <literal>--random</literal> option
449 will cause the output that &SCons; prints
450 to be inconsistent from invocation to invocation,
451 which may be an issue when
452 trying to compare output from different build runs.
458 If you want to make sure dependencies will be built
459 in a random order without having to specify
460 the <literal>--random</literal> on very command line,
461 you can use the &SetOption; function to
462 set the <literal>random</literal> option
463 within any &SConscript; file:
469 ['f1.c', 'f2.c', 'f3.c', 'f4.c', 'f5.c'])
471 SetOption('random', 1)
473 ['f1.c', 'f2.c', 'f3.c', 'f4.c', 'f5.c'])
481 <title>Troubleshooting Shared Caching: the &cache-debug; Option</title>
485 XXX describe the - - cache-debug option
486 XXX maybe point to the troubleshooting appendix?
500 XXX describe CacheDir management: monitoring, deleting, etc.