Fabian Groffen [Sat, 16 Aug 2008 09:41:32 +0000 (09:41 -0000)]
Make IUSE.invalid a warning only, as in Prefix we'll keep on having this as any modifications to metadata.xml will be wiped out on the next sync.
svn path=/main/branches/prefix/; revision=11421
Fabian Groffen [Sat, 16 Aug 2008 08:14:06 +0000 (08:14 -0000)]
Merged from trunk 11409:11419
| 11410 | Convert exceptions from parse_metadata_use() into |
| zmedico | "metadata.bad" errors instead of exiting. |
| 11411 | When a build fails, generate a status message showing which |
| zmedico | package failed and the path of the relevant log file if |
| | available. Thanks to _neuron_ for the suggestion. Here is |
| | some sample output: >>> Emerging (1 of 1) foo-bar/baz-1.0 |
| | >>> Failed to emerge foo-bar/baz-1.0, Log file: >>> |
| | '/var/log/portage/foo-bar:baz-1.0:
20080814-202327.log' |
| 11412 | * Replace vartree.dep_match() call with normal dbapi.match() |
| zmedico | call. * Don't silently swallow unexpected KeyError |
| | exceptions from dbapi.match() inside unmerge(). |
| 11413 | In order to avoid an unhandled InvalidAtom exception, |
| zmedico | validate atoms from command line arguments before passing |
| | them to unmerge(). Thanks to Cardoe for reporting. |
| 11414 | Validate atoms from command line args before calling |
| zmedico | action_depclean(). |
| 11415 | Pass atom instances into action_depclean() instead of the |
| zmedico | plain strings. |
| 11416 | For py3k compatibility, replace map() call with a normal for |
| zmedico | loop. Thanks to Ali Polatel <hawking@g.o>. |
| 11417 | Use a list comprehension instead of strange map() usage |
| zmedico | which yields odd results when fed to the py3k converter. |
| | Thanks to René 'Necoro' Neumann. |
| 11418 | For consistency, make get_open_fds() return a plain iterator |
| zmedico | in any case, rather than return a list in some cases. |
| 11419 | Bug #172496 - Add a PORTAGE_COMPRESS_EXCLUDE_SUFFIXES config |
| zmedico | variable which is used to filter ecompress arguments. This |
| | allows the user to control compression of files installed |
| | dodoc, excluding files with specific suffix patterns. The |
| | current default value includes all of the suffixes that are |
| | recognized by dohtml, as well as pdf. |
svn path=/main/branches/prefix/; revision=11420
Fabian Groffen [Thu, 14 Aug 2008 19:25:26 +0000 (19:25 -0000)]
Merged from trunk 11399:11408
| 11400 | Disable "waiting for lock" messages when in background mode. |
| zmedico | Thanks to Thargor for reporting. |
| 11401 | Bug #234478 - Fix config._getMissingLicenses() to only |
| zmedico | access USE when LICENSE is conditional since otherwise USE |
| | hasn't necessarily been calculated. |
| 11402 | Create a single EAPI section to contain all of the EAPI |
| zmedico | subsections and replace sect[1-3] elements with "section" |
| | elements which have unbounded depth. |
| 11404 | Fix quoting for iteration over PROFILE_PATHS. |
| zmedico | |
| 11405 | Make Scheduler._choose_pkg() parallelize more aggressively |
| zmedico | in cases when it will not result in a less optimal build |
| | order. This works by modifying the behavior of |
| | Scheduler._dependent_on_scheduled_merges() so that two cases |
| | of dependence are ignored: * A given dependency will be |
| | merged later anyway (due to dep cycles) and therefore |
| | delaying the merge of the parent package will not result in |
| | a more optimal build order. * A given dependency is an |
| | "uninstall" operation and it also is not a direct |
| | dependency, therefore it should be irrelevant to build order |
| | optimization. |
| 11406 | The fpformat module is deprecated and will be removed in |
| zmedico | py3k. The % string interpolation operator should be used |
| | instead. Thanks to Ali Polatel <hawking@g.o> for this patch. |
| 11407 | * Fix parse_metadata_use() to raise ParseError instead of |
| zmedico | the ExpatError that was reported by grobian. * Define |
| | __all__. |
svn path=/main/branches/prefix/; revision=11409
Fabian Groffen [Thu, 14 Aug 2008 19:16:02 +0000 (19:16 -0000)]
Merged from trunk 11391:11399
| 11392 | Remove unnecessary dict.keys() call. |
| zmedico | |
| 11393 | When testing userpriv write access in $DISTDIR, redirect the |
| zmedico | 'permission denied' error message to /dev/null. |
| 11394 | For pkg_config and pkg_info, add respective "config" and |
| zmedico | "info" values to EBUILD_PHASES. |
| 11395 | Increase Scheduler._job_delay_max to 10 seconds. |
| zmedico | |
| 11397 | In source_all_bashrcs(), only change $IFS while splitting |
| zmedico | $PROFILE_PATHS into an array, and then restore it before |
| | doing anything else. This avoids interference that can |
| | otherwise occur if code from profile.bashrc (or |
| | /etc/portage/env, be extension) needs to call any of the |
| | elog functions which use the save_IFS and restore_IFS |
| | aliases internally. |
| 11398 | Entirely disable the src_configure phase from the python |
| zmedico | side for EAPI values for which it's not supported. This |
| | avoids pointless spawning of ebuild.sh and also avoids |
| | having bashrc called during an invalid phase as reported by |
| | Arfrever. |
| 11399 | Fix ebuild(1) so that src_configure is only called for EAPIs |
| zmedico | for which it is supported. |
svn path=/main/branches/prefix/; revision=11408
Fabian Groffen [Sun, 10 Aug 2008 09:55:20 +0000 (09:55 -0000)]
check the same as what we will pass on
svn path=/main/branches/prefix/; revision=11391
Fabian Groffen [Sun, 10 Aug 2008 09:37:51 +0000 (09:37 -0000)]
Merged from trunk -r11372:11387
| 11373 | Bug #234157 - Remove BUILD_PREFIX from the docs since it's |
| zmedico | not a user config variable. |
| 11374 | Bug #234100 - Update date and portage version in man page |
| zmedico | headers. |
| 11375 | * Use shlex.split() for splitting COLLISION_IGNORE. * Define |
| zmedico | __all__. |
| 11376 | Add write_contents() and tar_contents() functions to |
| zmedico | __all__. |
| 11377 | Fix dyn_clean() to properly clean |
| zmedico | $PORTAGE_BUILDDIR/.configured. |
| 11378 | Make the scheduler's job delay ramp up faster by increasing |
| zmedico | the factor to 1.0 and adding an exponent of 1.5. |
| 11379 | Eliminate duplicate code by making _eapi1_src_compile() call |
| zmedico | _eapi2_src_configure(). |
| 11380 | Remove pointless `cd` calls in dyn_configure() and |
| zmedico | dyn_compile(). |
| 11381 | Remove INHERITED from the vardbapi aux_get cache since it's |
| zmedico | not really that useful (@live-rebuild currently pulls |
| | INHERITED from the highest visible ebuild). Also adjust keys |
| | used to pull metadata for Package instances, so that |
| | INHERITED won't get pulled from the vardbapi anymore. |
| 11382 | Fix typo. |
| zmedico | |
| 11383 | Add new sections for EAPI 0 and 1 (currently empty). |
| zmedico | |
| 11384 | Document the differences between EAPIs 0 and 1. |
| zmedico | |
| 11385 | Add sample code for the new src_configure and src_compile |
| zmedico | phase functions. |
| 11386 | Remove most references to Package.metadata_keys and replace |
| zmedico | them with approriate keys for the given package type. |
| 11387 | Move the default EAPI definition where it will work in any |
| zmedico | case it might be necessary, such as after loading an exising |
| | environment. |
svn path=/main/branches/prefix/; revision=11390
Fabian Groffen [Sun, 10 Aug 2008 09:33:08 +0000 (09:33 -0000)]
Merged from trunk -r11371:11372
| 11372 | Don't use "dir" for a variable name since it's also a |
| zmedico | builtin. |
svn path=/main/branches/prefix/; revision=11389
Fabian Groffen [Sun, 10 Aug 2008 09:27:21 +0000 (09:27 -0000)]
terminate quotes :/
svn path=/main/branches/prefix/; revision=11388
Fabian Groffen [Sat, 9 Aug 2008 12:09:02 +0000 (12:09 -0000)]
typo, thanks tabnanny
svn path=/main/branches/prefix/; revision=11371
Fabian Groffen [Sat, 9 Aug 2008 12:06:51 +0000 (12:06 -0000)]
implement Mach-O variants for listBrokenBinaries and listProviders
svn path=/main/branches/prefix/; revision=11370
Fabian Groffen [Sat, 9 Aug 2008 10:46:12 +0000 (10:46 -0000)]
Merged from trunk 11354:11357
| 11355 | In LinkageMap.rebuild(), filter empty strings that may be |
| zmedico | returned from split calls when processing the "needed" and |
| | "path" entries. Thanks to Lucian Poston for reporting. |
| 11356 | Fix broken reference to os.path.realpath() inside |
| zmedico | LinkageMap.getSoname(). |
| 11357 | Add new LinkageMap.listBrokenBinaries() and listProviders() |
| zmedico | methods. Thanks to Lucian Poston <lucianposton@gmail.com> |
| | for submitting this patch along with the missing-rebuild |
| | package set posted on the gentoo-portage-dev mailing list. |
svn path=/main/branches/prefix/; revision=11369
Fabian Groffen [Sat, 9 Aug 2008 10:44:29 +0000 (10:44 -0000)]
Merged from trunk 11362:11366
| 11363 | Update date in dyn_help copyright header. Thanks to grobian. |
| zmedico | |
| 11364 | Escape use flags before compiling regular expressions from |
| zmedico | them. Thanks to pchrist for reporting. |
| 11365 | Make eapi*_src_install die since they're not supported. |
| zmedico | |
| 11366 | Add a reference to the default sets.conf location. Thanks to |
| zmedico | dberkholz for the suggestion. |
svn path=/main/branches/prefix/; revision=11368
Fabian Groffen [Sat, 9 Aug 2008 10:42:15 +0000 (10:42 -0000)]
An attempt to get Prefix working with EAPI 2.
svn path=/main/branches/prefix/; revision=11367
Fabian Groffen [Sat, 9 Aug 2008 09:10:19 +0000 (09:10 -0000)]
Merged from trunk 11357:11360
| 11358 | Remove redundant default pkg_nofetch, src_unpack, and |
| zmedico | src_test definitions which are now handled inside |
| | _ebuild_phase_funcs(). |
| 11359 | Increase Scheduler._job_delay_factor from 0.5 to 0.75 so |
| zmedico | that the delay ramps up a little quicker, since the current |
| | value appears to be too weak in some cases. |
| 11360 | When calling dbaip.aux_get() inside |
| zmedico | depgraph._select_package(), only pull the keys that are |
| | cached. This avoids triggering useless cache pulls for |
| | INHERITED from the bindbapi. |
svn path=/main/branches/prefix/; revision=11362
Fabian Groffen [Sat, 9 Aug 2008 09:07:42 +0000 (09:07 -0000)]
Merged from trunk 11349:11354
| 11350 | Implement eapi$EAPI_* default phase functions that are |
| zmedico | equivalent to the default_* functions for the given EAPI. |
| | For example, a function named eapi0_src_compile provides |
| | access to the default src_compile implementation that EAPI 0 |
| | provides. This feature is supported only when EAPI=2_pre2 is |
| | set. Thanks to Brian Harring for suggesting this idea. |
| 11351 | Rename the eapi2_pre2_* functions to just eapi2_* and |
| zmedico | document all the new eapi* functions. |
| 11352 | Fix default phase functions in EAPI 2_pre2, since they |
| zmedico | should be labeled as "2" instead of "2_pre2. Thanks to |
| | Arfrever for reporting. |
| 11353 | Add missing default_src_configure() function to table. |
| zmedico | Thanks to Arfrever. |
| 11354 | Fix broken quoting in eval statements. Thanks to Arfrever. |
| zmedico | |
svn path=/main/branches/prefix/; revision=11361
Fabian Groffen [Fri, 8 Aug 2008 11:20:35 +0000 (11:20 -0000)]
Merged from trunk 11334:11346
| 11335 | Fix breakage in elog function newline handling with causes |
| zmedico | blank/empty lines to get lost. |
| 11336 | Remove redundant 'local line' definition from previous |
| zmedico | commit. |
| 11337 | Fix `svn status` output parsing to account for "replaced" |
| zmedico | files indicated by "R". Thanks to Arfrever. |
| 11338 | Remove the "ebuild.disjointed" check because it's obsolete |
| zmedico | since we've switched to manifest2 only. Thanks to Arfrever |
| | for this patch. |
| 11339 | Replace call to `svn list` with a call to `svn status |
| zmedico | --depth=files --verbose` since the former contacts the |
| | server while the latter does not. Thanks to Arfrever for |
| | this patch. |
| 11340 | In order to conserve some space, make _setitem() skip |
| zmedico | writing keys for which the value is empty. |
| 11341 | Fix a bug in vardbapi.removeFromContents() which sometimes |
| zmedico | prevents the CONTENTS from being updated when it should. |
| 11342 | Add support in EAPI 2_pre2 for a function named "default" is |
| zmedico | redefined for each phase so that it will call the default_* |
| | function corresponding to the current phase. Thanks to |
| | Thomas Anderson for the initial patch. |
| 11343 | In order to properly preserve --jobs and --load-average |
| zmedico | settings for --resume operations, store resume options in |
| | the mtimedb as a dict instead of a list. The dict type is |
| | supported in >=portage-2.1.3_rc8 (2.1.4.4 is current stable) |
| | so people switching between stable/unstable will be fine. |
| 11344 | When testing containment of elem[:1] in "MR", filter |
| zmedico | potential empty string matches. |
| 11345 | Shorten the scheduler status display in the xterm title by |
| zmedico | removing all consecutive spaces. |
| 11346 | * Fix the default() implementation so that it maps the |
| zmedico | ebuild argument to the correct phase function. * Add |
| | --exclude-init-phases option to save_ebuild_env() for |
| | filtering out unnecessary functions and variables when |
| | creating environment.bz2. * Add support for default() and |
| | default_* functions that die in all ebuild phases so that |
| | improper use of these functions is quickly discovered. |
svn path=/main/branches/prefix/; revision=11349
Fabian Groffen [Fri, 8 Aug 2008 10:40:13 +0000 (10:40 -0000)]
Avoid stupid message about missing AF_INET6 attribute on systems with completely no IPv6 support (such as Interix), guess it won't hurt to merge in trunk.
svn path=/main/branches/prefix/; revision=11347
Fabian Groffen [Tue, 5 Aug 2008 20:13:21 +0000 (20:13 -0000)]
Merged from trunk 11322:11332
| 11323 | Make dblink.unmerge() succeed even when ebuild phases are |
| zmedico | unsuccessful because the current behavior is even worse |
| | (results in orphan files). Thanks to Arfrever for this patch |
| | (UnsupportedAPIException hunk by me). |
| 11324 | Insert extra newline before status messages, since that's |
| zmedico | how it used to be. |
| 11325 | Make the status display say "Emerging" and "Emerging binary" |
| zmedico | instead of "Building" and "Extracting". Thanks to Joe |
| | Peterson for the suggestion (including extra newlines from |
| | previous commit). |
| 11326 | Remove code from bug #223447 which is now obsolete due to |
| zmedico | depclean using the depgraph class. |
| 11327 | Bug #233735 - Add support for src_configure and default_* |
| zmedico | functions with new EAPI=2_pre2. Thanks to Arfrever Frehtes |
| | Taifersar Arahesis for the initial patch which I've made a |
| | few adjustments to. |
| 11328 | Add some docs for EAPI 2_pre2. |
| zmedico | |
| 11329 | Remove the use.local.desc IUSE validation which is now |
| zmedico | replaced by glep56 metadata.xml validation. This was |
| | requested by Cardoe since he expects to complete the gleep56 |
| | conversion process pretty soon. |
| 11330 | Remove unused parse_use_local_desc() function. |
| zmedico | |
| 11331 | When there is no build log to display, it means that fetch |
| zmedico | failed, so display the fetch log in that case. |
| 11332 | Make doebuild() display manifest verification failures for |
| zmedico | each corrupt ebuild rather than just the first one. |
svn path=/main/branches/prefix/; revision=11334
Fabian Groffen [Tue, 5 Aug 2008 19:58:22 +0000 (19:58 -0000)]
Also install emerge-webrsync, like in normal Portage, also because it works, yay
svn path=/main/branches/prefix/; revision=11333
Fabian Groffen [Sun, 3 Aug 2008 19:17:54 +0000 (19:17 -0000)]
Merged from trunk 11302:11320
| 11303 | Bug #233487 - Fix --fetch-all-uri breakage: * Make |
| zmedico | --fetch-all-uri imply --fetchonly so that all the |
| | --fetchonly conditionals apply to --fetch-all-uri as well. * |
| | Add an EbuildFetcher.fetchall attribute, and also add the |
| | required support to portage.doebuild() for a "fetchall" |
| | phase that EbuildFetcher can invoke in the background via |
| | the ebuild(1) command. |
| 11304 | Bug #229033 - Use `svn propget -R svn:keywords` to detect |
| zmedico | which will have changed headers that require an additional |
| | manifest commit. Thanks to Fabian Groffen for this patch |
| | which I've made some modifications to: * For clarity, use |
| | separate "no_expansion" variable for cvs bin blobs. * Fix |
| | svn keyword parsing to properly handle multiple keywords |
| | delimited by newlines. |
| 11305 | Remove the list of FEATURES descriptions from |
| zmedico | make.conf.example and replace it with a reference to the |
| | make.conf(5) man page. |
| 11306 | Fix new rejects in cnf/make.conf.*.diff. |
| zmedico | |
| 11307 | Optimize `emaint` --fix binhost so that it the Packages file |
| zmedico | isn't re-read and re-written for each package that's |
| | updated. Instead, hold a lock for the whole time the command |
| | is running and just update it once. |
| 11308 | Make InheritSet more like DowngradeSet, and check INHERITED |
| zmedico | from the highest visible ebuild in the portage tree rather |
| | than from the installed package itself. |
| 11309 | Bug #233252 - Handle InvalidAtom exceptions that can be |
| zmedico | raised from depgraph._add_dep() calls inside |
| | _add_pkg_deps(). |
| 11310 | Add DowngradeSet and OwnerSet to __all__. |
| zmedico | |
| 11311 | Update the Parallel Scheduling section to describe the |
| zmedico | current algorithm. |
| 11312 | Make EbuildFetcher pass a copy of os.environ to the |
| zmedico | ebuild(1) subprocess so that any incremental variables have |
| | been overridden are correctly considered by the config |
| | instance in the subproccess. |
| 11314 | Add debug support to PackagesSystemSet since robbat2 is |
| zmedico | reporting an empty set for some unknown reason. |
| 11315 | * Add INHERITED to vardbapi._aux_cache_keys since |
| zmedico | Package.metadata_keys also includes this one now. * Fix |
| | pull_me calculation inside vardbapi.aux_get() so that it |
| | correctly subtracts _mtime_ from the list of keys to pull. |
| 11316 | In the file collision elog message, only say 'NOT merged' |
| zmedico | when collision-protect is enabled. Thanks to dberkholz for |
| | reporting. |
| 11317 | Add a new RestrictSet which contains all packages that match |
| zmedico | specified RESTRICT values. This could be useful for |
| | implementing something like a @live-rebuild based on |
| | RESTRICT metadata, as suggested in bug #233589. |
| 11318 | Document the fact that use of @installed/EverythingSet makes |
| zmedico | it impossible for emerge to solve blockers by automatic |
| | uninstallation of blocked packages. |
| 11319 | Handle missing sets, to avoid KeyError raised from |
| zmedico | unmerge(). |
| 11320 | Add a more generic VariableSet to replace the InheritSet and |
| zmedico | RestrictSet classes. Thanks to Arfrever Frehtes Taifersar |
| | Arahesis for this patch from bug #233589 (with some minor |
| | adjustments from me). |
svn path=/main/branches/prefix/; revision=11322
Fabian Groffen [Sun, 3 Aug 2008 19:14:53 +0000 (19:14 -0000)]
revert my manifest recommit patch (bug #229033), such that I can cleanly merge zmedico's slightly adapted version
svn path=/main/branches/prefix/; revision=11321
Fabian Groffen [Thu, 31 Jul 2008 13:30:22 +0000 (13:30 -0000)]
Merged from trunk 11287:11300
| 11288 | Bug #233421 - Fix grammar, missing "be" in --update |
| zmedico | description. Thanks to Mikael Magnusson for this patch. |
| 11289 | Split out a _spawn_fetch() function that will be useful for |
| zmedico | implementing a userpriv testcase for bug #233303. |
| 11290 | Fixes in portage.fetch() for bugs #233303 and #94133: * |
| zmedico | Totally skip $DISTDIR creation if the fetch_to_ro feature is |
| | enabled. * Don't touch $DISTDIR permissions unless unless |
| | usepriv and/or userfetch are enabled. * When usepriv and/or |
| | userfetch are enabled, test whether or not a process that |
| | has dropped privileges is able to create a file in the |
| | directory, and only adjust permissions if the test fails. * |
| | Completely |
| 11291 | disable default IONICE command as it breaks for non-root, |
| genone | and ionice might not always be available |
| 11292 | Implement a new @live-ebuilds which is generated from |
| zmedico | installed packages that inherit from know live eclasses such |
| | as cvs, darcs, git, mercurial, and subversion. The list of |
| | eclasses is controlled by an "inherits" attribute that is |
| | configure in sets.conf for and instance of InheritSet. This |
| | set serves a purpose similar to the -scm ebuild suffix that |
| | has been proposed in GLEP 54. |
| 11293 | Add a new @module-rebuild set which emulates the behavior of |
| zmedico | the module-rebuild tool. The /lib/modules path is set in |
| | sets.conf via a "files" attribute of an OwnerSet instance. |
| | This can be easily used to define similar sets based on |
| | paths on installed files. |
| 11294 | Remove quotes since the seem to cause incorrect results. |
| zmedico | |
| 11295 | * Rename @live-ebuilds to @live-rebuild, for consistency |
| zmedico | with the other *-rebuild sets. * Document the new sets. |
| 11296 | Describe InheritSet and OwnerSet. |
| zmedico | |
| 11297 | Bug #233458 - Fix AsynchronousTask exit listener handling so |
| zmedico | that an exit listener will never get called after it's been |
| | passed into removeExitListener(), since the caller of |
| | removeExitListener() needs to be able to be able to trust |
| | that the given exit listener will not be called under any |
| | circumstances. |
| 11298 | Always invalidate results from |
| zmedico | _userpriv_test_write_file_cache when adjusting permissions |
| | on a given directory. |
| 11299 | Bug #233253 - Implement a @downgrade set which selects |
| zmedico | packages for which the highest visible ebuild version is |
| | lower than the currently installed version. This is useful |
| | if you have installed packages from an overlay and you want |
| | to downgrade to the highest visible after removing the |
| | overlay, even though the packages that will be dowgraded are |
| | not necessarily masked in any way. |
| 11300 | Fix DowngradeSet so it's safe for cases when no ebuild is |
| zmedico | available. |
svn path=/main/branches/prefix/; revision=11302
Fabian Groffen [Thu, 31 Jul 2008 13:26:41 +0000 (13:26 -0000)]
revert 1124{2,3}, trunk solved it by simply disabling IONICE by default, which works for me too.
svn path=/main/branches/prefix/; revision=11301
Fabian Groffen [Wed, 30 Jul 2008 13:58:43 +0000 (13:58 -0000)]
Merged from trunk 11267:11284
| 11268 | Add a quiet signal handler for SIGINT and SIGTERM since |
| zmedico | emerge calls ebuild for fetchs and we don't want the user to |
| | see a traceback due to the ebuild process getting killed. |
| 11269 | Disable auto_schedule on all of Scheduler._task_queues and |
| zmedico | schedule them inside _schedule_tasks(). This should help |
| | solve a tight loop. |
| 11270 | Fix the last part of Scheduler._main_loop() to work |
| zmedico | correctly now that all the queues have auto_schedule |
| | disabled. |
| 11271 | Avoid triggering a tight loop AssertionError in |
| zmedico | Scheduler._main_loop(). |
| 11272 | Fix _choose_pkg() to always return something if nothing is |
| zmedico | running or merging. |
| 11273 | Count state changes in Scheduler._schedule_tasks() and avoid |
| zmedico | some extra calls when nothing changes. |
| 11274 | Change conditional USE deps syntax and expermental EAPI from |
| zmedico | 2_pre0 to 2_pre1. Conditional syntax examples: Compact Form |
| | Equivalent Expanded Form foo[bar?] bar? ( foo[bar] ) !bar? ( |
| | foo ) foo[!bar?] bar? ( foo ) !bar? ( foo[-bar] ) foo[bar=] |
| | bar? ( foo[bar] ) !bar? ( foo[-bar] ) foo[!bar=] bar? ( |
| | foo[-bar] ) !bar? ( foo[bar] ) |
| 11275 | * Optimize SequentialTaskQueue.schedule() by using a _dirty |
| zmedico | attribute to track whether anything relevant has changed |
| | since the last schedule call. Since we can always rely on |
| | exist listeners being called, set of running tasks is always |
| | pruned automatically and there is never any need to actively |
| | prune it. * Remove the unused |
| | SequentialTaskQueue.auto_schedule feature. |
| 11276 | Just return a boolean "state changed" value from |
| zmedico | Scheduler._schedule_tasks_imp(), since the other "remaining" |
| | part is now unused. |
| 11277 | Put a 5 second cap on the delay produced by |
| zmedico | Scheduler._job_delay(). |
| 11278 | When checking for news items in --pretend mode, pass |
| zmedico | update=False into the NewsManager.getUnreadItems() call. |
| | Emerge shouldn't have to update anything anyway when in |
| | --pretend mode, and this helps to avoid potential permission |
| | problems. In order to pass the relevant information about |
| | options (--pretend state) to the new code, supporting |
| | function parameters have been added to |
| | display_news_notification(), post_emerge(), and |
| | checkUpdatedNewsItems(). |
| 11279 | * Add more use dep validation. * Fix broken handling of |
| zmedico | !foo? reported by ABCD. |
| 11280 | Remove unused variable. |
| zmedico | |
| 11281 | Remove redundant use dep validation code that's handled by |
| zmedico | regular expression now. |
| 11282 | * Fix Scheduler._restart_if_necessary() breakage so that the |
| zmedico | given package is correctly compared to the last one in the |
| | merge list. * Update code in depgraph.display() to be more |
| | consistent with the code in |
| | Scheduler._is_restart_necessary(). |
| 11283 | Update foo[!bar=] syntax. |
| zmedico | |
| 11284 | Fix typo. |
| zmedico | |
svn path=/main/branches/prefix/; revision=11287
Fabian Groffen [Wed, 30 Jul 2008 13:55:41 +0000 (13:55 -0000)]
Merged from trunk 11248:11267
| 11249 | Bug #233165 - When waiting for jobs and merges to finish in |
| zmedico | Scheduler._main_loop(), keep scheduling the merge queue |
| | since it doesn't autoschedule, and skip the poll loop if |
| | there no event handlers due to synchronous merge tasks being |
| | the only things left to do. |
| 11250 | Fix broken reference to "categories" in |
| zmedico | CategorySet.singleBuilder(). Thanks to Thargor for |
| | reporting. |
| 11251 | Make use of the new config.iteritems() method. |
| zmedico | |
| 11252 | Document the doman language code path translation extension |
| zmedico | from bug #222439. |
| 11253 | Tweak the conditional USE deps examples. |
| zmedico | |
| 11254 | Fix alignment. |
| zmedico | |
| 11255 | Add syntax examples for unconditional USE deps. |
| zmedico | |
| 11256 | Redirect the FEATURES=buildsyspkg message to the log when in |
| zmedico | background mode. |
| 11257 | Fix _use_dep.__str__() to work correctly in the case when |
| zmedico | conditional USE deps have evaluated to nothing (empty string |
| | rather than []). Thanks to ABCD for reporting. |
| 11258 | Implement _use_dep.__nonzero__(). |
| zmedico | |
| 11259 | Tweak table alignment. |
| zmedico | |
| 11260 | In apply_recursive_permissions(), ignore InvalidLocation |
| zmedico | exceptions such as FileNotFound and DirectoryNotFound since |
| | sometimes things disappear, like when adjusting permissions |
| | on DISTCC_DIR. |
| 11261 | Add support for the --jobs option to be specified without an |
| zmedico | argument, and also support -j as a short option. Since |
| | optparse doesn't natively support options with non-required |
| | args, create an insert_optional_args() function that inserts |
| | the required argument into the args so that optparse is |
| | happy. The function inserts the string True as a substitute |
| | for the argument that is required. This string is later |
| | converted to the True constant when stored in the emerge |
| | opts dict (similar to how normal boolean options are |
| | stored). The PollScheduler and SequentialTaskQueue classes |
| | recognize the meaning of the True constant to mean unlimited |
| | concurrent jobs. |
| 11262 | Fix slightly broken loop logic in insert_optional_args() by |
| zmedico | converting it to pop args off of a stack. |
| 11263 | During the first minute of entering the main scheduler loop, |
| zmedico | if --load-average is enabled then limit the rate that new |
| | jobs are spawned, so that the load average measurement has |
| | time to respond to the new load introduced by the new jobs. |
| | The time between spawning new jobs is proportional to the |
| | number of currently running jobs. |
| 11264 | Enable Scheduler._job_delay() whenever --load-average is |
| zmedico | enabled, for whole time the scheduler is running. This |
| | protects against too many jobs being sheduled if the load |
| | average temporarily drops. |
| 11265 | Fix Scheduler._choose_pkg() so that it doesn't choose |
| zmedico | packages prematurely in some cases. |
| 11266 | Make sure Scheduler._choose_pkg() doesn't return a package |
| zmedico | too early when there's no digraph and the previous merge |
| | hasn't completed yet. |
| 11267 | Fix Scheduler._set_digraph() to correctly handle cases when |
| zmedico | max_jobs is True. |
svn path=/main/branches/prefix/; revision=11286
Fabian Groffen [Mon, 28 Jul 2008 20:26:09 +0000 (20:26 -0000)]
implement missing getSoname function in LinkageMapMachO, thanks calculus for testing my fix
svn path=/main/branches/prefix/; revision=11248
Fabian Groffen [Mon, 28 Jul 2008 20:07:56 +0000 (20:07 -0000)]
Prepare emerge-webrsync to get working, use gentooexperimental for distfiles
svn path=/main/branches/prefix/; revision=11247
Fabian Groffen [Mon, 28 Jul 2008 20:03:55 +0000 (20:03 -0000)]
Merged from trunk 11243:11245
| 11244 | Even though the message is split on $'\n' in elog_base(), |
| zmedico | it's still not entirely safe to use it as a delimiter in the |
| | log file since there can still be escaped newlines that will |
| | be expanded due to the echo -e parameter. |
| 11245 | Bug #233103 - In portage.fetch(), pass all config variables |
| zmedico | instead of just those returned by the environ() method which |
| | is filtered. |
svn path=/main/branches/prefix/; revision=11246
Fabian Groffen [Mon, 28 Jul 2008 18:38:40 +0000 (18:38 -0000)]
Avoid portage constantly complaining about IONICENESS not working, since one has to be root to do that (and in Prefix we usually aren't)
svn path=/main/branches/prefix/; revision=11243
Fabian Groffen [Mon, 28 Jul 2008 18:23:29 +0000 (18:23 -0000)]
fix misc issues causing eapi not to be valid
svn path=/main/branches/prefix/; revision=11242
Fabian Groffen [Mon, 28 Jul 2008 18:05:08 +0000 (18:05 -0000)]
I want a frikkin compiler to tell me I am not sane...
svn path=/main/branches/prefix/; revision=11241
Fabian Groffen [Mon, 28 Jul 2008 17:52:55 +0000 (17:52 -0000)]
Original commit:
| 11233 | Add support for an new EAPI="2_pre0" value so that people |
| zmedico | who want to test USE deps can set the EAPI to something |
| | other than 0 or 1. We can support as many different |
| | experimental EAPI values as we need, and drop support for |
| | them when the final EAPI 2 is defined. |
Changed into a Prefix enabled version, which is more generic. It build
further on the idea that multiple properties are being present in EAPI.
I split it up in "should have" properties ("prefix" for now) that must
be present, and "can have" properties, that may be present. This
patch is not perfect (can have EAPI="prefix 2_pre0 1 0") but fixing that
requires more thinking (successing properties, conflicting
properties...)
svn path=/main/branches/prefix/; revision=11240
Fabian Groffen [Mon, 28 Jul 2008 17:48:47 +0000 (17:48 -0000)]
Merged from trunk 11225:11238
| 11226 | Fix the new glep56 code to handle missing metadata.xml. |
| zmedico | Thanks to jmbsvicetto for reporting. |
| 11227 | Fix incorrect logging.exception() arguments. The exception |
| zmedico | instance is not supposed to be passed into this function. |
| 11228 | Bug #233137 - Implement Atom.split(). |
| zmedico | |
| 11229 | * Add support in dep_getusedeps() and isvalidatom() for |
| zmedico | comma separated USE deps that only have one set of square |
| | brackets. * Add test cases for the new comma separated USE |
| | deps syntax. |
| 11230 | Create a SlotDict constructor which can take an optional |
| zmedico | positional arg that is passed to the update() method |
| | (similar to the dict constructor), and also pass keyword |
| | arguments into the update() method if any are given. This |
| | makes it possible to use the constructor similarly to the |
| | way that the _emerge.SlotObject constructor is used. |
| 11231 | Implement new conditional USE dep syntax: Conditional |
| zmedico | evaluation behavior: parent state conditional result x x? x |
| | -x x? x -x? -x -x? -x x x= x -x x= -x x x!= -x -x x!= x |
| | Conditional syntax examples: compact form equivalent |
| | expanded form foo[bar?] foo bar? ( foo[bar] ) foo[-bar?] foo |
| | !bar? ( foo[-bar] ) foo[bar=] foo bar? ( foo[bar] ) !bar? ( |
| | foo[-bar] ) foo[bar!=] foo bar? ( foo[-bar] ) !bar? ( |
| | foo[bar] ) |
| 11232 | Disable the multi-bracket USE deps syntax, so only the comma |
| zmedico | separated syntax is now valid. |
| 11234 | Fix arg count when constructing a TypeError in the SlotDict |
| zmedico | constructor. |
| 11235 | Create a table to document conditional USE dependency |
| zmedico | syntax. |
| 11236 | Add note about support for EAPI="2_pre0". |
| zmedico | |
| 11238 | Remove pointless generator expression. |
| zmedico | |
svn path=/main/branches/prefix/; revision=11239
Fabian Groffen [Sun, 27 Jul 2008 14:54:44 +0000 (14:54 -0000)]
Improve svn properties reading, thanks to Arfrever in #-portage.
svn path=/main/branches/prefix/; revision=11225
Fabian Groffen [Sun, 27 Jul 2008 14:43:00 +0000 (14:43 -0000)]
Merged from trunk 11215:11223
| 11216 | Fixes from Arfrever: * Highlight the "Files built without |
| zmedico | respecting LDFLAGS" QA Notice with red. * Remove |
| | scanelf-ignored-LDFLAGS.log when empty. |
| 11217 | Fix breakage from r11213 in INHERITED values returned from |
| zmedico | portdbapi.aux_get(). |
| 11218 | Add support to elog_base() to split messages on newlines |
| zmedico | automatically. Thanks to Arfrever for the suggestion. |
| 11219 | Make all the elog functions split lines when displaying |
| zmedico | output, just like elog_base() does when saving it. |
| 11220 | Make elog_base() just use \n as the delimiter in the file |
| zmedico | since it's safe now that lines are always split. |
| 11221 | Use ${NORMAL} to properly restore color when necessary. |
| zmedico | |
| 11222 | Update path to make.conf.example. Thanks to Arfrever. |
| zmedico | |
| 11223 | Fix EbuildFetcher to pass all config variables to the |
| zmedico | fetcher so things like http_proxy are included. |
svn path=/main/branches/prefix/; revision=11224
Fabian Groffen [Sun, 27 Jul 2008 11:51:06 +0000 (11:51 -0000)]
Merged from trunk 11212:11213
| 11213 | Serialize src_unpack() $DISTDIR access for live ebuilds |
| zmedico | since otherwise they can interfere with eachother. This will |
| | have to rely on a hardcoded list of eclasses until we create |
| | way to properly tag this information in the metadata. |
svn path=/main/branches/prefix/; revision=11215
Fabian Groffen [Sun, 27 Jul 2008 11:49:38 +0000 (11:49 -0000)]
fix off by one
svn path=/main/branches/prefix/; revision=11214
Fabian Groffen [Sun, 27 Jul 2008 11:29:55 +0000 (11:29 -0000)]
Merged from trunk 11209:11210
| 11210 | * Define a frozenset of live eclasses for the LIVEVCS.stable |
| zmedico | check. * Add "mercurial" to the list of known live eclasses. |
svn path=/main/branches/prefix/; revision=11212
Fabian Groffen [Sun, 27 Jul 2008 11:28:26 +0000 (11:28 -0000)]
Do a humble attempt to implement something along the lines of bug #229033
Big problem with SVN is that retrieving the known files is slow and
expensive (it goes over the network), hence should be avoided.
Imlemented Manifest commit in the first batch, if no headers change.
This implementation is a (hopefully) 90% coverage one. First, not all
keywords are recognised (they never were), second we do not check in
SVN's case if the used keyword is the one that SVN expands. What we
just try to achieve is to avoid the single Manifest commit on SVN repos
which by default do not use header expansion. This objective seems to
be made with this commit.
svn path=/main/branches/prefix/; revision=11211
Fabian Groffen [Sun, 27 Jul 2008 09:46:46 +0000 (09:46 -0000)]
Fix repoman not seeing subversion auto-ignored files (such as patch rejects and editor backup copies), thereby causing messed up Manifests. This fix should be merged in the trunk.
svn path=/main/branches/prefix/; revision=11208
Fabian Groffen [Sun, 27 Jul 2008 09:31:10 +0000 (09:31 -0000)]
Merged from trunk 11201:11206
| 11202 | Fix AsynchronousTask.removeStartListener() and |
| zmedico | removeExitListener() to return early if there are no |
| | listeners. |
| 11203 | In PollScheduler._poll(), add a call to _schedule() in order |
| zmedico | to ensure that event handlers are registered if possible. If |
| | not, raise StopIteration in order to avoid endless blocking |
| | in a poll call with no file descriptors registered. Add |
| | StopIteration exception handling for all |
| | PollScheduler._poll._next_poll_event() calls. |
| 11204 | Schedule ebuild pkg_setup phases on the merge queue, in |
| zmedico | order to serialize unsandboxed access to the live |
| | filesystem. |
| 11205 | Bug #233077 - Add QA check which verifies that LDFLAGS are |
| zmedico | respected. For now this only works when LDFLAGS contains |
| | --hash-style=gnu since in this case the the elf files should |
| | not contain .hash sections and we can use scanelf to check |
| | whether or not the those sections exist. This adds a |
| | QA_DT_HASH variable that is analogous to existing QA control |
| | variables documented in the ebuild.5 man page. There is also |
| | a complementary QA_STRICT_DT_HASH variable that can be set |
| | in make.conf in order to ignore QA_DT_HASH settings in |
| | ebuilds. Thanks to Arfrever Frehtes Taifersar Arahesis for |
| | this patch which I've tweaked in just a few minor ways. |
| 11206 | Temporarily `set -o noglob` when splitting QA_DT_HASH. |
| zmedico | |
svn path=/main/branches/prefix/; revision=11207
Fabian Groffen [Sat, 26 Jul 2008 20:55:16 +0000 (20:55 -0000)]
Merged from trunk 11187:11200
| 11188 | Detect an invalid cpv passed into portdbapi.findname2() and |
| zmedico | raise an InvalidPackageName exception. This avoids silent or |
| | nonsense errors later on. |
| 11190 | Fix PORTAGE_BIN_PATH typo in EbuildFetcher._start(). |
| zmedico | |
| 11191 | Suppress "for /" and "to /" in status messages when ROOT=/. |
| zmedico | Thanks to Thargor for the suggestion. |
| 11192 | Add PORTAGE_IONICE_COMMAND setting for make.conf which is |
| zmedico | called to adjust IO priority on portage and it's |
| | subprocesses. |
| 11193 | Reduce the default PORTAGE_IONICE_COMMAND priority to "idle" |
| zmedico | priority since it's probably suitable for most people. This |
| | way, even when the IO load is very high, the impact on |
| | system responsiveness should be practically unnoticeable. |
| 11194 | * For bug 227625, automatically create the |
| zmedico | /usr/src/rpm/SOURCES directory when necessary. * Add `cd |
| | "${T}"` at the beginning of dyn_rpm() since the spec file is |
| | written to $PWD. |
| 11195 | Replace hardcoded i386 machine name in dyn_rpm() with the |
| zmedico | result of `uname -m`. |
| 11196 | Bug #228085 - In the event of a file collision, the |
| zmedico | explanation about the collision and how to solve it may not |
| | be visible via a scrollback buffer, especially if the number |
| | of file collisions is large. Therefore, show a summary at |
| | the end and refer the user to the elog messages. |
| 11197 | Bug #232924 - When the user specifies a non-existent set, |
| zmedico | display a list of existing sets. |
| 11198 | Split out an apply_priorities() function to apply nice and |
| zmedico | ionice settings. |
| 11199 | * Use pure bash ${PARAMETER%/*} instead of spawing`dirname`. |
| zmedico | * Break out of loop as early as possible when searching for |
| | libs in ${D}. |
| 11200 | Fix LinkageMap.findConsumers() to check whether the master |
| zmedico | link for a lib providing a given soname actually points to |
| | that lib. If there is another version of this lib with the |
| | same soname and the master link points to that other |
| | version, this lib will be shadowed and won't have any |
| | consumers. By eliminating false, positives this way, we |
| | avoid the following state after upgrade from |
| | media-libs/mesa-7.0.3 to media-libs/mesa-7.1_rc3: # scanelf |
| | -S /usr/lib64/libGLU.so* TYPE SONAME FILE ET_DYN libGLU.so.1 |
| | /usr/lib64/libGLU.so ET_DYN libGLU.so.1 |
| | /usr/lib64/libGLU.so.1 ET_DYN libGLU.so.1 |
| | /usr/lib64/libGLU.so.1.3 ET_DYN libGLU.so.1 |
| | /usr/lib64/libGLU.so.1.3.070003 ET_DYN libGLU.so.1 |
| | /usr/lib64/libGLU.so.1.3.070100 <- shadowed lib Thanks to |
| | Diego "Flameeyes" Pettenò for reporting this issue. |
svn path=/main/branches/prefix/; revision=11201
Fabian Groffen [Fri, 25 Jul 2008 21:08:05 +0000 (21:08 -0000)]
we also need EPREFIX in Prefix
svn path=/main/branches/prefix/; revision=11189
Fabian Groffen [Fri, 25 Jul 2008 07:37:53 +0000 (07:37 -0000)]
fix syntax error found by tabcheck
svn path=/main/branches/prefix/; revision=11187
Fabian Groffen [Fri, 25 Jul 2008 07:34:22 +0000 (07:34 -0000)]
Merged from trunk 11169:11185
| 11170 | Make --fetchonly disable background mode. |
| zmedico | |
| 11173 | Add GENTOO_MIRRORS to the variables exported by |
| zmedico | EbuildFetcher. |
| 11175 | Add sandbox write access to PORTAGE_PYM_PATH before |
| zmedico | importing portage in order to avoid sandbox violations after |
| | python upgrade. |
| 11176 | Fix --depclean/--prune regressions reported by selkin: * |
| zmedico | Make --with-bdeps default to "y" for removal actions. * |
| | Treat unsatisfied build time dependencies as "soft" for |
| | removal actions, even when --with-bdeps=y. |
| 11177 | Avoid "0 files checked ..." messages reported by Pesa. |
| zmedico | |
| 11178 | Fix typo, found by pyflakes. |
| zmedico | |
| 11179 | Fix bug in get_repo_path() when called with multiple args. |
| zmedico | |
| 11181 | Bug #228117 - Insert a trailing newline in the getconfig() |
| zmedico | input file as necessary to avoid a silent error in shlex |
| | that is triggered by a source statement at the end of the |
| | file without a trailing newline after the source statement. |
| | To solve this problem, derive a special file class that |
| | inserts a newline just before eof when necessary. |
| 11182 | Bug #206773 - Add a new PORTAGE_IONICE_COMMAND variable that |
| zmedico | emerge uses to adjust ionice priority, similar to |
| | PORTAGE_NICENESS but used more like FETCHCOMMAND since so |
| | that portage doesn't have to know anything about ionice |
| | options. The command should include a \${PID} place-holder |
| | to be substituted with an integer pid. |
| 11183 | * Refer user to make.conf(5) if PORTAGE_IONICE_COMMAND |
| zmedico | fails. * Document that PORTAGE_IONICE_COMMAND can be |
| | disabled by setting it to an empty string. |
| 11184 | When in background mode and there is a single failure, dump |
| zmedico | the whole build log for convenience. |
| 11185 | Cancel prefetchers if they're the only reason the main poll |
| zmedico | loop is still running. |
svn path=/main/branches/prefix/; revision=11186
Fabian Groffen [Wed, 23 Jul 2008 06:54:24 +0000 (06:54 -0000)]
Merged from trunk 11157:11169
| 11158 | Fix --depclean/--prune to add lib providers and their |
| zmedico | dependencies to the graph and create a new clean list when |
| | necessary. This completes the fix for bug #230053. |
| 11159 | Adjust --fetchonly failure messages to work better with |
| zmedico | background mode. |
| 11160 | Adjust status messages for --fetchonly mode. |
| zmedico | |
| 11161 | Redirect dblink._preserve_libs() messages to the log when in |
| zmedico | background mode. |
| 11162 | Make --pretend disable background mode and imply --jobs=1. |
| zmedico | |
| 11163 | Disable "Installing" status messages when in --pretend or |
| zmedico | --buildpkgonly mode. |
| 11164 | Redirect build log eqawarn messages to the log file when in |
| zmedico | background mode. |
| 11165 | Show the number of failed packages in the status display. |
| zmedico | |
| 11166 | Fix resume_depgraph() so that it doesn't raise an |
| zmedico | AssertionError due to unsatisfied PDEPEND. |
| 11167 | In dblink.unmerge(), redirect preserve-libs "!needed" |
| zmedico | messages to the log file when in background mode. |
| 11168 | Update --depclean and --prune, removing warnings about |
| zmedico | libraries since those are now automatically accounted for. |
| 11169 | Handle UnsatisfiedResumeDep in |
| zmedico | Scheduler._calc_resume_list(). |
svn path=/main/branches/prefix/; revision=11172
Fabian Groffen [Wed, 23 Jul 2008 06:52:10 +0000 (06:52 -0000)]
Merged from trunk 11149:11157
| 11150 | Fix broken references to the old elog_modules namespace. |
| zmedico | |
| 11151 | Redirect the 'Recording in "world" favorites file...' |
| zmedico | message through _status_msg(). |
| 11152 | For --depclean and --prune, eliminate duplicate graph |
| zmedico | creation code by re-using the depgraph class to do it. The |
| | depgraph class was used in the past for this purpose but |
| | back then it didn't handle USE flags correctly for installed |
| | packages. Now it works fine. |
| 11153 | Fix typo. |
| zmedico | |
| 11154 | Fix breakage in the depclean/prune unsatisfied deps |
| zmedico | handling. |
| 11155 | In dblink._preserve_libs(), strip leading $ROOT from |
| zmedico | contents paths so that it works when $ROOT != /. |
| 11156 | In dblink.findProviders(), clone the "path" variable before |
| zmedico | extending it since obviously it shouldn't grow each time |
| | that findProviders() is called. |
| 11157 | Add initial lib awareness for --depclean and --prune. |
| zmedico | Currently it will simply bail out if the clean list contains |
| | a package that will cause link breakage if removed. TODO: |
| | Calculate a new cleanlist instead of bailing out, and |
| | implement a package set for rebuilding consumers. |
svn path=/main/branches/prefix/; revision=11171
Fabian Groffen [Sun, 20 Jul 2008 10:13:34 +0000 (10:13 -0000)]
Merged from trunk 11139:11147
| 11140 | Use the memoized _choose_pkg() result to return earlier from |
| zmedico | _schedule_tasks() when possible. |
| 11141 | Calling _schedule() from Scheduler._poll() is unsafe since |
| zmedico | the _schedule() call itself can consume poll events. |
| | Therefore, until a safe way exists, don't do it. |
| 11142 | Fix Scheduler._schedule_tasks_imp() so that it never |
| zmedico | indicates that there are remaining tasks in cases when it's |
| | supposed to bail out due to a package failure. |
| 11143 | Merge tasks currently run synchronously which makes it |
| zmedico | necessary to disable auto_schedule on the merge queue in |
| | order to avoid excess recursion which prevents tasks from |
| | being marked complete as soon as they should be. |
| 11144 | Make PollScheduler._poll() add poll events to the stack |
| zmedico | instead of returning them. This makes it more useful as a |
| | hook, allowing Scheduler._poll() to safely call _schedule(). |
| 11145 | * Clean up _schedule_tasks_imp() a bit. * Make _task() |
| zmedico | access _background directly instead of as a parameter. |
| 11146 | Fix Scheduler emergelog() interaction to properly disable |
| zmedico | emergelog() xterm titles whenever the status display is |
| | enabled. |
| 11147 | Redirect some more unmerge() output through writemsg_level() |
| zmedico | so that it can filtered out when in background mode. |
svn path=/main/branches/prefix/; revision=11149
Fabian Groffen [Sun, 20 Jul 2008 10:12:19 +0000 (10:12 -0000)]
Merged from trunk 11125:11139
| 11126 | Bug #199788 - Make repoman parse GLEP 56 USE flag |
| zmedico | descriptions from metadata.xml (in addition to |
| | use.local.desc). Thanks to Doug Goldstein <cardoe@g.o> for |
| | this patch. |
| 11127 | Add a MergeListItem.statusMessage(msg) callback for |
| zmedico | displaying messages like "Building this" or "Installing |
| | that". This delegates the resposibility of generating \r and |
| | \n control characters, to guarantee that lines are created |
| | or erased when necessary and appropriate. TODO: Make |
| | JobStatusDisplay display the output. |
| 11128 | Add support to JobStatusDisplay() to update the current line |
| zmedico | on the terminal, which in the future can be used to erase |
| | the line and update it with new information such as the |
| | latest load average measurement. This uses curses to obtain |
| | the simple control codes that are needed, and has fallback |
| | codes in case the curses module is unavailable or raises an |
| | error. When stdout is not a tty then the line erasing |
| | behavior is automatically disabled and display updates are |
| | simply show on a new line. |
| 11129 | When JobStatusDisplay.displayMessage() is called while the |
| zmedico | status is currently displayed, call display() after showing |
| | the given message. |
| 11130 | Make all poll() calls pass through PollScheduler._poll(). |
| zmedico | |
| 11131 | Add a hook in Scheduler._poll() that updates the display |
| zmedico | once every 2 seconds if stdout is a tty, so that the load |
| | average numbers are always current. |
| 11132 | Display "Verifying ebuild manifests" and "Starting parallel |
| zmedico | fetch" via the status display instead of directly to stdout. |
| 11133 | Add a Scheduler._background_mode() method to control |
| zmedico | background mode, and trigger it if either --quiet is enabled |
| | or --jobs > 1. |
| 11134 | Fix typos. |
| zmedico | |
| 11135 | * Call self._schedule() from inside Scheduler._poll(). This |
| zmedico | ensures that new jobs can scheduled as soon a the load |
| | average is low enough, and one of the running jobs doesn't |
| | have to complete for _schedule() to get called. * Cache the |
| | result of _background_mode() so it doesn't have to be called |
| | inside _schedule(), since this method is called a lot more |
| | often now. * Move background related state adjustments into |
| | _background_mode(). * Call _status_display.display() |
| | unconditionally inside _schedule_tasks(), since we want to |
| | update the display to show current load average even if no |
| | new jobs are added. |
| 11136 | Fix JobStatusDisplay._init_term() to supply default codes in |
| zmedico | cases when curses.tigetstr() returns None. |
| 11137 | Instead of calling _background_mode(), use the cached |
| zmedico | result. |
| 11138 | Bug #231464 - Replace erroneous "return" statement in |
| zmedico | elog_process() with a continue statement. Thanks to TGL for |
| | this patch. |
| 11139 | When Scheduler._choose_pkg() is unable to find a package, |
| zmedico | memoize the result until at least one of the existing jobs |
| | completes. |
svn path=/main/branches/prefix/; revision=11148
Fabian Groffen [Fri, 18 Jul 2008 09:02:11 +0000 (09:02 -0000)]
Merged from trunk 11111:11124
| 11112 | Since it's not very interesting, disable the "merges" part |
| zmedico | of status display. |
| 11113 | Also, don't count "merges" as changes in the status display. |
| zmedico | |
| 11114 | Initialize JobStatusDisplay.merges since it's not |
| zmedico | initialized with the bound properties anymore. |
| 11115 | Also, reset "merges" in JobStatusDisplay.reset(). |
| zmedico | |
| 11116 | When dblink is looping over files for merge/unmerge, |
| zmedico | temporarily yield to the scheduler each time a fixed number |
| | of files are processed (currently 20). This gives the |
| | scheduler an opportunity to service pending poll events. |
| | This is implemented with a new |
| | PollScheduler._schedule_yield() method which calls poll() |
| | exactly once, without blocking, and any services any |
| | resulting poll events. |
| 11117 | Redirect dblink elog messages from stdout to the log file |
| zmedico | when in background mode. This is implemented by adding a new |
| | "out" parameter to the portage.elog.e* functions, which can |
| | be used to send output to an arbitary file instead of |
| | stdout. |
| 11118 | If an problem is detected in dblink._security_check(), log |
| zmedico | it via elog so that it's properly displayed in the |
| | foreground even when in background mode. |
| 11119 | Use dblink._eerror() to eliminate duplicate code. |
| zmedico | |
| 11120 | In dblink._dblink_display_merge(), when in background mode |
| zmedico | and no log is available, suppress output when level < |
| | logging.WARNING. This hides ">>> Regenerating |
| | /etc/ld.so.cache..." messages that appear after postrm when |
| | PORT_LOGDIR is unset (the log file if it exists is removed |
| | after postrm so there's no place to log the message). |
| 11121 | Change package status messages to give more accurate |
| zmedico | information about what is happening at a given time: * |
| | Change "Emerging" messages to "Building" for ebuilds and |
| | "Extracting" for binary packages. * Change "Merging" |
| | messages to "Installing" and "Uninstalling". |
| 11122 | Add background output redirection and async ebuild phase |
| zmedico | support for PackageUninstall (automatic uninstalls that |
| | solve blockers). |
| 11123 | Suppress counter initialization warning messages in |
| zmedico | vardbapi.get_counter_tick_core() in cases when the vdb is |
| | empty. This avoids pointless error messages when merging |
| | packages into a new root, like when building stage 1. |
| 11124 | Fix typo. |
| zmedico | |
svn path=/main/branches/prefix/; revision=11125
Fabian Groffen [Thu, 17 Jul 2008 14:09:19 +0000 (14:09 -0000)]
Fix identation (tabnanny complains about this), should be merged into trunk
svn path=/main/branches/prefix/; revision=11110
Fabian Groffen [Thu, 17 Jul 2008 14:06:14 +0000 (14:06 -0000)]
Merged from trunk 11106:11107
| 11107 | Reformat the status display. Here's an example: >>> Jobs: 0 |
| zmedico | of 2 complete, 2 running Load average: 2.9, 2.4, 2.0 >>> |
| | Jobs: 0 of 2 complete, 1 running, 2 merges Load average: |
| | 2.8, 2.4, 2.0 >>> Jobs: 2 of 2 complete Load average: 2.6, |
| | 2.3, 2.0 |
svn path=/main/branches/prefix/; revision=11109
Fabian Groffen [Thu, 17 Jul 2008 14:03:12 +0000 (14:03 -0000)]
Merged from trunk 11087:11105
| 11088 | Avoid raising an IndexError from deprecated_profile_check() |
| zmedico | if the deprecated file happens to be empty. Instead just |
| | refer to the "Gentoo Upgrading Guide". Thanks to armin76 for |
| | reporting. |
| 11089 | Remove TODO since it no longer exists. |
| zmedico | |
| 11090 | Add --anon option for anonsvn.gentoo.org users. |
| zmedico | |
| 11091 | Add --anon to the usage message. |
| zmedico | |
| 11092 | Eliminate duplicate code by making EbuildProcess inherit |
| zmedico | from SpawnProcess instead of SubProcess. |
| 11093 | When SpawnProcess._start() spawns a process in the |
| zmedico | background, use /dev/null so that any attempts to read from |
| | stdin will immediately return EOF instead of blocking |
| | indefinitely. TODO: Use job control functions like |
| | tcsetpgrp() to control access to stdin. |
| 11094 | Add the new --anon option to the getopt args. |
| zmedico | |
| 11095 | In SequentialTaskQueue.clear(), remove the exit listener |
| zmedico | from each task before cancelling it. |
| 11096 | Add a JobStatusDisplay class to encapsulate the status |
| zmedico | display. This object tracks whether or not any of it's |
| | attributes have changed since the last time it was |
| | displayed, so the it's only displayed when something has |
| | changed. Unlike before, now the display should always update |
| | whenever a relevant change occurs. |
| 11097 | Don't increment the "complete" count for uninstall tasks. |
| zmedico | |
| 11098 | * Add a new portage.util.writemsg_level() function which |
| zmedico | uses a "level" parameter to control whether the message is |
| | sent to stdout or stderr. When level >= logging.WARNING then |
| | the message is sent to stderr, otherwise it is sent to |
| | stdout. The noiselevel is passed directly to writemsg(). * |
| | Use writemsg_level() for handling merge messages from dblink |
| | and add appropriate noiselevel values to each call. * Add a |
| | writemsg_level parameter to portage.env_update() so that |
| | dblink can pass in dblink._display_merge() as a means to |
| | redirect output back to the scheduler for appropriate |
| | handling when running in the background. |
| 11099 | Tweak interaction between Scheduler and status display so |
| zmedico | that it doesn't display more often than necessary and avoids |
| | showing inconsistent counts. |
| 11100 | Handle the case in dblink.unmerge() when the log file is a |
| zmedico | temp file and so it doesn't exist after postrm cleaning. |
| 11101 | Don't show a blank newline before ">>> Emerging" when in |
| zmedico | background mode. |
| 11102 | Make BinpkgVerifier redirect output to the fetch log when in |
| zmedico | background mode. |
| 11103 | Update the status display in _build_exit(), after |
| zmedico | decrementing the job count. |
| 11104 | Make Binpkg send "Extracting" messages to the log file when |
| zmedico | in background mode, instead of stdout. |
| 11105 | Don't show "Merging" and "merged." messages when in |
| zmedico | background mode. |
svn path=/main/branches/prefix/; revision=11108
Fabian Groffen [Thu, 17 Jul 2008 13:58:15 +0000 (13:58 -0000)]
PORTAGE_CONFIGROOT includes EPREFIX since my recent changes, thanks Lothar Braun for reporting
svn path=/main/branches/prefix/; revision=11106
Fabian Groffen [Wed, 16 Jul 2008 18:11:32 +0000 (18:11 -0000)]
fix more prefix errors
svn path=/main/branches/prefix/; revision=11087
Fabian Groffen [Wed, 16 Jul 2008 18:02:33 +0000 (18:02 -0000)]
fix NameError
svn path=/main/branches/prefix/; revision=11086
Fabian Groffen [Wed, 16 Jul 2008 17:53:11 +0000 (17:53 -0000)]
Fix misc amount of out-of-prefix issues
svn path=/main/branches/prefix/; revision=11085
Fabian Groffen [Wed, 16 Jul 2008 15:33:03 +0000 (15:33 -0000)]
I forgot about EROOT, reorder to make a bit more sense
svn path=/main/branches/prefix/; revision=11084
Fabian Groffen [Wed, 16 Jul 2008 15:21:47 +0000 (15:21 -0000)]
expansion doesn't run on man-pages, document Prefix variables in ebuild.5
svn path=/main/branches/prefix/; revision=11083
Fabian Groffen [Wed, 16 Jul 2008 13:10:43 +0000 (13:10 -0000)]
We have a most confusing situation here, which is most of all pretty
weak for protecting us from making mistakes.
First there is a config_root (PORTAGE_CONFIGROOT) which can be a path
somewhere, from which all paths need to be relative (e.g.
etc/portage), hence those constants do NOT have EPREFIX, because
config_root contains EPREFIX by default -- overriding it loses the
EPREFIX as one would expect.
Second there is target_root (ROOT) which is to install somewhere
completely else, in Prefix of limited use. Because this is an offset
always given, the EPREFIX should always be applied in it. Those
constants (like VDB_PATH) DO have EPREFIX.
svn path=/main/branches/prefix/; revision=11082
Fabian Groffen [Wed, 16 Jul 2008 12:21:33 +0000 (12:21 -0000)]
Revert -r11077:11080, lstrip doesn't take a string but chars. Need to fix it differently.
svn path=/main/branches/prefix/; revision=11081
Fabian Groffen [Wed, 16 Jul 2008 12:07:46 +0000 (12:07 -0000)]
this is going to cause some headaches, strip more prefixes for now
svn path=/main/branches/prefix/; revision=11080
Fabian Groffen [Wed, 16 Jul 2008 11:52:52 +0000 (11:52 -0000)]
more prefix confusement
svn path=/main/branches/prefix/; revision=11079
Fabian Groffen [Wed, 16 Jul 2008 11:24:54 +0000 (11:24 -0000)]
messing around with/without EPREFIX, I'm affraid I lost the consistency here
svn path=/main/branches/prefix/; revision=11078
Fabian Groffen [Wed, 16 Jul 2008 11:11:49 +0000 (11:11 -0000)]
Merged from trunk 11060:11076
| 11061 | In Scheduler._restart_if_necessary(), use list.remove() |
| zmedico | instead of assuming that portage is first in the resume |
| | list. |
| 11062 | Make EbuildFetcher copy FETCHCOMMAND and RESUMECOMMAND to |
| zmedico | the fetch environment, in case the default values have been |
| | overridden. |
| 11063 | Add some cross references between emerge's new parallel |
| zmedico | build options and MAKEOPTS. |
| 11064 | When --jobs > 1, display a status message each time that |
| zmedico | Scheduler._schedule_tasks() is called. For example: >>> |
| | Jobs: 2 running, 0 merges, load average: 0.34, 0.46, 0.27 |
| 11065 | Redirect dblink.unmerge() output to the log file when --jobs |
| zmedico | > 1. |
| 11066 | Adapt the dblink merge display callback to use log levels |
| zmedico | from the logging module so that it can be used as an |
| | interface for both writemsg() and writemsg_stdout(). |
| 11067 | Make dblink._display_merge() detect scheduler presence |
| zmedico | internally. |
| 11068 | Redirect dblink.merge() output to the log file when --jobs > |
| zmedico | 1. |
| 11069 | Pass the whole dblink instance into |
| zmedico | Scheduler._dblink_display_merge() in case it's ever needed. |
| 11070 | Show LDFLAGS in emerge --info output. Thanks to dirtyepic |
| zmedico | for this patch. |
| 11071 | Remove the config._filter_calling_env attribute since it's |
| zmedico | not really needed. Replace it with an simple |
| | os.path.exists() call inside config.environ(). |
| 11072 | Add async support for calling functions in |
| zmedico | misc-functions.sh: * Create a new MiscFunctionsProcess class |
| | derived from SpawnProcess. * Add "background" support to the |
| | SpawnProcess class and split out a _spawn() template |
| | function so MiscFunctionsProcess can override it. * Rename |
| | the existing EbuildPhase class to EbuildProcess. * Create a |
| | new EbuildPhase class derived from CompositeTask. This class |
| | spawns the EbuildProcess and MiscFunctionsProcess classes as |
| | sub-tasks. |
| 11073 | * Add xterm titles support to Scheduler._display_status(). * |
| zmedico | Tweak emergelog() so that it won't interfere with the |
| | scheduler summary. |
| 11075 | Add a progress counter showing "x of y complete" to |
| zmedico | Scheduler._display_status(). |
| 11076 | Pass background=True into prefetcher constructors. |
| zmedico | |
svn path=/main/branches/prefix/; revision=11077
Fabian Groffen [Wed, 16 Jul 2008 08:34:04 +0000 (08:34 -0000)]
Do not check config_root + prefix, if we explicitly set PORTAGE_CONFIGROOT we mean that dir, not the Prefix variant
svn path=/main/branches/prefix/; revision=11074
Fabian Groffen [Tue, 15 Jul 2008 19:59:50 +0000 (19:59 -0000)]
try to be more correct in Prefix
svn path=/main/branches/prefix/; revision=11060
Fabian Groffen [Tue, 15 Jul 2008 19:40:07 +0000 (19:40 -0000)]
Prevent false positives on AIX for .a files. On AIX these are pseudo "dynamic libraries".
svn path=/main/branches/prefix/; revision=11059
Fabian Groffen [Tue, 15 Jul 2008 19:21:45 +0000 (19:21 -0000)]
Merged from trunk 11048:11057
| 11049 | Split out a generic SubProcess._unregister() method that's |
| zmedico | called from _wait() to make sure everything is cleaned up |
| | before wait() returns. This might solve an intermittent |
| | KeyError that is sometimes raised from the main poll() loop |
| | when a handler is no longer registered for a poll() event |
| | that has occurred. |
| 11050 | Split out a _main_loop_cleanup() method from |
| zmedico | Scheduler._merge() and call it before restarting in |
| | _restart_if_necessary() to kill any parallel-fetch tasks |
| | that may still be running. |
| 11051 | Add a PollScheduler._next_poll_event() method to maintain a |
| zmedico | central event queue between _poll_loop() and |
| | _schedule_wait(). This solves an intermittent KeyError from |
| | _poll_loop() which occured when an event handler called |
| | _schedule_wait() which unregistered an event handler for an |
| | event that _poll_loop() had already recieved but hadn't |
| | processed yet. |
| 11052 | Use self.registered to return early from SubProcess.poll() |
| zmedico | when possible, and remove the related code from |
| | SequentialTaskQueue.schedule() since this makes it obsolete. |
| 11053 | Make AsynchronousTask "registered" attributes private since |
| zmedico | poll() and wait() do the job now. Also, hide |
| | SubProcess.files since there's no reason to expose it. |
| 11054 | * Fix --keep-going breakage that occured when |
| zmedico | Scheduler._choose_pkg() was introduced. * Handle multiple |
| | failed builds when buildinding in parallel. * Fix |
| | Scheduler._do_merge_exit() to remove the correct package |
| | since it's not always at position 0 when using --jobs. in |
| | parallel, so |
| 11055 | When there is more than one failure, show a summary packages |
| zmedico | that failed to build or install. This is especially useful |
| | when using --jobs or --keep-going since there can be |
| | multiple failures. |
| 11056 | When packages fail with --jobs > 1 and the "echo" elog |
| zmedico | module is not enabled, emulate elog's mod_echo module to |
| | show the 'error' level messages, which should include the |
| | important die message. This is implemented by adding a |
| | private hook inside elog_process() which the scheduler uses |
| | to collect error messages. |
| 11057 | Since lots of packages will have to be fixed before stable |
| zmedico | portage can abort with "this code is not 64bit clean", |
| | convert the die call to an eerror message instructing the |
| | user to file a bug at bugs.gentoo.org. This should allow us |
| | to stabilize portage-2.2 without having to ensure that the |
| | entire stable tree is 64 bit clean first. |
svn path=/main/branches/prefix/; revision=11058
Fabian Groffen [Mon, 14 Jul 2008 19:20:17 +0000 (19:20 -0000)]
Merged from trunk 11041:11047
| 11042 | Ensure that that Scheduler._main_loop() doesn't return |
| zmedico | before the merge queue is empty. |
| 11043 | Remove redundant failed_packages check from |
| zmedico | Scheduler._main_loop() since it's now handled inside the |
| | _schedule() call. |
| 11044 | Add async execution support for pkg_preinst and |
| zmedico | pkg_postinst. This allows the scheduler's poll loop to run |
| | so that other parallel tasks aren't starved for output |
| | handling while pkg_preinst and pkg_postinst are executing. |
| 11045 | * Fix doebuild so "returnpid" works for preinst and |
| zmedico | postinst. * Add async support for pkg_prerm and pkg_postrm. |
| | * Add missing "treetype" parameter to dblink constructor |
| | calls. * Fix SubProcess._wait() to check self.returncode |
| | _after_ calling the scheduler. |
| 11046 | Call prepare_build_dirs() prior to pkg_preinst so that |
| zmedico | logging works properly. |
| 11047 | Unregister from poll() objects before closing file in order |
| zmedico | to hopefully avoid stale events that can trigger a KeyError |
| | in Scheduler._poll_loop() if the event's file descriptor no |
| | longer has a registered handler. |
svn path=/main/branches/prefix/; revision=11048
Fabian Groffen [Sun, 13 Jul 2008 14:28:38 +0000 (14:28 -0000)]
Merged from trunk 11038:11040
| 11039 | * Since can_poll_pty() is unsafe due to he possibility of |
| zmedico | the poll() call blocking forever, replace it with a simpler |
| | test on /dev/null that's derived from a test in a glib2 |
| | configure script. * Remove PtyReaderTestCase since it has |
| | the same problem as can_poll_pty(). |
| 11040 | Remove all traces of the "poll" parameter for the |
| zmedico | PollScheduler constructor since it was only used by |
| | can_poll_pty(). |
svn path=/main/branches/prefix/; revision=11041
Fabian Groffen [Sun, 13 Jul 2008 11:56:18 +0000 (11:56 -0000)]
Merged from trunk 11036:11037
| 11037 | In can_poll_tty(), direct bash's stderr into the pty since |
| zmedico | we don't want so see any error message it generates (due to |
| | PipeReader closing it's end of the pty). The error message |
| | is irrelevant since all we want to know is whether or not |
| | the data is successfully transmitted through the pty. |
svn path=/main/branches/prefix/; revision=11038
Fabian Groffen [Sun, 13 Jul 2008 11:28:59 +0000 (11:28 -0000)]
Merged from trunk 11017:11035
| 11018 | Fix a few typos found with pyflakes. Thanks to Cardoe for |
| zmedico | reporting. |
| 11019 | Fix SpawnProcess.start() to return early and notify exit |
| zmedico | listeners if the spawn call fails. |
| 11020 | For subclasses of Task, explicitly assign __hash__ = |
| zmedico | Task.__hash__ for compatibility with python-2.6_beta. Thanks |
| | to Arfrever for reporting. |
| 11021 | * Add a digraph.get() method, analogous to dict.get(), used |
| zmedico | to simplify Scheduler._pkg(). * Define digraph.__contains__ |
| | for containment tests via the "in" operator. |
| 11022 | Create a test case for the poll loop which uses the loop to |
| zmedico | read data from a pipe and assert that the data written to |
| | the pipe is identical to the data read from the pipe. In |
| | order to implement this test, several useful classes have |
| | been added: * PipeReader Reads output from one or more files |
| | and saves it in memory, for retrieval via the getvalue() |
| | method. This is driven by the scheduler's poll() loop, so it |
| | runs entirely within the current process. * QueueScheduler |
| | Add instances of SequentialTaskQueue and then call run(). |
| | The run() method returns when no tasks remain. * |
| | TaskScheduler A simple way to handle scheduling of |
| | AsynchrousTask instances. Simply add tasks and call run(). |
| | The run() method returns when no tasks remain. |
| 11023 | Derive a PtyReaderTestCase from PipeReaderTestCase. This |
| zmedico | test is expected to fail on some operating systems such as |
| | Darwin that do not support poll() on pty devices. |
| 11024 | Split out a portage._create_pty_or_pipe() function to avoid |
| zmedico | duplicate code. |
| 11025 | Implement a runtime test that checks whether it's possible |
| zmedico | to use poll() to read data from a pty device. The test only |
| | runs the first time that the can_poll_pty() function is |
| | called (currently happens when the first PollScheduler |
| | instance is created). The test is known to fail on Darwin |
| | and we're not sure how many other kernels might have a |
| | similar problem, this runtime test allows us to avoid having |
| | to maintain special cases for various kernels. |
| 11026 | Set self._mergelist inside Scheduler._calc_resume_list() |
| zmedico | since it needs to be set when calling _set_digraph(). |
| 11027 | Add missing self.wait() calls when tasks return |
| zmedico | unsuccessfully. |
| 11028 | Remember to close file descriptors before returning early |
| zmedico | from can_poll_pty(). |
| 11029 | Rename PollScheduler._schedule() to _schedule_wait(). |
| zmedico | |
| 11030 | * Wrap calls to PollScheduler._schedule_tasks() with a |
| zmedico | _schedule() method that automatically returns early from any |
| | recursive calls that the _schedule_tasks() call might |
| | trigger. This makes _schedule() safe to call from inside |
| | exit listeners. * Call _schedule() inside |
| | Scheduler._job_exit() to trigger a new job when possible. * |
| | Fix SequentialTaskQueue._task_exit() to avoid a potential |
| | KeyError if the task had already been removed from |
| | self.running_tasks for some reason. |
| 11031 | Decrement the job count prior to merging a given package |
| zmedico | (after it's been built or extracted). This will allow |
| | additional jobs to spawn sooner in some cases when there may |
| | be idle resources. |
| 11032 | Fix Scheduler._schedule_tasks() so that it never schedules a |
| zmedico | task when a package has already failed. |
| 11033 | In order to avoid having BlockerDB instantiate a new |
| zmedico | FakeVartree for each merge, add a FakeVartree.sync() method |
| | and so that the same FakeVartree can be reused. Package |
| | counters and timestamps are used to validate package |
| | instances during each sync() call. |
| 11034 | Fix broken binpkg path comparison in in binarytree.inject() |
| zmedico | so that duplicate entries are correctly from the Packages |
| | index. |
| 11035 | Changes to pym/_emerge/__init__.py from the previous commit |
| zmedico | weren't meant to be committed yet. |
svn path=/main/branches/prefix/; revision=11036
Fabian Groffen [Fri, 11 Jul 2008 07:41:11 +0000 (07:41 -0000)]
Merged from trunk 11001:11016
| 11002 | Use the EbuildFetcher class to implement async fetch for the |
| zmedico | EbuildBuild class. When in background mode, fetch output is |
| | logged to emerge-fetch.log. This solves a problem with |
| | calling doebuild(returnpid=1) and then doebuild calling |
| | fetch() synchronously. In that case, doebuild would return |
| | an int if fetch() failed (including when running |
| | pkg_nofetch), which would lead to a TypeError since |
| | doebuild(returnpid=1) is supposed to return a list |
| | containing a spawned pid. |
| 11003 | For parallel-fetch prefetchers, pass fetchonly=1 into the |
| zmedico | EbuildFetcher constructor. |
| 11004 | Bug #231358 - Fix 'NameError: global name 'p' is not |
| zmedico | defined' in getmaskingstatus() that's triggered by a |
| | conditional in LICENSE. |
| 11005 | Like in Binpkg, cancel or wait for the prefetcher in |
| zmedico | EbuildBuild. |
| 11006 | Fix Scheduler._is_restart_necessary() to return False if the |
| zmedico | replacement version of portage is the same exact version as |
| | the currently running instance. |
| 11007 | * Handle errno.ECHILD and errno.ESRCH from waitpid() and |
| zmedico | kill() calls. These errors commonly occur when ^C is used to |
| | kill emerge. * Remove spawned pids from |
| | portage.process.spawned_pids immediately in order to avoid a |
| | race condition for removing it. |
| 11008 | * Fix AsynchronousTask._wait_hook() to prevent potential |
| zmedico | recursive calls triggered when calling exit listeners. * |
| | Make SequentialTaskQueue use exit listeners to trigger |
| | scheduling as soon as a task exits. This makes it possible |
| | for the queues to automatically schedule new tasks while |
| | scheduler remains in it's poll() loop. |
| 11009 | Make digestcheck() skip the "checking ebuild checksums" part |
| zmedico | when fetching in the background. |
| 11010 | Bug #231382 - If the open() call in getconfig() raises an |
| zmedico | unexpected IOError, write a message to stderr showing the |
| | exact call that failed, including the path of the file, in |
| | order to aid in debugging. |
| 11011 | When a package replaces another one, add the corresponding |
| zmedico | uninstall task for the replaced package to |
| | Scheduler._completed_tasks so that it's properly accounted |
| | for in _dependent_on_scheduled_merges(). |
| 11012 | Add support for parallel --regen, using the existing --jobs |
| zmedico | and --load-average options: * Split out a PollLoop base |
| | class from Scheduler and derive a MetadataRegen class to |
| | handle parallel scheduling for --regen. * Add |
| | pordbapi._metadata_process() and _metadata_callback() |
| | methods to implement asynchronous metadata generation. One |
| | method returns an EbuildMetadataPhase instance to |
| | encapsulate the async task. The other method is called to |
| | save the metadata when the process completes successfully. |
| | These methods share code with aux_get to avoid duplicate |
| | code. |
| 11013 | Refactor and simplify the main task scheduling and poll |
| zmedico | loops: * Make output handlers unregister themselves and call |
| | wait() to notify exit listeners immediately. This makes the |
| | exit listeners more useful for scheduling tasks. This makes |
| | the poll loop nice an clean because it just calls the |
| | handlers and then the handlers can do the scheduling when |
| | necessary. * Make SequentialTaskQueue.add() and addFront() |
| | trigger scheduling internally, so that it's more of a chain |
| | reaction than something that has to be done explicitly. |
| 11014 | Document the new --jobs and --load-average options for |
| zmedico | parallelization. |
| 11015 | Call settings.reset() in EbuildMetadataPhase.start() since |
| zmedico | that's what portdbapi does too before generating metadata, |
| | and without it, a previous CATEGORY can leak into the ebuild |
| | env and corrupt it (breaks lots of packages that reference |
| | $CATEGORY). |
| 11016 | Copy INHERITED/_eclasses_ handling from |
| zmedico | portdbapi._metadata_callback() to portdbapi.aux_get() since |
| | it's still needed there or else aux_get won't properly |
| | return INHERITED. |
svn path=/main/branches/prefix/; revision=11017
Fabian Groffen [Wed, 9 Jul 2008 18:19:19 +0000 (18:19 -0000)]
Merged from trunk 10997:10999
| 10998 | Fix SubProcess._poll to handle a missing pid gracefully. |
| zmedico | |
| 10999 | * Unify the scheduler's "build" and "extract" queues into a |
| zmedico | single "jobs" queue. * Add support for logging fetches to |
| | /var/log/emerge-fetch.log when --jobs is enabled. Previously |
| | this log was only used for the parallel-fetch feature but |
| | now it's also used for --jobs. The scheduler's "prefetch" |
| | queue has been renamed to "fetch" since it's not exclusively |
| | used for parallel-fetch anymore. * Pass the "background" |
| | parameter from Binpkg in to the BinpkgFetcher instance, to |
| | send output to emerge-fetch.log instead of stdout. |
svn path=/main/branches/prefix/; revision=11001
Fabian Groffen [Wed, 9 Jul 2008 17:05:16 +0000 (17:05 -0000)]
Merged from trunk 10988:10996
| 10989 | * Call self._wait() when unregistering output handlers, in |
| zmedico | order to avoid triggering a tight loop. Also fix |
| | Subprocess._wait() to only schedule when self.registered is |
| | True. * Add an assertion inside Scheduler._schedule_main() |
| | to try and detect tight loops like the one above. * Fix typo |
| | in Scheduler._merge_exit(). |
| 10990 | Remove redundant call to _schedule_main(). |
| zmedico | |
| 10991 | Fix Scheduler._dependent_on_scheduled_merges() to properly |
| zmedico | distinguish "nomerge" nodes from "uninstall" nodes. |
| 10992 | In scheduler._schedule_main(), only consider a poll() event |
| zmedico | to be a state change if it results in a handler being |
| | unregistered. |
| 10993 | Make the "tight loop" assertion trigger whenever there is no |
| zmedico | state change detected for a given _schedule_main() call. |
| 10994 | * Fix _choose_pkg() to pop the first node if there are no |
| zmedico | other jobs running. * Allow _choose_pkg() to choose an |
| | uninstall node when it reaches the front of the queue. TODO: |
| | fix order for uninstall operations. * Fix |
| | _dependent_on_scheduled_merges() to handle direct circular |
| | deps correctly. |
| 10995 | Fix _prune_digraph() to correctly distinguish "nomerge" |
| zmedico | nodes from "uninstall" nodes. |
| 10996 | Fix Scheduler._choose_pkg() to perform uninstallation |
| zmedico | actions (due to blockers) in the correct order when building |
| | in parallel. |
svn path=/main/branches/prefix/; revision=11000
Fabian Groffen [Wed, 9 Jul 2008 07:17:53 +0000 (07:17 -0000)]
Merged from trunk 10972:10988
| 10973 | Add a new "SRC_URI.mirror" check which checks to if a uri |
| zmedico | listed in profiles/thirdpartymirrors is found in SRC_URI. |
| | Thanks to Betelgeuse for the initial patch which has now |
| | been optimized with a regex. |
| 10974 | * Add new CompositeTask._default_final_exit() method and use |
| zmedico | it as the new generic task exit callback. * Remove erroneous |
| | self.wait() calls inside MergeListItem.start(). |
| 10975 | Rename SubProcess.reg_id to _reg_id since it no longer needs |
| zmedico | to be exposed. |
| 10976 | Add a Scheduler._digraph attribute for use in parallel build |
| zmedico | scheduling. |
| 10977 | Use plain sequential search for the SRC_URI.mirror check |
| zmedico | since the regex approach is actually much slower. |
| 10978 | Add a missing check in Scheduler._main_loop() to ensure that |
| zmedico | max_jobs isn't exceeded. |
| 10979 | Add a MergeListItem.background attribute for putting a task |
| zmedico | in the background when max_jobs > 1. |
| 10980 | Also add "background" attributes to the EbuildBuild and |
| zmedico | Binpkg classes. |
| 10981 | Add AsynchronousTask.background attribute and add support |
| zmedico | for it to the EbuildPhase class. |
| 10982 | * Add background support to the SpawProcess class. * Make |
| zmedico | Binpkg pass the current background setting into subtasks. |
| 10983 | Implement parallel build support by adding new --jobs and |
| zmedico | --load-average options that are analogous to the |
| | corresponding `make` options. Input and output handling |
| | still need work to make it look better and act more friendly |
| | for things like interactive ebuilds that require input. |
| 10984 | Use a normal list instead of a deque for |
| zmedico | Scheduler._pkg_queue since deque doesn't have a remove() |
| | method until python-2.5. |
| 10985 | Fix ValueError triggered when Scheduler._choose_pkg() |
| zmedico | doesn't find a package and returns None. |
| 10986 | Fix infinite loop in CompositeTask._wait(). |
| zmedico | |
| 10987 | Raise an AssertionError in CompositeTask._wait() if it's |
| zmedico | detected that self._current_task hasn't been properly |
| | updated after calling wait on it. |
| 10988 | Derive PackageMerge from AsynchronousTask instead of |
| zmedico | CompositeTask since CompositeTask is designed for |
| | self._current_task to be set. |
svn path=/main/branches/prefix/; revision=10997
Fabian Groffen [Mon, 7 Jul 2008 19:47:15 +0000 (19:47 -0000)]
Merged from trunk 10968:10971
| 10969 | remove TODO list as it's not maintained anyway |
| genone | |
| 10970 | Make SubProcess._wait() call scheduler.schedule(self.reg_id) |
| zmedico | so it's encapsulated and callers don't have to know about |
| | it. |
| 10971 | Move the "registered" attribute to the SubProcess class too. |
| zmedico | |
svn path=/main/branches/prefix/; revision=10972
Fabian Groffen [Mon, 7 Jul 2008 09:18:40 +0000 (09:18 -0000)]
Merged from trunk 10944:10966
| 10945 | Correct TaskSequence docstring to refer to the |
| zmedico | addExitListener() method. |
| 10946 | Call _wait_hook() from poll() if the wait call occurs there. |
| zmedico | |
| 10947 | * Fix AsynchronousTask.poll() to call _wait_hook() when |
| zmedico | necessary. * Use the default poll() and cancel() |
| | implementations for BinpkgVerifier. |
| 10948 | Fix typo in xterm titles total package count which causes it |
| zmedico | to show the current package instead. Thanks to Arfrever for |
| | this patch. |
| 10949 | Add a CompositeTask._assert_current() method that |
| zmedico | asynchronous callbacks can use detect possible bugs. |
| 10950 | Split out a reusable CompositeTask._default_exit() method |
| zmedico | that subclasses can use as a generic task exit callback. |
| 10951 | Add CompositeTask._final_exit() method and use it to fix |
| zmedico | breakage from the previous commit. |
| 10952 | Split out a CompositeTask._start_task() for subclasses to |
| zmedico | use as a generic way to start a task. |
| 10953 | Add async support to EbuildBuild, and an synchronous |
| zmedico | install() method. |
| 10954 | * Fix broke return code handling from previous commit, in |
| zmedico | MergeListItem.execute(). * Fix |
| | TaskSequence._task_exit_handler() so it won't call |
| | final_exit() if _default_exit() has already set |
| | self._current_task to None. |
| 10955 | Fix --getbinpkg to inject downloaded packages into the local |
| zmedico | bintree. |
| 10956 | Make AsynchronousTask subclasses override _wait() and |
| zmedico | _poll() so that calls to public methods can be wrapped for |
| | implementing hooks such as exit listener notification. |
| 10957 | Fix parent class constructor call in the BinpkgFetcher |
| zmedico | constructor. |
| 10958 | Make BinpkgFetcher send output directly to stdout when |
| zmedico | appropriate, so that wget's progress bar works normally. |
| 10959 | Add async support to the Binpkg class. |
| zmedico | |
| 10960 | Add async support to MergeListItem. |
| zmedico | |
| 10961 | Add a PackageMerge class to serve as an asynchronous |
| zmedico | interface to package merges. For now it executes |
| | synchronously inside the start() method. |
| 10962 | * Implement MergeListItem._poll() and _wait(). * Fix |
| zmedico | BinpkgVerifier.start() to call wait() since it's not |
| | asynchronous. |
| 10963 | Fix typo in Binpkg.start() which prevents --genbinpkg |
| zmedico | prefetcher sync from working properly in some cases. |
| 10964 | Fix EbuildPhase._set_returncode() so that it correctly |
| zmedico | updates the returncode attrbute instead of just a local |
| | variable. |
| 10965 | Fix broken code in AsynchronousTask.poll(). |
| zmedico | |
| 10966 | * Implement CompositeTask._poll(). * Make AsynchronousTask |
| zmedico | classes call self.wait() to notify exit listeners. * Rewrite |
| | Scheduler._main_loop() to bring it closer to allowing |
| | parallel build scheduling. |
svn path=/main/branches/prefix/; revision=10968
Fabian Groffen [Mon, 7 Jul 2008 09:11:52 +0000 (09:11 -0000)]
Merged from trunk 10928:10943
| 10929 | Bug #230525 - Work around ObjectProxy breakage in `portageq |
| zmedico | vdb_path`. |
| 10930 | Add a PollConstants class to serve as a layer of indirection |
| zmedico | for accessing select.POLL* attributes which do not exist on |
| | interix. Thanks to grobian for reporting. |
| 10931 | TODO: For parallel scheduling, everything in MergeListItem |
| zmedico | needs asynchronous execution support (start, poll, and wait |
| | methods). |
| 10932 | Convert EbuildFetcher to an asynchronous interface: * Rename |
| zmedico | EbuildFetcher to EbuildFetchPretend. * Rename |
| | EbuildFetcherAsync to EbuildFetcher. |
| 10933 | Make EbuildBuild inherit from EbuildBuildDir since it's |
| zmedico | going to hold the build dir lock while it runs |
| | asynchronously. |
| 10934 | Make Binpkg inherit from EbuildBuildDir since it's going to |
| zmedico | hold the build dir lock while it runs asynchronously. |
| 10935 | * Implement a new binarytree.digestCheck() method and use it |
| zmedico | to check digests for binary packages. * Split out a |
| | AsynchronousTask base class from SubProcess. * Derive a new |
| | BinpkgVerifier class from AsynchronousTask. Even though this |
| | is not really asynchronous yet, it can fake the interface by |
| | doing everything in the start() method. |
| 10936 | Rename BinpkgFetcherAsync class to BinpkgFetcher and use it |
| zmedico | to replace the older class with the same name |
| 10937 | Remove unused variable. |
| zmedico | |
| 10938 | Use the EbuildPhase class to execute the "clean" phase |
| zmedico | asynchronously. |
| 10939 | Add async support to EbuildBinpkg by making it inherit from |
| zmedico | EbuildPhase. |
| 10940 | Add a new CompositeTask class which can be used to combine |
| zmedico | separate AsynchronousTask instances into a single instance. |
| | The CompositeTask instance used task exit listeners as a |
| | means to (asynchronously) trigger progression from one |
| | subtask to the next. This technique is used to group |
| | together all the ebuild phases executed by EbuildExecuter, |
| | and should be useful for grouping many more sets of tasks |
| | into similar composite tasks. |
| 10941 | Fix CompositeTask.cancel() so that it's safe to call when |
| zmedico | there is no running task. |
| 10942 | Make EbuildExecuter inherit from AsynchronousTask. |
| zmedico | |
| 10943 | Split out common code from CompositeTask and EbuildExecuter |
| zmedico | into a new CompositeTask class and rename the old |
| | CompositeTask class to TaskSequence. |
svn path=/main/branches/prefix/; revision=10967
Fabian Groffen [Thu, 3 Jul 2008 17:05:34 +0000 (17:05 -0000)]
Merged from trunk 10904:10913
| 10905 | Split out a new _add_prefetchers() method from |
| zmedico | Scheduler._merge(). |
| 10906 | Remove the Scheduler._post_merge() method, and create a new |
| zmedico | _show_failed_fetches() method. |
| 10907 | Remove unused BinpkgMerge class. |
| zmedico | |
| 10908 | Split out a _restart_if_necessary() method from |
| zmedico | Scheduler._merge(). |
| 10909 | Add a portage._disable_legacy_globals() function. This |
| zmedico | deletes the ObjectProxy instances that are used for lazy |
| | initialization of legacy global variables. The purpose of |
| | deleting them is to prevent new code from referencing these |
| | deprecated variables. This allows the removal of the |
| | PORTAGE_LEGACY_GLOBALS variable which used to serve the same |
| | purpose. |
| 10910 | Clean up Scheduler.merge() and split out a |
| zmedico | _save_resume_list() method. |
| 10911 | Split logging and world atom code out of |
| zmedico | Scheduler._execute() task, and trigger it inside |
| | EbuildPhase.execute(). |
| 10912 | Remove old unused vars inside _execute_task() and fix code |
| zmedico | not to use the old vars. |
| 10913 | Split out a _create_prefetcher() method from |
| zmedico | _add_prefetchers(). |
svn path=/main/branches/prefix/; revision=10914
Fabian Groffen [Thu, 3 Jul 2008 06:49:48 +0000 (06:49 -0000)]
Merged from trunk 10893:10903
| 10894 | Handle missing select.poll on interix by using select() to |
| zmedico | emulate a poll object. |
| 10895 | Fix EbuildBuildDir so that it works properly with binary |
| zmedico | packages. |
| 10896 | Fix broken except statement. Thanks to hawking. |
| zmedico | |
| 10897 | Use stdout.write() instead of "print", for py3k compat. |
| zmedico | |
| 10898 | Split out a SpawnProcess base class out of |
| zmedico | EbuildFetcherAsync. |
| 10899 | Move post ebuild phase code (originally from spawnebuild) |
| zmedico | from EbuildExecuter.execute() to |
| | EbuildPhase._set_returncode(). |
| 10900 | Add missing BinpkgFetcherAsync.registered attribute. |
| zmedico | |
| 10901 | Make BinpkgFetcherAsync inherit from SpawnProcess. |
| zmedico | |
| 10902 | Implement asynchronous binary package extraction, so that |
| zmedico | the scheduler can run while a package is extracting in the |
| | background. |
| 10903 | Split out a new _check_manifests() method from |
| zmedico | Scheduler._merge() since this method is not useful for |
| | parallel scheduling and any useful code needs to be |
| | relocated. |
svn path=/main/branches/prefix/; revision=10904
Fabian Groffen [Wed, 2 Jul 2008 10:49:22 +0000 (10:49 -0000)]
Merged from trunk 10884:10892
| 10885 | * Rename existing EbuildBuild class to EbuildExecuter. * |
| zmedico | Split more code out of Scheduler._execute_task() and use it |
| | to make a more comprehensive EbuildBuild class. |
| 10886 | Remove debugging code from the previous commit. |
| zmedico | |
| 10887 | Split out a Binpkg class from code inside |
| zmedico | Scheduler._execute_task(). |
| 10888 | Fix inverted buildpkgonly logic. |
| zmedico | |
| 10889 | Fix EbuildBuild.execute() to return os.EX_OK on success. |
| zmedico | |
| 10890 | Py3k compatibility patch by Ali Polatel <hawking@g.o>. Don't |
| zmedico | use the format raise Exception, "string" |
| 10891 | Split out a SubProcess base class for EbuildFetcherAsync, |
| zmedico | EbuildPhase, and BinpkgFetcherAsync. |
| 10892 | Remove $PORTAGE_BUILDDIR locking from portage.pkgmerge() and |
| zmedico | use an instance of EbuildBuildDir instead. |
svn path=/main/branches/prefix/; revision=10893
Fabian Groffen [Tue, 1 Jul 2008 18:47:14 +0000 (18:47 -0000)]
it works better if all files are available, and if you reference existing variables
svn path=/main/branches/prefix/; revision=10884
Fabian Groffen [Tue, 1 Jul 2008 18:39:57 +0000 (18:39 -0000)]
make sure we do not clash with standard names for expansion, as e.g. $BASH
svn path=/main/branches/prefix/; revision=10883
Fabian Groffen [Tue, 1 Jul 2008 17:40:20 +0000 (17:40 -0000)]
my attempt to do python 2.6 compatibility thing
svn path=/main/branches/prefix/; revision=10882
Fabian Groffen [Tue, 1 Jul 2008 17:32:45 +0000 (17:32 -0000)]
Merged from trunk 10869:10875
| 10870 | Py3k compatibility patch #1 by Ali Polatel <hawking@g.o>. |
| zmedico | Replace dict.has_key() calls with "in" and "not in" |
| | operators.. |
| 10871 | Py3k compatibility patch #2 by Ali Polatel <hawking@g.o>. |
| zmedico | Replace dict.has_key() calls with "in" and "not in" |
| | operators. |
| 10872 | Py3k compatibility patch #3 by Ali Polatel <hawking@g.o>. |
| zmedico | Replace dict.has_key() calls with "in" and "not in" |
| | operators. |
| 10873 | Py3k compatibility patch #4 by Ali Polatel <hawking@g.o>. |
| zmedico | Replace dict.has_key() calls with "in" and "not in" |
| | operators. |
| 10874 | Py3k compatibility patch #5 by Ali Polatel <hawking@g.o>. |
| zmedico | Replace dict.has_key() calls with "in" and "not in" |
| | operators. |
| 10875 | Py3k compatibility patch #6 by Ali Polatel <hawking@g.o>. |
| zmedico | Replace dict.has_key() calls with "in" and "not in" |
| | operators. |
svn path=/main/branches/prefix/; revision=10881
Fabian Groffen [Tue, 1 Jul 2008 17:22:29 +0000 (17:22 -0000)]
Merged from trunk 10853:10869
| 10854 | Avoid python-2.6 deprecation warnings for md5 and sha |
| zmedico | modules by trying to import hashlib first and then falling |
| | back to the deprecated modules if necessary. Thanks to |
| | ColdWind for reporting. |
| 10855 | Reimplement parallel-fetch by spawning the `ebuild fetch` |
| zmedico | command for each ebuild. The benefit of using this approach |
| | is that it can be integrated together with parallel build |
| | scheduling that's planned. Parallel-fetch support for |
| | binhost is not implemented yet, though it worked previously. |
| 10856 | Clear the self._task_queue to avoid duplicate parallel-fetch |
| zmedico | tasks in --keep-going mode. |
| 10857 | Add "(no inline comments)" to qualify "comments begin with |
| zmedico | #" statements. |
| 10858 | Bug #230245 - Pass the correct directory when calling `snv |
| zmedico | list` and `svn status` since repoman supports category-level |
| | and repo-level commits. |
| 10859 | Bug #230245 - Use os.path.basename() on paths returned from |
| zmedico | `svn list` and `svn status`. |
| 10860 | Bug #230249 - Disable the "ebuild.notadded" check when not |
| zmedico | in commit mode and running `svn list` and `svn status` calls |
| | in every package dir will be too expensive. |
| 10861 | Fix typo. |
| zmedico | |
| 10862 | add a call to pruneNonExisting() at the end of |
| zmedico | dbapi.vartree.PreservedLibsRegistry.__init__() |
| 10864 | Split out a write_contents() function and a |
| zmedico | vardbapi.removeFromContents() function. This is refactoring |
| | of code from the blocker file collision contents handling in |
| | dblink.treewalk(). Also, there is a new |
| | dblink._match_contents() method derived from isowner(). It |
| | returns the exact path from the contents file that matches |
| | the given path, regardless of path differences due to things |
| | such as symlinks. |
| 10865 | Handle potential errors in PreservedLibsRegistry.store() now |
| zmedico | that it can be called via pruneNonExisting(), due to things |
| | such as portageq calls where the user may not have write |
| | permission to the registry. |
| 10866 | Also avoid sandbox violations in |
| zmedico | PreservedLibsRegistry.store(), for running portage inside |
| | ebuild phases. |
| 10867 | Never do realpath() on an empty string for |
| zmedico | portdbapi.porttree_root since otherwise it can evaluate to |
| | $CWD which leads to undesireable results. |
| 10868 | Add a new BinpkgFetcherAsync class and use it to implement |
| zmedico | parellel-fetch for --getbinpkg. |
| 10869 | Add a "prefix" keyword parameter to slot_dict_class() which |
| zmedico | controls the prefix used when mapping attribute names from |
| | keys. Use this to change the syntax from files["foo"] to |
| | files.foo (it's fewer characters to look at). |
svn path=/main/branches/prefix/; revision=10880
Fabian Groffen [Tue, 1 Jul 2008 17:09:13 +0000 (17:09 -0000)]
Merged from trunk 10843:10853
| 10844 | Handle invalid atoms inside depgraph._add_pkg_deps(), to |
| zmedico | avoid an unhandled InvalidAtom exception from dep_expand() |
| | when matching against the vardb. |
| 10845 | Split out a _spawn_misc_sh() function from spawnebuild(). |
| zmedico | |
| 10846 | Split out a _post_src_install_uid_fix() function from |
| zmedico | spawnebuild(). |
| 10847 | Split out a _check_build_log() function from spawnebuild(). |
| zmedico | |
| 10848 | In EbuildBuild.execute(), call doebuild() separately for |
| zmedico | each phase since it will be easier to add asynchronous |
| | support to doebuild() if it's called this way. |
| 10849 | * Add "fd_pipes" and "returnpid" parameters to doebuild() |
| zmedico | and pass these into spawn calls, enabling ebuild processes |
| | to execute asynchronously. * Add a EbuildPhase class that's |
| | derived from the pty logging code inside portage.spawn(). * |
| | Integrate post-phase code from spawnebuild() into |
| | EbuildBuild.execute() so that it still gets called even |
| | though doebuild() calls execute asynchronously. |
| 10850 | Add EMERGE_FROM to the blacklisted vars in the config |
| zmedico | constructor. |
| 10851 | Centralize select.poll() event handling in |
| zmedico | MergeTask._schedule(). This will allow the parent process to |
| | handle output of multiple child processes running in |
| | parllel. |
| 10852 | * Tweak conditionals in doebuild() to fix emerge --fetch |
| zmedico | breakage reported by Arfrever. * Change a couple EMERGE_FROM |
| | conditionals to use the "returnpid" parameter instead. |
| 10853 | Rename the MergeTask class to Scheduler. |
| zmedico | |
svn path=/main/branches/prefix/; revision=10879
Fabian Groffen [Tue, 1 Jul 2008 16:59:40 +0000 (16:59 -0000)]
Merged from trunk 10833:10843
| 10834 | Use the EbuildMerge class instead of portage.merge(). |
| zmedico | |
| 10835 | Bug #229859 - Make portageq check atom validity when |
| zmedico | has_version() and best_version() are called from inside the |
| | ebuild environment. |
| 10836 | Copy portageq exit code validation from has_version() to |
| zmedico | best_version(). |
| 10837 | Bug #229859 - Also add atom validation to match(). |
| zmedico | |
| 10838 | * Remove unused EbuildBuild.pretend attribute. * Add TODO |
| zmedico | not about asynchronous execution. |
| 10839 | Make EbuildBinpkg delegate everything up to src_install() to |
| zmedico | the EbuildBuild class. |
| 10840 | Handle re.error exceptions raised from re.compile() when |
| zmedico | ACCEPT_CHOSTS is invalid. Thanks to solar for reporting. |
| 10841 | Split out an EbuildBuildDir class to handle locking of the |
| zmedico | build dir. |
| 10842 | Instead of calling the ebuild "clean" phase in |
| zmedico | MergeTask.merge(), move it inside doebuild() where it can be |
| | tied into the actionmap_deps framework. |
| 10843 | Move clean code, from the previous commit, from doebuild() |
| zmedico | to EbuildBuild.execute() where it fits better. |
svn path=/main/branches/prefix/; revision=10878
Fabian Groffen [Tue, 1 Jul 2008 16:55:38 +0000 (16:55 -0000)]
Merged from trunk 10824:10833
| 10825 | Split the MergeTask._merge() method into 3 smaller methods. |
| zmedico | The new methods are _execute_task() and _post_merge(). |
| 10826 | Split out an EbuildFetcher class. |
| zmedico | |
| 10827 | Split out a BinpkgFetcher class. |
| zmedico | |
| 10828 | * Split out a BinpkgMerge class. * Fix typos in |
| zmedico | _get_hash_key() methods. |
| 10829 | Split out an EbuildMerge class. |
| zmedico | |
| 10830 | Fix typo. |
| zmedico | |
| 10831 | Split out an EbuildBuild class. |
| zmedico | |
| 10832 | * Split out an EbuildBinpkg class. * Remove unused |
| zmedico | EbuildBuild.ldpath_mtimes attribute. |
| 10833 | Split out a PackageUninstall class. |
| zmedico | |
svn path=/main/branches/prefix/; revision=10877
Fabian Groffen [Tue, 1 Jul 2008 16:48:46 +0000 (16:48 -0000)]
Merged from trunk 10816:10824
| 10817 | Generate and log eerror messages for any packages that need |
| zmedico | to be dropped due to unsatisfied dependencies when |
| | --keep-going is enabled. |
| 10818 | Add documentation for the new --keep-going option. |
| zmedico | |
| 10819 | Fix typo. |
| zmedico | |
| 10820 | Pass the digraph into the MergeTask constructor. It's |
| zmedico | currently unused but it will be useful for parallel |
| | scheduling. |
| 10821 | Break references from Package instances in the digraph |
| zmedico | before passing it into the MergeTask constructor. |
| 10822 | Remove the MergeTask._blocker_db attribute since it can just |
| zmedico | be a local variable. |
| 10823 | Fix depgraph.break_refs() to work with DependencyArg |
| zmedico | instances (or anything else that may have a "root_config" |
| | attribute). |
| 10824 | Bug #226307 - Copy come code from |
| zmedico | depgraph._iter_atoms_for_pkg() that was used to solve bug |
| | #218854, and use it inside unmerge() when matching sets to |
| | packages. |
svn path=/main/branches/prefix/; revision=10876
Fabian Groffen [Tue, 1 Jul 2008 07:26:37 +0000 (07:26 -0000)]
Do not use hardcoded python, bug #230247
svn path=/main/branches/prefix/; revision=10863
Fabian Groffen [Fri, 27 Jun 2008 15:55:18 +0000 (15:55 -0000)]
more windows user/groupname quoting
svn path=/main/branches/prefix/; revision=10816
Fabian Groffen [Fri, 27 Jun 2008 14:53:20 +0000 (14:53 -0000)]
Merged from trunk 10801:10812
| 10802 | Handle KeyError from aux_get() inside |
| zmedico | dbapi._iter_match_slot() and _iter_match_use(). Thanks to |
| | grobian for reporting. |
| 10803 | Use settings["ROOT"] at the beginning of env_update() to |
| zmedico | avoid breakage due to "root" being an ObjectProxy instance. |
| 10804 | * Remove PORTAGE_LEGACY_GLOBALS hack for portage import |
| zmedico | since late initialization of portage.settings (via |
| | ObjectProxy) allows us to rely on being able to access the |
| | portage.exception namespace before portage.settings is |
| | initialized. * Use portage.settings["ROOT"] instead of |
| | portage.root to avoid potential ObjectProxy compatibility |
| | issues. |
| 10805 | Fix the PermissionDenied message for color.map. |
| zmedico | |
| 10806 | Add "PATH" to the set of allowed pacakge metadata keys for |
| zmedico | the PackageIndex. |
| 10807 | Call use_reduce() with matchall=1 when checking depstrings. |
| zmedico | Otherwise it's possible for invalid deps to be ignored. |
| | Thanks to grobian for reporting. |
| 10808 | For brevity, and consistency with pkgcore, rename |
| zmedico | "all-installed" to just "installed". Thanks to ferringb for |
| | the suggestion. |
| 10809 | Split the --skipfirst code into a resume_depgraph() function |
| zmedico | that will be useful for implementing a --keep-going option |
| | (bug #12768). |
| 10810 | Use bool() to simplify ObjectProxy.__nonzero__(). |
| zmedico | |
| 10811 | Bug #12768 - Add a --keep-going option. This uses the same |
| zmedico | resume_depgraph() function that's called for emerge --resume |
| | operations. Given the remaining tasks, depedencies are |
| | recalculated and any tasks with unsatisfied dependencies are |
| | automatically dropped. |
| 10812 | Call clear_caches() after the depgraph is out of scope, in |
| zmedico | MergeTask.merge(). |
svn path=/main/branches/prefix/; revision=10815
Fabian Groffen [Fri, 27 Jun 2008 14:47:19 +0000 (14:47 -0000)]
Merged from trunk 10788:10801
| 10789 | Swap out the whole Package.root_config attribute inside |
| zmedico | depgraph.break_refs(). |
| 10790 | Add a generic portage.cache.mappings.slot_dict_class() |
| zmedico | function which generates mapping classes that behave similar |
| | to a dict but store values as object attributes that are |
| | allocated via __slots__. Instances of these objects have a |
| | smaller memory footprint than a normal dict object. These |
| | classes are used to reduce the memory footprint of the |
| | dbapi.aux_get() caches and the Package.metadata attribute. |
| 10791 | Fix indentation. |
| zmedico | |
| 10792 | Fix KeyError constructor style. |
| zmedico | |
| 10793 | Fix typo. |
| zmedico | |
| 10794 | Add docstring to slot_dict_class(). |
| zmedico | |
| 10795 | Fix typo. |
| zmedico | |
| 10796 | Use SlotDict where appropriate in binarytree._populate(). |
| zmedico | |
| 10797 | Make PackageIndex use SlotDict for package metadata storage. |
| zmedico | The set of allowed keys is passed into the PackageIndex |
| | constructor (normal dict instances will be used if the set |
| | of keys is not passed in for some reason). A |
| | SlotDict.allowed_keys attribute now provides access to a |
| | frozenset of allowed keys. |
| 10798 | Implement lazy initialization of global "portdb", "settings" |
| zmedico | and other variables that pollute the portage module. This |
| | works by initializing the global variables with dummy |
| | "proxy" objects that serve as a means to trigger lazy |
| | initialization. As soon as the first attribute access or |
| | method call occurs on one of the proxy objects, it causes |
| | all the proxy objects to be replaced with the real ones. |
| | It's possible for an unsupported attribute access or method |
| | call on a proxy object to trigger an error, leading to |
| | breakage. However, hopefully these such corner cases will |
| | negligible (only time will tell). |
| 10799 | Use a separate proxy type to trigger portage.portdb |
| zmedico | initialization separately from the rest of the legacy global |
| | variables. This can be useful since sometimes the other |
| | variables are needed while the portdb is not. |
| 10800 | Also use a separate proxy type for portage.mtimedb since |
| zmedico | it's independent from the portdb and settings. |
| 10801 | change default name of EverythingSet to @all-installed (bug |
| genone | #229467) |
svn path=/main/branches/prefix/; revision=10814
Fabian Groffen [Fri, 27 Jun 2008 12:11:33 +0000 (12:11 -0000)]
need to quote more, on Windows user and groupnames can have spaces :(
svn path=/main/branches/prefix/; revision=10813
Fabian Groffen [Wed, 25 Jun 2008 20:26:36 +0000 (20:26 -0000)]
Merged from trunk 10784:10787
| 10785 | Fix Atom -> str breakage in depgraph.saveNomergeFavorites(). |
| zmedico | Thanks to grobian and darsiide. |
| 10786 | Clarify EverythingSet.description a bit. Thanks to Necoro. |
| zmedico | |
| 10787 | Clarify EverythingSet.description a bit. Thanks to Necoro. |
| zmedico | |
svn path=/main/branches/prefix/; revision=10788
Fabian Groffen [Wed, 25 Jun 2008 18:49:44 +0000 (18:49 -0000)]
Merged from trunk 10781:10783
| 10782 | After the depgraph caches the vardb state in a FakeVartree, |
| zmedico | to make some room on the heap, clear the vardbapi caches. |
| 10783 | Fix repoman breakage wrt _PackageMetadataWrapper._keys |
| zmedico | values. |
svn path=/main/branches/prefix/; revision=10784
Fabian Groffen [Wed, 25 Jun 2008 09:00:23 +0000 (09:00 -0000)]
Merged from trunk 10770:10780
| 10771 | Bug #229233 - Fix Atom -> str breakage in the |
| zmedico | depgraph.display_problems() package.provided warning |
| | message. |
| 10772 | Use weakref.WeakValueDictionary to make cached Atom |
| zmedico | instances eligible for garbage collection when no strong |
| | references remain. |
| 10773 | If dep calculation time exceeds 20 seconds then |
| zmedico | automatically enable "complete" mode since any performance |
| | difference is not as likely to be noticed by the user after |
| | this much time has passed. |
| 10774 | Fix spelling of depgraph._complete_threshold. |
| zmedico | |
| 10775 | Bug #229069 - Before deleting the depgraph, break references |
| zmedico | pointing to the depgraph from Package instances in the merge |
| | list. This helps reduce the heap size a lot. |
| 10776 | For the Package.metadata attribute, only implement the dict |
| zmedico | interface instead of actually inheriting from dict. This |
| | slightly decreases the memory footprint by defining |
| | __slots__ and storing items as object attributes. |
| 10777 | Remove the BlockerDB._blocker_cache attribute and just |
| zmedico | create new instances on demand instead of taking space on |
| | the heap. |
| 10778 | Make clear_caches() call portage.dircache.clear(). |
| zmedico | |
| 10779 | Use os.listdir() instead of portage.listdir() to avoid |
| zmedico | needless caching of directories that only need to be listed |
| | once. |
| 10780 | Implement __contains__, pop() and clear() for |
| zmedico | Package.metadata attributes. |
svn path=/main/branches/prefix/; revision=10781
projects / portage.git / log