portage.git
16 years agoBug #199788 - Make repoman parse GLEP 56 USE flag descriptions from
Zac Medico [Fri, 18 Jul 2008 12:40:58 +0000 (12:40 -0000)]
Bug #199788 - Make repoman parse GLEP 56 USE flag descriptions from
metadata.xml (in addition to use.local.desc). Thanks to Doug Goldstein
<cardoe@g.o> for this patch.

svn path=/main/trunk/; revision=11126

16 years agoFix typo.
Zac Medico [Fri, 18 Jul 2008 08:56:07 +0000 (08:56 -0000)]
Fix typo.

svn path=/main/trunk/; revision=11124

16 years agoSuppress counter initialization warning messages in
Zac Medico [Fri, 18 Jul 2008 07:47:12 +0000 (07:47 -0000)]
Suppress counter initialization warning messages in
vardbapi.get_counter_tick_core() in cases when the vdb is empty. This
avoids pointless error messages when merging packages into a new root,
like when building stage 1.

svn path=/main/trunk/; revision=11123

16 years agoAdd background output redirection and async ebuild phase support for
Zac Medico [Fri, 18 Jul 2008 03:45:44 +0000 (03:45 -0000)]
Add background output redirection and async ebuild phase support for
PackageUninstall (automatic uninstalls that solve blockers).

svn path=/main/trunk/; revision=11122

16 years agoChange package status messages to give more accurate information about what
Zac Medico [Fri, 18 Jul 2008 01:42:38 +0000 (01:42 -0000)]
Change package status messages to give more accurate information about what
is happening at a given time:

 * Change "Emerging" messages to "Building" for ebuilds and "Extracting" for
   binary packages.

 * Change "Merging" messages to "Installing" and "Uninstalling".

svn path=/main/trunk/; revision=11121

16 years agoIn dblink._dblink_display_merge(), when in background mode and no log is
Zac Medico [Fri, 18 Jul 2008 00:52:45 +0000 (00:52 -0000)]
In dblink._dblink_display_merge(), when in background mode and no log is
available, suppress output when level < logging.WARNING. This hides
">>> Regenerating /etc/ld.so.cache..." messages that appear after postrm
when PORT_LOGDIR is unset (the log file if it exists is removed after
postrm so there's no place to log the message).

svn path=/main/trunk/; revision=11120

16 years agoUse dblink._eerror() to eliminate duplicate code.
Zac Medico [Fri, 18 Jul 2008 00:34:42 +0000 (00:34 -0000)]
Use dblink._eerror() to eliminate duplicate code.

svn path=/main/trunk/; revision=11119

16 years agoIf an problem is detected in dblink._security_check(), log it via elog so
Zac Medico [Fri, 18 Jul 2008 00:13:17 +0000 (00:13 -0000)]
If an problem is detected in dblink._security_check(), log it via elog so
that it's properly displayed in the foreground even when in background mode.

svn path=/main/trunk/; revision=11118

16 years agoRedirect dblink elog messages from stdout to the log file when in background
Zac Medico [Thu, 17 Jul 2008 23:53:49 +0000 (23:53 -0000)]
Redirect dblink elog messages from stdout to the log file when in background
mode. This is implemented by adding a new "out" parameter to the
portage.elog.e* functions, which can be used to send output to an arbitary
file instead of stdout.

svn path=/main/trunk/; revision=11117

16 years agoWhen dblink is looping over files for merge/unmerge, temporarily yield to the
Zac Medico [Thu, 17 Jul 2008 22:41:04 +0000 (22:41 -0000)]
When dblink is looping over files for merge/unmerge, temporarily yield to the
scheduler each time a fixed number of files are processed (currently 20).
This gives the scheduler an opportunity to service pending poll events. This
is implemented with a new PollScheduler._schedule_yield() method which calls
poll() exactly once, without blocking, and any services any resulting poll
events.

svn path=/main/trunk/; revision=11116

16 years agoAlso, reset "merges" in JobStatusDisplay.reset().
Zac Medico [Thu, 17 Jul 2008 16:11:13 +0000 (16:11 -0000)]
Also, reset "merges" in JobStatusDisplay.reset().

svn path=/main/trunk/; revision=11115

16 years agoInitialize JobStatusDisplay.merges since it's not initialized with the bound
Zac Medico [Thu, 17 Jul 2008 16:09:15 +0000 (16:09 -0000)]
Initialize JobStatusDisplay.merges since it's not initialized with the bound
properties anymore.

svn path=/main/trunk/; revision=11114

16 years agoAlso, don't count "merges" as changes in the status display.
Zac Medico [Thu, 17 Jul 2008 16:05:27 +0000 (16:05 -0000)]
Also, don't count "merges" as changes in the status display.

svn path=/main/trunk/; revision=11113

16 years agoSince it's not very interesting, disable the "merges" part of status display.
Zac Medico [Thu, 17 Jul 2008 16:04:00 +0000 (16:04 -0000)]
Since it's not very interesting, disable the "merges" part of status display.

svn path=/main/trunk/; revision=11112

16 years agoFix identation (tabnanny complains about this) branches/prefix r11110
Zac Medico [Thu, 17 Jul 2008 14:21:58 +0000 (14:21 -0000)]
Fix identation (tabnanny complains about this) branches/prefix r11110

svn path=/main/trunk/; revision=11111

16 years agoReformat the status display. Here's an example:
Zac Medico [Thu, 17 Jul 2008 14:01:22 +0000 (14:01 -0000)]
Reformat the status display. Here's an example:

>>> Jobs: 0 of 2 complete, 2 running             Load average: 2.9, 2.4, 2.0
>>> Jobs: 0 of 2 complete, 1 running, 2 merges   Load average: 2.8, 2.4, 2.0
>>> Jobs: 2 of 2 complete                        Load average: 2.6, 2.3, 2.0

svn path=/main/trunk/; revision=11107

16 years agoDon't show "Merging" and "merged." messages when in background mode.
Zac Medico [Thu, 17 Jul 2008 13:08:11 +0000 (13:08 -0000)]
Don't show "Merging" and "merged." messages when in background mode.

svn path=/main/trunk/; revision=11105

16 years agoMake Binpkg send "Extracting" messages to the log file when in background
Zac Medico [Thu, 17 Jul 2008 13:02:59 +0000 (13:02 -0000)]
Make Binpkg send "Extracting" messages to the log file when in background
mode, instead of stdout.

svn path=/main/trunk/; revision=11104

16 years agoUpdate the status display in _build_exit(), after decrementing the job count.
Zac Medico [Thu, 17 Jul 2008 12:21:09 +0000 (12:21 -0000)]
Update the status display in _build_exit(), after decrementing the job count.

svn path=/main/trunk/; revision=11103

16 years agoMake BinpkgVerifier redirect output to the fetch log when in background mode.
Zac Medico [Thu, 17 Jul 2008 11:39:16 +0000 (11:39 -0000)]
Make BinpkgVerifier redirect output to the fetch log when in background mode.

svn path=/main/trunk/; revision=11102

16 years agoDon't show a blank newline before ">>> Emerging" when in background mode.
Zac Medico [Thu, 17 Jul 2008 11:21:17 +0000 (11:21 -0000)]
Don't show a blank newline before ">>> Emerging" when in background mode.

svn path=/main/trunk/; revision=11101

16 years agoHandle the case in dblink.unmerge() when the log file is a temp file and so
Zac Medico [Thu, 17 Jul 2008 10:57:38 +0000 (10:57 -0000)]
Handle the case in dblink.unmerge() when the log file is a temp file and so
it doesn't exist after postrm cleaning.

svn path=/main/trunk/; revision=11100

16 years agoTweak interaction between Scheduler and status display so that it doesn't
Zac Medico [Thu, 17 Jul 2008 10:40:44 +0000 (10:40 -0000)]
Tweak interaction between Scheduler and status display so that it doesn't
display more often than necessary and avoids showing inconsistent counts.

svn path=/main/trunk/; revision=11099

16 years ago* Add a new portage.util.writemsg_level() function which uses a "level"
Zac Medico [Thu, 17 Jul 2008 10:11:26 +0000 (10:11 -0000)]
* Add a new portage.util.writemsg_level() function which uses a "level"
  parameter to control whether the message is sent to stdout or stderr.
  When level >= logging.WARNING then the message is sent to stderr,
  otherwise it is sent to stdout. The noiselevel is passed directly to
  writemsg().

* Use writemsg_level() for handling merge messages from dblink and add
  appropriate noiselevel values to each call.

* Add a writemsg_level parameter to portage.env_update() so that dblink
  can pass in dblink._display_merge() as a means to redirect output back
  to the scheduler for appropriate handling when running in the background.

svn path=/main/trunk/; revision=11098

16 years agoDon't increment the "complete" count for uninstall tasks.
Zac Medico [Thu, 17 Jul 2008 09:57:30 +0000 (09:57 -0000)]
Don't increment the "complete" count for uninstall tasks.

svn path=/main/trunk/; revision=11097

16 years agoAdd a JobStatusDisplay class to encapsulate the status display. This object
Zac Medico [Thu, 17 Jul 2008 08:57:19 +0000 (08:57 -0000)]
Add a JobStatusDisplay class to encapsulate the status display. This object
tracks whether or not any of it's attributes have changed since the last
time it was displayed, so the it's only displayed when something has changed.
Unlike before, now the display should always update whenever a relevant change
occurs.

svn path=/main/trunk/; revision=11096

16 years agoIn SequentialTaskQueue.clear(), remove the exit listener from each task before
Zac Medico [Thu, 17 Jul 2008 06:31:00 +0000 (06:31 -0000)]
In SequentialTaskQueue.clear(), remove the exit listener from each task before
cancelling it.

svn path=/main/trunk/; revision=11095

16 years agoAdd the new --anon option to the getopt args.
Zac Medico [Thu, 17 Jul 2008 04:50:49 +0000 (04:50 -0000)]
Add the new --anon option to the getopt args.

svn path=/main/trunk/; revision=11094

16 years agoWhen SpawnProcess._start() spawns a process in the background, use /dev/null
Zac Medico [Thu, 17 Jul 2008 04:20:12 +0000 (04:20 -0000)]
When SpawnProcess._start() spawns a process in the background, use /dev/null
so that any attempts to read from stdin will immediately return EOF instead
of blocking indefinitely. TODO: Use job control functions like tcsetpgrp()
to control access to stdin.

svn path=/main/trunk/; revision=11093

16 years agoEliminate duplicate code by making EbuildProcess inherit from SpawnProcess
Zac Medico [Thu, 17 Jul 2008 03:52:24 +0000 (03:52 -0000)]
Eliminate duplicate code by making EbuildProcess inherit from SpawnProcess
instead of SubProcess.

svn path=/main/trunk/; revision=11092

16 years agoAdd --anon to the usage message.
Zac Medico [Wed, 16 Jul 2008 21:57:04 +0000 (21:57 -0000)]
Add --anon to the usage message.

svn path=/main/trunk/; revision=11091

16 years agoAdd --anon option for anonsvn.gentoo.org users.
Zac Medico [Wed, 16 Jul 2008 21:52:01 +0000 (21:52 -0000)]
Add --anon option for anonsvn.gentoo.org users.

svn path=/main/trunk/; revision=11090

16 years agoRemove TODO since it no longer exists.
Zac Medico [Wed, 16 Jul 2008 21:11:32 +0000 (21:11 -0000)]
Remove TODO since it no longer exists.

svn path=/main/trunk/; revision=11089

16 years agoAvoid raising an IndexError from deprecated_profile_check() if the deprecated
Zac Medico [Wed, 16 Jul 2008 20:43:12 +0000 (20:43 -0000)]
Avoid raising an IndexError from deprecated_profile_check() if the deprecated
file happens to be empty. Instead just refer to the "Gentoo Upgrading Guide".
Thanks to armin76 for reporting.

svn path=/main/trunk/; revision=11088

16 years agoPass background=True into prefetcher constructors.
Zac Medico [Wed, 16 Jul 2008 08:47:54 +0000 (08:47 -0000)]
Pass background=True into prefetcher constructors.

svn path=/main/trunk/; revision=11076

16 years agoAdd a progress counter showing "x of y complete" to
Zac Medico [Wed, 16 Jul 2008 08:37:30 +0000 (08:37 -0000)]
Add a progress counter showing "x of y complete" to
Scheduler._display_status().

svn path=/main/trunk/; revision=11075

16 years ago* Add xterm titles support to Scheduler._display_status().
Zac Medico [Wed, 16 Jul 2008 07:38:07 +0000 (07:38 -0000)]
* Add xterm titles support to Scheduler._display_status().
* Tweak emergelog() so that it won't interfere with the scheduler summary.

svn path=/main/trunk/; revision=11073

16 years agoAdd async support for calling functions in misc-functions.sh:
Zac Medico [Wed, 16 Jul 2008 07:09:51 +0000 (07:09 -0000)]
Add async support for calling functions in misc-functions.sh:

* Create a new MiscFunctionsProcess class derived from SpawnProcess.

* Add "background" support to the SpawnProcess class and split out a
  _spawn() template function so MiscFunctionsProcess can override it.

* Rename the existing EbuildPhase class to EbuildProcess.

* Create a new EbuildPhase class derived from CompositeTask. This class
  spawns the EbuildProcess and MiscFunctionsProcess classes as sub-tasks.

svn path=/main/trunk/; revision=11072

16 years agoRemove the config._filter_calling_env attribute since it's not really needed.
Zac Medico [Wed, 16 Jul 2008 03:32:53 +0000 (03:32 -0000)]
Remove the config._filter_calling_env attribute since it's not really needed.
Replace it with an simple os.path.exists() call inside config.environ().

svn path=/main/trunk/; revision=11071

16 years agoShow LDFLAGS in emerge --info <pkg> output. Thanks to dirtyepic for this
Zac Medico [Wed, 16 Jul 2008 02:38:53 +0000 (02:38 -0000)]
Show LDFLAGS in emerge --info <pkg> output. Thanks to dirtyepic for this
patch.

svn path=/main/trunk/; revision=11070

16 years agoPass the whole dblink instance into Scheduler._dblink_display_merge() in case
Zac Medico [Wed, 16 Jul 2008 01:42:42 +0000 (01:42 -0000)]
Pass the whole dblink instance into Scheduler._dblink_display_merge() in case
it's ever needed.

svn path=/main/trunk/; revision=11069

16 years agoRedirect dblink.merge() output to the log file when --jobs > 1.
Zac Medico [Wed, 16 Jul 2008 01:37:27 +0000 (01:37 -0000)]
Redirect dblink.merge() output to the log file when --jobs > 1.

svn path=/main/trunk/; revision=11068

16 years agoMake dblink._display_merge() detect scheduler presence internally.
Zac Medico [Wed, 16 Jul 2008 01:15:18 +0000 (01:15 -0000)]
Make dblink._display_merge() detect scheduler presence internally.

svn path=/main/trunk/; revision=11067

16 years agoAdapt the dblink merge display callback to use log levels from the logging
Zac Medico [Wed, 16 Jul 2008 01:09:46 +0000 (01:09 -0000)]
Adapt the dblink merge display callback to use log levels from the logging
module so that it can be used as an interface for both writemsg() and
writemsg_stdout().

svn path=/main/trunk/; revision=11066

16 years agoRedirect dblink.unmerge() output to the log file when --jobs > 1.
Zac Medico [Wed, 16 Jul 2008 00:44:58 +0000 (00:44 -0000)]
Redirect dblink.unmerge() output to the log file when --jobs > 1.

svn path=/main/trunk/; revision=11065

16 years agoWhen --jobs > 1, display a status message each time that
Zac Medico [Wed, 16 Jul 2008 00:11:44 +0000 (00:11 -0000)]
When --jobs > 1, display a status message each time that
Scheduler._schedule_tasks() is called. For example:

>>> Jobs:  2 running,  0 merges, load average: 0.34, 0.46, 0.27

svn path=/main/trunk/; revision=11064

16 years agoAdd some cross references between emerge's new parallel build options and
Zac Medico [Tue, 15 Jul 2008 23:19:16 +0000 (23:19 -0000)]
Add some cross references between emerge's new parallel build options and
MAKEOPTS.

svn path=/main/trunk/; revision=11063

16 years agoMake EbuildFetcher copy FETCHCOMMAND and RESUMECOMMAND to the fetch
Zac Medico [Tue, 15 Jul 2008 22:31:13 +0000 (22:31 -0000)]
Make EbuildFetcher copy FETCHCOMMAND and RESUMECOMMAND to the fetch
environment, in case the default values have been overridden.

svn path=/main/trunk/; revision=11062

16 years agoIn Scheduler._restart_if_necessary(), use list.remove() instead of assuming
Zac Medico [Tue, 15 Jul 2008 21:17:37 +0000 (21:17 -0000)]
In Scheduler._restart_if_necessary(), use list.remove() instead of assuming
that portage is first in the resume list.

svn path=/main/trunk/; revision=11061

16 years agoSince lots of packages will have to be fixed before stable portage can abort
Zac Medico [Tue, 15 Jul 2008 10:45:42 +0000 (10:45 -0000)]
Since lots of packages will have to be fixed before stable portage can abort
with "this code is not 64bit clean", convert the die call to an eerror message
instructing the user to file a bug at bugs.gentoo.org. This should allow us
to stabilize portage-2.2 without having to ensure that the entire stable tree
is 64 bit clean first.

svn path=/main/trunk/; revision=11057

16 years agoWhen packages fail with --jobs > 1 and the "echo" elog module is not enabled,
Zac Medico [Tue, 15 Jul 2008 09:50:41 +0000 (09:50 -0000)]
When packages fail with --jobs > 1 and the "echo" elog module is not enabled,
emulate elog's mod_echo module to show the 'error' level messages, which should
include the important die message. This is implemented by adding a private
hook inside elog_process() which the scheduler uses to collect error messages.

svn path=/main/trunk/; revision=11056

16 years agoWhen there is more than one failure, show a summary packages that failed to
Zac Medico [Tue, 15 Jul 2008 08:58:05 +0000 (08:58 -0000)]
When there is more than one failure, show a summary packages that failed to
build or install. This is especially useful when using --jobs or --keep-going
since there can be multiple failures.

svn path=/main/trunk/; revision=11055

16 years ago* Fix --keep-going breakage that occured when Scheduler._choose_pkg() was
Zac Medico [Tue, 15 Jul 2008 08:28:16 +0000 (08:28 -0000)]
* Fix --keep-going breakage that occured when Scheduler._choose_pkg() was
  introduced.

* Handle multiple failed builds when buildinding in parallel.

* Fix Scheduler._do_merge_exit() to remove the correct package since it's
  not always at position 0 when using --jobs.

in parallel, so

svn path=/main/trunk/; revision=11054

16 years agoMake AsynchronousTask "registered" attributes private since poll() and wait()
Zac Medico [Tue, 15 Jul 2008 03:03:42 +0000 (03:03 -0000)]
Make AsynchronousTask "registered" attributes private since poll() and wait()
do the job now. Also, hide SubProcess.files since there's no reason to expose
it.

svn path=/main/trunk/; revision=11053

16 years agoUse self.registered to return early from SubProcess.poll() when possible,
Zac Medico [Tue, 15 Jul 2008 02:49:31 +0000 (02:49 -0000)]
Use self.registered to return early from SubProcess.poll() when possible,
and remove the related code from SequentialTaskQueue.schedule() since
this makes it obsolete.

svn path=/main/trunk/; revision=11052

16 years agoAdd a PollScheduler._next_poll_event() method to maintain a central event
Zac Medico [Tue, 15 Jul 2008 02:11:39 +0000 (02:11 -0000)]
Add a PollScheduler._next_poll_event() method to maintain a central event
queue between _poll_loop() and _schedule_wait(). This solves an intermittent
KeyError from _poll_loop() which occured when an event handler called
_schedule_wait() which unregistered an event handler for an event that
_poll_loop() had already recieved but hadn't processed yet.

svn path=/main/trunk/; revision=11051

16 years agoSplit out a _main_loop_cleanup() method from Scheduler._merge() and call it
Zac Medico [Tue, 15 Jul 2008 00:50:13 +0000 (00:50 -0000)]
Split out a _main_loop_cleanup() method from Scheduler._merge() and call it
before restarting in _restart_if_necessary() to kill any parallel-fetch tasks
that may still be running.

svn path=/main/trunk/; revision=11050

16 years agoSplit out a generic SubProcess._unregister() method that's called from
Zac Medico [Mon, 14 Jul 2008 23:31:51 +0000 (23:31 -0000)]
Split out a generic SubProcess._unregister() method that's called from
_wait() to make sure everything is cleaned up before wait() returns.
This might solve an intermittent KeyError that is sometimes raised from
the main poll() loop when a handler is no longer registered for a poll()
event that has occurred.

svn path=/main/trunk/; revision=11049

16 years agoUnregister from poll() objects before closing file in order to hopefully
Zac Medico [Mon, 14 Jul 2008 09:00:03 +0000 (09:00 -0000)]
Unregister from poll() objects before closing file in order to hopefully
avoid stale events that can trigger a KeyError in Scheduler._poll_loop()
if the event's file descriptor no longer has a registered handler.

svn path=/main/trunk/; revision=11047

16 years agoCall prepare_build_dirs() prior to pkg_preinst so that logging works
Zac Medico [Mon, 14 Jul 2008 06:48:10 +0000 (06:48 -0000)]
Call prepare_build_dirs() prior to pkg_preinst so that logging works
properly.

svn path=/main/trunk/; revision=11046

16 years ago* Fix doebuild so "returnpid" works for preinst and postinst.
Zac Medico [Mon, 14 Jul 2008 06:38:30 +0000 (06:38 -0000)]
* Fix doebuild so "returnpid" works for preinst and postinst.
* Add async support for pkg_prerm and pkg_postrm.
* Add missing "treetype" parameter to dblink constructor calls.
* Fix SubProcess._wait() to check self.returncode _after_ calling
  the scheduler.

svn path=/main/trunk/; revision=11045

16 years agoAdd async execution support for pkg_preinst and pkg_postinst. This allows the
Zac Medico [Mon, 14 Jul 2008 03:47:47 +0000 (03:47 -0000)]
Add async execution support for pkg_preinst and pkg_postinst. This allows the
scheduler's poll loop to run so that other parallel tasks aren't starved for
output handling while pkg_preinst and pkg_postinst are executing.

svn path=/main/trunk/; revision=11044

16 years agoRemove redundant failed_packages check from Scheduler._main_loop() since
Zac Medico [Sun, 13 Jul 2008 23:25:23 +0000 (23:25 -0000)]
Remove redundant failed_packages check from Scheduler._main_loop() since
it's now handled inside the _schedule() call.

svn path=/main/trunk/; revision=11043

16 years agoEnsure that that Scheduler._main_loop() doesn't return before the merge
Zac Medico [Sun, 13 Jul 2008 22:13:52 +0000 (22:13 -0000)]
Ensure that that Scheduler._main_loop() doesn't return before the merge
queue is empty.

svn path=/main/trunk/; revision=11042

16 years agoRemove all traces of the "poll" parameter for the PollScheduler constructor
Zac Medico [Sun, 13 Jul 2008 13:38:24 +0000 (13:38 -0000)]
Remove all traces of the "poll" parameter for the PollScheduler constructor
since it was only used by can_poll_pty().

svn path=/main/trunk/; revision=11040

16 years ago* Since can_poll_pty() is unsafe due to he possibility of the poll() call
Zac Medico [Sun, 13 Jul 2008 13:24:39 +0000 (13:24 -0000)]
* Since can_poll_pty() is unsafe due to he possibility of the poll() call
  blocking forever, replace it with a simpler test on /dev/null that's
  derived from a test in a glib2 configure script.

* Remove PtyReaderTestCase since it has the same problem as can_poll_pty().

svn path=/main/trunk/; revision=11039

16 years agoIn can_poll_tty(), direct bash's stderr into the pty since we don't want
Zac Medico [Sun, 13 Jul 2008 11:54:13 +0000 (11:54 -0000)]
In can_poll_tty(), direct bash's stderr into the pty since we don't want
so see any error message it generates (due to PipeReader closing it's
end of the pty). The error message is irrelevant since all we want to
know is whether or not the data is successfully transmitted through the
pty.

svn path=/main/trunk/; revision=11037

16 years agoChanges to pym/_emerge/__init__.py from the previous commit weren't meant
Zac Medico [Sun, 13 Jul 2008 07:34:06 +0000 (07:34 -0000)]
Changes to pym/_emerge/__init__.py from the previous commit weren't meant
to be committed yet.

svn path=/main/trunk/; revision=11035

16 years agoFix broken binpkg path comparison in in binarytree.inject() so that duplicate
Zac Medico [Sun, 13 Jul 2008 07:30:43 +0000 (07:30 -0000)]
Fix broken binpkg path comparison in in binarytree.inject() so that duplicate
entries are correctly from the Packages index.

svn path=/main/trunk/; revision=11034

16 years agoIn order to avoid having BlockerDB instantiate a new FakeVartree for each
Zac Medico [Sun, 13 Jul 2008 05:29:35 +0000 (05:29 -0000)]
In order to avoid having BlockerDB instantiate a new FakeVartree for each
merge, add a FakeVartree.sync() method and so that the same FakeVartree
can be reused. Package counters and timestamps are used to validate package
instances during each sync() call.

svn path=/main/trunk/; revision=11033

16 years agoFix Scheduler._schedule_tasks() so that it never schedules a task when a
Zac Medico [Sun, 13 Jul 2008 04:30:06 +0000 (04:30 -0000)]
Fix Scheduler._schedule_tasks() so that it never schedules a task when a
package has already failed.

svn path=/main/trunk/; revision=11032

16 years agoDecrement the job count prior to merging a given package (after it's been
Zac Medico [Sun, 13 Jul 2008 01:10:45 +0000 (01:10 -0000)]
Decrement the job count prior to merging a given package (after it's been
built or extracted). This will allow additional jobs to spawn sooner in
some cases when there may be idle resources.

svn path=/main/trunk/; revision=11031

16 years ago* Wrap calls to PollScheduler._schedule_tasks() with a _schedule() method
Zac Medico [Sat, 12 Jul 2008 23:45:29 +0000 (23:45 -0000)]
* Wrap calls to PollScheduler._schedule_tasks() with a _schedule() method
  that automatically returns early from any recursive calls that the
  _schedule_tasks() call might trigger. This makes _schedule() safe to call from
  inside exit listeners.

* Call _schedule() inside Scheduler._job_exit() to trigger a new job when
  possible.

* Fix SequentialTaskQueue._task_exit() to avoid a potential KeyError if the
  task had already been removed from self.running_tasks for some reason.

svn path=/main/trunk/; revision=11030

16 years agoRename PollScheduler._schedule() to _schedule_wait().
Zac Medico [Sat, 12 Jul 2008 23:19:12 +0000 (23:19 -0000)]
Rename PollScheduler._schedule() to _schedule_wait().

svn path=/main/trunk/; revision=11029

16 years agoRemember to close file descriptors before returning early from can_poll_pty().
Zac Medico [Sat, 12 Jul 2008 22:54:35 +0000 (22:54 -0000)]
Remember to close file descriptors before returning early from can_poll_pty().

svn path=/main/trunk/; revision=11028

16 years agoAdd missing self.wait() calls when tasks return unsuccessfully.
Zac Medico [Sat, 12 Jul 2008 11:52:40 +0000 (11:52 -0000)]
Add missing self.wait() calls when tasks return unsuccessfully.

svn path=/main/trunk/; revision=11027

16 years agoSet self._mergelist inside Scheduler._calc_resume_list() since it needs to
Zac Medico [Sat, 12 Jul 2008 11:36:41 +0000 (11:36 -0000)]
Set self._mergelist inside Scheduler._calc_resume_list() since it needs to
be set when calling _set_digraph().

svn path=/main/trunk/; revision=11026

16 years agoImplement a runtime test that checks whether it's possible to use poll() to
Zac Medico [Sat, 12 Jul 2008 10:17:55 +0000 (10:17 -0000)]
Implement a runtime test that checks whether it's possible to use poll() to
read data from a pty device. The test only runs the first time that the
can_poll_pty() function is called (currently happens when the first
PollScheduler instance is created). The test is known to fail on Darwin and
we're not sure how many other kernels might have a similar problem, this
runtime test allows us to avoid having to maintain special cases for various
kernels.

svn path=/main/trunk/; revision=11025

16 years agoSplit out a portage._create_pty_or_pipe() function to avoid duplicate code.
Zac Medico [Sat, 12 Jul 2008 08:43:29 +0000 (08:43 -0000)]
Split out a portage._create_pty_or_pipe() function to avoid duplicate code.

svn path=/main/trunk/; revision=11024

16 years agoDerive a PtyReaderTestCase from PipeReaderTestCase. This test is expected to
Zac Medico [Sat, 12 Jul 2008 07:32:55 +0000 (07:32 -0000)]
Derive a PtyReaderTestCase from PipeReaderTestCase. This test is expected to
fail on some operating systems such as Darwin that do not support poll() on
pty devices.

svn path=/main/trunk/; revision=11023

16 years agoCreate a test case for the poll loop which uses the loop to read data from a
Zac Medico [Sat, 12 Jul 2008 06:22:26 +0000 (06:22 -0000)]
Create a test case for the poll loop which uses the loop to read data from a
pipe and assert that the data written to the pipe is identical to the data
read from the pipe. In order to implement this test, several useful classes
have been added:

 * PipeReader

     Reads output from one or more files and saves it in memory,
     for retrieval via the getvalue() method. This is driven by
     the scheduler's poll() loop, so it runs entirely within the
     current process.

 * QueueScheduler

     Add instances of SequentialTaskQueue and then call run().
     The run() method returns when no tasks remain.

 * TaskScheduler

     A simple way to handle scheduling of AsynchrousTask instances. Simply
     add tasks and call run(). The run() method returns when no tasks remain.

svn path=/main/trunk/; revision=11022

16 years ago* Add a digraph.get() method, analogous to dict.get(), used to simplify
Zac Medico [Sat, 12 Jul 2008 01:58:23 +0000 (01:58 -0000)]
* Add a digraph.get() method, analogous to dict.get(),  used to simplify
  Scheduler._pkg().

* Define digraph.__contains__ for containment tests via the "in" operator.

svn path=/main/trunk/; revision=11021

16 years agoFor subclasses of Task, explicitly assign __hash__ = Task.__hash__ for
Zac Medico [Sat, 12 Jul 2008 00:03:01 +0000 (00:03 -0000)]
For subclasses of Task, explicitly assign __hash__ = Task.__hash__ for
compatibility with python-2.6_beta. Thanks to Arfrever for reporting.

svn path=/main/trunk/; revision=11020

16 years agoFix SpawnProcess.start() to return early and notify exit listeners if the
Zac Medico [Fri, 11 Jul 2008 22:40:26 +0000 (22:40 -0000)]
Fix SpawnProcess.start() to return early and notify exit listeners if the
spawn call fails.

svn path=/main/trunk/; revision=11019

16 years agoFix a few typos found with pyflakes. Thanks to Cardoe for reporting.
Zac Medico [Fri, 11 Jul 2008 21:28:15 +0000 (21:28 -0000)]
Fix a few typos found with pyflakes. Thanks to Cardoe for reporting.

svn path=/main/trunk/; revision=11018

16 years agoCopy INHERITED/_eclasses_ handling from portdbapi._metadata_callback() to
Zac Medico [Fri, 11 Jul 2008 06:02:40 +0000 (06:02 -0000)]
Copy INHERITED/_eclasses_ handling from portdbapi._metadata_callback() to
portdbapi.aux_get() since it's still needed there or else aux_get won't
properly return INHERITED.

svn path=/main/trunk/; revision=11016

16 years agoCall settings.reset() in EbuildMetadataPhase.start() since that's what
Zac Medico [Fri, 11 Jul 2008 05:12:00 +0000 (05:12 -0000)]
Call settings.reset() in EbuildMetadataPhase.start() since that's what
portdbapi does too before generating metadata, and without it, a previous
CATEGORY can leak into the ebuild env and corrupt it (breaks lots of packages
that reference $CATEGORY).

svn path=/main/trunk/; revision=11015

16 years agoDocument the new --jobs and --load-average options for parallelization.
Zac Medico [Fri, 11 Jul 2008 04:16:17 +0000 (04:16 -0000)]
Document the new --jobs and --load-average options for parallelization.

svn path=/main/trunk/; revision=11014

16 years agoRefactor and simplify the main task scheduling and poll loops:
Zac Medico [Fri, 11 Jul 2008 02:38:35 +0000 (02:38 -0000)]
Refactor and simplify the main task scheduling and poll loops:

* Make output handlers unregister themselves and call wait() to notify
  exit listeners immediately. This makes the exit listeners more useful
  for scheduling tasks. This makes the poll loop nice an clean because
  it just calls the handlers and then the handlers can do the scheduling
  when necessary.

* Make SequentialTaskQueue.add() and addFront() trigger scheduling
  internally, so that it's more of a chain reaction than something that has
  to be done explicitly.

svn path=/main/trunk/; revision=11013

16 years agoAdd support for parallel --regen, using the existing --jobs and --load-average
Zac Medico [Thu, 10 Jul 2008 15:00:17 +0000 (15:00 -0000)]
Add support for parallel --regen, using the existing --jobs and --load-average
options:

 * Split out a PollLoop base class from Scheduler and derive a MetadataRegen
   class to handle parallel scheduling for --regen.

 * Add pordbapi._metadata_process() and _metadata_callback() methods to
   implement asynchronous metadata generation. One method returns an
   EbuildMetadataPhase instance to encapsulate the async task. The other
   method is called to save the metadata when the process completes
   successfully. These methods share code with aux_get to avoid duplicate
   code.

svn path=/main/trunk/; revision=11012

16 years agoWhen a package replaces another one, add the corresponding uninstall task
Zac Medico [Thu, 10 Jul 2008 10:33:03 +0000 (10:33 -0000)]
When a package replaces another one, add the corresponding uninstall task
for the replaced package to Scheduler._completed_tasks so that it's properly
accounted for in _dependent_on_scheduled_merges().

svn path=/main/trunk/; revision=11011

16 years agoBug #231382 - If the open() call in getconfig() raises an unexpected IOError,
Zac Medico [Thu, 10 Jul 2008 09:08:39 +0000 (09:08 -0000)]
Bug #231382 - If the open() call in getconfig() raises an unexpected IOError,
write a message to stderr showing the exact call that failed, including the
path of the file, in order to aid in debugging.

svn path=/main/trunk/; revision=11010

16 years agoMake digestcheck() skip the "checking ebuild checksums" part when fetching in
Zac Medico [Thu, 10 Jul 2008 04:41:27 +0000 (04:41 -0000)]
Make digestcheck() skip the "checking ebuild checksums" part when fetching in
the background.

svn path=/main/trunk/; revision=11009

16 years ago* Fix AsynchronousTask._wait_hook() to prevent potential recursive calls
Zac Medico [Thu, 10 Jul 2008 03:40:39 +0000 (03:40 -0000)]
* Fix AsynchronousTask._wait_hook() to prevent potential recursive calls
  triggered when calling exit listeners.

* Make SequentialTaskQueue use exit listeners to trigger scheduling as
  soon as a task exits. This makes it possible for the queues to
  automatically schedule new tasks while scheduler remains in it's
  poll() loop.

svn path=/main/trunk/; revision=11008

16 years ago* Handle errno.ECHILD and errno.ESRCH from waitpid() and kill() calls. These
Zac Medico [Thu, 10 Jul 2008 01:24:43 +0000 (01:24 -0000)]
* Handle errno.ECHILD and errno.ESRCH from waitpid() and kill() calls. These
  errors commonly occur when ^C is used to kill emerge.

* Remove spawned pids from portage.process.spawned_pids immediately in order
  to avoid a race condition for removing it.

svn path=/main/trunk/; revision=11007

16 years agoFix Scheduler._is_restart_necessary() to return False if the replacement
Zac Medico [Thu, 10 Jul 2008 00:58:08 +0000 (00:58 -0000)]
Fix Scheduler._is_restart_necessary() to return False if the replacement
version of portage is the same exact version as the currently running
instance.

svn path=/main/trunk/; revision=11006

16 years agoLike in Binpkg, cancel or wait for the prefetcher in EbuildBuild.
Zac Medico [Wed, 9 Jul 2008 23:53:10 +0000 (23:53 -0000)]
Like in Binpkg, cancel or wait for the prefetcher in EbuildBuild.

svn path=/main/trunk/; revision=11005

16 years agoBug #231358 - Fix 'NameError: global name 'p' is not defined' in
Zac Medico [Wed, 9 Jul 2008 23:20:32 +0000 (23:20 -0000)]
Bug #231358 - Fix 'NameError: global name 'p' is not defined' in
getmaskingstatus() that's triggered by a conditional in LICENSE.

svn path=/main/trunk/; revision=11004

16 years agoFor parallel-fetch prefetchers, pass fetchonly=1 into the EbuildFetcher
Zac Medico [Wed, 9 Jul 2008 21:51:25 +0000 (21:51 -0000)]
For parallel-fetch prefetchers, pass fetchonly=1 into the EbuildFetcher
constructor.

svn path=/main/trunk/; revision=11003

16 years agoUse the EbuildFetcher class to implement async fetch for the EbuildBuild
Zac Medico [Wed, 9 Jul 2008 21:39:39 +0000 (21:39 -0000)]
Use the EbuildFetcher class to implement async fetch for the EbuildBuild
class. When in background mode, fetch output is logged to emerge-fetch.log.
This solves a problem with calling doebuild(returnpid=1) and then doebuild
calling fetch() synchronously. In that case, doebuild would return an int
if fetch() failed (including when running pkg_nofetch), which would lead
to a TypeError since doebuild(returnpid=1) is supposed to return a list
containing a spawned pid.

svn path=/main/trunk/; revision=11002