Martin von Gagern [Tue, 13 Mar 2012 17:15:26 +0000 (10:15 -0700)]
Display: spawn pkg_nofetch when appropriate
Signed-off-by: Zac Medico <zmedico@gentoo.org>
Mike Frysinger [Mon, 12 Mar 2012 05:29:32 +0000 (01:29 -0400)]
QA: detect pkg-config files that bleed LDFLAGS
Signed-off-by: Mike Frysinger <vapier@gentoo.org>
Mike Frysinger [Mon, 12 Mar 2012 04:40:12 +0000 (00:40 -0400)]
etc-update: ignore changes in whitespace (-b) for automerge
Signed-off-by: Mike Frysinger <vapier@gentoo.org>
Mike Frysinger [Sun, 11 Mar 2012 04:15:31 +0000 (23:15 -0500)]
portageq: add "colormap" helper
Signed-off-by: Mike Frysinger <vapier@gentoo.org>
Mike Frysinger [Sun, 11 Mar 2012 19:39:50 +0000 (15:39 -0400)]
etc-update: fix logic bug with protected files
The rewrite introduced a slight bug with protected files. Fix that,
and rework the code slightly to make it obvious that we want a diff
set of options for files and dirs.
Signed-off-by: Mike Frysinger <vapier@gentoo.org>
Mike Frysinger [Sun, 11 Mar 2012 03:57:43 +0000 (22:57 -0500)]
etc-update: handle symlinked files
If the config file is symlinked somewhere, then update the target rather
then discarding the symlink and replacing it with a real file.
URL: http://bugs.gentoo.org/330221
Reported-by: Jack Byer <ftn768@gmail.com>
Signed-off-by: Mike Frysinger <vapier@gentoo.org>
Mike Frysinger [Sun, 11 Mar 2012 03:23:25 +0000 (22:23 -0500)]
etc-update: fix default "enter" behavior
Adding support for -[3579] command line flags inadvertently broke the
default behavior of hitting "enter" to automatically select the next
file. Make this work again.
Signed-off-by: Mike Frysinger <vapier@gentoo.org>
Mike Frysinger [Sun, 11 Mar 2012 02:54:59 +0000 (21:54 -0500)]
etc-update: add a --preen flag
Add a flag to merge just trivial updates and then quit.
URL: http://bugs.gentoo.org/159080
Reported-by: Iván Pérez Domínguez <iperez@babel.ls.fi.upm.es>
Signed-off-by: Mike Frysinger <vapier@gentoo.org>
Mike Frysinger [Sun, 11 Mar 2012 02:43:30 +0000 (21:43 -0500)]
etc-update: support scan paths on the command line
URL: http://bugs.gentoo.org/59235
Reported-by: Cory Visi <merlin@gentoo.org>
Signed-off-by: Mike Frysinger <vapier@gentoo.org>
Mike Frysinger [Thu, 8 Mar 2012 21:47:16 +0000 (16:47 -0500)]
dispatch-conf: do regex matching ourselves
This avoids having to pipe through multiple greps, as well as running
diff multiple times on the same set of files.
Signed-off-by: Mike Frysinger <vapier@gentoo.org>
Zac Medico [Sat, 10 Mar 2012 15:22:46 +0000 (07:22 -0800)]
ebuild.sh: skip EBUILD_HOOK_DIR for depend phase
Zac Medico [Fri, 9 Mar 2012 06:35:43 +0000 (22:35 -0800)]
install-info: avoid unnecessary shell
Zac Medico [Fri, 9 Mar 2012 05:08:40 +0000 (21:08 -0800)]
Exit after killed while waiting on prefetch.
Zac Medico [Fri, 9 Mar 2012 02:54:02 +0000 (18:54 -0800)]
Use Popen to avoid unnecessary shell.
Zac Medico [Thu, 8 Mar 2012 23:55:39 +0000 (15:55 -0800)]
Disable 'illegally inherited' check for nofetch.
It's disabled for nofetch, since that can be called by repoman and
that triggers bug #407449 due to repoman not exporting non-essential
variables such as INHERITED.
Mike Frysinger [Thu, 8 Mar 2012 20:19:55 +0000 (15:19 -0500)]
dispatch-conf: add --help/--version flags
Signed-off-by: Mike Frysinger <vapier@gentoo.org>
Mike Frysinger [Thu, 8 Mar 2012 20:13:05 +0000 (15:13 -0500)]
etc-update(1): flesh out a bit more
Signed-off-by: Mike Frysinger <vapier@gentoo.org>
Mike Frysinger [Thu, 8 Mar 2012 18:45:29 +0000 (13:45 -0500)]
etc-update: turn -[3579] flags into command line option
This lets people do `etc-update --automode -3` rather than having
to do something like `echo "-3" | etc-update`.
Signed-off-by: Mike Frysinger <vapier@gentoo.org>
Mike Frysinger [Thu, 8 Mar 2012 05:11:56 +0000 (00:11 -0500)]
etc-update: rewrite significantly
This tool is very inefficient, especially in the case of auto-merging
with -3/-5 and such. Rewrite the code to kill off dead paths, compress
multiple greps into a single sed, cache more calculated results, and
use bash code where possible rather than execing external tools. There
are also style updates here, many of which fix quoting issues and other
such things.
The next rewrite should probably be in python, but this version is
fast enough for me now even on my slow machines.
Signed-off-by: Mike Frysinger <vapier@gentoo.org>
Zac Medico [Mon, 5 Mar 2012 09:55:39 +0000 (01:55 -0800)]
whirlpool.py: fix WhirlpoolAdd for bug #406407
This file is a python port of Whirlpool.c the reference implementation:
http://www.larc.usp.br/~pbarreto/whirlpool.zip
Comparison of both implementations reveals a difference in loop logic
at the very beginning of the WhirlpoolAdd function, which is fixed now.
Zac Medico [Mon, 5 Mar 2012 07:30:28 +0000 (23:30 -0800)]
pruneNonExisting: handle eselect-opengl symlinks
Only count symlinks as preserved if they still point to a hardink
in the same directory, in order to handle cases where a tool such
as eselect-opengl has updated the symlink to point to a hardlink
in a different directory (see bug #406837). The unused hardlink is
automatically found by _find_unused_preserved_libs, since the soname
symlink no longer points to it. After the hardlink is removed by
_remove_preserved_libs, it calls pruneNonExisting which eliminates
the irrelevant symlink from the registry here.
Zac Medico [Mon, 5 Mar 2012 00:48:41 +0000 (16:48 -0800)]
repoman: say Git >=1.7.9 needed for signed commit
Zac Medico [Mon, 5 Mar 2012 00:01:29 +0000 (16:01 -0800)]
repoman: support overlays without repo_name
Zac Medico [Wed, 29 Feb 2012 15:49:25 +0000 (07:49 -0800)]
action_build: return 1 for "no" --ask answer
Zac Medico [Tue, 28 Feb 2012 04:57:35 +0000 (20:57 -0800)]
cvstree.getentries: handle "ignored" files in cvs
It's possible for files to be under version control even though they
match our ignore filter, so don't ignore them if they are listed in the
"Entries" file. Thanks to Mike Gilbert <floppym@gentoo.org> for
reporting in this blog post:
http://floppym.blogspot.com/2012/02/cvs-status-display-cvs-checkout-in-svn.html
Sebastian Luther [Sun, 26 Feb 2012 09:34:31 +0000 (10:34 +0100)]
autounmask: Avoid unmasking live versions if possible
Before this patch the allowed changes were:
1. USE
2. USE + ~arch + license
3. USE + ~arch + license + missing keywords + masks
With this patch:
1. USE
2. USE + ~arch + license
3. USE + ~arch + license + missing keywords
4. USE + ~arch + license + masks
5. USE + ~arch + license + missing keywords + masks
This avoids unmasking live versions, which are typically masked
and have missing keywords to be avoided if there is a regular
masked version available.
Sebastian Luther [Sun, 26 Feb 2012 08:49:13 +0000 (09:49 +0100)]
Reorganize how autounmask allows changes to be made
This patch does not change emerge's behaviour.
Sebastian Luther [Fri, 24 Feb 2012 21:13:52 +0000 (22:13 +0100)]
autounmask: Fix USE change messages to display the correct parent
Fixes bug 399863.
Zac Medico [Thu, 23 Feb 2012 05:30:19 +0000 (21:30 -0800)]
Unmerge orphan directory symlinks for bug 384397.
Zac Medico [Thu, 23 Feb 2012 03:07:11 +0000 (19:07 -0800)]
dblink: split out _unmerge_dirs method
This code will need to be called twice when safely unmerging symlinks
to directories (bug #384397).
Zac Medico [Wed, 22 Feb 2012 20:50:48 +0000 (12:50 -0800)]
ecompressdir: handle abs symlink, bug #405327
Zac Medico [Mon, 20 Feb 2012 11:01:12 +0000 (03:01 -0800)]
Scheduler: hang in _schedule_tasks, bug 404995
This is triggered whenever emerge bails out while parallel-fetch is
running in the background.
Zac Medico [Sat, 18 Feb 2012 04:03:41 +0000 (20:03 -0800)]
_eventloop: use explicit relative imports
Zac Medico [Sat, 18 Feb 2012 03:40:03 +0000 (19:40 -0800)]
MtimeDB: add JSON read/write
Support serialization as JSON instead of pickle, so that
/var/cache/edb/mtimedb is human readable/writable, for those rare
cases where it may be useful. Currently, pickle is still used for
writes. The plan is to migrate to JSON after JSON read has been
supported for some time.
Zac Medico [Sat, 18 Feb 2012 02:41:27 +0000 (18:41 -0800)]
MtimeDB._load: explicitly close file
Zac Medico [Sat, 18 Feb 2012 02:01:27 +0000 (18:01 -0800)]
PreservedLibsRegistry: add JSON read/write
Support serialization as JSON instead of pickle, so that
/var/lib/portage/preserved_libs_registry is human readable/writable,
for those rare cases where it may be useful. Currently, pickle is still
used for writes. The plan is to migrate to JSON after JSON read has
been supported for some time.
Zac Medico [Fri, 17 Feb 2012 23:56:56 +0000 (15:56 -0800)]
repoman: make virtual.oldstyle an error
Also, add "allow-provide-virtuals = true" setting for
metadata/layout.conf which reduces it to a warning.
Zac Medico [Fri, 17 Feb 2012 23:33:35 +0000 (15:33 -0800)]
portage.5: layout.conf manifest-hashes example
Zac Medico [Fri, 17 Feb 2012 23:23:39 +0000 (15:23 -0800)]
PollSelectAdapter: don't inherit PollConstants
Zac Medico [Fri, 17 Feb 2012 22:56:21 +0000 (14:56 -0800)]
Move Poll{Constants,SelectAdapter} to _eventloop.
Zac Medico [Fri, 17 Feb 2012 22:49:36 +0000 (14:49 -0800)]
Move _emerge.SlotObject to portage.util.
Zac Medico [Fri, 17 Feb 2012 22:31:37 +0000 (14:31 -0800)]
SlotObject: validate __slots__ and keyword args
Zac Medico [Fri, 17 Feb 2012 22:18:05 +0000 (14:18 -0800)]
Remove redundant inherited __slots__ values.
Zac Medico [Fri, 17 Feb 2012 22:16:59 +0000 (14:16 -0800)]
EventLoop: fix _io_handler_class "f" attribute
Arfrever Frehtes Taifersar Arahesis [Fri, 17 Feb 2012 21:33:13 +0000 (22:33 +0100)]
portage.package.ebuild.fetch.fetch(): Fix ResourceWarning with Python 3.2.
Zac Medico [Fri, 17 Feb 2012 10:20:35 +0000 (02:20 -0800)]
EventLoop.iteration: run timeouts last
Run timeouts last, in order to minimize latency in termination of
iteration loops that they may control.
Zac Medico [Fri, 17 Feb 2012 10:04:28 +0000 (02:04 -0800)]
SubProcess._waitpid_cb: fix args for glib compat
Zac Medico [Fri, 17 Feb 2012 09:29:15 +0000 (01:29 -0800)]
EventLoop: prune obsolete code
Zac Medico [Fri, 17 Feb 2012 06:29:17 +0000 (22:29 -0800)]
EventLoop.iteration: poll for blocking, not sleep
The effect is be mostly the same, but it's more conistent to use
_do_poll for all blocking, plus it has EINTR handling.
Zac Medico [Fri, 17 Feb 2012 06:03:30 +0000 (22:03 -0800)]
EventLoop.child_watch_add: dynamic IO watch
The IO watch is dynamically registered and unregistered as needed,
since we don't want to consider it as a valid source of events when
there are no child listeners. It's important to distinguish when there
are no valid sources of IO events, in order to avoid an endless poll
call if there's no timeout. This fixes possbible endless poll calls
since commit
1979a6cdfcd8c6bae4565982d82d862be07ba5be.
Zac Medico [Fri, 17 Feb 2012 03:08:07 +0000 (19:08 -0800)]
EventLoop: wakeup poll loop to receive sigchild
TODO: Find out why SIGCHLD signals aren't delivered during poll calls,
forcing us to wakeup in order to receive them. This fixes random hangs
in poll calls since commit
1979a6cdfcd8c6bae4565982d82d862be07ba5be.
Zac Medico [Thu, 16 Feb 2012 21:42:59 +0000 (13:42 -0800)]
EventLoop: fix signal race in _sigchld_init
Zac Medico [Thu, 16 Feb 2012 21:35:23 +0000 (13:35 -0800)]
EventLoop: fix _sigchld_io_cb to return True
This fixes an infinite loop since commit
1979a6cdfcd8c6bae4565982d82d862be07ba5be.
Zac Medico [Thu, 16 Feb 2012 20:58:29 +0000 (12:58 -0800)]
SubProcess: use child_watch_add
This fixes performance issues introduced by commit
9c664779a16f6cbca8a5ffe7f6b0c68572819723.
Zac Medico [Thu, 16 Feb 2012 20:56:50 +0000 (12:56 -0800)]
EventLoop: implement child_watch_add
Arfrever Frehtes Taifersar Arahesis [Thu, 16 Feb 2012 19:43:00 +0000 (20:43 +0100)]
portage.update.fixdbentries(): Fix ResourceWarnings with Python 3.2.
Zac Medico [Thu, 16 Feb 2012 05:13:38 +0000 (21:13 -0800)]
SubProcess: use non-blocking waitpid
This ensures that the EventLoop will not stop due to a waitpid call
blocking forever.
Zac Medico [Thu, 16 Feb 2012 04:58:10 +0000 (20:58 -0800)]
EventLoop.iteration: sleep if no IO handlers
Sleep so that we don't waste cpu time by looping too quickly. This
makes EventLoop useful for code that needs to wait for timeout
callbacks regardless of whether or not any IO handlers are currently
registered.
Zac Medico [Thu, 16 Feb 2012 04:41:16 +0000 (20:41 -0800)]
EventLoop.timeout_add: fix inverted min interval
Zac Medico [Thu, 16 Feb 2012 00:33:10 +0000 (16:33 -0800)]
Comment about _setup_pipes / PyPy GC interaction.
Zac Medico [Thu, 16 Feb 2012 00:25:31 +0000 (16:25 -0800)]
SpawnProcess: use os.open for /dev/null input
Zac Medico [Wed, 15 Feb 2012 22:28:41 +0000 (14:28 -0800)]
MergeProcess: inherit stdin for use with pdb
Zac Medico [Wed, 15 Feb 2012 22:17:38 +0000 (14:17 -0800)]
treewalk: handle EAGAIN from listdir for PyPy 1.8
Zac Medico [Wed, 15 Feb 2012 09:32:40 +0000 (01:32 -0800)]
get_open_fds: handle EAGAIN for PyPy 1.8
Zac Medico [Wed, 15 Feb 2012 03:01:16 +0000 (19:01 -0800)]
Scheduler: PyPy WeakValueDictionary.pop KeyError
KeyError observed from WeakValueDictionary.pop() with PyPy 1.8,
despite None given as default. Note that PyPy 1.8 has the same
WeakValueDictionary code as CPython 2.7, so it may be possible for
CPython to raise KeyError here as well.
Zac Medico [Wed, 15 Feb 2012 01:17:04 +0000 (17:17 -0800)]
SpawnProcess: for stdout use os.write, not fdopen
Zac Medico [Wed, 15 Feb 2012 00:04:34 +0000 (16:04 -0800)]
repoman: fix ebuild.badheader for bug #403705
Zac Medico [Tue, 14 Feb 2012 23:51:22 +0000 (15:51 -0800)]
SubProcess._wait: add debug code for bug #403697
Zac Medico [Tue, 14 Feb 2012 23:39:57 +0000 (15:39 -0800)]
Assert that fork returns int for bug 403697.
These cases should have been included with commit
6a94a074aa0475173a51f3f726377d4c407e986b.
Zac Medico [Tue, 14 Feb 2012 23:25:21 +0000 (15:25 -0800)]
AsynchronousTask: skip cancelled check in _start
These checks should be irrelevant, since the cancel() method is never
called on anything that hasn't been started already.
Zac Medico [Tue, 14 Feb 2012 16:27:01 +0000 (08:27 -0800)]
EventLoop: allow IO event handler re-entrance
IO event handlers may be re-entrant, in case something like
AbstractPollTask._wait_loop(), needs to be called inside a handler for
some reason.
Zac Medico [Tue, 14 Feb 2012 12:29:02 +0000 (04:29 -0800)]
After python fork, don't close fds for PyPy 1.8.
If we close all open file descriptors after a fork, with PyPy 1.8 it
triggers "[Errno 9] Bad file descriptor" later in the subprocess.
Apparently it is holding references to file descriptors and closing
them after they've already been closed and re-opened for other
purposes. As a workaround, we don't close the file descriptors, so
that they won't be re-used and therefore we won't be vulnerable to this
kind of interference.
The obvious caveat of not closing the fds is that the subprocess can
hold locks that belonged to the parent process, even after the parent
process has released the locks. Hopefully this won't be a major problem
though, since the subprocess has to exit at release the lock
eventually, when the EbuildFetcher or _MergeProcess task is complete.
Zac Medico [Tue, 14 Feb 2012 03:06:29 +0000 (19:06 -0800)]
EventLoop._do_poll: tweak EINTR handling
Silently handle EINTR, which is normal when we have received a signal
such as SIGINT. Also, raise StopIteration in order to break out of our
current iteration and respond appropriately to the signal as soon as
possible.
Zac Medico [Tue, 14 Feb 2012 02:35:03 +0000 (18:35 -0800)]
AsynchronousTask: don't wait for exit status
Synchronous waiting for status is not supported, since it would be
vulnerable to hitting the recursion limit when a large number of tasks
need to be terminated simultaneously, like in bug #402335.
Zac Medico [Tue, 14 Feb 2012 01:22:56 +0000 (17:22 -0800)]
EventLoop: make _poll/_run_timeouts re-entrant
This fixes infinite loops triggered by Ctrl-C, where timeout calls
would exhaust the poll event queue because _poll was not re-entrant.
Now, re-entrance is only prohibited for individual callback functions,
in order to protect against infinite recursion.
Zac Medico [Tue, 14 Feb 2012 00:41:38 +0000 (16:41 -0800)]
runtests.sh: test PyPy 1.8, and support prefix
Zac Medico [Mon, 13 Feb 2012 23:38:41 +0000 (15:38 -0800)]
Handle missing os.setgroups for PyPy.
See https://bugs.pypy.org/issue833 for details.
Zac Medico [Mon, 13 Feb 2012 21:57:31 +0000 (13:57 -0800)]
checksum.py: remove PyPy 1.7 workarounds
The corresponding hashlib issues are fixed in PyPy 1.8:
https://bugs.pypy.org/issue957
Zac Medico [Mon, 13 Feb 2012 06:43:46 +0000 (22:43 -0800)]
PollScheduler: use idle_add to check termination
This fixes a regression in termination signal handling since commit
8c1fcf5a9ba9fa4d406a4d0cc284fe73a84f5a63, which cause termination
signals to be ignored until a running job had exited. This regression
is not really noticeable for Ctrl-C handling, since in that case the
SIGINT propagets to subprocesses, causing them to exit and trigger a
_schedule() call whichtriggers a termination check.
Zac Medico [Sun, 12 Feb 2012 23:32:12 +0000 (15:32 -0800)]
repoman: check for env-update, bug #402339
Zac Medico [Sun, 12 Feb 2012 06:16:57 +0000 (22:16 -0800)]
Scheduler: simplify _sched_iface construction
Zac Medico [Sun, 12 Feb 2012 05:36:18 +0000 (21:36 -0800)]
EbuildIpcDaemon: handle POLLHUP, bug #401919
Zac Medico [Sun, 12 Feb 2012 04:08:59 +0000 (20:08 -0800)]
AbstractPollTask: merge _wait from subclass
Zac Medico [Sun, 12 Feb 2012 03:39:10 +0000 (19:39 -0800)]
xpak.unpackinfo: validate paths, bug #403149
Zac Medico [Sun, 12 Feb 2012 00:05:14 +0000 (16:05 -0800)]
QueueScheduler: tweak run loop logic
Zac Medico [Sat, 11 Feb 2012 23:56:07 +0000 (15:56 -0800)]
EventLoop.iteration: handle _poll StopIteration
Zac Medico [Sat, 11 Feb 2012 23:01:55 +0000 (15:01 -0800)]
MetadataRegen: use porttrees
Arfrever Frehtes Taifersar Arahesis [Sat, 11 Feb 2012 22:14:43 +0000 (23:14 +0100)]
_emerge.MetadataRegen.MetadataRegen._iter_metadata_processes():
Regenerate metadata for given cpv in each repository.
Zac Medico [Sat, 11 Feb 2012 21:41:18 +0000 (13:41 -0800)]
EventLoop: remove obsolete timeout_add comment
Zac Medico [Sat, 11 Feb 2012 21:11:10 +0000 (13:11 -0800)]
EventLoop.iteration: run timeouts if no fds
Zac Medico [Sat, 11 Feb 2012 20:05:17 +0000 (12:05 -0800)]
PollScheduler: return None from _schedule_tasks
The _keep_scheduling() template method is used instead.
Zac Medico [Sat, 11 Feb 2012 19:48:05 +0000 (11:48 -0800)]
QueueScheduler: timeout compat for GlibEventLoop
Raising StopIteration doesn't work with GlibEventLoop, since it catches
all exceptions and logs them. So, just use a boolean expression to
terminate the while loop. This depends on commit
dcb9fab8463996542d06d29bc383f5933bf0d677 so that IpcDaemonTestCase
timeouts work correctly with both EventLoop and GlibEventLoop.
Zac Medico [Sat, 11 Feb 2012 19:34:36 +0000 (11:34 -0800)]
EventLoop: shorten iteration poll if timeouts
This fixes EventLoop.iteration() so that it doesn't poll too long when
there are timeouts registered, since it's not nice to spend a long time
polling for an IO event that might never arrive even though a timeout
may have changed some state that would terminate a timeout loop like
in QueueScheduler.run() or AbstractPollTask._wait_loop(). This makes
EventLoop.iteration() behave more like GlibEventLoop.iteration().
Zac Medico [Sat, 11 Feb 2012 18:46:00 +0000 (10:46 -0800)]
Fix FEATURES=no{doc,info,man} for bug #403181
This has been broken since commit
93b654199a32fc3df1299b030317720b9294b0c3.
Zac Medico [Sat, 11 Feb 2012 05:42:57 +0000 (21:42 -0800)]
repoman: no ChangeLog for Manifest-only commit
Use --echangelog=force to force ChangeLog generation in this case.
See bug #398009.
Zac Medico [Sat, 11 Feb 2012 03:18:57 +0000 (19:18 -0800)]
display_autounmask: highlight config changes
As suggested here:
http://archives.gentoo.org/gentoo-dev/msg_06a47ad55977f4d16bfe18769c6c38b8.xml
Zac Medico [Sat, 11 Feb 2012 02:55:19 +0000 (18:55 -0800)]
GlibEventLoop: use gi.repository.GLib
Zac Medico [Sat, 11 Feb 2012 02:10:07 +0000 (18:10 -0800)]
PollScheduler: add generic _main_loop()
Zac Medico [Fri, 10 Feb 2012 23:19:06 +0000 (15:19 -0800)]
repoman: support PORTAGE_CONFIGROOT, bug 402577
Zac Medico [Fri, 10 Feb 2012 23:13:49 +0000 (15:13 -0800)]
repoman: remove redundant config constructor