portage.git
13 years agoTask.__eq__: allow _hash_key as input
Zac Medico [Sat, 21 May 2011 08:40:30 +0000 (01:40 -0700)]
Task.__eq__: allow _hash_key as input

This is used by depgraph._pkg() for lookups.

13 years agoambigous_merge_order: handle more failure modes
Zac Medico [Sat, 21 May 2011 04:14:49 +0000 (21:14 -0700)]
ambigous_merge_order: handle more failure modes

13 years agotest_merge_order: test hard buildtime blockers
Zac Medico [Sat, 21 May 2011 04:01:31 +0000 (21:01 -0700)]
test_merge_order: test hard buildtime blockers

13 years agotest_merge_order: test hard blocker resolution
Zac Medico [Sat, 21 May 2011 03:49:08 +0000 (20:49 -0700)]
test_merge_order: test hard blocker resolution

13 years agotest_merge_order: test blocker resolution
Zac Medico [Sat, 21 May 2011 03:30:30 +0000 (20:30 -0700)]
test_merge_order: test blocker resolution

13 years agotest_merge_order: remove uneeded ambigous flag
Zac Medico [Sat, 21 May 2011 01:16:46 +0000 (18:16 -0700)]
test_merge_order: remove uneeded ambigous flag

13 years agotest_merge_order: fix ambigous_merge_order bug
Zac Medico [Sat, 21 May 2011 01:11:56 +0000 (18:11 -0700)]
test_merge_order: fix ambigous_merge_order bug

13 years agotest_merge_order: test circular RDEPEND + PDEPEND
Zac Medico [Sat, 21 May 2011 01:00:02 +0000 (18:00 -0700)]
test_merge_order: test circular RDEPEND + PDEPEND

13 years agoResolverPlayground: support ambigous_merge_order
Zac Medico [Sat, 21 May 2011 00:51:09 +0000 (17:51 -0700)]
ResolverPlayground: support ambigous_merge_order

13 years agoFix grammar in previous commit.
Zac Medico [Fri, 20 May 2011 22:01:21 +0000 (15:01 -0700)]
Fix grammar in previous commit.

13 years agodepclean: adjust failure message wrt --with-bdeps
Zac Medico [Fri, 20 May 2011 21:56:08 +0000 (14:56 -0700)]
depclean: adjust failure message wrt --with-bdeps

We need to be more explicit about using --with-bdeps=y, since this
subtlety often leads to confusion.

13 years agoTask: optimize == and != _hash_key comparison
Zac Medico [Fri, 20 May 2011 20:40:13 +0000 (13:40 -0700)]
Task: optimize == and != _hash_key comparison

13 years agoAdd FEATURES=distcc-pump support.
MATSUU Takuto [Fri, 20 May 2011 19:20:58 +0000 (12:20 -0700)]
Add FEATURES=distcc-pump support.

This will fix bug #288597.

13 years agoTask: precalculate _hash_value
Zac Medico [Fri, 20 May 2011 06:31:55 +0000 (23:31 -0700)]
Task: precalculate _hash_value

13 years agoTask: remove unnecessary _get_hash_key() method
Zac Medico [Fri, 20 May 2011 06:23:51 +0000 (23:23 -0700)]
Task: remove unnecessary _get_hash_key() method

13 years agoPackage: precalculate _hash_key
Marat Radchenko [Fri, 20 May 2011 04:25:17 +0000 (08:25 +0400)]
Package: precalculate _hash_key

This eliminates an expensive getattr call in _get_hash_key(), which
greatly improves depgraph performance on a weak ARM cpu.

13 years agoFix typos in comments.
Arfrever Frehtes Taifersar Arahesis [Wed, 18 May 2011 19:46:09 +0000 (21:46 +0200)]
Fix typos in comments.

13 years ago_find_libs_to_preserve: always return set v2.2.0_alpha34
Zac Medico [Wed, 18 May 2011 05:38:07 +0000 (22:38 -0700)]
_find_libs_to_preserve: always return set

13 years agopreserve-libs: check for broken linkmap more
Zac Medico [Wed, 18 May 2011 05:27:07 +0000 (22:27 -0700)]
preserve-libs: check for broken linkmap more

13 years ago_pkg_visibility_check: if in graph then visible
Zac Medico [Wed, 18 May 2011 04:37:08 +0000 (21:37 -0700)]
_pkg_visibility_check: if in graph then visible

13 years ago_LockProcess.unlock: assert successful returncode
Zac Medico [Wed, 18 May 2011 04:00:39 +0000 (21:00 -0700)]
_LockProcess.unlock: assert successful returncode

13 years agoetc-update: clear term when appropriate
Zac Medico [Wed, 18 May 2011 03:46:36 +0000 (20:46 -0700)]
etc-update: clear term when appropriate

This solves an issue like bug #142508, but involving etc-update instead
of dispatch-conf. This is also relevant to bug #70668, which is the
same issue, though the reporter suggested to use colors to delimit the
output instead of using clear.

13 years ago--autounmask-write: use CONFIGROOT more
Zac Medico [Wed, 18 May 2011 02:15:58 +0000 (19:15 -0700)]
--autounmask-write: use CONFIGROOT more

13 years agoFix a typo in a comment.
Zac Medico [Wed, 18 May 2011 01:49:57 +0000 (18:49 -0700)]
Fix a typo in a comment.

13 years agodigraph: implement __bool__
Zac Medico [Wed, 18 May 2011 01:43:32 +0000 (18:43 -0700)]
digraph: implement __bool__

13 years agodepgraph._resolve: use temp set for autounmask
Zac Medico [Wed, 18 May 2011 01:31:15 +0000 (18:31 -0700)]
depgraph._resolve: use temp set for autounmask

13 years agodisplay_autounmask: optimize check_if_latest
Zac Medico [Wed, 18 May 2011 00:54:34 +0000 (17:54 -0700)]
display_autounmask: optimize check_if_latest

13 years agotest_asynchronous_lock: simulate SIGINT from tty
Zac Medico [Tue, 17 May 2011 22:19:20 +0000 (15:19 -0700)]
test_asynchronous_lock: simulate SIGINT from tty

13 years agoAsynchronousTask: tweak poll() logic
Zac Medico [Tue, 17 May 2011 21:32:33 +0000 (14:32 -0700)]
AsynchronousTask: tweak poll() logic

13 years agoAsynchronousLock: implement _poll() and test
Zac Medico [Tue, 17 May 2011 21:25:47 +0000 (14:25 -0700)]
AsynchronousLock: implement _poll() and test

13 years agotest_asynchronous_lock: test returncode more
Zac Medico [Tue, 17 May 2011 21:14:20 +0000 (14:14 -0700)]
test_asynchronous_lock: test returncode more

13 years agotest_asynchronous_lock: test wait and cancel
Zac Medico [Tue, 17 May 2011 21:02:12 +0000 (14:02 -0700)]
test_asynchronous_lock: test wait and cancel

13 years agotest_asynchronous_lock: test waiting
Zac Medico [Tue, 17 May 2011 20:56:15 +0000 (13:56 -0700)]
test_asynchronous_lock: test waiting

13 years agotest_autounmask: test missing keyword
Zac Medico [Tue, 17 May 2011 20:33:10 +0000 (13:33 -0700)]
test_autounmask: test missing keyword

13 years agoUpdate timestamps in headers of modified files.
Zac Medico [Tue, 17 May 2011 19:50:38 +0000 (12:50 -0700)]
Update timestamps in headers of modified files.

13 years ago--autounmask-write: don't write hidden files
Zac Medico [Tue, 17 May 2011 19:43:48 +0000 (12:43 -0700)]
--autounmask-write: don't write hidden files

13 years ago--autounmask-write: use CONFIGROOT and EROOT
Zac Medico [Tue, 17 May 2011 07:10:21 +0000 (00:10 -0700)]
--autounmask-write: use CONFIGROOT and EROOT

13 years ago--autounmask-write: handle non-existent file
Zac Medico [Tue, 17 May 2011 06:54:52 +0000 (23:54 -0700)]
--autounmask-write: handle non-existent file

13 years ago--autounmask-write: only prompt when necessary
Zac Medico [Tue, 17 May 2011 06:29:21 +0000 (23:29 -0700)]
--autounmask-write: only prompt when necessary

13 years ago--autounmask-write: handle config file recursion
Zac Medico [Tue, 17 May 2011 06:10:38 +0000 (23:10 -0700)]
--autounmask-write: handle config file recursion

13 years agoDocument that --autounmask is enabled by default.
Zac Medico [Tue, 17 May 2011 04:30:08 +0000 (21:30 -0700)]
Document that --autounmask is enabled by default.

13 years agoEnable --autounmask by default
Sebastian Luther [Mon, 16 May 2011 20:06:40 +0000 (22:06 +0200)]
Enable --autounmask by default

13 years ago--autounmask-write: respect --ask
Sebastian Luther [Mon, 16 May 2011 19:21:12 +0000 (21:21 +0200)]
--autounmask-write: respect --ask

13 years agoBinpkgFetcher: handle AsynchronousLock failure
Zac Medico [Mon, 16 May 2011 07:21:03 +0000 (00:21 -0700)]
BinpkgFetcher: handle AsynchronousLock failure

13 years agoEbuildBuildDir: handle AsynchronousLock failure
Zac Medico [Mon, 16 May 2011 07:10:07 +0000 (00:10 -0700)]
EbuildBuildDir: handle AsynchronousLock failure

13 years ago_LockProcess: handle process failure more
Zac Medico [Mon, 16 May 2011 06:54:49 +0000 (23:54 -0700)]
_LockProcess: handle process failure more

13 years ago_LockProcess: handle process failure if cancelled
Zac Medico [Mon, 16 May 2011 06:17:27 +0000 (23:17 -0700)]
_LockProcess: handle process failure if cancelled

13 years agopreserve-libs: lock vardb for removeFromContents
Zac Medico [Mon, 16 May 2011 01:25:59 +0000 (18:25 -0700)]
preserve-libs: lock vardb for removeFromContents

13 years agoRevert "counter_tick_core: don't lock if parallel-install"
Zac Medico [Sun, 15 May 2011 23:28:32 +0000 (16:28 -0700)]
Revert "counter_tick_core: don't lock if parallel-install"

This reverts commit 405ad9eed65393205ec28af8772f7ea45ce0371e.
The root problem is fixed by commit
8380ec9d7590a47b912e0d1e1db2ebb6049b903b.

13 years agotreewalk: unlockdb/lockdb typo triggers EDEADLK
Zac Medico [Sun, 15 May 2011 23:17:36 +0000 (16:17 -0700)]
treewalk: unlockdb/lockdb typo triggers EDEADLK

13 years agomake.conf.txt: remove because unused/unmaintained
Zac Medico [Sun, 15 May 2011 22:29:15 +0000 (15:29 -0700)]
make.conf.txt: remove because unused/unmaintained

13 years agoFix some typos.
Arfrever Frehtes Taifersar Arahesis [Sun, 15 May 2011 21:24:31 +0000 (23:24 +0200)]
Fix some typos.

13 years ago--autounmask: Treat missing keywords as masks
Sebastian Luther [Sun, 15 May 2011 19:05:19 +0000 (12:05 -0700)]
--autounmask: Treat missing keywords as masks

13 years ago--autounmask: Allow package.mask changes
Sebastian Luther [Sun, 15 May 2011 19:03:20 +0000 (12:03 -0700)]
--autounmask: Allow package.mask changes

13 years agoImplement --autounmask-write
Sebastian Luther [Sun, 15 May 2011 19:01:03 +0000 (12:01 -0700)]
Implement --autounmask-write

Enabling this option together with --autounmask writes proposed
changes to config files, honoring CONFIG_PROTECT.

13 years agocounter_tick_core: don't lock if parallel-install
Zac Medico [Sun, 15 May 2011 09:30:15 +0000 (02:30 -0700)]
counter_tick_core: don't lock if parallel-install

This is the same as commit 461564ae94ff936918eeaa18493bc1da3846796f
but this time with comments that make sense.

13 years agoEbuildMerge: inherit from CompositeTask
Zac Medico [Sun, 15 May 2011 09:02:08 +0000 (02:02 -0700)]
EbuildMerge: inherit from CompositeTask

13 years agoEbuildPhase: fix locking for setup phase
Zac Medico [Sun, 15 May 2011 08:14:09 +0000 (01:14 -0700)]
EbuildPhase: fix locking for setup phase

13 years agoEbuildPhase: AsynchronousLock for ebuild-locks
Zac Medico [Sun, 15 May 2011 08:03:14 +0000 (01:03 -0700)]
EbuildPhase: AsynchronousLock for ebuild-locks

13 years agoUse vardbapi.lock() where applicable.
Zac Medico [Sun, 15 May 2011 05:21:08 +0000 (22:21 -0700)]
Use vardbapi.lock() where applicable.

13 years agomovefile: handle EEXIST when os.symlink fails
Zac Medico [Sun, 15 May 2011 04:39:26 +0000 (21:39 -0700)]
movefile: handle EEXIST when os.symlink fails

13 years agoRevert "counter_tick_core: don't lock if parallel-install"
Zac Medico [Sun, 15 May 2011 04:29:48 +0000 (21:29 -0700)]
Revert "counter_tick_core: don't lock if parallel-install"

This reverts commit 461564ae94ff936918eeaa18493bc1da3846796f.
Disabling the lock here doesn't really make sense, and I'm still
triggering EDEADLK is some other places.

13 years agopreserve-libs: use vardbapi _fs_lock/unlock
Zac Medico [Sun, 15 May 2011 02:55:24 +0000 (19:55 -0700)]
preserve-libs: use vardbapi _fs_lock/unlock

The preserve-libs code is too dependent on the filesystem state to
allow merging/unmerge/env_update to execute concurrently.

13 years agovardbapi: add reentrant _fs_lock/unlock methods
Zac Medico [Sun, 15 May 2011 02:43:12 +0000 (19:43 -0700)]
vardbapi: add reentrant _fs_lock/unlock methods

13 years ago_LibGraphNode: re-use the _key attribute
Zac Medico [Sun, 15 May 2011 01:56:39 +0000 (18:56 -0700)]
_LibGraphNode: re-use the _key attribute

This allows us to avoid repeating any previous stat calls, which helps
to avoid potential race conditions due to inconsistent stat results
when the file system is being modified concurrently.

13 years agocounter_tick_core: don't lock if parallel-install
Zac Medico [Sat, 14 May 2011 23:53:35 +0000 (16:53 -0700)]
counter_tick_core: don't lock if parallel-install

Hopefully this avoids the following exception:

  File "/usr/lib/portage/pym/portage/locks.py", line 138, in lockfile
    fcntl.lockf(myfd, fcntl.LOCK_EX)
IOError: [Errno 35] Resource deadlock avoided

13 years agoFix a typo in a comment.
Arfrever Frehtes Taifersar Arahesis [Sat, 14 May 2011 22:56:35 +0000 (00:56 +0200)]
Fix a typo in a comment.

13 years agofind_unused_preserved_libs: fix unmerge handling
Zac Medico [Sat, 14 May 2011 22:14:45 +0000 (15:14 -0700)]
find_unused_preserved_libs: fix unmerge handling

Since commit c64d4abee145d083c70273be8fd23bd56dffe7ec (bug #286714),
preserved libs have not been properly garbage collected when packages
are unmerged without replacement, due to a difference in LinkageMap
state. It should behave correcly now.

13 years agoprune_plib_registry: display preserved libs
Zac Medico [Sat, 14 May 2011 21:41:22 +0000 (14:41 -0700)]
prune_plib_registry: display preserved libs

Since commit c64d4abee145d083c70273be8fd23bd56dffe7ec, sometimes we
preserve libs for unmerged packages here (bug #286714), so we should
display them like we do in _add_preserve_libs_to_contents().

13 years agotreewalk: fix plib_collisions with same cpv
Zac Medico [Sat, 14 May 2011 21:25:16 +0000 (14:25 -0700)]
treewalk: fix plib_collisions with same cpv

13 years ago--autounmask: Use >= atoms if possible
Sebastian Luther [Sat, 14 May 2011 20:44:22 +0000 (22:44 +0200)]
--autounmask: Use >= atoms if possible

If the package is the latest avaiable it now uses >=cat/pkg-ver instead
of =cat/pkg-ver. If the package is not the latest, but the latest in
its slot, >=cat/pkg-ver:slot is used. In all other case =cat/pkg-ver
is used. Rational is that it's more likely that newer versions, added
in the future, will work with the same configuration changes as the
currently latest version than not.

13 years agopreserve-libs: preserve during uninstall
Zac Medico [Sat, 14 May 2011 08:45:06 +0000 (01:45 -0700)]
preserve-libs: preserve during uninstall

This will fix bug #286714. The emerge --depclean-lib-check option will
now be ignored when FEATURES=preserve-libs is enabled, since any
libraries that have consumers will simply be preserved.

13 years agocache.sqlite: handle readonly more v2.2.0_alpha33
Zac Medico [Thu, 12 May 2011 19:04:54 +0000 (12:04 -0700)]
cache.sqlite: handle readonly more

13 years agoporttree: use pure volatile cache if necessary
Zac Medico [Thu, 12 May 2011 19:02:27 +0000 (12:02 -0700)]
porttree: use pure volatile cache if necessary

13 years agocache.volatile: discard "perms" param
Zac Medico [Thu, 12 May 2011 18:58:27 +0000 (11:58 -0700)]
cache.volatile: discard "perms" param

13 years agocache.sqlite: handle readonly
Zac Medico [Thu, 12 May 2011 18:42:08 +0000 (11:42 -0700)]
cache.sqlite: handle readonly

13 years agoexpand_new_virt: return early for non-virtual cat
Zac Medico [Thu, 12 May 2011 17:51:08 +0000 (10:51 -0700)]
expand_new_virt: return early for non-virtual cat

13 years agoexpand_new_virt: skip match for non-virtual cat
Zac Medico [Thu, 12 May 2011 07:09:19 +0000 (00:09 -0700)]
expand_new_virt: skip match for non-virtual cat

13 years agotreewalk: fix inverted counter init logic
Zac Medico [Thu, 12 May 2011 05:35:40 +0000 (22:35 -0700)]
treewalk: fix inverted counter init logic

13 years agoportdbapi: tweak cache permission handling
Zac Medico [Thu, 12 May 2011 01:14:54 +0000 (18:14 -0700)]
portdbapi: tweak cache permission handling

This is a reponse to the following issue:

  http://code.google.com/p/chromium-os/issues/detail?id=15234

13 years agoEbuildPhase: extract binpkg env for pkg_pretend
Zac Medico [Wed, 11 May 2011 23:50:15 +0000 (16:50 -0700)]
EbuildPhase: extract binpkg env for pkg_pretend

This will fix bug #366939.

13 years agounmerge: use frozenset for syslist
Zac Medico [Wed, 11 May 2011 20:24:22 +0000 (13:24 -0700)]
unmerge: use frozenset for syslist

13 years agodepgraph: in complete mode respect --with-bdeps
Zac Medico [Wed, 11 May 2011 19:00:27 +0000 (12:00 -0700)]
depgraph: in complete mode respect --with-bdeps

If you want the graph as complete as possible, now you'll have to use
--with-bdeps=y together with --complete-graph.

13 years agodep_check_composite_db: return early from cp_list
Zac Medico [Wed, 11 May 2011 04:03:02 +0000 (21:03 -0700)]
dep_check_composite_db: return early from cp_list

Since this implementation is only intended to check for existence of
new-style virtuals, it's a waste of time to return more than one cpv.

13 years agodepgraph: traverse ignored DEPEND if "complete"
Zac Medico [Tue, 10 May 2011 23:54:55 +0000 (16:54 -0700)]
depgraph: traverse ignored DEPEND if "complete"

13 years agoStrip build-time dependencies when they're ignored and we're not in --rebuild mode.
David James [Tue, 10 May 2011 20:41:06 +0000 (13:41 -0700)]
Strip build-time dependencies when they're ignored and we're not in --rebuild mode.

This should improve depgraph calculation performance.

TEST=Ran test suite.
BUG=chromium-os:15144

Change-Id: I9e06817201d7a17ae9b44b424d01c1a2bd89210a

13 years agoMergeProcess: lock vdb earlier when appropriate
Zac Medico [Tue, 10 May 2011 19:52:36 +0000 (12:52 -0700)]
MergeProcess: lock vdb earlier when appropriate

13 years agodepgraph: don't clear vardbapi cache in _load_vdb
Zac Medico [Tue, 10 May 2011 05:11:33 +0000 (22:11 -0700)]
depgraph: don't clear vardbapi cache in _load_vdb

Most of the memory is probably held on the heap by the installed
package instances anyway, and the cache is useful for being inherited
by subprocess in MergeProcess.

13 years agoget_counter_tick_core: use cpv_all
Zac Medico [Tue, 10 May 2011 05:05:40 +0000 (22:05 -0700)]
get_counter_tick_core: use cpv_all

It's more efficient to use cpv_all since cp_all calls cpv_all anyway,
and calls to cp_list induce additional stat calls.

13 years agoCache counter values, avoiding I/O when it doesn't change.
David James [Tue, 10 May 2011 04:11:47 +0000 (21:11 -0700)]
Cache counter values, avoiding I/O when it doesn't change.

This improves merge times by up to 25%, since looping over the vardb for
each package install is slow.

TEST=Emerge a bunch of packages, notice 25% speed improvement.
BUG=chromium-os:15112

Change-Id: I51dd617219cd1820ceeb702291bd790990995be4

13 years agodep_check_composite_db: remove unused _dep_expand
Zac Medico [Tue, 10 May 2011 01:03:27 +0000 (18:03 -0700)]
dep_check_composite_db: remove unused _dep_expand

This code has been unused since dep_check started passing
token_class=Atom to use_reduce in commit
c9f7930883d62fc26af72bff0c4623db0bbc8221. Since then, any atoms
without categories are dropped by dep_check because it returns
early after catching an InvalidDependString exception.

13 years agodep_check_composite_db: implement cp_list
Zac Medico [Mon, 9 May 2011 22:34:06 +0000 (15:34 -0700)]
dep_check_composite_db: implement cp_list

This is used by dep_check since commit
6f9ee9c508c1506cdf8eb0dc46796dbe30f268db.

13 years agoconfig: call _populate_treeVirtuals directly
Zac Medico [Mon, 9 May 2011 20:41:56 +0000 (13:41 -0700)]
config: call _populate_treeVirtuals directly

13 years agocpv_expand: avoid old-style virtuals more
Zac Medico [Mon, 9 May 2011 20:36:22 +0000 (13:36 -0700)]
cpv_expand: avoid old-style virtuals more

13 years agodep_check: avoid old-style virtuals code
Zac Medico [Mon, 9 May 2011 20:30:04 +0000 (13:30 -0700)]
dep_check: avoid old-style virtuals code

13 years agodisplay_news_notification: don't populate virts
Zac Medico [Mon, 9 May 2011 06:27:39 +0000 (23:27 -0700)]
display_news_notification: don't populate virts

This typically isn't useful anymore, since Gentoo has stopped using
old-style virtuals.

13 years agocpv_expand: populate old-style virts on demand
Zac Medico [Mon, 9 May 2011 06:23:07 +0000 (23:23 -0700)]
cpv_expand: populate old-style virts on demand

If old-style virtuals code is triggered here and we are given a
vartree, use it to avoid creating a temporary instance.

13 years agounmerge: skip getvirtuals() if cat is not virtual
Zac Medico [Mon, 9 May 2011 06:08:43 +0000 (23:08 -0700)]
unmerge: skip getvirtuals() if cat is not virtual

This allows us to avoid triggering the old-style virtual code which
scans /var/db/pkg/*/*/PROVIDE.

13 years agounmerge: use expand_new_virt for sys pkg warnings
Zac Medico [Mon, 9 May 2011 05:13:52 +0000 (22:13 -0700)]
unmerge: use expand_new_virt for sys pkg warnings

13 years agovardbapi: disable subprocess vdb cache updates
Zac Medico [Mon, 9 May 2011 04:51:47 +0000 (21:51 -0700)]
vardbapi: disable subprocess vdb cache updates

Since commit 7535cabdf2fab76fc55df83643157613dfd66be9,
vardbapi.flush_cache() is often called within subprocesses spawned
from MergeProcess. The _aux_cache_threshold doesn't work as designed
if the cache is flushed from a subprocess like this, can lead to the
vdb cache being flushed for every single merge. This is a waste of
disk IO, so disable vdb cache updates in subprocesses.