List Archive: gentoo-portage-dev
Note: Due to technical difficulties, the Archives are currently not up to date.
provides an alternative service for most mailing lists.c.f. bug 424647
On Mon, Apr 24, 2006 at 07:55:17PM -0700, Brian wrote:
> That however could change for the next major version of portage. I have
> not attempted to try an internal build in the imported pkgcore, but it
> may well work ok. That is something that is yet to be tested. Brian
> Harring does not know how pkgcore will handle being threaded. It is too
> early in it's development.
Locking has been setup in a few spots (not enabled mostly, fex vdb
repo locking)- what's not in place (and matters) is that internally,
pkgcore uses pkgcore.util.caching pretty heavily- used to generate
unique instances that are reused.
For example, if you get the package instance sys-apps/portage-2.0.51
from repo A and shove it in a list (thus holding onto it in memory),
if you grab sys-apps/portage-2.0.51 from repo A *again*, you get the
same instance back (literally same instance).
So... generation of the instance (or lookup of the old instance) is
a critical section- have to ensure that updates to the dict of what
pkgs are in memory (whether removal or addition of pkgs) is atomic so
you don't get N copies of pkg A.
Fortunately, python's GIL protects deals with the "pkg a is being
unalloc'd while it's about to be handed back"- GIL + ref counting
prevents that from being an issue.
Aside from that... the only other spots where locking is required is
1) build directory usage- don't want two instances executing
configure/make in the same directory fex, mainly because ebuild
execution may mangle things in install phase (which would break things
for compile phase).
2) sync locking. Debatable on that one (best you can manage is
invalidating any pkgs from that repo in memory, which can cause
3) cache updates (debatable also). Downside of races there is that
cache regen for a node occurs twice (rather then once), which needs to
be weighed against cost of locking overhead...
Usual locking requirements stable has (distdir fex) applies also.