1 |
On Mon, Apr 24, 2006 at 07:55:17PM -0700, Brian wrote: |
2 |
> That however could change for the next major version of portage. I have |
3 |
> not attempted to try an internal build in the imported pkgcore, but it |
4 |
> may well work ok. That is something that is yet to be tested. Brian |
5 |
> Harring does not know how pkgcore will handle being threaded. It is too |
6 |
> early in it's development. |
7 |
|
8 |
Locking has been setup in a few spots (not enabled mostly, fex vdb |
9 |
repo locking)- what's not in place (and matters) is that internally, |
10 |
pkgcore uses pkgcore.util.caching pretty heavily- used to generate |
11 |
unique instances that are reused. |
12 |
|
13 |
For example, if you get the package instance sys-apps/portage-2.0.51 |
14 |
from repo A and shove it in a list (thus holding onto it in memory), |
15 |
if you grab sys-apps/portage-2.0.51 from repo A *again*, you get the |
16 |
same instance back (literally same instance). |
17 |
|
18 |
So... generation of the instance (or lookup of the old instance) is |
19 |
a critical section- have to ensure that updates to the dict of what |
20 |
pkgs are in memory (whether removal or addition of pkgs) is atomic so |
21 |
you don't get N copies of pkg A. |
22 |
|
23 |
Fortunately, python's GIL protects deals with the "pkg a is being |
24 |
unalloc'd while it's about to be handed back"- GIL + ref counting |
25 |
prevents that from being an issue. |
26 |
|
27 |
Aside from that... the only other spots where locking is required is |
28 |
1) build directory usage- don't want two instances executing |
29 |
configure/make in the same directory fex, mainly because ebuild |
30 |
execution may mangle things in install phase (which would break things |
31 |
for compile phase). |
32 |
2) sync locking. Debatable on that one (best you can manage is |
33 |
invalidating any pkgs from that repo in memory, which can cause |
34 |
issues). |
35 |
3) cache updates (debatable also). Downside of races there is that |
36 |
cache regen for a node occurs twice (rather then once), which needs to |
37 |
be weighed against cost of locking overhead... |
38 |
|
39 |
Usual locking requirements stable has (distdir fex) applies also. |
40 |
~harring |