Gentoo Archives: gentoo-dev

From: Brian Harring <ferringb@×××××.com>
To: gentoo-dev@l.g.o
Subject: Re: [gentoo-dev] has_version etc parallelisability
Date: Sat, 05 Jan 2008 02:53:56
Message-Id: 20080105025056.GE5844@seldon
In Reply to: Re: [gentoo-dev] has_version etc parallelisability by Ciaran McCreesh
1 On Mon, Dec 31, 2007 at 02:28:44PM +0000, Ciaran McCreesh wrote:
2 > On Sun, 30 Dec 2007 20:11:16 -0800
3 > "Alec Warner" <antarus@g.o> wrote:
4 > > On 12/30/07, Ciaran McCreesh <ciaran.mccreesh@×××××××××××××.uk> wrote:
5 > > > Is it legal for ebuilds to call has_version and friends in
6 > > > parallel? Is it legal for ebuilds to call has_version and friends
7 > > > after the ebuild process has terminated? Discuss.
8 > >
9 > > If the pm implements read/write locking on the underlying datastore
10 > > (which it should probably have regardless of this request) then I
11 > > don't see a problem in parallel has_version calls.
12 >
13 > Actually, it's the communication channel that's the issue... If, for
14 > example, has_version is implemented in terms of a request on a pipe
15 > rather than execing a new package manager, we get into messy bash
16 > locking territory...
17 >
18 > > I don't get your second example..do you mean the ebuild is running
19 > > has_version in the background and then terminating?
20 >
21 > Yeah. Again, consider the pipe example. If the package manager closes
22 > off the pipe when it thinks the ebuild's done, calling has_version will
23 > get the backgrounded process SIGPIPEd.
24
25 Depends on the implementation; for pkgcore, if that comm pipe is
26 dead, the ebuild env *should* be dead, or dieing. Background'ing
27 processes from that env isn't valid imo, either.
28
29 If you're refering to an ebuild that parallelizes itself while
30 executing, iow, parallelization w/in the ebuild env/phase execution,
31 I'd look more at being able to batch commands instead of trying to run
32 them in parallel. Reasoning follows-
33
34 1) if doing an exec approach to service the request, this means
35 reparsing of involved files for each request- inefficient, potentially
36 horribly so on crappy hardware/setups.
37 2) screws up the pipe approach, should folks take it for control/env
38 introspection gains.
39
40 Summarizing, executing has_version (and friends)
41 concurrently has it's own issues performance wise, and implementation
42 wise; growing batch functionality into portageq however avoids those
43 issues, and would be faster- thus the route I'd advocate.
44
45 ~harring

Replies

Subject Author
Re: [gentoo-dev] has_version etc parallelisability Ciaran McCreesh <ciaran.mccreesh@×××××××××××××.uk>