Gentoo Archives: gentoo-dev

From: "Michał Górny" <gentoo@××××××××××.pl>
To: gentoo-dev@l.g.o
Subject: Re: [gentoo-dev] Multiple emerges in parallel (was: [RFC] RESTRICT=parallel for builds that can't be executed in parallel)
Date: Wed, 14 Apr 2010 14:20:32
Message-Id: 20100414162018.26e0524c@pomiot.lan
In Reply to: Re: [gentoo-dev] [RFC] RESTRICT=parallel for builds that can't be executed in parallel by Brian Harring
1 On Tue, 13 Apr 2010 23:10:16 -0700
2 Brian Harring <ferringb@×××××.com> wrote:
4 > Running multiple emerges in parallel is already a bad idea. The
5 > solution for that case is for the new/second emerge to feed the
6 > request into the original emerge (or a daemon).
8 Although such solution will be useful in many cases indeed, there are
9 still many advantages of having few separate emerge calls running
10 in parallel.
12 First of all, the 'emerge daemon' would have to be able to
13 transparently inject the additional packages onto the dependency tree.
14 And just appending missing dependencies and the real package at the end
15 doesn't seem sufficient.
17 For example, what if such operation would result in a conflict? If
18 that's only that simple as 'A and B can not be installed at the same
19 time', the second emerge call could reject appending the packages,
20 explaining the reason for that.
22 But what if the conflict could be resolved through pulling in some
23 other version of one of the packages? We could try to replace
24 the package in the dependency tree silently if it didn't yet start
25 being merged; but what if it did? Should we reject adding
26 the packages, encouraging user to 'try again later'? Or maybe abort
27 the running builds to merge another version of the depend? Same goes
28 for virtual dependencies which can be fulfilled by many different
29 packages.
31 Of course, that's just implementation-wide issue and running separate
32 emerge instances could result in even worse results of this problem.
33 But apart from that, there are side uses of multiple emerges which
34 would be hard to implement with your idea.
36 For example, I often use different PORTAGE_TMPDIR to build large
37 packages like xulrunner. Although it doesn't seem that hard
38 implementing the support for running merges in different temporary
39 directories at the same time, should we really waste time implementing
40 that? What about other possible configuration changes? What about
41 'USE' overrides within the environment? These ones should certainly be
42 respected.
44 So, now our daemon not only has to be able to efficiently mangle
45 dependency trees at runtime but also to support partial runtime
46 configuration changes.
48 The next thing is aborting merges. When running multiple emerges,
49 aborting one of them is as simple as pressing ^c. With daemon, we would
50 have to implement an ability of aborting/removing packages in runtime
51 -- and that would be another example of dependency tree mangling.
53 But in fact, implementing (at least limited) the support of such
54 mangling would be useful in some cases. With '--keep-going', it would
55 allow to drop the packages and continue merging the remaining ones
56 without waiting for current builds to complete.
58 And looking from the other side, the support for removing single
59 packages from the running emerge dependency tree would be useful too
60 with current emerge concept. Even if it just worked like
61 '--keep-going', it would be still better than the current workaround --
62 waiting for particular emerge to start and removing its ${WORKDIR} to
63 force failure.
65 --
66 Best regards,
67 Michał Górny
69 <>
70 <xmpp:mgorny@××××××.ru>


File name MIME type
signature.asc application/pgp-signature