Gentoo Archives: gentoo-portage-dev

From: Zac Medico <zmedico@g.o>
To: "Michał Górny" <mgorny@g.o>, gentoo-portage-dev@l.g.o
Cc: Zac Medico <zmedico@g.o>
Subject: Re: [gentoo-portage-dev] [PATCH] Enable FEATURES=parallel-install by default (bug 715110)
Date: Wed, 13 May 2020 06:28:44
Message-Id: 83f3b3b8-e88d-8fa7-d03d-75598ed0cad6@gentoo.org
In Reply to: Re: [gentoo-portage-dev] [PATCH] Enable FEATURES=parallel-install by default (bug 715110) by "Michał Górny"
1 On 5/12/20 11:22 PM, Michał Górny wrote:
2 > On Tue, 2020-05-12 at 13:18 -0700, Zac Medico wrote:
3 >> On 5/12/20 12:28 PM, Michał Górny wrote:
4 >>> W dniu wto, 12.05.2020 o godzinie 10∶05 -0700, użytkownik Zac Medico
5 >>> napisał:
6 >>>> On 5/12/20 3:39 AM, Michał Górny wrote:
7 >>>>> I'm sorry but I was asking of a more high-level implications.
8 >>>>>
9 >>>>> I presume that this means that more than files of more than one
10 >>>>> package
11 >>>>> can be merged simultaneously. However:
12 >>>>
13 >>>> No, an exclusive lock must be held on vardbapi._fs_lock for this.
14 >>>> This
15 >>>> is currently required at least to guarantee that access to the config
16 >>>> memory file is serialized (config memory is the thing that emerge
17 >>>> --noconfmem disables, but --noconfmem does not disable this lock).
18 >>>>
19 >>>> We assume that it's probably not worthwhile to try to merge files for
20 >>>> more than one package at a time, since that would cause them to
21 >>>> compete
22 >>>> for IO bandwidth.
23 >>>>
24 >>>>> 1. Are collisions handled correctly then? i.e. if you start
25 >>>>> installing
26 >>>>> A, and then B, and the two packages collide will portage fail
27 >>>>> before
28 >>>>> starting to install any file from B?
29 >>>>
30 >>>> There are no guarantees here. However, the risk is minimal, since
31 >>>> it's
32 >>>> unlikely that a file collision of this sort would occur. file
33 >>>> collisions
34 >>>> are a QA problem that is generally detected and corrected log before
35 >>>> we
36 >>>> would encounter a collision of this sort.
37 >>>>
38 >>>>> 2. Are preinst/postinst phases called simultaneously or serialized?
39 >>>>
40 >>>> They're serialized.
41 >>>
42 >>> Now I'm lost here. Could you try to explain to me, without getting
43 >>> into the deep technicalities, how parallel-install achieves better
44 >>> speed or at doing what is non-parallel-install so slow?
45 >>
46 >> It allows preinst/postinst/prerm/postrm phases to run for one package
47 >> while files are concurrently being merged or unmerged for another
48 >> package. This makes it possible to approach saturation of IO bandwidth.
49 >
50 > Doesn't this imply that programs run in these phases could fail if
51 > Portage is simultaneously replacing their files?
52
53 We've got the same potential issue with emerge --jobs, but we use the
54 dependency graph to schedule merges such that our dependencies do not
55 mutate underneath us.
56 --
57 Thanks,
58 Zac

Attachments

File name MIME type
signature.asc application/pgp-signature