Gentoo Archives: gentoo-portage-dev

From: "Michał Górny" <mgorny@g.o>
To: gentoo-portage-dev@l.g.o
Cc: Zac Medico <zmedico@g.o>
Subject: Re: [gentoo-portage-dev] [PATCH] Enable FEATURES=parallel-install by default (bug 715110)
Date: Wed, 13 May 2020 06:23:00
Message-Id: 7c383e90c4f8e6010bf1ab3489e5c8ccc08b4577.camel@gentoo.org
In Reply to: Re: [gentoo-portage-dev] [PATCH] Enable FEATURES=parallel-install by default (bug 715110) by Zac Medico
1 On Tue, 2020-05-12 at 13:18 -0700, Zac Medico wrote:
2 > On 5/12/20 12:28 PM, Michał Górny wrote:
3 > > W dniu wto, 12.05.2020 o godzinie 10∶05 -0700, użytkownik Zac Medico
4 > > napisał:
5 > > > On 5/12/20 3:39 AM, Michał Górny wrote:
6 > > > > I'm sorry but I was asking of a more high-level implications.
7 > > > >
8 > > > > I presume that this means that more than files of more than one
9 > > > > package
10 > > > > can be merged simultaneously. However:
11 > > >
12 > > > No, an exclusive lock must be held on vardbapi._fs_lock for this.
13 > > > This
14 > > > is currently required at least to guarantee that access to the config
15 > > > memory file is serialized (config memory is the thing that emerge
16 > > > --noconfmem disables, but --noconfmem does not disable this lock).
17 > > >
18 > > > We assume that it's probably not worthwhile to try to merge files for
19 > > > more than one package at a time, since that would cause them to
20 > > > compete
21 > > > for IO bandwidth.
22 > > >
23 > > > > 1. Are collisions handled correctly then? i.e. if you start
24 > > > > installing
25 > > > > A, and then B, and the two packages collide will portage fail
26 > > > > before
27 > > > > starting to install any file from B?
28 > > >
29 > > > There are no guarantees here. However, the risk is minimal, since
30 > > > it's
31 > > > unlikely that a file collision of this sort would occur. file
32 > > > collisions
33 > > > are a QA problem that is generally detected and corrected log before
34 > > > we
35 > > > would encounter a collision of this sort.
36 > > >
37 > > > > 2. Are preinst/postinst phases called simultaneously or serialized?
38 > > >
39 > > > They're serialized.
40 > >
41 > > Now I'm lost here. Could you try to explain to me, without getting
42 > > into the deep technicalities, how parallel-install achieves better
43 > > speed or at doing what is non-parallel-install so slow?
44 >
45 > It allows preinst/postinst/prerm/postrm phases to run for one package
46 > while files are concurrently being merged or unmerged for another
47 > package. This makes it possible to approach saturation of IO bandwidth.
48
49 Doesn't this imply that programs run in these phases could fail if
50 Portage is simultaneously replacing their files?
51
52 --
53 Best regards,
54 Michał Górny

Attachments

File name MIME type
signature.asc application/pgp-signature

Replies