1 |
On Sun, Oct 18, 2020 at 9:58 AM Felix Bier <Felix.Bier@×××××××××××××.com> wrote: |
2 |
> |
3 |
> Am Samstag, den 17.10.2020, 13:11 -0700 schrieb Matt Turner: |
4 |
> > On Sat, Oct 17, 2020 at 12:00 PM Felix Bier |
5 |
> > <Felix.Bier@×××××××××××××.com> wrote: |
6 |
> > > |
7 |
> > > This commit fixes the following issues: |
8 |
> > > |
9 |
> > > * The PORTDIR_OVERLAY variable has been deprecated by Gentoo. |
10 |
> > > |
11 |
> > > With this commit, the variable is no longer written to the |
12 |
> > > generated make.conf. Instead, a config file |
13 |
> > > /etc/portage/repos.conf/<repo-name>.conf |
14 |
> > > is generated for each overlay. The repo name is read from the |
15 |
> > > overlay using the portage API. Internally, portage parses |
16 |
> > > metadata/layout.conf and profiles/repo_name to obtain the name. |
17 |
> > > |
18 |
> > > References: |
19 |
> > > https://wiki.gentoo.org/wiki//etc/portage/make.conf |
20 |
> > > https://wiki.gentoo.org/wiki//etc/portage/repos.conf |
21 |
> > > |
22 |
> > > * All overlays were copied into the same target directory. If the |
23 |
> > > same file name occurred in multiple overlays, the last overlay |
24 |
> > > would overwrite all previous files with this name. In |
25 |
> > > particular, |
26 |
> > > only the metadata/layout.conf of the last overlay was retained, |
27 |
> > > so it was not possible to reference the other overlays e.g. via |
28 |
> > > the masters entry in the layout.conf or the portage-2 syntax |
29 |
> > > for specifying a parent profile from another overlay. Also, |
30 |
> > > this created problems when the overlays contained ebuilds |
31 |
> > > for the same package, but with differing versions, because |
32 |
> > > after copying, the target directory contained both versions of |
33 |
> > > the |
34 |
> > > ebuild but only the manifest file of the last overlay. |
35 |
> > > |
36 |
> > > With this commit, each overlay is copied into a separate |
37 |
> > > sub-directory, e.g. /var/gentoo/repos/local/<repo-name>/. |
38 |
> > > This directory is referenced via the location entry in the |
39 |
> > > generated /etc/portage/repos.conf/<repo-name>.conf. |
40 |
> > > --- |
41 |
> > |
42 |
> > Hello, |
43 |
> > |
44 |
> > Thank you for the patches. I'm happy to see them. |
45 |
> > |
46 |
> > I cannot apply the patches however. This one in particular is badly |
47 |
> > line wrapped by your mail client. I tried fixing it up, but either |
48 |
> > failed or don't know what this is supposed to apply to. |
49 |
> > |
50 |
> > It looks like the intention is for this to apply to the |
51 |
> > catalyst-3.0-stable branch since it discusses copying overlays into a |
52 |
> > subdirectory, rather than any mention of squashfs snapshots. |
53 |
> > |
54 |
> > I really don't want new feature work on the catalyst-3.0-stable |
55 |
> > branch, for example. If that is the target, then please consider |
56 |
> > rebasing the work onto the master branch. |
57 |
> > |
58 |
> > Matt |
59 |
> |
60 |
> Hello Matt, |
61 |
> thank you for the initial review. |
62 |
> |
63 |
> I'm sorry for the formatting issues, I will check my mail client's |
64 |
> format settings and resend the patches. |
65 |
> |
66 |
> The patches are intendend for the master branch. I agree that this |
67 |
> feature should not be in the stable branch. |
68 |
> |
69 |
> My understanding is that in the master branch, the squashfs is only |
70 |
> used for the main repository, whereas the overlays are still copied as |
71 |
> directories (in the StageBase.portage_overlay() function). |
72 |
> |
73 |
> This patch only removes PORTDIR_OVERLAY, not PORTDIR, so it should not |
74 |
> be a problem that the main repository is squashed. I kept the helper |
75 |
> functions that this patch adds general, so that they can be reused for |
76 |
> a later PORTDIR removal, but I would prefer to do that in a separate |
77 |
> step. |
78 |
|
79 |
Right. I guess I was expecting a transition from PORTDIR + |
80 |
PORTDIR_OVERLAY to the modern just-a-bunch-of-repos repos.conf system. |
81 |
That's totally fine to not make the leap all at once. |
82 |
|
83 |
There's actually a patch in https://bugs.gentoo.org/560518 that |
84 |
creates repos.conf, if you're interested in any follow on work :) |
85 |
|
86 |
> I don't know if there are plans to also switch the overlays to |
87 |
> squashfs. My intuition would be that the overlays would usually be much |
88 |
> smaller than the main repository, so squashing them would have less |
89 |
> benefit. Even if there is a plan to switch the overlays to squashfs (or |
90 |
> bind-mounting, which I would prefer), I think my patch would still be |
91 |
> helpful for that, since it would not be possible to create multiple |
92 |
> mounts with the same target directory (such as /var/db/repo/local). So |
93 |
> creating subdirectories (/var/db/repo/local/overlay1, |
94 |
> /var/db/repo/local/overlay2 etc.) would still be needed. |
95 |
|
96 |
An additional advantage of squashfs snapshots is that they're |
97 |
snapshots. In that way, we should be able to reproduce a build from |
98 |
exactly the same inputs. You're right that the other advantages of |
99 |
squashfs snapshots (smaller size, not copying 1GB of small files, |
100 |
quick cleanup, etc) don't really apply to the same degree to small |
101 |
overlays. |