Gentoo Archives: gentoo-portage-dev

From: Rich Freeman <rich0@g.o>
To: gentoo-portage-dev@l.g.o
Subject: Re: [gentoo-portage-dev] [RFC] Improving Gentoo package format
Date: Sun, 11 Nov 2018 10:56:30
Message-Id: CAGfcS_=VgD+Ho2pQxUBa1oO4rSu=+cZm5ZY-s1q4ZKVqsdCBRA@mail.gmail.com
In Reply to: Re: [gentoo-portage-dev] [RFC] Improving Gentoo package format by "Michał Górny"
1 On Sun, Nov 11, 2018 at 3:29 AM Michał Górny <mgorny@g.o> wrote:
2 >
3 > On Sat, 2018-11-10 at 09:37 -0500, Alec Warner wrote:
4 > > On Sat, Nov 10, 2018 at 8:09 AM Michał Górny <mgorny@g.o> wrote:
5 > >
6 > > >
7 > > > + If we can maintain reasonable level of vdb compatibility, the user can
8 > > > even emergency-install a package without causing too much hassle (as it
9 > > > will be recorded in vdb); ideally Portage would detect this vdb entry
10 > > > and support fixing the install afterwards.
11 > > >
12
13 IMO just overwriting vdb doesn't seem like a great idea. If somebody
14 does do a plain untar then the package manager has no ability to
15 sanitize anything going in, which means dealing with a potential mess
16 after-the-fact. Users wouldn't think to go messing with /var/db/pkg
17 on their own, but they certainly will be tempted to just untar a file.
18
19 Perhaps a package with the same name/version was already installed,
20 but the new files aren't the same as the old files. Now we have
21 orphans because the package manager never had a chance to clean up and
22 lost all its state regarding what was already there.
23
24 Plus, this is basically in-band signaling and that is the sort of
25 thing that usually ends up being regretted sooner or later.
26
27 I'm not sure if vdb is entirely optimal, but if we wanted to stick
28 with that metadata format, why not just stick it in a separate
29 tarball?
30
31 >
32 > Are you saying it's better that user emergency-installs a package
33 > without recording it in vdb, and ends up with mess of collisions
34 > and untracked files?
35 >
36
37 IMO this is no different than a user unpacking any other random
38 tarball that goes and creates orphans. I think the better solution is
39 some kind of tool to cleanly install a tarball, assuming it doesn't
40 already exist. Short of turning /usr into a squashfs or whatever I'm
41 not sure any distro has a great solution for keeping users from
42 bypassing the package manager entirely.
43
44 > > In short; why should we event a new format?
45 >
46 > Because nobody knows how to use 'ar', compared to how almost every
47 > Gentoo user can use 'tar' immediately? Of course we could alternatively
48 > just use a nested tarball but I wanted to keep the possibility
49 > of actually being able to 'tar -xf' it without having to extract nested
50 > archives.
51 >
52
53 IMO a nested tarball would be a better solution. I agree that ar is
54 obscure, and I don't see how it adds any value. If we were talking
55 about something going into a bootloader then optimizing for unpacking
56 efficiency might be a concern, but there is no reason not to use more
57 standard tools, unless there was something about the .deb format
58 itself we wanted to completely preserve (seems unlikely).
59
60 Overall though, I definitely think that a better binary format makes a
61 lot of sense, and I think you're on the right track.
62
63 One thing you didn't touch on is file naming. Right now two binary
64 packages with different USE/etc configurations are going to collide.
65 Would it make sense to toss in some kind of content hash of certain
66 metadata in the filename or something so that it would be much simpler
67 to host and auto-fetch binary packages? I realize this is going
68 beyond your initial scope, but if we wanted to do this it would be a
69 good time to do so.
70
71 --
72 Rich