1 |
On Wed, Feb 10, 2021 at 12:57 PM Andreas K. Hüttel <dilfridge@g.o> wrote: |
2 |
> |
3 |
> * what portage features are still needed or need improvements (e.g. binpkg |
4 |
> signing and verification) |
5 |
> * how should hosting look like |
6 |
|
7 |
Some ideas for portage enhancements: |
8 |
|
9 |
1. Ability to fetch binary packages from some kind of repo. |
10 |
2. Ability to have multiple binary packages co-exist in a repo (local |
11 |
or remote) with different build attributes (arch, USE, CFLAGS, |
12 |
DEPENDS, whatever). |
13 |
3. Ability to pick the most appropriate binary packages to use based |
14 |
on user preferences (with a mix of hard and soft preferences). |
15 |
|
16 |
One idea I've had around how #2-3 might be implemented is: |
17 |
1. Binary packages already contain data on how they were built (USE |
18 |
flags, dependencies, etc). Place this in a file using a deterministic |
19 |
sorting/etc order so that two builds with the same settings will have |
20 |
the same results. |
21 |
2. Generate a hash of the file contents - this can go in the filename |
22 |
so that the file can co-exist with other files, and be located |
23 |
assuming you have a full matching set of metadata. |
24 |
3. Start dropping attributes from the file based on a list of |
25 |
priorities and generate additional hashes. Create symlinked files to |
26 |
the original file using these hashes (overwriting or not existing |
27 |
symlinks based on policy). This allows the binary package to be found |
28 |
using either an exact set of attributes or a subset of higher-priority |
29 |
attributes. This is analogous to shared object symlinking. |
30 |
4. The package manager will look for a binary package first using the |
31 |
user's full config, and then by dropping optional elements of the |
32 |
config (so maybe it does the search without CFLAGs, then without USE |
33 |
flags). Eventually it aborts based on user prefs (maybe the user only |
34 |
wants an exact match, or is willing to accept alternate CFLAGs but not |
35 |
USE flags, or maybe anything for the arch is selected. |
36 |
5. As always the final selected binary package still gets evaluated |
37 |
like any other binary package to ensure it is usable. |
38 |
|
39 |
Such a system can identify whether a potentially usable file exists |
40 |
using only filename, cutting down on fetching. In the interests of |
41 |
avoiding useless fetches we would only carry step 3 reasonably far - |
42 |
packages would have to match based on architecture and any dynamic |
43 |
linking requirements. So we wouldn't generate hashes that didn't |
44 |
include at least those minimums, and the package manager wouldn't |
45 |
search for them. |
46 |
|
47 |
Obviously you could do more (if you have 5 combinations of use flags, |
48 |
look for the set that matches most closely). That couldn't be done |
49 |
using hashes alone in an efficient way. You could have a small |
50 |
manifest file alongside the binary package that could be fetched |
51 |
separately if the package manager wants to narrow things down and |
52 |
fetch a few of those to narrow it down further. |
53 |
|
54 |
Or you could skip the hash searching and just fetch all the manifests |
55 |
for a particular package/arch and just search all of those, but that |
56 |
is more data to transfer just to do a query. A metadata cache of some |
57 |
kind of might be another solution. Content hashes would probably |
58 |
still be useful just to allow co-existence of alternate builds. |
59 |
|
60 |
-- |
61 |
Rich |