Gentoo Archives: gentoo-dev

From: Joshua Kinard <kumba@g.o>
To: "Michał Górny" <mgorny@g.o>
Cc: gentoo-dev@l.g.o
Subject: Re: [gentoo-dev] new profile layout with flavors and mix-ins
Date: Thu, 03 Jul 2014 08:21:23
Message-Id: 53B5126F.5070308@gentoo.org
In Reply to: Re: [gentoo-dev] new profile layout with flavors and mix-ins by "Michał Górny"
1 On 07/03/2014 03:32, Micha³ Górny wrote:
2 > Dnia 2014-07-03, o godz. 02:18:23
3 > Joshua Kinard <kumba@g.o> napisa³(a):
4 >
5 >> [...]
6 >>
7 >> And so on. The goal was to have profiles/ be extremely flat, with limited
8 >> nesting only for categorization purposes. Each component would contain all
9 >> of the information specific to that component, and rely on OOP-like
10 >> inheritance to override parent-level variables only within that component
11 >> (although, <arch> and <kernel> would have to be a little bit more broad in
12 >> scope).
13 >
14 > Another sub-thread, the same question: how do you handle information
15 > that needs to be applied to the intersection of the profiles? CHOST for
16 > amd64 FreeBSD, for example.
17
18 Well, as I stated, I thought the bulk of this up *years* ago (at least 6-7
19 years ago, probably longer). The issues you're raising now weren't issues
20 back then. I only updated it slightly based on what I know now about how we
21 do things and what things are supported in Gentoo. I am only proposing it
22 as a base from which others to think about and either build upon or enhance,
23 or strike it down.
24
25
26 >> PROFILE also had a set order for the first few pieces, if only to make
27 >> parsing and building the profile stack saner for the Portage developers:
28 >> PROFILE="base:<KERNEL>:<ARCH>[:<SUBARCH>]:<LIBC>:<EVERYTHING_ELSE>"
29 >>
30 >> base - Core Gentoo/Portage/whatever information/vars/foobar/kittens
31 >
32 > I'm against plain 'base' since it quickly becomes a mess. I would
33 > prefer having flavor-specific bases, like for example arch/base to mask
34 > stuff available only in 1/2 arches and revert it in another arch/.
35
36 The only reason I put a 'base' folder in was so that where was some kind of
37 anchor point for Portage to build off of. I didn't put any thought into
38 what it might actually contain. For all intents and purposes, it might just
39 contain empty variable declarations, kinda like an abstract base class in an
40 OOP language. By itself, it's unusable and must be overridden, but it's the
41 point from which everything derives.
42
43 Maybe that's not needed now in current Portage. Dunno.
44
45
46 >> kernel - Specifies the OS kernel. At the time, only Linux existed, but
47 >> I *think* Debian was eyeballing kFreeBSD at the time. So I *think*
48 >> I accounted for it back then.
49 >
50 > What about kernel-ABI intersection? What about Prefix?
51
52 I know very little about Prefix. Any new profile system we create should
53 focus only what is in-tree *first*, then once we work the bugs and kinks
54 out, we can figure out how Prefixes will be incorporated.
55
56 But that said, what little I do know about Prefix is that it's Portage
57 ported to other OSes, like Solaris, MacOS, Interix, IRIX, etc. I guess an
58 out-of-tree Prefix overlay would just define missing bits in its profiles/*
59 folder. I.e., for Solaris, you'd have a kernel/solaris, libc/solaris-libc, etc.
60
61 I'm not the best at OOP-designs, but if we set the in-tree stuff up right,
62 it should be trivial for out-of-tree overlays to extend and override to
63 support things, without going down the endlessly-nested directory snafu that
64 the current profile system is. In-tree should be as flat as possible,
65 Prefix and other external overlays can extend it out however far they want.
66
67 As for kernel-ABI intersection, can you expand on that a little more? A
68 given kernel can support multiple ABIs simultaneously. Using MIPS as the
69 classic example, a 32-bit MIPS kernel only does o32, but a 64-bit kernel
70 does n64 natively, and you can enable o32 and n32 support individually
71 within the configuration. And MIPS has even more obscure ABIs that aren't
72 well-known, either.
73
74
75 >> arch - Machine arch-specific generic information -- doesn't handle
76 >> lower-level things like ISAs/ABIs/Endian, etc.
77 >>
78 >> subarch - Defines items specific given to given subarch of a main arch.
79 >> Items under this directory would carry their parent arch's
80 >> name for clarity only. Again, at the time, MIPS would have been
81 >> the only real user of this, and, back then, it would've dealt
82 >> with SGI-machine-specific packages and USE flags, such as
83 >> differentiating an ip32 machine from an ip27 machine or a
84 >> cobalt. Now that amd64/x86_64 is considering its own form of
85 >> n32 (x32), that would go here, and I imagine arm would
86 >> make heavy use of it as well.
87 >
88 > This is a no-go for multilib support. We need to work on a consistent
89 > arch profile tree, not more splitting.
90
91 Well, it'll have to be nested somehow. A given arch can handle multiple
92 ABIs, so there should at least be a top-level arch/ folder that defines very
93 high-level, very generic things specific to a given arch. How we fold the
94 multilib and multiple ABI stuff in probably needs a lot of thought. It may
95 not be possible with this proposed idea, so, if not, we'll need to think of
96 something else.
97
98 Also keep in mind that with MIPS, we up the fun fun with the ISA
99 (Instruction Set Architecture) mess (mips1 to mips4, mips32r2, mips64r2,
100 etc), which are also mostly independent of the ABI (meaning, n32/n64 on
101 mips3 or greater, but o32 only for mips1 and mips2. No idea about mips32r*
102 -- other MIPS devs will have to speak for those).
103
104
105 >> libc - Defines the main C library used. A bit redundant for *BSD, because
106 >> they really only have one, but might help if we ever add k*BSD
107 >> support (such as freebsd/glibc-based or such). For Linux...could be
108 >> tricky, since there are so many libc possibilities there. I feel it
109 >> fits better getting defined after <SUBARCH>, especially in the case
110 >> of MIPS.
111 >
112 > I think you need to handle kernel & libc in the same group.
113
114 If this was BSD, maybe. But Linux the kernel is independent from the
115 userland libc. You can have Linux/uclibc, Linux/glibc, and even Linux/musl,
116 among others. And as Debian has shown, kFreeBSD marries a FreeBSD kernel
117 with a GNU glibc-based userland (which probably chafes diehard BSD users).
118
119 Hence the purpose of me breaking them up that way was to make individual
120 elements as discrete as possible. I.e., the old UNIX philosophy of having
121 lots of little things that each do one thing very, very well. Not a few big
122 things that try to do many things poorly or halfway (like the IOC3 chip in
123 certain SGI systems, but I digress).
124
125 And, by organizing them one directory deep, it even adheres to our current
126 Portage philosophy of <category>/<package>, which also would keep
127 /usr/portage/profiles somewhat clean (once we got rid of the old stacking
128 system many years down the road).
129
130
131 >> eapi - EAPI didn't exist back when this topic was visited. Basically,
132 >> everything was EAPI=0 and there was only Portage (Paludis nor
133 >> pkgcore had been invented yet).
134 >
135 > And what is this supposed to do?
136
137 I dunno, I threw it in there just in case there was some nit about EAPIs
138 that I didn't know about that could be defined at a profile level. If it's
139 not relevant, then ignore it.
140
141
142 >> releases - I don't think this existed back then. How it'd operate
143 >> nowadays, I have no idea. Probably would contain
144 >> version-specific maskings for specific @system packages
145 >> to facilitate upgrading, and help us if we ever tried to cut
146 >> actual, fixed release points again (like what FreeBSD does
147 >> w/ -RELEASE-x.y)
148 >
149 > I don't think releases would make any sense without heavy intersection
150 > with other profiles.
151
152 Possibly. It's just another thing to think about, though. Arguably, if we
153 ever migrate the tree to git, setting release-specific branches is better
154 done there, though. But, it's just building on how we currently do
155 release-specific profiles in the tree. Maybe it's not need anymore either.
156
157 Brainstorming doesn't mean everything I am proposing is right. It might all
158 be wrong. Some of it might be right. Maybe it inspires someone to think of
159 something better. But if I don't put it out there, it'll never get
160 discussed in the first place.
161
162 --
163 Joshua Kinard
164 Gentoo/MIPS
165 kumba@g.o
166 4096R/D25D95E3 2011-03-28
167
168 "The past tempts us, the present confuses us, the future frightens us. And
169 our lives slip away, moment by moment, lost in that vast, terrible in-between."
170
171 --Emperor Turhan, Centauri Republic