[gentoo-dev] Proposal: profiles/arches.desc - improve repoman flexibility (with other benefits)

[Andreas K. Huettel]

Hey all,

here's the updated proposal for a new "arches.desc" file, after feedback.

In short:
We introduce a new file "arches.desc" which essentially describes if an arch 
(not a profile) is stable or not. The meaning of profiles.desc is not affected; 
profiles.desc still describes single profiles of each arch.
This is introduced in particular to help moving arches from stable to testing 
while keeping the "~arch" deptree consistent, or moving arches from testing to 
stable and easily preparing a consistent "arch" deptree.

Details:

1] File location:
profiles/arches.desc    or   metadata/arches.desc


2] File format: whitespace-separated, two columns, lines starting with # are 
comments
first column: arch name ("amd64")
second column: one of the three values "stable", "testing", "unstable"
Example:
===========================
# Example arches.desc file
amd64	stable
mips	testing
m68k	unstable
===========================


3] Meaning of the three values "stable", "testing", "unstable" for repoman

* stable: When a profile of arch is tested, then repoman checks consistency for 
"arch" and for "~arch" separately. 
Which profiles of the arch are tested is still controlled by profiles.desc (and 
-d / -e switches). 
This is the current behaviour and should be the default if nothing is specified 
for an arch.

* testing: When a profile of arch is tested, then repoman treats "arch" as 
"~arch", and tests consistency only for "~arch".
Which profiles of the arch are tested is still controlled by profiles.desc (and 
-d / -e switches). 
A new switch for repoman may be provided to temporarily upgrade an arch from 
"testing" to "stable" (for arch team work).

* unstable: When a profile of arch is tested, then repoman treats "arch" as an 
error and aborts. Consistency is only tested for "~arch".
Which profiles of the arch are tested is still controlled by profiles.desc (and 
-d / -e switches). 

* optionally, additional value "broken":
Do not test this arch at all. (In my opinion not necessary, since we can 
always set all profiles of an arch to exp.)


4] Meaning for other tools
All arches set to "stable" are considered "stable arches", meaning
* they get listed first in eshowkw
* they require stabilization requests in bugzilla
* ...


5] Initial value in the Gentoo repository
On introduction, the setting will be "stable" for all stable arches, "testing" 
for all arches where "inofficial" stable keywords exist (sh, s390, ...), and 
"unstable" everywhere else. 


6] Rationale
At the moment we have several cases where repoman finds its limits:

a) An arch loses its stable status (imagine s390), but the arch team doesnt 
want to drop all the stable keywords since they are useful for stage building. 
Since stable keywords on s390 are an arch-team-internal thing, everyone can 
drop the latest stable version of a package, and it's the arch team's 
responsibility to keep their "unofficial stable tree" straight. 
Right now this means that we have to set all *profiles* of this arch to "exp", 
otherwise repoman throws errors about a broken stable deptree. If we do that, 
repoman does however also not check ~s390 consistency, meaning that the 
~s390 deptree will soon be broken as well.
With arches.conf, one could set (arch: testing, profile: stable), which results 
in stable keywords being ignored, but the ~s390 deptree still being enforced.

b) An arch prepares for becoming a stable arch (think arm64).
So, first the ~arm64 deptree should be fine, and then stable keywords can be 
added. However, to avoid repoman errors as long as the stable deptree isnt 
complete yet the profiles need to be set to dev/exp, and again this brings the 
danger of the ~arm64 deptree getting inadvertently broken. Again the 
combination (arch: testing, profile: stable) helps.

Finally, at the moment the "semi-official" algorithm to figure out if an *arch* 
is stable (i.e., requires stabilization requests etc bla bla) is to check if 
the arch has any stable profile. This makes non-stable arches which want to 
keep a consistent deptree (think mips) impossible. Reading the arch status 
from arches.desc instead solves this problem.


7] Compatibility

a) No arches.desc and new system, or arch not listed in arches.desc
Arches are treated as "stable" by repoman (current behaviour), with profile 
status according to profiles.desc. So, business as usual.
Gentoolkit and other tools trying to determine a list of stable arches should 
fall back to current method of scanning profiles.desc for stable profiles.

b) arches.desc and old system
Tools ignore the unknown file (?).
Repoman and other tools may emit surplus dependency errors when profiles are 
checked on arches that are "testing" (they check the consistency of the stable 
tree alone, which is not OK, since "arch" is supposed to be treated like 
"~arch"). This affects only development work and can be fixed by updating 
repoman.

c) PMS may need to be amended to allow the additional file. 


8] Several repositories

If arches.desc is present in several repositories, then the strictest setting 
for an arch wins. [I don't really see many usecases for this though.]


Congratulations for getting this far. What's your opinion?

-- 
Andreas K. Hüttel
dilfridge@g.o
Gentoo Linux developer (council, perl, libreoffice)