Gentoo Archives: gentoo-dev

From: Fabian Groffen <grobian@g.o>
To: gentoo-dev@l.g.o
Subject: [gentoo-dev] [RFC] gentoo-x86 migration to repo-per-package
Date: Sat, 06 Aug 2011 14:14:58
1 All,
3 When we migrate away from CVS for gentoo-x86 (gx86), as it looks now,
4 the same structure will be kept as we have in CVS now. Policies to
5 reject merge commits and only allow rebases on e.g. the Git
6 infrastructure will even more closely match the central and server-based
7 way of working Gentoo is used to now.
9 In this email, I step away from the current model that Gentoo uses for
10 the gentoo-x86 repository. Instead, I consider a repo-per-package
11 model, as in use by e.g. Fedora [1] and Debian [2].
13 In short, the repo-per-package model means that each package
14 (my-cat/package) is a separate repository in some VCS.
15 Instead of having a huge tree that will only grow forever (gx86),
16 packages are just in their own repository.
18 This approach can potentially be interesting for a number of reasons:
19 - history is per package
20 + package is likely to be small enough not to have to consider any
21 history removal/splitting -- everything can be retained
22 + if a package is removed, it's repository is simply no longer
23 considered, hence its existence and history doesn't clobber a main
24 repository
25 + since the repository can move, its history can also easily move
26 along with its location, being either a category, or even as purpose
27 (e.g. packages that started on sunrise, or in developer overlays)
28 - tree generation is dynamic
29 + a full (rsync) tree has to be created by first getting all repositories,
30 and/or getting them up-to-date -- avoid those packages you don't
31 need
32 + easy to make different "trees", e.g. a server tree (no GNOME, KDE),
33 prefix tree (different versions of packages), etc.
34 + easy to move packages around, their category is specified by the
35 tree configuration, the repository the package lives in doesn't change,
36 probably overlays, betagarden, graveyard, sunset, etc. can all go
37 + no restriction to using only a single VCS, because packages are just
38 refreshed before inclusion in the tree, their (source) origin
39 doesn't matter
40 - per package branches
41 + instead of developing in overlays, simply branches could be used,
42 such that a single place is sufficient to for each package
43 + switching branches can implement atomic tree-wide changes for
44 complex situations
46 No restriction to using a only a single VCS.
47 While I don't think that allowing developers to use their VCS of choice
48 is very relevant when committing package changes, the ability to use
49 more than just *one* VCS when assembling the rsync tree is a huge
50 advantage if we want to migrate away from the current CVS tree slowly
51 during a migration period. It could enable the use of git (the obvious
52 choice of many) now, alongside the current gx86 tree.
54 Because the rsync tree would be generated by assembling all packages
55 that need to be in the tree, the only thing necessary for that
56 generation is to understand which VCS commands to use to acquire/update
57 a package and what files/directories to skip when copying the package to
58 its final destination in the rsync tree. This is easily scriptable,
59 given that only the old gx86 tree will be CVS, and the rest git.
61 When rsync tree generation would be based on a file with packages to
62 include, I can imagine a simple way define where the package comes from,
63 and where it should end up, e.g.:
65 my-cat;package1;git://git.g.o/foo-package.git;optionalbranch
66 my-cat;package2;cvs://cvs.g.o/gentoo-x86/my-cat/package2;
68 which defines the category, the package name, its source location, and
69 perhaps something like a branch or tag in case we ever want to e.g.
70 split development (what is now in overlays typically) from what's in the
71 tree. Branch support would also be useful for e.g. Prefix modifications
72 to a package, when only checked out by Prefix rsync tree generation. It
73 can as well be a solution for what is often referred to as the "slacker
74 arches" problem, when old versions of ebuilds that a maintainer wants to
75 drop, remain available for minority arches that need them, but only for
76 their rsync tree, without bothering the maintainer with it.
78 Obviously, in an ideal world all packages would be in the same VCS, git
79 in this case. With a system like this, however, CVS packages can slowly
80 be moved to git, as their maintainers see fit. Some developers aim to
81 benefit more from git than others. They can move their packages,
82 directly. For the remaining packages, eventual migration is necessary,
83 but they should block developers that want to use git for their packages
84 less.
86 There probably are drawbacks to this system as well. I, however, only
87 see big advantages for the moment.
88 Comments, thoughts, ideas welcome.
91 [1]
92 [2]
94 --
95 Fabian Groffen
96 Gentoo on a different level