Re: [gentoo-dev] [RFC] gentoo-x86 migration to repo-per-package

[Robin H. Johnson]

On Sat, Aug 06, 2011 at 04:13:52PM +0200, Fabian Groffen wrote:
> When we migrate away from CVS for gentoo-x86 (gx86), as it looks now,
> the same structure will be kept as we have in CVS now.  Policies to
> reject merge commits and only allow rebases on e.g. the Git
> infrastructure will even more closely match the central and
> server-based way of working Gentoo is used to now.
The discussion about rejecting merges was never completed IIRC. I think
there may be some very valid cases where we need merges still (esp the
big atomic commit cases from KDE/GNOME), but they should still be used
sparingly. Additionally, the rebase problem has problems of requiring
everybody else to hard-reset their trees if they have pushed to multiple
places, then rebase to push to the main tree, so I don't know if that
will actually fly.

> In this email, I step away from the current model that Gentoo uses for
> the gentoo-x86 repository.  Instead, I consider a repo-per-package
> model, as in use by e.g. Fedora [1] and Debian [2].
Everything you have mentioned here was previously covered in the
discussions about Git conversion models. Please consult the history of
this list, as well as the -scm list. Additionally, a large discussion
about the pros and cons of all 3 models (package per repo, category per
repo, single repo) was had at the GSoC mentor summit last year, and a
number of the core Git developers were involved in the discussion.

Problems:
- atomic/well-ordered commits that span packages, eclasses and profiles/
  directories. (Esp. committing to eclasses and then packages
  afterwards).
- Massive space overhead: Every .git directory requires a minimum of 25
  inodes [1], covering at least 100KiB. We have 15k packages in the tree
  right now. Assuming there is no tail-packing in use, that's a minimum
  of 1.5GiB on .git overhead.
- Massive space overhead(2): Having a repo per package also removes ANY
  git compression advantage that would be gained where ebuilds between
  packages are substantially similar. The _complete_ history packfile
  for the Tree right is under 1GiB [2].
- Pain in branching/forking: instead of being able to just have your own
  local clone of the single git repo, a user wanting to work on multiple
  packages together would need to have repos for ALL of them. No
  pull/merge ability at all.

[1] Git space usage testcase:
mkdir foo && cd foo && git init \
&& touch bar && git commit -m '.' bar \
&& git gc && du .git --exclude '*.sample' && find .git ! -name
'*.sample' |wc -l

[2] Packfile size:
The final proposal regarding packfile size was that we were going to
partition older history using grafts, similar to when Linus moved the
kernel into Git, and had a graft available of the old history. Initial
packfile size was under 50MiB.

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Trustee & Infrastructure Lead
E-Mail     : robbat2@g.o
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85