[gentoo-dev] RFC: git-r3, additional clone types

[Michał Górny]

Hello, all.

Following the late cleanup to git-r3, I'd like to discuss another idea
that aims to most effectively satisfy the needs of various users,
developers and packages.

The idea is to provide three different clone types: mirror clone, single
branch clone and shallow clone. User would select one of the types
through make.conf (and per-package environment), while packages could
specify a 'minimal' supported clone type.


Mirror clone
------------

- current git-r3 behavior (after the cleanup),

- all branches and tags are updated from the remote repo (and pruned
  following removals in remote),

- git notes are fetched,

- EGIT_COMMIT can name any commit in any branch or tag,

- best for developer use (ability to clone equivalent to remote)
  and local mirrors,

- repositories with many outdated and diverged branches may be
  unnecessarily large.


Single branch clone
-------------------

- git-2 eclass behavior,

- only requested branch (or tag) is fetched, and tags on the commits
  fetched as part of the branch/tag history,

- git notes are fetched,

- EGIT_COMMIT can only name a commit in the fetched EGIT_BRANCH,

- pruning unused branches/tags is hard (and possibly won't be
  implemented at first),

- more efficient than the mirror mode, most likely future default.


Shallow clone
-------------

- similar to old git-r3 behavior,

- minimal number of objects from the requested branch (or tag) is
  fetched,

- branch/tag history is not preserved,

- git notes are not fetched,

- EGIT_COMMIT can only name tags (using a hash auto-forces higher mode),

- changing branches may be very inefficient (since it implies
  re-fetching all objects implied by --depth 1),

- since the history is truncated, 'git describe' won't give pretty
  names,

- most space-saving, intended for embedded and other special use cases,

- does not guarantee repository consistency -- unsafe.


Control variables
-----------------

I'm thinking of two control variables (better names appreciated):

- EGIT_CLONE_MODE - set in make.conf by user to requested clone mode,

- EGIT_MIN_CLONE_MODE - optionally set by ebuilds that require more.

For example, if ebuilds sets EGIT_MIN_CLONE_MODE=single, and user sets
EGIT_CLONE_MODE=shallow, this single ebuild will be fetched
in single-branch mode. This can be used when build system operates
on repo history and doesn't work without it.

I can't think of a case when ebuild would need
EGIT_MIN_CLODE_MODE=mirror.


Mode upgrades and downgrades
----------------------------

Mode is not associated persistently with a repository. Therefore, using
a repository in a different mode than it was used before (e.g. due to
different ebuild or user preference change) results in mixed-mode
repository.

When mirror or single-branch mode is used on a shallow repository,
the repository is still marked 'shallow' even if the full history is
available. I don't know if this wouldn't break some of 'git foo' uses
in the checkout but that probably can't be predicted. Moreover, I don't
know if it is safe to remove 'shallow' after doing full-fetch in mirror
mode.

When single-branch or shallow mode is used on a mirror-mode repository,
only the requested branch/tag is updated. The repository may no longer
be correctly synced to the remote.

When shallow mode is used on *existing* branch or tag in a mirror-mode
or single-branch repository, new commits are fetched as
in single-branch mode. When shallow mode is used on *new* branch
or tag, the commits are fetched with '--depth 1' and the clone is
marked 'shallow'.


Old branch/tag pruning
----------------------

In mirror mode, pruning is pretty straightforward. Since all branches
and tags are requested, 'git fetch --prune' removes the branches
and tags that are removed upstream. The remaining branches and tags are
expected.

In single-branch and shallow mode, complete pruning is pretty much
impossible. Upstream-removed branches and tags could be supposedly
pruned using some explicit git-ls-remote magic.

Removing 'old' branches (that is, no longer referenced by ebuilds) is
pretty much impossible since one can't clearly determine whether
the particular branch/tag will be used by other ebuilds :). This
becomes especially blurry in case of mode switching.


That's pretty much it. Most of the stuff I've tested proof-of-concept.
It may not work as well in the actual eclass. Any thoughts?

-- 
Best regards,
Michał Górny