[gentoo-scm] Re: [gentoo-dev] Re: My masterplan for git migration (+ looking for infra to test it)

[Michał Górny]

Dnia 2014-09-14, o godz. 10:33:03
Rich Freeman <rich0@g.o> napisał(a):

> > Of course, that assumes infra is
> > going to cooperate quickly or someone else is willing to provide the
> > infra for it.
>
> The infra components to a git infrastructure are one of the main
> blockers at this point.  I don't really see cooperation as the issue -
> just lack of manpower or interest.

By 'cooperating' I simply meant offering the necessary resources
in a reasonable time.

> >
> > 1. send announcement to devs to explain how to use git,
>
> This is one of the blockers.  We haven't actually decided how we want
> to use git.
>
> Sure, everybody knows how to use git.  The problem is that there are a
> dozen different ways we COULD use git, and nobody has picked the ONE
> way we WILL use it.
>
> This isn't as trivial as you might think.  We have a fairly high
> commit rate and with a single repository that means that in-between a
> pull-merge/rebase-push there is a decent chance of another commit that
> will make the resulting push a non-fast-forward.
>
> People love to point out linux and its insane commit rate.  The thing
> is, the mainline git repo with all those commits has exactly one
> committer - Linus himself.  They don't have one big repo with one
> master branch that everybody pushes to.  At least, that is my
> understanding (and there are certainly others here who are more
> involved with kernel development).

It's hard to talk about commit rate when we combine crippled CVS with
awfully stupid two-part repoman committing. This forces us to commit
everything immediately, and makes some of us not committing anything
at all anymore...

With git, we can finally do stuff like preparing everything and pushing
in one go. Rebasing or merging will be much easier then, since
the effective push rate will be smaller than current commit rate.

> > On top of user sync repo rsync is propagated. The rsync tree is populated
> > with all old ChangeLogs copied from CVS (stored in 30M git repo), new
> > ChangeLogs are generated from git logs and Manifests are expanded.
>
> So, I don't really have a problem with your design.  I still question
> whether we still need to be generating changelogs - they seem
> incredibly redundant.  But, if people really want a redundant copy of
> the git log, whatever...

I don't want them too. However, I'm pretty sure people will bikeshed
this to death if we kill them... Especially that rsync has no git log.
Not that many users make real use of ChangeLogs, esp. considering
how useless messages often are there...

> > Main developer repo
> > -------------------
> >
> > I was able to create a start git repository that takes around 66M
> > as a git pack (this is how much you will have to fetch to start working
> > with it). The repository is stripped clean of history and ChangeLogs,
> > and has thin Manifests only.
> >
> > This means we don't have to wait till someone figures out the perfect
> > way of converting the old CVS repository. You don't need that history
> > most of the time, and you can play with CVS to get it if you really do.
> > In any case, we would likely strip the history anyway to get a small
> > repo to work with.
>
> We already have a migration process that coverts the old CVS
> repository, generating both a shallow repository that lacks history
> and a full repository that contains all of history. Additionally,
> these two are consistent - that is the last branch of the full
> repository has the same commit ID as the base of the shallow
> repository.  Basically we generate the full history and then trim out
> 99% of it so that the commit in the shallow repository points to a
> parent that isn't in the packed repository.
>
> Actually doing the conversion is basically a solved problem.  If this
> were actually the blocker I'd be all for just sticking the history in
> a different repo and starting from scratch with a new one.

Was the resulting tree actually verified? How long does the conversion
take? Can it be incremental, i.e. convert most of it, lock CVS, convert
the remaining new commits?

> > I think we should also merge gentoo-news & glsa & herds.xml into
> > the repository. They all reference Gentoo packages at a particular
> > state in time, and it would be much nicer to have them synced properly.
> >
>
> I can see the pros/cons here, but I don't personally have an issue
> with merging them.  As has been brought up elsewhere herds.xml may
> just go away.
>
> If somebody can come up with a set of hooks/scripts that will create
> the various trees and the only thing that is left is to get infra to
> host them, I think we can make real progress.  I don't think this is
> something that needs to take a long time.  The pieces are mostly there
> - they just have to be assembled.

Are you willing to champion that, then? :)

-- 
Best regards,
Michał Górny