[gentoo-scm] Re: [gentoo-dev] CVS -> git, list of where non-infra folk can contribute

[Rich Freeman]

On Tue, Oct 2, 2012 at 4:20 PM, Gregory M. Turner <gmt@×××××.us> wrote:
> Brian Harring wrote:
>>
>> replay it into git via tailor;
>>
>
> Never knew about that tool... not sure about the wisdom of adding an extra
> moving part just to keep the lights on for those few hours... Given the "2G
> of history" issue Diego mentioned, which if I understand correctly,
> effectively means that the future gentoo git can never rebase its commit
> history, why chance it?

I think that the reality is that we're going to have a million dress
rehearsals before we do the real thing.  Apparently right now the
conversion isn't quite right, and we can't validate that it is right
either.  I don't see any harm in having people look into being able to
keep the downtime low while others figure out how the migration works
in the first place.

Dress rehearsals don't need to even be announced.  You just grab a
snapshot of cvs at some random time and convert it and test it.  Then
you grab another snapshot at a later moment in time and try to use it
to catch up the converted repository.  Then you test it all again.  If
you can do that on demand without issue then I'd say we're ready to
go.

I do plan to mess around with validation as I posted yesterday.
Rather than dump a lot of time into a "clever" solution like Mapreduce
where I have no experience I'll probably just start with a single
threaded proof of concept and see just how long it takes.  I have
thought of ways to optimize things - you can descend the tree of all
the commits iteratively side-by-side, and at each step prune every
sub-tree that is a duplicate (with a little care to catch situations
where the tree might have been reverted).  That means that instead of
descending the entire tree for every commit you only actually descend
the branches that have changes on each commit, which in most cases
will just be a single branch anyway.  If the records don't proliferate
at each step then you're talking about an order of a few million
records to check each pass, with only a few passes - that might be
reasonable without much heavy equipment.  However, the job should
still be able to be run in parallel as long as you still run it in
stages.

I've got pseudocode for the git side - so I'll see what I can do with it.

Rich