Re: [gentoo-scm] Git Validation - Update

[Robin H. Johnson]

On Tue, Oct 16, 2012 at 08:42:40PM +0200, Michael Haggerty wrote:
> 1. It seems like you are doing an awful lot of work and using a lot of
> movable parts to reduce the switchover time.  Just how much traffic does
> your repo get?  Is it really worth the effort to reduce the downtime by
> a couple of hours?
gentoo-x86 CVS gets ~500 commits/day. ~250/day if you consider the
second pass of the Manifest commit as part of the same logical commit as
the first pass of the Manifest.

> 2. It seem to me to be overkill to insist on a repository validation
> between the conversion and switching the repository live.  I would suggest
>
>     do {
>         Adjust conversion scripts
>         Copy CVS repo
>         Do test conversion
>         Validate converted repository
>     } until (validation perfect);
>
>     Switch CVS repo to read-only
>     Do final conversion
>     Switch git repo live
>     Validate converted repository at leisure for your peace of mind
This is already the path we're taking, with the additional steps:
- Ensure that the conversion output is exactly reproducible, and record
  the commits that have already been validated as correct (because you
  can skip them on the next iteration of the loop).
- Validate the commits between the last validation success and the
  readonly point.

Realistically, we're going to iterate that loop until we're happy with
it, then lock read-only, take a final pass of the loop (with the
recording of already validated commits and reproducible output, it
shouldn't take long to validate the delta at all).

> > Once I have both I can start working on validation rules and perhaps
> > get feedback to the conversion team.  We'll need to work out what does
> > and doesn't count as OK.  We're doing transformation of data during
> > migration, so I need to take that into account.  Either the logic goes
> > into the compare function, or the logic goes into the dump side so
> > that the compares work out the same.  Timestamps might force us to do
> > logic during compare anyway.
> What kind of data transformations are you doing during the migration?
1. $Header$ expansion is needed to confirm that our Manifest files have
   entries matching the checksums of the files in a given commit.
1.1. Caveat: Make a list of CVS commits w/ known non-matches.
2. Potentially: validation of GPG signatures of Manifests.
3. Ongoing discussion about converting some non-signed Manifests to
   thin-Manifest format.

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Trustee & Infrastructure Lead
E-Mail     : robbat2@g.o
GnuPG FP   : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85