Gentoo Archives: gentoo-scm

From: "Robin H. Johnson" <robbat2@g.o>
To: gentoo-scm@l.g.o
Subject: [gentoo-scm] Progress summary, 2009/06/01
Date: Mon, 01 Jun 2009 23:46:32
Message-Id: 20090601234549.GA3042@orbis-terrarum.net
1 Update on this TODO list, and extension for new items.
2 I've deliberately broken the threading so that more people read it. This
3 was a response to the mail with the subject of "Converting a recent CVS
4 copy - Item 2: statistics"
5
6 Executive summary:
7 - We've gone from 18.5 hours to 9 hours, all in a single portion of the
8 conversion, thanks to help from upstream. There's lots more room for
9 improvement.
10 - C (for git) and Python (for cvs2svn) coders very welcome to challenge
11 the problems.
12 - Testers wanted
13 - Actually forming this up to a project with a team is probably due
14 soon. I've had interest/direct offers of help from: WilliamH, Calchan,
15 Betelgeuse over the last week.
16 If you want something, pick it from this mail, and try to flesh it out
17 on the list with me while you work on it.
18
19 New TODO items:
20 - Finish new hooks for git:
21 upload-pack.c - get_common_commits:
22 right before the 'return 0', check for and launch a new hook, passing
23 the have/want headers via stdin, and use the return code (AND the
24 stderr) to see if we should halt.
25 - Review commit signing
26 - pclouds (a former Gentoo dev) contributed this prototype:
27 http://thread.gmane.org/gmane.comp.version-control.git/115562/focus=118788
28 - I'm not entirely convinced the above is right, as the commit message
29 seems to end up unsigned.
30 - Wait for the commit-notes patches onto upstream Git?
31 - Test git-cvsserver usage.
32 - Both remote and local modes.
33 - mips and narrow checkouts may want this.
34
35 On Tue, Apr 14, 2009 at 01:33:24AM -0700, Robin H. Johnson wrote:
36 > TODO:
37 > - Could somebody with Python-foo please look hard at cvs2svn with an eye to
38 > making it multi-threaded?
39 > - Focus on pass1 and pass9.
40 > - pass8 maybe as well, but I think it will be harder by design.
41 - pass1 optimization was completed 20 minutes ago by mhagger (one of the
42 cvs2svn upstream lead developers). From an old time of 36204 seconds,
43 it's now only 1598 seconds (and we only used 2 CPU cores so far, we
44 have 6 more for later). 22x speedup :-).
45 - mhagger from upstream needed hardware to test on, so I hooked him up
46 with access to the experimental conversion box.
47 - TODO: pass9/pass8 remain.
48 - Need to validate output of new mode against the previous mode.
49
50 > - We need incremental conversion stuff badly.
51 Incremental may become practically unneeded if we can get the conversion
52 time under 2 hours.
53
54 > - I had to use the RCSRevisionReader, as InternalRevisionReader seemed to be
55 > broken. Would make pass1 faster as well.
56 This is now completely unneeded. The pass1 solution integrated the
57 rcsparse code into the new generate_blobs.py.
58
59 > - Should probably ignore the '.frozen' files.
60 No support to exclude files presently.
61 TODO: Implement inside def _generate_cvs_files
62
63 > - Review RCS state of ALL ,v files. There are a few non-dead files in Attic.
64 Thanks to William Hubbs (williamh) for his review. I've got a few more
65 items to process from this review.
66
67 > - Maybe trim out the Manifest/digest contents during the conversion, leaving
68 > only DIST lines?
69 > Con: _WOULD_ break old GPG signatures.
70 > Pro: probably help size a lot.
71 Additional con:
72 - I think it will massively slows down the conversion right now.
73 - Nothing more than a bad prototype I did. I'd like somebody else to
74 attack the problem without having seen my prototype, rather than be
75 infected with my bad ideas in it.
76
77 Upstream misc:
78 - Subtree checkouts
79 - Is progressing upstream. Now known as checkout modes: narrow,
80 sparse, shallow
81 = narrow: some directory that is not the root.
82 = sparse: a subset of files in a directory.
83 = shallow: subset of recent history.
84
85 --
86 Robin Hugh Johnson
87 Gentoo Linux Developer & Infra Guy
88 E-Mail : robbat2@g.o
89 GnuPG FP : 11AC BA4F 4778 E3F6 E4ED F38E B27B 944E 3488 4E85

Replies