Gentoo Archives: gentoo-scm

From: "Robin H. Johnson" <robbat2@g.o>
To: gentoo-scm@l.g.o
Subject: [gentoo-scm] Converting a recent CVS copy - Item 2: statistics
Date: Tue, 14 Apr 2009 08:33:31
Message-Id: 20090414T081305Z@curie.orbis-terrarum.net
1 Notes:
2 - Total conversion time on the order of 20 hours of wallclock.
3 - cvs2svn seems to be entirely single-threaded :-(.
4 - Dropped: chriswhite's vendorbranch
5 - Dropped: branches: 'RELEASE-1_4', 'MAIN'
6 - Dropped: tags: 'GENTOO_1_4_SNAP_2003010800', 'gentoo_1_0_rc5', 'gentoo_1_0_rc1'
7 - DID run with Linus's memory improvements. Helped :-).
8
9 TODO:
10 - Could somebody with Python-foo please look hard at cvs2svn with an eye to
11 making it multi-threaded?
12 - Focus on pass1 and pass9.
13 - pass8 maybe as well, but I think it will be harder by design.
14 - We need incremental conversion stuff badly.
15 - I had to use the RCSRevisionReader, as InternalRevisionReader seemed to be
16 broken. Would make pass1 faster as well.
17 - Should probably ignore the '.frozen' files.
18 - Review RCS state of ALL ,v files. There are a few non-dead files in Attic.
19 - Maybe trim out the Manifest/digest contents during the conversion, leaving
20 only DIST lines?
21 Con: _WOULD_ break old GPG signatures.
22 Pro: probably help size a lot.
23
24 Raw number fun:
25
26 cvs2svn Statistics:
27 ------------------
28 Total CVS Files: 353491
29 Total CVS Revisions: 2135193
30 Total CVS Branches: 0
31 Total CVS Tags: 0
32 Total Unique Tags: 0
33 Total Unique Branches: 0
34 CVS Repos Size in KB: 1484628
35 Total SVN Commits: 631683
36 First Revision Date: Fri Jul 28 00:35:42 2000
37 Last Revision Date: Sun Apr 12 21:54:28 2009
38 ------------------
39 Timings (seconds):
40 ------------------
41 36204 pass1 CollectRevsPass
42 69 pass2 CleanMetadataPass
43 0 pass3 CollateSymbolsPass
44 334 pass4 FilterSymbolsPass
45 5 pass5 SortRevisionSummaryPass
46 0 pass6 SortSymbolSummaryPass
47 443 pass7 InitializeChangesetsPass
48 4778 pass8 BreakRevisionChangesetCyclesPass
49 22499 pass9 RevisionTopologicalSortPass
50 95 pass10 BreakSymbolChangesetCyclesPass
51 382 pass11 BreakAllChangesetCyclesPass
52 338 pass12 TopologicalSortPass
53 801 pass13 CreateRevsPass
54 0 pass14 SortSymbolsPass
55 0 pass15 IndexSymbolsPass
56 704 pass16 OutputPass
57 66652 total
58
59 real 1110m55.837s
60 user 591m49.479s
61 sys 535m40.497s
62
63 git-fast-import statistics:
64 ---------------------------------------------------------------------
65 Alloc'd objects: 5115000
66 Total objects: 5114877 ( 128330 duplicates )
67 blobs : 1915756 ( 107039 duplicates 1635294 deltas)
68 trees : 2567439 ( 21291 duplicates 2311113 deltas)
69 commits: 631682 ( 0 duplicates 0 deltas)
70 tags : 0 ( 0 duplicates 0 deltas)
71 Total branches: 1 ( 1 loads )
72 marks: 1073741824 ( 2654477 unique )
73 atoms: 287640
74 Memory total: 244856 KiB
75 pools: 45051 KiB
76 objects: 199804 KiB
77 ---------------------------------------------------------------------
78 pack_report: getpagesize() = 4096
79 pack_report: core.packedGitWindowSize = 1073741824
80 pack_report: core.packedGitLimit = 8589934592
81 pack_report: pack_used_ctr = 1
82 pack_report: pack_mmap_calls = 1
83 pack_report: pack_open_windows = 1 / 1
84 pack_report: pack_mapped = 304718192 / 304718192
85 ---------------------------------------------------------------------
86 real 16m3.034s
87 user 14m18.158s
88 sys 1m42.582s
89
90 $ time git repack -adf --window=250 --depth=250 2>&1
91 real 19m59.339s
92 user 96m48.011s
93 sys 0m36.914s
94
95 Pack output size:
96 883MiB.
97
98 --
99 Robin Hugh Johnson
100 Gentoo Linux Developer & Infra Guy
101 E-Mail : robbat2@g.o
102 GnuPG FP : 11AC BA4F 4778 E3F6 E4ED F38E B27B 944E 3488 4E85