Gentoo Logo
Gentoo Spaceship




Note: Due to technical difficulties, the Archives are currently not up to date. GMANE provides an alternative service for most mailing lists.
c.f. bug 424647
List Archive: gentoo-scm
Navigation:
Lists: gentoo-scm: < Prev By Thread Next > < Prev By Date Next >
Headers:
To: gentoo-scm@g.o
From: "Robin H. Johnson" <robbat2@g.o>
Subject: Converting a recent CVS copy - Item 2: statistics
Date: Tue, 14 Apr 2009 01:33:24 -0700
Notes:
- Total conversion time on the order of 20 hours of wallclock.
- cvs2svn seems to be entirely single-threaded :-(. 
- Dropped: chriswhite's vendorbranch
- Dropped: branches: 'RELEASE-1_4', 'MAIN'
- Dropped: tags: 'GENTOO_1_4_SNAP_2003010800', 'gentoo_1_0_rc5', 'gentoo_1_0_rc1'
- DID run with Linus's memory improvements. Helped :-).

TODO: 
- Could somebody with Python-foo please look hard at cvs2svn with an eye to
  making it multi-threaded?
  - Focus on pass1 and pass9.
  - pass8 maybe as well, but I think it will be harder by design.
- We need incremental conversion stuff badly.
- I had to use the RCSRevisionReader, as InternalRevisionReader seemed to be
  broken. Would make pass1 faster as well.
- Should probably ignore the '.frozen' files.
- Review RCS state of ALL ,v files. There are a few non-dead files in Attic.
- Maybe trim out the Manifest/digest contents during the conversion, leaving
  only DIST lines? 
  Con: _WOULD_ break old GPG signatures.
  Pro: probably help size a lot.

Raw number fun:

cvs2svn Statistics:
------------------
Total CVS Files:            353491
Total CVS Revisions:       2135193
Total CVS Branches:              0
Total CVS Tags:                  0
Total Unique Tags:               0
Total Unique Branches:           0
CVS Repos Size in KB:      1484628
Total SVN Commits:          631683
First Revision Date:    Fri Jul 28 00:35:42 2000
Last Revision Date:     Sun Apr 12 21:54:28 2009
------------------
Timings (seconds):
------------------
36204   pass1    CollectRevsPass
   69   pass2    CleanMetadataPass
    0   pass3    CollateSymbolsPass
  334   pass4    FilterSymbolsPass
    5   pass5    SortRevisionSummaryPass
    0   pass6    SortSymbolSummaryPass
  443   pass7    InitializeChangesetsPass
 4778   pass8    BreakRevisionChangesetCyclesPass
22499   pass9    RevisionTopologicalSortPass
   95   pass10   BreakSymbolChangesetCyclesPass
  382   pass11   BreakAllChangesetCyclesPass
  338   pass12   TopologicalSortPass
  801   pass13   CreateRevsPass
    0   pass14   SortSymbolsPass
    0   pass15   IndexSymbolsPass
  704   pass16   OutputPass
66652   total

real    1110m55.837s
user    591m49.479s
sys     535m40.497s

git-fast-import statistics:
---------------------------------------------------------------------
Alloc'd objects:    5115000
Total objects:      5114877 (    128330 duplicates                  )
      blobs  :      1915756 (    107039 duplicates    1635294 deltas)
      trees  :      2567439 (     21291 duplicates    2311113 deltas)
      commits:       631682 (         0 duplicates          0 deltas)
      tags   :            0 (         0 duplicates          0 deltas)
Total branches:           1 (         1 loads     )
      marks:     1073741824 (   2654477 unique    )
      atoms:         287640
Memory total:        244856 KiB
       pools:         45051 KiB
     objects:        199804 KiB
---------------------------------------------------------------------
pack_report: getpagesize()            =       4096
pack_report: core.packedGitWindowSize = 1073741824
pack_report: core.packedGitLimit      = 8589934592
pack_report: pack_used_ctr            =          1
pack_report: pack_mmap_calls          =          1
pack_report: pack_open_windows        =          1 /          1
pack_report: pack_mapped              =  304718192 /  304718192
---------------------------------------------------------------------
real    16m3.034s
user    14m18.158s
sys     1m42.582s

$ time git repack -adf --window=250 --depth=250 2>&1
real    19m59.339s
user    96m48.011s
sys     0m36.914s

Pack output size: 
883MiB.

-- 
Robin Hugh Johnson
Gentoo Linux Developer & Infra Guy
E-Mail     : robbat2@g.o
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85
Attachment:
pgpY7dRY9Dgr7.pgp (PGP signature)
Navigation:
Lists: gentoo-scm: < Prev By Thread Next > < Prev By Date Next >
Previous by thread:
Converting a recent CVS copy - Item 1: mailmap fun
Next by thread:
Converting a recent CVS copy - Item 3: pushing, availability
Previous by date:
Converting a recent CVS copy - Item 1: mailmap fun
Next by date:
Converting a recent CVS copy - Item 3: pushing, availability


Updated Jun 17, 2009

Summary: Archive of the gentoo-scm mailing list.

Donate to support our development efforts.

Copyright 2001-2013 Gentoo Foundation, Inc. Questions, Comments? Contact us.