Re: [gentoo-dev] Re: My masterplan for git migration (+ looking for infra to test it)

[W. Trevor King]

On Wed, Sep 17, 2014 at 10:36:45AM +0200, Michał Górny wrote:
> Dnia 2014-09-16, o godz. 10:52:13
> W. Trevor King napisał(a):
> >   $ git pull --depth=1
> >
> > for subsequent syncs.  pym/_emerge/actions.py currently hardcodes
> > ‘git pull’ for the latter, and doesn't seem to have any code for
> > the former.  On the other hand, it wouldn't be too terrible to
> > force users to shallow their history manually whenever they felt
> > like it.
>
> This isn't a good idea at all :). For git, --depth=1 fetching is the
> same thing as --depth=1 clone. This way, you refetch everything
> rather than just getting the update.

You don't refetch everything, but the pull fails because it doesn't
know how the original and new shallow commits are related (so it can't
(ff-)merge them).  It works if you fetch and reset (skipping the
merge).  Here's my test script:

  #!/bin/sh

  rm -rf upstream local-1 local-2 &&
  mkdir upstream &&
  (
    cd upstream &&
    git init &&
    echo 'Some project' >README &&
    git add README &&
    git commit -m 'Start the project' &&
    for X in 1 2 3 4 5 6 7 8 9
    do
				echo "${X}" >"${X}" &&
    		git add "${X}"
    done &&
    git commit -m 'Add a bunch of dummy files'
  ) &&
  git clone --depth 1 "file://${PWD}/upstream" local-1 &&
  (
    cd upstream &&
    echo 'Build with ...' >>README &&
    git commit -am 'Add building instructions' &&
    echo 'Test with ...' >>README &&
    git commit -am 'Add testing instructions'
  ) &&
  (
    cd local-1 &&
    git fetch --depth 1 &&
    git reset --hard origin/master &&
    git --no-pager log --oneline --decorate &&
    du -s . &&
    git reflog expire --expire=now --all &&
    git gc --aggressive --prune=now &&
    du -s .
  ) &&
  git clone --depth 1 "file://${PWD}/upstream" local-2 &&
  (
    cd local-2 &&
    du -s .
  )

and here are some excerpts of it's output:

* The shallow fetch only pulls in three objects (the new README, tree,
  and commit):

  remote: Counting objects: 3, done.
  remote: Compressing objects: 100% (2/2), done.
  remote: Total 3 (delta 1), reused 0 (delta 0)
  Unpacking objects: 100% (3/3), done.
  From file:///tmp/upstream
   + 73f6253...5abbe64 master     -> origin/master  (forced update)
  HEAD is now at 5abbe64 Add testing instructions

* After the shallow fetch and reset, local-1 has a shallow history:

  5abbe64 (grafted, HEAD, origin/master, origin/HEAD, master) Add testing instructions

* But it still has references to the old master in the reflog, and
  that takes up some space:

  168     .

* Expiring the reflog and garbage collectiong gets us that space back
  (although in practice, I'd just let Git expire these automatically
  in the course of time):

  Counting objects: 12, done.
  Delta compression using up to 2 threads.
  Compressing objects: 100% (2/2), done.
  Writing objects: 100% (12/12), done.
  Total 12 (delta 0), reused 9 (delta 0)
  140     .

* A fresh shallow clone gets all 12 objects (not just the three new
  ones):

  Cloning into 'local-2'...
  remote: Counting objects: 12, done.
  remote: Compressing objects: 100% (2/2), done.
  remote: Total 12 (delta 0), reused 0 (delta 0)
  Receiving objects: 100% (12/12), done.
  Checking connectivity... done.

* And takes up as much space as our garbage-collected local-1:

  140     .

Again, I'm happy to leave it to users to manually

  $ git fetch --depth 1
  $ git reset --hard origin/master

to shorten their history, but I expect many will not bother, and then
get annoyed as that unpurged history takes up more and more space ;).
In any case, we don't have to resolve this before the transition.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy