Re: [gentoo-dev] Suggested sync method/Portage config for devs on ~arch?

[Brian Evans]

On 2/28/2017 5:14 AM, Thomas Deutschmann wrote:
> On 2017-02-28 10:52, James Le Cuirot wrote:
>> I use hasufell's repo too. I'm surprised we haven't made it more
>> official.
>
> The public Gentoo git mirror is
>
>   https://github.com/gentoo-mirror/gentoo
>
> This git mirror includes pre-generated metadata. No need for any
> hack/additional step.
>
> Devs maybe want to switch branch from stable (default branch) to master
> branch (stable branch has CI coverage and will only sync if everything
> is fine).
>
>

People, developers and users, may want to consider some facts when
cloning from that repository. (Please read the entire explanation below
before commenting.)

Git does a very poor job of data deduplication by default in this
repository and repacks are necessary to keep it sane.  Unfortunately,
there doesn't seem to be a way to trigger GitHub to do so.

This is not any developer's fault.  Simply a limitation or possible flaw
in git itself.

Shallow clones (--depth=1) are just fine.

Full clones are ridiculous in size because of needing a full repack
periodically.  GitHub even hung up on me for a full clone after about a
GiB downloaded. Infra's copy has been/is being repacked so the sizes are
not so bad.

If you don't need the full history, then this is very quick and small
footprint.
End users typically won't care but developers may.

Brian

Evidence:

Github...
> grknight@akame ~ $ git clone --depth=1 https://github.com/gentoo-mirror/gentoo.git
> Cloning into 'gentoo'...
> remote: Counting objects: 155625, done.
> remote: Compressing objects: 100% (128104/128104), done.
> remote: Total 155625 (delta 31631), reused 75529 (delta 26281), pack-reused 0
> Receiving objects: 100% (155625/155625), 80.36 MiB | 2.35 MiB/s, done.
> Resolving deltas: 100% (31631/31631), done.
> Checking out files: 100% (141011/141011), done.
> grknight@akame ~ $ du -sh gentoo/.git
> 101M    gentoo/.git
> grknight@akame ~ $ rm -fr gentoo
> grknight@akame ~ $ git clone https://github.com/gentoo-mirror/gentoo.git
> Cloning into 'gentoo'...
> remote: Counting objects: 3668662, done.
> remote: Compressing objects: 100% (677/677), done.
> error: RPC failed; curl 56 GnuTLS recv error (-54): Error in the pull function.
> fatal: The remote end hung up unexpectedly
> fatal: early EOF
> fatal: index-pack failed
> grknight@akame ~ $ rm -fr gentoo
> grknight@akame ~ $ git clone https://github.com/gentoo-mirror/gentoo.git
> Cloning into 'gentoo'...
> remote: Counting objects: 3668680, done.
> remote: Compressing objects: 100% (694/694), done.
> Receiving objects: 100% (3668680/3668680), 1.27 GiB | 2.19 MiB/s, done.
> remote: Total 3668680 (delta 326), reused 0 (delta 0), pack-reused 3667973
> Resolving deltas: 100% (3144716/3144716), done.
> Checking out files: 100% (141015/141015), done.
> grknight@akame ~ $ du -sh gentoo/.git
> 1.4G    gentoo/.git


Infra..
> grknight@akame ~ $ git clone --depth=1 https://anongit.gentoo.org/git/repo/sync/gentoo.git
> Cloning into 'gentoo'...
> remote: Counting objects: 155630, done.
> remote: Compressing objects: 100% (138925/138925), done.
> remote: Total 155630 (delta 44790), reused 68200 (delta 15465)
> Receiving objects: 100% (155630/155630), 75.28 MiB | 1.88 MiB/s, done.
> Resolving deltas: 100% (44790/44790), done.
> Checking out files: 100% (141015/141015), done.
> grknight@akame ~ $ du -sh gentoo/.git
> 96M     gentoo/.git
> grknight@akame ~ $ rm -fr gentoo
> grknight@akame ~ $ git clone https://anongit.gentoo.org/git/repo/sync/gentoo.git
> Cloning into 'gentoo'...
> remote: Counting objects: 3259105, done.
> remote: Compressing objects: 100% (472991/472991), done.
> remote: Total 3259105 (delta 2818449), reused 3193051 (delta 2758678)
> Receiving objects: 100% (3259105/3259105), 556.04 MiB | 1.02 MiB/s, done.
> Resolving deltas: 100% (2818449/2818449), done.
> Checking out files: 100% (141015/141015), done.
> grknight@akame ~ $ du -sh gentoo/.git
> 659M    gentoo/.git