Re: [gentoo-project] RFC: Dropping rsync as a tree distribution method

[Zac Medico]

On 12/15/18 9:17 PM, Alec Warner wrote:
>
>
> On Sun, Dec 16, 2018 at 12:13 AM Georgy Yakovlev <gyakovlev@g.o
> <mailto:gyakovlev@g.o>> wrote:
>
>     On Saturday, December 15, 2018 8:40:38 PM PST Matt Turner wrote:
>     > On Sat, Dec 15, 2018 at 11:16 PM Alec Warner <antarus@g.o
>     <mailto:antarus@g.o>> wrote:
>     > > - Disk usage for git vs rsync
>     >
>     > This is why I have not switched. With git you pull down increasing
>     > amounts of history, whereas with rsync the data fits easily in a <1GB
>     > partition.
>
>     Recent portage can use sync-depth = 1
>     repo dir no longer grows as it used to and it's works fine unlike
>     initial
>     implementation that was giving trouble
>
>     https://bugs.gentoo.org/552814
>
>     du -hs /var/db/repos/gentoo
>     350M    /var/db/repos/gentoo
>
>     example /etc/portage/repos.conf/gentoo.conf :
>     [DEFAULT]
>     main-repo = gentoo
>
>     [gentoo]
>     auto-sync = yes
>     location = /var/db/repos/gentoo
>     sync-type = git
>     sync-uri = https://github.com/gentoo-mirror/gentoo.git
>     sync-depth = 1
>     sync-git-clone-extra-opts = -b master
>     sync-git-verify-commit-signature = true
>
>
>     sync is almost instantaneous compared to rsync, but some folks not
>     going to
>     like github as a mirror in this case.
>
>
> I don't plan on using github for the mirror, so I'm not overly worried
> about that portion.
>
> +Zac Medico <mailto:zmedico@g.o> 
>
> My recollection was that git doesn't ship with ebuild metadata by
> default, so even if we make the first sync fast (by using depth=1 in the
> clone) do we have a good story for ebuild metadata? Is portage just
> faster than in the past for ebuilds with missing metadata? Does emerge
> --sync handle metadata regen for syncs with git origins?

The metadata has to be included in the git repostory, and we've
currently got "master" and "stable" branches which include everything
that the rsync tree has:

https://gitweb.gentoo.org/repo/sync/gentoo.git/log/?h=master
https://gitweb.gentoo.org/repo/sync/gentoo.git/log/?h=stable

Both branches are also mirrored on github:

https://github.com/gentoo-mirror/gentoo/commits/master
https://github.com/gentoo-mirror/gentoo/commits/stable

It would be interesting to see some garbage collection stats for
sync-deph = 1, people using it should post the output of this command:

git count-objects -v

> -A
>  
>
>
>
>     --
>     Georgy Yakovlev
>     Gentoo Linux Developer
>


-- 
Thanks,
Zac