| 1 |
On Wed, Jan 13, 2010 at 9:24 PM, Mike Frysinger <vapier@g.o> wrote: |
| 2 |
> i think our current work flows also significantly impede the smooth running of |
| 3 |
> this. if we had were using a dscm (git) on gentoo-x86, i feel like it'd be a |
| 4 |
> much smoother ride for Gentoo devs to pull from a proxy maintainer and push on |
| 5 |
> their behalf. |
| 6 |
> |
| 7 |
|
| 8 |
In theory, yes. In practice, git is too slow to handle 30,000 files. |
| 9 |
Even simple operations like git add become painful even if you put the |
| 10 |
whole of portage on tmpfs since git does a stat() on every single file |
| 11 |
in the repository with every operation. |
| 12 |
|
| 13 |
Simple test: do a git init followed by git add && git commit -m |
| 14 |
"Initial commit" in your portage dir (.gitignore packages/ and |
| 15 |
distfiles/) |
| 16 |
|
| 17 |
Once this is done, you can test how it'll feel like to use a DSCM on |
| 18 |
portage (without history). Unless you have a really fast SSD and |
| 19 |
processor, you'll want to go back to the good old days of CVS with its |
| 20 |
network-bound latencies on just 5-6 files in the current dir. |
| 21 |
|
| 22 |
Besides this, there is the problem of accommodating people who use a |
| 23 |
subtree of gentoo-x86, and those who don't want the entire CVS history |
| 24 |
on their hard drives. In summation, robbat2 needs *our* help in the |
| 25 |
following: |
| 26 |
|
| 27 |
a) Push functionality in shallow clones (patches exist upstream) |
| 28 |
b) Partial-tree checkouts (patches exist upstream) |
| 29 |
c) Optimize git so it can handle 30,000 files |
| 30 |
- Maybe maintain a cache of directory timestamps and only stat() |
| 31 |
directories? |
| 32 |
- Implement recursive timestamps on directories in various |
| 33 |
filesystems and then in git (via xattrs perhaps)? People want to do |
| 34 |
this for things like Tracker too. Prelim patches might exist. |
| 35 |
d) Implement scripts/infra for people to fetch repository (shallow and |
| 36 |
deep) bundles to initialize their local git clones (similar to portage |
| 37 |
snapshots) |
| 38 |
- git clone from scratch taxes the server too much, just like |
| 39 |
rsync from scratch |
| 40 |
e) Server-side scripts for pushing to CIA.vc for pretty stats like we do in CVS |
| 41 |
- We want this for overlays right now too. |
| 42 |
f) (Optional) Fix http cloning in git to make it "smarter" to help |
| 43 |
people behind firewalls get anonymous clones (patches exist upstream) |
| 44 |
|
| 45 |
Did I miss something Robin? |
| 46 |
|
| 47 |
-- |
| 48 |
~Nirbheek Chauhan |
| 49 |
|
| 50 |
Gentoo GNOME+Mozilla Team |
Archives