1 |
On Mon, 2004-12-04 at 12:17 -0500, Andrew Gaffney wrote: |
2 |
> Todd Berman wrote: |
3 |
> > On Mon, 2004-12-04 at 11:59 -0500, Andrew Gaffney wrote: |
4 |
> > |
5 |
> >>Todd Berman wrote: |
6 |
> >> |
7 |
> >>>On Mon, 2004-12-04 at 11:22 -0500, Andrew Gaffney wrote: |
8 |
> >>> |
9 |
> >>> |
10 |
> >>>>Troy Dack wrote: |
11 |
> >>>> |
12 |
> >>>> |
13 |
> >>>>>Another point against a monolithic zip containing all the ebuilds (or |
14 |
> >>>>>even per directory zips) is the performance hit that slow machines would |
15 |
> >>>>>take, not everybody runs gentoo on a 2GHz plus machine (eg: my little |
16 |
> >>>>>PII-400 in the corner) |
17 |
> >>>> |
18 |
> >>>>Or my little P233 Thinkpad... |
19 |
> >>>> |
20 |
> >>> |
21 |
> >>> |
22 |
> >>>And with the current setup of writing thousands of 1K files that little |
23 |
> >>>p233 thinkpad really flys i bet... |
24 |
> >> |
25 |
> >>I'm not sure if that was supposed to be sarcastic, but yes, it does fly. It only takes |
26 |
> >>slightly longer to sync than my Athlon 1.3GHz desktop. The only part that takes forever is |
27 |
> >>updating the portage cache. That's why I just use a NFS shared portage tree from my |
28 |
> >>desktop machine now. |
29 |
> > |
30 |
> > In a way it was and in a way it wasn't. I honestly don't understand how |
31 |
> > you can explain to me that a compression-less zip file be any slower |
32 |
> > than the current setup. |
33 |
> > |
34 |
> > With compression I could understand, but without I don't think any speed |
35 |
> > difference would be noticeable. However, even with compression the |
36 |
> > operation should be fairly fast. zip is not like a tar.gz or tar.bz2, |
37 |
> > ie, you can read a file out of it and search through it fairly fast, and |
38 |
> > you don't have to uncompress the entire archive to get a single file. |
39 |
> |
40 |
> Even without compression, there is still a little bit of overhead with having ebuilds and |
41 |
> such contained in ZIP files. The overhead comes into play both during sync and emerge. It |
42 |
> wouldn't be noticable on a fast machine (e.g. my Athlon 1.3GHz desktop) but it would make |
43 |
> an operation like 'emerge -uDpv world' take quite a bit longer on my Thinkpad. |
44 |
> |
45 |
|
46 |
This is going to be a slow operation on that thinkpad regardless. |
47 |
|
48 |
> As someone else pointed out, the ZIP files wouldn't make syncs faster either. Rsync |
49 |
> transfers only the differences in files and not the entire app-text directory or whatever |
50 |
> category you're working with. |
51 |
> |
52 |
|
53 |
Portage does not use this as it takes more cpu server and client side to |
54 |
figure out what changed with 80 thousand 1K files than it does to just |
55 |
transfer the files across. |
56 |
|
57 |
> As another person pointed out, it would be more difficult to modify ebuilds in the tree by |
58 |
> hand when they're contained in ZIP files. |
59 |
> |
60 |
|
61 |
This could easily be dealt with. the cvs->rsync process could create zip |
62 |
files, and portage would use the zip stuff by default for /usr/portage/ |
63 |
and use the old directory structure code for your overlays. |
64 |
|
65 |
> It was a good idea, but I don't think it is practical. |
66 |
|
67 |
It is absolutely practical, and I think would cut down on the time taken |
68 |
to sync (because now you could turn compression back on, and actually |
69 |
transfer just changes because you wouldnt be syncing 80 thousand files). |
70 |
It might potentially have small speed loses when running emerge -vuDp |
71 |
world, but as that process takes a long time regardless, I don't think |
72 |
it would be noticeable. The time it takes to read a single file out of |
73 |
an uncompressed zip is not noticeably longer than the time it takes to |
74 |
read a single file off the hard drive. |
75 |
|
76 |
Note, as of writing this the portage tree takes up 333MB of space on my |
77 |
hard drive and a compression-less zipfile of the tree takes up 74MB. To |
78 |
me, the potential savings are incredible. |
79 |
|
80 |
--Todd |
81 |
|
82 |
|
83 |
-- |
84 |
gentoo-dev@g.o mailing list |