| 1 |
On Mon, Feb 08, 2010 at 05:23:03AM +0000, Robin H. Johnson wrote: |
| 2 |
> On Sun, Feb 07, 2010 at 05:02:22PM -0800, Brian Harring wrote: |
| 3 |
> > On Sun, Jan 31, 2010 at 10:04:40AM +0000, Robin H. Johnson wrote: |
| 4 |
> > > Changes: |
| 5 |
> > > - This GLEP can stand independently of GLEP58. |
| 6 |
> > > - Add XZ to compression types list. |
| 7 |
> > > - Move cutoff to 32KiB. Provide size example w/ 32KiB+gzip. |
| 8 |
> > > - Split specification into generation and validation. |
| 9 |
> > One concern w/ this glep- the intention seems to be to reduce on disk |
| 10 |
> > space requirements but the addition of compression raises questions |
| 11 |
> > for rsync transferance of the manifests. |
| 12 |
> > |
| 13 |
> > Have you done any testing to quantify how much of an increase in rsync |
| 14 |
> > bandwidth this will add? Specifically thinking about the metamanifest |
| 15 |
> > on this one. |
| 16 |
> |
| 17 |
> I think the best course of action is to end up generating the compressed |
| 18 |
> MetaManifests when we start generated the MetaManifests themselves, but |
| 19 |
> not placing them into the tree yet. Instead simply use them to measure |
| 20 |
> rsync transfer size impact on the generation server and produce |
| 21 |
> statistics to see if the cutoff could benefit from being altered, or if |
| 22 |
> the disk space should be wasted in favour of smaller transfer size. |
| 23 |
|
| 24 |
Works for me- I do strongly suspect that if we use compression we'll |
| 25 |
be purely trading disk space decrease for the cost of transfering each |
| 26 |
changed compressed manifest in full though... rsync + compression do |
| 27 |
not get along at all. |
| 28 |
|
| 29 |
Either way, stats would be useful when you've got time. |
| 30 |
~brian |
Archives