| 1 |
Hi all |
| 2 |
|
| 3 |
Quick summary: Improve the performance of the Gentoo Linux mirrors by |
| 4 |
reducing the overall bandwidth load, allowing people to fetch binary |
| 5 |
patches from the mirrors, instead of the full source tarballs, when |
| 6 |
updating some package. This project is partially based on GLEP 25. |
| 7 |
|
| 8 |
The project is a bit ahead of schedule. |
| 9 |
|
| 10 |
== Progress == |
| 11 |
|
| 12 |
- Finished the integration of DeltaDB with existing code. A sample |
| 13 |
DeltaDB for sys-kernel/gentoo-source is available [1] |
| 14 |
- Finished the implementation of the delta reconstructor script. The |
| 15 |
script (distpatcher.py) gets a CPV, creates a list of all the |
| 16 |
distfiles needed by that version, check which ones are already |
| 17 |
available in $DISTDIR, walk the delta db in the reverse order, and |
| 18 |
reconstruct the needed distfiles, if possible. It also handle the |
| 19 |
checksums and is able to say if the checksum of the reconstructed |
| 20 |
distfile compressed is the same of the original distfile compressed, |
| 21 |
saving it in a diferent place, if needed, to be handled separately by |
| 22 |
Portage (to be implemented in a next task). |
| 23 |
- Generated a simple stats class. distdiffer.py produced 45 deltas for |
| 24 |
sys-kernel/gentoo-sources, then I created a plot [2] with the |
| 25 |
percentage of savings for each delta, if people download the delta |
| 26 |
instead of the full destination distfile. The deltas are sorted by |
| 27 |
percentage of savings, to make the visualization easy. This is just a |
| 28 |
visualization bonus, there's no really useful info about which |
| 29 |
distfiles are saving more download size. You need to check the DeltaDB |
| 30 |
file [1] if want such info. |
| 31 |
|
| 32 |
== Next steps == |
| 33 |
|
| 34 |
- Create the script to generate deltas for the full gentoo-x86 tree. |
| 35 |
- Improve error handling for existing scripts. |
| 36 |
- Rewrite some bits of existing code. |
| 37 |
- Try a full tree run, if possible. |
| 38 |
|
| 39 |
The project homepage now lives on the Gentoo infrastructure project |
| 40 |
page: http://www.gentoo.org/proj/en/infrastructure/distpatch/ |
| 41 |
|
| 42 |
[1] http://paste.pocoo.org/raw/404807/ |
| 43 |
[2] http://i.imgur.com/nKqpM.png |
| 44 |
|
| 45 |
That's all for now. |
| 46 |
|
| 47 |
Thanks! |
| 48 |
|
| 49 |
-- |
| 50 |
Rafael Goncalves Martins |
| 51 |
http://rafaelmartins.eng.br/ |
Archives
