1 |
Hi all |
2 |
|
3 |
Quick summary: Improve the performance of the Gentoo Linux mirrors by |
4 |
reducing the overall bandwidth load, allowing people to fetch binary |
5 |
patches from the mirrors, instead of the full source tarballs, when |
6 |
updating some package. This project is partially based on GLEP 25. |
7 |
|
8 |
The project is a bit ahead of schedule. |
9 |
|
10 |
== Progress == |
11 |
|
12 |
- Finished the integration of DeltaDB with existing code. A sample |
13 |
DeltaDB for sys-kernel/gentoo-source is available [1] |
14 |
- Finished the implementation of the delta reconstructor script. The |
15 |
script (distpatcher.py) gets a CPV, creates a list of all the |
16 |
distfiles needed by that version, check which ones are already |
17 |
available in $DISTDIR, walk the delta db in the reverse order, and |
18 |
reconstruct the needed distfiles, if possible. It also handle the |
19 |
checksums and is able to say if the checksum of the reconstructed |
20 |
distfile compressed is the same of the original distfile compressed, |
21 |
saving it in a diferent place, if needed, to be handled separately by |
22 |
Portage (to be implemented in a next task). |
23 |
- Generated a simple stats class. distdiffer.py produced 45 deltas for |
24 |
sys-kernel/gentoo-sources, then I created a plot [2] with the |
25 |
percentage of savings for each delta, if people download the delta |
26 |
instead of the full destination distfile. The deltas are sorted by |
27 |
percentage of savings, to make the visualization easy. This is just a |
28 |
visualization bonus, there's no really useful info about which |
29 |
distfiles are saving more download size. You need to check the DeltaDB |
30 |
file [1] if want such info. |
31 |
|
32 |
== Next steps == |
33 |
|
34 |
- Create the script to generate deltas for the full gentoo-x86 tree. |
35 |
- Improve error handling for existing scripts. |
36 |
- Rewrite some bits of existing code. |
37 |
- Try a full tree run, if possible. |
38 |
|
39 |
The project homepage now lives on the Gentoo infrastructure project |
40 |
page: http://www.gentoo.org/proj/en/infrastructure/distpatch/ |
41 |
|
42 |
[1] http://paste.pocoo.org/raw/404807/ |
43 |
[2] http://i.imgur.com/nKqpM.png |
44 |
|
45 |
That's all for now. |
46 |
|
47 |
Thanks! |
48 |
|
49 |
-- |
50 |
Rafael Goncalves Martins |
51 |
http://rafaelmartins.eng.br/ |