| 1 |
Elaborating on the previous subject an idea come in mind, |
| 2 |
avoid the $PORTDIR/metadata/cache sync, It's 80 MB, 20000 files |
| 3 |
|
| 4 |
So I've tryed the first python patch of my life, obviously it's also the |
| 5 |
first portage one (subliminal message, take the result with care and |
| 6 |
check it yourself) |
| 7 |
|
| 8 |
how? the patch add the option "--withregen", the to |
| 9 |
exclude /metadata/cache from the sync and to issue a regen of the cache |
| 10 |
before to update the metadata. |
| 11 |
calling "emerge --withregen --sync" do the work. |
| 12 |
|
| 13 |
The following test has run with a dual opteron as rsync server and a |
| 14 |
Athlon XP 2500+ with an old and slow disk, cabled with gigabit |
| 15 |
ethernet. The rsync is forced removing the timestamp. |
| 16 |
|
| 17 |
The portage tree has been synced _before_ starting the bench, so the |
| 18 |
overhead of rsync is really reduced at the bare minimum, check the |
| 19 |
differences between the files on server and client. |
| 20 |
|
| 21 |
The surprising result is that recreating the cache is faster than rsync |
| 22 |
it. (real time) |
| 23 |
|
| 24 |
===================================|=================================== |
| 25 |
===================================|=================================== |
| 26 |
1st try, portage [2.0.53_rc5] + cac|1st try, portage [2.0.53_rc5] + cac |
| 27 |
===================================|=================================== |
| 28 |
sync with cache and _no_ regen |sync without cache and regen after |
| 29 |
|Regenerating cache entries... |
| 30 |
|done regen! |
| 31 |
| |
| 32 |
real 6m23.727s |real 4m14.837s |
| 33 |
user 0m12.373s |user 0m18.681s |
| 34 |
sys 0m13.849s |sys 0m7.744s |
| 35 |
===================================|=================================== |
| 36 |
===================================|=================================== |
| 37 |
2nd try, portage [2.0.53_rc5] + cac|2nd try, portage [2.0.53_rc5] + cac |
| 38 |
===================================|=================================== |
| 39 |
sync with cache and _no_ regen |sync without cache and regen after |
| 40 |
|Regenerating cache entries... |
| 41 |
|done regen! |
| 42 |
| |
| 43 |
real 6m53.649s |real 2m40.794s |
| 44 |
user 0m12.361s |user 0m18.361s |
| 45 |
sys 0m14.117s |sys 0m6.800s |
| 46 |
===================================|=================================== |
| 47 |
===================================|=================================== |
| 48 |
3rd try, portage [2.0.53_rc5] + cac|3rd try, portage [2.0.53_rc5] + cac |
| 49 |
===================================|=================================== |
| 50 |
sync with cache and _no_ regen |sync without cache and regen after |
| 51 |
|Regenerating cache entries... |
| 52 |
|done regen! |
| 53 |
| |
| 54 |
real 6m46.973s |real 4m19.261s |
| 55 |
user 0m12.605s |user 0m18.593s |
| 56 |
sys 0m13.733s |sys 0m7.648s |
| 57 |
| |
| 58 |
|
| 59 |
To run the test yourself (please with a local rsync mirror) |
| 60 |
|
| 61 |
================ |
| 62 |
mkdir tmptest ; cdtmptest |
| 63 |
# download the patch here |
| 64 |
bunzip2 emerge.patch.bz2 |
| 65 |
cp /usr/bin/emerge . |
| 66 |
patch -i emerge.patch emerge |
| 67 |
grep -B1000 "###end test with" emerge.patch > test_emerge |
| 68 |
# modify test_emerge for your needs |
| 69 |
. test_emerge |
| 70 |
================== |
| 71 |
(/please with a local rsync mirror) |
| 72 |
|
| 73 |
Please check the correctness of what the patch do before test it. |
| 74 |
|
| 75 |
Cheers |
Archives