1 |
- ======================================================== - |
2 |
- Project IDFetch - Weekly report #3 ("Strawberry issues") - |
3 |
- ======================================================== - |
4 |
|
5 |
Switching development environment is usually not an easy thing, and it |
6 |
was not so easy this time when i decided to move to village. Thought |
7 |
you get clear air, lungs full of oxygen, and mouthful of strawberries [1] |
8 |
there're some drawbacks - Internet connection choices are limited to |
9 |
GPRS. Without going into details of my experiments on attaching my cell |
10 |
phone to the 3rd store of the 2-stored house let's just say that taking |
11 |
things for granted when you have them is simple, but fighting for |
12 |
things you used to have can be quite a deal. In conclusion results are: |
13 |
|
14 |
I) No 3G yet, but DTR is up to 15 Kb/s (not so bad); |
15 |
|
16 |
II) 2 broken battery-chargers for my cellphone (one got broken connector |
17 |
when the phone was falling from the 3rd floor, i still can't |
18 |
understand how i managed to brake the 2nd one); |
19 |
|
20 |
III) My cat "MOOrcello Mastroianni" AKA "Moorchik" gone missing ;-(((( |
21 |
|
22 |
IV) Few more results on idfetch project itself (read further). |
23 |
|
24 |
|
25 |
|
26 |
IDFetch project progress |
27 |
======================== |
28 |
|
29 |
1. [ Exporting duplicate distfiles ] |
30 |
|
31 |
Sometimes exporting pkg.list by #emerge -f pkg command produces |
32 |
duplicate distfiles. Some fixes for fetch.py have been added to |
33 |
control/skip duplicate distfiles. |
34 |
|
35 |
|
36 |
2. [ Skipping downloaded distfiles ] |
37 |
|
38 |
Skipping distfiles is fun, so skipping for already downloaded distfiles |
39 |
has been added to the segget. |
40 |
|
41 |
|
42 |
3. [ Skipping downloaded segments ] |
43 |
|
44 |
More skipping. If settings.resume_on option is set then fully downloaded |
45 |
segments are also skipped. |
46 |
|
47 |
|
48 |
4. [ Interface improvements ] |
49 |
Introduction of Ncurses [2] into segget for more intuitive view on the |
50 |
downloading processes [3]. |
51 |
|
52 |
|
53 |
5. [ Gethering stats ] |
54 |
|
55 |
5.1. Runtime visualization of progress for each connection has been |
56 |
implemented [3]. |
57 |
|
58 |
5.2. Each connection provided with speed measurement [3]. |
59 |
For now it's only current speed, though it's actually avg over 1 second. |
60 |
TO-DO: avg speed measurement for the whole connection time period. |
61 |
|
62 |
5.3. Total speed is also measured [3]. |
63 |
TO-DO: Avg speed measurement from the moment segget started. |
64 |
Note: Later in daemon mode - times when daemon is idle won't be counted. |
65 |
|
66 |
5.4. In case of unsuccessful segment fetch, segget increases |
67 |
segment->try_num and restarts fetching. |
68 |
|
69 |
|
70 |
6. [ Combining segments into distfiles ] |
71 |
There're few improvements/fixes to the process of combining segments |
72 |
into distfiles. Segments get deleted after being joined. |
73 |
|
74 |
|
75 |
7. [ RMD160, SHA1, SHA256, SHA512, WHIRLPOOL, MD5, CRC32 ] |
76 |
|
77 |
7.1. libcrypto++[4] employed to check RMD160, SHA1 and SHA256 hashes of |
78 |
downloaded distfiles. Errors are logged. |
79 |
|
80 |
7.2. Support for SHA512, WHIRLPOOL, MD5, CRC32 have also been added |
81 |
after remark from Robin H.Johnson. |
82 |
|
83 |
7.3. Later libcrypto++ might be replaced by one of: libmhash, NSS, |
84 |
openssl. |
85 |
|
86 |
|
87 |
8. [ Logs ] |
88 |
Ncurses is a real jam but there's no life without logs: segget.log, |
89 |
error.log and debug.log have been added. |
90 |
|
91 |
|
92 |
9. [ Strawberries in virtual boxes ] |
93 |
Cost of going for strawberries is more expensive and slow Internet |
94 |
connection. Hence, emulating mirrors became quite useful: |
95 |
- no need to consume Internet traffic; |
96 |
- no cable in my laptop (no wifi limitations either); |
97 |
- more control on experimental environment (changing speed, connection |
98 |
limits on the server side). |
99 |
For this purpose VirtualBox with Host-only Ethernet adapters [5] is used |
100 |
and some modifications to pkg.list file has been made (see json fragment |
101 |
with pkg dict): |
102 |
|
103 |
{ |
104 |
"CATEGORY": "dev-libs", |
105 |
"distfile_list": [ |
106 |
{ |
107 |
"RMD160": "82e5061ec76f23643ba5477ab28bbee6eebd393a", |
108 |
"SHA1": "b836783ebd72d5bc6a916620ab2b1ecec316fef1", |
109 |
"SHA256":"b522f0b5f850b50e9917823ea986f855295407380fafbe30f358875c41998bc5", |
110 |
"name": "cryptopp560.zip", |
111 |
"size": 1049029, |
112 |
"url_list": [ |
113 |
"ftp://192.168.6.11/cryptopp560.zip", |
114 |
"ftp://192.168.6.12/cryptopp560.zip", |
115 |
"ftp://192.168.6.13/cryptopp560.zip", |
116 |
"ftp://192.168.6.14/cryptopp560.zip" |
117 |
] |
118 |
} |
119 |
], |
120 |
"pkg_name": "crypto++-5.6.0-r1" |
121 |
} |
122 |
|
123 |
[1] http://soc.dev.gentoo.org/~simka/wr3/strawberry.jpg |
124 |
[2] http://en.wikipedia.org/wiki/Ncurses |
125 |
[3] http://soc.dev.gentoo.org/~simka/wr3/ui.jpg |
126 |
[4] http://www.cryptopp.com/ |
127 |
[5] http://www.virtualbox.org/manual/ch06.html#network_hostonly |
128 |
|
129 |
Best regards, |
130 |
Kostyantyn Ovechko AKA simka |