Gentoo Archives: gentoo-user

From: Fernando Rodriguez <cyklonite@×××××.com>
To: gentoo-user@l.g.o
Subject: Re: [gentoo-user] Genlop oddity
Date: Sun, 31 Jul 2016 22:51:21
Message-Id: f6f00fa7-18d6-f33b-f487-2b483464ff1a@gmail.com
In Reply to: Re: [gentoo-user] Genlop oddity by Peter Humphrey
1 -----BEGIN PGP SIGNED MESSAGE-----
2 Hash: SHA256
3
4 On 07/31/2016 12:14 PM, Peter Humphrey wrote:
5 > On Sunday 31 Jul 2016 10:48:33 Fernando Rodriguez wrote:
6 >> On 07/31/2016 06:47 AM, Peter Humphrey wrote:
7 >>> I've just encountered something I can't explain. Can anyone here?
8 >>>
9 >>> ~ $ genlop -c -f /mnt/rescue/var/log/emerge.log
10 >>> using logfile /mnt/rescue/var/log/emerge.log
11 >>> Currently merging 281 out of 287
12 >>> * net-libs/gnutls-3.3.24
13 >>> current merge time: 1 minute and 44 seconds.
14 >>> ETA: less than a minute.
15 >>> Currently merging 264 out of 287
16 >>>
17 >>> * sys-devel/gcc-4.9.3
18 >>> current merge time: 7 minutes and 12 seconds.
19 >>> ETA: 3 minutes and 13 seconds.
20 >>>
21 >>> $ genlop -c -f /mnt/rescue/var/log/emerge.log
22 >>> using logfile /mnt/rescue/var/log/emerge.log
23 >>> Currently merging 264 out of 287
24 >>> * sys-devel/gcc-4.9.3
25 >>> current merge time: 15 minutes and 19 seconds.
26 >>> ETA: 3 minutes and 41 seconds.
27 >>>
28 >>> How is it possible for genlop's reported ETA to increase while its time
29 >>> spent so far also increases?
30 >
31 > ...or to change at all, for that matter.
32 >
33 >> It is an estimate (a prediction) so it's subject to change.
34 >
35 > Yes, but the estimate is derived from an average of the durations found in
36 > the log file, so it can't change until the emerge is complete.
37
38 I haven't looked at the code but if it changes I think it's doing more than
39 just averaging previous builds. It can and should try better than that. Even
40 if all it has to work with is the logs a simple algorithm could (once it starts
41 taking longer) look at the build time differences and come up with a better guess.
42 My guess it's doing something along those lines. After a bit it just says "anytimme
43 now."
44
45 >>> Could the concurrent gnutls merging have affected it? Surely not.
46 >>
47 >> Probably, I don't know if genlop takes that into account. But there's
48 >> countless factors that affect build time that it's practically impossible
49 >> to predict accurately. So it means nothing.
50 >
51 > I think that's too pessimistic. If all your emerges have -j1, the accuracy
52 > is pretty good, or it used to be in the days when I did that. The major
53 > factor affecting the durations nowadays is -j > 1, so any other packages
54 > could be emerging at the same time, thus skewing the log record.
55 >
56 > Another, even bigger log skewer is my practice, if I need an emerge -e, of
57 > doing it in two stages: -eB first, then reboot to a minimal system and -eK.
58 > That avoids things like kdelibs being different when I come to reboot the
59 > system next time and hanging up during shutdown.
60 >
61
62 There's the -j option, there's distcc, ccache, etc. Many system settings,
63 the amount of junk in /tmp if it's tmpfs, etc etc.
64
65 Even in the simplest of cases using -j1 and controlling everything else some
66 versions of the same package take significantly longer than others, what you
67 have installed on your system can affect compile times, some compiler versions
68 are faster, etc. etc.
69
70 For me they're totally useless. gcc compile time ranges from 24mins-13hrs.
71 Firefox 24mins-1day 3hrs. LibreOfffice 5hrs-14hrs.
72
73 >> Does that estimate even look reasonable to you?
74 >> gnutls compiles in a few minutes but gcc takes significantly longer for
75 >> me.
76
77 > No, as I said, genlop's own estimates are poor nowadays, but I can make a
78 > rough estimate myself from comparing "genlop -c" with "genlop -t <package> |
79 > grep minutes".
80
81 Just out of curiosity what are the differences between the original genlop
82 calculation and yours, and how long did it actually take? and what is the
83 output of 'genlop -t <cat/pkg>'. I think it would be inaccurate in most cases
84 (at least it is for me) but if you think the calculation is wrong you should
85 file a bug. For me it seems reasonably accurate por packages with consistent
86 build times.
87
88
89
90 - --
91
92 Fernando Rodriguez
93 -----BEGIN PGP SIGNATURE-----
94 Version: GnuPG v2
95
96 iQIcBAEBCAAGBQJXnoEnAAoJEPbOFX/5Ulwc3c4P/i26UPhwodsQ2TMS28fUyv4q
97 BAyMvxGZG/nACbmzAYwPPbT8cZwIQomXhfuozokWGFsRzWLFdNms73HQ0YoQokeV
98 Q8fHBolQdT3UyNgoROcDn7MJopr0QXWcZ2sWSpLFm2jZVmzz01d4ZkrQ2rXD22q/
99 6qK3293svTfHpsft0h9ADeFW5r8BJ8BuLT+Q1TNHFQETILUHxmB34R7cHyJ6eMYX
100 B0vGUXVSb1yYPReNKM3g+ok0KHMh5Q8iUGW44pzXLEg2Iw1Rnk2nwb7SYj0FjyzX
101 rFreLC5iiVVPRLUkI9NKn6k3Mg3y3TIn8fNqI44r8RMMt8o96sXKGsUIsWamXBzp
102 Sgorb+hvzTmiffD9Lk+1/reA5ne9nWfXGYlNfJY+JKgF0peGSOiWQ90w/T1x97si
103 tYs1MqqMnvna+vkDW+lZZcditRE1NnLpmSIkHtvBuPi0XcjeiRNlmUEI6s6k9ZPO
104 vTn0obIyT9f6XvUDNWZw5XlAZaP+w0Tf9r2mV6gJrx11OpjiKlEvWSN/F+rvRlsh
105 2Ki+QdrCjHpfnPbyQirUzfMeY2Ch4qzo8Ucnz/mJntE/4D9jPcmXisHV9k6KpYuX
106 phE6m+DmU48QRys1pbh9prQTtvIPDm9LkuG2Hc5XJq+q+nAGeMUIH6SskCtE1Lsy
107 5UFWcFsuzZLAm7ShTAuZ
108 =TiEU
109 -----END PGP SIGNATURE-----

Replies

Subject Author
Re: [gentoo-user] Genlop oddity Peter Humphrey <peter@××××××××××××.uk>