1 |
Zac Medico posted on Mon, 14 Nov 2011 07:22:19 -0800 as excerpted: |
2 |
|
3 |
> On 11/14/2011 12:47 AM, Dirkjan Ochtman wrote: |
4 |
>> On Mon, Nov 14, 2011 at 02:59, Zac Medico <zmedico@g.o> wrote: |
5 |
>>> Well, it's much easier to gather interest and get feedback if we |
6 |
>>> deploy the change and ask questions later. |
7 |
>> |
8 |
>> I like the new output, but find it kind of annoying that there's very |
9 |
>> little feedback on how far the progress is within a single job. Perhaps |
10 |
>> we could show the currently executing ebuild phase in order to give a |
11 |
>> little more feedback? Maybe most packages are completely dominated by |
12 |
>> src_compile(), but for smaller packages it would be helpful, IMO. |
13 |
> |
14 |
> I think that would be an interesting option. I imagine that it would be |
15 |
> too much information for many people, so I don't think that we'd want to |
16 |
> enable it by default. |
17 |
|
18 |
Thanks for thinking "out of the box". =:^) |
19 |
|
20 |
This is one thing that I've found frustrating with parallel emerging, |
21 |
myself. |
22 |
|
23 |
The first thing I discovered were that some phases (at least the merge |
24 |
phase, and I believe install) aren't counted, so the numbers don't |
25 |
entirely add up (complete + running + not-started + failed <> total). |
26 |
Thus, some packages appear to be simply sitting there doing nothing. |
27 |
When a whole bunch of packages are in that state, it appears emerge is |
28 |
doing nothing but consuming cycles and real-time, for no visible reason |
29 |
at all. |
30 |
|
31 |
Since for a number of packages (I seem to notice it most on kde packages, |
32 |
but perhaps it's all C++ or most CMAKE or some such, plus of course |
33 |
primarily data packages that don't have a significant build phase but |
34 |
install many or large files) the install phase can be as intense as the |
35 |
build phase if not more so, so if some phases aren't counted in |
36 |
"running", then I'd definitely prefer they be counted /somewhere/. |
37 |
|
38 |
Of course, listing the number of packages in each phase (which in single- |
39 |
mode would by definition list the phase the single package is in) would |
40 |
cure the above issue at the same time. |
41 |
|
42 |
Another problem, but one I'm not sure how to fix (tho the interactive |
43 |
display would offer opportunities here), is that it'd be nice to get a |
44 |
list of completed packages, in-process packages, and still not started |
45 |
packages. That doesn't so easily fit in a neat summary, but as noted, |
46 |
the interactive proposal introduces quite some opportunity, here. |
47 |
|
48 |
Alternatively, since often the real info desired is the status of a |
49 |
particular package, it'd be useful to have either an interactive command |
50 |
or a simple separate querying command, that can be run to query the |
51 |
status of a particular package. |
52 |
|
53 |
*** Which brings up an alternative to the whole interactive emerges idea |
54 |
as well -- what about a separate command, say "emerging", that when run, |
55 |
would simply output a bunch of detail on any pending emerges? |
56 |
|
57 |
This would certainly offer a safer initial implementation, as well, since |
58 |
it's less likely to break the current setup due to bugs, etc. Another |
59 |
noted benefit is that it leaves the existing defaults alone, so it's not |
60 |
going to be threatening anyone's sacred cows (or scripted assumptions) in |
61 |
terms of how portage has always behaved. =:^) |
62 |
|
63 |
|
64 |
So, perhaps improve the current "quiet" display marginally, either |
65 |
including all phases in "running" or adding another listing so the |
66 |
numbers add up, but more importantly... |
67 |
|
68 |
*** Add a new "emerging" (name up for bikeshedding) command, that |
69 |
displays a nice multiline summary, perhaps something like this (the |
70 |
second emerge command was for a failure in the first, with a second |
71 |
attempt made before completion of the main emerge @world has completed): |
72 |
|
73 |
Current emerge status: |
74 |
|
75 |
2 emerge command(s) running. |
76 |
7 emerge jobs outstanding. |
77 |
|
78 |
Commands: |
79 |
|
80 |
** emerge --upgrade --deep --newuse --keep-going @world |
81 |
|
82 |
Status: |
83 |
|
84 |
123 of 257 jobs complete, 1 failed, 6 running, 120 not yet started, |
85 |
7 dependencies removed due to failures. |
86 |
|
87 |
Running (6): |
88 |
|
89 |
1 pkgsetup (cat-egory/package-ver) |
90 |
|
91 |
1 configure (cat-egory/package-ver, cat-egory/package-ver) |
92 |
|
93 |
2 build (cat-egory/package-ver, cat-egory/package-ver) |
94 |
|
95 |
1 install (cat-egory/package-ver) |
96 |
|
97 |
1 merge (cat-egory/package-ver) |
98 |
|
99 |
Failed (1): |
100 |
|
101 |
dev-libs/boost-1.47.0-r1 |
102 |
|
103 |
Complete (123): |
104 |
|
105 |
<list in nice columns> |
106 |
... |
107 |
|
108 |
Not started (120): |
109 |
|
110 |
<list in nice columns> |
111 |
... |
112 |
|
113 |
Removed dependencies due to failure (7): |
114 |
|
115 |
<list in nice columns> |
116 |
... |
117 |
|
118 |
|
119 |
** emerge -u1 boost |
120 |
|
121 |
Status: |
122 |
|
123 |
0 of 1 jobs complete, 1 running. |
124 |
|
125 |
Running (1): |
126 |
|
127 |
1 build (dev-libs/boost-1.47.0-r1) |
128 |
|
129 |
Obviously the first implementation might not have all the information |
130 |
above, and there might have been some nice information I missed and other |
131 |
information that could be displayed better, but it's a very cool idea |
132 |
even if I /do/ say so myself. =:^) |
133 |
|
134 |
Note that zeroed-out listings aren't displayed, so the the --oneshot |
135 |
doesn't list failed, not yet started, etc, and inactive phases are also |
136 |
not displayed. |
137 |
|
138 |
Also note how easy this sort of detailed display would make it to retry |
139 |
failed jobs and the resulting removed dependencies, once the failure has |
140 |
been resolved. |
141 |
|
142 |
"emerging" could have a "repeat" mode as well as one-shot, with a polling- |
143 |
delay parameter set to something sane like 30 or 60 seconds by default. |
144 |
That way, I wouldn't have to feed it to "watch" =:^) |
145 |
|
146 |
Because this is a separate command, worries about TMI/TLI should be |
147 |
eliminated. Additionally, it should go some way toward easing the whole |
148 |
quiet/noisy debate as well, since there's now another command available |
149 |
with an intermediate level of information. |
150 |
|
151 |
Something like this: |
152 |
|
153 |
edisplaylog |
154 |
If we then throw in one more tool, I'll call it edisplaylog for lack of a |
155 |
better name, that reads make.conf to find the elog dir, and can |
156 |
automatically pick out the latest log for a package, thus eliminating the |
157 |
trouble of doing it manually, that should help eliminate another |
158 |
objection to quiet-by-default. So... |
159 |
|
160 |
edisplaylog boost |
161 |
|
162 |
... would find the last elog for boost and display it. |
163 |
|
164 |
To make things easy for the user, simply ... |
165 |
|
166 |
edisplaylog |
167 |
|
168 |
... could be made to filetime search and display the last active log in |
169 |
tail -f mode. If after say a couple seconds of no activity, it re-polled |
170 |
the log dir and switched to displaying a different log (have it list the |
171 |
log it's displaying at the top) if it was newer, that could pretty much |
172 |
replace "noisy" mode entirely. =:^) |
173 |
|
174 |
Obviously running simply "edisplaylog" by itself during a parallel-jobs |
175 |
emerge would be nearly as confusing as portage not automatically going |
176 |
into quiet mode for parallel emerging would be, tho not quite since it |
177 |
would display the logfile name at the top, but "edisplaylog <pkg>" would |
178 |
still be very useful, ESPECIALLY for those using --keep-going, so they |
179 |
could track down and fix whatever failed a build before the parallel |
180 |
emerge spit out the results at the end. |
181 |
|
182 |
Talking about which... an "edisplaylog failed" mode, which would |
183 |
automatically find the last failure and displayed the log for it, could |
184 |
be very helpful as well. =:^) |
185 |
|
186 |
Obviously all these ideas entail a lot of coding, and I realize it's easy |
187 |
to sit here and make requests without doing the coding, but implementing |
188 |
them as separate commands first and incremental implementation should |
189 |
help a lot, and I hadn't seen anything like this proposed yet, so... |
190 |
|
191 |
I'll close with a thanks to Zac and everyone else who has already made |
192 |
portage the great tool it is today, because without them, we'd obviously |
193 |
not be having this discussion in the first place. =:^) |
194 |
|
195 |
-- |
196 |
Duncan - List replies preferred. No HTML msgs. |
197 |
"Every nonfree program has a lord, a master -- |
198 |
and if you use the program, he is your master." Richard Stallman |