1 |
Alex Efros posted on Sat, 20 Oct 2012 23:07:40 +0300 as excerpted: |
2 |
|
3 |
> Hi! |
4 |
> |
5 |
> On Sat, Oct 20, 2012 at 08:07:31PM +0200, Dominique Michel wrote: |
6 |
>> USE="-consolekit -policykit -udisks -upower" |
7 |
> |
8 |
> I'm using fluxbox, but, of course, I need ability to run gnome and kde |
9 |
> applications (but not gnome/kde itself). Last time I tried to switch off |
10 |
> these USE-flags it doesn't work. Best I got is: |
11 |
> |
12 |
> /etc/make.conf: |
13 |
> USE="${USE} -policykit -upower" |
14 |
> /etc/portage/package.use |
15 |
> # required by udisks |
16 |
> sys-fs/udev extras gudev hwdb |
17 |
> # required by polkit, consolekit |
18 |
> sys-auth/pambase consolekit |
19 |
> # required by polkit, udisks, clementine |
20 |
> sys-auth/consolekit policykit |
21 |
|
22 |
FWIW, as my gentoo experience lengthens, I've gotten *MUCH* tighter with |
23 |
my installed-packages and USE flags policy. |
24 |
|
25 |
I run a kde desktop, but have this in my global USE file[1]: |
26 |
|
27 |
-consolekit -policykit -udisks -udisks2 -upower |
28 |
|
29 |
Also given my kde4 desktop it's worth noting: |
30 |
|
31 |
-raptor -redland -semantic-desktop -virtuoso |
32 |
|
33 |
(Of course that means no kdepim packages at all. I got fed up with |
34 |
akonadi problems and switched to the gtk-based claws-mail after nearly a |
35 |
decade on kmail! That allowed me to kill kmail and akonadi, which |
36 |
allowed me to kill semantic-desktop for all of kde, which allowed me to |
37 |
kill rasqal, redland, virtuoso, soprano... I did it mainly to get rid of |
38 |
akonadi and the problems it brought, but WOW, the kde4 desktop was faster |
39 |
without all that extra bloatware! And I had already turned off nepomuk |
40 |
and strigi too, and it was STILL dramatically faster when kde was built |
41 |
without that junk! Sort of reminds me of all those MSWormOS users and |
42 |
how surprised they often are to learn how badly the malware was affecting |
43 |
system performance once it's cleaned up. Yes, I DID just compare the |
44 |
semantic-desktop crap to malware!) |
45 |
|
46 |
|
47 |
But: USE=udev |
48 |
|
49 |
No udisks (1 or 2), upower, consolekit or policykit installed. |
50 |
|
51 |
|
52 |
But, I actually prefer manual mounting of removables (significantly safer |
53 |
that way!), etc, and the fact that automount functionality requires |
54 |
udisks, which (in v1) ultimately pulls in lvm2, and I'm NOT interested in |
55 |
having that on my system[2], so while I tolerated it for awhile, |
56 |
ultimately I got rid of it. |
57 |
|
58 |
|
59 |
Meanwhile, the below is an admitted rather long tangent, but readers may |
60 |
find this output from emerge --depclean ($>> is the last line of my |
61 |
triple-line customized bash $PS1 prompt) rather interesting indeed, and |
62 |
it DOES continue the "tightened up system" policy theme I presented above: |
63 |
|
64 |
$>>emerge --depclean -p |
65 |
|
66 |
[snip the standard boilerplate] |
67 |
|
68 |
!!! You have no system list. |
69 |
!!! Proceeding is likely to break your installation. |
70 |
|
71 |
[more boilerplate] |
72 |
|
73 |
Packages installed: 865 |
74 |
Packages in world: 0 |
75 |
Packages in system: 0 |
76 |
Required packages: 865 |
77 |
Number to remove: 0 |
78 |
|
79 |
|
80 |
Empty system warning, empty world, yet 865 required packages? WTF? |
81 |
|
82 |
The empty @system set warning is accurate; the empty world not quite so, |
83 |
tho it's anything but standard. Here's the explanation: |
84 |
|
85 |
World first, as it's easier. =^) |
86 |
|
87 |
I'm running the (still masked) portage-2.2-alpha series for sets support, |
88 |
which (AFAIK) hasn't yet hit the 2.1 series. My world file is INDEED |
89 |
empty, but that's because I took advantage of sets support to break up |
90 |
the long and unsorted world list into a bunch of sets, each of which is |
91 |
listed in the world_sets file. (That's comparable to what I did with |
92 |
make.conf, as noted in footnote [1] from the mention of my global USE |
93 |
file.) |
94 |
|
95 |
So my /var/lib/portage/world_sets file lists sets like (jed are my |
96 |
initials, used here to indicate that they are my custom sets, distinct |
97 |
from for example the sets that the kde overlay ships, tho my custom kde |
98 |
sets are simply the sets from the overlay, with various packages |
99 |
commented out, I diff them against the overlay set twice a year when I |
100 |
upgrade to the next kde 4.x version minor, making adjustments as |
101 |
necessary then): |
102 |
|
103 |
@jed.admin |
104 |
@jed.kde.base.kdegames |
105 |
@jed.portage |
106 |
|
107 |
That's just picking three from the list of about two dozen. |
108 |
|
109 |
Here's the contents of my @jed.portage set as an example: |
110 |
|
111 |
$>>cat /etc/portage/sets/jed.portage |
112 |
app-portage/eclass-manpages |
113 |
app-portage/esearch |
114 |
app-portage/gentoolkit |
115 |
app-portage/layman |
116 |
app-portage/mirrorselect |
117 |
app-portage/portage-utils |
118 |
app-doc/pms |
119 |
|
120 |
|
121 |
So while my world file is empty, my world_sets file has about two dozen |
122 |
sets listed, each of which contains its own list of packages I want that |
123 |
sort into that category. Yes, my world file is empty, but the @world / |
124 |
set/ is not empty. But the depclean summary hasn't yet been updated for |
125 |
sets. I filed a feature-request-bug[3] on that some time ago, but |
126 |
obviously it's down Zac's priority list quite a way, until such time as |
127 |
they decide to introduce proper sets support to an unmasked portage. |
128 |
|
129 |
Worst-case, they drop the still experimental sets feature instead, and I |
130 |
have to recombine all the packages listed in my individual sets into one |
131 |
big world file again. No big deal tho sets support is nice and I'd miss |
132 |
it. At least I've been able to use sets in the mean time. |
133 |
|
134 |
|
135 |
OK, that explains the "empty" world, the world _file_ is empty, but not |
136 |
the world_sets file and thus not @world, but the depclean summary doesn't |
137 |
know about world_sets yet. |
138 |
|
139 |
What about that empty @system warning? |
140 |
|
141 |
The technical information's available in the portage (5) manpage, but it |
142 |
comes down to this: /etc/portage/profile/* files can be used to override |
143 |
the normal tree profile where a gentoo users finds it necessary to do |
144 |
so. Basically it gets applied on top of the normal cascading profile in |
145 |
the tree, adding or subtracting from it just as the contents of the |
146 |
make.profile dir override the contents of its cascaded parents. |
147 |
|
148 |
More specifically, the packages file contains entries like this (with or |
149 |
without the negation, but with the *): |
150 |
|
151 |
-*sys-devel/make |
152 |
-*>=sys-devel/patch-2.6.1 |
153 |
|
154 |
Each (un-negated) starred package atom found in the packages file in one |
155 |
of the cascaded profile dirs adds that package to the @system set. |
156 |
Negating the entry removes a package from @system that was added by one |
157 |
of the cascading parents. |
158 |
|
159 |
Now here's the deal. There are two purposes that the @system set |
160 |
fulfills. First, the list of packages in @system is used by catalyst to |
161 |
create the stage tarballs used to bootstrap a gentoo system at initial |
162 |
installation. Basically, it's the list of packages found in a stage3. |
163 |
|
164 |
Second, once a gentoo system is installed and running, the list of |
165 |
packages in @system form a core set of basic dependencies that can be |
166 |
assumed to be installed, so they can be omitted from an ebuild's |
167 |
dependencies list, dramatically shortening the dependencies list and |
168 |
decreasing the maintenance burden both for individual package maintainers |
169 |
since they have a core set of packages that can be assumed to be |
170 |
installed, and for core system and arch maintainers, since that core list |
171 |
is nicely centralized, thus much easier to change that it would be to |
172 |
edit thousands of individual ebuild dependencies. |
173 |
|
174 |
For the gentoo user, among other things, if you mistakenly try to unmerge |
175 |
something in the system set, portage gives you a much stronger warning |
176 |
than it would otherwise, since it's very possible doing so will break the |
177 |
system beyond easy recoverability, forcing a boot to a rescue disk or |
178 |
backup in ordered to fix the problem. |
179 |
|
180 |
But there is a very practical non-zero cost to having a package in |
181 |
@system. Portage is more careful with these packages and tends to merge |
182 |
them one at a time, thus breaking emerge's parallel merge ability and the |
183 |
--jobs and --load-average options that control it, when merging an |
184 |
@system package or a dependency of one, forcing portage back to the bad |
185 |
old days of serialized one-package-at-a-time merging. For people with |
186 |
even a quad-core system and enough memory to nicely handle multiple |
187 |
merges who have accordingly enabled parallel merging using the --jobs and |
188 |
--load-average options, then, any package that's in @system or a |
189 |
dependency of something in @system, dramatically slows down system |
190 |
updates, and even more so, the emerge --empty-tree @world that many |
191 |
people like to do after a major gcc upgrade or change to CFLAGS/CXXFLAGS/ |
192 |
LDFLAGS. Keeping @system as small as possible directly affects the speed |
193 |
at which updates and particularly --empty-tree rebuilds complete! |
194 |
|
195 |
So the @system package list has two purposes, only one of which actually |
196 |
matters once you're beyond the stage-3 point of an install, while every |
197 |
package on that list and all its dependencies have a dramatically |
198 |
increased cost in terms of merge time on a modern multicore system, since |
199 |
those packages can't take advantage of portage's parallel emerges ability. |
200 |
|
201 |
Once a user has been on gentoo awhile and knows his way around the system |
202 |
well enough that he's unlikely to make the mistake of unmerging a package |
203 |
he well enough knows is critical, thus making that extra warning for such |
204 |
a mistake unnecessary, the cost of that @system list begins to look |
205 |
bigger and bigger in relation to the actual benefit it continues to |
206 |
provide. |
207 |
|
208 |
My first gentoo install was 2004.0... 8.5 years ago. If I'm sure enough |
209 |
about an emerge -C to do it in the first place, that extra @system |
210 |
warning isn't going to stop me, so I might as well not have to deal with |
211 |
its cost. |
212 |
|
213 |
But here, an emerge --pretend --emptytree @system wanted to remerge 300+ |
214 |
packages! I forgot how many were actual @system packages, but most of |
215 |
them were dependencies. That's a *LOT* of packages for portage to be |
216 |
forcing to one-at-a-time merging! |
217 |
|
218 |
So at first I tried whittling down the number of @system deps by tweaking |
219 |
USE flags. That did drop the total some, but not enough! And trying to |
220 |
sort out individual package.use exceptions was getting complex! |
221 |
|
222 |
Simple enough solution, be rid of the whole @system list! |
223 |
|
224 |
emerge --pretend @system provided a list of package in that set, without |
225 |
dependencies. First I did an emerge --pretend --depclean to ensure there |
226 |
was nothing it wants to remove with the existing system set that I needed |
227 |
to decide about. (There wasn't, I routinely run revdep-rebuild and |
228 |
emerge --depclean after I'm finished updating. Everything was in world, |
229 |
or in my case in the world_sets, that needed to be. No package cruft |
230 |
left hanging!) |
231 |
|
232 |
Then I took that emerge --pretend @system list and create /etc/portage/ |
233 |
profile/packages negating entries for each of them. =:^) |
234 |
|
235 |
Reran emerge --pretend @system. What was this? Two packages still in |
236 |
@system along with 200+ dependencies (--emptytree lists the deps too, |
237 |
without it, only the actual @system packages are listed)! WTF? |
238 |
|
239 |
While I tried to figure that out, I played around with USE flags some |
240 |
more and with just those two packages in @system, I was able to get the |
241 |
total @system with deps down to 120 or so. MAJOR PROGRESS, especially |
242 |
when I had started with 300+, and still had 200+ with only two packages |
243 |
in @system. BUT NOT GOOD ENOUGH! |
244 |
|
245 |
Meanwhile, while I tried to figure out what was going on there, I did |
246 |
another --depclean --pretend to see what packages in @system weren't |
247 |
still protected as dependencies of something in @world. Turns out most |
248 |
of them were already dependencies of something, so only a few packages |
249 |
came up to be depcleaned. |
250 |
|
251 |
And actually, several of those were virtuals (example, virtual/editor, |
252 |
default provider nano). For most of those, I already had the package |
253 |
that I wanted to fill that virtual in @world (in one or another of my |
254 |
custom @jed.* sets), so unmerging the virtual wasn't going to hurt |
255 |
anything because the package I wanted to fill that virtual was still |
256 |
installed and in @world. So I unmerged the unnecessary virtuals. |
257 |
|
258 |
For the handful of others, after figuring out which @jed.* set I wanted |
259 |
each one in, I added it there. After that a --depclean --pretend came up |
260 |
clean again and the installed package list was stable, tho @system along |
261 |
with its depends was still too big for comfort. |
262 |
|
263 |
I let it sit at that point for a few days, while I tried to figure out |
264 |
where the other two @system entries were coming from. At first I thought |
265 |
they must be hard-coded, which made some sense for the one, baselayout-2. |
266 |
But the other one was patch, and it just made no sense that /patch/, of |
267 |
ALL things, would be hardcoded, but things like gcc or sed and grep |
268 |
weren't. |
269 |
|
270 |
Turns out there was a simple explanation. Those two @system package |
271 |
entries weren't simply generic entries like this: |
272 |
|
273 |
*sys-apps/baselayout |
274 |
*sys-devel/patch |
275 |
|
276 |
They actually appeared with version specifiers like this: |
277 |
|
278 |
*>=sys-apps/baselayout-2 |
279 |
*>=sys-devel/patch-2.6.1 |
280 |
|
281 |
So the generic negation wasn't cutting it. I had to do this, negating the |
282 |
exact same entries that were added by the in-tree profile: |
283 |
|
284 |
-*>=sys-apps/baselayout-2 |
285 |
-*>=sys-devel/patch-2.6.1 |
286 |
|
287 |
Viola! THAT DID IT! TOTALLY ZEROED OUT @system LIST!! =:^) |
288 |
|
289 |
Repeated the --depclean --pretend but those last couple packages were deps |
290 |
of something already in @world so they didn't need added. |
291 |
|
292 |
So now I have an entirely empty @system, and packages parallel-merge much |
293 |
more efficiently. =:^) |
294 |
|
295 |
I was so happy, I took the opportunity to do a full emerge --emptytree |
296 |
@world, something I hadn't done since upgrading cpu/gpu/mobo/ram to a 6- |
297 |
core bulldozer and changing CFLAGS/CXXFLAGS accordingly, back in July, |
298 |
tho I had been keeping up with updates including a kde bump, so it wasn't |
299 |
too bad. |
300 |
|
301 |
Between the better parallel merging of the empty @system and the upgrade |
302 |
from a dual-dual-core opteron 290 @2.8 (so four cores of k8 tech) on the |
303 |
old system, to the new six-core buldozer (fx6100) slightly overclocked to |
304 |
3.6, I was rather happy with the full --emptytree @world merge time |
305 |
improvement. =:^) |
306 |
|
307 |
In fact, I was so happy with that, that I updated my 32-bit chroot on the |
308 |
same machine, where I build the image I rsync to the (32-bit-only atom |
309 |
n270) netbook, did an upgrade there, something that I hadn't done in a |
310 |
year and a half so it was a bit complicated to sort out but I managed, |
311 |
negated its entire @system and adjusted its custom sets appropriately, |
312 |
and then to be sure, did an emerge --emptytree @world there as well. =:^) |
313 |
|
314 |
|
315 |
Bottom line, an empty @system set really does make a noticeable |
316 |
difference in parallel merge handling, speeding up especially --emptytree |
317 |
@world rebuilds but also any general update that has a significant number |
318 |
of otherwise @system packages and deps, dramatically. I'm happy. =:^) |
319 |
|
320 |
--- |
321 |
[1] Global use file: I take advantage of the make.conf source statement |
322 |
to source a bunch of individual files. make.conf itself simply sources a |
323 |
master file (dead easy to recreate make.conf that way if it gets deleted/ |
324 |
overwritten), which in turn sources a bunch of individual files. ($>> is |
325 |
the third line of my customized bash $PS1 prompt, the first is a blank |
326 |
line and the second is deleted for posting.) |
327 |
|
328 |
$>>cat /etc/portage/make.conf |
329 |
source /etc/portage/make/master |
330 |
|
331 |
*$>>cat /etc/portage/make/master |
332 |
source /etc/portage/make/cflags |
333 |
source /etc/portage/make/collision-ignore |
334 |
source /etc/portage/make/features |
335 |
source /etc/portage/make/fs |
336 |
source /etc/portage/make/layman |
337 |
source /etc/portage/make/ldflags |
338 |
source /etc/portage/make/log |
339 |
source /etc/portage/make/makeopts |
340 |
source /etc/portage/make/mirrors |
341 |
source /etc/portage/make/net |
342 |
source /etc/portage/make/use |
343 |
source /etc/portage/make/use.expand |
344 |
source /etc/portage/make/other |
345 |
|
346 |
So I have a file, /etc/portage/make/use, that contains only my globally |
347 |
applied USE flags. There's another for CFLAGS/CXXFLAGS, another for |
348 |
LDFLAGS, another for the filesystem (fs) settings, another for MAKEOPTS, |
349 |
another for FEATURES, etc. |
350 |
|
351 |
[2] LVM2: I tried lvm2 at one point, and decided the additional |
352 |
complexity and negative effect on my confidence in my own ability to |
353 |
sanely manage disaster recovery and restore access to my data, was |
354 |
nothing I was interested in, and it's not worth having around on the |
355 |
system just to handle automount, either! |
356 |
|
357 |
FWIW I DID run with an md/raid configured system for awhile altho I |
358 |
recently upgraded and didn't have money for another disk so am not |
359 |
running it now, but md/raid was nice! =:^) |
360 |
|
361 |
[3] depclean summary: world_sets line request |
362 |
Clear-Text: http://bugs.gentoo.org/show_bug.cgi?id=298298 |
363 |
Secure: https://bugs.gentoo.org/show_bug.cgi?id=298298 |
364 |
|
365 |
-- |
366 |
Duncan - List replies preferred. No HTML msgs. |
367 |
"Every nonfree program has a lord, a master -- |
368 |
and if you use the program, he is your master." Richard Stallman |