[gentoo-amd64] Re: CFLAGS question from a AMD64 newbie

[Duncan]

Branko Badrljica <brankob@××××××××××.com> posted
49485D48.9000001@××××××××××.com, excerpted below, on  Wed, 17 Dec 2008
03:00:40 +0100:

> Duncan wrote:
>
>> The feature is somewhat limited, in that these files are sourced in the
>> bash portion of portage (basically, ebuild.sh) -- the python portion
>> knows nothing about them.
> GREAT !
> Can I use them for switching gcc compiler etc, not just for simple
> variable settings ?

I've never tried anything but variable settings, but it's sourced as 
bash, so you should be able to do pretty much anything you could do in a 
bash script, so yes, gcc-config ..., etc, should work... again, for the 
bash side only.

I'm not sure how much the python might depend on gcc, etc, however.  I do 
know that any deps (say if the package is in fortran or java and it tests 
gcc to see if it was merged with the appropriate USE flags) are resolved 
in the python bit, in emerge itself after it has sourced the ebuilds for 
deps, but before it actually starts on any of the ebuild phases.  Thus, 
if you go messing with that on a package where it's a dependency, you're 
likely to have "undesired results".  However, I'm not sure if portage/
emerge depends on gcc and/or anything else staying stable beneath it or 
not.

There's also the parallel merge ability in newer portage to think about.  
If you use it and invoke one of these env scripts, who knows /what/ stage 
some other package might be at in parallel merge!  That'd DEFINITELY be 
an invitation to trouble, since gcc-config switches things system-wide, 
and switching versions of gcc in the middle of compiling say glibc could 
literally bring the entire system crashing down around your ears when it 
tries to qmerge that half-n-half glibc to the running system!

Thus I'd certainly recommend sticking to stuff that's going to affect 
only that single package, setting environment variables, etc.  Don't do 
anything that will affect the system globally unless you are seriously 
looking for trouble.

What I'd suggest, if you want to push the boundaries, is to either just 
try it and see what you can get away with, or go looking in the forums 
and etc.  There's gotta be a thread or two on the subject.  You could 
also see if there's anything on the gentoo-user list archive (I use 
gmane, Gentoo has its own archive, and I think there's at least one more 
archive, Mark's or some such) about it, and/or ask there.  There's some 
pretty knowledgeable people out there -- way more than just what I know 
on it.

> And btw, is there a way to switch gcc back after emerge without changing
> ebuild file ?

Well, yes, but you're getting yet another layer deeper into the advanced 
stuff.  I haven't played around with them myself directly, but there are 
pre-phase and post-phase hooks for each phase.  Presumably you'd stick 
the switch back in the post-compile-hook (and I'm not sure I got the name 
exactly right) routine.  More below...

> Or maybe there is some other mechanism to be able to compile package
> with particular gcc ?

This doesn't directly answer that question, but it should give you a 
better idea the sort of flexibility available...

In addition to the per-package env files, there's the general 
/etc/portage/bashrc (see the file list and descriptions in
man (5) portage).  This is actually cascaded in a specific order (which 
IDR ATM) with the profile.bashrc(s) at the various levels of your 
(cascading) profile.  Among other things, the Gentoo/amd64 project was I 
believe the first to use this method to filter invalid cflags (say you 
use gcc-4.3.x normally, with certain cflags only valid from it on, and 
gcc-config back to a gcc-4.2.x version that doesn't recognize these 
cflags, one of the bashrc layers filters the invalid ones).  Take a look 
at the various *bashrc files in the profiles to get an idea of what 
Gentoo is already using them for!  (Actually, I believe the env files 
were first implemented this way but at a global not profile level, tho in 
newer portage they may be called directly.)

There's actually some pretty advanced bashrc and phase hooks 
implementations around.  Ed Catmur, the guy who wrote udept, has a nice 
one that allows all sorts of interesting stuff.  Unfortunately, he seems 
to have disappeared (and udept is in the process of being masked since 
there's no upstream any more and it doesn't support the latest EAPIs, 
I've no idea if he simply disappeared without warning or if he got in an 
argument with a Gentoo dev or what, but the Gentoo stuff on his site now 
says it needs a password, which could as easily mean it doesn't exist 
there anymore at all), but I have a copy of his tarballs from a year or 
so ago.

I use only one of the several features his bashrc and phase hooks scripts 
made possible, FEATURES=patchtree.  The idea and implementation is 
similar to but not identical to the env tree, in this case, a
category/package based tree under /etc/portage/patches/, only the package 
is in this case another subdir instead of a file as it is with the env 
tree.  

Here's an example of how I use it.  As I regularly install and set as my 
normal system gcc still-hard-masked versions, I'm used to checking bugs 
for patches to enable various packages to compile with the newer gcc, 
well before the patches actually find their way into the tree.  If I can 
find an appropriate patch, I can dump the patch file into an 
appropriately versioned patches/category/package-ver subdir, and with his 
scripts in place and that feature turned on, portage will automatically 
apply the patch post-unpack-phase.  His scripts are even smart enough to 
determine in many cases whether auto-conf, etc, needs rerun, and handle 
it automatically, altho they don't handle every possibility.  While I 
used to nearly always have at least a half dozen ebuilds in my overlay, 
with various patches and etc applied so they'd build with the latest gcc 
or glibc or whatever, now I just dump the patches in an appropriately 
created subdir and let portage and Ed's FEATURES=patchtree handle it!  
Between that and the env tree, I seldom have to carry patched ebuilds in 
my overlay any more, and it's often empty and seldom has more than a 
couple more complex case ebuilds in it.

Another feature I do NOT use from his scripts, is 
FEATURES=installsources, which keeps the sources around for reference.  I 
think a number of Gentoo devs are actually using this feature, so they 
can keep sources around for what they're working on.  I think it can be 
toggled per package but as I said I don't use it so I haven't looked too 
deeply into how it works.

There's some more, as well.  IDR the details, but looked at them and 
decided I didn't need them here.

If you're interested in the tarballs, maybe even only to take a look at 
for ideas to see what's actually possible, ask, and I'll mail them to 
you.  Of course with him apparently disappeared and no longer involved 
with Gentoo, if/when they eventually start to break, you'll need to deal 
with that, but they're working fine so far with the portage-2.2-rcs I've 
been running, and I think the portage hooks they use are intended to stay 
around for awhile, so if it breaks, it'll probably be inadvertent (as I 
said I believe a number of devs use those scripts...), and a portage bug 
report and/or a bit of bash hacking should hopefully get it working again.

> One more question:
>
> What if two or more files match the package ?
>
> Like:
>
> env/sys-kernel/vanilla-sources
> env/sys-kernel/vanilla-sources-2
> env/sys-kernel/vanilla-sources2.6
> env/sys-kernel/vanilla-sources-2.6.27
>
> Do they all get included in some /un/specified order or just
> first/last/unspecified one ?

I don't remember the specific details, in part because as I described, I 
have the patch tree to think about as well, and I never remember the 
versioning formats for either.  I know the first and the last example 
above should work, and 2.6.27 would work for -rX versions as well (found 
that out the other day, the ~ and = prefixes don't work, but just the 
version applies for all revisions, too), but generally get confused and 
often end up simply trying it until I get it to work, for the odd cases.

Matter of fact, I spent a bit of time looking for documentation on 
exactly what version format to use for env files myself, the other day, 
but didn't find it.  That's when I discovered the versioning info 
mentioned above, simply by testing until I got it working.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman