[gentoo-amd64] Re: gcc 4.1 upgrade - bad desktop interactivity anyone?

[Duncan]

"Mark Knecht" <markknecht@×××××.com> posted
5bdc1c8b0609141743y7c5f6042ncd0a596eb290c8ba@××××××××××.com, excerpted
below, on  Thu, 14 Sep 2006 17:43:19 -0700:

>    Now, you are very adept at this. You're explanations make sense to
> the level I've considered them. (Not very far right now...) Main
> questions:

Adept, perhaps, but don't take my observations as being from God or
anything! =8^)  I try to be fairly cautious with my CFLAGS, but if
anything quits working, I know how to undo them and try with a more
generic set, and in fact do so from time to time on individual packages,
before filing bugs on them.  Sometimes it's my CFLAGS, tho usually my
config doesn't matter a whit to the bug, as I've been reasonably cautious
in my choices to begin with and don't tend to enable stuff like the unsafe
floating-point math options that give folks problems from time to time.

In particular, as you can see from the -ftree-vectorize subthread, I tend
to stay with the defaults when I can't explain with some degree of
confidence exactly what the effect of a flag might be and why I might or
might not want it.  I don't know enough about that area to do that, so
I've stayed well away from it in my CFLAGS.

> 1) What can be done to test this out at my end without making a 2-day
> commitment to rebuild the complete machine. Is it possibly to rebuild
> only portions of the machine using a different set of flags or is it a
> system wide commitment requiring that I rebuild 575 packages as I did
> last weekend?

In general, you /can/ rebuild only a part of your system and test that,
before making further changes.  However, it's important to use a bit of
(un?)common sense when doing so, or your results won't be worth much. 
Basically, in ordered to see how an optimization affects something, you
must have some awareness of the shared libraries it uses and to what
extent it uses them, recompiling enough of the heavily used dependencies
that the critical parts of your test applications (including the libraries
they load) are using the new optimizations.

One lib that all applications make some use of is glibc, so it can be worth
recompiling.  It's a big recompile on its own, but of course nowhere near
as big as recompiling the entire system. =8^)  However, glibc is a special
case in some aspects for a number of reasons. The glibc ebuild is pretty
conservative with the flags it allows, and actually replaces -Os with -O2,
due to problems -Os had mainly on x86, back in the gcc-3.2 and 3.3 era. 
Since the system is pretty horribly broken if glibc breaks, to the point
you are likely to have to boot to a backup or liveCD to fix it, this isn't
an unreasonable policy at all.

None-the-less, after making doubly sure I had tested-working backups, I
decided to see just what the effect of taking out that -Os -> -O2 replace
in the glibc ebuild might be.  For awhile I actually ran a glibc I had
built after having removed that replace.  The system continued to work
just fine with a -Os compiled glibc, it didn't break or anything, but it
didn't seem to be much better either and in some cases seemed worse.  It
turns out that glibc is built in a much more modular fashion than many
libraries, so an app will only load the parts of it it needs, not the
parts it doesn't, and that -Os doesn't work so well with this rather
extreme (compared to most libs) modularization.  As well, as I said, glibc
is used by everything on the system, which meant that having bypassed one
of the safeties in the glibc ebuild, I could never be sure whether a bug I
was experiencing was due to my strange glibc, or to some problem with the
package the bug was showing up in or one of its other dependencies.  I
concluded that it simply wasn't worth bypassing the safeties in the
ebuild, and since then, have left them there.

Thus, with glibc anyway, simply switching to -Os in your CFLAGS won't make
any difference, since the ebuild replaces that with -O2 anyway.  The
/other/ CFLAGS might make a difference, but -Os it self won't, unless you
bypass the replace in the ebuild, and as my experimentation demonstrated
well enough for me, that's really not worth the trouble.  As I said, the
other CFLAGS may make a bit of difference tho, so you might consider it
anyway, if you decide to try them.

For X users, another library that's going to be commonly used is libX11. 
You'll probably want to recompile xorg-server (assuming modular-X) as
well, plus whatever xf86-video-* driver you use, and libXcomposite if you
use the composite extension (transparent windows and the like).  Together,
those will be pretty critical for performance of any X app.  For OpenGL
accelerated apps, mesa is likely to be critical to performance as well,
for any functions not handled by hardware.

For anything written in C++, almost anything KDE among other packages, gcc
libstdc++, a part of gcc, will be critical.  Other than for C++
apps/libraries, recompiling gcc with new CFLAGS shouldn't make that much
difference in how the app runs, tho it might make some difference in how
fast compiles the app.  (Of course, note that compile speed can be
dramatically affected by the optimizations being compiled, since many of
them cause additional passes in the process, to catch that little bit of
extra optimization you are telling gcc to enable.  Tell gcc to do more
work and of course it'll take longer doing it!)

You mentioned gnome.  For anything gnome related, you'll want to recompile
glib, gtk, libgnome, possibly orbit, and maybe others (your window
manager). I don't know enough about gnome internals to go further with it.
With KDE, it'd be qt and kdelibs (plus as I mentioned gcc, for libstdc++),
kwin, probably kicker and konqueror, etc, plus the packages behind various
kparts as used in whatever app you are testing, as well as the app itself.

> 2) What about building the kernel? How do the standard
>
> make && make modules_install
>
> command make any use of the flags in /etc/make.conf?

They normally don't.  However, as is commonly the case with manually
compiled packages, the kernel build process makes use of CFLAGS if it
finds that environment variable set.  Thus, you can source make.conf,
export the desired variables, and go from there.  In fact, I have a small
utility script installed as /usr/local/bin/buildflags.sh that contains the
following:

# Source this previous to doing a manual build to import portage buildflags

[ -z "$CFLAGS" ] && CFLAGS="`. /etc/make.conf 2>/dev/null; echo $CFLAGS`"
[ -z "$CXXFLAGS" ] && CXXFLAGS="`. /etc/make.conf 2>/dev/null; echo $CXXFLAGS`"
[ -z "$LDFLAGS" ] && LDFLAGS="`. /etc/make.conf 2>/dev/null; echo $LDFLAGS`"
export CFLAGS CXXFLAGS LDFLAGS

As you can see, that gathers and exports the three build variables CFLAGS,
CXXFLAGS, and LDFLAGS.  If I source that into my running shell before I
build anything (including the kernel) manually, the build should see my
existing environment and compile accordingly.

As it happens, I use a script to compile and install the kernel, as well,
and it sources buildflags.sh before it does the kernel compile.

/usr/local/sbin/ki (for kernel install:

#!/bin/bash
# kernel compile and install
echo
echo sourcing buildflags
ki_buildflagfile=/usr/local/bin/buildflags.sh
if [ -f $ki_buildflagfile ] ; then
    . $ki_buildflagfile
else
    pause 10 x \\a$ki_buildflagfile doesn\'t exist
fi
unset buildflagfile

echo
echo cd-ing to /usr/src/linux
cd /usr/src/linux
echo Next: make
sleep 1
echo
make || exit 1
echo
echo Next: make modules_install
sleep 1
echo
make modules_install || exit 2
echo
echo Next: mount /boot
sleep 1
echo
mount /boot &>/dev/null
echo
echo Next: make install
sleep 1
echo
make install || exit 3
echo
echo Next: umount /boot
sleep 1
echo
umount /boot &>/dev/null
echo Done.
echo

In addition to using buildflags.sh, that script also uses a utility script
I've written called pause, installed as /usr/local/bin/pause (and also as
/bin/pause here, so I can use it to debug initscripts before /usr/local is
mounted).

#!/bin/bash

# putting everything in a function to allow local scoped vars
function doit {
echo
local PROMPT="Pause for char"
case $# in
        0)      read -n1 -p "$PROMPT:";;
        1)      if [ $1 == "--help" -o $1 == "-h" -o $1 == "-?" ]
                then
                        echo $PROMPT script utility.
                        echo
                        echo Usage: pause [timeoutsec [defaultchar [promptstring]]]
                        echo \ \ \ \ \ \ \ pause \(--help\|-h\|-?\)
                        echo
                        echo Source pause to have \$REPLY set to the returned char.
                        echo
                        echo A timeoutsec of 0 indicates no timeout.  Otherwise,
                        echo on timeout, defaultchar if provided is returned in \$REPLY.
                        echo
                        echo With timeout and defaultchar, promptstring defaults to:
                        echo $PROMPT \(default defaultchar in timeoutsec\):
                        echo With just a timeout, prompt default is:
                        echo $PROMPT \(timeout timeoutsec\):
                        echo Without a timeout, the default is:
                        echo $PROMPT:
                        echo
                        echo
                elif [ $1 == 0 ]
                then read -n1 -p "$PROMPT:"
                else read -n1 -t$1 -p "$PROMPT (timeout $1):"; echo
                fi;;
        2)      if [ $1 == 0 ]
                then read -n1 -p "$PROMPT:"
                else
                        read -n1 -t$1 -p "$PROMPT (default $2 in $1):"; echo
                        REPLY=${REPLY:-$2}
                fi;;
        *)      if [ $1 == 0 ]
                then
                        shift 2
                        echo -e $*
                        read -n1
                else
                        local TIMEOUT=$1 DEFAULTCHAR=$2; shift 2
                        echo -e $*
                        read -n1 -t$TIMEOUT; echo
                        REPLY=${REPLY:-$DEFAULTCHAR}
                fi;;
esac
echo
}

doit $*

Obviously, I've taken a bit more time to properly document pause, so you
can run pause -h or --help to get the usual usage summary.  The other two
scripts aren't so nicely --help equipped; you have to look at the source
to see what they are doing.  I've found pause to be very useful over the
years. =8^)

As you can see, I routinely compile the kernel with my chosen CFLAGS, and
it works fine here.  Of course, your kernel config will be different and I
can't vouch for every single driver, but the concept is sane and my CFLAGS
are demonstrated to work with the core kernel and the drivers I use,
anyway.

As Greg mentions, there's also a compile for size option in the kernel
config, and I have it enabled as well.  It enables -Os, but I think it
does a few other kernel specific things as well.  I've never looked into
it in detail, but I know I've had it enabled every since its introduction
some kernels ago, with the accompanying coverage in LWN.  That coverage in
fact was what got me started on the whole -Os thing, before I'd even
switched to Gentoo with 2004.1 and was still on Mandrake IIRC, so it has
been a kernel option and I've been using it for over two years now.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@g.o mailing list