[gentoo-dev] IMPORTANT: append-flags -fPIC

[Daniel Robbins]

On Wed, 2003-12-10 at 18:07, Aron Griffis wrote:
> I think it's important all developers see this, so I'm forwarding it to
> gentoo-core and gentoo-dev...  (I'm referring to the text from Daniel,
> not necessarily the fact that I screwed up ;-)

Thanks Aron :)

Just to give some context, we were talking about situations where a line
this is needed to get an ebuild to compile on non-x86 arches:

append-flags -fPIC

Here is what you need to know about situations like this:

1) If you need to add append-flags -fPIC to get something to compile,
the Makefile is broken for all arches including x86. This is true *even
if* it seems to compile and run OK without -fPIC on x86. Here's why.

2) If it needs an -fPIC to compile on a non-x86 arch, it actually also
needs -fPIC on x86 -- not to get the thing to compile correctly of
course but for another reason: to avoid wasting memory when the code is
used. Non-PIC library code on x86 wastes memory when used, even though
it technically "works". The dynamic loader is forced to create a private
copy for each process using the non-PIC code, defeating one of the main
reasons of using shared libraries in the first place -- to conserve RAM.

3) The proper solution for things that need -fPIC is to create a patch
for the Makefiles to add -fPIC to only those sources that specifically
need it (shared library source onlt), send the patch upstream, and then
have the ebuild apply it for *all* arches, not just the non-x86 arches.
This does 2 things:

	a) gets the code to compile on non-x86 systems
	b) makes the code run with optimal memory usage on x86 systems

append-flags -fPIC is a band-aid that gets the program to work, but
isn't an optimal fix. Why can't we just enable -fPIC globally? Because
you may be enabling it for non-shared-lib code. Compiled PIC code uses
up an extra machine register for storing a memory offset. This means
that by globally enabling -fPIC, you will be hurting performance for
things that don't need relocatable code by causing an extra machine
register to be unecessarily used. (According to pipacs, it is rumored
that amd64 has some new addressing mode that allows PIC code to be just
as efficient as non-PIC code, although he has not confirmed this yet.)

(Incidentally, you may have noticed that some 'l33t' shared libraries
purposely use non-PIC code on x86 -- I think ffmpeg may be one such lib
-- by doing this, they are trying to gain one extra machine register,
and are willing to pay the price of forcing each process to have their
own personal copy of the shared library in RAM for the sake of the extra
performance they can squeeze out of having that extra register.)
NVIDIA's OpenGL libraries appear to do this on x86. On AMD64 they do not
(probably because non-PIC .so's on AMD64 appear to be illegal.)

The vast majority of this info comes from pipacs, who is working with
the hardened team on security stuff. It's amazing what you can learn
from a low-level assembly programmer in a couple of hours :)

Regards,

Daniel