Re: [gentoo-hardened] TESTING kevquinn_overlays: amd64 hardened gcc - (bugzilla is down, i hope the mails are not)

[Peter S. Mazinger]

On Sat, 6 Jan 2007, Kevin F. Quinn wrote:

> On Sat, 6 Jan 2007 20:56:22 +0100 (CET)
> "Peter S. Mazinger" <ps.m@×××.net> wrote:
>
> > On Mon, 18 Dec 2006, Kevin F. Quinn wrote:
> >
> > > On Mon, 18 Dec 2006 11:55:31 +0100 (CET)
> > > "Peter S. Mazinger" <ps.m@×××.net> wrote:
> > >
> > > > On Tue, 12 Dec 2006, Alexander Gabert wrote:
> > > >
> > > > > Kevin: this is the build failures, i would have made them end up
> > > > > adding into bugzilla but one of us both wasn't awake.
> > > > >
> > > > > PSM: if you're reading up: i don't have a clue.
> > > > >
> > > > > In the meantime, working with bugs.g.o. it's getting a real
> > > > > hassle, so can someone please kick that pos in the shins so we
> > > > > can work on bugs again?  TIA
> > > > >
> > > > > I think we should have a talk or two and maybe a test case for
> > > > > building that libiberty on all arches with either PIC or PIE
> > > > > code. I honestly don't care howe we build it, i just want it to
> > > > > work on all Gentoo supported arches for hardened.
> > > > >
> > > > > For those of you having issues with the file: here is the log
> > > > > for pitr and miranda:
> > > > >
> > > > > http://dev.gentoo.org/~pappy/tmp/amd64-pie-libiberty-failure.txt
> > > >
> > > > It seems some of you are mixing up PIC and PIE. For the record,
> > > > there are no PIE libraries and if you want to run a hardened box
> > > > where each file is text relocation free, then you have to compile
> > > > all code that goes to libraries (even into lib*.a) with
> > > > -fPIC|-fpic. To quickly check this statement, compile some object
> > > > with -fPIE, put into libx.a and compile a shared lib with -fPIC
> > > > using code from libx.a. readelf -d shared_lib | grep TEXTREL.
> > >
> > > Agreed - however lib*.a are not generally used to build shared
> > > libraries; they're mostly for building static binaries.  Gcc deals
> > > with libiberty (normally) by building a pic version
> > > (libiberty_pic.a) for linking to shared libraries when it needs
> > > to.  We do mess that up, I think - I'm still looking into that.
> >
> > we do not mess it up, libiberty_pic.a is not used to compile the
> > intermediate gen* binaries and if the build specs defaults to PIE
> > binaries and then these binaries have text relocations, thus failing
> > on a PaX box if the text relocations are blocked (or it is an uClibc
> > hardened box, where libc blocks text relocations as well)
>
> What we mess up, is that gcc's configure thinks that the compiler
> always generates PIC code (which it doesn't, quite) and thus doesn't do
> -fPIC -DPIC in places where it normally would do (it just does -DPIC).
> This means that some objects that should be built PIC, get built PIE
> with the hardened compiler.  Historically there was a -D_IN_GCC_ (or
> something like that) filter in the specs which meant that sometimes gcc
> generated PIE code and sometimes not, depending on what part of gcc is
> being built.
>
> In order to have gcc build all its intermediate executables PIE (and to
> be honest, I don't see any point in doing so), in a way that works on
> all architectures, we would have to modify the build process a lot more
> than we already do.

If we want to have gcc's final binaries "hardened", then we can't probably 
avoid compiling the intermediate files "hardened" too, else we have to 
tweak too many parts of the build-system. Some will probably say that gcc 
is not needed hardened itself, because it is not used for runtime 
(exceptions being libgcc.a/libgcc_s.so/libsupc++.a/libstdc++.* and the 
crt files that go into each binary), I personally think that it is a good 
test for hardened to see if gcc fails somewhere, thus warning us about 
expected problems.
 
> > > > The pie patches compile everything with -fPIE unless -fPIC was
> > > > used, the app itself has to take care (and most apps using
> > > > libtool do it by using -fPIC -DPIC) that code that goes into libs
> > > > is -fPIC compiled. libtool fails only in cases where configure
> > > > does not recognize correctly the target (thats why
> > > > gnuconfig_update/libtoolize are needed).
> > > >
> > > > Some will argue, that it does not make sense to compile code in
> > > > lib*.a with -fPIC. Those shouldn't use hardened, because they end
> > > > up with text relocations and a hardened box should block text
> > > > relocations either in kernel (PaX feature) or libc (possible with
> > > > uClibc, but not glibc)
> > >
> > > I think it's more complicated than that.  What should go into a
> > > lib*.a archive depends on how it's going to be used.  If it's going
> > > to be used to build a shared library, then as you say the code
> > > needs to be PIC - however this is rare (examples are
> > > libc_*nonshared.a).  If it's being used to build a non-PIE
> > > executable - which usually means building a static executable -
> > > then the code needs to be normal (non-PIC).  If it's going to be
> > > used to build a (PIE) executable, the code needs to be PIE - in
> > > practice this is very rare, since linking to lib*.a instead of
> > > lib*.so means building static binaries, and that's not sensible
> > > when building PIEs as by definition they need the dynamic loader.
> >
> > No, the code needs to be PIC, else the generated PIE binary has text
> > relocation (PIE binary is more like a shared library needing though
> > an interpreter/dynamic loader).
>
> The only time this is an issue is when lib*.a archives are being used
> to link executables.  This is rather rare - indeed the only places I've
> seen it done are in gcc and glibc.
>
>
> ok; maybe I wasn't clear:
>
> 1) Objects to be linked into shared libraries must be PIC
>
> 2) Objects to be linked into position-independent executables should be
> PIE (but PIC objects are work too, they just put more through the PLT
> than necessary)
>
> 3) Objects to be linked into normal executables (i.e. not PIEs) should
> be normal - neither PIC nor PIE.
>
> I think you'll agree those statements are true.

not with 2)
if that would be true, then the intermediate files would not have text 
relocations, libiberty.a's objects are compiled -fPIE instead of -fPIC 
(-DPIC does not influence the result)

>
> So, to lib*.a archives.  These are collections of object files - how
> they should be built depends on what they're going to be used for, and
> the above rules apply in the same way.  In the vast majority of cases,
> whenever someone is linking lib*.a archives into something, it's into
> static binaries as the lib*.a is the static equivalent of lib*.so.  For
> these cases, we should have normal (no-PIE) objects.
>
> The cases internal to gcc/glibc are exceptions, in that the archives are
> used to build shared libraries (which use the lib*_nonshared.a archives
> which should always be PIC anyway) and those executables - which if we
> want them to be PIE of course need PIE objects.  I think a better way
> to approach this sort of case is to modify the build process explicitly
> to build PIE objects where needed, rather than using the altered
> defaults.

there are many exceptions, but you do not see them with a glibc/gnu 
system. On an uClibc hardened system, if configure fails to detect 
correctly the system (needing gnuconfig-update/libtoolize), it will 
probably build only lib*.a instead of both lib*.so/lib*.a, and the next 
app compiled against lib*.a will have text relocations.

> > > Strictly speaking, on a hardened system, the -vanilla and
> > > -hardenednopie compiler cannot be guaranteed to generate working
> > > executables - indeed it creates garbage on sparc (x86 seems to be
> > > more forgiving; you can mix PIE objects into a non-PIE executable
> > > and they appear to work).
> >
> > because lib*.a has to be PIC.
>
> Like I say, whether they should be normal, PIC or PIE depends on what
> they're used for - and in general you just don't know.  The most common
> use for lib*.a files from the system (/usr/lib etc) is to build static
> binaries.
>
> Consider also the state of the crt*.o objects - on existing hardened
> systems crtbegin.o and friends are built -fPIE.  This makes the

no, crt*S.o are compiled -fPIC at least on linux, see gcc/config/t-linux 
CRTSTUFF_T_CFLAGS_S, only those without S are compiled -fPIE (if gcc 
defaults to hardened and ... and ..., depending on the version of the 
piepatches)

> switching of startfile/endfile in the specs achieve almost nothing - on
> x86, crtend.o and crtendS.o are identical, and crtbegin.o/crtbeginS.o
> are almost the same.  Have we really analysed the source for those
> objects, to verify that building them -fPIE gives sensible results?  I
> suspect not.

maybe we should add -fno-PIE to crtbegin.o/crtend.o/crtbeginT.o to guard 
against the above case.

> > > I'm working on a sane solution to this problem using ABIs, but it's
> > > quite involved, and it's quite a departure from what we currently
> > > do. Once I have my thoughts clear, I'll post a proposal.
> > >
> > > For the moment, however, we can continue as we did before, but
> > > highlight that -vanilla and -hardenednopie may not generate working
> > > executables. Mostly this affects bootloaders, and things like
> > > static busybox.
> >
> > Wondering why you mention static binaries,
>
> Static binaries should not be PIE - the two concepts don't mix.  It
> doesn't make sense to build an ET_EXEC executable (which is what static
> executables are) with PIE code.

sadly, when compiling we do not know if the object will go into 
static/ET_EXEC/ET_DYN, else we could handle that (we handle only the case 
if the final binary is compiled and linked in one step)

> > I have no problem on a
> > clean hardened box with static binaries,
>
> What you have aren't really static binaries - they're ET_EXECs with PIE
> objects inside. This kinda works on x86, but on sparc it just segfaults
> on startup (due to the contents of crt*.o).

I know ppc/arm does not have any problem with it (mips is out of question 
because it is always PIC except the kernel)

give fno-PIE a try for those, maybe it solves the issue.

> > even if the lib*.a are PIC
> > compiled (mine are for sure, and most of you have it too, if an app
> > is using autotools and running ./configure shows that the system
> > supports shared libraries and for "dependant libraries" it says
> > "pass_all"), I though agree that it is overkill for static binaries
> > to use libraries that were PIC compiled.
>
> The problem with static binaries is that they're not correct on all
> platforms, when built with PIE objects.  sparc in particular
> segfaults.  It does create executables that run on x86, but they're a
> bizarre PIE/ET_EXEC hybrid that makes little sense.

sparc or sparc64 or both? have you compared assembler output? does it make 
a difference if you provide -K PIC to the assembler or not?

Peter

> What I've done for now, and am proposing we enter into the tree, is to
> build gcc itself (and glibc) non-PIE.  This means the crt files are
> built correctly, also libc.a and the gcc lib*.a's are non-PIE and
> suitable for properly static binaries.
>
> This is still not entirely satisfactory; in a perfect world we'd have
> lib*.a available both ET_EXEC and PIE.  The only half-sensible way I
> can think of to do that, would be to set things up like a multilib
> system.  However until portage supports multilib
> (http://dev.gentoo.org/~kanaka/auto-multilib/ would be my preferred
> approach) I don't think it's worth the trouble.
>
>

-- 
Peter S. Mazinger <ps dot m at gmx dot net>           ID: 0xA5F059F2
Key fingerprint = 92A4 31E1 56BC 3D5A 2D08  BB6E C389 975E A5F0 59F2

-- 
gentoo-hardened@g.o mailing list