Re: [gentoo-hardened] TESTING kevquinn_overlays: amd64 hardened gcc - (bugzilla is down, i hope the mails are not)

[Kevin F. Quinn]

On Mon, 8 Jan 2007 13:46:51 +0100 (CET)
"Peter S. Mazinger" <ps.m@×××.net> wrote:

> On Sat, 6 Jan 2007, Kevin F. Quinn wrote:
>
> > [...]
> > What we mess up, is that gcc's configure thinks that the compiler
> > always generates PIC code (which it doesn't, quite) and thus
> > doesn't do -fPIC -DPIC in places where it normally would do (it
> > just does -DPIC). This means that some objects that should be built
> > PIC, get built PIE with the hardened compiler.  Historically there
> > was a -D_IN_GCC_ (or something like that) filter in the specs which
> > meant that sometimes gcc generated PIE code and sometimes not,
> > depending on what part of gcc is being built.
> >
> > In order to have gcc build all its intermediate executables PIE
> > (and to be honest, I don't see any point in doing so), in a way
> > that works on all architectures, we would have to modify the build
> > process a lot more than we already do.
>
> If we want to have gcc's final binaries "hardened", then we can't
> probably avoid compiling the intermediate files "hardened" too, else
> we have to tweak too many parts of the build-system. Some will
> probably say that gcc is not needed hardened itself, because it is
> not used for runtime (exceptions being
> libgcc.a/libgcc_s.so/libsupc++.a/libstdc++.* and the crt files that
> go into each binary), I personally think that it is a good test for
> hardened to see if gcc fails somewhere, thus warning us about
> expected problems.

Well, building PIE doesn't find anything as such.  My preference at the
moment is to build gcc itself with SSP, NOW and RELRO (which don't
have any conflict from one stage to the next), but not PIE.  glibc I'm
building with both SSP and PIE switched off.

The crt files should definitely never be built -fPIE.  For the lib*.a
provided by gcc and libc, it may be worth building them both PIE and
no-PIE.  How they would be selected on use is another problem...

> > > [...]
> > > No, the code needs to be PIC, else the generated PIE binary has
> > > text relocation (PIE binary is more like a shared library needing
> > > though an interpreter/dynamic loader).
> >
> > The only time this is an issue is when lib*.a archives are being
> > used to link executables.  This is rather rare - indeed the only
> > places I've seen it done are in gcc and glibc.
> >
> >
> > ok; maybe I wasn't clear:
> >
> > 1) Objects to be linked into shared libraries must be PIC
> >
> > 2) Objects to be linked into position-independent executables
> > should be PIE (but PIC objects are work too, they just put more
> > through the PLT than necessary)
> >
> > 3) Objects to be linked into normal executables (i.e. not PIEs)
> > should be normal - neither PIC nor PIE.
> >
> > I think you'll agree those statements are true.
>
> not with 2)
> if that would be true, then the intermediate files would not have
> text relocations, libiberty.a's objects are compiled -fPIE instead of
> -fPIC (-DPIC does not influence the result)


If it's false, then we're in trouble - because PIEs would always have
textrels.

The way to build a PIE, as I understand it, is to:

(1) compile all objects -fPIE
(2) link with -fPIE -pie

If that creates executables with textrels, then we're in trouble
everywhere.  However I don't think that's the case - I understand we
get textrels when:

a) linking normal or -fPIE objects into shared libraries
b) linking normal objects in to PIE executables

and I think the textrels in the intermediate files occur when starting
from a vanilla compiler, where libiberty.a is built normally (i.e. not
PIE or PIC) in the first pass.  Possibly it's also affected by
configure thinking the compiler generates PIC by default.


> > So, to lib*.a archives.  These are collections of object files - how
> > they should be built depends on what they're going to be used for,
> > and the above rules apply in the same way.  In the vast majority of
> > cases, whenever someone is linking lib*.a archives into something,
> > it's into static binaries as the lib*.a is the static equivalent of
> > lib*.so.  For these cases, we should have normal (no-PIE) objects.
> >
> > The cases internal to gcc/glibc are exceptions, in that the
> > archives are used to build shared libraries (which use the
> > lib*_nonshared.a archives which should always be PIC anyway) and
> > those executables - which if we want them to be PIE of course need
> > PIE objects.  I think a better way to approach this sort of case is
> > to modify the build process explicitly to build PIE objects where
> > needed, rather than using the altered defaults.
>
> there are many exceptions, but you do not see them with a glibc/gnu
> system. On an uClibc hardened system, if configure fails to detect
> correctly the system (needing gnuconfig-update/libtoolize), it will
> probably build only lib*.a instead of both lib*.so/lib*.a, and the
> next app compiled against lib*.a will have text relocations.

I was not aware of that.  I would have expected embedded systems to be
keen on shared libraries, as they reduce physical memory requirements.
Would it be better to have autotools working properly with uclibc?  How
many packages are we talking about?  Would it be reasonable to suggest
they be modified to build shared libraries properly (perhaps by
directing configure explicitly)?

> > > > Strictly speaking, on a hardened system, the -vanilla and
> > > > -hardenednopie compiler cannot be guaranteed to generate working
> > > > executables - indeed it creates garbage on sparc (x86 seems to
> > > > be more forgiving; you can mix PIE objects into a non-PIE
> > > > executable and they appear to work).
> > >
> > > because lib*.a has to be PIC.
> >
> > Like I say, whether they should be normal, PIC or PIE depends on
> > what they're used for - and in general you just don't know.  The
> > most common use for lib*.a files from the system (/usr/lib etc) is
> > to build static binaries.
> >
> > Consider also the state of the crt*.o objects - on existing hardened
> > systems crtbegin.o and friends are built -fPIE.  This makes the
>
> no, crt*S.o are compiled -fPIC at least on linux, see
> gcc/config/t-linux CRTSTUFF_T_CFLAGS_S, only those without S are
> compiled -fPIE (if gcc defaults to hardened and ... and ...,
> depending on the version of the piepatches)

Yes.  It was just the non-S versions I was talking about (crtbegin.o,
crtend.o and friends, crt1.o from libc).

> > switching of startfile/endfile in the specs achieve almost nothing
> > - on x86, crtend.o and crtendS.o are identical, and
> > crtbegin.o/crtbeginS.o are almost the same.  Have we really
> > analysed the source for those objects, to verify that building them
> > -fPIE gives sensible results?  I suspect not.
>
> maybe we should add -fno-PIE to crtbegin.o/crtend.o/crtbeginT.o to
> guard against the above case.

I did try that, also building crt1.o (libc) with -fno-PIE.  The
modifications to the toolchain build files got more and more involved,
which I really want to avoid.  In addition, I suspect that will break
your "PIE/static" binaries; where perhaps it would be better to use
crt*S.o/Scrt1.o.

> > > > I'm working on a sane solution to this problem using ABIs, but
> > > > it's quite involved, and it's quite a departure from what we
> > > > currently do. Once I have my thoughts clear, I'll post a
> > > > proposal.
> > > >
> > > > For the moment, however, we can continue as we did before, but
> > > > highlight that -vanilla and -hardenednopie may not generate
> > > > working executables. Mostly this affects bootloaders, and
> > > > things like static busybox.
> > >
> > > Wondering why you mention static binaries,
> >
> > Static binaries should not be PIE - the two concepts don't mix.  It
> > doesn't make sense to build an ET_EXEC executable (which is what
> > static executables are) with PIE code.
>
> sadly, when compiling we do not know if the object will go into
> static/ET_EXEC/ET_DYN, else we could handle that (we handle only the
> case if the final binary is compiled and linked in one step)

Yep.  The only way I can think of to deal with that would be a
normal/PIE multilib setup.  For glibc systems it is probably not worth
the trouble (since we rarely use the lib*.a archives anyway).  For
uclibc, it would depend how much breakage there is in building shared
libraries.

> > > I have no problem on a
> > > clean hardened box with static binaries,
> >
> > What you have aren't really static binaries - they're ET_EXECs with
> > PIE objects inside. This kinda works on x86, but on sparc it just
> > segfaults on startup (due to the contents of crt*.o).
>
> I know ppc/arm does not have any problem with it (mips is out of
> question because it is always PIC except the kernel)
>
> give fno-PIE a try for those, maybe it solves the issue.

I did try that, I think, but without any success.


> > > even if the lib*.a are PIC
> > > compiled (mine are for sure, and most of you have it too, if an
> > > app is using autotools and running ./configure shows that the
> > > system supports shared libraries and for "dependant libraries" it
> > > says "pass_all"), I though agree that it is overkill for static
> > > binaries to use libraries that were PIC compiled.
> >
> > The problem with static binaries is that they're not correct on all
> > platforms, when built with PIE objects.  sparc in particular
> > segfaults.  It does create executables that run on x86, but they're
> > a bizarre PIE/ET_EXEC hybrid that makes little sense.
>
> sparc or sparc64 or both? have you compared assembler output? does it
> make a difference if you provide -K PIC to the assembler or not?

sparc64 (sparcv9).  The problem was symbols not being resolved
properly, iirc.  I don't remember exactly what I tried, now - I tried
many things, although not -K PIC (I would assume that would be passed
to the assembler when building -fPIE or -fPIC).

>
> Peter
>
> > What I've done for now, and am proposing we enter into the tree, is
> > to build gcc itself (and glibc) non-PIE.  This means the crt files
> > are built correctly, also libc.a and the gcc lib*.a's are non-PIE
> > and suitable for properly static binaries.
> >
> > This is still not entirely satisfactory; in a perfect world we'd
> > have lib*.a available both ET_EXEC and PIE.  The only half-sensible
> > way I can think of to do that, would be to set things up like a
> > multilib system.  However until portage supports multilib
> > (http://dev.gentoo.org/~kanaka/auto-multilib/ would be my preferred
> > approach) I don't think it's worth the trouble.
> >
> >
>


-- 
Kevin F. Quinn