pieworld (was Re: [gentoo-hardened] TESTING kevquinn_overlays: amd64 hardened gcc - (bugzilla is down, i hope the mails are not))

[Kevin F. Quinn]

On Mon, 15 Jan 2007 19:58:37 +0100 (CET)
"Peter S. Mazinger" <ps.m@×××.net> wrote:

> On Mon, 8 Jan 2007, Kevin F. Quinn wrote:
>
> > Well, building PIE doesn't find anything as such.  My preference at
> > the moment is to build gcc itself with SSP, NOW and RELRO (which
> > don't have any conflict from one stage to the next), but not PIE.
> > glibc I'm building with both SSP and PIE switched off.
>
> Why do you not add -fPIC to libiberty.a and all the PIE issues within
> intermediate files are gone?

The main reason is that forcing libiberty.a is fixing the symptom,
rather than the cure.  I think the cure is to fix the configure script
so that it understands default -fPIE doesn't mean the platform is all
default PIC.  I'm pretty sure that's the root cause.


> The other easy way is guarding pie with !D_INGCC:!D_LIBGCCx, where x
> is maybe empty, maybe 2, cant recall now
>
> > The crt files should definitely never be built -fPIE.  For the
> > lib*.a provided by gcc and libc, it may be worth building them both
> > PIE and no-PIE.  How they would be selected on use is another
> > problem...
>
> agreed that crt*.o should not be PIC/PIE
> I dont think it is worth the trouble to have PIE versions of the
> crt*S versions, lets have only the PIC versions as it is, Scrt1.o
> could be though -fPIE.

Yep.

> libgcc.a/libgcc_eh.a/libstdc++.a/libsupc++.a are all PIC and I cant
> think of a good way to handle them multilib-alike, so I would keep
> them PIC.

Yep.

> The multilib option could be applied though for the libs
> provided by glibc

I'm going to give up on the multilib-alike - it's just too messy.  So
I'm returning to 'pieworld' but fixed to avoid -fPIE crtstuff, and have
static PIEs working consistently everywhere.


> > > > > [...]
> > > > > No, the code needs to be PIC, else the generated PIE binary
> > > > > has text relocation (PIE binary is more like a shared library
> > > > > needing though an interpreter/dynamic loader).
> > > >
> > > > The only time this is an issue is when lib*.a archives are being
> > > > used to link executables.  This is rather rare - indeed the only
> > > > places I've seen it done are in gcc and glibc.
> > > >
> > > >
> > > > ok; maybe I wasn't clear:
> > > >
> > > > 1) Objects to be linked into shared libraries must be PIC
> > > >
> > > > 2) Objects to be linked into position-independent executables
> > > > should be PIE (but PIC objects are work too, they just put more
> > > > through the PLT than necessary)
> > > >
> > > > 3) Objects to be linked into normal executables (i.e. not PIEs)
> > > > should be normal - neither PIC nor PIE.
> > > >
> > > > I think you'll agree those statements are true.
> > >
> > > not with 2)
> > > if that would be true, then the intermediate files would not have
> > > text relocations, libiberty.a's objects are compiled -fPIE
> > > instead of -fPIC (-DPIC does not influence the result)
> >
> >
> > If it's false, then we're in trouble - because PIEs would always
> > have textrels.
> >
> > The way to build a PIE, as I understand it, is to:
> >
> > (1) compile all objects -fPIE
> > (2) link with -fPIE -pie
>
> (2) link with -pie (-fPIE does not/should not influence linking)

I would have thought so, however gcc info says:

`-pie'
     Produce a position independent executable on targets which support
     it.  For predictable results, you must also specify the same set
     of options that were used to generate code (`-fpie', `-fPIE', or
     model suboptions) when you specify this option.

so I was just following that advice.  I suspect it makes no difference
on x86, amd64, ppc - but perhaps it does on some platforms.

I think that when linking, the gcc driver also processes the cc1 spec,
(where you've added ${pie:-fPIE}) so our default is to add '-fPIE -pie'
anyway.


> > If that creates executables with textrels, then we're in trouble
> > everywhere.  However I don't think that's the case - I understand we
> > get textrels when:
> >
> > a) linking normal or -fPIE objects into shared libraries
> > b) linking normal objects in to PIE executables
>
> due to a) we have to have lib*.a PIC, unless we do the multilib
> approach with a separate set of static libs compiled -fPIE

Why would anyone link a lib*.a into a shared library?  Surely you would
just link to the .so?


> > and I think the textrels in the intermediate files occur when
> > starting
> > >from a vanilla compiler, where libiberty.a is built normally (i.e.
> > >not
> > PIE or PIC) in the first pass.  Possibly it's also affected by
> > configure thinking the compiler generates PIC by default.
>
> earlier the PIC patch for libiberty.a was ok, why is now a sed
> expression a so big trouble adding -fPIC to libiberty.a, wont hurt
> anyone, does not make anything bigger and so on..., I dont really
> understand all these objections that keep us discussing and not
> solving anything

See above (top) - I think fixing configure sorts this out properly,
because the build process then proceeds as upstream wrote it to.


> Note: Kevin, you disabled 2 patches in hardened overlay, the
> libstdc++_pic one is coming from uClibc, better consult vapier about
> that if you want it removed.

ok; I just disabled that to be sure I wasn't getting any forced -fPIC
stuff, to check the configure fix.  If libstdc++.a is acceptable -fPIE
then I think we're fine - if you really need it -fPIC, then that
implies you're linking it into shared libraries which is strange.


> > > > So, to lib*.a archives.  These are collections of object files
> > > > - how they should be built depends on what they're going to be
> > > > used for, and the above rules apply in the same way.  In the
> > > > vast majority of cases, whenever someone is linking lib*.a
> > > > archives into something, it's into static binaries as the
> > > > lib*.a is the static equivalent of lib*.so.  For these cases,
> > > > we should have normal (no-PIE) objects.
> > > >
> > > > The cases internal to gcc/glibc are exceptions, in that the
> > > > archives are used to build shared libraries (which use the
> > > > lib*_nonshared.a archives which should always be PIC anyway) and
> > > > those executables - which if we want them to be PIE of course
> > > > need PIE objects.  I think a better way to approach this sort
> > > > of case is to modify the build process explicitly to build PIE
> > > > objects where needed, rather than using the altered defaults.
> > >
> > > there are many exceptions, but you do not see them with a
> > > glibc/gnu system. On an uClibc hardened system, if configure
> > > fails to detect correctly the system (needing
> > > gnuconfig-update/libtoolize), it will probably build only lib*.a
> > > instead of both lib*.so/lib*.a, and the next app compiled against
> > > lib*.a will have text relocations.
> >
> > I was not aware of that.  I would have expected embedded systems to
> > be keen on shared libraries, as they reduce physical memory
> > requirements. Would it be better to have autotools working properly
> > with uclibc?  How many packages are we talking about?  Would it be
> > reasonable to suggest they be modified to build shared libraries
> > properly (perhaps by directing configure explicitly)?
>
> autotools works mostly correct now for uClibc and many patches were
> added to solve all these issues if you make use of autoconfig
> option/gnuconfig_update/(e)libtoolize, so the tools are provided by
> gentoo, it depends only on the devs using them properly

ok; great.  Best thing here is to bug where devs are using autotools
badly - that's of benefit to everyone.


> > > > > > Strictly speaking, on a hardened system, the -vanilla and
> > > > > > -hardenednopie compiler cannot be guaranteed to generate
> > > > > > working executables - indeed it creates garbage on sparc
> > > > > > (x86 seems to be more forgiving; you can mix PIE objects
> > > > > > into a non-PIE executable and they appear to work).
> > > > >
> > > > > because lib*.a has to be PIC.
> > > >
> > > > Like I say, whether they should be normal, PIC or PIE depends on
> > > > what they're used for - and in general you just don't know.  The
> > > > most common use for lib*.a files from the system (/usr/lib etc)
> > > > is to build static binaries.
> > > >
> > > > Consider also the state of the crt*.o objects - on existing
> > > > hardened systems crtbegin.o and friends are built -fPIE.  This
> > > > makes the
> > >
> > > no, crt*S.o are compiled -fPIC at least on linux, see
> > > gcc/config/t-linux CRTSTUFF_T_CFLAGS_S, only those without S are
> > > compiled -fPIE (if gcc defaults to hardened and ... and ...,
> > > depending on the version of the piepatches)
> >
> > Yes.  It was just the non-S versions I was talking about
> > (crtbegin.o, crtend.o and friends, crt1.o from libc).
> >
> > > > switching of startfile/endfile in the specs achieve almost
> > > > nothing
> > > > - on x86, crtend.o and crtendS.o are identical, and
> > > > crtbegin.o/crtbeginS.o are almost the same.  Have we really
> > > > analysed the source for those objects, to verify that building
> > > > them -fPIE gives sensible results?  I suspect not.
> > >
> > > maybe we should add -fno-PIE to crtbegin.o/crtend.o/crtbeginT.o to
> > > guard against the above case.
> >
> > I did try that, also building crt1.o (libc) with -fno-PIE.  The
> > modifications to the toolchain build files got more and more
> > involved, which I really want to avoid.  In addition, I suspect
> > that will break your "PIE/static" binaries; where perhaps it would
> > be better to use crt*S.o/Scrt1.o.
>
> I wanted some time ago to add some corrections into gcc upstream due
> to incorrect handling of HAVE_LD_PIE within the generation of specs,
> crtbegin/crtend related (the issue was, that if binutils does not
> support -pie, then it does not make sense to use Scrt*.o files to
> link) The collection of the proposed patches is in
> piepatches/upstream subdir. They were not accepted upstream, the
> comment from Jakub Jelinek was something like:
> it does not matter to use Scrt*.o files if no pie linking is done, it
> adds only 1-2 cycles more.
> so living with the 1-2 cycles performance loss, we could always use
> Scrt* files on a hardened box.

Yeah.  I think I have this working; doing it an easy way by patching
the makefiles directly when USE=hardened (instead of trying to modify
the whole make process to do it perfectly - which is hard, and a waste
of time, really).


> > > > > > I'm working on a sane solution to this problem using ABIs,
> > > > > > but it's quite involved, and it's quite a departure from
> > > > > > what we currently do. Once I have my thoughts clear, I'll
> > > > > > post a proposal.
> > > > > >
> > > > > > For the moment, however, we can continue as we did before,
> > > > > > but highlight that -vanilla and -hardenednopie may not
> > > > > > generate working executables. Mostly this affects
> > > > > > bootloaders, and things like static busybox.
> > > > >
> > > > > Wondering why you mention static binaries,
> > > >
> > > > Static binaries should not be PIE - the two concepts don't
> > > > mix.  It doesn't make sense to build an ET_EXEC executable
> > > > (which is what static executables are) with PIE code.
> > >
> > > sadly, when compiling we do not know if the object will go into
> > > static/ET_EXEC/ET_DYN, else we could handle that (we handle only
> > > the case if the final binary is compiled and linked in one step)
> >
> > Yep.  The only way I can think of to deal with that would be a
> > normal/PIE multilib setup.  For glibc systems it is probably not
> > worth the trouble (since we rarely use the lib*.a archives
> > anyway).  For uclibc, it would depend how much breakage there is in
> > building shared libraries.
>
> well, my overlay is almost 30MB big, the most of it solves these
> issues, -z defs/-z now problems, probably needs a cleanup, but I
> expect about 15MB will stay...
> but you know, I am crazy, I do use uClibc on my work/development box
> natively, so I use probably much more of the apps, that a normal
> gentoo-embedded box wouldnt need ;)
>
> > > > > I have no problem on a
> > > > > clean hardened box with static binaries,
> > > >
> > > > What you have aren't really static binaries - they're ET_EXECs
> > > > with PIE objects inside. This kinda works on x86, but on sparc
> > > > it just segfaults on startup (due to the contents of crt*.o).
> > >
> > > I know ppc/arm does not have any problem with it (mips is out of
> > > question because it is always PIC except the kernel)
> > >
> > > give fno-PIE a try for those, maybe it solves the issue.
> >
> > I did try that, I think, but without any success.
> >
> >
> > > > > even if the lib*.a are PIC
> > > > > compiled (mine are for sure, and most of you have it too, if
> > > > > an app is using autotools and running ./configure shows that
> > > > > the system supports shared libraries and for "dependant
> > > > > libraries" it says "pass_all"), I though agree that it is
> > > > > overkill for static binaries to use libraries that were PIC
> > > > > compiled.
> > > >
> > > > The problem with static binaries is that they're not correct on
> > > > all platforms, when built with PIE objects.  sparc in particular
> > > > segfaults.  It does create executables that run on x86, but
> > > > they're a bizarre PIE/ET_EXEC hybrid that makes little sense.
> > >
> > > sparc or sparc64 or both? have you compared assembler output?
> > > does it make a difference if you provide -K PIC to the assembler
> > > or not?
> >
> > sparc64 (sparcv9).  The problem was symbols not being resolved
> > properly, iirc.  I don't remember exactly what I tried, now - I
> > tried many things, although not -K PIC (I would assume that would
> > be passed to the assembler when building -fPIE or -fPIC).
>
> I am not sure about -K PIC usage, it is true, that if you use
> -fPIE|-fPIC on the command-line, it will add for sparc -K PIC.
> the scenario I am thinking of:
> a hardened-pie specs will enforce -K PIC/-fPIE on all objects with
> some exceptions. could it be. I wonder if the assembler wouldnt
> autoget -K PIC, but the objects are compiled with -fPIE, would the
> object be usable on sparc in a static executable?
> To do such a test you have to disable the autoadding -K PIC in specs
> (asm_pie section)

Haven't tested yet (getting pieworld right first).  It seems '-K PIC'
is an undocumented feature of ld on sparc - indeed it means the
documented '-K' option fails to parse (!).  I thought it was just a
compatibility thing for the Sun 'as' (which is often used in preference
to GNU gas on Solaris) where -K PIC is meaningful, but it looks like
it's more than that.

-- 
Kevin F. Quinn