Re: [gentoo-embedded] Re: uclibc base system

[Ned Ludd]

Peter,

Ok the only remaining things now are uClibc itself and any package that
conditionally has a PROVIDE=. For the ones that conditionally provide a
PROVIDE= I assume we can just leave them out of the profile in the first
place (not sure about the nocompress one till uclibc works).

Next the uClibc.
I had to look around a bit but found the missing do_rem patch on the
uClibc mailing list, after getting that one in I went to merge the
uClibc on a hardened system and we failed with misc ssp errors. 

Attempted to drop the uclibc-patches tarball and compile without any of
those misc patches as they all seem pie-ssp related or unneeded cruft
that really does not apply to uClibc at all like -z relro.

USE="-*" ebuild uclibc-0.9.26-r1.ebuild clean unpack compile
....

*
* uClibc development/debugging options
*
Build uClibc with debugging symbols (DODEBUG) [N/y/?] n
Build uClibc with run-time assertion testing (DOASSERTS) [N/y/?] n
Build the shared library loader with debugging support
(SUPPORT_LD_DEBUG) [N/y/?] n
Build the shared library loader with early debugging support
(SUPPORT_LD_DEBUG_EARLY) [N/y/?] n
Manuel's hidden warnings (UCLIBC_MJN3_ONLY) [N/y/?] n
+ ./extra/scripts/fix_includes.sh -k /usr -t i386


The file /usr/Makefile is missing!
Perhaps your kernel source is broken?
----------------------------------------------------

Next try with arch=arm (thanks spanky)

Tested on an arm glibc system that's completely non hardened and
encountered more or less the same ssp problem.

USE="-*" CFLAGS="-fno-stack-protector"
TARGET_CFLAGS="-fno-stack-protector" 
DISTDIR=/home/solar/overlay/distfiles/
PORTDIR_OVERLAY=/home/solar/overlay/ ebuild uclibc-0.9.26-r2.ebuild
clean unpack compile
And we fail with ldso errors.


Anyway if you care to take another stab at the uclibc ebuild I'd be more
than happy to test it on some arches and commit it when it's ready.


On Tue, 2004-06-15 at 11:51, Ned Ludd wrote:
> I've mirrored two more of the files you have sent me to the following
> location so others can get to them.
> http://dev.gentoo.org/~solar/uclibc/peter_mirror/uClibc-0.9.26-cvs-update-20040613.patch.bz2
> http://dev.gentoo.org/~solar/uclibc/peter_mirror/uClibc-0.9.26-patches-1.0.tar.bz2
>
> I've merged a small portion of the app-arch -> sys-apps
> .ebuilds+uclibc/nls diffs last night till I about passed out.
>
> Saving binutils/gcc/uclibc for last.
> All the .ebuilds with use uclibc &&|| in the global context or requiring
> changes to virtual/* or PROVIDE= will likely be the ones that will take
> me/us longer to get in. I want ask SpanKY/vapier to QA those parts.
>
> On Tue, 2004-06-15 at 09:13, Peter S. Mazinger wrote:
> > On 15 Jun 2004, Ned Ludd wrote:
> >
> > > Quite impressive Peter.
> > > I have mirrored your files to
> > > http://dev.gentoo.org/~solar/uclibc/peter_mirror/portage-uclibc-overlay-20040614.tar.bz2
> > > and exploded the tarball to
> > > http://dev.gentoo.org/~solar/uclibc/peter_mirror/portage-uclibc/
> > > then diffed out the .org files and the .ebuilds the ebuild's patch is
> > > here
> > > http://dev.gentoo.org/~solar/uclibc/peter_mirror/portage-uclibc-ebuilds-20040614.patch
> >
> > this is what I really meant, so others can check what changed
> >
> > > and the profile/script data is here
> >
> > the script data is yet untested, I have only removed glibc reference from
> > there
> >
> > > http://dev.gentoo.org/~solar/uclibc/peter_mirror/org-uclibc-20040614.patch
> > > This will be quite a bit of an undertaking I'm hoping mutex, dragonheat
> > > can help with some of these commits.
> > >
> > > How may megs is your resulting stage/images after the initial bootstrap
> > > process?
> >
> > I can't really tell, I do not have managed to build stages (any help
> > appreciated how to do it from tbz2 files), and my env. has left over files
> > from my earlier rpms (wouldn't be relevant if counted)
> >
> > I can tell that the packages/All directory is 58MB (for emerge system) +
> > ccache, catalyst
> >
> > bigger than 1MB are kbd (the keyboard files are next candidates to strip
> > down),miscfiles(although stripped, gzipped), ncurses (although not so
> > many terminfo files, and no additional libs, like menu,panel,form), db4,
> > automake
> > bigger than 2MB are libperl, openssl
> > bigger than 3MB are binutils
> > bigger than 4MB are python
> > bigger than 12MB gcc, perl (13MB)
> >
> > Is there some way to query portage to tell how much the installed stuff
> > is?
> >
> > I haven't checked how much of this is man-pages and info-files, if the
> > binaries are really stripped all of them where possible.
> >
> > I have attached 2 missing files from distfiles (for uClibc)
> >
> > Busybox is not used at all yet.
> >
> > There are some things that have to be decide:
> > 1. will gcc get a c++ use flag?
> > 2. should groff/man/man-pages/info/install-info be in a stage3
> > 3. should ncurses include the full stuff (all libs)
> > 4. I would remove all the *.so handling by scripts, if they are installed
> > in /lib, they really only should be installed directly into /usr/lib.
> > 5. what to do w/ perl (mini/micro-perl are alternatives for the build
> > system (autotools should work w/ it) but not for a full featured one, no
> > support for addons)
> > 6. gettext: as I already said, I would put the *.m4 files into autotools
> > and remove gettext from the stages
> > 7. locale/nls support: the current only usable variant is to have uClibc
> > w/o locale support, and use libintl.{a,h,so} from gettext.
> >
> > Peter
> >
> > > I'm CC: the hardened mailing list as others there may have an interest
> > > in your work as this uses the hardened profile and all :)
> > >
> > > On Mon, 2004-06-14 at 19:25, Peter S. Mazinger wrote:
> > > > Hello!
> > > >
> > > > This is the overlay directory I used parallel to portage (it has to be
> > > > there for now, else the included links won't work), that allowed me to
> > > > build gentoo fully uclibc based (starting from a buildroot config,
> > > > building manually python/portage, running emerge sync ...)
> > > >
> > > > 1. the files directories have only new files and links to the originally
> > > > used (for x86), the digest/Manifest files were needed to rebuild fully
> > > > with these configs as an overlay directory, the links because portage
> > > > can't handle "properly (my opinion)" the overlay directory
> > > >
> > > > 2. the ebuilds can be diffed to the corresponding version (as of emerge
> > > > sync 20040613) to see what I have done
> > > >
> > > > 3. some of the changes are not directly uclibc related, they correct
> > > > typos etc. in the originals, add support to build w/o nls, or strip down
> > > > the package somewhat
> > > >
> > > > 4. the directories profiles, scripts include the original version (*.org)
> > > > of files too, the new ones have to be copied over the original tree, the
> > > > overlay support does not allow to have these files at another location.
> > > >
> > > > 5. distfiles include new patches for binutils-2.14.90/15.91 and gcc-3.3.3
> > > > (these have to be copied to the main distfiles, because again the overlay
> > > > structure does not support it in another location)
> > > >
> > > > 6. I haven't tried yet cascaded profiles, the only profile tested is what
> > > > I delivered.
> > > >
> > > > 7. it builds as it is (haven't tried w/ nls, and that is not really
> > > > correct in uclibc yet), don't enable nls for now
> > > >
> > > > 8. stage building and bootstraping was not tested, because I didn't find
> > > > an "elegant" way to make a stage1/2/3 from .tbz2 files (any help
> > > > appreciated, then I could also provide a stage1)
> > > >
> > > > 9. for now gettext, yacc (replaced by bison -y), ncompress
> > > > (uncompress replaced by gzip), bc, bin86, groff, man[-pages] are not a
> > > > part of an 'emerge system', cracklib got support for gzipped files (so
> > > > miscfiles is much smaller), w/o groff and man-pages it is not a
> > > > requirement to have c++ compiler either (this is not implemented, should
> > > > probably be a flag in gcc, like f77, objc), gnuconfig_update is only
> > > > needed where configure is run directly, not by econf (econf is hacked to
> > > > provide the same functionality, as gnuconfig_update), ncurses does not
> > > > deliver the addon libraries (menu,panel,form). Some told me that gettext
> > > > can't be removed, else autotools won't run, well I think, the .m4 from
> > > > gettext could be added to autotools, and than it should be no problem w/o
> > > > it.
> > > >
> > > > 10. added also my make.conf and package.keywords, to show which versions
> > > > where used, the most is stable stuff, but some have to be ~x86.
> > > >
> > > > 11. mainly the shared libs will have problems, to add support for new
> > > > libs, look at the libtool patches (ltconfig-uclibc for older configures
> > > > and libtool-1.4.3-uclibc for newer ones)
> > > >
> > > > 12. be aware that you have to build the buildroot w/ the same config (and
> > > > patches), as deduced from the uclibc.ebuild (using in both places the
> > > > same cvs too). Do not start from uclibc-0.9.26 stable, because it is not
> > > > binary compatible w/ the current cvs.
> > > >
> > > > 13. hardened stuff: gcc uses pie and ssp, but relro/now are disabled,
> > > > relro is also completely removed from binutils, uclibc does not have
> > > > support for it (any volunteer to add this to the uclibc's ldso?)
> > > >
> > > > 14. CHOST has to be set to *linux-uclibc (not linux-gnu)
> > > >
> > > > Peter
> > >
-- 
Ned Ludd <solar@g.o>
Gentoo (hardened,security,infrastructure,embedded,toolchain) Developer