Re: [gentoo-embedded] Re: uclibc base system

[Ned Ludd]

On Tue, 2004-06-22 at 07:41, Peter S. Mazinger wrote:
> On 21 Jun 2004, Ned Ludd wrote:
>
> > Peter,
> >
> > Ok the only remaining things now are uClibc itself and any package that
> > conditionally has a PROVIDE=. For the ones that conditionally provide a
> > PROVIDE= I assume we can just leave them out of the profile in the first
> > place (not sure about the nocompress one till uclibc works).
>
> could be left out, the only that we really need (but can be replaced by a
> versioning condition is gnuconfig-uclibc)
>
> >
> > Next the uClibc.
> > I had to look around a bit but found the missing do_rem patch on the
> > uClibc mailing list, after getting that one in I went to merge the
> > uClibc on a hardened system and we failed with misc ssp errors.
>
> sorry, if I have forgotten to send it to you
No biggie. 

>
> >
> > Attempted to drop the uclibc-patches tarball and compile without any of
> > those misc patches as they all seem pie-ssp related or unneeded cruft
> > that really does not apply to uClibc at all like -z relro.
>
> I do not use the relro/now patches, the ebuild removes them for now (they
> are only for completeness)

After talking to mjn3 I don't think relro will even be considered for
uClibc till some time after 1.0

>
> >
> > USE="-*" ebuild uclibc-0.9.26-r1.ebuild clean unpack compile
>
> have you done this in a buildroot, or on a glibc portage based system?

glibc.. Has to be glibc at first as there exists no seed stage yet with
supporting portage shared objects.
Point being is that the uclibc.ebuild is a no go.. I can't merge
something that fails to test properly. If it fails on a glibc system
then it will never be accepted into portage. uClibc (old wrapper style)
is already used by misc gentoo projects for PXE netbooting etc.

> I do not support any glibc system, only uclibc based (cross-compiling and
> so on should be left out),

> also nls should be disabled (as I said nls the
> only usable way would be to have uclibc w/o locale and get libintl.* files
> from gettext.

Yeah I encountered the nls thing on the arm at first and then opted to
USE="-*"



>
> > ....
> >
> > *
> > * uClibc development/debugging options
> > *
> > Build uClibc with debugging symbols (DODEBUG) [N/y/?] n
> > Build uClibc with run-time assertion testing (DOASSERTS) [N/y/?] n
> > Build the shared library loader with debugging support
> > (SUPPORT_LD_DEBUG) [N/y/?] n
> > Build the shared library loader with early debugging support
> > (SUPPORT_LD_DEBUG_EARLY) [N/y/?] n
> > Manuel's hidden warnings (UCLIBC_MJN3_ONLY) [N/y/?] n
> > + ./extra/scripts/fix_includes.sh -k /usr -t i386
> >
> >
> > The file /usr/Makefile is missing!
>
> you need kernel-headers installed in /usr/include and my Makefile patch
>
> > Perhaps your kernel source is broken?
> > ----------------------------------------------------
> >
> > Next try with arch=arm (thanks spanky)
> >
> > Tested on an arm glibc system that's completely non hardened and
> > encountered more or less the same ssp problem.
> >
> > USE="-*" CFLAGS="-fno-stack-protector"
> > TARGET_CFLAGS="-fno-stack-protector"
> > DISTDIR=/home/solar/overlay/distfiles/
> > PORTDIR_OVERLAY=/home/solar/overlay/ ebuild uclibc-0.9.26-r2.ebuild
> > clean unpack compile
> > And we fail with ldso errors.
>
> it's normal, because you haven't applied the ssp patches, that add
> -fno-stack-protector[-all] to ldso and libc build makefiles
>
> >
> >
> > Anyway if you care to take another stab at the uclibc ebuild I'd be more
> > than happy to test it on some arches and commit it when it's ready.
>
> maybe I can upload tomorrow my tbz2 files, so anybody could start from a
> "clean" uclibc env.
>
> Peter
>
> > On Tue, 2004-06-15 at 11:51, Ned Ludd wrote:
> > > I've mirrored two more of the files you have sent me to the following
> > > location so others can get to them.
> > > http://dev.gentoo.org/~solar/uclibc/peter_mirror/uClibc-0.9.26-cvs-update-20040613.patch.bz2
> > > http://dev.gentoo.org/~solar/uclibc/peter_mirror/uClibc-0.9.26-patches-1.0.tar.bz2
> > >
> > > I've merged a small portion of the app-arch -> sys-apps
> > > .ebuilds+uclibc/nls diffs last night till I about passed out.
> > >
> > > Saving binutils/gcc/uclibc for last.
> > > All the .ebuilds with use uclibc &&|| in the global context or requiring
> > > changes to virtual/* or PROVIDE= will likely be the ones that will take
> > > me/us longer to get in. I want ask SpanKY/vapier to QA those parts.
> > >
> > > On Tue, 2004-06-15 at 09:13, Peter S. Mazinger wrote:
> > > > On 15 Jun 2004, Ned Ludd wrote:
> > > >
> > > > > Quite impressive Peter.
> > > > > I have mirrored your files to
> > > > > http://dev.gentoo.org/~solar/uclibc/peter_mirror/portage-uclibc-overlay-20040614.tar.bz2
> > > > > and exploded the tarball to
> > > > > http://dev.gentoo.org/~solar/uclibc/peter_mirror/portage-uclibc/
> > > > > then diffed out the .org files and the .ebuilds the ebuild's patch is
> > > > > here
> > > > > http://dev.gentoo.org/~solar/uclibc/peter_mirror/portage-uclibc-ebuilds-20040614.patch
> > > >
> > > > this is what I really meant, so others can check what changed
> > > >
> > > > > and the profile/script data is here
> > > >
> > > > the script data is yet untested, I have only removed glibc reference from
> > > > there
> > > >
> > > > > http://dev.gentoo.org/~solar/uclibc/peter_mirror/org-uclibc-20040614.patch
> > > > > This will be quite a bit of an undertaking I'm hoping mutex, dragonheat
> > > > > can help with some of these commits.
> > > > >
> > > > > How may megs is your resulting stage/images after the initial bootstrap
> > > > > process?
> > > >
> > > > I can't really tell, I do not have managed to build stages (any help
> > > > appreciated how to do it from tbz2 files), and my env. has left over files
> > > > from my earlier rpms (wouldn't be relevant if counted)
> > > >
> > > > I can tell that the packages/All directory is 58MB (for emerge system) +
> > > > ccache, catalyst
> > > >
> > > > bigger than 1MB are kbd (the keyboard files are next candidates to strip
> > > > down),miscfiles(although stripped, gzipped), ncurses (although not so
> > > > many terminfo files, and no additional libs, like menu,panel,form), db4,
> > > > automake
> > > > bigger than 2MB are libperl, openssl
> > > > bigger than 3MB are binutils
> > > > bigger than 4MB are python
> > > > bigger than 12MB gcc, perl (13MB)
> > > >
> > > > Is there some way to query portage to tell how much the installed stuff
> > > > is?
> > > >
> > > > I haven't checked how much of this is man-pages and info-files, if the
> > > > binaries are really stripped all of them where possible.
> > > >
> > > > I have attached 2 missing files from distfiles (for uClibc)
> > > >
> > > > Busybox is not used at all yet.
> > > >
> > > > There are some things that have to be decide:
> > > > 1. will gcc get a c++ use flag?
> > > > 2. should groff/man/man-pages/info/install-info be in a stage3
> > > > 3. should ncurses include the full stuff (all libs)
> > > > 4. I would remove all the *.so handling by scripts, if they are installed
> > > > in /lib, they really only should be installed directly into /usr/lib.
> > > > 5. what to do w/ perl (mini/micro-perl are alternatives for the build
> > > > system (autotools should work w/ it) but not for a full featured one, no
> > > > support for addons)
> > > > 6. gettext: as I already said, I would put the *.m4 files into autotools
> > > > and remove gettext from the stages
> > > > 7. locale/nls support: the current only usable variant is to have uClibc
> > > > w/o locale support, and use libintl.{a,h,so} from gettext.
> > > >
> > > > Peter
> > > >
> > > > > I'm CC: the hardened mailing list as others there may have an interest
> > > > > in your work as this uses the hardened profile and all :)
> > > > >
> > > > > On Mon, 2004-06-14 at 19:25, Peter S. Mazinger wrote:
> > > > > > Hello!
> > > > > >
> > > > > > This is the overlay directory I used parallel to portage (it has to be
> > > > > > there for now, else the included links won't work), that allowed me to
> > > > > > build gentoo fully uclibc based (starting from a buildroot config,
> > > > > > building manually python/portage, running emerge sync ...)
> > > > > >
> > > > > > 1. the files directories have only new files and links to the originally
> > > > > > used (for x86), the digest/Manifest files were needed to rebuild fully
> > > > > > with these configs as an overlay directory, the links because portage
> > > > > > can't handle "properly (my opinion)" the overlay directory
> > > > > >
> > > > > > 2. the ebuilds can be diffed to the corresponding version (as of emerge
> > > > > > sync 20040613) to see what I have done
> > > > > >
> > > > > > 3. some of the changes are not directly uclibc related, they correct
> > > > > > typos etc. in the originals, add support to build w/o nls, or strip down
> > > > > > the package somewhat
> > > > > >
> > > > > > 4. the directories profiles, scripts include the original version (*.org)
> > > > > > of files too, the new ones have to be copied over the original tree, the
> > > > > > overlay support does not allow to have these files at another location.
> > > > > >
> > > > > > 5. distfiles include new patches for binutils-2.14.90/15.91 and gcc-3.3.3
> > > > > > (these have to be copied to the main distfiles, because again the overlay
> > > > > > structure does not support it in another location)
> > > > > >
> > > > > > 6. I haven't tried yet cascaded profiles, the only profile tested is what
> > > > > > I delivered.
> > > > > >
> > > > > > 7. it builds as it is (haven't tried w/ nls, and that is not really
> > > > > > correct in uclibc yet), don't enable nls for now
> > > > > >
> > > > > > 8. stage building and bootstraping was not tested, because I didn't find
> > > > > > an "elegant" way to make a stage1/2/3 from .tbz2 files (any help
> > > > > > appreciated, then I could also provide a stage1)
> > > > > >
> > > > > > 9. for now gettext, yacc (replaced by bison -y), ncompress
> > > > > > (uncompress replaced by gzip), bc, bin86, groff, man[-pages] are not a
> > > > > > part of an 'emerge system', cracklib got support for gzipped files (so
> > > > > > miscfiles is much smaller), w/o groff and man-pages it is not a
> > > > > > requirement to have c++ compiler either (this is not implemented, should
> > > > > > probably be a flag in gcc, like f77, objc), gnuconfig_update is only
> > > > > > needed where configure is run directly, not by econf (econf is hacked to
> > > > > > provide the same functionality, as gnuconfig_update), ncurses does not
> > > > > > deliver the addon libraries (menu,panel,form). Some told me that gettext
> > > > > > can't be removed, else autotools won't run, well I think, the .m4 from
> > > > > > gettext could be added to autotools, and than it should be no problem w/o
> > > > > > it.
> > > > > >
> > > > > > 10. added also my make.conf and package.keywords, to show which versions
> > > > > > where used, the most is stable stuff, but some have to be ~x86.
> > > > > >
> > > > > > 11. mainly the shared libs will have problems, to add support for new
> > > > > > libs, look at the libtool patches (ltconfig-uclibc for older configures
> > > > > > and libtool-1.4.3-uclibc for newer ones)
> > > > > >
> > > > > > 12. be aware that you have to build the buildroot w/ the same config (and
> > > > > > patches), as deduced from the uclibc.ebuild (using in both places the
> > > > > > same cvs too). Do not start from uclibc-0.9.26 stable, because it is not
> > > > > > binary compatible w/ the current cvs.
> > > > > >
> > > > > > 13. hardened stuff: gcc uses pie and ssp, but relro/now are disabled,
> > > > > > relro is also completely removed from binutils, uclibc does not have
> > > > > > support for it (any volunteer to add this to the uclibc's ldso?)
> > > > > >
> > > > > > 14. CHOST has to be set to *linux-uclibc (not linux-gnu)
> > > > > >
> > > > > > Peter
> > > > >
> >
>
>
>
> --
> gentoo-embedded@g.o mailing list
-- 
Ned Ludd <solar@g.o>
Gentoo (hardened,security,infrastructure,embedded,toolchain) Developer