Re: [gentoo-portage-dev] Re: mkinitrd

[Ned Ludd]

On Wed, 2004-10-27 at 18:43, Ned Ludd wrote:
> On Wed, 2004-10-27 at 15:20, Peter S. Mazinger wrote:
> > On Wed, 27 Oct 2004, Ned Ludd wrote:
> >
> > > I've been thinking about initrd and I'd like to move down that path at some point for my own use.
> > > Perhaps kernel+busybox for the initd then another partition for packages.
> > > I'd like to go right from building packages on my own system to dishing (http:// || ftp:// ) them up from my $PKGDIR and installing right to the device when it's mounted +rw.
> > > Problem I see however with the idea now is that the native portage format is a tad bloated (noman|nodoc|noinfo) all run at the installing phase vs the before packages phase which would force me to put a strip on the runtime device to keep packages small enough.
> > > So my first goal is to get the dev-portage guys talking with me about redefining/extending the portage binary format a little.
> > >
> > > hrmm I guess I'll subscribe to gentoo-portage-dev-subscribe@g.o
> > > and spark up a conversation.
> >
> > Are you thinking of adding subpackaging and/or foreign format support,
> > like ipkg?

Here is a mocked up version to do what I(we?) need for embedded devices.
while trying to cope with the existing portage binary format.

uCpie etc # cat make.conf 
PKGDIR=/dev/shm/portage/packages/
PORTAGE_TMPDIR=/dev/shm/portage/tmp
PORTAGE_BINHOST=http://oc12.net/~solar/uclibc/gentoo/x86/packages/world/
INDEX_DIR=$PKGDIR
FEATURES="noman noinfo nodoc distclean"
INSTALL_MASK="*.a *.o *.h *.pl *.sample /usr/include /etc/init.d /etc/conf.d /usr/share/locale/"
INSTALL_MASK="${INSTALL_MASK} /usr/share/gcc-data/"

As we can see there is no easy way to overcome the (pre|post)inst functions if we pay attention to the flow of the script.
If we had stripping and no{man,doc,info} and INSTALL_MASK handled by portage before the  dyn_package() and a logical way to parse the extra metadata that a tbz2 provides then we could be in much better shape. 
One major advantage to a new binary format for us to not have to incur thr extra overhead from having to provide a strip util at runtime. You know now that I think of it the whole stripping setup in prepall/prepstrip phase is wrong.. So this is a bug..

Anybody from the portage team have anything in mind with portage's binary handling anytime soon? 
Extended it? Expanding it? Roadblocking changes to it?

>
> I have very simple ways todo ipkg format now that I can share with you if you want to deploy it now.
> But the topic of binary format handling is a hot topic (but nobody is really talking about it)
> Guess I'll CC:  this to gentoo-portage-dev as my first post.
>
> Right now portage only handles the .tbz2 format. This current format poses a few problems for it to really be useful for us on embedded devices.
> The tarballs contain man files/docs/info pages (lots of bloat to us).  None of the ELF's are stripped in the package.
> Normally on our development systems we don't see these files as they don't get installed to the runtime system because FEATURES="noman noinfo nodoc" are set and this is handled in the dyn_install phase. So in order to get the desired behaviors inside we want it's almost like we need to do.
> emerge pkg ; quickpkg pkg vs emerge -B package
>
> Another problem that exists for us is that embedded systems usually never have python/perl/(your fav scripting lang here) installed. Without python installed I see no easy way to get at the additional package metadata thats joined into the .tbz2
> This metadata is what holds the basic info like (what arch is this package for. USE/IUSE etc..) but even half of that is sealed up in a file called environment.bz2 so we have to decompress & extract the entire .tbz2 then parse (not source) the decompressed 'environment' file just to find out if a package is for our ARCH or not.
> This is by no means very efficient.
> Also the INSTALL_MASK feature needs work ( http://bugs.gentoo.org/show_bug.cgi?id=67190 )
>
> Now to answer your orig question.
> ipkg format. This is sorta of a yes & no thing.  In order for portage to better support more formats it should have abstraction layers between diff formats.
> So first we must decide what formats we want to support.
> I'd think out of the gate we should support the most primitive of these pkg formats which is the classic .tar.gz (.tgz) format for the actual file system data.
>
> Note to self:
> Look at source code for /usr/bin/tbz2tool
> (busybox emerge binary applet?)
>
> >
> > Peter
-- 
Ned Ludd <solar@g.o>
Gentoo (hardened,security,infrastructure,embedded) Developer