[gentoo-osx] Re: [RFC] Separate alt-prefix repo for base-system packages.

[Finn Thain]

On Mon, 29 Aug 2005, Grobian wrote:

> Kito wrote:
> >
> > On Aug 27, 2005, at 4:32 PM, Grobian wrote:
> >

[snip]

>
> > Do the initial import of the minimum required packages from the main
> > tree, which shouldn't be too hard, basically a stage1 (see a
> > packages.build file in one of the linux profiles for instance) give or
> > take some things.
> >
> > Create a new profile(s), which should probably be a complete tear
> > down, Finn Thain has had some great suggestions in this area. FINN!
> > Care to jump in here?
>
> ^^^ Finn ^^^ :D

Sure. The idea was this profile inheriance tree:


                                 .-> ppc-od
                 .-> ppc-darwin <
                /                `-> ppc-macos
default-darwin <
                \                .-> x86-od
                 `-> x86-darwin <
                                 `-> x86-macos


The purpose being, "progressive" and "collision-protect" are best given 
their own profiles in order to avoid having ebuilds test for the 
"collision-protect" feature.

So, I was advocating that an "upstream darwin" profile, {x86,ppc}-darwin, 
should be used for anything that didn't bow down to mac os. Hence, the 
ppc-macos profile would be taken to mean "behave as a secondary package 
manager: play nice with mac os, and test the mac os package database for 
dependencies, and use the mac os installer to install them if they are 
available in a .pkg".

At the moment, ppc-macos would need the collision-protect feature, but 
once we have a feature for prefixed installs, that should be used instead 
(by default, at least).

> > The user interface would need to be hashed out as well of course. How
> > do you install/bootstrap it?

It would be nice to have ebuilds that could invoke the Darwin Build 
Scripts (and merge the result on a ROOT).

http://www.opendarwin.org/projects/darwinbuild/releases/

Given such ebuilds, surely catalyst can bootstrap it.

> > Where is the local configuration data stored? This is an area that I
> > think would be acceptable to take some Mac OS specific indulgences,
> > such as plists for the main config data(prefix info, search paths,
> > etc), pkgs/dmgs to bootstrap/install, and I also think we should abuse
> > the umbrella Framework mechanism when feasible.
>
> Wow, using plists would be a first start on getting portage widely
> accepted because it includes the buzz word XML.  LOL.  On a serious
> note, I think Apple has shown XML can work somehow.  At least it has an
> open character, and it's great when you can 'script' your Keynote
> presentation by just doing string manipulation in a big XML file.
>
> So I would say, let's try to use this horrible XML on a pilot base for
> small configuration files.  Maybe we should do it better than Apple at
> some point because their XML does not always make use of the tree
> structure of XML.  For XML I would say: only deal with it if it looks
> appropriate for the case and it is relatively easy to extract the data
> (which is often very flat, as in the .plist files).

I reckon XML is important, though perhaps not in the way you describe. As 
I see it, where ever portage is deployed as a secondary package manager, 
it needs to consult the primary one. That means that there needs to be a 
standard protocol for one package manager to query another.

In the mac os case, we can write a shim to talk XML to portage.

(BTW, a similar protocol, but allowing one package manager to _update_ 
another could be used to make a universal binary installer. But it would 
need rpm, apt-get, portage etc to all implement a standard XML protocol 
and API. But this is only really interesting in a closed source context. 
And I've digressed. Mentioning XML just reminded me.)

> Let's indeed make it a 'native' application for OSX users.  Native in
> the sense of how it installs and looks like. I may give file locations a
> thought today, but maybe I should know the Framework stuff a bit better
> first.  Can we install the whole Gentoo stuff in a Framework? or is it
> better to just use a shortest path algorithm and end with /gentoo?
> Actually the user should be able to select a disk to install to during
> install...

I reckon get it working (with an "upstream darwin" profile) in a chroot 
stage first (which could double as a boot disk).

[snip]

> > Some random broad philosophy/design goals that I think should be
> > stated...:
> >
> >   The repo should never ever never ever EVER rely on anything it
> > doesn't know how to supply itself with, whether that be a tool, a
> > library, knowledge of a filesystem, a host, a protocol, whatever. It
> > doesn't matter how it gets it, it just needs to know at least one way
> > to get it. This implies of course proper implementation of ferringbs
> > BDEPENDS idea.
>
> so, you mean if there is something (a filesystem) portage hasn't
> installed, then we should create the proper handles to deal with the OS
> provided one?  As in create a compatibility tool for "fdisk.HFS+".  I'm
> a bit clueless on how exactly you want to achieve what you want.  Maybe
> I don't understand correctly what you want exactly too.

This is how I would handle that case:

If a program (say fsck_hfs) is available upstream, build it with Dawin 
Build, merge it with portage (I expect an eclass for darwin build is 
required, and of course an ebuild for diskdev_cmds.)

If a package (say macext2fs) is available as a .pkg or .dmg, download it 
with portage, and install it with mac os installer(8). Again, an eclass to 
handle these would be useful (this is probably how they are handled 
already?)

> >   Although we want this for ppc-macos at the moment, it should not be
> > specific to either of those things, ppc, or macos. Adhering to this
> > rule assumes a lot...again the BDEPENDS issue, and keeping as close to
> > the main tree as possible, with those things in place, and done
> > properly(i.e. what it REALLY takes to bootstrap an
> > {x86,amd_64,ppc,ppc64,mips,sh,whatever}-{linux,*bsd,darwin,macos,whatever}
> > toolchain) , you have a sane cross-compile ready repo, that is pretty
> > damn portable.

Darwin is not self-hosting (out of the box). You need a carefully 
contrived a build environment to make it so, and this is what the Darwin 
Build Scripts do: install that build environment, and automate the builds.

Darwin build uses plists to describe all the packages in a mac os release, 
along with build dependencies, and so forth. These build dependencies are 
confined to the darwin build chroot, as I understand it. Portage wouldn't 
have to know about them, I think it just has to do the merge of the build 
product and install the runtime dependencies thereof (of course).

> That would be really great.
>
> >   Binary packages have to work. Thats a fun one. But all this done
> > properly, should allow for at least a little more consistency than the
> > current situation. I'm still sold on using xar[1] for this despite the
> > rather heavy deps (they are deps I would want in most any environment
> > anyway - libcurl,ssl,libxml,zlib), and it just fits the bill too well
> > imho, support for most standard arch specific data such bsd flags,
> > ACLs, resource forks ,.etc as well an excellent TOC structure that is
> > perfect for storing environment settings and package metadata.
>
> Again XML.  If you do it XML, you have to do it all XML, something Apple
> apparently understood.  It appears we will have the blessings if we use
> XML, so I think we should.  In the end we can always dump all that XML
> into MonetDB/XQuery to have extremely fast querying over all the files,
> tree based. I think it would seriously be the first project to use
> XQuery and XML for it's configuration.  However, if you come to think of
> it, it is a tree, an extensible tree, and might be a much more a logical
> choice than it appears to be.

Why not use one of the open source .pkg tools to generate binary packages? 
Kito tells me he has already been able to unpack .pkg format and install 
it via portage.

> >   Avoid package.provided or anything of its likeness like the plague.
> > This repo needs to describe a toolchain from the ground up, regardless
> > of the host. "What CAN it build and how?", not "What WON'T the host OS
> > let me do".
>
> Uhm, yes please!

Hear, hear!

package.provided must die. In the long run, it is not sustainable to 
attempt to use profiles to describe a moving target. If you do that, you 
end up with not one profile for 10.4, but profiles for 10.4, 10.4.1 and 
10.4.2 because, for example, 10.4.2 fixed a bug in apple's xyz package 
that forced apples xyz to be masked in the 10.4.1 profile.

And then you still have the problem that the user may actually be running 
10.4.6, thanks to a recent Software Update, while the latest profile is at 
10.4.4, thanks to an old emerge --sync, all while /etc/make.profile is 
still a symlink to the 10.4.1 profile, thanks to the user neglecting to 
re-point it since installation!

> >   Keep the number of ebuilds to a bare minimum. We can't get too
> > carried away with maintaining a complete tree, or we risk drifting too
> > far downstream and the zealots pushing Darwin/Mac OS support out of
> > the main tree entirely. That would be bad. This can't go in the
> > direction of a fork, just an experimental branch that will be merged
> > back in at some point.

IMHO, this sounds like a "gentoo-darwin" sub-project to gentoo-alt, 
along-side os x and bsd. It isn't really a fork except in as much as the 
profile arrangement doesn't really accomodate work on darwin proper (but 
then the profile arrangemnet is flawed anyway: it only exists this way 
because of the package.provided crutch).

-f
-- 
gentoo-osx@g.o mailing list