Re: [gentoo-catalyst] Couple of questions

[Mikey]

On Thursday 15 December 2005 09:58, Eric Edgar spammed:

> You also have to concider that you were doing things incorrectly.  We
> only have so much time in the day and have to go after the low hanging
> fruit first per say.  While you are correct that something is wrong in
> the tree the initial results did not indicate that.

It took me a few iterations to understand how the tool is _supposed_ to be 
used.  At no place in the documentation does it clearly explain that the 
syntax of stage4/use: does not migrate to stage2/use:, for example, or that 
USE flags are NOT to be used anywhere but stage4, or that USE flags are not 
to go into the envscript.  There is one generic stage example, which has 
been subsequently changed to explain that cflags and such only apply on 
stage2, but no mention of use flags and so on.  I assumed that:

"The catalyst tool is intended to be used by those who wish to create their 
own customized versions of Gentoo Linux, or their own customized LiveCDs. 
Our goal is to make catalyst a powerful tool that's a pleasure to use, and 
to ensure that the code we write is maintainable and of high-quality."

..meant that the tool was intended to be used to create customized versions, 
what I didn't understand was the customization only applies in very limited 
ways to stage4...

So I have had to learn the hard way and have had to bug the hell out of you 
guys.  I would be glad to work on some of the documentation after wrapping 
my mind around it more completely :)

> put out a new gentoo cd.  There are only 2 main developers working on
> catalyst at the moment, Myself and wolf31o2.

Geez, no wonder the fuse is kinda short.  I had no idea it was only you two.

> from my standpoint I have seen you as quite firm in your convictions
> that you were right.  I agree that the stage1 bug you posted is valid,
>      but you could have also performed the few requested steps to follow
>      wolf31o2's suggestions.

After making a few boneheaded mistakes I did follow Chris's suggestions, and 
arrived at the same results.  I also understood that the actual problem is 
not caused by those mistakes.  Stage1 is the only stage that installs into 
a separate ROOT as far as I can tell.  Catalyst is not taking into account 
the order that portage.grab_stacked() and portage.stack_lists() is 
returning package lists in.  Glibc requires headers to be installed first.  
The solution is either to hack portage to always sort lists that involve 
the toolchain correctly, or to account for it in catalyst.  I have never 
coded in python, so I don't quite know how to implement the solution, but I 
do have a guess at what the solution is.  Push any toolchain package to the 
top (or bottom, whichever) of the buildpkgs array in their proper order, 
don't rely on portage.grab_stacked() or portage.stack_lists() to order it 
correctly because it does not gracefully take into account the install is 
happening in a different ROOT.  The DEPEND check in the glibc ebuild is 
looking for headers in the running system, not what is installed in ROOT, 
so it does not see it as a dependency that needs to be installed first.  
<rant>As an anal purist, I believe the toolchain should be installed first 
and in the proper order explicitly one package at a time instead of in a 
stacked list.  To do otherwise is heaping problems on yourself on down the 
road if you ask me.  You also avoid this weird dependency problem when 
installing into a ROOT.</rant>


> Making a new profile is really very easy to do and with catalysts
> portage_overlay features quite painless to do.  It works just like an
> overlay on a host gentoo system.  I can understand that it might have
> sounded difficult or time consuming to follow the recommendation but a
> better course of action from you would have been how can I do this .. or
> how do you guys do this.  If it sounds like it sucks for you maybe it
> sucks for us too, but we just havent had the time to code a simpler way
> yet.  We are not out to make anyones life difficult, remember we are
> volunteers here with limited time.

Can you put profiles in overlays?  Duh, that would be much easier.

> you are also missing the fact that the profile stays consistent from one
> stage to the next.  custom use flags in a spec file can and will very
> easily be missed or dropped or typoed etc.  It really is a better design
> choice to do it with profiles.

Call this a feature request, or a suggestion.

Allow cflags, chosts, cxxflags, FEATURES, and use flags in only ONE place - 
stage2.  Stage2 can generate the entire make.conf that will be used through 
any future stages, including the user specified USE flags. Stage2 runs the 
bootstrap.sh, which correctly throws out non-applicable USE flags, so it is 
not a problem, all the catalyst developers would have to do is count on 
bootstrap.sh to work correctly.  

From that point forward every stage done by catalyst uses the USE flags 
generated in step 2 and never changes.  If a USE flag breaks the install, 
it would have broken during the install in stage4 anyway.  This would 
greatly simplify everything in my opinion.

Stage1 would continue to be the proper place to sanitize the build to 
pre-selected cflags and use flags, etc...

Am I crazy, or what?

> You assume that its a simple thing to compile a tool chain in order.
> But what you miss is the fact that upstream developers consistently
> change how things are done.  We have coded catalyst the best way for us.
> Using profiles allows us to leverage other developers time in the tree
> to catch a majority of the issues for us.  They can help reorganize the
> way things are built if necessary via a profile.  Otherwise we would be
> constantly updating our spec files and never know exactly what was going
> on.

It _is_ a simple thing to bootstrap a toolchain in order.  The order has 
always been the exact same and probably always will be.  binutils, gcc, 
headers, glibc.  Repeat, using tools from previous step.  This is the 
_only_ way as far as I know to do it correctly, and to do it correctly 
every time, and to be absolutely certain it has been done correctly.  It 
has been this way for over 10 years if I recall correctly, it is linux 
system building 101.