Re: [gentoo-catalyst] Couple of questions

[Eric Edgar]

On 08:47 Thu 15 Dec     , Mikey wrote:
> On Thursday 15 December 2005 07:34, Chris Gianelloni spammed:
>
> > It is a bug in the snapshot/tree, as I stated in bug #115445.  I'm
> > almost getting tired of explaining things seeing as how I have to do it
> > multiple times.  Rather than me repeating myself, please read each
> > message a few times instead? ;P
>
> > know will not work as expected.  We are not stupid.  We have been
> > working with portage and Gentoo for a long time and know what is and is
> > not possible quite well by now.
>
> As I am tired of repeating over and over that something is wrong with the
> stage1 target and all you seem to be able to do is blame me.  It only took
> me 5 freaking times to explain something was wrong.
You also have to concider that you were doing things incorrectly.  We
only have so much time in the day and have to go after the low hanging
fruit first per say.  While you are correct that something is wrong in
the tree the initial results did not indicate that.
> So hey, you are not
> the only one who gets tired of repeating shit to thick headed people.  My
> snapshot version was right there in the spec files I posted, a simple 20
> minute test on your part would have confirmed the bug.
For you it might be a simple 20 minute test.  For us it takes much
longer.  We are working on getting catalyst ready to deploy a release.
Catalyst is the tool that allows us to do this .. that is its primary
purpose from our perspective.  While we do accept patches and generally
take the view point that catalyst should be as generic of a tool as
possible we have to shift gears into release mode when it comes time to
put out a new gentoo cd.  There are only 2 main developers working on
catalyst at the moment, Myself and wolf31o2.  

We are fully aware that there are bugs and inconsistencies and warts and
who knows what else.  Thats what happens when you have numerous
developers working in the tree.  I dont want to get into a political
flamewar on how it should be different.  Frankly we dont have the time.
We are developers who are donating our time to try and do things that
interest us.  

I am a catalyst developer now because the concept interested me.  I
started hacking on it to make it better.  I know its not perfect but its
good enough right now for its stated purpose of releasing gentoo install
media.  I am working on generalizing things and if you look at the code
between catalyst1 and catalyst2 you will see significant changes that
include additional error checks and user warnings for the most common
issues we see.
> Apparently you
> portage people are not as smart as you would like me to believe, portage
> has its share of bugs and rarely behaves consistently.  I have been
> compiling linux from source for over 12 years, I happen to know a little
> bit about what I am talking about as well.  Probably not as much as you
> know about gentoo, but hey, I can recognize an obvious glaring problem when
> I see it.
from my standpoint I have seen you as quite firm in your convictions
that you were right.  I agree that the stage1 bug you posted is valid,
     but you could have also performed the few requested steps to follow
     wolf31o2's suggestions.  

Catalyst is designed to be as repeatable as possible.  We havent found
every issue yet and are always looking at smoothing those bumps over.  
Making a new profile is really very easy to do and with catalysts
portage_overlay features quite painless to do.  It works just like an
overlay on a host gentoo system.  I can understand that it might have
sounded difficult or time consuming to follow the recommendation but a
better course of action from you would have been how can I do this .. or
how do you guys do this.  If it sounds like it sucks for you maybe it
sucks for us too, but we just havent had the time to code a simpler way
yet.  We are not out to make anyones life difficult, remember we are
volunteers here with limited time.
> If you would have looked at what I posted in my bug report you would find a
> myriad of bugs.  Catalyst let me do things it is not supposed to do, like
> use a subarch during stage1 when it shouldn't.  Emerge told you it was
> going to install packages in a certain order, yet it didn't (not your bug).
> For some reason, baselayout was missing from the packages that were going
> to be installed, even though catalyst asked for it to be installed.  You
> indicate that usage of an envscript in catalyst.conf is normal usage, but
> fail to inform the user NOT to use USE flags in there or it will break
> something (that should be handled by catalyst).  You tell me catalyst is
> designed to not allow customization in the lower stages, then tell me the
> bug is invalid because I tried customization in lower stages.
>
> But hey, I am sure it is a winning strategy to just treat all of your
> debuggers like they are idiots.
You are not an idiot, you were just very firm in thinking the tool
should be used in X way and basically need to learn more of how the tool
works.  Suggestions were made to get you in the right direction yet it
seemed like you were not interested in following those initially.

> > Because making changes outside of the scope of the profile is broken.
> > It produces an inconsistent stage, since the USE flags passed via an
> > envscript *never* see the inside of make.conf, giving you a stage that
> > *will not* behave as you expect it to.  Why leave this option available
> > when it is *obviously* misunderstood and causes errors?  You have proven
> > here exactly *why* the option is not there.
>
> Gee, listen to what you are saying.  Using USE flags in stage2 and beyond is
> broken and produces inconsistent stages.
He did NOT say using use flags was broken.  He said using use flags via
an environment variable was broken.  The envscript is a tool to help
build the initial hardened stages for example.  It is for developers who
need additional functionality that only a small number of the developers need.
We get our use flags out of the profiles because then every one can
build a stage1 the same way.  We grab the use flags and the packages to
be built right out of portage.  So this is not something that is broken.  
If you want different use flags for example setup a new profile and use
that ..  That is how catalyst is designed to work.  It works with the
other tools such as portage instead of trying to work around them.
Obviously the tree changes and we have to deal with that .. but that has
already been discussed.
> I would say that is a major
> problem that needs to be fixed before anything else, considering it is the
> most basic functionality that sets gentoo apart from all other linux
> distributions.  Stage2 ignores improper use flags via bootstrap.sh, stage3
> should be able to handle whatever use flag you throw at it, portage itself
> should be able to handle that.  So what is the problem with letting
> catalyst generate the use flags and put them into the make.conf right along
> with the cflags, mirrors, etc?  Don't worry, it is a rhetorical question.
The problem is we already have a way to do that via profiles as I have
discussed.  Why code 2 methods to do something when we already have one
in place that works as expected for us.  Doing it this way allows us to
have very simple stage1 stage2 and stage3 spec files rather than
cluttering them up with ever changing use flags that may or may not be
missed because some developer changed something in the tree and didnt
inform us.  As it stands now a developer can change it in the tree and
update the profiles and most of the time it just works.  


> > > nls, userlocales, nptl, nptlonly, multilib I think, and runs with
> > > USE="-* build bootstrap allowed_flags", so why the restriction from
> > > using nptl and
> >
> > There is no such restriction, as stated over and over and over and over
> > again.  Use a profile to make these changes.
>
> I see, using USE flags in the profile is not broken, letting the user
> specify USE flags anywhere else is.  Encouraging users to dork around with
> the profile is a good idea, set USE flags bad idea.  Brilliant.
> Particularly when you consider the fact that you can build "safe" USE flags
> into catalyst just like you build in "safe" cflags.  Which, by the way, are
> deprecated for gcc 3.4.4...
>
> > No, you're being dense, rather.  I have explained it.  You just seem to
> > dislike my answer.  Stages 2 and 3 must match the profile or they will
> > not install properly in all cases.  Period.

you are also missing the fact that the profile stays consistent from one
stage to the next.  custom use flags in a spec file can and will very
easily be missed or dropped or typoed etc.  It really is a better design
choice to do it with profiles.


> Then something needs to be fixed, because the underlying process is broken.
> It needs to be the USE flags that match, it shouldn't matter where they
> come from.  Catalyst is not producing consistent results right now in spite
> of locking it down to the profile.  Locking catalyst down to extremely
> narrow functionality because portage might break at any moment is a shame,
> because it is a great tool.
>
> But I just don't buy your explanation, USE flags are calculated the same way
> regardless of where you put them, in the profile or the environment.  The
> precedence might change, but they are still calculated the same way.  If I
> create a profile and take out nls, catalyst should handle it the exact same
> way as if I stuck it in the environment....
>
> > Then I wouldn't accept them.  Catalyst is modular.  You're more than
> > welcome to replace any of the scripts to do things the way you want
>
> I don't think it is catalyst that is broken, but catalyst can be used to
> insure consistent results.  Simple things like install the toolchain in the
> proper order instead of using multi-package emerge statements that might or
> might not work correctly depending on what day of the week it is...
>
You assume that its a simple thing to compile a tool chain in order.
But what you miss is the fact that upstream developers consistently
change how things are done.  We have coded catalyst the best way for us.
Using profiles allows us to leverage other developers time in the tree
to catch a majority of the issues for us.  They can help reorganize the
way things are built if necessary via a profile.  Otherwise we would be
constantly updating our spec files and never know exactly what was going
on.


> > them.  Just don't come asking us for help when you do things that we
> > know will not work as expected.  We are not stupid.  We have been
> > working with portage and Gentoo for a long time and know what is and is
> > not possible quite well by now.
>
> Did I say you were stupid?  And after this little experience, _I_ would be
> the stupid one for asking for help, as long as this is the response I get.

Rocket