Re: [gentoo-alt] Allow non-interactive bootstrap to be stopped at arbitrary stages

[Sam Pfeiffer]

Hello Benda,

On Mon, 27 Jan 2020 at 21:26, Benda Xu <heroxbd@g.o> wrote:

> Hi Sam,
>
> Sam Pfeiffer <sammypfeiffer@×××××.com> writes:
>
> > It took me a while to figure out how to do it with minimal
> > changes... but this is the new version I came up with:
> >
> > https://gist.github.com/awesomebytes/c930100ca97dd4195123104c5396b645
> >
> > The changes to the current bootstrap_prefix.sh are these (in .patch
> > format):
> > https://gist.github.com/awesomebytes/ab1f237feea2f51ae65a5ecca203842e
> >
> > That patch in words: I've wrapped the setup stuff in a function called
> >bootstrap_get_defaults (as that's what it is doing). Then the complete
> >bootstrap process is named bootstrap_all_stages. I also separated the
> >last step calling it bootstrap_emerge_emptytree_system.
> >
> > This keeps the usage of the script the same for the interactive case,
> > the non interactive case, but...There is a 'maybe' breaking change, as
> > now giving the name of the function to execute first calls
> > bootstrap_get_defaults, then calls the function...  and does a couple
> > of necessary workarounds.
> >
> > I say 'maybe' breaking, as it doesn't 'exactly' do the same it did
> > before when executing ./bootstrap-prefix.sh EPREFIX stage2 (but I
> > think it was either broken or I don't know how it was used, which is
> > the most likely option).
>
> > If you don't like this new behaviour, my alternative implementation
> > was to create extra functions like: bootstrap_stage1_noninteractive,
> > bootstrap_stage2_noninteractive, bootstrap_stage3_noninteractive,
> > bootstrap_emerge_emptytree_system_noninteractive Which would do the
> > bootstrap_get_defaults and the workarounds.
> >
> > Let me know what you think. I can prepare a patch with that too. If
> > that's the case, I'd love an explanation on how to use the current
> > behaviour of the script.
>
> I read your patches.  But I don't understand the motivation for
> completely reproduce the interactive use of bootstrap-prefix.sh by
> bootstrap_get_defaults() and bootstrap_emerge_emptytree_system().
>
> The most important part of the CI is to test stages 1~3, because both
> the hosts and the gentoo repository are moving targets.  In the past 10
> years, there were few problems happening in bootstrap_get_defaults() and
> bootstrap_emerge_emptytree_system(), or equivalents.  I would like to
> propose the 3rd way:
>
> 1. set the PATH, CHOST, etc. with your CI script,
> 2. bootstrap-prefix.sh stage1
> 3. bootstrap-prefix.sh stage2
> 4. bootstrap-prefix.sh stage3
> 5. call emerge -e @system with your CI script.
>
> What do you think?
>

First, thanks for explaining your proposal in a detailed way. I sincerely
didn't understand exactly what you meant.

Let me explain further. I found "complicated" to know what environment
variables I needed to set manually in order
to make use of the bootstrap script calling it in the way you just
presented. I did attempt to do it previously and failed miserably.
When looking at the bootstrap-prefix.sh script there are a lot of variables
in a 3000+ lines script, it's kind of daunting.
And when your goal is "I just need this to bootstrap, and it looks very
complicated" you want to the most automated way
(someone smart wrote the automated way!).

Now I understood what do I need to set up.
In hindsight, it was pretty obvious, as the interactive script itself spits
out for me:
EPREFIX=/tmp/gentoo
CHOST=x86_64-pc-linux-gnu
PATH=/tmp/gentoo/usr/bin:/tmp/gentoo/bin:/tmp/gentoo/tmp/usr/bin:/tmp/gentoo/tmp/bin:/tmp/gentoo/tmp/usr/local/bin:/usr/bin:/bin
MAKEOPTS=-j5

as the variables it needs when executing the interactive/noninteractive way.
And now that I've seen that, I also see that the instructions found here:
https://wiki.gentoo.org/wiki/Project:Prefix/Manual_Bootstrap
Are not out of date as reported at the top of the document. Even though it
doesn't say you need to set CHOST and MAKEOPTS, which I guess
they are actually automatically detected. It does briefly talk about CHOST
tho.

So now I see I could write my own lines of the script that does exactly
what you said... but I still need to configure a bunch of things
by hand.

Meanwhile using the same codepath we are already using "guarantees" I don't
need to fight any of that. Neither me, you or the next
maintainer. To me, the mental model I need to create and maintain in my
head from the version I'm proposing is tinier
than from the one where I need to figure out the environment variables and
also copy-paste the MAKEOPTS guesswork.

I mean, ideally, I would just call* ./bootstrap-prefix.sh EPREFIX
noninteractive* and call it a day... but the current CI infrastructure
can't support it. So I'm trying to get the best trade-off I can making this
easier to use from my point of view.

I must say that this email thread has helped me understand better this
process. I still think that being able to call the
different stages in an automated fashion is beneficial, but I understand if
you don't want to change that.



>
> Yours,
> Benda
>
>
>

-- 

*Sammy Pfeiffer*
PhD Candidate at The Magic Lab within UTS.