Re: [gentoo-alt] Allow non-interactive bootstrap to be stopped at arbitrary stages

[Sam Pfeiffer]

Hello Benda,



On Sat, 25 Jan 2020 at 23:58, Benda Xu <heroxbd@g.o> wrote:

> Hi Sam,
>
> Sam Pfeiffer <sammypfeiffer@×××××.com> writes:
>
> > To be more specific... To easily go thru it I made a gist with the
> > latest version of bootstrap-prefix.sh and I'll be pointing to specific
> > lines (this also helps me recap why I did this, as I remember using
> > stage{1,2,3} didn't work for me when I first came into this):
> >
> > When you execute bootstrap-prefix.sh without arguments (other than the
> > EPREFIX, ./bootstrap-prefix.sh /MY/EPREFIX) it calls this line:
> >
> https://gist.github.com/awesomebytes/ba4df9c6ef3ab5742cb9aceed7998c3d#file-bootstrap-prefix-sh-L3033
> > Calling bootstrap_interactive
> >
> > If you give an extra argument (./bootstrap-prefix.sh /MY/EPREFIX
> > [noninteractive, stage{1,2,3}]) it calls this line:
> >
> https://gist.github.com/awesomebytes/ba4df9c6ef3ab5742cb9aceed7998c3d#file-bootstrap-prefix-sh-L3082
> > bootstrap_${TODO#non} || exit 1
> >
> > Which, on the noninteractive case, goes to the same place than no
> > arguments (bootstrap_interactive). But in the stage{1,2,3} calls
> > directly the bootstrap_stageX function.
> >
> > The thing is, noninteractive does some checks, finds resources and
> > sets up environment variables like:
> >
> > * Check for bad flags:
> >
> https://gist.github.com/awesomebytes/ba4df9c6ef3ab5742cb9aceed7998c3d#file-bootstrap-prefix-sh-L2200-L2247
> > * We find a gcc:
> >
> https://gist.github.com/awesomebytes/ba4df9c6ef3ab5742cb9aceed7998c3d#file-bootstrap-prefix-sh-L2273-L2433
> > * We get a cpu count and adjust -j for Make:
> >
> https://gist.github.com/awesomebytes/ba4df9c6ef3ab5742cb9aceed7998c3d#file-bootstrap-prefix-sh-L2443-L2501
> > * Check if we deal with multilib systems:
> >
> https://gist.github.com/awesomebytes/ba4df9c6ef3ab5742cb9aceed7998c3d#file-bootstrap-prefix-sh-L2503-L2576
> > * Check if we are boostrapping from a Gentoo system:
> >
> https://gist.github.com/awesomebytes/ba4df9c6ef3ab5742cb9aceed7998c3d#file-bootstrap-prefix-sh-L2578-L2608
> > * Set our EPREFIX:
> >
> https://gist.github.com/awesomebytes/ba4df9c6ef3ab5742cb9aceed7998c3d#file-bootstrap-prefix-sh-L2610-L2681
> >
> > Which is summarised at the end by setting the environment variables
> EPREFIX, CHOST, PATH, MAKEOPTS.
> >
> > And then finally calls bootstrap_stage1_log, which runs
> > bootstrap_stage1 itself.
> >
> https://gist.github.com/awesomebytes/ba4df9c6ef3ab5742cb9aceed7998c3d#file-bootstrap-prefix-sh-L2724-L2726
> >
> > Then, we compare with calling bootstrap-prefix.sh EPREFIX stage1...
> > Calls bootstrap_stage1():
> >
> https://gist.github.com/awesomebytes/ba4df9c6ef3ab5742cb9aceed7998c3d#file-bootstrap-prefix-sh-L1355
> >
> > And as you can see it won't do any of the extra work done in
> > bootstrap_interactive().
>
> Perfect! You are correct in understanding the script.
>

Great, that's kind of a relief.

>
> > I'm not that versed in this script, so I may be missing something easy
> > to overcome this (or I don't know enough about what is going on), but
> > there is no separate function that does everything in
> > bootstrap_interactive without calling bootstrap_stage1. And
> > bootstrap_stage1 does not do all that setup that bootstrap_interactive
> > does. So that's how I came to my proposal.
> >
> > If there is some patch to enable that setup from bootstrap_interactive
> > with bootstrap_stage1, that would be a good alternative, I guess. I
> > still like the fact that with my approach you are just saying
> > "bootstrap everything until this step" instead of needing to manually
> > do bootstrap_setup (imaginary function I just named) then
> > bootstrap_stage1, then bootstrap_stage2, then bootstrap_stage3, and
> > then no argument for the last emerge -e system, to get to a specific
> > bootstrap stage, but that's just commodity.
>
> Why manually do bootstrap_setup then stage 1-3, emerge -e @system a bad
> thing?  That gives you more control of what to do in the CI system.  I
> would prefer to have the explicit steps than environmental switches.
>

It took me a while to figure out how to do it with minimal changes... but
this is the new version I came up with:

https://gist.github.com/awesomebytes/c930100ca97dd4195123104c5396b645

The changes to the current bootstrap_prefix.sh are these (in .patch
format):
https://gist.github.com/awesomebytes/ab1f237feea2f51ae65a5ecca203842e

That patch in words: I've wrapped the setup stuff in a function called
*bootstrap_get_defaults* (as that's what it is doing). Then the complete
bootstrap
process is named *bootstrap_all_stages*. I also separated the last step
calling it *bootstrap_emerge_emptytree_system*.

This keeps the usage of the script the same for the interactive case, the
non interactive case, but...There is a 'maybe' breaking change,
as now giving the name of the function to execute first calls
*bootstrap_get_defaults*, then calls the function...
and does a couple of necessary workarounds.

I say 'maybe' breaking, as it doesn't 'exactly' do the same it did before
when executing
*./bootstrap-prefix.sh EPREFIX stage2 *
(but I think it was either broken or I don't know how it was used, which is
the most likely option).

If you don't like this new behaviour, my alternative implementation was to
create extra functions like:
*bootstrap_stage1_noninteractive, bootstrap_stage2_noninteractive,
bootstrap_stage3_noninteractive,
bootstrap_emerge_emptytree_system_noninteractive*
Which would do the bootstrap_get_defaults and the workarounds.

Let me know what you think. I can prepare a patch with that too. If that's
the case, I'd love an explanation on how to use the current behaviour of
the script.


>
> > Let me know if I'm wrong somewhere or you think of any better solution!
>
> You are nowhere wrong.  I just prefer keeping the manual bootstrap
> options up to date and have it drive our CI.
>
> Yours,
> Benda
>


-- 

*Sammy Pfeiffer*
PhD Candidate at The Magic Lab within UTS.