Re: [gentoo-catalyst] need clarification

[Chris Gianelloni]

On Sat, 2005-01-29 at 10:16 +0100, h2o wrote:
> Hi
>
> i am currently trying out catalyst but have run into several questions
> and problems.
>
> 1. what gets carried over from a stage1 to stage2 while bootstraping?
> the faq point out that when seeding a stage3 to create a stage1 the
> linux-headers of the seed will be used. but nowhere is described what
> happens when progressing from 1 to 2, and from 2 to 3.

Well, it isn't documented only because it is identical to what happens
when building a Gentoo system.  If you want to know more about the
building of Gentoo, the Gentoo Handbook is the authoritative source.

> 2. if nothing from the stage1 gets carried to the stage2 why should i
> even use anything but the stage1 from a gentoo-mirror?

Everything from the stage1 gets carried into the stage2 and that is used
to bootstrap.  If you have 2.4 headers, then you'll bootstrap with 2.4
headers.

> at this point i should possibly point out why i want to use catalyst:

> i plan to build a stage3 which already includes all packages which i
> always need. this could be things like gentoo-sources, grub, metalog,
> fcron. this would mean that the choices that i have already made will be
> reflected in the stages i use to setup new machines with. i would like
> to have several different stages for desktop, server and other
> specialised boxes. the desktop version eg would already contain X, my
> prefered wm, firefox ... , but this would already be more like a stage4.

You honestly shouldn't modify the stages, as you'll find yourself having
way more trouble than it is worth, and you definitely won't get support
from development on it.  The files included in stage1 and stage2 are set
in catalyst.  The contents of a stage 3 is set by the profile used.  You
would need to keep your own profile to build your custom stages, but
even then none of it would be configured.

> i also need some way to replace configuration files inside the stages
> with already modified versions of my current machine. (->6)

Yeah.

Catalyst cannot do this and I can tell you that it probably won't be
extended to do this.  Catalyst is designed to build custom installation
media, not custom stages.

> 3. is catalyst intended to create such a stage3/4 or should this rather
> be done with some seperate scripts?

You would do better to use separate scripts.  My suggestion would be to
build through a livecd-stage1, then take the packages that you created
as your pkgcache and use those for installation.  Even then, they
wouldn't be customized.  One of the main strengths of catalyst is that
it produces the same results given the same input, so making
modifications is quite a bit harder than you might realize at first.

> i am uncertain of how i can customise my stages. i think these ways are
> possible:
> - modify the profile that is used to build the stage (->4)

You could add packages to the "system" part of your profile, but then
you would never be able to remove them without making modifications to
the profile.  This is probably a very bad idea, but would conceivably
work.

You still would have no configuration on any of your packages.

> - when building a livecd options like "livecd/packages" can be used in
> the spec file. (->5)

This installs them on the LiveCD, but does nothing for creating a stage.
You could conceivably modify the temporary catalyst cache of this once
you had all of your packages merged, to include your customizations,
then tar up the entire thing as a "stage4" tarball.  This is probably
the closest you would get to what you want, and it would require no
catalyst work, only manual work from you.

I can also suggest using livecd-stage2 as a way to modify your stage.
Since livecd-stage2 is built from livecd-stage1, you would simply feed
it a fsscript to make your modifications, along with not cleaning any
packages.  You could even feed it your kernel config.  At this point you
would have a binary kernel, created by genkernel, plus your modules.
You would need to extract your kernel into the livecd-stage2 temporary
catalyst cache's /boot directory and modify your bootloader.  You would
then tar the livecd-stage2 cache up as your stage4.  The only
requirement you would have after unpacking the tarball on a new machine
would be to re-install the bootloader into the MBR/partition.

> instead of using /usr/portage/profiles/default-linux/x68/2004.3/ i made
> a copy of it. i made the copy as a subdir and also as a directory on the
> same level but both times i got this error:

Like I said, profile modification is probably the wrong way to go about
it.

> !!! ARCH is not set... Are you missing the /etc/make.profile symlink?
> !!! Is the symlink correct? Is your portage tree complete?

Did you set the profile in your spec file to be this one?

> i checked the suggested causes for this but didnt find anything unusal.
> /etc/make.profile does not point to the new profile though. when i use
> another already profided profile which also isnt symlinked from
> /etc/make.profile catalyst procedes past this point without any problems.

Catalyst makes the symlink itself based on the profile listed in the
spec files.  The symlink doesn't matter, as it is relinked by catalyst.

> 4. how do i create and use my own profile? is this the correct way to
> costumise a stage?

Profiles are pretty simple.  So long as you have a valid one with valid
inheritance and call it properly from your spec file, you should have no
problems.  However, this is not the way to go around modifying stages.

> 5. why are there no options like "livecd/use" for normal stages? or are
> they just not listed?

They are not there because the stages are not supposed to be changed
from what is in the profile.  It would break the stage drastically.

> 6. how are files that are not related to a package, or that replace
> files profides by a package moved into a stage? is there a spec option
> for this?

They cannot be.

> 7. how is the profile used when building a stage1/2? the following
> suggests that profile only has an effect on `emerge system`:

The profile is pretty much only used to calculate minimal versions of
the toolchain and default virtuals during stage1 and stage2.

> from /usr/portage/profile/base/packages:
> OK, you're staring at this file and you have no idea what these stars
> are for.  Here's the scoop.  An initial "*" marks a package that is part
> of the official BASE system profile.  If there is a "*" then `emerge
> system` will use the line in its calculations of what should be
> installed for the base profile.  Lines without a "*" prefix will be
> ignored for profile system calculations.

Correct.  The profile only affects the stage3 in this manner.

> E2. i sucessfully build a stage1 but when i tried to use it as a seed
> for a stage2 it turned out it did not contain gcc. i have used the
> unmodified /usr/portage/default-linux/x86/2004.3/.

Newer versions of catalyst (> 1.1.1) require a modified bootstrap.sh
script that will become the default once 2005.0 is released.  You can
find the script in /usr/portage/scripts as bootstrap-new.sh and it can
simply be renamed to bootstrap.sh to get a working system.  The
internals of stage building have changed a good bit in catalyst.  I
would suggest checking out the ChangeLog for the package
in /usr/share/doc to try to keep up with each set of changes.

> i added "*>=sys-devel/gcc-3.3.4-r1" to
> /usr/portage/profiles/default-linux/x86/packages in the hope in getting
> a stage1 with gcc only to run into another problem.

You have to change it in the profile you're actually calling, otherwise
inheritance will most likely overwrite it.  Profiles to the right, or
further down the tree, have a higher weight than the profiles to the
left, or higher up the tree.  The 2004.3 profile settings will always be
taken over the x86 settings, just as the x86 setting will always be
taken over the default-linux settings, etc.

> E3. after much copyling finally:
>
> Running command "/bin/bash /usr/lib/catalyst/targets/stage1/stage1.sh
> preclean"
> i386-pc-linux-gnu-3.3.5 i386-pc-linux-gnu-3.3.5-hardened
> i386-pc-linux-gnu-3.3.5-hardenednopie i386-pc-linux-gnu-3.3.5-hardenednossp
> * /usr/bin/gcc-config: Too many arguments!  Run /usr/bin/gcc-config
> without parameters for help.

You're going to have to run one of the newer versions of catalyst and
use the modified bootstrap.sh, as I mentioned earlier.

> thx ALOT
> to anyone willing to enlighten me on any of these questions/problems.
> there is quite a few of them i know... but there is also little
> documentation :-)

Most of what you want isn't documented because it isn't possible or
isn't supported. ;]

-- 
Chris Gianelloni
Release Engineering - Strategic Lead/QA Manager
Games - Developer
Gentoo Linux