Re: [gentoo-dev] new glep draft: Portage as a secondary package manager

[Jason Stubbs]

On Friday 20 May 2005 21:30, Michael Haubenwallner wrote:
> Jason Stubbs wrote:
> <snip>
>
> > I intend that the package to be installed should not assume anything
> > about where its dependencies are and should query portage for them all.
>
> Oh no, now many things get much clearer to me :(
>
> But - aren't there many settings left over to the packages to decide,
> at least to choose the package-defaults ?

Any package that does this is broken. There are a couple of cases where 
there's no other choice because portage doesn't allow for anything else but 
that will change.

> > Requiring ebuilds to have special handling for when their dependencies
> > are in the same ${PREFIX} and when in a different ${PREFIX} just seems
> > crazy to me.
>
> Agreed.
> You intend to use some portage-API in ebuilds which knows where and how
> to look for dependencies, did i get this right ?
> But having portage to behave differently seems crazy to me though.

I would fiercely disagree with implementing ${PREFIX} support without doing 
major changes in portage to support it. There's enough hacks in portage 
already.

> > If portage doesn't tell the packages what to build against, they'll go
> > their own merry way which would definitely make the binary packages
> > mentioned below impossible.
>
> Disagreed - binary packages right now can only be shared between highly
> identical machines...

In general, I do not be the case.

> It seems that there are two philosophies of how packages find their
> dependencies:
>
> 1) The Gentoo-Linux one:
>    All the depency information comes from the package manager and is
>    passed to packages by commandline, skipping the whole
>    autoconf-intelligence.
>
> pro:
> + The patching required for packages is less complex, most of the
>     time the autoconf-intelligence has to be disabled if there's a
>     commandline parameter missing for a particular feature.

+ Portage is able to know what a package requires and can ensure system
  consistency.

+ Binary packages are possible.

> contra:
> - To get this work on different platforms, an autoconf alike
>     intelligence has to be re-implemented to portage and the ebuilds,
>     including access to provided/injected packages or to the
>     primary pkg mgr's database.

Wrong. What do you think *DEPEND is if it's not "autoconf alike intelligence"?

> - External packages installed by hand into the primary prefix will not
>     be stored in the primary pkg mgr's database and therefore cannot
>     be seen by portage as the secondary mgr once portage could read it.

No different to the current system. This is what package.provided is for.

> - There's always a rest of decicions left to the package's
>     autoconf-intelligence.

Such as? I don't know if this is a pro or a con or what.

> - This works for the primary package mgr, but would become
>     unmaintainable for secondary installations - which is your
>     legitimate fear.

Without it, there is no guarantee of system consistency and hence no reason 
for adding support for it into portage at all. If you don't want portage to
maintain system consistency for you, why not just do "./configure; make; make 
install" ?

> - Different behaviour seems to be required within the ebuilds or
>     (through an API) portage if installed to / or prefix or ~.

Different behaviour between prefix or ~ only. / is just another prefix. This 
is definitely not a con of philosophy 1. It is a requirement regardless.

> 2) The one necessary for secondary installation:
>    Let the packages decide from environment (PATH,PKG_CONFIG_PATH,...)
>    and filesystem where to find their dependencies, and the package
>    manager just has to prepare the environment variables and the
>    filesystem to let the autoconf intelligence work.

"necessary"? I think not.

> pro:
> + Many of the packages already do compile and run on several platforms,
>     by checking the environment and filesystem, and are shipped with the
>     autoconf-intelligence required for that.
>     This intelligence is used and therefore not needed in portage and
>     ebuilds.

Works the same for "/", no? Tell me again what the point of portage is?

> + This is what autoconf/libtool are created for.

Do you use a new point to reiterate your last point just to make it look like 
your way is better?

> + If there are external packages installed by hand into the primary
>     prefix, they are normally recognized, because they appear on the
>     filesystem and in the environment.

Woops. Exactly the same point again.

> + When installing within a primary portage, this continues to work.

And again. Except that here you're saying that system consistency should be 
thrown out the window altogether. It seems to me like your concept of portage 
is:

emerge() {
	PKG="$1"
	tar zxf ${PKG}
	cd ${PKG/.tar.gz/}
	./configure
	make && make install
}

> + The pkg mgr does nearly the same commands as an administrator likes to
>     do by hand if she has no pkg mgr at hand:
>        ./configure [--prefix=/my/prefix]
>        make
>        make install [DESTDIR=/tmp/inst]
>     without any more arguments ideally.

Again, why use portage at all?

> + Seems to be much less need for ebuild/portage to behave differently if
>     installed to / or prefix or ~

Eh? Where did that come from?

> contra:
> - If the autoconf-intelligence does not work, there's more effort needed
>     to create the patches to get it work.

Where do USE flags come into this?

> - There's always a rest of issues needed to be dealt with in some
>     ebuilds.

This isn't a pro or con either. It's just a simple fact that exists 
regardless.

> Portage itself does not predetermine which philosophy to use - it is to
> the ebuild-dev to choose one - but maybe portage's full-support for the
> former "doesn't exist and isn't going to exist for a very long time, if
> ever" (to say it with ciaranm's words).

Wrong. Portage does predetermine which philosophy to use. With your option 2, 
there is no way to do:

1) A working version of depclean
2) Safe unmerging of packages
3) Binary packages
4) Parallel compiles (future goal)
5) Dispatched compiles in a portage cluster (future goal)

> To me now, my real problem seems to be the philosophy established
> in the ebuild-devs right now, could that be the case ?

It really seems that Gentoo doesn't suit your way of thinking at all.

> > Until portage supports other package formats, an equivalent of
> > package.provided would be used for this. However, this has nothing to do
> > with how ebuilds would find out where their dependencies are.
>
> Agreed, but just to ensure unterstanding:
> One thing is the dependency tree for the pkg mgr to install all the
> prerequisites, the other is how packages then find those prerequisites,
> right ?

Building a dependency graph implies gaining the knowledge of where packages 
are or will be installed. Passing that information to ebuilds is a minor 
thing.

> >>>7  Portage needs to be able to configured with one-way dependency
> >>>allowance between installed package databases. For example, ~ installed
> >>>packages are allowed to depend on / installed packages.
>
> When installing a package into ~, a dependency install to / or prefix
> becomes triggered or sth. like that, do you mean this ?

Forget this. It's an implementation detail but we're not even past concept let 
alone design.

> > This is just silly, in my opinion. Home-support may have issues beyond
> > prefix-support, but most of them are the same. Why would you only want to
> > consider prefix-support and implement it if considering home-support
> > might invalidate assumptions you make? Doing this pathspec stuff is a
> > huge project so it's not something that should be rushed into.
>
> Agreed from your point now since i know the Gentoo-Linux philosophy of
> what ebuilds have to know and what packages are allowed to know (see
> above).
>
> Since i've never installed plugins into my ~, some questions here:
>
> Can those plugins be installed into prefix or / too ?
>
> If so, what are the general differences in how to do that against
> installing into ~ ?

I haven't either. However, Ciaran posted about this. The main difference is 
that the directory structure changes completely.

> > Doing it properly should also lay down most of the ground work for
> > getting full cross-compile support into portage as well. Personally, I
> > wouldn't push this GLEP for approval at all until those issues are at
> > least looked and fleshed out a bit as well.
>
> I've not used cross-compiling yet, but there are cross-compiling bits in
> the ebuilds - so what does not work now or how can these bits work now ?

I haven't worked with it either. I know that there's extensive 
cross-compilation support in the toolchain but I'm pretty sure there's little 
to none before that. Using portage for cross-compilation at the moment is a 
touch-and-go thing that requires a lot of hacking.

> > I'll reiterate; doing this is a huge amount of work. There really needs
> > to be a guarantee that the effort will be worth it. And to be honest, if
> > the only benefit is prefix-support then most everybody will consider the
> > effort to outweigh the return.
>
> I want to distinguish between the effort for portage itself, which does
> not immediately hit the ebuilds - it's just a portage-feature not used
> by the gentoo-ebuild-tree - and the effort to get ebuilds supporting
> prefix/home install.
>
> There's always the way to have a separate ebuild-tree for prefix-
> installation, not containing ebuilds for linux-kernel and the like...

It's a huge amount of work to get full support into portage as well. However, 
most of it is working through the concepts involved and designing 
accordingly. The actual implementation differs little to what is required for 
the current system.

Regards,
Jason Stubbs