Re: [gentoo-dev] Dynamic SLOTs

[Matthieu Sozeau]

On Wed, Aug 04, 2004 at 08:34:24AM +0900, Jason Stubbs wrote:
> On Monday 02 August 2004 06:27, Martin Schlemmer wrote:
> > On Sun, 2004-08-01 at 04:01, Jason Stubbs wrote:
> > > On Sunday 01 August 2004 01:29, Mike Frysinger wrote:
> > > > On Saturday 31 July 2004 04:32 am, Jason Stubbs wrote:
> > > > > Well, the idea was that to create local USE flags of the form
> > > > > cc-${ARCH}, have the gcc ebuilds use them to configure the cross
> > > > > compiler and also have them control the the slot. The only real
> > > > > difference to what there is now would be that portage would be able
> > > > > to forecast the SLOT correctly.
> > > >
> > > > so the user would do USE="cc-arm" to get an arm cross compiler ?
> > > >
> > > > and then in the ebuild we'd have to do something like:
> > > > SLOT="cc-arm? ( blah )" ?
> > > > how'd would we specify a default slot ?
> > >
> > > SLOT="!cc-arm? ( !cc-foo? ( !cc-bar? ( !cc-baz ( defaultslot ) ) )"
> >
> > I really do not know why there hasn't been a '&&' and '||' logic
> > added yet?  Will it really be that difficult to implement?
>
> It would take some time to implement and then testing time and what not. The
> main reason it hasn't been done is that it doesn't add anything that can't be
> done already.
>
> > > Terrible, no? This is gone anyway.
> >
> > What exactly do you mean by this?
>
> There is no point adding complexity to SLOT (and making dep calculation
> intricate and very slow) for the benefit of four ebuilds that could be
> implemented without the use of dynamic SLOTs anyway.
>
> As far as I know, allowing ${KV} as a SLOT was a quick hack to prevent
> previously installed kernel modules from being unmerged. It was/is a
> temporary solution until something better is found. That better solution is
> what I'm looking for here.
>
> Regards,
> Jason Stubbs
>

I'm happy those issues are being discussed as i didn't have much incentive to
post a feature request like what follows until now. First, some context:
I maintain the ML packages and particularly the ocaml ones. As some of
you may know, ocaml is not binary compatible from version to version and
it is required to recompile all dependent libraries on compiler updates,
(in the right order of course). This situation is very similar with Perl
or Python actually. Managing more than one version of such packages is
currently impossible (using portage only) AFAIK.

What I would like is a proper framework to deal with this
compilers/interpreters, and hopefully one general enough to deal with
cross-compilation too. I have hacked a solution on my own to deal with
the problem of incompatible compilers, using dynamic SLOTS, an ocaml
eclass and an ocaml-config script (similar to gcc-config). All ocaml
packages use this eclass which sets the slot to the compiler version and
sets some env vars (pathes) to allow flawless cohabitation of the
different compiler versions. It works just like a charm. I can update
all my packages for new versions of ocaml, do backward-compatibility
tests (i'm an ocaml dev) and if i wanted to, i could manage the CVS
version of ocaml and all ocaml packages using only portage and my
ocaml-config script.

The problem is obviously that i'm using portage for something it was not
intended to do (cleanly, at least). I have some (clear?) ideas about how it
could be done though, and i'm willing to give help. Here's my wish:

	      *** Portage support for compilers ***

  I never heard anybody mention it, but I think it is required to get a
  clean implementation. It would introduce some changes:

  - Factorized, standard config files and utilities for all compilers/interpreters,
  gcc, python, perl, ocaml, you name it. Configurations would be simply a set
  of env vars, much like what gcc-config uses today.
  You could set different configurations for different users etc...
  A set of compiler policies (in the tree) would also be needed to 
  mention the fact that no ocaml compilers are compatible, but that 
  different gcc's are. A new flag could be added to emerge telling him
  to rebuild automatically all packages that depended on a previous
  compiler that is incompatible with a newer one.

  - An ebuild metadata variable mentioning the compiler configurations
  it depends on: a CDEPEND in fact, mentioning which compiler versions
  it is compatible with. Unset would mean any, but when set, it could 
  well be cached and maybe some pruning in the dep tree could be done too.

  - When invoking an ebuild for some package, portage would simply set 
  the compiler variables (for the compilers mentionned in CDEPEND).
  The process of dependency checking would introduce one more step on
  each depended-on pkg check : When a package dependent on the same language
  as one mentionned is found, try to find an installed one matching the
  dep requirements and the compiler policies, if none matches, use the
  same policies as the current one for dep-checks etc... Informative
  messages could be easily crafted in case of errors. 

Now for the critics, i will point out that my scheme doesn't extend
ebuild syntax, which is good as most compiler compatibility problems can
be handled at once and for all packages. It would also factorize some
code that's between the various *-config scripts and as an added benefit
it would make every compiler update easy, getting rid of rebuild
scripts. The main advantage is a finer, more robust dependency
checking. 

Here's a test case:
I've got ocaml-3.07, lablgl and lablgtk merged, i want the latest
mldonkey which requires lablgtk-1.2.7 which in turn works only with
ocaml-3.08. Portage will see that mldonkey has no particular preference
regarding compiler versions, it will check lablgtk which will tell that
it needs ocaml-3.08 and no other, so, as you have the opengl USE flag
set, portage will check ocaml compiler policies to see if lablgl in 3.07
would be compatible with lablgtk compiled with 3.08, the policy says no,
hence portage checks that lablgl can be compiled with 3.08 and finally
it sets its roadmap to: install 3.08, lablgl for 3.08, lablgtk for 3.08 and
finally merge mldonkey for 3.08.

Now for the cross-compiling, it would greatly help to have such support
built into portage. If i want to build ppc packages on my athlon,
i can tell portage to use a (more general than just compilers)
configuration that's declared incompatible with my current one 
(as it should be) that sets the root to some other place, have a
particular make.conf, cross-compiler config etc... I'm just prospecting here.

That was just my 2 euros, i'm more than open to criticism about my
proposal which i ruminate for some time now (see bug #39246), and 
i'll get it as far as I can, writing a GLEP and coding included (i'm in
holidays after all :).

Thanks for your patience !

-- mattam