[gentoo-dev] Re: [RFC] Masterplan for solving LINGUAS problems

[Martin Vaeth]

Mart Raudsepp <leio@g.o> wrote:
> Ühel kenal päeval, K, 01.06.2016 kell 20:24, kirjutas Martin Vaeth:
>> Gentoo has chosen this name so that as a side effect of setting
>> USE=linguas_* you also get a correct LINGUAS variable exported
>> (according to the USE-settings and your settings and according
>> to the ebuild's IUSE.)
>> These are the package with LINGUAS options in their USE flags
>> which you mentioned above.
>
> People already set LINGUAS for autotools control; using the same for
> USE_EXPAND probably seemed natural to re-use that, but the effects on
> PM variable mangling wasn't thought about at the time.

At least, it looks like it was well-thought:
It avoided errors like "language xxx not supported",
it gave the user control to set it easily on a per-peckage basis,
and most important, it made the package manager aware of the LINGUAS
used and handle binary packages correctly.
From the user perspective it had only advantages.
If all these positive side effects were not the original intention,
it was a very good accident that it happened.

> There were no problems, and there still are no problem with implicit
> LINGUAS handling (by means of not listing them in IUSE), as long as the
> package manager doesn't modify the variable set in make.conf.

As long as there is only one setting valid for all packages
and all systems on which one will ever install the binary package.
A quite unrealistic setting for quite some use cases.

> The user has set it explicitly somewhere (make.conf most likely) and
> simply gets honored in upstream build system, just like CFLAGS.

That handling of CFLAGS is bad enough - no reason to repeat this mistake:
This is usable only if you have only one architecture but not in a
hybrid setup with e.g. roughly compatible machines where e.g. you change
CFLAGS by /etc/portage/env or /etc/portage/package.env (or locally
in the environment) to compile for another machine or in a ways
compatible for another machine, etc.

Well, for CFLAGS, CXXFLAGS, LDFLAGS at least the
variables actually used are stored in
/var/db/cat/pkg/{CFLAGS,CXXFLAGS,LDFLAGS}
and similarly stored in the metadata of binary packages (not
hidden in some hard-to-parse compressed environment file),
so that the package manager _could_ check their validity
(and even if it does not - which AFAIK is the case currently -
it is not very hard to write a script which does this).

That's why I suggested that - as the very least - if you really want
to kill all the current advantages for no reason, at the very least
handle LNGUAS not even *worse* than CFLAGS and at least store them
in a new /var/db variable so that - if you already let the user do
the job which the package manager is supposed to do - at least the
user can write his own scripts to do portage's job.
(Of course, this scripts can never work as good as the current
solution if you really kill it, because these scripts are lacking
the information which LINGUAS are actually supported by the package.)

> LINGUAS does not imply any order whatsoever. Packages that assumed so
> were seriously buggy and if any still remain, this needs to be fixed
> ASAP.

So if you consider this as a non-issue, I see even less reason
why to remove the *transpararent* (to the package manager and user)
IUSE=linguas_* and go back to the intransparant LINGUAS=...
(I am not complaining about a renaming of the IUSE, as long as
this IUSE in the end affects LINUGAS)

> Including them all in IUSE for simple translation catalogs is not
> feasible to maintain.

I disagree. The list of packages is in most cases very easy to get
(e.g. ls po/*.po) and inserting it correctly is a usually a
one-time thing. For bumping, one will have to check for dependency
changes anyway, and that a package adds a new language without listing
it in its ChangeLog is also very rare.

> And yes, there are packages that have 196 different language and
> dialects translations. Even core GNOME packages [...]

Nevertheless these packages are very rare. One could also manage
*one* list of all languages globally and allow to use it for
convenience for such cases (if a few of the listed languages are
actually not provided, nothing critical will happen: The
user's output has perhaps a few more entries than necessary and
perhaps an unnecessary rebuild can happen if the user
selects/deselects an exotic language.)

> It also clutters emerge --verbose --pretend or --ask output hard with
> LINGUAS="long list of 196 language codes"

About the situation which we currently have (e.g. k3b): For some
packages the LINGUAS list is not 1 line but 2-4. So what?
If this really is an issue for somebody (I don't consider it some),
then one can discuss to add an option to suppress the LINGUAS output
for those not interested in it.
But it's certainly not a reason to make it *intransparent* for the
package manager and user.

> these and mixes it all up for when the information is more useful
> when it implies huge extra downloads.

This happens always with USE-flags: Unless you look at the
ebuild source, you never know whether it will trigger dependencies,
increase the download time, increase the build time (and how much),
increase mount of installed data (and how much) etc.
Just because that information is not listed, it does not make having
the USE-flags less useful. There is no difference to linguas-specific
USE-flags.

> The point is that if the USE_EXPAND is renamed, then a LINGUAS as found
> in make.conf will just get passed verbatim into the environment (as
> parsed in via shlex - make.conf is not bash sourced), and then honored
> by upstream build system.

Yes, exeactly: Intransparent implicit handling as a side effect,
instead of a transparent and reliable handling through the
package manager.
We had this hackish handling before IUSE=linguas_*
was introduced, which was a big relief for a lot of people.
Now the suggestion is to turn back to the hacks, excusing
the hack by declaring setting/changing LINGUAS a
thing for the "advanced" user, only.

> I don't see anything hackish in just setting L10N for extra language
> support downloads and LINGUAS for UI translations.

Setting LINGUAS without awareness of the package manager of it
*is* a hack.

> You can find it in the VDB packed in environment.* in case of portage.

Now, don't tell me that the necessity to write code which uncompresses
a library and parses the tricky escape handling of the environment
file to get eventually you own hackish script to do what the
package manager should be suppposed to do, is anything else than
an even bad hack.

> Exporting it specially like CFLAGS, CXXFLAGS, FEATURES and others are,
> is a separate matter if one wants to propose that later.

It would at least turn a really bad hack into a little bit less bad hack.
But it is of course never as good as a real transparent solution
with USE-flags.

>> Where our opinions strongly differ is whether a way to
>> cleanly set LINGUAS should be provided (e.g. by allowing
>> IUSE=l10n_* to modify LINGUAS appropriately, and make this
>> the "good" way instead calling it a "broken" way which
>> should be avoided.)
>
> I don't believe I have read such a claim from his e-mails anywhere, so
> not sure what this is about.

My the suggestion to make this (export LINGUAS based on
IUSE=linguas_*/l10n_*) as the default solution in l10n.eclass
(which is the current place where this thing happens implicitly
through the name) was replied with:

: This eclass should be killed with fire as ugly nonsense [...]
: As far as I'm concerned, it may not exist, be broken or whatever.

>> For instance, the default language of mplayer will depend on
>> the first entry of LINGUAS [...]
>
> This is not the case since long ago as far as I look, as it was a bug
> and fixed long ago.

Then even more, I think that LINGUAS should be implicitly set by through
USE-flags: If order does not matter, there is absolutely no reason to do
not do it transparatenly this way.