Re: [gentoo-dev] Re: [RFC] Masterplan for solving LINGUAS problems

[Mart Raudsepp]

Ühel kenal päeval, K, 01.06.2016 kell 20:24, kirjutas Martin Vaeth:
> Gentoo has chosen this name so that as a side effect of setting
> USE=linguas_* you also get a correct LINGUAS variable exported
> (according to the USE-settings and your settings and according
> to the ebuild's IUSE.)
> These are the package with LINGUAS options in their USE flags
> which you mentioned above.

People already set LINGUAS for autotools control; using the same for
USE_EXPAND probably seemed natural to re-use that, but the effects on
PM variable mangling wasn't thought about at the time.
This was introduced only to have a natural way to refer to them in
SRC_URI and beyond, for extra language pack downloads (like firefox
langpacks).
Things escalated badly from there.

> This had the advantage that no additional code except a correct
> IUSE is needed for these ebuilds. (That's why gentoo has chosen
> this name).

> It had the disadvantage that:
> 1. Ebuilds without handling IUSE=linguas_* explicitly (or
> with wrong values, because maintainers did not care about the
> values) had problems.

There were no problems, and there still are no problem with implicit
LINGUAS handling (by means of not listing them in IUSE), as long as the
package manager doesn't modify the variable set in make.conf.
The user has set it explicitly somewhere (make.conf most likely) and
simply gets honored in upstream build system, just like CFLAGS.
The problem is when package manager mangles it per PMS rules, for which
I'm told portage has special PMS ignoring exceptions.

> 2. Some packages which needed a different handling of LINGUAS
> (like respecting the order) also had problems.

LINGUAS does not imply any order whatsoever. Packages that assumed so
were seriously buggy and if any still remain, this needs to be fixed
ASAP. We have LC_* and LANG environment variables to choose what the
localization is at runtime, not some arbitrary choice at build time.
This is not a concern whatsoever, forget about it.

> There are at least the following solutions to these disadvantages:
>
> a)
> One _could_ solve problem 1 simply by not touching LINGUAS if
> IUSE=linguas_* is not in the ebuild. (Main problem with this
> solution: It is not PMS compatible; one needs e.g.
> an exception for LINGUAS).

This is a problem when they are in IUSE too. For example a package
provides translations via gettext, but has extra downloads for some
languages support (lets say grammar checking support for a language in
an office application). The latter would get marked up with IUSE for
usage in SRC_URI and install. Package manager will intersect them and
remove the LINGUAS values not found in IUSE, but simple UI translation
gettext catalogs for these languages might still exist upstream. They
now get removed due to IUSE=linguas_* not including them.

Including them all in IUSE for simple translation catalogs is not
feasible to maintain. It also clutters emerge --verbose --pretend or --
ask output hard with LINGUAS="long list of 196 language codes" for
these and mixes it all up for when the information is more useful when
it implies huge extra downloads.

And yes, there are packages that have 196 different language and
dialects translations. Even core GNOME packages would have such an
amount, see https://l10n.gnome.org/languages/

> In a similar manner, one could solve problem 2 by allowing
> ebuilds to modify LINGUAS at build time (which is perhaps
> also not PMS compatible).
>
> b)
> Or one could, for all packages with LINGUAS in their USE-flag
> simply rename IUSE=linguas_* to IUSE=l10n_* and set
> export LINGUAS=$L10N
> in these ebuilds (this would require practically no manual ebuild
> editing if one does it in the l10n.eclass).

The idea is not to export this anywhere in ebuilds. We'd have to do
that in pretty much all.
The point is that if the USE_EXPAND is renamed, then a LINGUAS as found
in make.conf will just get passed verbatim into the environment (as
parsed in via shlex - make.conf is not bash sourced), and then honored
by upstream build system.
That's because it isn't in USE_EXPAND list anymore, so it doesn't have
such PMS rules to modify it.
One can modify the environment via package.env as well, to change
things per-package if one fancies for some reason.

> I had originally understood mgorny's suggestion such that b)
> is meant, and I would complain neither against a) nor b).
>
> But in his reply, mgorny says that, moreover, he wants to
> remove the l10n.eclass, more precisely, that he considers
> it as broken that packages use IUSE=l10n_* for setting
> LINGUAS.
>
> This suggestion means that setting LINGUAS can be done
> *only* in a hackish way by the user, "hidden" from the
> package manager, not in the clean way as it is currently
> done by those ebuilds with LINGUAS in their use-falgs.

I don't see anything hackish in just setting L10N for extra language
support downloads and LINGUAS for UI translations.

> I agree with him that a hidden setting is a bad idea.

You can find it in the VDB packed in environment.* in case of portage.
Exporting it specially like CFLAGS, CXXFLAGS, FEATURES and others are,
is a separate matter if one wants to propose that later.

> Where our opinions strongly differ is whether a way to
> cleanly set LINGUAS should be provided (e.g. by allowing
> IUSE=l10n_* to modify LINGUAS appropriately, and make this
> the "good" way instead calling it a "broken" way which
> should be avoided.)

I don't believe I have read such a claim from his e-mails anywhere, so
not sure what this is about.

> The INSTALL_MASK is an independent thing which for _many_
> packages "by accident" can be used with a similar effect as
> setting LINGUAS (because many packages use LINGUAS only to
> determine the files which they install).
> However, it is a different thing, and for some packages
> it is not a replacement at all.

I agree that INSTALL_MASK is a different thing, and that's why I don't
think we need to do step 1) in his masterplan; we can do that in
parallel and push out later, as a means to at least filter the specific
language files easily when installing a binary package which includes
all translations (was built with LINGUAS unset to not filter any with
autotools; or a superset of what you need on install on that machine).

> For instance, the default language of mplayer will depend on
> the first entry of LINGUAS, independent of INSTALL_MASK.
> That's why I suggested that  INSTALL_MASK should actually
> not be discussed at all in this context.

This is not the case since long ago as far as I look, as it was a bug
and fixed long ago.
USE_EXPAND can not express any ordering, portage probably sorts the
list alphabetically too.
LINGUAS as used upstream does not imply any order either.
Runtime locale environment variables like LC_MESSAGES are supposed to
affect what the "default" language is at startup; we are not windows.


Mart