Re: [gentoo-portage-dev] [PATCH 2/2] pym/portage/util/locale.py: add a C module to check locale

[Michał Górny]

On Fri, 20 May 2016 06:59:14 -0400
"Anthony G. Basile" <basile@××××××××××××××.edu> wrote:

> On 5/19/16 9:38 AM, Michał Górny wrote:
> > Dnia 19 maja 2016 14:43:38 CEST, "Anthony G. Basile" <basile@××××××××××××××.edu> napisał(a):
> >> From: "Anthony G. Basile" <blueness@g.o>
> >>
> >> The current method to check for the system locale is to use python's
> >> ctypes.util.find_library() to construct a full library path to the
> >> system libc.so which is then passed to ctypes.CDLL().  However,
> >> this gets bogged down in implementation dependant details and
> >> fails with musl.
> >>
> >> We work around this design flaw in ctypes with a small python module
> >> written in C called 'portage_c_check_locale', and only fall back on
> >> the current ctypes-based check when this module is not available.
> >>
> >> This has been tested on glibc, uClibc and musl systems.
> >>
> >> X-Gentoo-bug: 571444
> >> X-Gentoo-bug-url: https://bugs.gentoo.org/show_bug.cgi?id=571444
> >> Signed-off-by: Anthony G. Basile <blueness@g.o>
> >
> > To be honest, I don't like this. Do we really want to duplicate all this, including duplicating the complete messages? This really looks messy and unmaintainable.
> >
> > The only reason libc functions were being used was to workaround the hacky built-in case conversion functions. And this is the only part where CDLL was really used.
> >
> > So please change this to provide trivial wrappers around the C library functions, and use them alternatively to ones obtained using CDLL. With a single common code doing all the check logic and messaging.
> >
> >
>
> ctypes is itself a problem and so hacky python built-ins were replaced
> by more hacky python.  CDLL shouldn't be used at all.  The other problem
> is that non utf-8 encodings cause problems much earlier in the portage
> codebase than the test for a sane environment, which is a bigger problem
> than this small issue.  No one checked what happens with python2.7 +
> portage + exotic locale.  Since I don't know where this might head in
> the future, I kinda like the standalone potential.  The only repeated
> code here is the message.  Nonetheless, I can reduce this to just two
> functions, and do something like the following.  I assume that's what
> you're suggesting:
>
> try:
> 	from portage_c_convert_case import _c_toupper, _c_tolower
> 	libc_toupper = _c_toupper
> 	libc_lolower = _c_tolower
> except ImportError:
> 	libc_fn = find_library("c")
> 	if libc_fn is None:
> 		return None
> 	libc = LoadLibrary(libc_fn)
> 	if libc is None:
> 		return None
> 	libc_toupper = libc.toupper
> 	libc_tolower = libc.tolower
>
>
> Incidentally, another approach, one that I use in bash is as follows.  I
> think it requires bash4, and I'm not sure how to pull this into python
> gracefully.
>
> l="abcdefghijklmnopqrstuvwxyz"
> u="ABCDEFGHIJKLMNOPQRSTUVWXYZ"
> ru=${l^^}
> rl=${u,,}
> [[ $l == $rl ]] && echo "lower case okay" || echo "lower case bad"
> [[ $u == $ru ]] && echo "upper case okay" || echo "upper case bad"

Exactly as pointed out above. You have to import tolower()
and toupper() from libc as Python case conversion functions don't give
the same results as 'pure' libc used by bash.

-- 
Best regards,
Michał Górny
<http://dev.gentoo.org/~mgorny/>