Gentoo Archives: gentoo-dev

From: Brian Harring <ferringb@×××××.com>
To: gentoo-dev@l.g.o
Subject: Re: [gentoo-dev] Locale check in python_pkg_setup()
Date: Fri, 30 Jul 2010 18:47:44
Message-Id: 20100730184518.GA32513@hrair
In Reply to: Re: [gentoo-dev] Locale check in python_pkg_setup() by "Paweł Hajdan
On Fri, Jul 30, 2010 at 09:49:21AM -0700, "Paweee Hajdan, Jr." wrote:
> On 7/29/10 8:48 PM, Brian Harring wrote: > > It's basically annoying people into changing to partially > > sidestep a couple of bugs, instead of fixing the issue- and that's the > > wrong course of action. > > I think that with python earlier than python-3 unicode handling is quite > complicated, and I'm not surprised there are problems with that.
encoding handling wasn't that bad under py2k. Py3k just enforces the boundaries- meaning you can't just skid by.
> Arfrever, does python-3 have the same problem with non-UTF8 locales?
ascii is a subset of utf-8 and ascii is a subset of latin-1; latin-1 and utf-8 aren't compatible in encoded form however. What this means is that the same set of bugs I ran down still will go boom if you have a utf-8 locale and the code in question was dealing w/ a latin-1 encoded file.
> Another thing we can consider is making UTF8 the default setup in > Gentoo. I think most people (including me) don't care whether it's C or > UTF8 as long as it works.
"as long as it works" in this case means "fix the code" as I've laid out. Forcing locale's to sidestep it leaves the latin-1/utf8 incompatibility to go 'boom'. Basically, forcing utf8 doesn't "make it work". It reduces the cases breakage will show up while leaving those issues still there- frankly this is worse, can't fix those screwups without them breaking (for better or worse, and preferably breaking in a testcase). We've got 4 bugs, and only one of them is semi complex fix (dodcutils needs to require that html it's fed is utf8 compatible- valid enough req anyways since html shouldn't be latin-1, it should be ascii or utf8). So.. get fixing, instead of dodging the work imo. ;) ~brian