[gentoo-catalyst] Python 3 support

[Matthew Ruffalo]

Hi all-

First, thanks for the great tool. I've been using Gentoo for about 10 
years, and I use Catalyst to build custom Gentoo live CDs that contain 
extra software (e.g. netcat6, rsync, ddrescue, etc.).

At some point in the past the stage3 tarballs only contained Python 3, 
though it looks like the 20130516 amd64 stage3 contains both 2.7 and 
3.2. It's reasonable to assume that any current Gentoo system has a 
Python 3.2+ interpreter, so is there any objection to making Catalyst 
work on Python 3? The fewer things in the world that depend on Python 2, 
the better (in my opinion). Portage has supported Python 3 for some time 
and it makes sense for Catalyst to follow suit.

I use Python 3 for all of my own work and have a reasonable amount of 
experience porting projects to Python 3. This typically involves a lot 
of code churn even when there aren't any logic changes -- every print 
statement has to be converted to calling the print function, for 
instance. Explicit conversion between bytes and str can require a bit 
more thought but isn't difficult.

This conversion might require other miscellaneous cleanup, including 
some whitespace-only commits. Python 3 does not allow mixing tabs and 
spaces for indentation, and (e.g.) catalyst_support.py is a *mess* with 
indentation. PyCharm also notices things like unnecessary "\" line 
continuation characters, unused variables, mutable default arguments 
(e.g. catalyst_support.spawn's "env"), equality comparisons against 
None, and unreachable code (lines 183 - 220 of catalyst_lock.py).

It looks like the project has mostly standardized on using tabs for 
indentation, so that's what I'll use when changing indentation is necessary.

In any case, over the next few weeks/months I'll prepare a series of 
patches in decreasing order of priority:

 1. Changes required to run 2to3, like changing string exceptions (e.g.
    catalyst_lock.py:34) to 'raise Exception("message")'
 2. Automated 2to3 conversion
 3. Minimal indentation changes required for Python 3 to parse the code
    (as I mentioned, the diff of catalyst_support.py will still be quite
    large)
 4. Any required bytes/str fixes and other things that 2to3 isn't smart
    enough to do on its own
 5. Miscellaneous style tweaks (some of which are suggested by PEP 8), like:
     1. using os.path.join where appropriate (e.g. grp_target.py:38,
        catalyst_lock.py:80)
     2. using context managers ("with open(filename) as f:")
     3. replacing mutable default function arguments
     4. using argparse instead of getopt in the main catalyst script (maybe)
     5. catching Exception instead of using a raw 'except:' -- probably
        don't want to catch SystemExit, StopIteration and
        KeyboardInterrupt in a lot of these places
 6. Other PEP 8-inspired changes like using spaces around operators and
    function arguments

The intention is that categories 1 through 4 would be the minimum set of 
changes required to make Catalyst run on Python 3. Categories 5 and 6 
would be a mix of PEP 8 and newer Python idioms and applying these would 
be optional.

I'm not a Gentoo developer and I'll be making a lot of small commits, so 
sharing my changes will probably be easiest by publishing a clone of the 
Catalyst repository on GitHub. (I plan to make/test/release all of these 
changes whether or not they'll make it in to Catalyst in the near 
future. Even if the Gentoo releng team doesn't care about supporting 
Python 3 now, having a working port available might make it easier to 
justify that change in the future :) )

Thanks for your time and any feedback,
MMR...
-- 
Matthew Ruffalo
Department of Electrical Engineering and Computer Science
Case Western Reserve University