Re: [gentoo-python] Re: [PATCH eselect-python 1/2] Store per-version interpreter preference in a file as well.

[Brian Harring]

On Thu, Nov 01, 2012 at 10:27:19PM +0100, Micha?? G??rny wrote:
> On Thu, 1 Nov 2012 14:48:55 -0400
> Mike Gilbert <floppym@g.o> wrote:
>
> > On Thu, Nov 1, 2012 at 11:44 AM, Mike Gilbert <floppymaster@×××××.com> wrote:
> > > On Thu, Nov 1, 2012 at 7:54 AM, Micha?? G??rny <mgorny@g.o> wrote:
> > >> The idea is very simple: /etc/env.d/python/python[23] with a one-line
> > >> value similar to the main interpreter /config file.
> > >>
> > >> That should be simpler & more reliable than reading a symlink. And at
> > >> some point we can replace the symlink with an $EPYTHON-aware wrapper
> > >> as well.
> > >
> > > I don't understand the point of this. Do we have some need to enable
> > > EPYTHON usages for scripts that have a python2 or python3 shebang?
> > >
> > > I also don't understand how a text file is more reliable than a
> > > symlink; they are basically the same thing, but the symlink has a
> > > different file mode.
>
> Ah, and I'd forget. I have the following dream:
>
>   /etc/env.d/python/python2
>   /etc/env.d/python/python3
>   /etc/env.d/python/config -> python2
>
> So, python{2,3} keep the per-version interpreters, and config is just
> a symlink to one of them. Two bird with one stone -- readlink to get
> which group is enabled, cat to get the exact interpreter. I'm proud
> of myself!

I too have a dream; git://pkgcore.org/eslect-python .

That's a shebang based version of what I proposed a while back.  It 
works now and has tests.  It's written to be basically a drop in for 
existing python.eclass generation of shebangs, w/ the 
hardlinking/de-duplication/farking-fast/fix python3.2 
/usr/bin/sphinx-build scenario.

The remaining work for that is thus:

1) If the EPYTHON targets aren't given via shebang arg, then it needs
 to fallback to grabbing the targets from the file (easy enough).

2) Add a few helpers/wrappers to make it easier to do the 
deduplication/shebang rewriting.

Frankly, you should be looking at this imo, rather than trying 
standalone files.  Yes, files is simpler- but you'll wind up sooner or 
later rebuilding it into what I already built out here.

Either way, will let the commit message speak for it:

"""
Add a python-shebang utility, slave python-wrapper to it.

Going forward, the intent is to push the PYTHON_ABIs of a given
script down into it's shebang, pointing the shebang at python-shebang.

In this way, all known/supported abi's are available at the time of
execution; further, if the target is told to respect-EPYTHON (meaning
no searching for something to execute, either active python version or
EPYTHON var), we can push this down into that list and have that code
handle it.

Using sphinx-build as an example, we go from the current setup of:
echo > sphinx-build-2.7 <<EOF
#!/usr/bin/python2.7
<code>
EOF

echo > sphinx-build-3.2 <<EOF
#!/usr/bin/python3.2
<code>
EOF

echo > sphinx-build << EOF
#!/usr/bin/env python
<python wrapper code to decide which of the above to execute>
EOF

to the simpler form of:
for x in sphinx{,-{2.7,3.2}}; do echo > $x <<EOF
#!/usr/bin/python-shebang python2.7,python3.2
<code>
EOF
done

The gains of this are thus:

1) This simplifies the pypy EPYTHON->script_name parsing code; since
 we have the list of PYTHON_ABIs available at the script level, we
 can use that info to map it directly (thus we don't have to update
 that code till pypy changes their interp name).

2) Measurably faster implementation; minimally, via moving this to
 c we're not paying the VM overhead that the current wrapper solution
 requires; further, via accessing the underlying python config file,
 we can get the active version without spawning bash (the python.eclass
 wrapper solution semi-validly doesn't do this since the given wrapper
 isn't part of eselect-python; python-shebang is, thus we can do the
 faster path).  The code will still fallback to eselect python show as
 a protective measure, but that pathway should never be executed
 realistically.

3) Via this approach, we can do de-duplication of the versioned
 scripts; since each (including the versioned forms) carry the
 python ABIs as an argument, their content is identical- so we can just
 hardlink them together (which the code looks for and exploits to avoid
 a double exec, instead exec'ing directly to the target).

4) The underlying python-wrapper in turn, just becomes a dumb re-exec
 to `python-shebang ACTIVE_PYTHON`- a special mode of python-shebang,
 which is essentially respect-EPYTHON, while defaulting EPYTHON to
 the active python version.  Single implementation basically.

5) java-config-2's jython cycle can be explicitly broken by the script itself
 now; since the script now specifies the ABIs it targets, java-config-2's shebang
 just would drop jython2.5 from the list of targets.  Via this, on a system that
 has jython2.5 as the active interp, when it goes to invoke java-config-2 (a necessary
 step to actually start the jython interp itself) it would use an implementation other
 than jython2.5- no fork bomb/exec cycle.

6) python3.2 sphinx-build now does the correct thing- execute sphinx-build-3.2, even
 if the active is python2.7.
"""