1 |
On Thu, Oct 18, 2012 at 11:54:21PM +0200, Micha?? G??rny wrote: |
2 |
> On Thu, 18 Oct 2012 02:15:43 -0700 |
3 |
> Brian Harring <ferringb@×××××.com> wrote: |
4 |
> |
5 |
> > There's a trick to this; currently, those generated scripts hardcode |
6 |
> > the allowed/known python versions for that package. We obviously have |
7 |
> > to preserve that; I propose we shove it into the symlink path. |
8 |
> > |
9 |
> > Basically, we add a /usr/libexec/python directory; within it, we have |
10 |
> > a wrapper binary (explained below), and a set of symlinks pointing at |
11 |
> > the root of that directory. To cover our current python versions, the |
12 |
> > following would suffice: |
13 |
> > |
14 |
> > for x in {2.{4,5,6,7},3.{0,1,2,3,4}}-cpy 2.5-jython 2.7-pypy-1.{7,8} |
15 |
> > \2.7-pypy-1.9; do |
16 |
> > ln -s ./ /usr/libexec/python/$x |
17 |
> > done |
18 |
> > |
19 |
> > While that seems insane, there is a reason; via that, we can encode |
20 |
> > the allowed versions into the symlink. Using pkgcore's pquery for |
21 |
> > example (which should support cpy: 2.5, 2.6, 2.7, 3.1, 3.2, 3.3) |
22 |
> > instead of a wrapper script at /usr/bin/pquery, we'd have thus: |
23 |
> > |
24 |
> > targets=( 2.{5,6,7}-cpy 3.{1,2,3}-cpy ) |
25 |
> > targets=$(IFS=/;echo -n "${targets[*]}") |
26 |
> > # This results in |
27 |
> > # targets=2.5-cpy/2.6-cpy/2.7-cpy/3.1-cpy/3.2-cpy/3.3-cpy |
28 |
> > ln -s "/usr/libexec/python/${targets}/wrapper" \ |
29 |
> > /usr/bin/pquery |
30 |
> > |
31 |
> > /usr/libexec/python/wrapper upon invocation, takes a look at argv[0]; |
32 |
> > sees how it was invoked basically. This will be the /usr/bin/whatever |
33 |
> > pathway. It reads the symlink, in the process getting the allowed |
34 |
> > versions and preferred order of the versions. |
35 |
> > |
36 |
> > Few notes; vast majority of filesystems will store the symlink target |
37 |
> > into the core inode if at all possible- in doing so, this avoids |
38 |
> > wasting an inode and is only limited by the length of the target. |
39 |
> > That length is capped by PATH_MAX- which can range from 256 to 4k (or |
40 |
> > higher). |
41 |
> > |
42 |
> > For the pquery example above, that comes out to ~73 bytes for the |
43 |
> > symlink pathway; well under PATH_MAX. |
44 |
> > |
45 |
> > For the scenarios where PATH_MAX caps the symlink pathway, or for |
46 |
> > whatever reason we don't want to use that trick, a tree of files |
47 |
> > contained within /usr/libexec/python/ holding the allowed versions for |
48 |
> > the matching pathway would suffice. |
49 |
> |
50 |
> While I agree that it's a clever trick, I doubt it's worth the effort. |
51 |
> Did you got any numbers proving it being superior over, say, trying to |
52 |
> exec() scripts like I do in python-exec? |
53 |
> |
54 |
> While I can imagine, that in an worst case that bunch of exec()s is |
55 |
> going to be definitely slower than storing the list anyway, I doubt |
56 |
> such a bad case is often. |
57 |
|
58 |
The difference in performance there is going to be negligable; I'm not |
59 |
particularly concerned about that, and it shouldn't be a debate point |
60 |
between my notion and yours (it's only a debate point when one is |
61 |
talking about a c binary vs a python script). |
62 |
|
63 |
One thing you're ignoring here is that the route(s) I mentioned all |
64 |
allow for control/specifying what the order of preference is for |
65 |
lookup/fallback (each link/shebang can encode that order how ever the |
66 |
hell it wants). |
67 |
|
68 |
|
69 |
> Considering that the most common Python version used now is Python 2, |
70 |
> how often doesn't the script support that Python version? That's a very |
71 |
> rare case, and often just executing "foo-${EPYTHON}" works. In your |
72 |
> case, that common case involves readlink() + parsing + exec(). |
73 |
> |
74 |
> Even in case of Python 3 being selected, I doubt the overhead |
75 |
> of multiple exec()s on the scripts not supporting Python 3 is really |
76 |
> relevant. Please measure it if you believe so. |
77 |
> |
78 |
> To be honest, I don't see any real advantage in this solution. It is |
79 |
> complex; understanding it requires explanation or some thinking. |
80 |
> The code will be fragile, and I'm not even sure if I'm not missing |
81 |
> something important here. |
82 |
|
83 |
Not particularly sure how you claim this is fragile, but whatever, |
84 |
your view. |
85 |
|
86 |
I suggest you read my response to floppym. There's differing ways to |
87 |
tackle this while getting the required speed, and covering what I want |
88 |
(python3.2 sphinx-build working). |
89 |
|
90 |
|
91 |
> Thus, unless you've got a good arguments how this solution is superior |
92 |
> to the straightforward one done in python-exec, or numbers proving that |
93 |
> it is more efficient in a way noticeable to our users, -1. Smart hack, |
94 |
> yes, but not really beneficial. |
95 |
|
96 |
Your -1 is duly noted. |
97 |
|
98 |
|
99 |
> > Either proposal here would be far faster than what we've got now; also |
100 |
> > will use less space (ancillary benefit). |
101 |
> > |
102 |
> > One subtle aspect here is that if we did this, it makes it possible to |
103 |
> > avoid losing the invocation information- currently if you did |
104 |
> > `/usr/bin/python3.2 $(which sphinx-build) blah`, because of how things |
105 |
> > are implemented now (specifically the two layers of wrappers)- you'll |
106 |
> > get python2.7 running that sphinx-build invocation. |
107 |
> > |
108 |
> > This is wrong (it's directly discarding what the invocation |
109 |
> > requested), although you're only going to see it for scripts that |
110 |
> > do python introspection. |
111 |
> > |
112 |
> > Via doing the restructuring I'm mentioning above, that issue can be |
113 |
> > fixed, while making things faster/saner. |
114 |
> |
115 |
> I don't see how this is relevant to the wrapper. As Mike pointed out, |
116 |
> python3.2 is the actual Python executable, and the wrapper you're |
117 |
> suggesting is a C executable -- something that does not really work |
118 |
> together like that. |
119 |
> |
120 |
> So please elaborate on how you are actually going to solve this. Hope |
121 |
> it's not through patching Python... |
122 |
|
123 |
Same thing, read my responses to floppym; not going to reiterate those |
124 |
points over here. |
125 |
|
126 |
I will note your python-exec bit still doesn't fly for python3.2 |
127 |
sphinx-build- which frankly, is the wrong direction to go in my views. |
128 |
|
129 |
~harring |