1 |
On Wed, Mar 7, 2018 at 11:52 AM, Alec Warner <antarus@g.o> wrote: |
2 |
> On Wed, Mar 7, 2018 at 11:51 AM, Michael Orlitzky <mjo@g.o> wrote: |
3 |
>> |
4 |
>> On 03/07/2018 11:06 AM, anoteros@××××××.io wrote: |
5 |
>> > Why should portage download some outdated second copy of the |
6 |
>> > sources for 'bar', rebuild it, and scatter it around the file system |
7 |
>> > where it cannot be used by other programs installed by cabal? |
8 |
>> |
9 |
> |
10 |
> I'm really not happy with the tone of this email, so I'm going to comment on |
11 |
> it a bit. |
12 |
> |
13 |
|
14 |
I can't help but agree with Mr. Orlitzky's sentiment. All language |
15 |
package managers suffer from the same sophomoric problems: |
16 |
|
17 |
1) I usually don't know where things are downloaded from. |
18 |
2) I can't integrate these changes with my distrbution (Gentoo, |
19 |
Ubuntu, Debian, Fedora, CentOS) safely without serious work. |
20 |
3) I can't figure out easily what dependencies a package has. Usually |
21 |
I see if there are compile or runtime errors. Sometimes the |
22 |
dependencies are listed somewhere. If the dependency is not what is |
23 |
currently in e.g. Ubuntu's repository, I may have to maintain separate |
24 |
versions to be compatible. |
25 |
4) Sometimes they aren't set up to be built at all. Let the magic |
26 |
package manager do everything for you. This works, except when your |
27 |
shared objects are not in the right places. (But it makes me feel |
28 |
dirty.) |
29 |
|
30 |
>> |
31 |
>> These other package managers don't solve any hard problems -- they're |
32 |
>> basically a fancy interface around wget and "git clone." Portage on the |
33 |
>> other hand has ~20 years of good ideas and hard work on the hard |
34 |
>> problems in package management. For example... |
35 |
> |
36 |
> |
37 |
> Portage is also full of not-good ideas; many of these we papered over with |
38 |
> PMS and EAPI to make |
39 |
> the actual API people use less horrific. Lets not preach from our ivory |
40 |
> tower here. |
41 |
> |
42 |
|
43 |
The magnitude of "not good" is, I would suggest, very different. |
44 |
|
45 |
Cabal is a pretty hilarious example. Have you ever tried to build it |
46 |
without using the release binaries? I suppose this is a second problem |
47 |
though, where people want to be "self-reliant" and instead just end up |
48 |
making things impossible to verify or make reproducible. |
49 |
|
50 |
For the longest time Cabal did not authenticate or verify the code it |
51 |
would run (as root). Very recently this was fixed, but I still feel |
52 |
bad any time I let it run, even if it's on a separate development |
53 |
system. |
54 |
|
55 |
>> |
56 |
>> > It seems reasonable to me to 'hook' portage into these other package |
57 |
>> > managers, so that running 'emerge bar' would actually run 'cabal install |
58 |
>> > bar' |
59 |
>> |
60 |
>> Can "cabal install" build or even identify the C libraries that your |
61 |
>> Haskell package needs? No, because nobody ever thought of that, and it |
62 |
>> seems kind of hard now that the cabal build system has no ability to |
63 |
>> build non-Haskell packages, so no one is ever going to work on it. |
64 |
>> |
65 |
>> Can "cabal install" rebuild your Haskell packages when the ABI of some |
66 |
>> library changes? No, because "cabal install" doesn't have any idea |
67 |
>> what's installed on your system. |
68 |
>> |
69 |
>> Can "cabal install" uninstall a package? Nope, it has no idea what was |
70 |
>> done during the installation, and thus no idea what to undo. |
71 |
>> |
72 |
>> Can "cabal install" verify the integrity of your downloaded source code? |
73 |
>> No, because by design it fetches and runs code from complete strangers. |
74 |
>> |
75 |
>> Can "cabal install" use a local tarball to function without network |
76 |
>> access? Etc. We're dead in the water. |
77 |
>> |
78 |
>> Every other language-specific package manager has the same problems, |
79 |
>> because they're all written by people who didn't know anything about |
80 |
>> package management and then got bored when they realized that there's |
81 |
>> more to it than parsing a json file of dependencies. |
82 |
>> |
83 |
> |
84 |
> They are written by people who are not you, who have different problems than |
85 |
> you and often don't care about the above use cases. |
86 |
> It turns out this stuff exists because: |
87 |
> |
88 |
> 1) Upstream wants to push 1 single thing and have it work in all distros. |
89 |
> 2) pip / virtualenv / cargo / whatever work reasonable well. |
90 |
> 3) Rolling-based distros couldn't keep up with packaging. |
91 |
> 4) Snapshot-based distros (debian stable / ubuntu) were not designed with |
92 |
> this in mind as much; because packages were developed with a high velocity. |
93 |
> |
94 |
|
95 |
1) What do you mean by this? Distributions are usually not binary |
96 |
compatible. This "works" by having each distribution customize and |
97 |
build a project by hand. |
98 |
2) Virtualenv works well, and cabal now has a local installation |
99 |
option. Still, these are not perfect. |
100 |
3) Use a git ebuild, or target stable versions. If a project has so |
101 |
much churn that I can't keep up with it I will find something more |
102 |
stable. |
103 |
4) True, but prefix would be a lot better at fixing that problem. If |
104 |
not that, something like virtualenv. |
105 |
|
106 |
In the case of either #3 or #4, the distribution developers prefer you |
107 |
use their package manager to install packages. You are only safe doing |
108 |
anything else in an environment like virtualenv, which does not exist |
109 |
for the vast majority of languages. This is why developers will pass |
110 |
around VM images, or devote an entire VM to development on a project. |
111 |
Languages and their packages are not designed to be compartmentalized |
112 |
and will trash your installation. |
113 |
|
114 |
There is also (going along with #4): |
115 |
5) The software may need to work on Windows. They could have used |
116 |
Cygwin or MSYS2. Anything else is an exercise in futility. People have |
117 |
come before you and done it better. |
118 |
|
119 |
Taken together these issues boil down to "the people rewriting package |
120 |
managers don't realize the problems they actually have" which agrees |
121 |
with what was said. As above, the typical workflows of these people |
122 |
involve lots of VMs or containers. Ask yourself "why?" |
123 |
|
124 |
> |
125 |
>> |
126 |
>> If you want to eliminate the duplication of effort, tell these people to |
127 |
>> use Gentoo Prefix instead of writing the N+1st crippled PM. Doing it the |
128 |
>> other way around won't work because we'd be replacing one good thing |
129 |
>> with 75 shitty things. |
130 |
> |
131 |
> |
132 |
> I agree that in theory they could have published ebuilds for Gentoo prefix |
133 |
> and it would have 'worked everywhere' but I think that boat sailed about 10 |
134 |
> years ago. |
135 |
> |
136 |
> https://wiki.gentoo.org/wiki/Project:Perl/g-cpan is a project is in a |
137 |
> similar space and basically reads perl CPAN metadata to generate stub |
138 |
> ebuilds. |
139 |
> Portage tracks these stub ebuilds (and so for example, it tracks what these |
140 |
> cpan packages install and can remove them afterwards.) |
141 |
> |
142 |
> I think this is the most pragmatic approach I've seen used as its mostly an |
143 |
> adapter (to cpan) that just generates ebuilds. |
144 |
> Its plausible that with some careful eclass magic you might be able to make |
145 |
> the installed packages compatible with pip, cargo, etc. |
146 |
> |
147 |
> I think its more of a struggle to make it compatible with things like |
148 |
> virtualenv or pip --user though. |
149 |
> |
150 |
|
151 |
This might be a good way to relieve the amount of intervention |
152 |
required when repackaging code for an actual package manager. The |
153 |
information should be there. The other option is convincing people to |
154 |
package for multiple systems at once, which diffuses the effort to the |
155 |
point people tend to not mind. |
156 |
|
157 |
1) Language package manager (usually used by Windows consumers). |
158 |
2) .debs for Ubuntu/Debian. |
159 |
3) .rpms for Fedora/CentOS. |
160 |
4) Sometimes there's a Gentoo or Arch release. |
161 |
|
162 |
If you could sell #4 as a way to generate 1-3 it would likely be |
163 |
possible to reduce the proliferation of language specific package |
164 |
managers over time. Prefix would likely play an important role. |
165 |
|
166 |
Cheers, |
167 |
R0b0t1 |