1 |
On Wed, Mar 7, 2018 at 1:15 PM, Alec Warner <antarus@g.o> wrote: |
2 |
> |
3 |
> |
4 |
> On Wed, Mar 7, 2018 at 1:22 PM, R0b0t1 <r030t1@×××××.com> wrote: |
5 |
>> |
6 |
>> On Wed, Mar 7, 2018 at 11:52 AM, Alec Warner <antarus@g.o> wrote: |
7 |
>> > On Wed, Mar 7, 2018 at 11:51 AM, Michael Orlitzky <mjo@g.o> |
8 |
>> > wrote: |
9 |
>> >> |
10 |
>> >> On 03/07/2018 11:06 AM, anoteros@××××××.io wrote: |
11 |
>> >> > Why should portage download some outdated second copy of the |
12 |
>> >> > sources for 'bar', rebuild it, and scatter it around the file system |
13 |
>> >> > where it cannot be used by other programs installed by cabal? |
14 |
>> >> |
15 |
>> > |
16 |
>> > I'm really not happy with the tone of this email, so I'm going to |
17 |
>> > comment on |
18 |
>> > it a bit. |
19 |
>> > |
20 |
>> |
21 |
>> I can't help but agree with Mr. Orlitzky's sentiment. All language |
22 |
>> package managers suffer from the same sophomoric problems: |
23 |
>> |
24 |
>> |
25 |
>> 1) I usually don't know where things are downloaded from. |
26 |
>> 2) I can't integrate these changes with my distrbution (Gentoo, |
27 |
>> Ubuntu, Debian, Fedora, CentOS) safely without serious work. |
28 |
>> 3) I can't figure out easily what dependencies a package has. Usually |
29 |
>> I see if there are compile or runtime errors. Sometimes the |
30 |
>> dependencies are listed somewhere. If the dependency is not what is |
31 |
>> currently in e.g. Ubuntu's repository, I may have to maintain separate |
32 |
>> versions to be compatible. |
33 |
>> 4) Sometimes they aren't set up to be built at all. Let the magic |
34 |
>> package manager do everything for you. This works, except when your |
35 |
>> shared objects are not in the right places. (But it makes me feel |
36 |
>> dirty.) |
37 |
>> |
38 |
>> >> |
39 |
>> >> These other package managers don't solve any hard problems -- they're |
40 |
>> >> basically a fancy interface around wget and "git clone." Portage on the |
41 |
>> >> other hand has ~20 years of good ideas and hard work on the hard |
42 |
>> >> problems in package management. For example... |
43 |
>> > |
44 |
>> > |
45 |
>> > Portage is also full of not-good ideas; many of these we papered over |
46 |
>> > with |
47 |
>> > PMS and EAPI to make |
48 |
>> > the actual API people use less horrific. Lets not preach from our ivory |
49 |
>> > tower here. |
50 |
>> > |
51 |
>> |
52 |
>> The magnitude of "not good" is, I would suggest, very different. |
53 |
>> |
54 |
>> Cabal is a pretty hilarious example. Have you ever tried to build it |
55 |
>> without using the release binaries? I suppose this is a second problem |
56 |
>> though, where people want to be "self-reliant" and instead just end up |
57 |
>> making things impossible to verify or make reproducible. |
58 |
>> |
59 |
>> For the longest time Cabal did not authenticate or verify the code it |
60 |
>> would run (as root). Very recently this was fixed, but I still feel |
61 |
>> bad any time I let it run, even if it's on a separate development |
62 |
>> system. |
63 |
> |
64 |
> |
65 |
> Gentoo just got signature checking enabled by default...in ~arch? I'm not |
66 |
> sure if that version of portage is stable yet. |
67 |
> Like I said, be careful how one preaches from the ivory tower. |
68 |
> |
69 |
|
70 |
webrsync-gpg has been an option for a long time. |
71 |
|
72 |
Package maintenance for Gentoo is not the ivory tower. If anything, |
73 |
the ivory tower is language development, in that tower the language |
74 |
developers are isolated from the wider reaching consequences of their |
75 |
actions. |
76 |
|
77 |
>> |
78 |
>> |
79 |
>> >> |
80 |
>> >> > It seems reasonable to me to 'hook' portage into these other package |
81 |
>> >> > managers, so that running 'emerge bar' would actually run 'cabal |
82 |
>> >> > install |
83 |
>> >> > bar' |
84 |
>> >> |
85 |
>> >> Can "cabal install" build or even identify the C libraries that your |
86 |
>> >> Haskell package needs? No, because nobody ever thought of that, and it |
87 |
>> >> seems kind of hard now that the cabal build system has no ability to |
88 |
>> >> build non-Haskell packages, so no one is ever going to work on it. |
89 |
>> >> |
90 |
>> >> Can "cabal install" rebuild your Haskell packages when the ABI of some |
91 |
>> >> library changes? No, because "cabal install" doesn't have any idea |
92 |
>> >> what's installed on your system. |
93 |
>> >> |
94 |
>> >> Can "cabal install" uninstall a package? Nope, it has no idea what was |
95 |
>> >> done during the installation, and thus no idea what to undo. |
96 |
>> >> |
97 |
>> >> Can "cabal install" verify the integrity of your downloaded source |
98 |
>> >> code? |
99 |
>> >> No, because by design it fetches and runs code from complete strangers. |
100 |
>> >> |
101 |
>> >> Can "cabal install" use a local tarball to function without network |
102 |
>> >> access? Etc. We're dead in the water. |
103 |
>> >> |
104 |
>> >> Every other language-specific package manager has the same problems, |
105 |
>> >> because they're all written by people who didn't know anything about |
106 |
>> >> package management and then got bored when they realized that there's |
107 |
>> >> more to it than parsing a json file of dependencies. |
108 |
>> >> |
109 |
>> > |
110 |
>> > They are written by people who are not you, who have different problems |
111 |
>> > than |
112 |
>> > you and often don't care about the above use cases. |
113 |
>> > It turns out this stuff exists because: |
114 |
>> > |
115 |
>> > 1) Upstream wants to push 1 single thing and have it work in all |
116 |
>> > distros. |
117 |
>> > 2) pip / virtualenv / cargo / whatever work reasonable well. |
118 |
>> > 3) Rolling-based distros couldn't keep up with packaging. |
119 |
>> > 4) Snapshot-based distros (debian stable / ubuntu) were not designed |
120 |
>> > with |
121 |
>> > this in mind as much; because packages were developed with a high |
122 |
>> > velocity. |
123 |
>> > |
124 |
>> |
125 |
> |
126 |
> This == upstream has a git tag (release X) and they sync the tag into pip |
127 |
> and they tell people to pip install X and their job is done. |
128 |
> Software, distributed. |
129 |
> |
130 |
>> |
131 |
>> 1) What do you mean by this? Distributions are usually not binary |
132 |
>> compatible. This "works" by having each distribution customize and |
133 |
>> build a project by hand. |
134 |
> |
135 |
> |
136 |
> Right, and because "distributions" move a different speeds (Ubuntu, Debian, |
137 |
> Gentoo) and have different branches (testing, stable, unstable, ~arch, |
138 |
> pmask) |
139 |
> this is all even *more* overhead for developers to worry about. So they use |
140 |
> pip, and they don't (worry about it.) |
141 |
> |
142 |
|
143 |
At some level pip "works" but the solution to more overhead seems to |
144 |
have been more overhead. |
145 |
|
146 |
> |
147 |
>> |
148 |
>> 2) Virtualenv works well, and cabal now has a local installation |
149 |
>> option. Still, these are not perfect. |
150 |
>> 3) Use a git ebuild, or target stable versions. If a project has so |
151 |
>> much churn that I can't keep up with it I will find something more |
152 |
>> stable. |
153 |
> |
154 |
> |
155 |
> I don't think Ubuntu ships git-ebuilds (because it doesn't have ebuilds) so |
156 |
> I'm stuck consuming some PPA (either one that upstream builds, or some shady |
157 |
> PPA run by who knows who that build my binaries.) |
158 |
> |
159 |
> Easier to pip install! |
160 |
> |
161 |
|
162 |
Pip may not work, or you may need conflicting versions of something |
163 |
installed. It doesn't solve these issues. |
164 |
|
165 |
>> |
166 |
>> 4) True, but prefix would be a lot better at fixing that problem. If |
167 |
>> not that, something like virtualenv. |
168 |
>> |
169 |
>> In the case of either #3 or #4, the distribution developers prefer you |
170 |
>> use their package manager to install packages. You are only safe doing |
171 |
>> anything else in an environment like virtualenv, which does not exist |
172 |
>> for the vast majority of languages. This is why developers will pass |
173 |
>> around VM images, or devote an entire VM to development on a project. |
174 |
>> Languages and their packages are not designed to be compartmentalized |
175 |
>> and will trash your installation. |
176 |
> |
177 |
> |
178 |
> Containers are certainly awesome, yep. |
179 |
> |
180 |
>> |
181 |
>> There is also (going along with #4): |
182 |
>> 5) The software may need to work on Windows. They could have used |
183 |
>> Cygwin or MSYS2. Anything else is an exercise in futility. People have |
184 |
>> come before you and done it better. |
185 |
>> |
186 |
>> |
187 |
>> Taken together these issues boil down to "the people rewriting package |
188 |
>> managers don't realize the problems they actually have" which agrees |
189 |
>> with what was said. As above, the typical workflows of these people |
190 |
>> involve lots of VMs or containers. Ask yourself "why?" |
191 |
> |
192 |
> |
193 |
> Because containers are awesome and are way easier to use. |
194 |
> |
195 |
> -A |
196 |
> |
197 |
|
198 |
I think you missed my point: Why are they easier to use? |
199 |
|
200 |
Cheers, |
201 |
R0b0t1 |
202 |
|
203 |
>> |
204 |
>> |
205 |
>> > |
206 |
>> >> |
207 |
>> >> If you want to eliminate the duplication of effort, tell these people |
208 |
>> >> to |
209 |
>> >> use Gentoo Prefix instead of writing the N+1st crippled PM. Doing it |
210 |
>> >> the |
211 |
>> >> other way around won't work because we'd be replacing one good thing |
212 |
>> >> with 75 shitty things. |
213 |
>> > |
214 |
>> > |
215 |
>> > I agree that in theory they could have published ebuilds for Gentoo |
216 |
>> > prefix |
217 |
>> > and it would have 'worked everywhere' but I think that boat sailed about |
218 |
>> > 10 |
219 |
>> > years ago. |
220 |
>> > |
221 |
>> > https://wiki.gentoo.org/wiki/Project:Perl/g-cpan is a project is in a |
222 |
>> > similar space and basically reads perl CPAN metadata to generate stub |
223 |
>> > ebuilds. |
224 |
>> > Portage tracks these stub ebuilds (and so for example, it tracks what |
225 |
>> > these |
226 |
>> > cpan packages install and can remove them afterwards.) |
227 |
>> > |
228 |
>> > I think this is the most pragmatic approach I've seen used as its mostly |
229 |
>> > an |
230 |
>> > adapter (to cpan) that just generates ebuilds. |
231 |
>> > Its plausible that with some careful eclass magic you might be able to |
232 |
>> > make |
233 |
>> > the installed packages compatible with pip, cargo, etc. |
234 |
>> > |
235 |
>> > I think its more of a struggle to make it compatible with things like |
236 |
>> > virtualenv or pip --user though. |
237 |
>> > |
238 |
>> |
239 |
>> This might be a good way to relieve the amount of intervention |
240 |
>> required when repackaging code for an actual package manager. The |
241 |
>> information should be there. The other option is convincing people to |
242 |
>> package for multiple systems at once, which diffuses the effort to the |
243 |
>> point people tend to not mind. |
244 |
>> |
245 |
>> 1) Language package manager (usually used by Windows consumers). |
246 |
>> 2) .debs for Ubuntu/Debian. |
247 |
>> 3) .rpms for Fedora/CentOS. |
248 |
>> 4) Sometimes there's a Gentoo or Arch release. |
249 |
>> |
250 |
>> If you could sell #4 as a way to generate 1-3 it would likely be |
251 |
>> possible to reduce the proliferation of language specific package |
252 |
>> managers over time. Prefix would likely play an important role. |
253 |
>> |
254 |
>> Cheers, |
255 |
>> R0b0t1 |
256 |
>> |
257 |
> |