Gentoo Archives: gentoo-soc

From: mmacleod@××××××××××.za
To: gentoo-soc@l.g.o
Subject: Re: [gentoo-soc] Improved binary package support
Date: Tue, 24 Mar 2009 07:45:34
Message-Id: 200903240945.31116.mmacleod@webmail.co.za
In Reply to: Re: [gentoo-soc] Improved binary package support by Ethan Hall
1 > Would getting hashes to agree be a hard thing to accomplish?
2 There are probably some packages that will give difficulty, and a few things
3 to consider but it should not be unreasonably hard.
4 > If you use
5 > the binary build feature in portage you could save the build in a
6 > compressed format.
7 This is the idea yes, to improve the binary feature of portage and then take
8 advantage of it as part of this system.
9 > Then have a hash generate set up where it would take
10 > the name, version, and use flags, cflags and hash just that information.
11 We are talking about two different types of hashes.
12 There would be a hash in the package names in order to tell the difference
13 between package foo compiled with use flag "bar" and package foo compiled
14 without useflag "baa"(It would also have to take into account cflags and
15 dependency versions), this is part of the "improved binary support idea".
16
17 The second kind of hash that I am talking about now is a security hash
18 computed over the final package file. By having multiple users compile the
19 package and generate a security hash of it one can ensure(within reasonable
20 doubt) that the package has not been tampered with by the contributor, by for
21 example adding a rootkit to the source code.
22 > Have a program that would run as a deamond on compter it could update
23 > themselves to something like a torrent tracker. The torrent name could the
24 > the hashed values. Then we could have the data that would need to be
25 > installed passed via torrent to all other users. It would be an interesting
26 > implementation. Also you could use a DC++ idea and have a hub set up where
27 > you could only share compiled programs?
28 This is roughly what I am proposing, except with a bit more security built in
29 to prevent tampering with packages as well as a bit more work required to
30 decide who will host/seed what.
31 > This would make the idea distributed so it would not have to have ~10451G
32 > of space used on any single server. People would host 'packages' as they
33 > compiled them. The more comon the program was, the faster the downloads
34 > would be.
35 Yes, except there are security concerns so its a bit more complex then just
36 one user compiling something and then everyone using it. There would have to
37 be redundant compiles to prevent tampering, as well as tracking of where
38 packages came from.
39 The main concern isn't really fast downloads though but rather distributing
40 the large quantity of hard drive usage required and the CPU time of compiles.
41 > Another use would be releasing common programs/sets as packages? Example,
42 > Xorg. There are tons of programs that go along with it. If you host
43 > commonly used programs on a service then people could download the program
44 > and use it while they compiled it with their own use flags. We could offer
45 > programs with all flags enabled, the package manager would go get the
46 > prebuild program, keep track of all the files and then start the custom
47 > build according to the users own settings. After the build is done, before
48 > it is moved from its sandbox, purdge the old files from the downloaded,
49 > pre-compiled code, and then move the files from the sand box.
50 The idea is that binaries of as many packages with as many different use
51 flag/cflag etc.. combinations would be available as possible. It would be self
52 regulating in that the more people who use a particular use flag/cflag
53 combination the more likely a binary will already exist so the more popular
54 stuff will always have binaries available first. If you use some really wacky
55 cflag combination then you can't really expect that binaries will be
56 available.
57
58 People will be able to use these binaries with a normal emerge command
59 provided they enable support for it...
60
61 So emerge Xorg would download the appropriate Xorg binary if one existed, if
62 there was not one it would instead compile it as normal.

Replies

Subject Author
Re: [gentoo-soc] Improved binary package support Philipp Riegger <lists@××××××××××××.de>