Gentoo Archives: gentoo-portage-dev

From: Thomas Horsten <thomas@×××××××.com>
To: gentoo-portage-dev@l.g.o
Subject: [gentoo-portage-dev] emerge feature request: Downloads managed by lock file system
Date: Sat, 07 Feb 2004 13:18:53
Message-Id: 200402071318.47224.thomas@horsten.com
1 Hi,
2
3 I have a request for an emerge feature that I think would be useful for a lot
4 of people.
5
6 I have quite a fast machine with RAID and 3GHz CPU, and only a 512kbit/s
7 Internet connection. When I do an "emerge -u world" most of the time actually
8 goes by waiting for packages to download. For most ebuilds, It takes roughly
9 the same time to download as to compile.
10
11 So to speed up my builds, I usually start an "emerge -f -u world". Once the
12 first couple of packages have downloaded I then start an "emerge -u world" in
13 another shell.
14
15 This works great as long as the fetcher process is ahead of the builder
16 process. But if some packages are built too quickly, the builder process
17 catches up. It then starts downloading the same file as the downloader
18 process, resuming from wherever the downloader had gotten to, so now two
19 downloaders are writing to the same file (waste of bandwidth and a possible
20 source of corrupted archives).
21
22 What I suggest is that emerge will put a lockfile in place when it's
23 downloading a file, eg. "name-of-package.tar.bz2.lock", or by using fcntl
24 locks. Then if the "builder" catches up, and finds the output file is already
25 open, it waits on the lock for the download to complete and then continues
26 unpacking after that.
27
28 I already love gentoo and the portage system, it's by far the best available,
29 but adding this feature would make it even better IMHO.
30
31 Regards,
32
33 Thomas
34
35
36 --
37 gentoo-portage-dev@g.o mailing list

Replies