Gentoo Archives: gentoo-portage-dev

From: Zac Medico <zmedico@g.o>
To: gentoo-portage-dev@l.g.o
Subject: Re: [gentoo-portage-dev] [RFC/PATCH] prepstrip/ecompressdir: parallelize operations
Date: Tue, 15 May 2012 00:07:58
Message-Id: 4FB18A3F.9080302@gentoo.org
In Reply to: Re: [gentoo-portage-dev] [RFC/PATCH] prepstrip/ecompressdir: parallelize operations by Mike Frysinger
1 On 05/14/2012 01:10 PM, Mike Frysinger wrote:
2 > On Monday 14 May 2012 15:08:32 Zac Medico wrote:
3 >> Actually, the inode_var_name thing will not work unless it's all in
4 >> one process.
5 >
6 > hmm, true, but that's the level we currently parallelize at, so it's fine. we
7 > do one subprocess per ELF and that includes the strip/splitdebug/splitsources.
8 >
9 > parallelizing more than on a per-ELF basis will require much finer grained
10 > queues which, while possible, would make the file much harder to hack on and
11 > extend. and i'm not sure we'd see that much of a gain.
12 > -mike
13
14 The thing is, in the case of hardlinks, we're parallelizing multiple
15 times on the *same* elf. Anyway, I've fixed it by using a directory full
16 of hardlinks, in these commits:
17
18 http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=ad944275b88a50d2a1f694826b127cceb9221e78
19 http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=9ed00a9e70a3705164a5349145ff467e5c40ddfd
20 --
21 Thanks,
22 Zac

Replies