Re: [gentoo-dev] EJOBS variable for EAPI 5?

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 12/09/12 12:58 PM, Zac Medico wrote:
> On 09/12/2012 09:33 AM, Hans de Graaff wrote:
>> On Wed, 2012-09-12 at 08:58 -0400, Ian Stakenvicius wrote:
>>
>>> So essentially what you're saying here is that it might be
>>> worthwhile to look into parallelism as a whole and possibly
>>> come up with a solution that combines 'emerge --jobs' and
>>> build-system parallelism together to maximum benefit?
>>
>> Forget about jobs and load average, and just keep starting jobs
>> all around until there is only 20% (or whatever tuneable amount)
>> free memory left. As far as I can tell this is always the real
>> bottleneck in the end. Once you hit swap overall throughput has
>> to go down quite a bit.
>
> Well, I think it's still good to limit the number of jobs at
> least, since otherwise you could become overloaded with processes
> that don't consume a lot of memory at first but by the time they
> complete they have consumed much more memory than desired (using
> swap).

I think this would need to be dealt with by having the parent emerge
process monitor all children and specifically block individual
processes (ie, 'make' , 'ld' , etc) once resources are unavailable
until they become so.  Swap may be hit by the big processes but they
wouldn't continue to be processed while in swap, at least.

I don't have a solution to the potential 'thrashing' issue, though,
which i expect would be a problem even if there's enough memory.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iF4EAREIAAYFAlBQwFoACgkQ2ugaI38ACPAiAwD/foU8Xw1BQM3jeO6IiVfCGOnw
xHtIufwVmMpsGVdJQRIA/3W7Utg92foSc6ZtKMzBP5Fj0qB2BXMt/RS2I4pLsCQm
=gy9K
-----END PGP SIGNATURE-----