Re: [gentoo-user] per package parallel build

[Dale]

Mick wrote:
> On Friday, 25 October 2019 18:01:57 BST Dale wrote:
>> Mick wrote:
>>> PS.  In an ideal AI world, portage would know how much memory is necessary
>>> for a given package and would auto-adjust the number of jobs to minimise
>>> swapping given any amount of RAM.  In an even more ideal world, it would
>>> be able to do this in real time.  :-)
>> I agree that it would be nice if emerge could do that automatically,
>> although I have no clue how to do that or even if it can be done at
>> all.  Back when I had less memory, I could let FF, LOo or another
>> package run at full speed but only if it was only one of those packages
>> at a time.  Thing is, on occasion two or more of those updates would hit
>> and due to the long compile times, end up compiling at the same time.
>> Do you think there is a way for the devs to set up a method to tell
>> emerge not to emerge certain packages at the same time?  In other words,
>> if Firefox is emerging, LOo is held until it is done or vice versa.
>> Maybe even have it so others can be listed.  The list of large packages
>> are likely small but they can have a huge impact on systems with less
>> memory.
>>
>> You think that a feature worth asking the devs about?  Maybe they can
>> figure out a way to implement that??
>>
>> Dale
>>
>> :-)  :-)
> Well, ebuilds with known large compiles have specific tests at present, e.g.
> chromium, FF, et al., run checks on available disk space for /var/tmp/portage
> and warn the user accordingly.  In firefox-70.0.ebuild we see this:
>
> ...
> pkg_pretend() {
>         # Ensure we have enough disk space to compile
>         if use pgo || use lto || use debug || use test ; then
>                 CHECKREQS_DISK_BUILD="8G"
>         else
>                 CHECKREQS_DISK_BUILD="4G"
>         fi
>
>         check-reqs_pkg_setup
> }
>
> I guess a lot can be included in such checks, but I have no idea how to do it,
> or how complicated an ebuild would get as a result.  The problem with RAM
> usage is that it changes continuously.  You open a browser, or some large PDF,
> or LO and more RAM is being used, just at the moment an ebuild decides to run
> a check on available RAM.  It can't guess if you will shut down your
> applications and log out of your desktop the next moment to free up more
> memory, if you will shut down web servers, databases or whatever else you may
> be running in the background and drop to a console.  For this reason I assume
> such a measure won't be implemented in rush and even if it did, it would annoy
> people, especially if they are asked to add USE="I-really-know-what-I-am-
> doing" to allow the compile to start.


I was hoping for something more simple.  Nothing in the ebuild just a
list of packages that emerge won't run at the same time.  Let's say the
big ones are included in the list, Seamonkey, Firefox, LOo and maybe a
couple others.  When emerge is run, Firefox and LOo is to be updated. 
Emerge itself knows not to update both so if one is updating and the
other is next in line, then it waits until the other is finished before
starting the next big one.  I would think the code should be in emerge
itself not the ebuild.  That way the user can determine what their
particular machine can handle. 

Like you, I'm not sure how to code that.  Of course, I've read about the
devs doing some complicated things that I don't grasp.  After all, it
calculates dependencies a lot faster than I could do on paper.  lol 

Dale

:-)  :-)