Re: [gentoo-user] Emerge and CPU core usage

[Dale]

J. Roeleveld wrote:
> Top posting? You?
>
>
> On 13 December 2019 08:36:08 CET, Dale <rdalek1967@×××××.com> wrote:
>> Howdy,
>>
>> I don't recall seeing the thread on the forums but it sort of sounds a
>> lot like what I was reading on -dev as to why it is not.  Basically, it
>> would be a complex and difficult piece of code.  According to some, it
>> could even create problems that don't exist now, depending on what
>> dependencies may pop into the process.
>>
>> It would be nice IF it could be done and really help with the time it
>> takes to calculate but it sounds like it might not even help much if it
>> was coded that way.  I guess it won't be done anytime soon.  It seems
>> like for good reason but it would be nifty. 
>>
>> I might add, I think that forum thread was way earlier than the posts
>> on
>> -dev.  I think the discussion on -dev was only a few years ago.  All in
>> all, the emerge command has come a long ways since I started uses
>> Gentoo
>> back in 2003. 
>>
>> Thanks for the info.
> I do think it can be done and should make it quicker.
> If done correctly, the hack of "backtracking" won't be needed anymore as well.
>
> Portage is a database and should be handled that way as well. Then for every package in the "install set" a thread can be started to check all the requirements.
> For each requirement, if it is already met, fine. If not, a new thread to check that. Result of each is what is needed. Any conflicts can be identified and handled by the calling thread.
>
> This should scale, provided the entire tree can be loaded into memory quickly.
>
> --
> Joost
>

I top posted because the previous person did.  I don't know what device
they are using and thought it be easier for them.  Yes, it does throw
the order of things into reverse but as we know, some devices aren't
logical.  LOL 

I think part of the problem is that emerge was never intended to be
multi-threaded.  It reminds me of Firefox.  I seem to recall several
years ago the devs there practically started from scratch in order to
fix some issues and make multi-threading work correctly.  Yea, some
complained about it breaking things and all but using the old code base
could have resulted in a worse piece of software.  I read all sorts of
comments on the reasoning behind the drastic change.  Sometimes, you
just have to throw out the bath water and start with fresh water.  I
suspect that quite often, it is even easier to do so. 

If things keep getting more complex, I suspect at some point it will
have to be done.  Even with the limited knowledge I have of how emerge
works, I feel sorry for that coder.  They may have to invent a new slide
rule.  ;-)  Still, emerge being 4 or 5 times faster would be nice.  For
those who have dozens of threads and even multi-CPU systems with lots of
threads, it would be even faster. 

I do want to be clear.  I'm not complaining about emerge.  It does a
awesome job, except for that cryptic error output sometimes.  :/  I was
sort of hoping to learn a bit and maybe even make someone think about a
way to work around this.  Even if it takes a year or two to get the
brain fully engaged, it would be worth it. 

Speaking of backtracking, that's one reason mine takes so long.  I have
some settings set as defaults because it makes emerge resolve issues
without having to set those options and starting it again.  I think I
have backtrack set to 100.  I started with the default which is much
lower but found that quite often it just wasn't deep enough. 

Interesting thread tho. 

Dale

:-)  :-)