1 |
Hi Fabian, |
2 |
|
3 |
Just eyeballing htop's RES column while emerge is running, the max |
4 |
value I see during "emerge -uDvpU --with-bdeps=y @world" increases |
5 |
from 272 MB to 380 MB, so it looks to be around 110 MB of extra memory |
6 |
usage on my system (with 1,094 total packages installed). |
7 |
|
8 |
Chun-Yu |
9 |
|
10 |
On Sat, Jun 27, 2020, 12:35 AM Fabian Groffen <grobian@g.o> wrote: |
11 |
> |
12 |
> Hi Chun-Yu, |
13 |
> |
14 |
> On 26-06-2020 23:34:12 -0700, Chun-Yu Shei wrote: |
15 |
> > Hi, |
16 |
> > |
17 |
> > I was recently interested in whether portage could be speed up, since |
18 |
> > dependency resolution can sometimes take a while on slower machines. |
19 |
> > After generating some flame graphs with cProfile and vmprof, I found 3 |
20 |
> > functions which seem to be called extremely frequently with the same |
21 |
> > arguments: catpkgsplit, use_reduce, and match_from_list. In the first |
22 |
> > two cases, it was simple to cache the results in dicts, while |
23 |
> > match_from_list was a bit trickier, since it seems to be a requirement |
24 |
> > that it return actual entries from the input "candidate_list". I also |
25 |
> > ran into some test failures if I did the caching after the |
26 |
> > mydep.unevaluated_atom.use and mydep.repo checks towards the end of the |
27 |
> > function, so the caching is only done up to just before that point. |
28 |
> > |
29 |
> > The catpkgsplit change seems to definitely be safe, and I'm pretty sure |
30 |
> > the use_reduce one is too, since anything that could possibly change the |
31 |
> > result is hashed. I'm a bit less certain about the match_from_list one, |
32 |
> > although all tests are passing. |
33 |
> > |
34 |
> > With all 3 patches together, "emerge -uDvpU --with-bdeps=y @world" |
35 |
> > speeds up from 43.53 seconds to 30.96 sec -- a 40.6% speedup. "emerge |
36 |
> > -ep @world" is just a tiny bit faster, going from 18.69 to 18.22 sec |
37 |
> > (2.5% improvement). Since the upgrade case is far more common, this |
38 |
> > would really help in daily use, and it shaves about 30 seconds off |
39 |
> > the time you have to wait to get to the [Yes/No] prompt (from ~90s to |
40 |
> > 60s) on my old Sandy Bridge laptop when performing normal upgrades. |
41 |
> > |
42 |
> > Hopefully, at least some of these patches can be incorporated, and please |
43 |
> > let me know if any changes are necessary. |
44 |
> |
45 |
> This sounds like a good job to me! Do you have any idea what the added |
46 |
> memory pressure for these changes are? |
47 |
> |
48 |
> Thanks, |
49 |
> Fabian |
50 |
> > |
51 |
> > Thanks, |
52 |
> > Chun-Yu |
53 |
> > |
54 |
> > |
55 |
> > |
56 |
> |
57 |
> -- |
58 |
> Fabian Groffen |
59 |
> Gentoo on a different level |