1 |
Hi Chun-Yu, |
2 |
|
3 |
On 26-06-2020 23:34:12 -0700, Chun-Yu Shei wrote: |
4 |
> Hi, |
5 |
> |
6 |
> I was recently interested in whether portage could be speed up, since |
7 |
> dependency resolution can sometimes take a while on slower machines. |
8 |
> After generating some flame graphs with cProfile and vmprof, I found 3 |
9 |
> functions which seem to be called extremely frequently with the same |
10 |
> arguments: catpkgsplit, use_reduce, and match_from_list. In the first |
11 |
> two cases, it was simple to cache the results in dicts, while |
12 |
> match_from_list was a bit trickier, since it seems to be a requirement |
13 |
> that it return actual entries from the input "candidate_list". I also |
14 |
> ran into some test failures if I did the caching after the |
15 |
> mydep.unevaluated_atom.use and mydep.repo checks towards the end of the |
16 |
> function, so the caching is only done up to just before that point. |
17 |
> |
18 |
> The catpkgsplit change seems to definitely be safe, and I'm pretty sure |
19 |
> the use_reduce one is too, since anything that could possibly change the |
20 |
> result is hashed. I'm a bit less certain about the match_from_list one, |
21 |
> although all tests are passing. |
22 |
> |
23 |
> With all 3 patches together, "emerge -uDvpU --with-bdeps=y @world" |
24 |
> speeds up from 43.53 seconds to 30.96 sec -- a 40.6% speedup. "emerge |
25 |
> -ep @world" is just a tiny bit faster, going from 18.69 to 18.22 sec |
26 |
> (2.5% improvement). Since the upgrade case is far more common, this |
27 |
> would really help in daily use, and it shaves about 30 seconds off |
28 |
> the time you have to wait to get to the [Yes/No] prompt (from ~90s to |
29 |
> 60s) on my old Sandy Bridge laptop when performing normal upgrades. |
30 |
> |
31 |
> Hopefully, at least some of these patches can be incorporated, and please |
32 |
> let me know if any changes are necessary. |
33 |
|
34 |
This sounds like a good job to me! Do you have any idea what the added |
35 |
memory pressure for these changes are? |
36 |
|
37 |
Thanks, |
38 |
Fabian |
39 |
> |
40 |
> Thanks, |
41 |
> Chun-Yu |
42 |
> |
43 |
> |
44 |
> |
45 |
|
46 |
-- |
47 |
Fabian Groffen |
48 |
Gentoo on a different level |