Gentoo Archives: gentoo-portage-dev

From: Chun-Yu Shei <cshei@××××××.com>
To: gentoo-portage-dev@l.g.o
Cc: Chun-Yu Shei <cshei@××××××.com>
Subject: [gentoo-portage-dev] [PATCH 1/3] Add caching to catpkgsplit function
Date: Sat, 27 Jun 2020 06:34:26
Message-Id: 20200627063415.936177-2-cshei@google.com
In Reply to: [gentoo-portage-dev] Add caching to a few commonly used functions by Chun-Yu Shei
1 According to cProfile, catpkgsplit is called up to 1-5.5 million times
2 during "emerge -uDvpU --with-bdeps=y @world". Adding a dict to cache its
3 results reduces the time for this command from 43.53 -> 41.53 seconds --
4 a 4.8% speedup.
5 ---
6 lib/portage/versions.py | 7 +++++++
7 1 file changed, 7 insertions(+)
8
9 diff --git a/lib/portage/versions.py b/lib/portage/versions.py
10 index 0c21373cc..ffec316ce 100644
11 --- a/lib/portage/versions.py
12 +++ b/lib/portage/versions.py
13 @@ -312,6 +312,7 @@ def _pkgsplit(mypkg, eapi=None):
14
15 _cat_re = re.compile('^%s$' % _cat, re.UNICODE)
16 _missing_cat = 'null'
17 +_catpkgsplit_cache = {}
18
19 def catpkgsplit(mydata, silent=1, eapi=None):
20 """
21 @@ -331,6 +332,11 @@ def catpkgsplit(mydata, silent=1, eapi=None):
22 return mydata.cpv_split
23 except AttributeError:
24 pass
25 +
26 + cache_entry = _catpkgsplit_cache.get(mydata)
27 + if cache_entry is not None:
28 + return cache_entry
29 +
30 mysplit = mydata.split('/', 1)
31 p_split = None
32 if len(mysplit) == 1:
33 @@ -343,6 +349,7 @@ def catpkgsplit(mydata, silent=1, eapi=None):
34 if not p_split:
35 return None
36 retval = (cat, p_split[0], p_split[1], p_split[2])
37 + _catpkgsplit_cache[mydata] = retval
38 return retval
39
40 class _pkg_str(_unicode):
41 --
42 2.27.0.212.ge8ba1cc988-goog

Replies