Gentoo Archives: gentoo-portage-dev

From: Fabian Groffen <grobian@g.o>
To: gentoo-portage-dev@l.g.o
Subject: Re: [gentoo-portage-dev] [PATCH v3] collision_protect: use dynamic report interval
Date: Wed, 09 Jan 2019 08:09:42
Message-Id: 20190109080928.GA7770@gentoo.org
In Reply to: Re: [gentoo-portage-dev] [PATCH v3] collision_protect: use dynamic report interval by "M. J. Everitt"
1 On 08-01-2019 20:59:34 +0000, M. J. Everitt wrote:
2 > On 08/01/19 19:15, Zac Medico wrote:
3 > > On 1/8/19 5:42 AM, Fabian Groffen wrote:
4 > >> The reporting of files remaining can look somewhat odd since the report
5 > >> interval is hardcoded to be per 1000 objects. Adjust this interval to
6 > >> be time based. This means that modern (fast) machines likely will never
7 > >> see the countdown messages at all. On slow setups the message will be
8 > >> informative that there is progress, albeit rather slowly. While at it,
9 > >> report percentage done.
10 > >>
11 > >> Output before this patch:
12 > >>
13 > >> * checking 6158 files for package collisions
14 > >> 5158 files remaining ...
15 > >> 4158 files remaining ...
16 > >> 3158 files remaining ...
17 > >> 2158 files remaining ...
18 > >> 1158 files remaining ...
19 > >> 158 files remaining ...
20 > >>
21 > >> Possible output after this patch on a slower machine:
22 > >>
23 > >> * checking 6158 files for package collisions
24 > >> 48% done, 3145 files remaining ...
25 > >> 96% done, 192 files remaining ...
26 > >> 100% done
27 > >>
28 > >> Signed-off-by: Fabian Groffen <grobian@g.o>
29 > >> ---
30 > >> lib/portage/dbapi/vartree.py | 11 +++++++++--
31 > >> 1 file changed, 9 insertions(+), 2 deletions(-)
32 > >>
33 > >> diff --git a/lib/portage/dbapi/vartree.py b/lib/portage/dbapi/vartree.py
34 > >> index 4b91caea8..244195fad 100644
35 > >> --- a/lib/portage/dbapi/vartree.py
36 > >> +++ b/lib/portage/dbapi/vartree.py
37 > >> @@ -3475,13 +3475,18 @@ class dblink(object):
38 > >> symlink_collisions = []
39 > >> destroot = self.settings['ROOT']
40 > >> totfiles = len(file_list) + len(symlink_list)
41 > >> + tnow = time.time()
42 > >> + tinterv = 2 # seconds
43 > >> + ninterv = tnow + tinterv
44 > >> showMessage(_(" %s checking %d files for package collisions\n") % \
45 > >> (colorize("GOOD", "*"), totfiles))
46 > >> for i, (f, f_type) in enumerate(chain(
47 > >> ((f, "reg") for f in file_list),
48 > >> ((f, "sym") for f in symlink_list))):
49 > >> - if i % 1000 == 0 and i != 0:
50 > >> - showMessage(_("%d files remaining ...\n") % (totfiles - i))
51 > >> + if time.time() > ninterv:
52 > >> + showMessage(_("%3d%% done, %d files remaining ...\n") %
53 > >> + (i * 100 / totfiles, totfiles - i))
54 > >> + ninterv = time.time() + tinterv
55 > >>
56 > >> dest_path = normalize_path(
57 > >> os.path.join(destroot, f.lstrip(os.path.sep)))
58 > >> @@ -3570,6 +3575,8 @@ class dblink(object):
59 > >> break
60 > >> if stopmerge:
61 > >> collisions.append(f)
62 > >> + if tnow + tinterv < ninterv:
63 > >> + showMessage(_("100% done\n"))
64 > >> return collisions, dirs_ro, symlink_collisions, plib_collisions
65 > >>
66 > >> def _lstat_inode_map(self, path_iter):
67 > >>
68 > > Please replace time.time() with portage.util.monotonic.monotonic().
69
70 Will do.
71
72 > It's a bit of a nit-pick, granted, but can we ensure that the count-down's
73 > remain padded/justified such that the numbers line up for easy at-a-glance
74 > inspection ? The optimal standard looks somewhat like the pre-merge Sizes:
75 >
76 >  * Final size of build directory: 2696 KiB (2.6 MiB)
77 >  * Final size of installed tree:  5372 KiB (5.2 MiB)
78 >
79 > Otherwise, I think this will be quite helpful. Thanks.
80
81 I think I understand what you mean, let me see if I can do something
82 without too much complexity.
83
84 Thanks,
85 Fabian
86
87 --
88 Fabian Groffen
89 Gentoo on a different level

Attachments

File name MIME type
signature.asc application/pgp-signature