1 |
On 04/01/2018 11:29 AM, Michał Górny wrote: |
2 |
> W dniu nie, 01.04.2018 o godzinie 08∶59 -0700, użytkownik Zac Medico |
3 |
> napisał: |
4 |
>> On 04/01/2018 03:57 AM, Michał Górny wrote: |
5 |
>>> W dniu sob, 31.03.2018 o godzinie 19∶46 -0700, użytkownik Zac Medico |
6 |
>>> napisał: |
7 |
>>>> Since key refresh is prone to failure, retry using exponential |
8 |
>>>> backoff with random jitter. This adds the following sync-openpgp-* |
9 |
>>>> configuration settings: |
10 |
>>>> |
11 |
>>>> sync-openpgp-key-refresh-retry-count = 40 |
12 |
>>>> |
13 |
>>>> Maximum number of times to retry key refresh if it fails. Between |
14 |
>>>> each key refresh attempt, there is an exponential delay with a |
15 |
>>>> constant multiplier and a uniform random multiplier between 0 and 1. |
16 |
>>>> |
17 |
>>>> sync-openpgp-key-refresh-retry-delay-exp-base = 2 |
18 |
>>>> |
19 |
>>>> The base of the exponential expression. The exponent is the number |
20 |
>>>> of previous refresh attempts. |
21 |
>>>> |
22 |
>>>> sync-openpgp-key-refresh-retry-delay-max = 60 |
23 |
>>>> |
24 |
>>>> Maximum delay between each retry attempt, in units of seconds. This |
25 |
>>>> places a limit on the length of the exponential delay. |
26 |
>>>> |
27 |
>>>> sync-openpgp-key-refresh-retry-delay-mult = 4 |
28 |
>>>> |
29 |
>>>> Multiplier for the exponential delay. |
30 |
>>>> |
31 |
>>>> sync-openpgp-key-refresh-retry-overall-timeout = 1200 |
32 |
>>>> |
33 |
>>>> Combined time limit for all refresh attempts, in units of seconds. |
34 |
>>>> |
35 |
>>>> Bug: https://bugs.gentoo.org/649276 |
36 |
>>>> |
37 |
>>>> Zac Medico (4): |
38 |
>>>> Add ForkExecutor (bug 649588) |
39 |
>>>> Add ExponentialBackoff and RandomExponentialBackoff |
40 |
>>>> Add retry decorator (API inspired by tenacity) |
41 |
>>>> rsync: add key refresh retry (bug 649276) |
42 |
>>>> |
43 |
>>>> cnf/repos.conf | 5 + |
44 |
>>>> man/portage.5 | 19 +++ |
45 |
>>>> pym/portage/repository/config.py | 22 ++++ |
46 |
>>>> pym/portage/sync/modules/rsync/rsync.py | 16 ++- |
47 |
>>>> pym/portage/sync/syncbase.py | 85 +++++++++++- |
48 |
>>>> pym/portage/tests/util/futures/test_retry.py | 147 +++++++++++++++++++++ |
49 |
>>>> pym/portage/util/_eventloop/EventLoop.py | 45 ++++++- |
50 |
>>>> pym/portage/util/backoff.py | 48 +++++++ |
51 |
>>>> pym/portage/util/futures/executor/__init__.py | 0 |
52 |
>>>> pym/portage/util/futures/executor/fork.py | 130 +++++++++++++++++++ |
53 |
>>>> pym/portage/util/futures/futures.py | 6 + |
54 |
>>>> pym/portage/util/futures/retry.py | 178 ++++++++++++++++++++++++++ |
55 |
>>>> 12 files changed, 697 insertions(+), 4 deletions(-) |
56 |
>>>> create mode 100644 pym/portage/tests/util/futures/test_retry.py |
57 |
>>>> create mode 100644 pym/portage/util/backoff.py |
58 |
>>>> create mode 100644 pym/portage/util/futures/executor/__init__.py |
59 |
>>>> create mode 100644 pym/portage/util/futures/executor/fork.py |
60 |
>>>> create mode 100644 pym/portage/util/futures/retry.py |
61 |
>>>> |
62 |
>>> |
63 |
>>> This essentially looks like ~700 lines of code to try to workaround |
64 |
>>> broken networking. I would rather try to do that using 5 lines of code |
65 |
>>> but that's just me, and my programs aren't enterprise quality. I just |
66 |
>>> hope it actually solves as many problems as it's going to introduce. |
67 |
>> |
68 |
>> The vast majority of this code is generic and reusable, and I do intend |
69 |
>> to reuse it. For example, the executor support will be an essential |
70 |
>> piece for the asyncio.AbstractEventLoop for bug 649588. |
71 |
> |
72 |
> Sure it is and sure you will. But tell me: who is going to maintain it |
73 |
> all? Because as far as I can see, we're still dealing with a bus factor |
74 |
> of one and all you're doing is making it worse. More code, more |
75 |
> complexity, more creeping featurism and hacks. |
76 |
|
77 |
In the worst case, people can disable the retry feature and all of the |
78 |
associated code can be disabled simply by setting |
79 |
sync-openpgp-key-refresh-retry-count = 0 in repos.conf. |
80 |
|
81 |
In order to reduce the bus factor, I'm modeling the code on the standard |
82 |
library's asyncio framework. This allows use to leverage people's |
83 |
experience with asyncio, and also introduces people to asyncio concepts |
84 |
if they are not yet familar. |
85 |
|
86 |
> Last time you went away, you left us with a horribly unmaintainable |
87 |
> package manager full of complexity, hacks and creeping featurism |
88 |
> and a Portage team whose members had barely any knowledge of the code. |
89 |
> Just when things started moving again, you came back and we're back to |
90 |
> square one. |
91 |
|
92 |
I'd love to help reduce the bus factor, and I'm working to do that, like |
93 |
with the asyncio stuff. |
94 |
|
95 |
> Today Portage once again is a one-developer project, full of more |
96 |
> complexity, more hacks and more creeping featurism. And we once again |
97 |
> have a bus factor of one -- one developer who apparently knows |
98 |
> everything, does everything and tries to be nice to everyone, except he |
99 |
> really ignores others, makes a lot of empty promises and consistently |
100 |
> makes the health of the project go from bad to worse. |
101 |
|
102 |
I disagree with your synopsis. |
103 |
|
104 |
> So, please tell me: what happens when you leave again? How have you used |
105 |
> your time in the project? What have you done to make sure that |
106 |
> the project stays alive without you? Because as far as I can see, adding |
107 |
> few thousand of lines of practically unreviewed code every month does |
108 |
> not help with that. |
109 |
|
110 |
Again, I'd love to help reduce the bus factor. |
111 |
|
112 |
> I forked Portage because I didn't want to fight with you. When I forked |
113 |
> it, I declared that I will merge mainline changes regularly for |
114 |
> the benefit of my users. But after a week, I really start feeling like |
115 |
> that's going to be a really bad idea. Like it's time to forget about |
116 |
> mainline Portage as a completely dead end, and go our separate ways. |
117 |
|
118 |
That's your decision to make, but I think you'll benefit from merging my |
119 |
mainline changes. |
120 |
|
121 |
> I'm seriously worried about the future of Gentoo. I'd really appreciate |
122 |
> if you started focusing on that as well. |
123 |
|
124 |
I feel like I'm doing my part, and I appreciate your role as well. |
125 |
Thanks very much for your work. |
126 |
|
127 |
> I get that all this stuff looks |
128 |
> cool on paper but few months or years from now, someone will curse |
129 |
> 'whoever wrote that code' while trying to fix some nasty bug. Or get |
130 |
> things moving forward. Or implement EAPI 8. |
131 |
|
132 |
Again, I'd love to help reduce the bus factor, and the asyncio stuff |
133 |
represents movement toward that goal. |
134 |
-- |
135 |
Thanks, |
136 |
Zac |