| 1 |
On 04/01/2018 03:57 AM, Michał Górny wrote: |
| 2 |
> W dniu sob, 31.03.2018 o godzinie 19∶46 -0700, użytkownik Zac Medico |
| 3 |
> napisał: |
| 4 |
>> Since key refresh is prone to failure, retry using exponential |
| 5 |
>> backoff with random jitter. This adds the following sync-openpgp-* |
| 6 |
>> configuration settings: |
| 7 |
>> |
| 8 |
>> sync-openpgp-key-refresh-retry-count = 40 |
| 9 |
>> |
| 10 |
>> Maximum number of times to retry key refresh if it fails. Between |
| 11 |
>> each key refresh attempt, there is an exponential delay with a |
| 12 |
>> constant multiplier and a uniform random multiplier between 0 and 1. |
| 13 |
>> |
| 14 |
>> sync-openpgp-key-refresh-retry-delay-exp-base = 2 |
| 15 |
>> |
| 16 |
>> The base of the exponential expression. The exponent is the number |
| 17 |
>> of previous refresh attempts. |
| 18 |
>> |
| 19 |
>> sync-openpgp-key-refresh-retry-delay-max = 60 |
| 20 |
>> |
| 21 |
>> Maximum delay between each retry attempt, in units of seconds. This |
| 22 |
>> places a limit on the length of the exponential delay. |
| 23 |
>> |
| 24 |
>> sync-openpgp-key-refresh-retry-delay-mult = 4 |
| 25 |
>> |
| 26 |
>> Multiplier for the exponential delay. |
| 27 |
>> |
| 28 |
>> sync-openpgp-key-refresh-retry-overall-timeout = 1200 |
| 29 |
>> |
| 30 |
>> Combined time limit for all refresh attempts, in units of seconds. |
| 31 |
>> |
| 32 |
>> Bug: https://bugs.gentoo.org/649276 |
| 33 |
>> |
| 34 |
>> Zac Medico (4): |
| 35 |
>> Add ForkExecutor (bug 649588) |
| 36 |
>> Add ExponentialBackoff and RandomExponentialBackoff |
| 37 |
>> Add retry decorator (API inspired by tenacity) |
| 38 |
>> rsync: add key refresh retry (bug 649276) |
| 39 |
>> |
| 40 |
>> cnf/repos.conf | 5 + |
| 41 |
>> man/portage.5 | 19 +++ |
| 42 |
>> pym/portage/repository/config.py | 22 ++++ |
| 43 |
>> pym/portage/sync/modules/rsync/rsync.py | 16 ++- |
| 44 |
>> pym/portage/sync/syncbase.py | 85 +++++++++++- |
| 45 |
>> pym/portage/tests/util/futures/test_retry.py | 147 +++++++++++++++++++++ |
| 46 |
>> pym/portage/util/_eventloop/EventLoop.py | 45 ++++++- |
| 47 |
>> pym/portage/util/backoff.py | 48 +++++++ |
| 48 |
>> pym/portage/util/futures/executor/__init__.py | 0 |
| 49 |
>> pym/portage/util/futures/executor/fork.py | 130 +++++++++++++++++++ |
| 50 |
>> pym/portage/util/futures/futures.py | 6 + |
| 51 |
>> pym/portage/util/futures/retry.py | 178 ++++++++++++++++++++++++++ |
| 52 |
>> 12 files changed, 697 insertions(+), 4 deletions(-) |
| 53 |
>> create mode 100644 pym/portage/tests/util/futures/test_retry.py |
| 54 |
>> create mode 100644 pym/portage/util/backoff.py |
| 55 |
>> create mode 100644 pym/portage/util/futures/executor/__init__.py |
| 56 |
>> create mode 100644 pym/portage/util/futures/executor/fork.py |
| 57 |
>> create mode 100644 pym/portage/util/futures/retry.py |
| 58 |
>> |
| 59 |
> |
| 60 |
> This essentially looks like ~700 lines of code to try to workaround |
| 61 |
> broken networking. I would rather try to do that using 5 lines of code |
| 62 |
> but that's just me, and my programs aren't enterprise quality. I just |
| 63 |
> hope it actually solves as many problems as it's going to introduce. |
| 64 |
|
| 65 |
The vast majority of this code is generic and reusable, and I do intend |
| 66 |
to reuse it. For example, the executor support will be an essential |
| 67 |
piece for the asyncio.AbstractEventLoop for bug 649588. |
| 68 |
-- |
| 69 |
Thanks, |
| 70 |
Zac |
Archives