| 1 |
On 09/25/2018 01:36 PM, Michał Górny wrote: |
| 2 |
> On Tue, 2018-09-25 at 13:29 -0700, Zac Medico wrote: |
| 3 |
>> On 09/25/2018 01:07 PM, Michał Górny wrote: |
| 4 |
>>> On Tue, 2018-09-25 at 12:58 -0700, Zac Medico wrote: |
| 5 |
>>>> On 09/25/2018 12:55 PM, Zac Medico wrote: |
| 6 |
>>>>> On 09/25/2018 12:18 PM, Michał Górny wrote: |
| 7 |
>>>>>> Fix crash due to race condition in handling the same file being present |
| 8 |
>>>>>> both in compressed and uncompressed variants. If that is the case, |
| 9 |
>>>>>> just queue the uncompressed variant for compression, and ignore |
| 10 |
>>>>>> the other compressed variants. |
| 11 |
>>>>>> |
| 12 |
>>>>>> Bug: https://bugs.gentoo.org/667072 |
| 13 |
>>>>>> Signed-off-by: Michał Górny <mgorny@g.o> |
| 14 |
>>>>>> --- |
| 15 |
>>>>>> bin/ecompress | 7 +++++++ |
| 16 |
>>>>>> 1 file changed, 7 insertions(+) |
| 17 |
>>>>>> |
| 18 |
>>>>>> diff --git a/bin/ecompress b/bin/ecompress |
| 19 |
>>>>>> index 36bdb585b..d5ff3796c 100755 |
| 20 |
>>>>>> --- a/bin/ecompress |
| 21 |
>>>>>> +++ b/bin/ecompress |
| 22 |
>>>>>> @@ -49,6 +49,13 @@ while [[ $# -gt 0 ]] ; do |
| 23 |
>>>>>> find_args+=( -size "+${PORTAGE_DOCOMPRESS_SIZE_LIMIT}c" ) |
| 24 |
>>>>>> |
| 25 |
>>>>>> while IFS= read -d '' -r path; do |
| 26 |
>>>>>> + # if both compressed and uncompressed variant exists, |
| 27 |
>>>>>> + # skip the compressed variants (bug #667072) |
| 28 |
>>>>>> + case ${path} in |
| 29 |
>>>>>> + *.Z|*.gz|*.bz2|*.lzma|*.xz) |
| 30 |
>>>>>> + [[ -s ${path%.*} ]] && continue |
| 31 |
>>>>>> + ;; |
| 32 |
>>>>>> + esac |
| 33 |
>>>>> |
| 34 |
>>>>> In theory, we'd still have a problem if the file existed with muliple |
| 35 |
>>>>> compressions, right? Maybe a good solution is to strip the compression |
| 36 |
>>>>> extension here, and then have ecompress-file check for duplicates? |
| 37 |
>>>> |
| 38 |
>>>> Alternatively, we could uncompress the pre-compressed files right here. |
| 39 |
>>> |
| 40 |
>>> That would cause them to be decompressed even if the path is eventually |
| 41 |
>>> exclude via 'docompress -x'. |
| 42 |
>> |
| 43 |
>> I suppose we could record the extensions in a shared .ecompress file, |
| 44 |
>> and then ecompress-file could read the existing extensions from that file. |
| 45 |
> |
| 46 |
> Could we please solve the problem here instead of inventing super- |
| 47 |
> complex solutions to a non-existing theoretical problem that are |
| 48 |
> eventually going to cause more issues than real gain? Just like this |
| 49 |
> problem wouldn't have occurred if we haven't added entirely pointless |
| 50 |
> parallel compression of tiny files. |
| 51 |
|
| 52 |
Yeah, for now let's just revert "ecompress: Run compression in parallel": |
| 53 |
|
| 54 |
https://gitweb.gentoo.org/proj/portage.git/commit/?id=289d9a17dc9d9287e5dcb75f84b38ad0388e5fde |
| 55 |
-- |
| 56 |
Thanks, |
| 57 |
Zac |
Archives