1 |
commit: 06c577a0e72864859fbb2fb1cb7b7e8d60a78d79 |
2 |
Author: Michał Górny <mgorny <AT> gentoo <DOT> org> |
3 |
AuthorDate: Sun Sep 11 11:54:56 2022 +0000 |
4 |
Commit: Michał Górny <mgorny <AT> gentoo <DOT> org> |
5 |
CommitDate: Wed Sep 21 17:31:41 2022 +0000 |
6 |
URL: https://gitweb.gentoo.org/data/glep.git/commit/?id=06c577a0 |
7 |
|
8 |
glep-0074: Specify compressed file formats |
9 |
|
10 |
Signed-off-by: Michał Górny <mgorny <AT> gentoo.org> |
11 |
|
12 |
glep-0074.rst | 81 +++++++++++++++++++++++++++++++++++++++++++++++++++-------- |
13 |
1 file changed, 71 insertions(+), 10 deletions(-) |
14 |
|
15 |
diff --git a/glep-0074.rst b/glep-0074.rst |
16 |
index 3d7bbbd..7f53302 100644 |
17 |
--- a/glep-0074.rst |
18 |
+++ b/glep-0074.rst |
19 |
@@ -27,7 +27,8 @@ Changes |
20 |
======= |
21 |
|
22 |
v1.3 |
23 |
- Formally specified the current set of hash algorithms supported. |
24 |
+ Formally specified the current set of hash algorithms and compressed |
25 |
+ Manifest formats supported. |
26 |
|
27 |
v1.2 |
28 |
Specified the newline convention used for Manifests. |
29 |
@@ -432,9 +433,8 @@ compression and this specification. |
30 |
|
31 |
The compressed Manifest files are required to be suffixed for their |
32 |
compression algorithm. This suffix should be used to recognize |
33 |
-the compression and decompress Manifests transparently. The exact list |
34 |
-of algorithms and their corresponding suffixes are outside the scope |
35 |
-of this specification. |
36 |
+the compression and decompress Manifests transparently. The supported |
37 |
+formats are specified in `compressed file formats`_ section. |
38 |
|
39 |
The top-level Manifest file must not be compressed. Since the OpenPGP |
40 |
signature covers the uncompressed text and is compressed itself, |
41 |
@@ -455,6 +455,46 @@ uncompressed content and the specification is free to choose either |
42 |
of the files using the same base name. |
43 |
|
44 |
|
45 |
+Compressed file formats |
46 |
+----------------------- |
47 |
+ |
48 |
+.. table:: Table 2. Defined compressed file formats |
49 |
+ :widths: auto |
50 |
+ |
51 |
+ =========== ====== ==================== =========== |
52 |
+ Tool name Suffix Specification Notes |
53 |
+ =========== ====== ==================== =========== |
54 |
+ bzip2 .bz2 (none known) |
55 |
+ gzip .gz RFC 1952 [#RFC1952]_ Recommended |
56 |
+ lz4 .lz4 (none known) |
57 |
+ lzip .lz RFC draft [#LZIP]_ |
58 |
+ lzma .lzma (none known) Deprecated |
59 |
+ lzop .lzo (none known) |
60 |
+ xz .xz xz [#XZ]_ |
61 |
+ zstd .zst RFC 8878 [#RFC8878]_ |
62 |
+ =========== ====== ==================== =========== |
63 |
+ |
64 |
+Any new formats must be added to this specification prior to being used |
65 |
+for Manifest files. Adding a new compressed file format is considered |
66 |
+a backwards-compatible change to the GLEP. It is recommended that new |
67 |
+formats use their reference (most common) file suffixes. |
68 |
+ |
69 |
+An implementation can implement an arbitrary subset of the listed |
70 |
+formats. For best interoperability, it should implement at least |
71 |
+the recommended formats. Using deprecated formats should be avoided. |
72 |
+ |
73 |
+If multiple Manifest variants coexist using different compressed file |
74 |
+formats, the implementation may choose to use an arbitrary subset |
75 |
+of them. However, all of them must be verified against the hashes stored |
76 |
+in the containing Manifest. Should they be decompressed, the resulting |
77 |
+contents must be identical. |
78 |
+ |
79 |
+If the compressed file format is unsupported and a variant using |
80 |
+a supported format coexists, the other variant should be used. However, |
81 |
+at least one supported variant must exist for the verification |
82 |
+to succeed. |
83 |
+ |
84 |
+ |
85 |
Combining multiple Manifest trees (informational) |
86 |
------------------------------------------------- |
87 |
|
88 |
@@ -1033,12 +1073,19 @@ into a compressed sub-Manifest in the top directory (e.g. |
89 |
``Manifest.sub.gz``), and including a ``MANIFEST`` entry for this file |
90 |
in a signed, uncompressed top-level Manifest. |
91 |
|
92 |
-The existence of additional entries for uncompressed Manifest checksums |
93 |
-was debated. However, plain entries for the uncompressed file would |
94 |
-be confusing if only the compressed file existed, and conflicting |
95 |
-if both uncompressed and compressed variants existed. Furthermore, |
96 |
-it has been pointed out that ``DIST`` entries do not have |
97 |
-an uncompressed variant either. |
98 |
+The existence of additional entries for checksums of Manifest contents |
99 |
+after uncompressing was debated. However, plain entries for |
100 |
+the uncompressed file would be confusing if only the compressed file |
101 |
+existed. Furthermore, it has been pointed out that ``DIST`` entries |
102 |
+do not have an uncompressed variant either. |
103 |
+ |
104 |
+The specification permits coexistence of multiple variants of the same |
105 |
+Manifest file using different compression for historical compatibility. |
106 |
+However, there does not seem to be any real benefit from including |
107 |
+a compressed Manifest file if the uncompressed variant needs to exist |
108 |
+anyway. Providing different compressed variants could technically |
109 |
+improve interoperability, though the same result could probably |
110 |
+be achieved by using a more commonly supported format (e.g. gzip). |
111 |
|
112 |
|
113 |
Performance considerations |
114 |
@@ -1171,6 +1218,20 @@ References |
115 |
(archived at 2017-11-29) |
116 |
(https://web.archive.org/web/20171129084214/http://www.larc.usp.br/~pbarreto/WhirlpoolPage.html) |
117 |
|
118 |
+.. [#RFC1952] RFC 1952: GZIP file format specification version 4.3 |
119 |
+ (https://www.rfc-editor.org/rfc/rfc1952) |
120 |
+ |
121 |
+.. [#LZIP] RFC draft: Lzip Compressed Format and the 'application/lzip' |
122 |
+ Media Type |
123 |
+ (https://datatracker.ietf.org/doc/html/draft-diaz-lzip) |
124 |
+ |
125 |
+.. [#XZ] The .xz File Format |
126 |
+ (https://tukaani.org/xz/xz-file-format.txt) |
127 |
+ |
128 |
+.. [#RFC8878] RFC 8878: Zstandard Compression and the 'application/zstd' |
129 |
+ Media Type |
130 |
+ (https://www.rfc-editor.org/rfc/rfc8878) |
131 |
+ |
132 |
.. [#C08] Cappos, J et al. (2008). "Attacks on Package Managers" |
133 |
(https://www2.cs.arizona.edu/stork/packagemanagersecurity/attacks-on-package-managers.html) |