1 |
commit: b7d7db208c421db0a30170f80d3e41328a4dc7db |
2 |
Author: Michał Górny <mgorny <AT> gentoo <DOT> org> |
3 |
AuthorDate: Thu Nov 2 18:43:14 2017 +0000 |
4 |
Commit: Michał Górny <mgorny <AT> gentoo <DOT> org> |
5 |
CommitDate: Mon Nov 13 16:33:01 2017 +0000 |
6 |
URL: https://gitweb.gentoo.org/data/glep.git/commit/?id=b7d7db20 |
7 |
|
8 |
glep-0074: Deprecate MISC and remove non-strict behavior |
9 |
|
10 |
glep-0074.rst | 93 +++++++++++++++++++++++++++++++++++++---------------------- |
11 |
1 file changed, 59 insertions(+), 34 deletions(-) |
12 |
|
13 |
diff --git a/glep-0074.rst b/glep-0074.rst |
14 |
index f256451..eee863a 100644 |
15 |
--- a/glep-0074.rst |
16 |
+++ b/glep-0074.rst |
17 |
@@ -49,16 +49,10 @@ This specification is designed with the following goals in mind: |
18 |
1. It should provide means to ensure the authenticity of the complete |
19 |
repository, including preventing the injection of additional files. |
20 |
|
21 |
-2. Like the original Manifest2, the files should be split into two |
22 |
- groups — files whose authenticity is critical, and those whose |
23 |
- mismatch may be accepted in non-strict mode. The same classification |
24 |
- should apply both to files listed in Manifests, and to stray files |
25 |
- present only in the repository. |
26 |
- |
27 |
-3. The format should be universal enough to work both for the Gentoo |
28 |
+2. The format should be universal enough to work both for the Gentoo |
29 |
repository and third-party repositories of different characteristics. |
30 |
|
31 |
-4. The Manifest files should be verifiable stand-alone, that is without |
32 |
+3. The Manifest files should be verifiable stand-alone, that is without |
33 |
knowing any details about the underlying repository format. |
34 |
|
35 |
|
36 |
@@ -205,15 +199,9 @@ The Manifest files can specify the following tags: |
37 |
verification (always pass). |
38 |
|
39 |
``DATA <path> <size> <checksums>…`` |
40 |
- Specifies a file subject to obligatory Manifest verification. |
41 |
- The file is required to pass verification. Used for all files directly |
42 |
- affecting package manager operation (ebuilds, eclasses, profiles). |
43 |
- |
44 |
-``MISC <path> <size> <checksums>…`` |
45 |
- Specifies a file subject to non-obligatory Manifest verification. |
46 |
- The package manager may ignore a verification failure if operating |
47 |
- in non-strict mode. Used for files that do not affect the installed |
48 |
- packages (``metadata.xml``, ``use.desc``). |
49 |
+ Specifies a regular file subject to Manifest verification. The file |
50 |
+ is required to pass verification. Used for all files that do not match |
51 |
+ any other type. |
52 |
|
53 |
``DIST <filename> <size> <checksums>…`` |
54 |
Specifies a distfile entry used to verify files fetched as part |
55 |
@@ -233,6 +221,11 @@ allowed at the package directory level: |
56 |
``EBUILD <filename> <size> <checksums>…`` |
57 |
Equivalent to the ``DATA`` type. |
58 |
|
59 |
+``MISC <path> <size> <checksums>…`` |
60 |
+ Equivalent to the ``DATA`` type. Historically indicated that |
61 |
+ the package manager may ignore a verification failure if operating |
62 |
+ in non-strict mode. However, that behavior is deprecated. |
63 |
+ |
64 |
``AUX <filename> <size> <checksums>…`` |
65 |
Equivalent to the ``DATA`` type, except that the filename is relative |
66 |
to ``files/`` subdirectory. |
67 |
@@ -378,11 +371,11 @@ for a package directory would have the following content:: |
68 |
|
69 |
DATA SphinxTrain-0.9.1-r1.ebuild 932 SHA256 3d3b.. SHA512 be4d.. |
70 |
DATA SphinxTrain-1.0.8.ebuild 912 SHA256 f681.. SHA512 0749.. |
71 |
+ DATA metadata.xml 664 SHA256 97c6.. SHA512 1175.. |
72 |
DATA files/gcc.patch 816 SHA256 b56e.. SHA512 2468.. |
73 |
DATA files/gcc34.patch 333 SHA256 c107.. SHA512 9919.. |
74 |
DIST SphinxTrain-0.9.1-beta.tar.gz 469617 SHA256 c1a4.. SHA512 1b33.. |
75 |
DIST sphinxtrain-1.0.8.tar.gz 8925803 SHA256 548e.. SHA512 465d.. |
76 |
- MISC metadata.xml 664 SHA256 97c6.. SHA512 1175.. |
77 |
|
78 |
|
79 |
Rationale |
80 |
@@ -521,21 +514,48 @@ it. Those directories were ``distfiles``, ``local`` and ``packages``. It |
81 |
could be also used to ignore VCS directories such as ``CVS``. |
82 |
|
83 |
|
84 |
-Non-obligatory Manifest verification |
85 |
------------------------------------- |
86 |
+Non-strict Manifest verification |
87 |
+-------------------------------- |
88 |
|
89 |
-While this specification recommends all tools to use strict verification |
90 |
-by default, it allows declaring some files as non-obligatory like |
91 |
-the original Manifest2 format did. This could be used on files that do |
92 |
-not affect the normal package manager operation. |
93 |
+Originally the Manifest2 format provided a special ``MISC`` tag that |
94 |
+was used for ``metadata.xml`` and ``ChangeLog`` files. This tag |
95 |
+indicated that the Manifest verification failures could be ignored for |
96 |
+those files unless the package manager was working in strict mode. |
97 |
|
98 |
-It aims to account for two use cases: |
99 |
+The first versions of this specification continued the use of this tag. |
100 |
+However, after a long debate it was decided to deprecate it along with |
101 |
+the non-strict behavior, and require all files to strictly match. |
102 |
|
103 |
-1. Stripping down files that are not strictly required to install |
104 |
- packages from repository checkouts. |
105 |
+Two arguments were mentioned for the usefulness of a ``MISC`` type: |
106 |
|
107 |
-2. Accounting for automatically generated files that might be updated |
108 |
- by standard tooling. |
109 |
+1. being able to reduce the checkout size by stripping unnecessary |
110 |
+ files out, and |
111 |
+ |
112 |
+2. being able to run update automatically generated files locally |
113 |
+ without causing unnecessary verification failures. |
114 |
+ |
115 |
+However, the usefulness of ``MISC`` in both cases is doubtful. |
116 |
+ |
117 |
+The cases for stripping unnecessary files mostly focused around space |
118 |
+savings. For this purpose, stripping ``metadata.xml`` and similar files |
119 |
+has little value. It is much more common for users to strip whole |
120 |
+categories which can not be handled via the ``MISC`` type, and needs |
121 |
+a dedicated package manager mechanism. The same mechanism can also |
122 |
+handle files that used the ``MISC`` type. |
123 |
+ |
124 |
+The cases for autogenerated files involve such cache files |
125 |
+as ``use.local.desc``. However, we can not include ``md5-cache`` there |
126 |
+due to security concerns which results in inconsistent cache handling. |
127 |
+Furthermore, the tools were historically modified to provide stable |
128 |
+output which means that their content can not change without |
129 |
+a non-``MISC`` content being changed first. This practically defeats |
130 |
+the purpose of using ``MISC``. |
131 |
+ |
132 |
+Finally, the non-strict mode could be used as means to an attack. |
133 |
+The allowance of missing or modified documentation file could be used |
134 |
+to spread misinformation, resulting in bad decisions made by the user. |
135 |
+A modified file could also be used e.g. to exploit vulnerabilities |
136 |
+of an XML parser. |
137 |
|
138 |
|
139 |
Timestamp field |
140 |
@@ -569,17 +589,22 @@ be not suitable to safe use. |
141 |
New vs deprecated tags |
142 |
---------------------- |
143 |
|
144 |
-Out of the four types defined by Manifest2, two are reused and two are |
145 |
-marked deprecated. |
146 |
+Out of the four types defined by Manifest2, only one is reused |
147 |
+and the remaining three is replaced by a single, universal ``DATA`` |
148 |
+type. |
149 |
|
150 |
-The ``DIST`` and ``MISC`` tags are reused since they can be relatively |
151 |
-clearly marked into the new concept. |
152 |
+The ``DIST`` tag is reused since the specification does not change |
153 |
+anything with regard to distfile handling. |
154 |
|
155 |
The ``EBUILD`` tag could potentially be reused for generic file |
156 |
verification data. However, it would be confusing if all the different |
157 |
data files were marked as ``EBUILD``. Therefore, an equivalent ``DATA`` |
158 |
type was introduced as a replacement. |
159 |
|
160 |
+The ``MISC`` tag and the relevant non-strict mode has been removed |
161 |
+as being of little value, as detailed in the `Non-strict Manifest |
162 |
+verification`_ section. |
163 |
+ |
164 |
The ``AUX`` tag is deprecated as it is redundant to ``DATA``, and has |
165 |
the limiting property of implicit ``files/`` path prefix. |
166 |
|
167 |
@@ -622,7 +647,7 @@ Normally we are not including those files to reduce the checkout size. |
168 |
However, some users have shown interest in them and Infra is working |
169 |
on providing them via an additional rsync module. |
170 |
|
171 |
-If such files were injected into the repository, they would cause strict |
172 |
+If such files were injected into the repository, they would cause |
173 |
verification failures of Manifests. To account for this, Infra could |
174 |
provide ``IGNORE`` entries to allow them to exist. |