1 |
Hi, |
2 |
|
3 |
Here's an updated text of GLEP 65 for review. Besides some editorial |
4 |
changes, I've added the concept of post-merge (aka post-postinst.d) |
5 |
checks as currently run by Portage. |
6 |
|
7 |
If there are no major issues, I'd like to push this version |
8 |
to the agenda for the next Council meeting. |
9 |
|
10 |
For the record, the problem of lack of signature verification still |
11 |
holds. However, it is irrelevant to the GLEP at hand, and it should be |
12 |
solved by GLEP 58 (MetaManifest). |
13 |
|
14 |
|
15 |
--- |
16 |
GLEP: 65 |
17 |
Title: Post-install QA checks |
18 |
Author: Michał Górny <mgorny@g.o> |
19 |
Type: Standards Track |
20 |
Status: Draft |
21 |
Version: 2 |
22 |
Created: 2014-10-26 |
23 |
Last-Modified: 2017-10-17 |
24 |
Post-History: 2017-10-17, 2014-10-30 |
25 |
Content-Type: text/x-rst |
26 |
--- |
27 |
|
28 |
Abstract |
29 |
======== |
30 |
|
31 |
This GLEP provides two kinds of QA check: checks run on the installation image |
32 |
once ``src_install`` returns, and checks run on the live system once |
33 |
``pkg_postinst`` returns. The checks can be provided by the Package Manager, |
34 |
repositories, packages (installed system-wide) and the system administrator. |
35 |
The QA checks can inspect the installation image or live system respectively, |
36 |
output and store both user- and machine-oriented QA warning logs, manipulate |
37 |
the files and abort the install, as necessary. |
38 |
|
39 |
|
40 |
Motivation |
41 |
========== |
42 |
|
43 |
The current Portage versions have two main QA check mechanisms: repoman |
44 |
and post-install QA checks. While repoman is usually considered more |
45 |
important, it has severe limitations. In particular, it is run on repository |
46 |
without executing the ebuild, therefore it is incapable of inspecting |
47 |
the installed files. This is where post-install QA checks become useful. |
48 |
|
49 |
Over time, many different QA checks have been added to Portage. That includes |
50 |
checks corresponding to generic Gentoo rules (like filesystem hierarchy, |
51 |
security requirements), checks enforcing Gentoo team policies and correct |
52 |
eclass uses. Some of the checks depend on external tools being present. |
53 |
|
54 |
Keeping those checks directly in Portage sources has two major disadvantages: |
55 |
|
56 |
1. The checks can not be properly updated without Portage upgrade. |
57 |
In particular, a change in a QA check becomes fully effective when |
58 |
the relevant Portage version becomes stable and the user upgrades. |
59 |
There is no easy way to keep QA checks in sync with eclasses. |
60 |
|
61 |
2. Gentoo-specific checks are enforced for all repositories and derived |
62 |
distributions. Modifying the QA check list requires forking Portage. |
63 |
|
64 |
|
65 |
Specification |
66 |
============= |
67 |
|
68 |
QA check types |
69 |
-------------- |
70 |
|
71 |
There are two kinds of QA checks defined in this specification: |
72 |
|
73 |
1. Post-install QA checks (``install-qa-check.d``), |
74 |
|
75 |
2. Post-merge (postinst) QA checks (``postinst-qa-check.d``). |
76 |
|
77 |
The post-install QA checks are are executed after the ``src_install`` ebuild |
78 |
phase finishes but before the binary package is built or the ``pkg_preinst`` |
79 |
phase is executed. They can use the same commands as are permitted |
80 |
in ``src_install``, and access the installation image ``${D}`` |
81 |
and the temporary directory ``${T}``. |
82 |
|
83 |
In case of severe QA issues, the checks are allowed to alter the contents of |
84 |
the installation image in order to sanitize them, or call the ``die`` function |
85 |
to abort the build. |
86 |
|
87 |
The post-merge QA checks are executed after the ``pkg_postinst`` ebuild phase |
88 |
finishes. They can use the same commands as are permitted in ``pkg_postinst``, |
89 |
and access the installed system location ``${ROOT}`` and the temporary |
90 |
directory ``${T}``. |
91 |
|
92 |
The checks are allowed to alter the contents of the filesystem to the same |
93 |
degree as ``pkg_postinst`` phase is. They must not call ``die``. |
94 |
|
95 |
QA check file format & locations |
96 |
-------------------------------- |
97 |
|
98 |
QA checks are stored as bash scripts. The checks are identified and ordered |
99 |
by file name. If files with same names are present in multiple locations, |
100 |
the file in location with the highest priority is used. |
101 |
|
102 |
The specification defines four sources of QA checks, listed in the order |
103 |
of increasing priority: |
104 |
|
105 |
1. internal checks included in the Package Manager, |
106 |
2. repository-specific QA checks, |
107 |
3. package-installed QA checks, |
108 |
4. sysadmin-defined QA checks. |
109 |
|
110 |
The internal checks are stored in Package Manager-specific location and should |
111 |
be installed along with the Package Manager. It is recommended that only |
112 |
generic checks are included in the Package Manager and not checks specific to |
113 |
Gentoo policies, packages or eclasses included in Gentoo. |
114 |
|
115 |
Repository-specific QA checks are included in ``metadata/install-qa-check.d`` |
116 |
and ``metadata/postinst-qa-check.d`` directories of a repository. |
117 |
For an ebuild in question, the repository containing it and its masters are |
118 |
traversed for QA checks, with priority decreasing with each inheritance level. |
119 |
|
120 |
The package-installed QA checks are located in ``/usr/lib/install-qa-check.d`` |
121 |
and ``/usr/lib/postinst-qa-check.d``, and are intended to be installed |
122 |
by packages. The sysadmin-defined QA checks are located |
123 |
in ``/usr/local/lib/install-qa-check.d`` |
124 |
and ``/usr/local/lib/postinst-qa-check.d``. |
125 |
|
126 |
QA check script format |
127 |
---------------------- |
128 |
|
129 |
QA checks are sourced as bash scripts by the Package Manager. QA scripts are |
130 |
run in an isolated subshell, and therefore can safely alter the environment |
131 |
and change the working directory. QA scripts must always end with a command |
132 |
terminating with a successful exit code. |
133 |
|
134 |
Aside to the standard PMS functions, two additional commands are provided: |
135 |
|
136 |
1. ``eqawarn`` to output QA warnings to user, |
137 |
2. ``eqatag`` to store machine-readable information about QA issues. |
138 |
|
139 |
Repository-defined QA checks are allowed to ``inherit`` eclasses from |
140 |
the repository providing the check or any of its masters. The same |
141 |
inheritance rules apply as to ebuilds in the particular repository. Sourced |
142 |
eclasses do not affect the final ebuild metadata or phase functions. |
143 |
|
144 |
Function specification |
145 |
---------------------- |
146 |
|
147 |
eqawarn |
148 |
~~~~~~~ |
149 |
Synopsis |
150 |
``eqawarn <message>...`` |
151 |
|
152 |
Output the specified message to the user as a QA warning, if the QA warning |
153 |
output is enabled. The message can contain escape sequences, and they will be |
154 |
interpreted according to the rules for ``echo -e`` bash built-in. |
155 |
|
156 |
The mechanism for enabling QA warning output and the specific output |
157 |
facilities are defined by the Package Manager. |
158 |
|
159 |
eqatag |
160 |
~~~~~~ |
161 |
Synopsis |
162 |
``eqatag [-v] <tag> [<key>=<value>...] [<file>...]`` |
163 |
|
164 |
Tag the package with specific QA issues. The *tag* parameter is |
165 |
a well-defined name identifying specific QA issue. The tag can be additionally |
166 |
associated with some data in key-value form and/or one or more *files*. |
167 |
The file paths are relative to installation image (``${D}``), and need to |
168 |
start with a leading slash. |
169 |
|
170 |
If ``-v`` (verbose) parameter is passed, the function will also output |
171 |
newline-delimited list of files using ``eqawarn``. This is intended |
172 |
as a short-hand for both storing machine-readable and outputting user-readable |
173 |
QA warnings. |
174 |
|
175 |
The mechanism used to store tags is defined by the Package Manager. The tag |
176 |
names are defined by the specific QA checks. However, it is recommended that |
177 |
tags are named hierarchically, with words being concatenated using a dot |
178 |
``.``, and that the first word matches QA check filename. For example, |
179 |
the tags used by ``60bash-completion`` check would be named |
180 |
``bash-completion.missing-alias`` and ``bash-completion.deprecated-have``. |
181 |
|
182 |
|
183 |
Rationale |
184 |
========= |
185 |
|
186 |
QA check types |
187 |
-------------- |
188 |
|
189 |
The two types of QA checks were created to account for different kinds |
190 |
of common mistakes in ebuilds. |
191 |
|
192 |
Post-install QA checks can be used to verify the installation image before |
193 |
it is merged to a live system or published as a binary package. They can |
194 |
account for various problems caused by the ebuild code up to and including |
195 |
``src_install``, the upstream code executed as part of any of those phases |
196 |
and the supplied files. |
197 |
|
198 |
Post-postinst QA checks can be used to verify the state of system after |
199 |
the package is merged and its ``pkg_postinst`` phase is executed. They mostly |
200 |
aim to detect missing postinst actions but can do other live system integrity |
201 |
checks. |
202 |
|
203 |
QA check file format & locations |
204 |
-------------------------------- |
205 |
|
206 |
The multiple locations for QA checks aim to get the best coverage for various |
207 |
requirements. |
208 |
|
209 |
The checks installed along with the Package Manager are meant to cover |
210 |
the generic cases and other checks that rely on Package Manager internals. |
211 |
Unlike other categories of QA checks, those checks apply to a single Package |
212 |
Manager only and can therefore use internal API. However, it is recommended |
213 |
that this category is used scarcely. |
214 |
|
215 |
Storing checks in the repository allows developers to strictly bind them to |
216 |
a specific version of the distribution and update them along with the relevant |
217 |
policies and/or eclasses. In particular, rules enforced by Gentoo policies |
218 |
and eclasses don't have to apply to other distributions using Portage. |
219 |
|
220 |
The QA checks are applied to sub-repositories (via ``masters`` attribute) |
221 |
likewise eclasses. This makes sure that the majority of repositories don't |
222 |
lose QA checks. The QA checks related to eclasses are inherited the same way |
223 |
as eclasses are. Similarly to eclasses, sub-repositories can override |
224 |
(or disable) QA checks. |
225 |
|
226 |
System-wide QA checks present the opportunity of installing QA checks along |
227 |
with packages. In the past, some QA checks were run only conditionally |
228 |
depending on existence of external checker software. Instead, the software |
229 |
packages can install their own QA checks directly. |
230 |
|
231 |
The administrative override via ``/usr/local`` is a natural extension |
232 |
of system-wide QA checks. Additionally, it can be used by the sysadmin |
233 |
to override or disable practically any other QA check, either internal Portage |
234 |
or repository-wide. |
235 |
|
236 |
Sharing the QA checks has the additional advantage of having unified QA tools |
237 |
for all Package Managers. |
238 |
|
239 |
QA check script format |
240 |
---------------------- |
241 |
|
242 |
Use of bash is aimed to match the ebuild format. The choice of functions aims |
243 |
at portability between Package Managers. |
244 |
|
245 |
The scripts are run in isolated subshell to simplify the checks and reduce |
246 |
the risk of accidental cross-script issues. |
247 |
|
248 |
The script need to end with a successful command as a result of bash |
249 |
limitation:: |
250 |
|
251 |
source foo || die "source failed" |
252 |
|
253 |
The ``source`` call either returns the exit code of last command in the script |
254 |
or unsuccessful exit code in case of sourcing error. In order to distinguish |
255 |
between the two, we need to guarantee that the script always returns |
256 |
successfully. |
257 |
|
258 |
The extra ``eqawarn`` log function aims to provide the user with distinction |
259 |
between important user-directed warnings and developer-oriented QA issues. |
260 |
The ``eqatag`` function aims to store check results in a machine-readable |
261 |
format for further processing. |
262 |
|
263 |
Inheriting eclasses makes it possible to reuse code and improve |
264 |
maintainability. The possibility is mostly intended for eclass-specific checks |
265 |
that may want to e.g. obtain search paths from the eclass. |
266 |
|
267 |
Inheriting is allowed only in repository-specific since it is the only |
268 |
location where availability of eclasses can be assumed. For system-wide |
269 |
checks, we can't assume that the source repository will be available when |
270 |
ebuild in question is processed. |
271 |
|
272 |
Function specification |
273 |
---------------------- |
274 |
eqawarn |
275 |
~~~~~~~ |
276 |
|
277 |
This function is already considered well-defined at the time of writing. It is |
278 |
supported by Portage and stubbed in ``eutils.eclass``. Therefore, |
279 |
the specification aims to be a best match between the current implementation |
280 |
and the PMS definition of ``ewarn`` function. The latter specifically involves |
281 |
making the output and output control mechanisms PM-defined. |
282 |
|
283 |
eqatag |
284 |
~~~~~~ |
285 |
|
286 |
This functions is defined in order to allow external tools to parse results |
287 |
of QA checks easily, tinderbox in particular. The name ``eqatag`` alludes |
288 |
to the process of 'tagging' files with QA labels. |
289 |
|
290 |
The original proposal has used the name ``eqalog`` but it was rejected because |
291 |
of potential confusion with user-oriented ``elog`` function. |
292 |
|
293 |
The tags can be associated both with files and abstract data to accommodate |
294 |
the widest range of checks. The additional data is provided in key-value form |
295 |
to allow extending or changing the format easily. The file path format is |
296 |
meant to match the canonical ``/usr/bin/foo`` paths. |
297 |
|
298 |
The requirement of leading slash allows the function to safely distinguish |
299 |
between key-value data (assuming the key name must not start with a slash) |
300 |
and files. |
301 |
|
302 |
The ``-v`` argument works as a short-hand for an expected-to-be-common |
303 |
practice of:: |
304 |
|
305 |
eqawarn "The following files are frobnicated incorrectly:" |
306 |
eqawarn |
307 |
eqatag -v frobnicate "${files[@]}" |
308 |
eqawarn |
309 |
eqawarn "Please consult http://example.com/frobnicate for more details." |
310 |
|
311 |
which would be output as:: |
312 |
|
313 |
* The following files are frobnicated incorrectly: |
314 |
* |
315 |
* /usr/bin/frobnicatee |
316 |
* /usr/bin/other-frobnicatee |
317 |
* |
318 |
* Please consult http://example.com/frobnicate for more details. |
319 |
|
320 |
The mechanism for storing the results is left implementation-defined because |
321 |
both the method of running builds and their location varies through Package |
322 |
Managers. The original proposal used a well-defined format in ``${T}/qa.log``. |
323 |
|
324 |
|
325 |
Backwards Compatibility |
326 |
======================= |
327 |
|
328 |
Past versions of the Package Managers will only use their own built-in checks, |
329 |
and will not be affected by the specification. |
330 |
|
331 |
Compliant versions of the Package Manager will split the built-in checks into |
332 |
multiple files. When particular checks are moved into the repository, the name |
333 |
will be retained so that the repository copy will override the built-in check |
334 |
and no duplicate checking will happen. |
335 |
|
336 |
The transferred checks will be removed in the future versions of the Package |
337 |
Manager. However, since they will support this GLEP, the relevant checks will |
338 |
be used from the repository anyway. |
339 |
|
340 |
|
341 |
Reference implementation |
342 |
======================== |
343 |
|
344 |
The reference implementation of ``install-qa-check.d`` is available in Portage |
345 |
starting with version 2.2.15 (released 4 Dec 2014). The support |
346 |
for ``postinst-qa-check.d`` was added in 2.3.9 (released 19 Sep 2017). |
347 |
|
348 |
|
349 |
Copyright |
350 |
========= |
351 |
|
352 |
This work is licensed under the Creative Commons Attribution-ShareAlike 3.0 |
353 |
Unported License. To view a copy of this license, visit |
354 |
http://creativecommons.org/licenses/by-sa/3.0/. |
355 |
|
356 |
-- |
357 |
Best regards, |
358 |
Michał Górny |