1 |
On Thu, 17 Nov 2016 18:16:27 +1100 |
2 |
Michael Palimaka <kensington@g.o> wrote: |
3 |
|
4 |
> ==== Runtime testing ==== |
5 |
> |
6 |
> Consider the level of runtime testing that is required for the target |
7 |
> package. Remember, the focus of stabilisation is to integrate a testing |
8 |
> ebuild into the stable tree and not to identify routine bugs or |
9 |
> regressions - that is the purpose of the package's waiting time in ~arch. |
10 |
> |
11 |
> The level of runtime testing required will vary wildly based on a |
12 |
> variety of factors. Consider the following examples: |
13 |
> |
14 |
> * Multiple days of "normal use" testing may be appropriate for a new |
15 |
> version of {{package|sys-libs/glibc}} |
16 |
> * Basic functionality testing, such as browsing some web pages, may make |
17 |
> sense for a new version of {{package|www-client/firefox}} |
18 |
> * Passing tests might be enough for {{package|dev-python/yenc}} |
19 |
> * A leaf package such as {{package|kde-apps/kcalc}} may not require any |
20 |
> runtime testing at all |
21 |
|
22 |
Could we maybe include some place (metadata.xml?) to state what is |
23 |
the best way to test a package? I'm thinking it could include things |
24 |
like: |
25 |
|
26 |
- whether the test of the package are reliable, |
27 |
|
28 |
- whether runtime testing is required and what kind of, |
29 |
|
30 |
- how likely it is that revdeps need to be checked. |
31 |
|
32 |
For example, in LLVM I would like to ask arch testers to always check |
33 |
a few common clang calls. |
34 |
|
35 |
-- |
36 |
Best regards, |
37 |
Michał Górny |
38 |
<http://dev.gentoo.org/~mgorny/> |