1 |
Hi! |
2 |
|
3 |
On Tue, 17 May 2016, Kent Fredric wrote: |
4 |
|
5 |
> On 17 May 2016 at 19:37, Pallav Agarwal <pallavagarwal07@×××××.com> wrote: |
6 |
> > For normal users we wouldn't. But currently, arch-testers need to make a |
7 |
> > judgement call on what to test when a stable-req bug is filed. Tests run in |
8 |
> > src_test are provided by upstream, and does not guarantee that a package |
9 |
> > that has been merged will actually run on the system. |
10 |
> > If the maintainer could add a couple small scripts to check basic |
11 |
> > functionality |
12 |
> > of the merged package, it would make testing for arch testers much easier |
13 |
> > and reliable. |
14 |
> > Let me give an example. Let's say |
15 |
> > app-misc/screenfetch-2.7.7 is the current stable package for screenfetch in |
16 |
> > the portage tree. |
17 |
> > However, on running, it produces an error on the top, along with the proper |
18 |
> > output. |
19 |
> > If screenfetch-3.0.0 happens to fix that error, maintainer can add a simple |
20 |
> > script |
21 |
> > |
22 |
> > if [ "$(screenfetch 2>&1 1>/dev/null)" != "" ] then |
23 |
> > eerror "Still producing error" |
24 |
> > fi |
25 |
> > |
26 |
> > To make sure the build is properly updating the screenfetch version, and |
27 |
> > that |
28 |
> > the bug has in fact been fixed. This is the only way I can see to reliabily |
29 |
> > and automatically test packages that have been merged successfully |
30 |
> |
31 |
> |
32 |
> I don't think this needs to be an EAPI change. And if we can acheive |
33 |
> the goal without one, the better. |
34 |
|
35 |
Agreed. |
36 |
|
37 |
> [... lots of interesting stuff ...] |
38 |
|
39 |
Or maintainers and teams could do what the Emacs team does: |
40 |
provide test plans[0]. |
41 |
|
42 |
Naturally, I'd prefer something automated over the very hands-on |
43 |
tests that are a bit of a necessity for an application like |
44 |
Emacs, but they are still better than nothing. |
45 |
|
46 |
Either way, I think it is preferable to have non-upstream tests |
47 |
of whatever shape we pick to be separate from the ebuilds and the |
48 |
source files, for the simple reason that most users won't care |
49 |
about them. Those who do can retrieve them in the same manner as |
50 |
the ATs. |
51 |
|
52 |
And as for my pet peeve, tests that are known to fail, can we |
53 |
also annotate that somehow so I don't waste hours running a test |
54 |
suite that gives zero signal on whether I should add the stable |
55 |
keyword? Even a one-line hin in the stabilization request would |
56 |
be nice. As it is, I keep a list of known-to-fail packages and my |
57 |
testing machinery tells me to not bother with FEATURES=test in |
58 |
those case. |
59 |
|
60 |
Not ideal, but less time-wasty. |
61 |
|
62 |
Regards, |
63 |
Tobias |
64 |
|
65 |
[0] https://wiki.gentoo.org/wiki/Project:Emacs/Test_plans |
66 |
|
67 |
|
68 |
-- |
69 |
if (user_specified) |
70 |
/* Didn't work, but the user is convinced this is the |
71 |
* place. */ |
72 |
linux-2.4.0-test2/drivers/parport/parport_pc.c |