Gentoo Archives: gentoo-dev

From: Pallav Agarwal <pallavagarwal07@×××××.com>
To: gentoo-dev@l.g.o, "Sébastien Fabbro" <sebfabbro@×××××.com>, Nitin Agarwal <nitinagarwal3006@×××××.com>
Subject: [gentoo-dev] Continuous Stabilization (Regarding Post Install Test Scripts)
Date: Thu, 19 May 2016 07:46:01
Message-Id: CAK23ojRoW-c3xmECFcUjbpk1UXeOw5BcCU9uXRB3qt4HVtLaBg@mail.gmail.com
1 This is a continuation of "[gentoo-dev] Proposal for changes for the next
2 EAPI
3 version" thread. Since the former subject isn't very relevant anymore.
4
5 Target:
6 A place to provide scripts that can be used by an automated arch-tester bot
7 to
8 stabilize Gentoo packages. These scripts can be as simple as a one liner
9
10 `python -c "import module"`
11
12 for example in case of a python module. Or a little complex, for example to
13 test
14 different features in case of different USE flags.
15 The main target is to alleviate the problems and unreliability from ARCH
16 Testing
17 and to more thoroughly and automatically test packages.
18
19 1 - Some important factors to consider from the previous thread:
20 a) This must be run POST-MERGE so that problems after merging can be
21 detected
22 b) It should be hopefully scalable in both directions - similar tests for
23 multiple packages
24 - different tests for different versions of same package.
25 c) If a users want, they should be able to run these tests too (If they do,
26 bug reports
27 collected will be very valuable.
28 d) Most common opinion was to not have an EAPI change unless this gains
29 enough
30 use or demand.
31 e) If implemented as a part of ebuild, it can have all benefits of an
32 ebuild syntax.
33 Different versions of a package can have different test scripts. Test code
34 part can
35 have its own conditional runtime dependencies (only to take effect IF
36 testing is on).
37 Also, it can have specific parts of test run or not dependent on USE flags
38 enabled.
39
40 2 - Some suggested solutions
41 a) The original proposed solution:
42 To have an extra phase ci_test() or something similar that would be run
43 ONLY when
44 emerge is called with ci-test option. That way rest of the ebuild doesn't
45 have to change,
46 and for normal users, this wouldn't run unless they explicitly want it to.
47
48 PROS:
49 i) Full use of ebuild syntax
50 ii) Users can also use.
51 CONS:
52 i) Requires EAPI change
53 ii) Burdens the ebuild with more stuff
54 iii) Existing ebuilds need to be changed (one more revision per ebuild)
55
56 b) Having an extra global USE flag "ci_test" and putting test code in
57 pkg_postinst
58 phase under `if use ci_test` condition.
59
60 PROS:
61 i) Full use of ebuild syntax
62 ii) Users can enable ci-test too
63 iii) Changes to current ebuild standard is negligible
64 CONS:
65 i) Burdens the ebuild with more stuff
66 ii) Existing ebuilds need to be changed (one more revision per ebuild)
67
68 c) Creating a Gentoo Overlay with ebuilds which do test packages. And the
69 testing
70 script would be part of the pkg_postinst existing function.
71 For example, overlay could have a package dev-python/numpy-ci-test with:
72 ```
73 DEPEND="dev-python/numpy"
74 pkg_postinstall() {
75 python -c "import numpy; numpy.test()" || die "failed"
76 }
77 ```
78 as it's content.
79
80 PROS:
81 i) No change to existing ebuilds or EAPI
82 ii) Can almost use all benefits of ebuild syntax
83 iii) Anybody can submit tests for ebuilds to the corresponding ebuild on
84 overlay
85 iv) Eclasses can define CI tests for multiple packages at once
86 CONS:
87 i) Users will almost never use an overlay just for testing
88 ii) Maintainer will have to submit one script for the portage tree and one
89 for the overlay.
90 Many maintainers would prefer to skip that.
91 iii) Ebuild names need to be different from the ones in portage tree, so
92 the process isn't
93 transparent anymore. We first need to check if -ci-test package exists or
94 not.
95
96
97 Any more suggestions/comments/methods are invited.
98
99 ---
100 Regards,
101 Pallav

Replies

Subject Author
Re: [gentoo-dev] Continuous Stabilization (Regarding Post Install Test Scripts) Harald Alfred Weiner <harald.weiner@×××.at>