Gentoo Archives: gentoo-user

From: James <wireless@×××××××××××.com>
To: gentoo-user@l.g.o
Subject: [gentoo-user] automated code validation
Date: Mon, 01 Dec 2014 17:41:10
Message-Id: loom.20141201T175046-420@post.gmane.org
1 Howdy!
2
3 Software validation has a long, often divisive history, we should all
4 be aware of. I've seen several Computer Scientists (PH.Dumasses) go to
5 fists over "validation". It seems surreal now, but, it was hilarious
6 at the time as none of the (3) involved in the fists_to_cuff had a clue
7 about fighting. I'm sure other's have their stories.
8
9
10 But, seriously, Gentoo has taken a big hit recently, with the loss of
11 Tinderbox and Diego [1]. I surely do not want to dredge up that dispute. I
12 liked both individuals quite a lot and the outcome is pitiful, to say the
13 least. One was/is known to be a "buss_crack", but I never had issues with
14 him. What I could never figure out is why Tinderbox, did not exist in
15 several different locations; this would allow everyone to test x_64 bit
16 and then focus on different architectures. The code used should be
17 open source and mainlined, as everyone in Opensource and elsewhere has
18 need of robust, automated code validation testing, imho. [2]
19
20
21 Now, an old idea of mine (several racks of "gentoo" systems of various
22 types and diverse hardware) has an updated sense of urgency. Automated
23 code testing and development, was the goal, with a twist. I also wanted
24 a method to rapidly change the entire codebase on a piece of hardware;
25 and that was the show stopper until recently. Granted this project "was"
26 to be focused on embedded gentoo, but, I see no reason it could not be
27 expanded to include testing gentoo distro codes for x86 (32 and 64 bit
28 codes) and some other arches too.
29
30 (OK so far?) So you say, great go build it! OK, I've been working on it.
31 I do need to flesh out some ideas and refine some codes and models I have
32 found; for that I would greatly appreciate constructive criticism, ideas
33 and code contributions. I have the resources to scale up this effort.
34 I would expect "Gentoo" to also encourage others to build up an automated
35 test suite environment, at other locations.
36
37 Existing resource-codes I expext to leverage:
38
39 (1) running many of the test systems from USB devices aka likewhoa_usb.
40 This solves changing out platforms for discrete tests, by manually moving
41 platforms around with discrete usb sticks. Sure part of the automated
42 code testing will be all scripts and ethernet, particular for the mainstream
43 arches, but, I also wanto to support testing on standalone
44 boards/system for things such as embedded systems, firewalls, micro-dns
45 servers, etc etc.
46
47 (2) support for linux containers/vm/emulation for some test platforms
48 (here folks are especially encourage to chim in, as this is not my forte.
49
50 (3) The state of the art in massive automated testing is Linaro, imho. [3,4]
51
52
53 Granted Linaro is focused on the latest arm cores, not limited to 64 bit[5],
54 but what they use works, albeit for rolling out regular binary builds
55 and not built for a rolling release model. [6] Hopefully, with some
56 input, we can address this aspect and come up with something (a model).
57
58
59 Maybe some forward looking folks could use "gaming theory" to enhance this
60 automated test environment that quickly links to code repositories
61 where codes are developed and modified. That part of the effort would
62 be experimental, as I see the opportunity maybe working with one or 2
63 Overlays that are specifically about a single category of code.
64
65
66 And, just in case you are wondering, yet the cluster I'm working on is
67 to be use for a myriad of admin tasks and cross compiling to support
68 an automated testing platform.
69
70
71 All input is welcome, positive or negative....(obviously, security
72 is a critical component, after the specification/model is established
73 realizing that the spec. will be modified based on continuous security
74 testing and enhancements).
75
76 James
77
78
79
80 [1] https://blog.flameeyes.eu/tag/tinderbox
81
82 [2] http://wiki.gentoo.org/wiki/Tinderbox_log_collection_and_analysis
83
84 [3] https://launchpad.net/lava
85
86 [4] https://validation.linaro.org/
87
88 [5] http://www.linaro.org/projects/test-validation/
89
90 [6] http://www.linaro.org/blog/lava-blog/lava-fundamentals/

Replies

Subject Author
Re: [gentoo-user] automated code validation Alex Brandt <alunduil@g.o>