1 |
Howdy! |
2 |
|
3 |
Software validation has a long, often divisive history, we should all |
4 |
be aware of. I've seen several Computer Scientists (PH.Dumasses) go to |
5 |
fists over "validation". It seems surreal now, but, it was hilarious |
6 |
at the time as none of the (3) involved in the fists_to_cuff had a clue |
7 |
about fighting. I'm sure other's have their stories. |
8 |
|
9 |
|
10 |
But, seriously, Gentoo has taken a big hit recently, with the loss of |
11 |
Tinderbox and Diego [1]. I surely do not want to dredge up that dispute. I |
12 |
liked both individuals quite a lot and the outcome is pitiful, to say the |
13 |
least. One was/is known to be a "buss_crack", but I never had issues with |
14 |
him. What I could never figure out is why Tinderbox, did not exist in |
15 |
several different locations; this would allow everyone to test x_64 bit |
16 |
and then focus on different architectures. The code used should be |
17 |
open source and mainlined, as everyone in Opensource and elsewhere has |
18 |
need of robust, automated code validation testing, imho. [2] |
19 |
|
20 |
|
21 |
Now, an old idea of mine (several racks of "gentoo" systems of various |
22 |
types and diverse hardware) has an updated sense of urgency. Automated |
23 |
code testing and development, was the goal, with a twist. I also wanted |
24 |
a method to rapidly change the entire codebase on a piece of hardware; |
25 |
and that was the show stopper until recently. Granted this project "was" |
26 |
to be focused on embedded gentoo, but, I see no reason it could not be |
27 |
expanded to include testing gentoo distro codes for x86 (32 and 64 bit |
28 |
codes) and some other arches too. |
29 |
|
30 |
(OK so far?) So you say, great go build it! OK, I've been working on it. |
31 |
I do need to flesh out some ideas and refine some codes and models I have |
32 |
found; for that I would greatly appreciate constructive criticism, ideas |
33 |
and code contributions. I have the resources to scale up this effort. |
34 |
I would expect "Gentoo" to also encourage others to build up an automated |
35 |
test suite environment, at other locations. |
36 |
|
37 |
Existing resource-codes I expext to leverage: |
38 |
|
39 |
(1) running many of the test systems from USB devices aka likewhoa_usb. |
40 |
This solves changing out platforms for discrete tests, by manually moving |
41 |
platforms around with discrete usb sticks. Sure part of the automated |
42 |
code testing will be all scripts and ethernet, particular for the mainstream |
43 |
arches, but, I also wanto to support testing on standalone |
44 |
boards/system for things such as embedded systems, firewalls, micro-dns |
45 |
servers, etc etc. |
46 |
|
47 |
(2) support for linux containers/vm/emulation for some test platforms |
48 |
(here folks are especially encourage to chim in, as this is not my forte. |
49 |
|
50 |
(3) The state of the art in massive automated testing is Linaro, imho. [3,4] |
51 |
|
52 |
|
53 |
Granted Linaro is focused on the latest arm cores, not limited to 64 bit[5], |
54 |
but what they use works, albeit for rolling out regular binary builds |
55 |
and not built for a rolling release model. [6] Hopefully, with some |
56 |
input, we can address this aspect and come up with something (a model). |
57 |
|
58 |
|
59 |
Maybe some forward looking folks could use "gaming theory" to enhance this |
60 |
automated test environment that quickly links to code repositories |
61 |
where codes are developed and modified. That part of the effort would |
62 |
be experimental, as I see the opportunity maybe working with one or 2 |
63 |
Overlays that are specifically about a single category of code. |
64 |
|
65 |
|
66 |
And, just in case you are wondering, yet the cluster I'm working on is |
67 |
to be use for a myriad of admin tasks and cross compiling to support |
68 |
an automated testing platform. |
69 |
|
70 |
|
71 |
All input is welcome, positive or negative....(obviously, security |
72 |
is a critical component, after the specification/model is established |
73 |
realizing that the spec. will be modified based on continuous security |
74 |
testing and enhancements). |
75 |
|
76 |
James |
77 |
|
78 |
|
79 |
|
80 |
[1] https://blog.flameeyes.eu/tag/tinderbox |
81 |
|
82 |
[2] http://wiki.gentoo.org/wiki/Tinderbox_log_collection_and_analysis |
83 |
|
84 |
[3] https://launchpad.net/lava |
85 |
|
86 |
[4] https://validation.linaro.org/ |
87 |
|
88 |
[5] http://www.linaro.org/projects/test-validation/ |
89 |
|
90 |
[6] http://www.linaro.org/blog/lava-blog/lava-fundamentals/ |