| 1 |
On Wed, Jul 20, 2005 at 10:32:29AM -0700, Donnie Berkholz wrote: |
| 2 |
> Duncan wrote: |
| 3 |
> | The AT guidelines (available for amd64 ATs here: |
| 4 |
> | http://www.gentoo.org/proj/en/base/amd64/tests/index.xml ) specifically |
| 5 |
> | mention testing "every conceivable permutation", |
| 6 |
> That's an absolute joke for reasonably large packages with say, >=10 USE |
| 7 |
> flags. I had to look up my combinations math, but it looks like you're |
| 8 |
> already into thousands of possibilities with even 10 flags. |
| 9 |
For PHP with 41 flags (98 in current revisions of PHP5), the numbers |
| 10 |
make it simply impossible to test every combination. |
| 11 |
|
| 12 |
I'm perfectly fine with looking at it via coverage tests instead. |
| 13 |
"Do our tests cover a large part of the source base (say 80%) and all |
| 14 |
critical functionality?" |
| 15 |
|
| 16 |
Im this case it's clear that ~5 specific cases of building dev-php/php |
| 17 |
would ensure sufficient testing. |
| 18 |
|
| 19 |
Automated code-coverage tools would probably go a long way in this. |
| 20 |
|
| 21 |
-- |
| 22 |
Robin Hugh Johnson |
| 23 |
E-Mail : robbat2@××××××××××××××.net |
| 24 |
Home Page : http://www.orbis-terrarum.net/?l=people.robbat2 |
| 25 |
ICQ# : 30269588 or 41961639 |
| 26 |
GnuPG FP : 11AC BA4F 4778 E3F6 E4ED F38E B27B 944E 3488 4E85 |
Archives