Re: [gentoo-dev] New gen-b0rk repository specifically for Q/A tools testing

[Justin]

On 02/05/2016 12:57 am, M. J. Everitt wrote:
> On 02/05/16 00:53, Brian Dolbec wrote:
>> In order to further improve the chances of Q/A tools catching
>> errors.  I have created a new repo (overlay) which will contain minimal
>> test case ebuilds.  The idea is to have test case ebuilds to run
>> repoman code against.  The outcome of these runs should be comparable
>> to pre-recorded output.  In that way as more code changes are applied
>> as part of the stage3 re-write as well as new test cases and checks to
>> be added to it's capabilities.  It should minimize the bugs introduced
>> in releases.
>>
>> Repoman does have some unit tests, but it is far from 100% coverage.
>> Also with the major structural changes that the code has been
>> undergoing, it is not always possible for the unit tests to be
>> compatible with the new code.
>>
>> This new repository is open to all Gentoo developers to contribute to.
>> All we ask is that you follow some simple common sense rules for adding
>> additional test ebuilds.
>>
>> The repo is located at:
>>
>> https://gitweb.gentoo.org/repo/proj/gen-b0rk.git/
>>
>> Here is the README included in the base directory.
>>
>> This repository is for the primary purpose of testing Q/A tools like repoman.
>>
>> The ebuilds it contains are for testing specific areas of tests that are
>> performed as part of the normal operation of that Q/A tool.
>>
>> This repository is open to all Gentoo developers under the following rules:
>>
>> 1) The master branch is to remain the stable Q/A testing branch.
>>
>> 2) All ebuilds are to be minimal test cases.
>>
>> 3) All ebuilds in it are to have no more than 3 or 4 flaws to detect.
>>    This makes it easier to spot errors during code development.  Simply running
>>    repoman in a category should be enough to test everything the module tests.
>>    This excludes some commit only checks which can be run in a local only branch.
>>
>> 4) All category names are to represent the Q/A category being tested.
>>       ie:
>>           ebuild-test - tests various aspects of the ebuild repoman module
>>           eclass-test - various eclass module tests
>>           ...
>>
>> 5) like the category naming, the package naming will follow the test
>>    being performed.
>>        ie:
>>            eclass-test/live-keywords - test the eclass module LiveEclassChecks
>>                                        keywords check
>>            ebuild-test/invalid - test for invalid package name detection
>>
>> 6) Profiles ... Not sure about this one, but probaly will have masters=gentoo
>>    That should ensure it maintains co-ordination with the main gentoo repo.
>>    All new or modified eclasses that affect pkg metadata should be validated in
>>    a branch.
>>
>> 7) New module development and test ebuilds will be done in a branch or personal
>>    repo and submitted to the gentoo-portage-dev email list for review and
>>    approval to merge into master.
>>    NOTE: This rule is lifted for the initial creation and population of
>>          test ebuilds to use to test out the repoman code.  An anouncemnt to
>>          the gentoo-project email list will be made when this initial population
>>          period is being ended.
>>
>> 8) Signed commits only, also signed-pushes are mandatory
>>
>> 9) The metadata category will get files of validated output that can be used
>>    to verify code changes in the various categories and repo wide runs.
>>    Diffing the output, should help to verify code changes did not break anything.
>>
>> 10) See rules 1-9 :-)
>>
> +1 be good to have somewhere central for this stuff :]
>
We can also run this on our new recruits. :)