[gentoo-releng] software engineering standards document remarks

[Pieter Van den Abeele]

Hi,

I've read through the gentoo software engineering standards draft. I've 
written down some remarks below.

> This document contains comprehensive standards for Gentoo software
> development
> projects. These standards should benefit nearly any Gentoo project,
> but are
> particularly critical for projects that demand a very high level of
> refinement
> and overall quality.

Software engineering standards are like licensing terms. If the GPL 
said: nearly all users using GPL software should obey to its terms, 
there would be a problem. It is not clear from the above paragraph, 
that all gentoo project should look into this document. We also need a 
formal definition of a 'project'. (modifying an ebuild is not a project 
that requires USE cases, design, approval, ...)

It is a bad idea to suggest we have projects that need to follow 
standards, and those that don't or to a lesser extend. Simply all 
projects need to follow the standards. I wouldn't document what 
projects you are going to look harder at, and what projects you don't. 
It is clear from this document that portage is something you'll look 
hard at. That's good to know, but I'd leave that fact out of the 
document, unless of course you expect to use the document only for 
portage.

My gentoo experience tells me that high standard projects are affected 
by projects without or with only very low standards/ethics: in the high 
standard project it would be possible to schedule things, but deadlines 
will be missed -without doubt- because requirements for other projects 
are changed suddenly during design or implementation, leading to at 
least a complete rescheduling for the high-standards project because 
they need to accommodate for the manpower loss. If you create a fire 
somewhere, people from other projects will have to go put it out, and 
those people will not be able to do something else while they are 
putting out the fire.

Note that uncontrolled fires do not obey to engineering standards. More 
specifically, their timetable is too constraining. For instance: there 
are currently 3 features in the 2004.1 I'm guessing I need to invest 
time into, but afaik work has only started on one of them. If catalyst 
starts changing five minutes for the release and an incredible amount 
of ebuilds are required to be masked stable on ppc in the last five 
minutes, I'll be doing nothing but releasing again, in best case, only 
miss the 2004.1 release by a few days. Even though I have been building 
releases every two days, I can still end up with an unstable release.

These standards will without doubt be another fire too if they are 
suddenly applied to projects without giving them time to adapt. One 
fire can be put out really quickly, but 10 uncontrolled fires will lead 
to some project stalling. I wonder whether this is open-source related; 
will there always be new volunteers to replace the firemen who burned 
themselves (out) ? My suggestion would be to switch to controlled 
fires, and to increase the amount of fires when the amount of manpower 
increases, but I'd stop burning people.

> Goals

> It is intended to define a waterfall model for

It might be a good idea to explain what the waterfall model is;

1. Requirements
2. Client approves functional/non-functional requirements
3. Design
4. Client approves design
5. Implementation and documentation
6. Requirements check - project failure or success
7. Revisit requirements, (add/remove/change)
8. Client approves revisited functional/non-functional requirements
9. Revisit design
10. Client approves revisited design
11. Implementation and documentation
12. Requirements met? - iteration failure or success
13. goto 7

In this case the client is our userbase. This is a special client 
because:

- our client (a huge set of people) wants to see all their requirements 
met in the first iteration.
- our client cannot approve something unanimously very easily

We therefore need to find somebody who represents the client.

My suggestion would be to let the management team represent the client, 
with the store as an advisor. The releng team could represent the 
'developer'. Releng should therefore be represented in the management 
team. Care should be taken with people seated in the management team, 
part of releng and in the store, they should make sure they keep their 
roles separate, otherwise we will end up in an unhealthy situation 
(where for instance releng solely represents the store and does not 
take into account development constraints). Management needs to look 
for a compromis between store, userbase and developer 
requirements/constraints.

When we define requirements standards, design standards and standards 
to see if an implementation meets requirements, we can then still 
define how hard to be about these things. Saying in advance that some 
projects do not need requirements, design etc is a recipe for the 
disaster. Please note that updating an ebuild is not a project. We also 
need a good definition of a "project".

Gentoo so far has followed a waterfall model, without visible design 
and requirements phases, or visible but approved and tested by the 
developer.

> First Steps
>
>  target audience

Too much examples here. Nothing really concrete.

> Invite Participation

"Invite participation" alone is not good, and is not a software 
engineering standard, rather a marketing requirement.

This should be a guideline at max: Identify USE cases by inviting 
participation, document user USE cases, and ask people to read your USE 
cases before sending it of to the management/releng team.
Please note that USE cases are not only USER USE cases but there are 
also DEVELOPER USE cases. It is highly unlikely that inviting 
participation will not lead to feature-bloat and unrealistic first 
iteration requirements. Actively recruiting people is also not a 
requirement (we have a separate project for that). If requirements can 
be met with only passively recruiting or no recruiting at all, why not? 
We have a recruiting team continuously looking for additional forces?

Keep in mind that the target audience is not necessarily a set of 
developers. Also, geeks do not necessarily dislike software which 
newbies can use (cfr. OS X).

> Create a High-Level Development timeline

A timeline is not a good idea.

Completion dates are prone to change in an open source project because 
people work when they have time. A schedule in which the amount of time 
needed to realize each USE flag is discussed is to be preferred 
(possibly in combination with a weekly status update showing how much 
time was used) over a hard deadline. Hard deadlines are only good for 
project wide releases (2004.1, 2004.2), to help determing whether a 
completed features falls in a release. And it is possible to see when a 
feature will be completed based on the status updates. No status 
updates means no project progress.

> Next, develop a tenative development roadmap for your effort that sets
> general
> completion dates for:
>
> 1) project start (public invitation)
> 2) initial work on design/requirements doc
> 3) completion of design/requirements doc (should be at least a month
> after #2)

cfr supra.

This is not a good template because it lacks the approval argument. 
Approving your own requirements will lead to half-baked projects, 
features that shouldn't be included because they're half-baked, ...

> In addition, your roadmap should also define how the design,
> development and
> testing process should proceed, so it should define steps such as:

We should have standards for those too.

> Gentoo is a volunteer project, and the nature of volunteer work can
> make it
> quite challenging to meet deadlines. Fortunately, there are quite a few
> techniques that can be used to make your development effort resilient,
> so that
> it can meet the unexpected challenges of volunteer development,
> whether this
> means flexibility in dealing with needed adjustments to deadlines,
> coping with
> changes in developer interest and activity, etc.

> One absolutely critical aspect to ensuring success in an uncertain
> environment
> is to *clearly define the requirements for the results of your
> effort*.  "Hard
> requirements" are specific, testable aspects of your end-product --
> aspects
> that must be achieved in order for your development effort to be
> considered a
> success. If you're behind schedule, you can't compromise on a hard
> requirement
> -- doing so would undermine the entire purpose of the development
> effort. Instead,
> your schedule will need to be adjusted to reflect the your current
> progress.

Not a good text. The schedule is hard linked to the work performed. 
That argument does not hold in a project where people can only work on 
things in their free time. The only people that can 'cope', 'deal' or 
'compromise' are those having time to do so, and wanting to do so. Time 
needed + time spend can lead to estimates on when to expect something.  
I would agree to your schedule if there was a paycheck in my bus for 
every project, but unfortunately it isn't.

It looks like some parts for the document were cut/pasted from an 
email, and I'm not sure the target audience is the same. It might help 
to base the document on 
http://standards.ieee.org/reading/ieee/std_public/description/se/

I'd have at least standards for  SQAP, SCMP, SPMP, SRS, SDS, and STD . 
It would be a good idea to provide template documents to fill in (like 
the release notes) instead of a long list of requirements, otherwise 
you'll not be seeing much documents.
Not all people read the DTD's or xml schema's. It would be a good idea 
to have a tuturial on how to write gentoo XML. Otherwise templates are 
needed.

SQAP: software quality assurance plan - coding conventions - 
documentation standards?
SCMP: software configuration management plan - CVS artifacts to be used 
- who is allowed to commit, who isn't...
SPMP: project management plan - the management bits (see IEEE format 
example)
SRS: software requirements (USE cases, UML?, functional/non-functional 
requirements)
SDS: software design document (design patterns, diagrams, ...)
STD: software test description (how to test the software works)

I have some good tex templates if those are needed.

Best regards,

Pieter Van den Abeele


--
gentoo-releng@g.o mailing list