Re: [gentoo-dev] Revisiting GLEP 19

[Stuart Herbert]

On Wednesday 21 July 2004 17:34, aeriksson@××××××××.fm wrote:
> > Two things.  First off, I'd hope to achieve a lot more than just adding a
> > [S] entry to emerge -pv.
>
> Care to elaborate? "It's nifty for the future" is a bad argument. Far
> too many times have I came across people who thinks all problems goes
> away if the solution is implemented using xml (I'm NOT saying you're
> one of them). Present the problems you want to fix, and we'll discuss
> them.

Let's deal with the XML vs plain text argument first.

I'm not an XML-everywhere person, but I do believe that XML is *the* right 
technology when you want to add meaning to structured data - meaning that can 
be re-used in software tools.  It's nowhere near as difficult to use as 
ASN.1, not as clumbsy as the plain-text markups in common-use, and if you do 
read the raw XML in a text editor, most people can follow what the file says.

You *can* do everything I'm suggesting below using a plain text file - 
provided everyone follows the same conventions.  You might even be successful 
in getting everyone to keep to the conventions.  But using XML for the job 
gives your tool-writers instant access to rich libraries for parsing, 
manipulating, and verifying the data.  These libraries are available to 
programmers in all the major languages (except bash alas ... perhaps it's 
time for someone to change that? ;-)  Tools that work on XML data are much 
easier to write, and much easier to maintain.

Our changelogs are supposed to contain the following information:

- package version affected
- list of changed files
- reason for change
- bug(s) addressed by the change (inc security issues announced through GLSAs)
- developer who made the change
- user(s) who originally suggested the change (credit for patches, ebuilds, 
and the like)

Let's look at what we could do with this sort of data.

Now, if I'm reading a changelog, if I want to see what issues are fixed, the 
changelog isn't normally enough.  Why?  Because it's very common for the 
changelog to only contain a bug number and not a description.  Normally, you 
have to open up a browser, and search for each bug in turn on 
bugs.gentoo.org.  It's not exactly hard work, but we could make a better 
experience.

There's no reason why the changelog couldn't contain both the bug id and the 
summary line from bugzilla.  There's no reason why a GUI tool like Porthole - 
or even the command-line based emerge - couldn't provide you with a [more] 
link to go and get the full discussion from bugzilla.  Instead of unconnected 
data stored in different places, we now have connected data and a means to 
connect them.  That idea - of making it possible for tools to connect the 
data - of making it possible for the tools to understand the data - is where 
the real value of XML lies.

If we were really wanting to take this further, for those packages that 
provide a list of upstream bugs that are fixed in a release, we could store 
that list in the changelogs too, with summaries and click-throughs.  That's a 
bit more work, and not everyone's idea of fun, but it could be done.  At the 
very least we could provide a link to the upstream changelog.

That's just looking at one aspect of bugs and changes.  Being able to say 
whether a bug fix is security-related or not is another aspect.  That's just 
another type of change information.  It shouldn't have to be stored and 
treated as a special case - which is where the glsa-check tool seems to be 
going.

Let's look at the reasons why a new version of a package is added to Portage.  
I believe 'version bump' will prove the most popular in our changelogs right 
now.  Is it to address serious bugs?  Is it for security reasons?  We could 
standardise that into a choice of one or more reasons.  That's information 
that can then be used by GUIs such as Porthole and packages.gentoo.org.  
That's information that users could search on.  

Where is the upstream changelog?  Something else that can be added.  If you 
write your tools correctly, you can introduce new types of information into 
your XML data without having to change your tools.

I'm in the information business.  I work on publishing tools for developers.  
I work on a CMS tool (yes, it's XML based).  I write about my work.  I work 
with information probably more than I work with code.  Most information just 
doesn't exist in isolation.  Most information is really just information 
about yet more information, information that people don't read because it 
isn't served to them on a plate.

It's not about getting left behind.  I didn't buy that argument during the 
dotcom boom, and I definitely don't buy it now.  What it's about is making a 
lot more of what we already have, by making it more available.  To make it 
more available to people, you have to make it more available to the tools 
first.

I'm not looking for problems to fix.  "It ain't broke, so don't fix it" isn't 
always the best advice.  Sometimes, the thing to do is to look at what is 
possible, rather than just accept what you're stuck with today.

Now, maybe this information should be part of the metadata for an ebuild 
instead of being in a 'ChangeLog' - as Weeve was aluding to earlier in this 
thread.  From an SCM point of view, it's all configuration information 
related to a change, whether you want to call it a 'ChangeLog' or 
'metadata.xml' file or whatever.

> Why not add that bit of info the the ebuild which incorporates the
> fix? "SECURITYFIX=url1 url2 http://www.gentoo.org/GLSA1234whatever"
> would be all need.

That's one way to do it.  Build another special case into ebuilds and into 
Portage.  It'll work.  But if the information was somewhere in XML, it'd be 
an easier change to introduce.  All you'd have to do would be to update an 
XML schema.  You shouldn't have to update tools.

> Or put the other way around, why not move
> all the other headers in ebuilds to xml, and use xml-aware tools to
> execute on them? (joke)

That's a topic that keeps cropping up.  If you look at our GLEPs, there's one 
currently open with people chomping at the bit to make that happen, at least 
for some of the information.

Best regards,
Stu
-- 
Stuart Herbert                                              stuart@g.o
Gentoo Developer                                       http://www.gentoo.org/
                                                   http://stu.gnqs.org/diary/

GnuPG key id# F9AFC57C available from http://pgp.mit.edu
Key fingerprint = 31FB 50D4 1F88 E227 F319  C549 0C2F 80BA F9AF C57C
--