Re: [gentoo-dev] Uncoordinated changes

[Michał Górny]

On Sun, 14 Feb 2016 12:37:33 +0100
Patrick Lauer <patrick@g.o> wrote:

> On 02/11/2016 09:15 PM, Patrick Lauer wrote:
> > Now instead of looking up [metadata.xml] -> (herd name) -> [herds.xml]
> > -> email it goes backwards:
> > [metadata.xml] -> (maintainer type=project) -> email -> [projects.xml]
> > -> Project name
> >
> > Since this involves XML and python's ElementTree library it's a
> > nontrivial change that also removes a few now useless helpers
> > (_get_herd_email has no reason to be, but we'd need a _get_herd_name
> > helper instead. Err, get_proj ... ah well, whatever name works)
> >
> > And all that just so (1) gentoolkit output works and (2) euscan updates
> > properly. Both of which I don't really care about much, but now that
> > I've invested ~4h into debugging and trying to fix it I'm a tiny bit
> > IRRITATED.
> >
> So this turns out to be more fun than expected.
>
> Having spent a little bit of time staring at XML, DTDs and wondering why
> we do things the most difficult way ...
>
> Previously the herd tag was defined as:
> <!ELEMENT herd (#PCDATA)>
>
> So we end up with, for example:
> <herd>kde</herd>
>
> The new schema collapses herd (err, project!) into maintainers (err,
> sustainers ... staff ... linchpin?)
> And maintainer is defined as:
> <!ELEMENT maintainer ( email, (description| name)* )>
>
> Which means that only email is mandatory. So instead of search by name
> you are now required to search by email.

Congratulations! After the whole discussion, GLEP, explanatory blog
post and explanatory mails, you finally figured out how things work!

> And it leads to inconsistent (partial) duplication: Some metadata.xml
> entries carry Name, some Description, and some are Email only.
>
> For example for gentoolkit this means that instead of search by name now
> it needs to be search by email, and the previous search by name
> functionality requires herds.xml, err, projects.xml to figure out the
> name of a project. Which might not match the one in metadata.xml!
> (And you may need to filter out maintainers-that-are-not-projects, and
> what about maintainers that are undefined? So much extra code complexity!)

Everything becomes complex if you try hard to make it sound complex.

Maintainer is a single entity, and it's identified by e-mail. So if you
want to group packages, you just group by e-mail. Simple as that. No
special magic needed. No extra files needed. The basic functionality
works without any special needs. You can also use it straight to
contact the maintainer or assign bugs.

If you want to split maintainers into people and projects, you've got
type="". Which is also in-place, with no extra files needed.

If you want pretty project names, then you can use projects.xml.
Or the name in metadata.xml. Both are fine by definition.

Now, the surprise: current people-maintainers could already have
different (or no) names in different metadata.xml files! You didn't
expect that, did you? In other words, that's not a new issue, neither
a major problem.

> And this is why I avoided the topic and hoped that the 'migration' would
> make sense:
> (1) Using XML is mildly insane. Neither machine- nor human-readable

Wrong. It works for machines well.

> (2) The DTD is even more insane, and few people have the patience to
> figure it out

And what do you need the DTD for? Furthermore, it's in process of being
replaced.

> (3) The recent changes to the DTD change the data model in subtle ways
> so that there's even *more* denormalization possible
> (4) The tooling is, due to XML, wonderfully horrible and requires things
> like XPATH to get the required data (because query by attribute is
> harder than query by tag)

Of course it does. Because no modern programming languages provide such
complex features as conditionals!

> There's fundamental questions that should be handled before doing more
> modifications - for example, should the data be more normalized (e.g.
> name only in projects.xml / maintainers.xml and only email in
> metadata.xml)? If we allow denormalization, do we have tools to check
> and autocorrect (e.g. a maintainer changing name)?
>
> Once we decide to abstract it away so that people should use tools and
> not mangle it manually (have you looked at herds.xml ?! omg ...) there's
> the question ... why XML? It's about the worst format for this job, INI
> format is sufficient and easier to parse. Or JSON, or YAML, or whatever
> is trendy now. Or do we autogenerate from templates?

What is the gain? Who is going to fix all the tools?

> Another funny thing: projects.xml is not in the same repository, so
> synchronizing changes gets more tricky. And the metadata.dtd is in yet
> another place. Wouldn't it make sense to have this organized in a less
> confusing way?

projects.xml is autogenerated from wiki. Yes, the place you refuse to
visit. Which means you'll never exist in projects.xml.

DTDs are not needed for anything, except for doing poor man's
correctness verification.

> You see where this is going - and why I didn't object loud enough to the
> changes: I want to not care about this whole cluster of topics and do
> things that are more rewarding. But that choice got taken away when
> things broke (oh, they didn't break, they Function Differently now) and
> I had to spend some time investigating why things deviate.

Of course you had to. Because reading is hard.

-- 
Best regards,
Michał Górny
<http://dev.gentoo.org/~mgorny/>