Re: [gentoo-dev] glep 42 (news) round six

[Brian Harring]

On Sun, Dec 18, 2005 at 06:23:55AM +0000, Ciaran McCreesh wrote:
> On Sat, 17 Dec 2005 21:50:47 -0800 Brian Harring <ferringb@g.o>
> wrote:
> | Drop the magic-chicken crap (especially in light of your comments
> | about 'professional' news inline in the glep).
>
> Eh, there aren't any magic chickens in the glep.

Intention was a nudge about keeping snarky comments out of the glep, but yep.


> | Not really.  Just requires your code to be space safe.
> |
> | You write good code, right?  Especially since you need to keep our
> | path handling safe for osx (/volumes) and for users who do strange
> | things?
>
> The problem isn't my code. My code's fine. So is eselect. The problem is
> people who write their own handler scripts to suit their own needs, and
> then get nastily bitten because they don't realise we're allowing
> spaces in filenames.

People writing their own plugins/readers are responsible for what they 
create.  They *still* need to handle space's in file paths, thus as I 
stated the requirement is arbitrary and uneeded.

Already addressed this in irc; you provide a framework.  If plugins to 
it suck, that's not your problem, nor is it a valid reason to 
constrain things just because someone might manage something stupid 
(remember this is gentoo, that line of logic would lock CFLAGS down 
to a sane set).

To head off the "don't make features that are easily screwed up", this 
isn't one of them- this is expecting code to behave correctly from the 
path standpoint.

Yes it's harder on bash crap, but that's also a reflection of bash 
(it's not an issue in python :-P ).


> | > News items should be signed with a detached GPG signature: ::
> |
> | why should vs must?
> | Should leaves open the possibility for a tree compromise and a news
> | item with Very Bad(tm) instructions stuck into it.
> |
> | Moving towards getting the whole tree signed, introducing new
> | components that aren't required signed works against that.
>
> I don't want to introduce any signing requirements that we don't have
> already. When signing for everything else becomes mandatory, signing
> for news would be too. If I make this a 'must', someone will ask me to
> specify how we're handling the Gentoo keyring...

Pawn the keyring off on others.  The issues isn't an established trust 
ring, it's required signing- yes, a trust ring makes things a helluva 
lot easier on the user front, but it's useless without a required 
signing policy.

We've already had this conversation also btw, in the 
beginning of glep42 iirc.  Obviously I don't agree 
with your reasoning "I'll do it when it's required I do it".  It's 
useful now, it becomes massively more useful when a trust ring is 
available.


> | > ``Revision:``
> | >     Initially 1. Incremented every time a non-trivial change is
> | > made.  Changes which require a re-read of the news item should
> | > instead use a new news item file. Mandatory.
> |
> | non-trivial changes that don't require a re-read sounds like a
> | contradiction.  Clarify, especially since portage will mark this as
> | read _once_ and only once.
>
> Hrm, word it as "Changes other than minor formatting tweaks", or remove
> "non-trivial"?

It's not a wording thing, I'm pointing out sans spelling corrections 
and trivial word mangling, any new info jammed in requires a new item 
bump so readers can display the changes.

In light of that, wording above needs correction.


> | This isn't incredibly useful if ranged versions are ever introduced.
> | Ammending the glep for that seems stupid, looser language might be
> | wise.
>
> What's the syntax for ranged versions? And how do they differ from SLOT
> versions?

>=kde-base/kde-libs-3.0 <=kde-base/kde-libs-4.0
It's not syntax as much as a boolean and of atoms.


> | That's an implementation note, but I'm bringing it up since the time
> | to do the filter/cache instantiation is during portage initialization
> | (yes it sucks, but your stuck with stable)... so start thinking about
> | ways to make it fast, since python -c'import portage' is the slowest
> | part of portage queries
>
> Looks like I might have to do that thing in Python rather than bash
> then...

Do it in bash if you like being on the receiving end of atomic 
wedgies.


> Once an hour would work fine. On the other hand, the merge is just
> copying a few small files -- time-wise, it's less than generating the
> cache for a couple of ebuilds.

More then a couple; this beast will bloat up to hundred or so files 
I'd expect (remember translation serves as a multiplier).

Any signed item would need to be verified also, although fortunately 
this chunk can be done in parallel to regen run.


> | > The main rsync tree will **not** use the ``yyyy/mm/`` subdirectory
> | > layout.
> |
> | What shall it use?  Again, be explicit.
>
> + The news item directories will be moved into the single top level
> directory.
>
> Not sure whether we really want to do that. It makes the client code
> slightly easier...

Your decision.


> | You haven't stated how the 'package manager' will trigger the user's
> | reader of choice for these targets.  Should also extend this to allow
> | a way to disable any news notices, lest someone's cronjob get hung
> | displaying news (feature or not, it's needed).
>
> The same way the package manager handles updating config files: it
> won't. It'll just tell the user that some news items need reading.

And you'll personally handle all of the bug spam from feature requests 
that --ask trigger $news_reader?

It's a logical extension, thus people will ask for it.


> | > The package manager must keep track of news items that have already
> | > been added to the unread list to avoid repeatedly marking a deleted
> | > news item. This could be handled via a ``news-repoid.skip`` file
> | > containing the IDs of news items that have already been added to a
> | > ``news-repoid.unread`` file, but this method is not required by
> | > this GLEP.
> |
> | Specifics.
>
> I'm trying to avoid forcing a particular implementation on the skip
> file, since I can think of at least three different ways of doing it
> and I'm not sure which will be fastest...

Name them, and let others in on the sekret...


> | > When a news item is read, its name should be removed from the
> | > ``news-repoid.unread`` file. If a news client acts as an
> | > interactive reader rather than a gateway, it should then add the
> | > name to a ``news-repoid.read`` file in the same directory with the
> | > same file format.
> |
> | implementation issue; you need locking on this.  If portage has to
> | farm out to the users reader of choice, then it needs to lock the
> | file to keep readers/writers from screwing with each other.
>
> We do? Marius told me it wasn't worth it.

I disagree.  It's mainly contingent upon $news_reader being spawned 
via --ask, although it *also* matters heavily if the user is flipping 
through their news while a sync in the background is running- if the 
utility updates on the way out, unless their is some advisorial 
locking involved, you've just lost all of the new synced unread news.


> | Flock preferably, since it's faster then resorting to hardlink
> | trickery (yes this may be harder for shell crap, but so it goes since
> | hardlink locking sucks).
>
> What's wrong with mkdir?

Note the 'preferably'.  mkdir's reliant on all code cleaning up on the 
way out (that and sleep polling).  Flock and friends don't suffer that 
class of issues.


> | > News items can be removed (by removing the news file from the main
> | > tree) when they are no longer relevant, if they are made obsolete
> | > by a future news item or after a long period of time. This is the
> | > same as the method used for ``updates`` entries.
> |
> | implementation issue; updating unread/skip crap so it doesn't bloat
> | is required, but time frame also matters (crap sync resulting in news
> | getting removed, yet it still being valid, just missing from the
> | local tree).
>
> Hrm. We can't take the same approach here as we do with expiring
> updates entries?

We expire updates?  If so, someone might want to look at the updates 
from 3 years back...


> | > There is an existing automated tool [#forums-glsa]_ for posting
> | > GLSAs to the forums. A similar tool can be used for these news
> | > items.
> |
> | Pawned it off on someone, or something you'll be doing?
>
> Hopefully the former. I have it on reasonably good authority that it
> won't take more than half an hour if I end up having to do it though...

Get cracking then (regardless if it's pawning or coding).

~harring