Re: [gentoo-portage-dev] Plugin backport PATCH (1/2)/(2/2)

[Brian Harring]

On Sun, Nov 20, 2005 at 11:33:02AM +0900, Jason Stubbs wrote:
> On Sunday 20 November 2005 01:11, Brian Harring wrote:
> > On Sun, Nov 20, 2005 at 12:50:25AM +0900, Jason Stubbs wrote:
> > <regarding FsLocks and keeping fds open while unlocked>
> > > I still don't see why fds should remain open after all locks are
> > > released. What my point above is about is that if some external process
> > > creates a bunch of FsLock objects that it keeps around to obtain future
> > > locks (bad design, but) fds will quickly become used up when they are not
> > > actually being used for anything.
> >
> > How about a configurable?
> >
> > From where I'm sitting, running out of fds *really* shouldn't occur.
> > This locking is strictly fs orientated, synchronization points on the
> > fs shouldn't be too many.
> >
> > Via a configurable, we can avoid open/close for every unlocked -> lock
> > and lock -> unlocked transition; if fd exhaustion becomes an issue,
> > we can just modify the class to ignore the configurable, so that it
> > falls back to open/close during transitions.
> >
> > Compromise of sorts, plus it's probably useful for the class. :)
>
> Still not seeing the benefit... All usage of FsLock within the patch is to
> create an instance, lock it, unlock it and then destroy it. I can't think of
> any usage that differs from that in the rest of portage code either.

Common code.  Stable may not use it, but I intend to in savior...

> > Offhand... it's a bit nuts, but conversion of stable to the class
> > based instance might be worthwhile, rather then the crap func route we
> > have right now.
> >
> > One issue though is that portage_locks will cycle through flock/lockf,
> > then falling back to the evil hardlink locking- this code is strictly
> > flock.  Thoughts on trying to clean up that crap in stable, or just
> > leave the mess as it is and try and replace it down the line?
>
> Moving over sounds good to me. I don't really trust the current lock_method
> cycling. It usually comes into play with network mounts, right?
Reiserfs managed to kick the cycling into lockf rather then flock in a 
few occasions.  Pretty much if the setup is questionable/stupid/laggy, 
falling from flock to lockf to hardlink probably will occur.

Would have to do some digging to find the scenarios; nfs is one of 
'em, as is samba.

> So unless all
> machines that are trying to access happen to choose the same lock_method,
> locking is essentially disabled anyway.
Not disabled... silently thinking it works.
It's evil. :)

> Being able to configure what locking
> method is used would be much saner...
Agreed, although locking per fs is the real need, not global (I make 
no comments about complexity involved, since I don't know).

> > > > Curious, qualms about using a massively cleaned up version of this?
> > > > The reason I wanted this seperated out and backported was to go with
> > > > an actual registry, and abuse the existing code- specific complaints
> > > > with that route, beyond patch sucking?
> > >
> > > None here. For what it's worth at this late stage, this patch is much
> > > better than my attempt. :)
> >
> > This is version 3 of the savior plugins registry.  You *really* do not
> > want to see the first two I created (hence them never being in cvs).
> > It's a fun bit to attempt :)
> >
> > So... assuming a plugin registry with on disk files is an agreeable
> > approach, what would be the initial targets?  Cache comes to mind, but
> > what else?
> >
> > The problem with a registry is it just points at the func/code to load
> > up; for elog (fex), it's not binding any configuration information to
> > it- it *could* be managed by pointing at a func that holds the
> > configuration info, but that's massively unclean (plus it'll stomp on
> > 3.0 goals of code/config seperation).
>
> This sounds like a "fun" issue. Might leave it dangle for a bit. ;)
Agreed.  Just throwing it out there so people don't go and do it and I 
have to yell at them about mucking things up. :)

> Speaking of versions, nothing has been decided yet in the 2.0.54/2.1.0/2.2.0
> debate. It's pretty much decided that 2.0.x will be only small sets of bug
> fixes from now on, right?
No complaints over a week...ish + everyone is around == works for me.

Marius, willing to live with 2.1 as next minor release?  I'm not 
really much for skipping 2.1, but just because I'm noisy doesn't mean 
I'm right (nor that I speak for the majority)...
~harring