Re: [gentoo-dev] rfc: openrc service script dependency checker

[Andrew Savchenko]

Hello,

On Thu, 4 Dec 2014 08:59:22 -0500 Rich Freeman wrote:
> On Thu, Dec 4, 2014 at 7:54 AM, Andrew Savchenko <bircoph@g.o> wrote:
> >
> > 1. There are multiple services having "after $all" statement (an
> > analog in Gentoo is "after *", which is currently used only by
> > local init.d script).
> >
>
> Seems to me that the solution to this is to ban this sort of syntax
> entirely, and use proper deps.

There is no serious problem in Gentoo here, because local is _the
only_ service that uses this kind of dependency. But in Debian
"after $all" syntax is in common use and we can't change that.

The prime cause of these patches was to bring OpenRC as usable init
system for other distributions and not to limit its scope only to
Gentoo. And Debian has requirement of additional features in order
to allow OpenRC to be used as a reliable replacement of other init
systems.

> WHY do we have to run local.d last?
> Why do we have to run any of that other stuff last?

I doubt this is in POSIX standard or whatever, but it is a quite
common custom among distributions to have some rc.local service
where admins may put their custom daemons to be started in quick and
dirty way, so that admins without prior knowledge of how particular
init system works may put their stuff to that rc.local script/
service/whatever. This is mostly needed when out-of-tree packages
are used or in-tree packages lack init script. The former is quite
common in production, unfortunately.

> > 2. LSB dependencies are allowed to be asymmetrical relative to start
> > and stop, while in OpenRC they are symmetrical. This yields to
> > loops in OpenRC while in LSB the same services work fine. Example
> > follows:
>
> Seems like the solution to this is to allow a different syntax for
> start vs stop dependencies then, with the default being to keep them
> the same if the current syntax is used.

Yes, this particular case can be solved with optionally separated
start and stop dependencies.

But look at the whole picture. What happens _now_ if there is a loop
in service dependencies?

For the sake of unambiguity under "soft" loop I mean loop where
there is at least one "after" or "use" dependency and "hard" loop
is supposed to consist only of "need" dependencies.

1. OpenRC fails to start _all_ packages in the loop (both soft and
hard).
2. This takes enormous amount of time: 60 seconds per loop and
this time may be precious (e.g. system is on low battery).

What proposed solutions do?

1a. For the first issue early loop solver allows to start all
services from a *soft* loop gracefully, so that all services should
work, because all mandatory "need" dependencies will be satisfied.

1b. If loop is *hard*, then later loop solver will intelligently
break loop at some point, so that as much services as possible will
have their "need" dependencies satisfied. Of course, some services
may fail in this case, but this is still better than fail _all_ of
them.

2. No time will be lost in case of *soft* loop and time loss will
be minimized in case of *hard* loop.

> > So it is a statement of fact that OpenRC
> > should be compatible with LSB dependencies.
>
> It seems to be a statement of fact that OpenRC ISN'T compatible with
> LSB dependencies.  What it "should" be is anything but a statement of
> fact, which is what the word "should" means...

Yes, it is not compatible now. And if OpenRC wants to step out of
Gentoo scope to other distros, it should be. But as one can see,
Gentoo will also benefit from proposed fixes, which will made OpenRC
init system more robust and error-prone.

> I don't get why Gentoo gets by just fine with things as they are, but
> nobody else apparently can. Just fix your dependencies.

I don't have broken dependencies on my systems right now. And this
is not a discussion of my personal issues at all. What I'm trying
to do is to make OpenRC robust and resilient to errors which may
occur as well as expand its scope outside of Gentoo as a mature
init system.

> > Warnings for users about loops is a good idea for Gentoo, but will
> > produce a lot of not always wanted output on Debian, that's why
> > this option should be configurable.
>
> I'd call these situations errors - a warning should certainly be used.

They are errors in Gentoo, but in some cases are normal in Debian,
because dependencies are asymmetric there. That's why switch to
omit them was requested. 

> > As for Gentoo it is desirable too, becase it is better to boot
> > system somehow instead of not booting it at all (or with long
> > delays due to 60-seconds timeout on service startup). This is
> > crucial for remote servers, e.g. admin needs to reboot machine due
> > to critical security kernel update ASAP and having it hang during
> > boot is really a very bad idea.
>
> So, I'm fine with some kind of emergency mode, though expecting it to
> work on a remote server might be asking a bit much.

Why? Especially if this is doable and there is already working code
present.

> Dropping to shell is certainly preferable to just hanging.

Yes. But booting as much services as possible is even more
preferable, especially when box is remote.

> You certainly couldn't
> guarantee a successful boot, since the configuration contains errors.

Nobody can. But we can raise the probability of successful boot.
 
> > Another example from my experience
> > is emergency shutdown due to power failure and low battery signal
> > from sys-power/nut. I had several nasty cases when system failed to
> > shutdown properly due to 60-second timeouts for services failed to
> > shutdown — battery just ran out of charge while OpenRC was try to
> > do thing "right way".
>
> Seems like the solution to this is to let you configure the timeout
> per-service and globally, and maybe even give you the ability to
> override it at time of shutdown.  During a routine shutdown you don't
> want the service manager randomly killing stuff.

It is already configurable globally IIRC. What will be probably
more useful is to have separate start and stop timeout values and/or
maybe a way to set emergency shutdown timeout value.

Best regards,
Andrew Savchenko