Re: [gentoo-soc] GSoC suggestion - Emerge or some of the Portage features implemented in Common Lisp

[Andrew Savchenko]

Hi!

On Sat, 24 Mar 2018 04:48:53 -0400 Crystalsun wrote:
> Hello everybody!
> I want to share some thoughts on a project that can be
> implemented during Google SUmmer of Code.

Thank you for your interest in the Gentoo GSoC.

> So, moving on to the proposal - I suggest that implementing
> emerge or some of its features in Common Lisp may be a great thing.

I'm not a Lisp guy, so I'm CC'ing Lisp community.

> The first and main reason for that is the interactiveness Common
> Lisp provides (and Python doesn't).

You should know that the portage (where the emerge tool belongs) is
not the only one implementation. We have Gentoo Package Manager
Specification (PMS)[1] detailed enough to implement alternative
implementations. Right now we have three of them: portage[2],
pkgcore[3] and paludis[4]; the first two are in python, the later
one is in C++.

> For example, when emerging a package, if an error occures, the
> whole process will die since the merge of the current atom.
> However, if emerge was written in Common Lisp, it could be possible
> to interactively debug the error and continue the process, which,
> overall, means more user control of what's happening and better
> hotfixes, thanks to the fact that the REPL is part of the language
> itself, not provided by the *outer environment*.

I do not understand what you mean as interactiveness. Python
support interactive execution shell with ipython. But it should not
be an issue here anyway, see below.

What exactly do you mean as an error? If we are talking about
internal portage crash, such cases are extremely rare and are
usually promptly fixed.

If you mean failed dependency calculation, this usually happens due
to impossibility of building dep graph for present packages, USE
flags, package manager (PM) options and other constraints. In order
to fix them changes must be made (e.g. user should change some USE
flags) and dependency tree graph needs to be build again. This is
the most time consuming part and it doesn't matter if PM will be
restarted as an application or not, because most time will be spent
on graph building, not on application restart.

If you mean failed package builds, there is no need to start emerge
from scratch. It has --resume option to resume operation,
--skipfirst may be used to skip last failed package. Option
--keep-going allows to try to build packages as long as possible:
after a package build failure a dependency tree is being
recalculated in order to rebuild all packages not blocked by one
failed to build.

So I don't understand what knew user visible features will be
available thanks to Lisp.

> Nevertheless, changing emerge also means changing ebuilds, which
> is truly a big change.

Why ebuilds must be changed? The PMS is PM agnostic. It doesn't
matter using what language PM will be implemented: PMS defines
abstract API and this API can be implemented in any Turing-complete
programming language.

> Thus, I understand it may be too hard/long to implement, so a
> compromising, yet useful solution might be available. For instance,
> it could be fairly great to localize some task Portage is slow/bad
> at, and reimplement it (what about resolving dependencies
> interactively?). That might be a good place to start already.

Frankly I really doubt that Lisp will be faster than Python for
this task. Both are quite high level languages.

Of course you idea is far out of the scope of the GSoC time frame,
but if Lisp team will found it useful, maybe you can start with
some skeleton implementation.

[1] https://projects.gentoo.org/pms/6/pms.html
    https://wiki.gentoo.org/wiki/Project:Package_Manager_Specification
[2] https://wiki.gentoo.org/wiki/Project:Portage
[3] https://github.com/pkgcore/pkgcore
[4] https://wiki.gentoo.org/wiki/Paludis/Guide
    https://paludis.exherbo.org/


Best regards,
Andrew Savchenko