Re: [gentoo-portage-dev] Multiple language use, `component-based' design, and other issues

[Pieter Van den Abeele]

Hi,

On 12 Jan 2004, at 06:14, Jeremy Maitin-Shepard wrote:

> The main advantage cited for using multiple languages has been that
> certain languages are `better' for doing certain types of things.  For
> example, it has been argued that Prolog should be used for dependency
> calculation because programs written in it can be proven correct.  I
> fail to understand why it is particularly important that the portage
> dependency checker be ``provably correct.''  To my knowledge, there
> have
> been no significant problems with there being bugs in the current
> dependency checker.  It is also useful to note that there are tens of
> thousands or more software programs, far more critical than a portage
> dependency checker, which are not written in languages in which they
> can
> be proven correct, but which operate quite adequately.  It is perhaps
> useful for the software on the Spirit Mars rover to be provably
> correct,
> but that is simply not a useful guideline for the portage dependency
> checker.  It has also been argued that it is more convenient to write a
> dependency checker in Prolog, compared to other languages.  I can
> assure the reader that implementing a topological sort algorithm in any
> language is not overly complex; certainly not sufficiently complex to
> justify doing it in another language and dealing with language
> interoperability problems and adding an additional compile-time and
> possibly runtime dependency.

Ah, this is a common misconception. It is true that a topological sort 
can be realized in every language. There is certainly no clear 
advantage of using prolog for that. However, what happens -before- 
applying that ordering and which is often forgotten, is 'computing a 
configuration' or 'computing a model' if you prefer. Let me explain 
with a simplified example.

Right now current portage performs a walk through the dependency graph. 
Please consider the following simplified example:

kde -> linux.
kde -> bsd.

means that kde depends on either linux or bsd. So if a user told 
portage he/she wanted kde, portage could suggest either linux or bsd. 
However if a user also told portage he/she wanted component 'foo' and 
it is known that

'foo cannot be installed on bsd'

the bsd option for kde is no longer a relevant option, because it is 
excluded by a constraint introduced by adding foo to the configuration. 
Portage currently does not feature such constraints (which are quite 
similar to the 'conflict field used in debian packages'. I think it 
will be hard to implement similar behaviour. Also, this is a simplified 
example. An example from real life shows that:

kde -> qt(use kde)

So if you want to install kde, qt should be installed with option 'kde' 
enabled. But similarily, a component should be able to constrain use 
flags of another package. Or a combination of components could be 
constrained, ...

Reasoning about these things is much much easier in something like 
prolog. But the real advantage here is that prolog can be regarded as a 
proof engine and thus is able to provide a proof why a component with a 
given specification is in a configuration. It is important to ensure 
that the configuration returned is correct and satisfies user 
constraints/requirements. So in that sense, it will be 'provable'. But 
again, I don't see prolog as an absolute requirement; everything which 
provides the equivalent of a Turing Machine should can be used to 
implement what I'm doing now in Prolog. The set of features that can be 
reused in prolog happens to be bigger than for instance C++ or anything 
else for this problem, which speeds up development of an explicatory 
prototype.

> It has been mentioned that portage-ng will have a `component-based'
> design.  It is not clear what the term `component-based' means exactly,
> and so it would be useful to get some clarification.  To me that term
> suggests that portage-ng will use some sort of complex runtime dynamic
> linking model.  I would argue that it may be simpler to simply handle
> all optional functionality, if there is any, through the use of USE
> flags, rather than going to the trouble of supporting such a complex
> model.  Clearly, some type of static linking could only be supported if
> portage-ng did not depend on a runtime dynamic linking model.
> Advantages to allow static linking include greater robustness in cases
> of failure of various shared libraries, and the dependencies of portage
> will not need to be handled as carefully, because there will be no risk
> of breaking a statically-linked portage.

You make some valid points in this last section. My interpretation of 
component based design is that we'll allow users to create their own 
component (which uses a specific interface) and link those to the 
program. Whether that linking happens at runtime or compile time is not 
specified yet, but I wouldn't exclude one.

Examples of such components that were given in the past include a 
'logging strategy', a 'compression strategy', a 'parallel compilation 
strategy', an 'installation strategy', ... The bottleneck areas in 
current portage. What we don't want portage-ng to be is one huge file 
with over 10 thousand lines of code. 'Components' is a generic term 
we've introduced and which includes 'patterns', 'objects', 'modules', 
'tiers', 'layers', all concepts which facilitate parallel development, 
community development without focusing explicitly on one paradigm.

Pieter


--
gentoo-portage-dev@g.o mailing list