Re: [gentoo-dev] Portage programming question

[Robin H. Johnson]

On Sun, Mar 30, 2003 at 01:13:24AM -0800, George Shapovalov wrote:
> On Saturday 29 March 2003 23:51, Robin H. Johnson wrote:
> > I'm going to write some C++ or Java instead for what I want. Python
> > seems to lack the Sets and other sequence datatypes that would be
> > greatly useful.
> Well, not sure what are you referring to, but python sure does have sequence
> types and sets. In fact all sequence types provide set abilities.

> You might want to take a look here:
> http://www.python.org/doc/current/lib/typesseq.html
> and here:
> http://www.python.org/doc/current/tut/node7.html
I looked at those originally when learning python properly earlier
today.

My definitions:
A LIST is a group of unordered and possibly duplicate items. As in
LINKED LIST. Commonly O(n) time to access any item. O(n/2) or better if
cleverly implemented. O(n) time for add/remove/contains/size. O(1) for
add/remove (after search).

An ARRAY is a LIST with additional pointers to each element allowing
constant time O(1) access to each element. O(n) time for arbitrary
add/remove. 

A SET is a group of items that are all unique and are accessed in
naturally sorted order. O(1) time for access/add/remove/contains/size.

A MAP is defined as a SET with each element being the key to another
data structure. (Python provides this as a dictionary). Same big-O as
SET.

A BIMAP is best explained as a pair of SETs doubly linked in MAPs. It is
MAP that can be used in both directions, with significent memory, speed
and consistancy improvements, over just using two MAPs.
An explaination of BIMAP:
A is the set of (A1,A2,A3)
B is the set of (B1,B2,B3)
BIMAP.parents() might be: {A1=>(B1,B3),A2=>(B2),A3=(B2,B3)}
BIMAP.children() would then be: {B1=>(A1),B2=>(A2,A3),B3=(A1,A3)}
The actual data of A and B is stored seperately and references to them
are used since each item will appear at least twice.
BIMAPs offer the major advantage that an element from set of data can be
used as a key to obtain related items in constant amortized time.
O(1) for almost all operations.

Hashtable implementation of SET/MAP/BIMAP is cruicial for high
performance.

Python's implementation of a MAP (dictionary) implemented via hash
tables, but I can't find anything on tweaking the load factor and
initial capacity, which makes a significent difference on data sets of
large but known sizes.

For these structures, various iterators are required as well, as this is
the best way to traverse the data. Functional programming methods to
apply to the data is also required, such as (Python provides these first
three) filter, map, reduce, (Python doesn't seem to have these last
three) set union, set difference, set intersection.

For a SET, I considered using MAP and just ignoring the data field, but
that seems like a terrible waste of cpu and memory. I find it really
strange that Python doesn't have a SET while it does have a MAP, esp.
given that MAPs are a logical extension of SETs.

It seems Python did at one stage consider adding SETS:
http://www.python.org/peps/pep-0218.html
But I can't make much sense of the outcome.

It seems I would have to implement BIMAP myself, since there isn't one
in Python that I can find at all. I would admit that it is significently
less common data structure than a list/map/set. I did have to write my
own Java implementation of it, but it is available in other functional
programming languages (Mercury provides a very good example).

BIMAP is the main structure that I would be using, but SET and MAP also
to a slightly lesser degree. I want to create a BIMAP containing the
relative paths to each ebuild as one key, and the IUSE data from that
ebuild as the other key.

-- 
Robin Hugh Johnson
E-Mail     : robbat2@××××××××××××××.net
Home Page  : http://www.orbis-terrarum.net/?l=people.robbat2
ICQ#       : 30269588 or 41961639
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85