Re: [gentoo-portage-dev] Few things, which imho would make portage better

[tvali]

Ok, i send a lot of them, but hopefully they're interesting :)

I did research a bit about adding SQL support to portage -- as much as
i see, mysql is smallest sql server, which could be emerged with
python module.

In beginning, i think that SQL database structure should be created,
which supports basic fields from ebuilds (DESCRIPTION, HOMEPAGE,
SRC_URI, LICENSE, SLOT, KEYWORDS, IUSE, DEPEND, RDEPEND).

Then emerge --createsql command should be set up, which adds data from
/usr/portage/ filetree into this sql database.

Then, portage_db files should be copied into new folder, where queries
will be brought partially into sql. Emerge --usesql should affect
sys.path = ["/usr/lib/portage/pym"]+sys.path in emerge.py so that user
can control, if she wants to use experimental sql functions. Search
and tree-building are first functions, which should get sql-support,
as they are slowest.

Objective of adding sql support would getting portage search and
pretend functions fast -- so that all real use of portage would be
without long waiting in any place -- only waiting would be in --update
command, then, which could be run when noone is nearby.

I have some questions now:
* Where i could find basic ebuild specification?
* Where i could find basic portage tree datastructure specification,
if there is any?
* Does portage have some code documentation for faster learning?
(or should i just read the code?)

I think that i will start with sql support, then go to UI -- as sql
support seems to have far more important functionality.

I would like to build sql table-structure after getting the
information and send it into this list here so that you could hek out
if it has something missing or poorly optimized. PS. i dont like rules
of database normalizing, so my strutures usually have some "hacks" in
them -- i hope that noone complains about that; normalized structures
are just dull and optimized only for dumb-user-human-readability,
which is imho not the biggest virtue of software code.

2006/3/14, tvali <qtvali@×××××.com>:
> Another place, where it would be good, is that it could interact well with GUI and it's Apply button.
>
>  Another thing, what would be imho improved, is the speed of following functions:
>  * emerge -s, emerge -S
>  They should be cached somehow.
>
>  Also, i think that when speaking about *speed* of portage, which is imho notably slow right now (speed of searching, building trees and so on), sql-support should be considered. In /usr/portage/, i see many folders and files, which tend to contain one or two words, lists of hashes and so on. Actually i see relational database there. With many caches and other optimizations it could be tuned fast, but having SQL doing all that, it would be achieved very simply (maybe it would be a jump to have all linux configurations in one SQL "filesystem"? ...ok, just fantasy:))
>
>  There are several small and simple SQL databases. With such tool, big parts of portage become unnessecary, but it's speed will become much faster. This should be optional if one uses some tiny sql, mysql or big clustered sql with nas and load-balancer, but by default some tiny version of SQL should be built.
>
>  Also, i currently dont see a possibility to see dependency tree of installed package.
>
>
>
> 2006/3/14, tvali <qtvali@×××××.com>:
> >  I did think about some priorities too, so that it could be perfect for me.
> >
> >  It should be possible to add package with a priority. I will give you an use case and explanation how i would use portage.
> >
> >  emerge --justadd kdebase-meta kicker --priority 10
> >  emerge --justadd kdenetwork-meta kdeutils-meta kdeadmin-meta --priority 9
> >  emerge --justadd kdeedu-meta kdetoys-meta kdegames-meta kdeartwork-meta --priority 1
> >  emerge --justadd koffice-meta kdegraphics-meta kdemultimedia-meta --priority 5
> >  emerge --justadd kdewebdev-meta kdevelop htmltidy kompare cervisia --priority 7
> >  emerge --justadd  kdepim-meta --priority 3
> >  emerge --justadd kdeaddons-meta kde-meta --priority 20
> >  emerge --changepriority kdeaddons-meta kde-meta --priority 0
> >
> >  Now, when i run emerge --update, it should:
> >
> >  * Sort packages in such order that packages with highest priority would be installed as soon as possible
> >  * Start installing
> >  * If something goes wrong, then skip this package and all, which depend on it. This should be optional, if those, which dont depend on it, but are needed by same package, will stay in their positions (go to end of same priority packages) or go to end.
> >
> >  When i need to use my computer and need much resources, i will press ctrl-c, use it, then start (continue) emerge --update.
> >
> >
> > 2006/3/14, tvali <qtvali@×××××.com>:
> >
> > >  2006/3/14, Simon Stelling <blubb@g.o>:
> > >
> > > >    tvali wrote:
> > > > > * Add package to "world" checking all dependencies, but not emerging
> > > > > * Remove package from "world" without unmerging
> > > >
> > > > Uhm, why would you want that?
> > > >
> > >
> > >
> > >  On my computer, building takes several hours on some packages. I have used it in such way that it builds in several windows on the same time, but i guess that that's not at all the best practice, especially with some combinations of packages. So i would like to add all packages, seeing all blocks and inter dependencies, then start building when ready (and go out of home, when it builds, and be sure that it will do it's best when i'm away).
> > >
> > >
> > > > > * Save all important messages into file instead of beeping (i may be away
> > > > > from home); optionally replay them at end of update
> > > >
> > > > elog can do this.
> > >
> > >
> > >  Will check out.
> > >
> > >
> > > > > * When updating or installing new package, if something goes wrong, still
> > > > > emerge other packages, which havent failed packages as dependencies -- in my
> > > > > case, for example, when i do --update, for example, then the fact that
> > > > > update of first package fails does not definitely mean that i dont want to
> > > > > update others
> > > >
> > > > --resume
> > >
> > >
> > >   I know --resume, but that's not what i meant.
> > >
> > >  Imagine the situation:
> > >  emerge -p package1 package2
> > >
> > >  part1 (needed by package1)
> > >  part2 (needed by part3)
> > >  part3 (needed by package2)
> > >  part4 (needed by package2)
> > >  part5 (needed by both package1 and package2)
> > >  package1
> > >  package2
> > >
> > >  now i start "emerge package1 package2" and go to work (or sleep or party or sauna).
> > >
> > >  part2 fails -- some bug or misconfiguration.
> > >
> > >  currently it means that when i came back home, i see that part1 is built, but nothing more and 4-5 hours are actually just wasted.
> > >
> > >  There should be a way to have all those built when i came home:
> > >
> > >  part1 (needed by package1)
> > >  part4 (needed by package2)
> > >   part5 (needed by both package1 and package2)
> > >   package1
> > >
> > >  There should be an error message listing all others. Also, i would add them to world and then emerge so that --update would try to emerge them again, asking me about all previously failed packages when i start it, if i want to retry this time (yes, no, all, none).
> > >
> > >
> > > > --
> > > > Kind Regards,
> > > >
> > > > Simon Stelling
> > > > Gentoo/AMD64 Developer
> > > >  --
> > > >  gentoo-portage-dev@g.o mailing list
> > > >
> > > >
> > >
> > >
> > >
> > >
> > > --
> > > tvali
> > > (e-mail: "  qtvali@×××××.com"; msn: " qtvali@×××××.com";
> > > icq: "317-492-912")
> > >
> > > Ühe eesti internetifirma lehel kohtasin tsitaati:
> > > If you don't do it excellently, dont do it at all. Because if it's not excellent, it won't be profitable or fun, and if you're not in business for fun or profit, what the hell are you doing here?
> > > Robert Townsend
> >
> >
> >
> >
> > --
> > tvali
> > (e-mail: " qtvali@×××××.com"; msn: "qtvali@×××××.com";
> > icq: "317-492-912")
> >
> > Ühe eesti internetifirma lehel kohtasin tsitaati:
> > If you don't do it excellently, dont do it at all. Because if it's not excellent, it won't be profitable or fun, and if you're not in business for fun or profit, what the hell are you doing here?
> > Robert Townsend
>
>
>
> --
> tvali
> (e-mail: "qtvali@×××××.com"; msn: "qtvali@×××××.com";
> icq: "317-492-912")
>
> Ühe eesti internetifirma lehel kohtasin tsitaati:
> If you don't do it excellently, dont do it at all. Because if it's not excellent, it won't be profitable or fun, and if you're not in business for fun or profit, what the hell are you doing here?
> Robert Townsend



--
tvali
(e-mail: "qtvali@×××××.com"; msn: "qtvali@×××××.com";
icq: "317-492-912")

Ühe eesti internetifirma lehel kohtasin tsitaati:
If you don't do it excellently, dont do it at all. Because if it's not
excellent, it won't be profitable or fun, and if you're not in
business for fun or profit, what the hell are you doing here?
Robert Townsend

-- 
gentoo-portage-dev@g.o mailing list