Re: [gentoo-portage-dev] Few things, which imho would make portage better

[tvali]

I did think about it now and it seems to me that probably it would be
much faster if esearch is not just another package, but part of
portage.

I mean -- functions of portage, which query db, should use esearch
index wherever they need information, which exists in that index.

As much as i can understand, /var/cache/edb/ contains esearch database
in many files and esearchdb.py is search index as python script.

I think that SQL functions for searching and other things are maybe
not much faster than py (as py seems to be fast enough, at least if
there are more important things to do), but simpler to use. Therefore
i think that rewriting the whole thing with SQL support may be still
better.

I will make it clear for me, what those emerge db functions do and
still try to do something, what connects them to SQL.

Anyway, i am still interested, is there some code or other
documentation about how portage works, especially how it keeps it's
data?

2006/3/14, tvali <qtvali@×××××.com>:
> Another question now is about sync.
>
> I did read somewhere, that this is not good user behavior to sync more
> than once per day. I understand that as if this is a huge download
> even if there is nothing changed.
>
> Isnt it nice idea to have this database just optimized?
>
> I mean (assuming portage using SQL now) -- that would be really simple
> to log every change in portage tree as series of SQL queries, which
> would reproduce this change.
>
> There could be table, which contains two fields -- Id and query (for
> example -- in reality, it could be a bit more complex for whatever
> reason; datetime, for example -- or filename and it's content to
> support current portage filetree). If i do emerge --sync, then one
> query could be done, SELECT query FROM updates WHERE Id >
> LastDownloadedUpdateId ORDER BY Id, then run all queries.
>
> I guess that in such case, portage --sync would be faster and less
> server-expensive. I dont know about this cache it builds, but i
> suppose that this could be made faster, too :)
>
> Ok, i will work now a bit, but i hope you have time to explain to me,
> if something i tell here is stupid or not-so-important for whatever
> reason.
>
> 2006/3/14, tvali <qtvali@×××××.com>:
> > Thank you much!
> >
> > This is what i meant about searching :) Would it be now good to mix
> > this esearch code with emerge, then adding fields to db and making
> > emerge treebuilding fast, too? Or is there some utility to emerge,
> > which does all that?
> >
> > 2006/3/14, Devon Miller <devon.c.miller@×××××.com>:
> > > Try "emerge esearch". Replace "emerge --sync" with "esync", "emerge -s" with
> > > "esearch", and "emerge -S" with "esearch -S".
> > > After running "emerge --update ...", run "eupdatedb".
> > >
> > > As for dependency tree: "emerge --pretend --tree world"
> > > Or, to also check additional dependencies: "emerge --deep --pretend --tree
> > > world"
> > > And, if you want to see how *everything* relates: "emerge --deep --emptytree
> > > --pretend --tree"
> > >
> > > dcm
> > >
> > >
> > >
> > > On 3/14/06, tvali <qtvali@×××××.com> wrote:
> > > >
> > > > Another place, where it would be good, is that it could interact well with
> > > GUI and it's Apply button.
> > > >
> > > > Another thing, what would be imho improved, is the speed of following
> > > functions:
> > > > * emerge -s, emerge -S
> > > > They should be cached somehow.
> > > >
> > > > Also, i think that when speaking about *speed* of portage, which is imho
> > > notably slow right now (speed of searching, building trees and so on),
> > > sql-support should be considered. In /usr/portage/, i see many folders and
> > > files, which tend to contain one or two words, lists of hashes and so on.
> > > Actually i see relational database there. With many caches and other
> > > optimizations it could be tuned fast, but having SQL doing all that, it
> > > would be achieved very simply (maybe it would be a jump to have all linux
> > > configurations in one SQL "filesystem"? ...ok, just fantasy:))
> > > >
> > > > There are several small and simple SQL databases. With such tool, big
> > > parts of portage become unnessecary, but it's speed will become much faster.
> > > This should be optional if one uses some tiny sql, mysql or big clustered
> > > sql with nas and load-balancer, but by default some tiny version of SQL
> > > should be built.
> > > >
> > > > Also, i currently dont see a possibility to see dependency tree of
> > > installed package.
> > > >
> > > >
> > > >
> > > > 2006/3/14, tvali < qtvali@×××××.com>:
> > > > > I did think about some priorities too, so that it could be perfect for
> > > me.
> > > > >
> > > > > It should be possible to add package with a priority. I will give you an
> > > use case and explanation how i would use portage.
> > > > >
> > > > > emerge --justadd kdebase-meta kicker --priority 10
> > > > > emerge --justadd kdenetwork-meta kdeutils-meta kdeadmin-meta --priority
> > > 9
> > > > > emerge --justadd kdeedu-meta kdetoys-meta kdegames-meta kdeartwork-meta
> > > --priority 1
> > > > > emerge --justadd koffice-meta kdegraphics-meta kdemultimedia-meta
> > > --priority 5
> > > > > emerge --justadd kdewebdev-meta kdevelop htmltidy kompare cervisia
> > > --priority 7
> > > > > emerge --justadd  kdepim-meta --priority 3
> > > > > emerge --justadd kdeaddons-meta kde-meta --priority 20
> > > > > emerge --changepriority kdeaddons-meta kde-meta --priority 0
> > > > >
> > > > > Now, when i run emerge --update, it should:
> > > > >
> > > > > * Sort packages in such order that packages with highest priority would
> > > be installed as soon as possible
> > > > > * Start installing
> > > > > * If something goes wrong, then skip this package and all, which depend
> > > on it. This should be optional, if those, which dont depend on it, but are
> > > needed by same package, will stay in their positions (go to end of same
> > > priority packages) or go to end.
> > > > >
> > > > > When i need to use my computer and need much resources, i will press
> > > ctrl-c, use it, then start (continue) emerge --update.
> > > > >
> > > > >
> > > > > 2006/3/14, tvali <qtvali@×××××.com>:
> > > > >
> > > > > > 2006/3/14, Simon Stelling <blubb@g.o>:
> > > > > >
> > > > > > > tvali wrote:
> > > > > > > > * Add package to "world" checking all dependencies, but not
> > > emerging
> > > > > > > > * Remove package from "world" without unmerging
> > > > > > >
> > > > > > > Uhm, why would you want that?
> > > > > > >
> > > > > >
> > > > > >
> > > > > > On my computer, building takes several hours on some packages. I have
> > > used it in such way that it builds in several windows on the same time, but
> > > i guess that that's not at all the best practice, especially with some
> > > combinations of packages. So i would like to add all packages, seeing all
> > > blocks and inter dependencies, then start building when ready (and go out of
> > > home, when it builds, and be sure that it will do it's best when i'm away).
> > > > > >
> > > > > >
> > > > > > > > * Save all important messages into file instead of beeping (i may
> > > be away
> > > > > > > > from home); optionally replay them at end of update
> > > > > > >
> > > > > > > elog can do this.
> > > > > >
> > > > > >
> > > > > > Will check out.
> > > > > >
> > > > > >
> > > > > > > > * When updating or installing new package, if something goes
> > > wrong, still
> > > > > > > > emerge other packages, which havent failed packages as
> > > dependencies -- in my
> > > > > > > > case, for example, when i do --update, for example, then the fact
> > > that
> > > > > > > > update of first package fails does not definitely mean that i dont
> > > want to
> > > > > > > > update others
> > > > > > >
> > > > > > > --resume
> > > > > >
> > > > > >
> > > > > > I know --resume, but that's not what i meant.
> > > > > >
> > > > > > Imagine the situation:
> > > > > > emerge -p package1 package2
> > > > > >
> > > > > > part1 (needed by package1)
> > > > > > part2 (needed by part3)
> > > > > > part3 (needed by package2)
> > > > > > part4 (needed by package2)
> > > > > > part5 (needed by both package1 and package2)
> > > > > > package1
> > > > > > package2
> > > > > >
> > > > > > now i start "emerge package1 package2" and go to work (or sleep or
> > > party or sauna).
> > > > > >
> > > > > > part2 fails -- some bug or misconfiguration.
> > > > > >
> > > > > > currently it means that when i came back home, i see that part1 is
> > > built, but nothing more and 4-5 hours are actually just wasted.
> > > > > >
> > > > > > There should be a way to have all those built when i came home:
> > > > > >
> > > > > > part1 (needed by package1)
> > > > > > part4 (needed by package2)
> > > > > > part5 (needed by both package1 and package2)
> > > > > > package1
> > > > > >
> > > > > > There should be an error message listing all others. Also, i would add
> > > them to world and then emerge so that --update would try to emerge them
> > > again, asking me about all previously failed packages when i start it, if i
> > > want to retry this time (yes, no, all, none).
> > > > > >
> > > > > >
> > > > > > > --
> > > > > > > Kind Regards,
> > > > > > >
> > > > > > > Simon Stelling
> > > > > > > Gentoo/AMD64 Developer
> > > > > > > --
> > > > > > > gentoo-portage-dev@g.o mailing list
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > tvali
> > > > > > (e-mail: " qtvali@×××××.com"; msn: " qtvali@×××××.com";
> > > > > > icq: "317-492-912")
> > > > > >
> > > > > > Ühe eesti internetifirma lehel kohtasin tsitaati:
> > > > > > If you don't do it excellently, dont do it at all. Because if it's not
> > > excellent, it won't be profitable or fun, and if you're not in business for
> > > fun or profit, what the hell are you doing here?
> > > > > > Robert Townsend
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > tvali
> > > > > (e-mail: " qtvali@×××××.com"; msn: "qtvali@×××××.com";
> > > > > icq: "317-492-912")
> > > > >
> > > > > Ühe eesti internetifirma lehel kohtasin tsitaati:
> > > > > If you don't do it excellently, dont do it at all. Because if it's not
> > > excellent, it won't be profitable or fun, and if you're not in business for
> > > fun or profit, what the hell are you doing here?
> > > > > Robert Townsend
> > > >
> > > >
> > > >
> > > > --
> > > > tvali
> > > > (e-mail: "qtvali@×××××.com"; msn: " qtvali@×××××.com";
> > > > icq: "317-492-912")
> > > >
> > > > Ühe eesti internetifirma lehel kohtasin tsitaati:
> > > > If you don't do it excellently, dont do it at all. Because if it's not
> > > excellent, it won't be profitable or fun, and if you're not in business for
> > > fun or profit, what the hell are you doing here?
> > > > Robert Townsend
> > >
> > >
> >
> >
> > --
> > tvali
> > (e-mail: "qtvali@×××××.com"; msn: "qtvali@×××××.com";
> > icq: "317-492-912")
> >
> > Ühe eesti internetifirma lehel kohtasin tsitaati:
> > If you don't do it excellently, dont do it at all. Because if it's not
> > excellent, it won't be profitable or fun, and if you're not in
> > business for fun or profit, what the hell are you doing here?
> > Robert Townsend
> >
>
>
> --
> tvali
> (e-mail: "qtvali@×××××.com"; msn: "qtvali@×××××.com";
> icq: "317-492-912")
>
> Ühe eesti internetifirma lehel kohtasin tsitaati:
> If you don't do it excellently, dont do it at all. Because if it's not
> excellent, it won't be profitable or fun, and if you're not in
> business for fun or profit, what the hell are you doing here?
> Robert Townsend
>


--
tvali
(e-mail: "qtvali@×××××.com"; msn: "qtvali@×××××.com";
icq: "317-492-912")

Ühe eesti internetifirma lehel kohtasin tsitaati:
If you don't do it excellently, dont do it at all. Because if it's not
excellent, it won't be profitable or fun, and if you're not in
business for fun or profit, what the hell are you doing here?
Robert Townsend

-- 
gentoo-portage-dev@g.o mailing list