Re: [gentoo-portage-dev] search functionality in emerge

[tvali]

There is one clear problem:

   1. Some other app opens some portage file.
   2. Tree is mounted and indexed.
   3. Other app changes this file.
   4. Index is out-of-date.

To disallow such thing it should be first suggested that all scripts change
portage tree only after mount. As defence against those, which dont listen
to that suggestion, portage should just not use this altered data - portage
should totally rely on it's internal index and when you change some file and
index is not updated, you change should be as well as lost. Does this make
portage tree twice as big as it is?

I guess not, because:

   - Useflags can be indexed and refferred with numbers.
   - Licence, homepage and such data is not needed to be duplicated.

Also, as overlay directories are suggested anyway, is it needed at all to
check *all* files for updates? I think that when one does something wrong,
it's OK when everything goes boom and if someone has update scripts, which
dont use overlays and other suggested ways to do thing, then adding one more
thing, which breaks, is not bad. Hashing those few files isnt bad idea and
keeping internal duplicate of overlay directory is not so bad, too - then
you need to "emerge --commithandmadeupdates" and that's all.

Some things, which could be used to boost:

   - Dependancy searches are saved - so that "emerge -p pck1 pck2 pck3"
   saves data about deps of those 3 packages.
   - Package name list is saved.
   - All packages are given integer ID.
   - List of all words in package descriptions are saved and connected to
   their internal ID's. This could be used to make smaller index file. So when
   i search for "al", then all words containing those chars like "all" are
   considered and -S search will run only on those packages.
   - Hash file of whole portage tree is saved to understand if it's changed
   after last remount.

2008/11/24 tvali <qtvali@×××××.com>

> So, mornings are smarter than evenings (it's Estonian saying) ...at night,
> I thought more about this filesystem thing and found that it simply answers
> all needs, actually. Now I did read some messages here and thought how it
> could be made real simple, at least as I understand this word. Yesterday I
> searched if custom filesystems could have custom functionality and did not
> find any, so I wrote this list of big bunch of classes, which might be
> overkill as I think now.
>
> First thing about that indexing - if you dont create daemon nor filesystem,
> you can create commands "emerge --indexon", "emerge --indexoff", "emerge
> --indexrenew". Then, index is renewed on "emerge --sync" and such, but when
> user changes files manually, she has to renew index manually - not much
> asked, isn't it? If someone is going to open the cover of her computer, she
> will take the responsibility to know some basic things about electricity and
> that they should change smth in bios after adding and removing some parts of
> computer. Maybe it should even be "emerge --commithandmadechanges", which
> will index or do some other things, which are needed after handmade changes.
> More such things might emerge in future, I guess.
>
> But about filesystem...
>
> Consider such thing that when you have filesystem, you might have some
> directory, which you could not list, but where you can read files. Imagine
> some function, which is able to encode and decode queryes into filesystem
> format.
>
> If you have such function: search(packagename, "dependencies") you can
> write it as file path:
> /cgi-bin/search/packagename/dependencies - and packagename can be encoded
> by replacing some characters with some codes and separating long strings
> with /. Also, you could have API, which has one file in directory, from
> where you can read some tmp filename, then write your query to that file and
> read the result from the same or similarly-named file with different
> extension. So, FS provides some ways to create custom queries - actually
> that idea came because there was idea of creating FS as cgi server on LUFS
> page, thus this "cgi-bin" starting here is to simplify. I think it's similar
> to how files in /dev/ directory behave - you open some file and start
> writing and reading, but this file actually is zero-sized and contains
> nothing.
>
> Under such case, API could be written to provide this filesystem and
> nothing more. If it is custom-mapped filesystem, then it could provide
> search and such directories, which can be used by portage and others. If
> not, it would work as it used to.
>
> So, having filesystem, which contains such stuff (i call this subdir "dev"
> here):
>
>    - /dev/search - write your query here and read the result.
>    - /dev/search/searchstring - another way for user to just read some
>    listings with her custom script.
>    - /portage/directory/category/packagename/depslist.dev - contains
>    dynamic list of package dependencies.
>    - /dev/version - some integer, which will grow every time any change to
>    portage tree is made.
>
> Then, other functions would be added eventually.
>
> Now, things simple:
>
>    - Create standard filesystem, which can be used to contain portage
>    tree.
>    - Add all nessecary notifications to change and update files.
>    - *Mount this filesystem to the same dir, where actual files are placed
>    - if it's not mounted, portage will almost not notice this (so in emergency,
>    things are just slower). You can navigate to a directory, then mount new one
>    - I am not on linux box right now, but if I remember correctly, you can use
>    files in real directory after mounting smth other there in such way.*
>    - Create indexes and other stuff.
>
> 2008/11/24 Fabian Groffen <grobian@g.o>
>
> On 24-11-2008 10:34:28 +0100, René 'Necoro' Neumann wrote:
>> > tvali schrieb:
>> > > There is daemon, which notices about filesystem changes -
>> > > http://pyinotify.sourceforge.net/ would be a good choice.
>> >
>> > Disadvantage: Has to run all the time (I see already some people crying:
>> > "oh noez. not yet another daemon...").
>>
>> ... and it is Linux only, which spoils the fun.
>>
>>
>> --
>> Fabian Groffen
>> Gentoo on a different level
>>
>>
>
>
> --
> tvali
>
> Kuskilt foorumist: http://www.cooltests.com - kui inglise keelt oskad.
> Muide, üle 120 oled väga tark, üle 140 oled geenius, mingi 170 oled ju mingi
> täica pea nagu prügikast...
>



-- 
tvali

Kuskilt foorumist: http://www.cooltests.com - kui inglise keelt oskad.
Muide, üle 120 oled väga tark, üle 140 oled geenius, mingi 170 oled ju mingi
täica pea nagu prügikast...