Re: [gentoo-user] Eix takes ridiculously long to update cache of overlays

[Vaeth]

>> I guess I'm missing some settings specific to this? I have 3 overlays
>> installed via layman, and this eix takes ridiculously long to index
>> through them, I don't know why.
>>
>> The portage tree is indexed quickly.
>>
>
> There is usually not much you can do there. This typically happens with
> overlays that contain no metadata cache.
>
> Look at your overlay directory into metadata/layout.conf. If it doesn't
> contain a "cache-formats" line you are probably out of luck since eix
> has to parse each ebuild.

It is not the "cache-formats" line which is crucial but whether
there is an (up-to-date) metadata/cache or metadata/md5-cache
subdireectory.
If the overlay does not have such a directory (or it is not
updated regularly), You can let portage generate this directory
by using "egencache --repo=... update".
You can eix-sync automatically call this at every think
(see the example lines for eix-sync in the eix manpage).

> The default method for overlays is "parse|ebuild*" meaning that if there
> is a cache, it will be used and double-checked with a fast parser
> ("parse"). If there is no cache, the parser will be used alone. If that
> does not give satisfying results (e.g. because variables are set in
> eclasses instead of ebuilds), the real ebuild parser (i.e. bash) will be
> used to parse them ("ebuild*").

Almost: CACHE_METHOD_PARSE is appended to the default method.
Thus, the actual default method for overlays is something like
"parse|ebuild*#metadata-md5#metadata-flat#assign"
The choices after "#" are various cache methods; if the corresponding
cache is available, this method is used (in the specified order),
if they fail then "parse" is used (which is a fast parser, as you
mentioned, but not reliable) and if that does not give a satisfying
result then the real ebuild parser is used (which is very slow).
An alternative way to speed up eix is to remove the "|ebuild*"
in the overlay cache method: This means that eix will take
the result of "parse" even if it is not perfect. Of course, this
gives some "wrong" information (e.g. wrong SLOT name, missing
dependency information, etc.)

> Things might get better if they ever get libbash [1] finished.

I do not think that eix will be using it - it would be a lot
of work and only slightly improve the heuristics (main issue
are that inheriting eclasses currently requires some "wrapper"
interaction, so this cannot be solved with libbash alone.)
Perhaps some day portage might use libbash in "egencache", but
egencache is already surprisingly fast:

> If you need to update the eix cache on several systems, it might be
> faster to generate a cache on one machine and distribute it to all
> other. See `man egencache`.

Actually, egencache is already *much* faster than parsing each ebuild
separately. Moreover, I guess at least with cache-format=md5-dict
it parses really only those ebuilds for which the md5 checksum has
changed (i.e. the first execution might be slow, but after updates
there is not so much delay, usually).