| 1 |
On Sat, 1 Nov 2014 15:46:18 -0700 |
| 2 |
Zac Medico <zmedico@g.o> wrote: |
| 3 |
|
| 4 |
> In addition to indexed search, this patch series implements |
| 5 |
> stream-based incremental display of search results (fixing bug |
| 6 |
> #412471). |
| 7 |
> |
| 8 |
> I'll be maintaining this patch series in the following branch: |
| 9 |
> |
| 10 |
> https://github.com/zmedico/portage/tree/bug_525718_stream |
| 11 |
> |
| 12 |
> The old non-incremental version of this patch series is still |
| 13 |
> available here: |
| 14 |
> |
| 15 |
> https://github.com/zmedico/portage/tree/bug_525718 |
| 16 |
> |
| 17 |
> |
| 18 |
|
| 19 |
Zac, while the code looks good, testing has showed a significant drop |
| 20 |
in performance for a -s search while it is greatly improves for a -S |
| 21 |
search. |
| 22 |
|
| 23 |
big_daddy portage # time emerge -s porthole |
| 24 |
!!! Repository 'vdr-devel' has sync-type attribute set to unsupported value: 'layman' |
| 25 |
!!! Repository 'wtk' has sync-type attribute set to unsupported value: 'layman' |
| 26 |
|
| 27 |
[ Results for search key : porthole ] |
| 28 |
Searching... |
| 29 |
* app-portage/porthole |
| 30 |
Latest version available: 0.6.1-r3 |
| 31 |
Latest version installed: 0.6.1-r3 |
| 32 |
Size of files: 937 KiB |
| 33 |
Homepage: http://porthole.sourceforge.net |
| 34 |
Description: A GTK+-based frontend to Portage |
| 35 |
License: GPL-2 |
| 36 |
|
| 37 |
[ Applications found : 1 ] |
| 38 |
|
| 39 |
|
| 40 |
real 0m5.509s |
| 41 |
user 0m5.181s |
| 42 |
sys 0m0.321s |
| 43 |
big_daddy portage # time /usr/bin/emerge -s porthole |
| 44 |
!!! Repository 'vdr-devel' has sync-type attribute set to unsupported value: 'layman' |
| 45 |
!!! Repository 'wtk' has sync-type attribute set to unsupported value: 'layman' |
| 46 |
Searching... |
| 47 |
[ Results for search key : porthole ] |
| 48 |
[ Applications found : 1 ] |
| 49 |
|
| 50 |
* app-portage/porthole |
| 51 |
Latest version available: 0.6.1-r3 |
| 52 |
Latest version installed: 0.6.1-r3 |
| 53 |
Size of files: 937 KiB |
| 54 |
Homepage: http://porthole.sourceforge.net |
| 55 |
Description: A GTK+-based frontend to Portage |
| 56 |
License: GPL-2 |
| 57 |
|
| 58 |
|
| 59 |
real 0m3.431s |
| 60 |
user 0m3.024s |
| 61 |
sys 0m0.401s |
| 62 |
big_daddy portage # emerge --version |
| 63 |
!!! Repository 'vdr-devel' has sync-type attribute set to unsupported value: 'layman' |
| 64 |
!!! Repository 'wtk' has sync-type attribute set to unsupported value: 'layman' |
| 65 |
Portage 2.2.14_p28 (python 3.3.5-final-0, default/linux/amd64/13.0/desktop, gcc-4.8.3, glibc-2.19-r1, 3.16.1-gentoo x86_64) |
| 66 |
big_daddy portage # /usr/bin/emerge --version |
| 67 |
!!! Repository 'vdr-devel' has sync-type attribute set to unsupported value: 'layman' |
| 68 |
!!! Repository 'wtk' has sync-type attribute set to unsupported value: 'layman' |
| 69 |
Portage 2.2.14 (python 3.3.5-final-0, default/linux/amd64/13.0/desktop, gcc-4.8.3, glibc-2.19-r1, 3.16.1-gentoo x86_64) |
| 70 |
big_daddy portage # time esearch porthole |
| 71 |
[ Results for search key : porthole ] |
| 72 |
[ Applications found : 1 ] |
| 73 |
|
| 74 |
* app-portage/porthole |
| 75 |
Latest version available: 0.6.1-r3 |
| 76 |
Latest version installed: 0.6.1-r3 |
| 77 |
Size of downloaded files: 936 kB |
| 78 |
Homepage: http://porthole.sourceforge.net |
| 79 |
Description: A GTK+-based frontend to Portage |
| 80 |
License: GPL-2 |
| 81 |
|
| 82 |
|
| 83 |
real 0m0.189s |
| 84 |
user 0m0.149s |
| 85 |
sys 0m0.038s |
| 86 |
big_daddy portage # |
| 87 |
|
| 88 |
======================== |
| 89 |
|
| 90 |
times for -S search |
| 91 |
|
| 92 |
======================== |
| 93 |
|
| 94 |
big_daddy portage # time /usr/bin/emerge -S porthole |
| 95 |
!!! Repository 'vdr-devel' has sync-type attribute set to unsupported value: 'layman' |
| 96 |
!!! Repository 'wtk' has sync-type attribute set to unsupported value: 'layman' |
| 97 |
Searching... / * ERROR: app-portage/c-layman-9999::gentoo-guis failed (depend phase): |
| 98 |
* git.eclass could not be found by inherit() |
| 99 |
* |
| 100 |
* Call stack: |
| 101 |
* ebuild.sh, line 550: Called source '/home/brian/Dev/git/gentoo-guis/app-portage/c-layman/c-layman-9999.ebuild' |
| 102 |
* c-layman-9999.ebuild, line 9: Called inherit 'git' |
| 103 |
* ebuild.sh, line 257: Called die |
| 104 |
* The specific snippet of code: |
| 105 |
* [[ -z ${location} ]] && die "${1}.eclass could not be found by inherit()" |
| 106 |
* |
| 107 |
* If you need support, post the output of `emerge --info '=app-portage/c-layman-9999::gentoo-guis'`, |
| 108 |
* the complete build log and the output of `emerge -pqv '=app-portage/c-layman-9999::gentoo-guis'`. |
| 109 |
* Working directory: '/usr/lib64/python3.3/site-packages' |
| 110 |
* S: '/var/tmp/portage/app-portage/c-layman-9999/work/c-layman-9999' |
| 111 |
emerge: search: aux_get() failed, skipping |
| 112 |
| * Manifest not found for '/home/brian/Dev/git/gentoo-guis/app-portage/ufed/ufed-9999.ebuild' |
| 113 |
| * ERROR: dev-vcs/git-bzr-9999::The-Pit failed (depend phase): |
| 114 |
* git.eclass could not be found by inherit() |
| 115 |
* |
| 116 |
* Call stack: |
| 117 |
* ebuild.sh, line 550: Called source '/usr/local/portage/dev-vcs/git-bzr/git-bzr-9999.ebuild' |
| 118 |
* git-bzr-9999.ebuild, line 4: Called inherit 'git' |
| 119 |
* ebuild.sh, line 257: Called die |
| 120 |
* The specific snippet of code: |
| 121 |
* [[ -z ${location} ]] && die "${1}.eclass could not be found by inherit()" |
| 122 |
* |
| 123 |
* If you need support, post the output of `emerge --info '=dev-vcs/git-bzr-9999::The-Pit'`, |
| 124 |
* the complete build log and the output of `emerge -pqv '=dev-vcs/git-bzr-9999::The-Pit'`. |
| 125 |
* Working directory: '/usr/lib64/python3.3/site-packages' |
| 126 |
* S: '/var/tmp/portage/dev-vcs/git-bzr-9999/work/git-bzr-9999' |
| 127 |
emerge: search: aux_get() failed, skipping |
| 128 |
|
| 129 |
[ Results for search key : porthole ] |
| 130 |
[ Applications found : 1 ] |
| 131 |
|
| 132 |
* app-portage/porthole |
| 133 |
Latest version available: 0.6.1-r3 |
| 134 |
Latest version installed: 0.6.1-r3 |
| 135 |
Size of files: 937 KiB |
| 136 |
Homepage: http://porthole.sourceforge.net |
| 137 |
Description: A GTK+-based frontend to Portage |
| 138 |
License: GPL-2 |
| 139 |
|
| 140 |
|
| 141 |
real 1m15.121s |
| 142 |
user 1m1.664s |
| 143 |
sys 0m12.891s |
| 144 |
big_daddy portage # time emerge -S porthole |
| 145 |
!!! Repository 'vdr-devel' has sync-type attribute set to unsupported value: 'layman' |
| 146 |
!!! Repository 'wtk' has sync-type attribute set to unsupported value: 'layman' |
| 147 |
|
| 148 |
[ Results for search key : porthole ] |
| 149 |
Searching... |
| 150 |
* ERROR: app-portage/c-layman-9999::gentoo-guis failed (depend phase): |
| 151 |
* git.eclass could not be found by inherit() |
| 152 |
* |
| 153 |
* Call stack: |
| 154 |
* ebuild.sh, line 575: Called source '/home/brian/Dev/git/gentoo-guis/app-portage/c-layman/c-layman-9999.ebuild' |
| 155 |
* c-layman-9999.ebuild, line 9: Called inherit 'git' |
| 156 |
* ebuild.sh, line 257: Called die |
| 157 |
* The specific snippet of code: |
| 158 |
* [[ -z ${location} ]] && die "${1}.eclass could not be found by inherit()" |
| 159 |
* |
| 160 |
* If you need support, post the output of `emerge --info '=app-portage/c-layman-9999::gentoo-guis'`, |
| 161 |
* the complete build log and the output of `emerge -pqv '=app-portage/c-layman-9999::gentoo-guis'`. |
| 162 |
* Working directory: '/home/brian/Dev/git/portage/pym' |
| 163 |
* S: '/var/tmp/portage/app-portage/c-layman-9999/work/c-layman-9999' |
| 164 |
emerge: search: aux_get() failed, skipping |
| 165 |
* app-portage/porthole |
| 166 |
Latest version available: 0.6.1-r3 |
| 167 |
Latest version installed: 0.6.1-r3 |
| 168 |
Size of files: 937 KiB |
| 169 |
Homepage: http://porthole.sourceforge.net |
| 170 |
Description: A GTK+-based frontend to Portage |
| 171 |
License: GPL-2 |
| 172 |
|
| 173 |
* ERROR: dev-vcs/git-bzr-9999::The-Pit failed (depend phase): |
| 174 |
* git.eclass could not be found by inherit() |
| 175 |
* |
| 176 |
* Call stack: |
| 177 |
* ebuild.sh, line 575: Called source '/usr/local/portage/dev-vcs/git-bzr/git-bzr-9999.ebuild' |
| 178 |
* git-bzr-9999.ebuild, line 4: Called inherit 'git' |
| 179 |
* ebuild.sh, line 257: Called die |
| 180 |
* The specific snippet of code: |
| 181 |
* [[ -z ${location} ]] && die "${1}.eclass could not be found by inherit()" |
| 182 |
* |
| 183 |
* If you need support, post the output of `emerge --info '=dev-vcs/git-bzr-9999::The-Pit'`, |
| 184 |
* the complete build log and the output of `emerge -pqv '=dev-vcs/git-bzr-9999::The-Pit'`. |
| 185 |
* Working directory: '/home/brian/Dev/git/portage/pym' |
| 186 |
* S: '/var/tmp/portage/dev-vcs/git-bzr-9999/work/git-bzr-9999' |
| 187 |
emerge: search: aux_get() failed, skipping |
| 188 |
[ Applications found : 1 ] |
| 189 |
|
| 190 |
|
| 191 |
real 0m12.515s |
| 192 |
user 0m11.795s |
| 193 |
sys 0m0.660s |
| 194 |
big_daddy portage # |
| 195 |
|
| 196 |
I know the above times are probably slowed by not having overlays indexed, but it is still a significant speedup. |
| 197 |
esearch is only marginally slower doing a -S search than a regular pkg-name search, typically just over 0.2 seconds |
| 198 |
|
| 199 |
|
| 200 |
The above was done with only the gentoo repo indexed, plus I need to clean out some old ebuilds in overlays. |
| 201 |
But I was shocked to see the normal -s searches going from 3.4s to 5.5s with your new index. |
| 202 |
|
| 203 |
Also I did not see a way to specify all repos to be index updated. It is I believe a requirement of this new system. Re-running is for each repo installed individually is something to be reserved for the new postsync() to do when it lands in master. |
| 204 |
|
| 205 |
-- |
| 206 |
Brian Dolbec <dolsen> |
Archives