1 |
-----BEGIN PGP SIGNED MESSAGE----- |
2 |
Hash: SHA1 |
3 |
|
4 |
Emma Strubell wrote: |
5 |
> Actually, do you think more work on faster search would be an adequate |
6 |
> project for soc? |
7 |
|
8 |
I don't think it would be enough work for a soc project. |
9 |
|
10 |
> In order for it to actually be a plausible |
11 |
> modification to portage I still need to implement regex search and |
12 |
> support for overlays. As I realized too soon before I had to submit |
13 |
> the project to my prof, integrating regex search into the |
14 |
> suffix-tree-like index that I created would require a pretty |
15 |
> substantial overhaul of my implementation, if not a separate |
16 |
> implementation to deal with regex queries. I'm sure it would be |
17 |
> possible though. The biggest problem remaining with my implementation, |
18 |
> as I believe I said before, is that cPickle unpickles (and pickles?) |
19 |
> the index wayy to slowly. Without a better serialization module my |
20 |
> implementation is pretty much useless, but ignoring the time it takes |
21 |
> to unpickle the index, my implementation is something like an order of |
22 |
> magnitude faster than the current search implementation. That's |
23 |
> promising, right? |
24 |
|
25 |
Well, your serialization problem is somewhat surprising. How big is |
26 |
that index file? Considering the speed with which esearch is able to |
27 |
load and search its 2 MB database, I suspect that the approach |
28 |
you're using may not be optimal for the data set. |
29 |
|
30 |
> I haven't tried out any other picklers yet so I'm |
31 |
> not sure how much of an improvement it might be possible to get. I |
32 |
> sure, however, that writing my own superior pickler would be beyond my |
33 |
> abilities. Unless serialization is simpler than I think it is, and I |
34 |
> could maybe throw together something that is optimized for this |
35 |
> specific data structure. |
36 |
- -- |
37 |
Thanks, |
38 |
Zac |
39 |
-----BEGIN PGP SIGNATURE----- |
40 |
Version: GnuPG v2.0.11 (GNU/Linux) |
41 |
|
42 |
iEYEARECAAYFAknVpTIACgkQ/ejvha5XGaMYqwCgtYkLaIWGBVPetvcBORs/Fnn8 |
43 |
/GoAnAgZc9zuodpL4FFj4mvMZ5JdpEwk |
44 |
=+aJx |
45 |
-----END PGP SIGNATURE----- |