1 |
On Mon, Apr 21, 2003 at 03:44:21AM -0400, Evan Powers wrote: |
2 |
> Hmm.... Have you timed your script yet? What sorts of run times are you |
3 |
> getting with that implementation? |
4 |
Comparing the execution time of my implementation to yours... |
5 |
|
6 |
Lets say mine has some issues. It runs find on the entire |
7 |
filesystem tree, reads the complete output of find into memory, |
8 |
and performs regex-based filtering after the fact instead of |
9 |
as command-line parameters to find like you do. |
10 |
|
11 |
To be honest, I'm ashamed that creating two sorted manifests and |
12 |
comparing didn't occur to me. Your solution is annoyingly compact |
13 |
and simple :) |
14 |
|
15 |
I tweaked your script to not use qpkg -nc -l but inline Perl |
16 |
parsing, and got some amazing results (roughly eight times faster on |
17 |
this system). |
18 |
|
19 |
The numbers... |
20 |
|
21 |
Your original script: |
22 |
|
23 |
$ time ./script-cruft.sh |
24 |
./script-cruft.sh 14.56s user 0.92s system 100% cpu 15.473 total |
25 |
|
26 |
My tweaked version: |
27 |
|
28 |
$ time ./cruft-script-fast.sh |
29 |
./script-cruft-fast.sh > fast 1.33s user 0.46s system 99% cpu 1.792 total |
30 |
|
31 |
My Python script (oh the shame): |
32 |
|
33 |
$ time ./gtfilelint -C gtfilelint.conf -o output.list |
34 |
./gtfilelint <...> 15.86s user 7.15s system 99% cpu 23.105 total |
35 |
|
36 |
These times are after Linux caching has kicked in. Executed |
37 |
the scripts multiple times, reported only the final times. Before |
38 |
caching, my script took about 160s, then 50, then 40, then 32, and |
39 |
finally 23. |
40 |
|
41 |
I guess I'll be using the tweaked version of your script from now on |
42 |
(attached) :) |
43 |
|
44 |
Interestingly, delving into the innards of epm and qpkg, revealed a |
45 |
bug in their CONTENTS parsing code...They can't handle filenames with |
46 |
spaces in them. They truncate the filename at the place a space occurs. |
47 |
|
48 |
Other than that, the output generated by my tweaked version and your |
49 |
original should be identical for the same set of paths to exclude. |
50 |
|
51 |
Leon |
52 |
|
53 |
-- |
54 |
in the beginning, was the code. |