Gentoo Archives: gentoo-user

From: Justin <justin@×××××××××.net>
To: gentoo-user@l.g.o
Subject: Re: [gentoo-user] Speed up with pbzip2 or not!?
Date: Sun, 27 Jan 2008 18:21:43
Message-Id: 479CCBAD.4060707@j-schmitz.net
In Reply to: Re: [gentoo-user] Speed up with pbzip2 or not!? by Enrico Weigelt
1 Enrico Weigelt schrieb:
2 > * Florian Philipp <lists@f_philipp.fastmail.net> wrote:
3 >
4 > Hi folks,
5 >
6 >
7 >> On Sun, 2008-01-27 at 11:40 +0100, Justin wrote:
8 >>
9 >>> Is there a way to use 7zip for decompression with emerge?
10 >>>
11 >> I fear, no easy one. Of course, since emerge is just a python script,
12 >> you could rewrite it but 7z-syntax isn't compatible with bzip2 or
13 >> gnu-tar so it will be some work. I also tried to use star as a drop-in
14 >> replacement for gnu-tar, once. Didn't work well although their syntax is
15 >> nearly identical, but just nearly...
16 >>
17 >
18 > Why not an universal wrapper script ?
19 >
20 > maybe something like:
21 >
22 > magic-uncompress [-t <format>] [-c] [-o <output>] <input>
23 >
24 > This script could try to find out the input format automatically
25 > (the optional -t parameter allows to explicitly specifiy the format)
26 >
27 > Once we have this script, ebuilds could be rewritten step by step
28 > and no one (else than magic-uncompress) has to care about the actual
29 > commands behind.
30 >
31 > Some could be done w/ tar and zip, etc.
32 >
33 >
34 > cu
35 >
36
37 But than there is still the incompability problem with pbzip2 with bzip2
38 compressed files. The best solution at this point would be to compress
39 all official tarballs with pbzip2, because now and especially in future
40 parallel processing is the solution and second there is no problem
41 decompressing parallel compressed tarballs in single mode.
42
43
44 For an temporary work around I figured out this hack:
45
46 in /usr/lib/portage/bin/ebuild.sh there is a function defined for each
47 type of compression algorithm. I changed following lines
48
49 --- ebuild.sh.orig 2008-01-27 19:16:22.000000000 +0100
50 +++ ebuild.sh 2008-01-27 19:15:36.000000000 +0100
51 @@ -479,7 +479,7 @@
52 ;;
53 bz2|bz)
54 if [ "${y}" == "tar" ]; then
55 - 7za x -so "${srcdir}${x}" | tar
56 xof - ${tar_opts}
57 + bzip2 -dc "${srcdir}${x}" | tar
58 xof - ${tar_opts}
59 assert "$myfail"
60 else
61 bzip2 -dc "${srcdir}${x}" >
62 ${x%.*} || die "$myfail"
63
64 ebuild.sh.orig:
65 real 0m16.962s
66 user 0m16.289s
67 sys 0m2.780s
68
69
70 ebuild.sh:
71 real 0m12.805s
72 user 0m13.209s
73 sys 0m2.666s
74
75 With this trick you can change every kind of decompression command.
76
77 In this example you can save 25% of the time.
78 --
79 gentoo-user@l.g.o mailing list