Gentoo Archives: gentoo-performance

From: Miguel Sousa Filipe <miguel.filipe@×××××.com>
To: gentoo-performance@l.g.o
Subject: Re: [gentoo-performance] performance testing
Date: Mon, 30 Apr 2007 09:41:51
Message-Id: f058a9c30704300238m182855ye25cdab075ec7e78@mail.gmail.com
In Reply to: [gentoo-performance] performance testing by "Peter A. H. Peterson"
1 Hi there,
2
3 On 4/29/07, Peter A. H. Peterson <pedro@×××××××××××.net> wrote:
4 > Hi Everyone,
5 >
6 > My name is Peter Peterson and I represent a group of a grad students
7 > at UCLA. We're in a computer systems performance analysis course and we
8 > were hoping to do a general performance comparison of gentoo vs. a
9 > popular binary i386-compatible distribution (probably ubuntu) in some
10 > "real-world" server tests to try and meaningfully calculate the
11 > performance gains that local compilation provides. (For example,
12 > apache2 requests processed per second on the same hardware.)
13 >
14 > I've subscribed to this list because we want the gentoo community to
15 > be involved in helping us design the tests so that we can hopefully
16 > all feel good about what and how we are testing the systems.
17 >
18 > We have no particular outcome in mind; our group represents a wide
19 > range of computer users, from Mac, Linux, and Windows enthusiasts, and
20 > we have all used a wide variety of Linux distributions. We have simply
21 > noticed that much of the discussion of gentoo's performance advantage
22 > is anecdotal and we're genuinely hoping to provide some meaningful
23 > experimental data for discussion. Also, if anyone knows of any
24 > available benchmark data or papers on this subject, we'd love to hear
25 > about them. There was apparently a paper on slashdot a couple of years
26 > ago, but the host it was on appears to now be squatted. For that
27 > matter, if this is a well understood or closed issue (for example, if
28 > the statistics that people quote are actually from good experimental
29 > data) please let us know.
30 >
31 > Is anyone here interested in discussing this project? We are
32 > specifically interested in discussing methodology, testing suits,
33 > CFLAGS and other options. Our desire is not to "trick out" gentoo or
34 > ubuntu, but rather quantify the performance benefit that gentoo has
35 > over binary distributions with "normal" compile flags (whatever normal
36 > is).
37 >
38
39 A good CFLAGS would be something not very agressive, something like:
40 -march=<cpuType> -O3 or -O2 and at most -fomit-frame-pointer.
41 (Scientific workloads can speedup considerably with: -ffast-math)
42
43 Having experienced and done some benchmarks with gentoo and other
44 distros on servers and on scientific workstations.
45 What I found is that sometimes gentoo lacks critical performance
46 patches in glibc that are applied to mainstream distros (redhat,
47 suse..etc) that provide boosts in memcpy, memset, etc..(I remmember a
48 discussion about that some years ago).
49 What I also found out is that the compiler flags only affect
50 workloads that are very compute intensive. not something that depends
51 almost completely on FSB load or IO load.. like most server
52 workloads... -O3 doesn't do much to a working set full of
53 unpredictable branches (like server workloads usually are) and low IPC
54 rate.
55
56 I really do believe performance boost from gentoo to be practically
57 negligible. The difference will only be apreciable in very few corner
58 cases. Most distros also optimize critical aplications such has:
59 openssl, mplayer.. reducing the possible corner cases.
60
61 Anyways, doing a "academic" benchmark would be a good idea.
62
63 Something like:
64 micro-benchmarks:
65 - stream (mem bandwith benchmark)
66 - ??
67
68 macro-benchmarks:
69 - apache2 + gzip + php(make it cpu intensive, not IO intensive)
70 - xmlmark ?
71 - kernelbench
72 - pybench ?
73 - openssl bench
74
75
76 about methodology:
77 - same system, same bios version, same disks.
78 - All OSes must be installed in the same disk partitions.
79 - the will be trouble about the kernel config:
80 - for mainstream distros you should use the kernel that is provided.
81 - for gentoo, gentoo-sources configured by someone which is
82 experienced, and informed about configuration impacts on performance
83 (ideally a kernel hacker?).
84
85 - should use the stable versions in gentoo portage?
86 - or should use the same application versions used on mainstream distro?
87
88
89
90 --
91 Miguel Sousa Filipe
92 --
93 gentoo-performance@g.o mailing list

Replies

Subject Author
Re: [gentoo-performance] performance testing Michael W Spitzer <mwspitzer@×××××.com>
Re: [gentoo-performance] performance testing Paul de Vrieze <pauldv@g.o>