Re: [gentoo-performance] performance testing

[Michael W Spitzer]

I'd agree with Miguel. In general, I can't tell the difference in most
applications. I did some tests on OpenOffice a few years ago and found the 8
hour compile time wasn't worth it in terms of performance. Where Gentoo
shines is in giving you the increased ability to shrink your installed size.
I have nothing on this machine that I don't specifically want there and I
work overly hard to keep my compile flags exactly right.

That said, I'd be interested in seeing your results. It would be nice to
know that all of this work is beneficial for more than just keeping me
happy.

Mike

On 4/30/07, Miguel Sousa Filipe <miguel.filipe@×××××.com> wrote:
>
> Hi there,
>
> On 4/29/07, Peter A. H. Peterson <pedro@×××××××××××.net> wrote:
> > Hi Everyone,
> >
> > My name is Peter Peterson and I represent a group of a grad students
> > at UCLA. We're in a computer systems performance analysis course and we
> > were hoping to do a general performance comparison of gentoo vs. a
> > popular binary i386-compatible distribution (probably ubuntu) in some
> > "real-world" server tests to try and meaningfully calculate the
> > performance gains that local compilation provides. (For example,
> > apache2 requests processed per second on the same hardware.)
> >
> > I've subscribed to this list because we want the gentoo community to
> > be involved in helping us design the tests so that we can hopefully
> > all feel good about what and how we are testing the systems.
> >
> > We have no particular outcome in mind; our group represents a wide
> > range of computer users, from Mac, Linux, and Windows enthusiasts, and
> > we have all used a wide variety of Linux distributions. We have simply
> > noticed that much of the discussion of gentoo's performance advantage
> > is anecdotal and we're genuinely hoping to provide some meaningful
> > experimental data for discussion. Also, if anyone knows of any
> > available benchmark data or papers on this subject, we'd love to hear
> > about them. There was apparently a paper on slashdot a couple of years
> > ago, but the host it was on appears to now be squatted. For that
> > matter, if this is a well understood or closed issue (for example, if
> > the statistics that people quote are actually from good experimental
> > data) please let us know.
> >
> > Is anyone here interested in discussing this project? We are
> > specifically interested in discussing methodology, testing suits,
> > CFLAGS and other options. Our desire is not to "trick out" gentoo or
> > ubuntu, but rather quantify the performance benefit that gentoo has
> > over binary distributions with "normal" compile flags (whatever normal
> > is).
> >
>
> A good CFLAGS would be something not very agressive, something like:
> -march=<cpuType> -O3 or -O2 and at most -fomit-frame-pointer.
> (Scientific workloads can speedup considerably with: -ffast-math)
>
> Having experienced and done some benchmarks with gentoo and other
> distros on servers and on scientific workstations.
> What I found is that sometimes gentoo lacks critical performance
> patches in glibc that are applied to mainstream distros (redhat,
> suse..etc) that provide boosts in memcpy, memset, etc..(I remmember a
> discussion about that some years ago).
> What I also found out is that the compiler flags only affect
> workloads that are very compute intensive. not something that depends
> almost completely on FSB load or IO load.. like most server
> workloads... -O3 doesn't do much to a working set full of
> unpredictable branches (like server workloads usually are) and low IPC
> rate.
>
> I really do believe performance boost from gentoo to be practically
> negligible. The difference will only be apreciable in very few corner
> cases. Most distros also optimize critical aplications such has:
> openssl, mplayer.. reducing the possible corner cases.
>
> Anyways, doing a "academic" benchmark would be a good idea.
>
> Something like:
> micro-benchmarks:
> - stream (mem bandwith benchmark)
> - ??
>
> macro-benchmarks:
> - apache2 + gzip + php(make it cpu intensive, not IO intensive)
> - xmlmark ?
> - kernelbench
> - pybench ?
> - openssl bench
>
>
> about methodology:
> - same system, same bios version, same disks.
> - All OSes must be installed in the same disk partitions.
> - the will be trouble about the kernel config:
>   - for mainstream distros you should use the kernel that is provided.
>   - for gentoo, gentoo-sources configured by someone which is
> experienced, and informed about configuration impacts on performance
> (ideally a kernel hacker?).
>
> - should use the stable versions in gentoo portage?
> - or should use the same application versions used on mainstream distro?
>
>
>
> --
> Miguel Sousa Filipe
> --
> gentoo-performance@g.o mailing list
>
>