Re: [gentoo-dev] Re: Packages up for grabs

[Tom Wijsman]

On Sun, 16 Jun 2013 19:33:53 +0000 (UTC)
Duncan <1i5t5.duncan@×××.net> wrote:

> TL;DR: SSDs help. =:^)

TL;DR: SSDs help, but they don't solve the underlying problem. =:-(

I have one; it's great to help make my boot short, but it isn't really
a great improvement for the Portage tree. Better I/O isn't a solution
to computational complexity; it doesn't deal with the CPU bottleneck.

Sadly, an improvement to the CPU as good as the switch from HDD to SSD,
I'm yet to see such a hardware improvement. Maybe if we stack the
transistors into the third dimension, something Intel was working on.

> Quite apart from the theory and question of making the existing code
> faster vs. a new from-scratch implementation, there's the practical
> question of what options one can actually use to deal with the
> problem /now/.

Don't rush it: Do you know the problem well? Does the solution
properly deal with it? Is it still usable some months / years from now?

> FWIW, one solution (particularly for folks who don't claim to have
> reasonable coding skills and thus have limited options in that
> regard) is to throw hardware at the problem.

Improvements in algorithmic complexity (exponential) are much bigger
than improvements you can achieve by buying new hardware (linear).

> I recently upgraded my main system to SDD. ... SNIP ... Between that
> and the 6-core bulldozer[3] I upgraded to last year, I'm quite happy
> with portage's current performance, ... SNIP ...

Ironically, you don't even fully use the CPU, but only one core of it;
I'm glad you have a 6-core processor, but to Portage it is a 1-core
during dependency tree calculation.

Portage becomes slower at a faster rate than your hardware get faster;
this will continue to be that way until you make Portage benefit of
it, or failing that you would need to come up with an alternative PM.

I didn't get my short boot from upgrading hardware alone; quite the
opposite, it was rather the results of the efforts spent on it.

> ---
> [1] I'm running ntp and the initial ntp-client connection and time
> sync takes ~12 seconds a lot of the time, just over the initial 10
> seconds down, 50 to go, trigger on openrc's 1-minute timeout.

Why do you make your boot wait for NTP to sync its time?

How could hardware make this time sync go any faster?

> [2] ... SNIP ... runs ~1 hour ... SNIP ...

Sounds great, but the same thing could run in much less time. I have
worse hardware, and it doesn't take much longer than yours do; so, I
don't really see the benefits new hardware bring to the table. And that
HDD to SSD change, that's really a once in a lifetime flood.

> [3] Also relevant, 16 gigs RAM, PORTAGETMPDIR on tmpfs.

Sounds all cool, but think about your CPU again; saturate it...

Building the Linux kernel with `make -j32 -l8` versus `make -j8` is a
huge difference; most people follow the latter instructions, without
really thinking through what actually happens with the underlying data.
The former queues up jobs for your processor; so the moment a job is
done a new job will be ready, so, you don't need to wait on the disk.

Something completely different; look at the history of data mining,
today's algorithms are much much faster than those of years ago.

Just to point out that different implementations and configurations have
much more power in cutting time than the typical hardware change does.

Though, this was pretty much OT; we're talking about the dependency tree
calculation, not about emerging which is rather a result of building
(eg. your compiler) than it has anything to do with the package manager.

PS: A take home thought: What if the hardware designers decided to not
R&D storage, then we wouldn't have a SSD; same story, different level.
Another level higher; we have physics, maybe CERN can improve hardware?
But when will that happen? Can we rely and wait on that to happen?

-- 
With kind regards,

Tom Wijsman (TomWij)
Gentoo Developer

E-mail address  : TomWij@g.o
GPG Public Key  : 6D34E57D
GPG Fingerprint : C165 AF18 AB4C 400B C3D2  ABF0 95B2 1FCD 6D34 E57D