| 1 |
Volker Armin Hemmann wrote: |
| 2 |
> On Mittwoch, 13. Februar 2008, Richard Freeman wrote: |
| 3 |
>> Do you have benchmarks to support this? |
| 4 |
> |
| 5 |
> which numbers? that swap is horrible slow compared to ram? |
| 6 |
> |
| 7 |
|
| 8 |
No - that compiling in tmpfs is horrible compared to compiling to disk. |
| 9 |
Neither of us is proposing getting rid of disk access entirely - we're |
| 10 |
just debating two different ways of doing it. |
| 11 |
|
| 12 |
|
| 13 |
> think about this: instead of having the compiler in ram and files on disk, you |
| 14 |
> have to swap the compiler out, read the files from swap into ram, swap the |
| 15 |
> compiler in, do some stuff, repeat. Slowdown. |
| 16 |
> |
| 17 |
|
| 18 |
Uh, every file that you compile has to be read into RAM and then |
| 19 |
binaries have to be written back to disk with your proposal. With swap |
| 20 |
stuff only gets written to disk if RAM gets low - which in the worst |
| 21 |
case is equivalent to writing everything to disk - which is your typical |
| 22 |
case. The kernel would not be swapping the compiler in and out in these |
| 23 |
kinds of circumstances - it would probably be swapping in and out the |
| 24 |
tmpfs. However, if some pages in the compiler are completely idle they |
| 25 |
might get swapped out, making more RAM available for tmpfs use. |
| 26 |
|
| 27 |
> |
| 28 |
> nope. Stuff that is written and deleted some seconds later never hit the disk. |
| 29 |
> That is why cache&buffer exist. |
| 30 |
|
| 31 |
If "some seconds" is greater than about 10 it will be flushed to disk |
| 32 |
100% of the time with most filesystems. The exact figures probably vary |
| 33 |
by filesystem. Most filesystems sync their buffers within some time |
| 34 |
after they are written. tmpfs does not have this feature - which is |
| 35 |
good if you care about your files and bad if you don't. |
| 36 |
|
| 37 |
>> |
| 38 |
>> As opposed to wasting all your RAM on cache&buffers to hold all those |
| 39 |
>> files being accessed? During compilation that RAM is going to be |
| 40 |
>> heavily used - no getting around that. Once you're done any files left |
| 41 |
>> sitting on a tmpfs will just get paged out until accessed. They |
| 42 |
>> shouldn't really use any RAM at all. Even if those files were on disk |
| 43 |
>> they would consume RAM in the form of caching until they're considered |
| 44 |
>> unneeded. |
| 45 |
> |
| 46 |
> cached&buffered information can be discarded anytime if the ram is needed |
| 47 |
> elsewhere. tmpfs has to be shoved into swap. |
| 48 |
> |
| 49 |
|
| 50 |
Uh, data written to a file has to be written to disk before the |
| 51 |
cache/buffer is flushed. The difference is that the data is written to |
| 52 |
disk within about 10 seconds of being written to the buffer, and then |
| 53 |
retained in cache maybe for a few hours longer (optimistically). With |
| 54 |
tmpfs the write doesn't happen until the space is needed. Either way |
| 55 |
every byte gets written - with tmpfs it is deferred as late as possible |
| 56 |
but with disk-based filesystems the write is done as early as possible. |
| 57 |
|
| 58 |
> |
| 59 |
> with the files on disk, the kernel can hold as much data in ram as he likes |
| 60 |
> to. with a big tmpfs (and 512mb out of 1gb) he is always shoving someting |
| 61 |
> into swap or get something from there. |
| 62 |
> |
| 63 |
|
| 64 |
You can put 6GB of data in a 7GB tmpfs filesystem on a system with 512MB |
| 65 |
of RAM and still have 400MB of free memory. tmpfs does not reserve |
| 66 |
memory - if it did it would certainly use tons of memory. |
| 67 |
|
| 68 |
> Extreme example? |
| 69 |
> |
| 70 |
> kdepim with the enablefinal flag. On amd64 every single gcc instance needs |
| 71 |
> ~900mb at two points of compilation. With 1gb ram and no tmpfs it sucks. |
| 72 |
> |
| 73 |
> With 1gb ram and 512mb of that reserved for tmpfs, you'll get a swapstorm. |
| 74 |
|
| 75 |
Oh, it will certainly swap a great deal. However, I don't see why it |
| 76 |
would be any slower - either way you're doing a ton of disk access. Do |
| 77 |
you have actual benchmarks? |
| 78 |
|
| 79 |
Again, tmpfs doesn't "reserve" memory - it uses memory - just like |
| 80 |
cache/buffers. |
| 81 |
|
| 82 |
>> I certainly agree that |
| 83 |
>> swap is slow compared to RAM, but it isn't slow compared to a disk-based |
| 84 |
>> filesystem. |
| 85 |
> |
| 86 |
> really? Every really swapped? IMHO it feels like every single byte is fetched |
| 87 |
> by a mule. |
| 88 |
> |
| 89 |
|
| 90 |
Right now I've got 589MB of free RAM (-/+ buffers/cache) with 2GB of RAM |
| 91 |
total. I've got 705MB of swap used. Works just fine IMHO. Sure, if I |
| 92 |
make it really busy it can get slow, although with nice/ionice there |
| 93 |
isn't much visible impact. And it doesn't really seem much slower using |
| 94 |
swap vs on-disk. I'd expect some swap slowdown in my case as I'm using |
| 95 |
encrypted swap, but I don't think most users will be doing that. |
| 96 |
|
| 97 |
> Of course adding more RAM will never hurt, but that incurs |
| 98 |
>> significant cost and you can at least maximize your current hardware |
| 99 |
>> before investing in more of it. |
| 100 |
> |
| 101 |
> significant costs like 30euros for 2gb? |
| 102 |
|
| 103 |
Sure. Compared to about $1 for 2GB of hard drive space 30 euros is |
| 104 |
significant. And even if he had 2GB of RAM I submit that it would |
| 105 |
probably work better if he compiled in tmpfs than on disk. There is no |
| 106 |
question that a $550 computer will outperform a $500 computer, and a |
| 107 |
$600 computer will outperform a $550 computer, and so on. You can |
| 108 |
always spend 30 more euros and improve your system. What I'm interested |
| 109 |
in is maximizing the performance of the system I have now. I can always |
| 110 |
spend $50 and make it even faster. However, there are lots of things I |
| 111 |
can spend $50 on - I'd rather spend it on something else all things |
| 112 |
being equal. |
| 113 |
-- |
| 114 |
gentoo-amd64@l.g.o mailing list |
Archives