Gentoo Archives: gentoo-amd64

From: Duncan <1i5t5.duncan@×××.net>
To: gentoo-amd64@l.g.o
Subject: [gentoo-amd64] Re: Memory usage; 32 bit vs 64 bit.
Date: Tue, 04 Jan 2011 02:05:06
In Reply to: Re: [gentoo-amd64] Memory usage; 32 bit vs 64 bit. by Alex Alexander
Alex Alexander posted on Mon, 03 Jan 2011 21:37:08 +0200 as excerpted:

> On 3 Jan 2011, at 20:34, Dale <rdalek1967@×××××.com> wrote: > >> I recently built me a new 64 bit system. My old 32 bit system has 2Gbs >> and my new system has 4Gbs. I was expecting it to use about the same >> amount of memory but noticed it uses a good bit more on the new system >> than the old one. With just the normal stuff open, I use about 1.5Gbs >> of ram. My old system would use a little over half that. I have the >> same settings on both. >> >> Is this difference because 64 bit programs use more memory, maybe they >> are larger than 32 bit programs? Just curious. I notice that >> Seamonkey uses more and KDE's plasma-desktop uses more. Those are >> generally the biggest users. >> >> I'm not complaining about the usage, just curious as to why the >> difference. >> > Are you sure you're checking your free ram correctly? run "free" and > check the buffers/cache line :)
Linux memory usage is notoriously confusing for the uninitiated and not entirely simple to explain or figure out the "real" per-app usage even for those who know /something/ about it. First, to directly answer the question. 64-bit memory usage /will/ be somewhat higher, yes, but shouldn't be double. The reason usage is higher is because address pointers are now 64-bit, not 32-bit, so /they/ take twice the space. However, according to the gcc manpage: -m32 -m64 Generate code for a 32-bit or 64-bit environment. The 32-bit environment sets int, long and pointer to 32 bits and generates code that runs on any i386 system. The 64-bit environment sets int to 32 bits and long and pointer to 64 bits and generates code for AMD's x86-64 architecture. So the common "utility integer" standard C/C++ int types remain 32-bit. This actually one of the bigger issues in porting sources from 32-bit to 64-bit, as for years, lazy 32-bit-only programmers were used to thinking of int, long and (memory) pointer as the same size, 32-bits, and being able to directly convert between them and use them nearly interchangeably, but that's no longer possible on amd64, because pointers and ints are no longer the same size. But the point (not pointer! =:^) we're interested in for purposes of this discussion is that the very commonly used "utility integer" known simply as "int" remains 32-bit. Because the 32-bit int is /so/ commonly used, to the point that it's the "default" integer type even on 64-bit, with only memory pointers and integers requiring 64-bit size getting full 64-bit, memory usage doesn't normally double, only increasing by some smaller factor, depending on the app and its particular mix of 32-bit int vs 64- bit memory pointer and 64-bit long integers. This additional memory usage is one of the negatives of 64-bit, and the reason that on archs other than x86, it's common to see 64-bit kernels for the ability to address > 4GB at the system level, with a 32-bit user-land since few individual apps (with noted exceptions) actually benefit from being able to address > 1-4 GB of RAM in a single app. (Note the 1-4 GB range. This is due to the common user-space/kernel-space split of the 4 GB address space on 32-bit systems, meaning individual apps may be limited to only a gig of usable user-address-space, depending on whether the split is 1:3/2:2/3:1 or separate 4GB spaces for user and kernel. Of course full 64-bit doesn't have to worry about this.) x86 is somewhat different in this regard, however, because traditional 32- bit x86 is known as a "register starved" architecture -- the number of available full-CPU-speed registers on 32-bit x86 is comparatively limited, forcing code to depend on slower L1 cache (tho that's still way faster than L2/L3, which is way faster than main memory, which is way faster than typical spinning-disk main storage) where other archs could be using their relative abundance of CPU registers. When it was designing amd64, AMD pretty much (I'm not sure if exactly) doubled the number of registers in their 64-bit hardware spec as compared to 32-bit (where they kept the same limited number of registers for compatibility reasons), with the result being that on amd64/x86_64 the speed-boost from access to these additional available registers often more than offsets the negative of the comparative double-size memory pointers. The precise balance, whether the cost of dealing with double-size memory pointers or the benefit of access to all those additional registers wins, depends on the app in question, but in general the benefit of the extra registers on amd64/x86_64 as opposed to x86_32/ia32 is sufficient that it's far less common to see the 64-bit kernel, 32-bit userland that is often seen on other archs. That takes care of the direct answer. Now to expand on what Alex referred to and what I mentioned in my intro as well, the topic of measuring Linux memory usage in general. The uninitiated will often look at "free memory" (the value in the Mem: line of the "free" command, run at the command line) on Linux, and wonder why it's so small -- why Linux seems to use so much memory. But, as Alex mentioned, that line is rather misleading, again, to the uninitiated. Linux, like most OSs, considers "empty" memory "wasted" memory. If the memory is available to use, therefore, Linux, as other OSs, will try to use it for something, normally for disk cache, mainly, with a bit used for other "buffering" as well. When/if the system needs that memory for other stuff (apps), the cache and buffers can be dumped. The confusion comes not in this, but rather, in the number actually exposed as "free" memory, which can be two very different values, either the actual "free" (unused=wasted) memory, or the "free for use if needed" (including memory used for cache and buffers) memory, depending on how the OS chooses to present it. On Linux, the "free" memory as reported by the "free" command on the Mem: line is the first (unused=wasted), while that on the -/+ buffers/cache line is the second (free for use if needed). Swap, of course, can be thrown in as another factor, since within context that can be seen as the reverse of disk cache -- app memory swapped out to disk as opposed to disk data cached in memory. Thus free's Swap: line. It's worth noting here the existence of the Linux kernel's swappiness parameter, exposed in the filesystem as /proc/sys/vm/swappiness . This file contains a number 0-100 (attempting to set it > 100 results in an error), 60 being the default, indicating the desired balance between swapping apps out to retain disk cache and keeping apps in memory thus having less room for disk cache. 0 means always prefer keeping apps in memory, dumping cache when needed to do so, 100 means always prefer dumping apps to swap, retaining cache if at all possible. As mentioned, the kernel swappiness default is 60, slightly preferring cache to apps. A common recommendation found on the net, however, is to lower swappiness to something like 20, preferring with some strength retention of apps in memory to retention of cache. Here, OTOH, I run swappiness=100, because swap is striped across four disks, while most of the filesystem is RAID-1 mirrored on the same four disks, so swap I/O should be faster than rereading formerly cached data back in off disk. And, at least with my current 6 gigs RAM, with PORTAGE_TMPDIR on tmpfs (which is reported in free's cache value) and with parallel merging parameters carefully controlled so that even with swappiness=100 I only end up a few MB (perhaps a couple hundred) into swap, swappiness=100 works very well for me. I don't notice the bit of swapping, and typically when I'm done, I might have 16 or 32 MB swapped out, that stays that way until I swapoff -a or reboot, indicating that I don't really use that bit of swapped apps much anyway or it'd be swapped back in when I did. If you wish to experiment with swappiness, you can cat it to see the value as a normal user, but of course only save/echo a new value to it as root. When you're done experimenting, if you want to make a permanent change, add a line ... vm.swappiness = 100 ... to your /etc/sysctl.conf file. (Other /proc/sys/* settings can be similarly set this way, or of course with a simple echo-redirect line in /etc/conf.d/local or the like. You can google for info on most or all of the other files under /proc/sys/, if interested.) OK, back from the swappiness detour, to memory usage. What sort of memory usage is reasonable? Of course that depends on what you do with your computer. =:^) But, as you know, I'm a KDE user as well, and of course a gentoo/amd64 user. Currently, I have an uptime of a week, which was when I last synced and updated both Gentoo and the kernel (thus the week uptime, since I rebooted into the new kernel then). So I've not done a full update since I rebooted, tho I did emerge a few new packages (phonon-vlc and dependencies, including vlc, I was running phonon- xine and still have it installed, but decided to try vlc and phonon-vlc) a couple days ago. Of course I'm in KDE (4.5.4) ATM. With that general system state and keeping in mind that I have 6 gigs RAM (the -m tells free to report in MB): $free -m total used free shared buffers cached Mem: 5925 3334 2590 0 319 1571 -/+ buffers/cache: 1443 4481 Swap: 20479 0 20479 So ~ 2.5 gigs is entirely unused (empty, effectively wasted, ATM), with the ~ 3.25 gigs of used memory split between ~ 1.4 gigs used for apps and ~ 1.8 gigs of cached and buffer memory, currently used to store data that can be dumped to make room for actual apps, if necessary. Tho in my experience, even the 1.4 gigs of app usage isn't entirely required. It has been awhile ago now, but at one point I was running 1 gig of total RAM, with no swap. At that time, app-memory usage seemed to run ~ half a gig. When I upgraded RAM to 8 gigs (I since lost a stick that I've not replaced, thus the current 6 gigs), app memory usage increased as well, to closer to a gig (IIRC it was about 1.2 gig after a week's uptime, back then, to compare apples to apples as they say), without changing what I was running or the settings. So given the memory to use, the apps I run apparently use it, up to perhaps a gig and a half. But if they're constrained to under a gig, they'll be content with less, perhaps half a gig. I'm not sure of the mechanisms involved there except that apps do have access to the memory info as well, and perhaps some of them are more liberal with their own caching (in-memory web-page cache for browsers, etc) and the like, given memory room to work with. But there's clearly a point at which they have their fill, as at a gig of RAM, apps were using half of it (half a gig), while when I upgraded to 8 gig, 8 times the RAM, app-memory usage only just over doubled. I suspect 4 gigs and 8 gigs would have about the same usage, but below 4 gigs, the apps start to be a bit more conservative with their own usage. That covers overall system memory usage. But what about individual apps? Individual app memory usage on Linux is unfortunately a rather complex subject. Top is a useful app for reporting on and controlling (nicing, killing, etc) other apps. Top's manpage has a nice description of the various memory related stats and how they relate to each other, so I'll refer you to that for some detail I'm omitting here. Meanwhile, on non- swapping systems, resident memory (top's RES column) is about as accurate a first-order approximation of app memory usage as you'll get, but it's only reporting physical memory, so won't include anything swapped out. Also, the memory one could expect to free by terminating that app will be somewhat less than resident memory, due to libraries and data that may be shared between multiple apps. Top has a SHR (shared) column to report potentially shared memory, but doesn't tell you how many other apps (maybe none) are actually sharing it. Some memory reporting apps won't count shared memory as belonging to the app at all, others (like top, AFAIK) report the full memory shared as belonging to each app, while still others try to count how many apps are sharing what bits, and divide the shared memory by the number of apps sharing it. Which way is "right" depends on what information you're actually looking for. If you want the app totals to match actual total memory usage, apportioned share reporting is the way to go. If you want to know what quitting the app will actually free, only count what's not shared by anything else. If you want to know how much memory an app is actually using, regardless of other apps that may be sharing it too, count all the memory it's using, shared or not. Then there's swapping. Due to the way Linux works, the data available on swapped out memory is limited. To get all the normal data would require swapping all that data back in, rather defeating the purpose of swap, so few if any memory usage reporting utils give you much detail about anything that's swapped out. For people with memory enough to do so, a swapoff (or simply running without swap at all) force-disables swap, thus making full statistics available, but as mentioned above, to a point, many apps will use more memory if it's available, conserve if it's not, so running without swap on systems that routinely report non-zero swap usage doesn't necessarily give a true picture of an app's memory usage with swap enabled, either. Conclusion: While the output of the free command (and by extension, other references to free memory in Linux) may initially seem a bit unintuitive, it's straightforward enough, once one understands what's there. Unfortunately, the same can't be said about individual application memory usage, which remains somewhat difficult to nail down and even more so to properly describe, even after one understands the basics. FWIW, however, I don't claim to be a programmer or to understand all that much beyond the basics. Should someone believe I'm in error with the above, or if they have anything to add or especially if they have a reasonably accurate simpler way to describe things, please post! I love to learn, and definitely do NOT believe I've reach my limit in learning in this area! -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman


Subject Author
Re: [gentoo-amd64] Re: Memory usage; 32 bit vs 64 bit. Volker Armin Hemmann <volkerarmin@××××××××××.com>
Re: [gentoo-amd64] Re: Memory usage; 32 bit vs 64 bit. Enrico Weigelt <weigelt@×××××.de>