[gentoo-amd64] Re: I can use only 3GB aout of install 6GB

[Duncan]

Volker Armin Hemmann <volkerarmin@××××××××××.com> posted
200905061337.38907.volkerarmin@××××××××××.com, excerpted below, on  Wed,
06 May 2009 13:37:38 +0200:

> On Mittwoch 06 Mai 2009, Nadav Horesh wrote:
>> I recently added 4GB to the existing 2GB I had. The BIOS and lshw
>> recognize the memory configuration (2 dimm of 1G and 2 of 2G), but
>> /prc/meminfo and a test program I ran recognize 3.2GB.

> get a non broken bios?

... Or possibly, reconfigure the BIOS you (OP) have.

The story is this.  Legacy PCI devices were designed for 32-bit memory 
only.  That means their hardware I/O addresses must be below the 32-bit
4-gig memory ceiling.  Back before memory got cheap and plentiful enough 
that people commonly ran 3+ gigs, that was no problem, the hardware 
addresses simply lived at the top of the 4-gig memory area, typically 
3.5-4 gig.

Now that common memory installations exceed that 3.5 gig barrier, the 
BIOS must be prepared to create a "memory hole", mapping memory that 
would otherwise fall behind the PCI hardware I/O area up beyond the 4-gig 
barrier.  IOMMUs (input/output memory management units) or software 
bounce-buffers are then used to remap DMAs such that legacy PCI hardware 
operating with addresses in this memory hole can still handle DMAs 
working with memory addresses above the 4-gig barrier.

This was in fact one of the reasons early AMD64 hardware was considered 
superior to early Intel em64t hardware, AMD had a GART based hardware 
IOMMU, while Intel hardware of the era required a software emulated IOMMU 
bounce-buffer, AMD hardware could thus handle DMAs directly, while Intel 
hardware had to emulate it in software.  The kernel now has support for 
two additional forms of IOMMU, the newer AMD hardware and IBM's Calgary 
IOMMUs.  I honestly don't know what modern Intel hardware uses, whether 
it's still software bounce-buffer based (the GART based IOMMU option 
includes the software emulator as well), or if they have a hardware IOMMU 
covered by one of the above kernel config options.

If you've been in computers long enough to remember, the situation is 
somewhat analogous to the way early IBM PC hardware had conventional 
memory to 640 kB, a hardware memory area 640 kB to 1 MB, then expanded 
(early tech) or extended (later) memory beyond the 1 MB barrier.  The 640 
kB to 1 MB area was also called a "memory hole" with a BIOS option for 
handling it.  If I'm not mistaken, it can still be seen in some modern 
BIOS in the form of an OS specifier, OS/2 or not.  (It is however 
possible that I'm confused and that the OS/2 BIOS option is unrelated.)

Anyway, check your BIOS.  It's obviously difficult to refer to mine while 
I'm typing this, and I'm running AMD hardware (older first/second 
generation socket-940 Opteron, AMD 8xxx chipset) anyway, which may have 
different options or have it labeled differently, but look for an option 
mapping the memory hole as contiguous, discrete, or left undescribed, and 
try playing with that.  If I recall correctly, my BIOS actually has two 
places to set something like that.  I'm not sure how the two relate to 
each other nor do I recall precisely which each is labeled, but 
basically, when I upgraded memory from a gig to 8 gig, I played with 
these two BIOS options and the related kernel options until I got both 
BIOS and the kernel seeing the additional memory.  Interestingly enough, 
the BIOS now counts to ~10 gig of memory, obviously counting the memory 
hole as well, but regardless, the kernel gets it right so I've left well 
enough alone since then.

The related kernel config options I can actually check while I'm posting. 
=:^)  These options can be found in menuconfig under processor type and 
features.  As mentioned above, you'll want to figure out which IOMMU 
option matches your hardware and enable it.  (The GART based option 
should work in any case, as it includes the software bounce-buffer 
emulation, but it may not be the fastest option if you have a hardware 
IOMMU OTHER than the original AMD 8xxx GART based version.)

Then, look at memory model.  Here, with current kernels, I have only one 
option, Sparse, evidently limited by my choice of hardware (Processor 
family and/or Supported processor vendor options, higher on the page, I'd 
guess or perhaps probed from the BIOS).  However with older kernels, and 
presumably now if I were to choose hardware other than AMD Opteron/K8 (or 
choose a different BIOS option), there are Flat and Discontiguous options 
as well.  Unfortunately, I can't tell you which options are correct for 
your (Intel) hardware.  If you have multiple memory model choices, you 
may just have to try them and see.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman