1 |
Boyd Stephen Smith Jr. posted <200603251758.33840.bss03@××××××××××.net>, |
2 |
excerpted below, on Sat, 25 Mar 2006 17:58:15 -0600: |
3 |
|
4 |
> I had something like this happening on my 4GB system. With IOMMU off, I'd |
5 |
> only have access to like 2G of memory. With IOMMU on, I had various |
6 |
> results, ranging from kernel panic before initrd is loaded to missing |
7 |
> memory (small, like maybe the size of my PCI IO window?) to a completely |
8 |
> working system with devices properly mapped to memory addresses beyond my |
9 |
> physical RAM. These various behaviors were controlled, oddly enough, by |
10 |
> my BIOS settings. |
11 |
> |
12 |
> I'm running a Tyan Dual-Opteron Dual-PCIe board... If you've got a similar |
13 |
> board, I can share my BIOS and kernel settings and maybe you can resolve |
14 |
> your IOMMU issues. |
15 |
|
16 |
Not the same one, but similar, probably similar enough here to suffice. I |
17 |
see you mention the Tyan 2895 below, mine's the 2885, dual Opteron but |
18 |
PCI-X and 8x AGP, not yet PCI-E. Mine's the full AMD 8xxx chipset, both |
19 |
south and north/tunnel. Tyan is about the best choice there is in terms |
20 |
of Linux support for Opteron. Other makers do Opteron boards that do |
21 |
Linux, but they don't very well actively support Linux, the way Tyan does |
22 |
-- even to supplying lm_sensors config files and the like. I've also been |
23 |
very happy in that my board was like generation 1.3 or 1.5, but Tyan has |
24 |
fully supported the board with dual-core capable firmware since that came |
25 |
out with about generation 2.0. That is in fact one of the reasons I |
26 |
upgraded the memory -- I plan on upgrading to dual core in 6 months or so, |
27 |
to have 2 by dual-core = 4 cores on two sockets, and I don't expect to |
28 |
grow into the memory until then. |
29 |
|
30 |
The board was $400-some, more than I'd ever paid for a mobo before, but it |
31 |
has been /well/ worth it, going on 2 1/2 years now, and I'm looking at at |
32 |
least that long before I upgrade again, so 5 years easily, possibly 7 |
33 |
years, the longest I've ever run on the same mobo by far (I think 3-4 |
34 |
years was the longest previous). I could easily end up having to change |
35 |
out the battery in the thing, for the first time in my life. The thing |
36 |
is, with 8 gigs memory, 4x300gig 5-yr warrantee Seagate SATA drives in |
37 |
RAID, and 2xdual-cores, even 5 years in (2 1/2 from now), it's going to be |
38 |
no slouch of a system. It's hard to even comprehend a seven year old |
39 |
system (well, mobo anyway) being anything but antique, and it'll certainly |
40 |
be showing it's age by then, but it should still be reasonable. Consider |
41 |
what a seven year old system is now, maybe half a GHz Athlon original? The |
42 |
thing should be far better than that in comparison, considering it'll be 7 |
43 |
years old! |
44 |
|
45 |
As for the issue at hand, I've had a similar experience. If I use a mem= |
46 |
kernel command line between 3 and 5 gig, it sees 2 gig, period. I haven't |
47 |
tried over 5 gig, but I was shocked that telling it 5 gig only gave me two. |
48 |
|
49 |
Setting the BIOS memory hole between contiguous and discrete seems to make |
50 |
no difference at all. Setting the other one (I forget what it's called) |
51 |
gives me 8 gig POST detect, 7-odd gig kernel detect one way, 10 gig POST |
52 |
detect, the full 8 gig kernel detect, the other. Naturally, I have it set |
53 |
to the 10-POST/8-kernel. |
54 |
|
55 |
With kernel 2.6.15, configuring IOMMU won't let me boot (without a mem= |
56 |
kernel command line). Configuring it in but forcing it off (iommu=off on |
57 |
the kernel command line) gives me some IOMMU needed errors and it still |
58 |
won't boot (again, without a mem=). Configuring it out, the system boots |
59 |
just fine. |
60 |
|
61 |
With kernel 2.6.16, configuring IOMMU in or out, on or off, won't boot |
62 |
(without a mem=). |
63 |
|
64 |
Setting mem=2, 4, or 5 gig, results in 2 gig usable, and IOMMU |
65 |
automatically turned off as it doesn't need it because it sees only 2 gig |
66 |
of memory. The system boots fine, but of course without full memory. |
67 |
That's with both kernels. |
68 |
|
69 |
Writing this, it's obvious I should try mem=8g, just to see what it does, |
70 |
but I haven't tried that yet. |
71 |
|
72 |
So... yeah, the combination of BIOS and kernel IOMMU settings that work |
73 |
for you, particularly if you have libata based SATA or regular SCSI |
74 |
running fine on kernel 2.6.16, would be very useful. |
75 |
|
76 |
The other thing I should look at is whether the Gentoo kernel includes any |
77 |
patches that look apropos to >=4G on amd64, since I've only tried vanilla |
78 |
kernels. I know I didn't need anything in the Gentoo patch set before, |
79 |
but it's possible they have something in there to do with big memory, that |
80 |
I now need. |
81 |
|
82 |
-- |
83 |
Duncan - List replies preferred. No HTML msgs. |
84 |
"Every nonfree program has a lord, a master -- |
85 |
and if you use the program, he is your master." Richard Stallman in |
86 |
http://www.linuxdevcenter.com/pub/a/linux/2004/12/22/rms_interview.html |
87 |
|
88 |
|
89 |
-- |
90 |
gentoo-amd64@g.o mailing list |