1 |
On 10/16/05, Duncan <1i5t5.duncan@×××.net> wrote: |
2 |
> Mark Knecht posted |
3 |
> <5bdc1c8b0510151118p447b72a9la959017a0de1dd08@××××××××××.com>, excerpted |
4 |
> below, on Sat, 15 Oct 2005 11:18:12 -0700: |
5 |
> |
6 |
> > Is anyone else seeign these troubling messages from tie to time? |
7 |
> > |
8 |
> > mtrr: base(0xe8020000) is not aligned on a size(0x400000) boundary |
9 |
> > mtrr: type mismatch for e8000000,8000000 old: write-back new: write-combining |
10 |
> > mtrr: type mismatch for e8000000,8000000 old: write-back new: write-combining |
11 |
> > mtrr: type mismatch for e8000000,8000000 old: write-back new: write-combining |
12 |
> > mtrr: base(0xe8020000) is not aligned on a size(0x400000) boundary |
13 |
> > mtrr: type mismatch for e8000000,8000000 old: write-back new: write-combining |
14 |
> > mtrr: base(0xe8020000) is not aligned on a size(0x400000) boundary |
15 |
> > mtrr: type mismatch for e8000000,8000000 old: write-back new: write-combining |
16 |
> > mtrr: base(0xe8020000) is not aligned on a size(0x400000) boundary |
17 |
> > mtrr: type mismatch for e8000000,8000000 old: write-back new: write-combining |
18 |
> > |
19 |
> > Is there somethign I should be doing to fix this? |
20 |
> |
21 |
> You may well know more about this than I do, but on the off chance this |
22 |
> may be new to you and for others... (and because I'm googling and learning |
23 |
> a bit in the process myself... =8^) |
24 |
|
25 |
Duncan, |
26 |
Great post. I'm getting some really good stuff out of it. |
27 |
|
28 |
Time to fess up. Based on reading this and looking again this |
29 |
morning I'm embarrassed to say that the machine that is failing is |
30 |
actually an Intel P4HT machine, not an AMD64 machine. I do have an |
31 |
AMD64 machine and I guess I blanked out at the moment I sent the |
32 |
email. None the less I think the topic is valuable on either list and |
33 |
I cannot imagine getting a more detailed and comprehensive response |
34 |
from anyone. Thanks! |
35 |
> |
36 |
> mtrr=memory type range (or region) register. This is definitely kernel |
37 |
> domain we are talking about here, often video drivers (so xorg related as |
38 |
> well). |
39 |
|
40 |
Yes, this I knew. Until reading you post I thought it was only used |
41 |
for video memory, and from my hardware background (PC chipset/AGP |
42 |
stuff some years ago) specifically with write-combining. I see that it |
43 |
really can have more wide ranging uses. |
44 |
<SNIP> |
45 |
> |
46 |
> http://www.meduna.org/txt_mtrr_en.html |
47 |
> |
48 |
> Paraphrasing from the above link... |
49 |
> |
50 |
> Basically, the MTRRs determine the behavior of cache vs regular memory on |
51 |
> memory-write, for various memory regions/ranges. |
52 |
> |
53 |
<SNIP> |
54 |
> |
55 |
> * Write-combining is sort of in-between the two above choices. The data |
56 |
> is allowed to sit in cache without updating main memory only a |
57 |
> comparatively short period, in ordered to allow the possibility of |
58 |
> combining several smaller writes into a larger, single, more efficient |
59 |
> write. (More efficient because each write has a set amount of overhead in |
60 |
> setup and take-down time and data. Thus, combining 8 128-byte |
61 |
> transactions into a single 1KB transaction means 1/8 the overhead, thus |
62 |
> more effective payload bandwidth, at a cost of more latency, due to |
63 |
> waiting for several transactions to accumulate, provided of course they |
64 |
> come in before the expiry time forces what's there to be written even if |
65 |
> it's not yet a full size transfer.) |
66 |
|
67 |
There was a totally related, but slightly additional use for write |
68 |
combining at the hardware level. It was introduced by Intel when they |
69 |
first wrote the AGP 1.0 spec. The idea at the time was that game |
70 |
software doing 3D work does not naturally output memory chunks that |
71 |
are the same width as the AGP bus. Typically they are smaller blocks, |
72 |
like 2 bytes, 4 bytes, etc. For this reason, to keep transfers on the |
73 |
AGP bus more efficient, write combining could be set up so that the |
74 |
processor would hold a number of small writes in a single cache line |
75 |
and only flush this cache line over the AGP bus to the video display |
76 |
when the software moved off the cache line or over-wrote some dirty |
77 |
part of the cache line. The idea was that there was no reason to send |
78 |
a bunch of bytes in separate writes, using more AGP bus bandwidth, |
79 |
when these writes could be sent as a group using less bandwidth. The |
80 |
description above supports this but it's not specifically graphics |
81 |
oriented. |
82 |
> |
83 |
> * There's also uncachable, which turns off caching for reads as well as |
84 |
> writes. This will be /very/ slow. |
85 |
> |
86 |
> Where graphics gets involved is that these MTRRs are often used to program |
87 |
> access to video memory over the AGP or whatever bus. The link above lists |
88 |
> some of the (then) xfree86 operations that MTRR settings affect and by how |
89 |
> much. |
90 |
|
91 |
Yep. |
92 |
|
93 |
> |
94 |
> Another link with some interesting info on the kernel config option |
95 |
> (CONFIG_MTRR) and the userland interface to MTRRs (/proc/mtrr) once |
96 |
> enabled. The two paragraphs below are excerpted: |
97 |
> |
98 |
> http://developer.osdl.org/dev/robustmutexes/src/fusyn.hg/Documentation/mtrr.txt |
99 |
|
100 |
This one is interesting in that it opens my eyes to the possibility of |
101 |
defining more MTRRs, a concept that I hadn't considered before. |
102 |
|
103 |
<SNIP> |
104 |
|
105 |
> Memory type region registers control the caching on newer Intel and non |
106 |
> Intel processors. This function allows drivers to request an MTRR is |
107 |
> added. The details and hardware specifics of each processor's |
108 |
> implementation are hidden from the caller, but nevertheless the caller |
109 |
> should expect to need to provide a power of two size on an equivalent |
110 |
> power of two boundary. |
111 |
|
112 |
The 'power of two' comments are interesting. |
113 |
<SNIP> |
114 |
> |
115 |
> That last contains a mention of boundaries ("power of two size on an |
116 |
> equivalent power of two boundary") that appears to pertain to your |
117 |
> problem. |
118 |
|
119 |
Seems to... |
120 |
|
121 |
> |
122 |
> So... now we know a bit about what MTRRs actually do (control the |
123 |
> interaction between cache and a specified portion of memory for write |
124 |
> transactions), what they are most often adjusted for (to increase graphics |
125 |
> performance, by changing the way writes to graphics memory are cached), |
126 |
> and can make a bit of sense out of the messages (the size doesn't match |
127 |
> the required base address for the MTRR, something's trying to change the |
128 |
> caching method, but using the wrong address and the adjustment is |
129 |
> therefore getting a type mismatch). |
130 |
|
131 |
OK. |
132 |
|
133 |
> |
134 |
> How does that translate into something you can do to fix it? |
135 |
> |
136 |
> Well, first, you can actually go take a look at /proc/mtrr (don't try to |
137 |
> write anything to it, unless you are sure you know what you are doing, but |
138 |
> reading it should be fine), and see if you can figure out which entry it's |
139 |
> supposed to be changing, if there's one close to that address at all, or |
140 |
> if not, what needs created. |
141 |
|
142 |
Yes, on the IA32 machine that actually has the problem I see strange |
143 |
results. It's a machine with 512MB, but possibly with shared memory? |
144 |
Not sure. Anyway, look at this: |
145 |
|
146 |
mark@dragonfly ~ $ cat /proc/mtrr |
147 |
reg00: base=0x00000000 ( 0MB), size=983552MB: write-back, count=1 |
148 |
reg01: base=0x1f000000 ( 496MB), size=983056MB: uncachable, count=1 |
149 |
mark@dragonfly ~ $ |
150 |
|
151 |
where as my AMD64 machine has a single MTRR operating and shows this: |
152 |
|
153 |
mark@lightning ~/.wine/drive_c/VST $ cat /proc/mtrr |
154 |
reg00: base=0x00000000 ( 0MB), size= 512MB: write-back, count=1 |
155 |
mark@lightning ~/.wine/drive_c/VST $ |
156 |
|
157 |
It seems that the IA32 machine is having some trouble with these |
158 |
registers. The size numbers look strange to me. |
159 |
|
160 |
<SNIP> |
161 |
|
162 |
>Whatever your video card driver is, that's |
163 |
> probably (but not for certain) what's causing the log messages. |
164 |
|
165 |
I agree. It's a built in Intel device. Part of the chipset I think: |
166 |
|
167 |
dragonfly ~ # lspci |
168 |
0000:00:00.0 Host bridge: Intel Corporation 82865G/PE/P DRAM |
169 |
Controller/Host-Hub Interface (rev 02) |
170 |
0000:00:02.0 VGA compatible controller: Intel Corporation 82865G |
171 |
Integrated Graphics Controller (rev 02) |
172 |
|
173 |
> Therefore, take a look at /var/log/Xorg.0.log, and see if you can match up |
174 |
> any possible MTRR messages listed there. |
175 |
|
176 |
No MTRR messages in Xorg.0.log.... |
177 |
|
178 |
> |
179 |
> Next, take a look at the driver documentation and your xorg.conf file |
180 |
> and boot loader config, and see what sort of adjustments you might need to |
181 |
> make. |
182 |
> |
183 |
> Of course, if it's /not/ video driver related, you'll likely have to |
184 |
> figure out what else is accessing the MTRRs and how to reconfigure it |
185 |
> correctly. Taking a wild guess, I'd say check anything that's likely to |
186 |
> be using DMA, thus, stuff like NICs or storage devices, and their drivers. |
187 |
|
188 |
I'm thinking that the strange sizes on the first MTRR are what caused |
189 |
the messages and possibly relate to something not really happy at boot |
190 |
time. The first one shows up in this set of operations: |
191 |
|
192 |
agpgart: Detected an Intel 865 Chipset. |
193 |
agpgart: Detected 16252K stolen memory. |
194 |
agpgart: AGP aperture is 128M @ 0xe8000000 |
195 |
[drm] Initialized drm 1.0.0 20040925 |
196 |
ACPI: PCI Interrupt 0000:00:02.0[A] -> GSI 16 (level, low) -> IRQ 19 |
197 |
mtrr: type mismatch for e8000000,8000000 old: write-back new: write-combining |
198 |
[drm] Initialized i915 1.1.0 20040405 on minor 0: Intel Corporation |
199 |
82865G Integrated Graphics Controller |
200 |
|
201 |
Not sure if it's ACPI related or graphics related. I'm guessing the latter... |
202 |
|
203 |
<SNIP> |
204 |
> |
205 |
> From an Opteron BIOS integrator's pdf @ amd... they recommend one of the |
206 |
> variable MTRRs (there are some fixed ones covering the memory space from |
207 |
> 640k to 1 MB as well, that must be what those common settings in BIOS must |
208 |
> be for) be set to cover the entire physical memory range... And so I |
209 |
> see... I have a gig of memory and see a 1024 MB MTRR set @ base-address |
210 |
> 0x 0000 0000 (0 MB), type write-back (so read/write cacheable). That makes |
211 |
> sense as it's telling the CPU(s, 2 in my case) that all of my main memory |
212 |
> is fully cacheable, no special restrictions needed! |
213 |
> |
214 |
> Apparently, some CPUs only have two variable MTRRs, and if one is used to |
215 |
> cover all of physical main-memory, that leaves only one available, which |
216 |
> would be used by the video driver, so that's how Linux is normally setup. |
217 |
> Again apparently, modern x86 (and presumably x86_64 as well) CPUs from |
218 |
> both AMD and Intel have eight such MTRRs, so more ranges can be programed |
219 |
> as needed. |
220 |
|
221 |
Intel P4 with Hyper-threading here. I would assume it has a reasonable |
222 |
set of these things. |
223 |
|
224 |
<SNIP> |
225 |
> |
226 |
> Note that at least here (Tyan s2885 dual Opteron board), the BIOS actually |
227 |
> has two related settings controlling the way > 3.5 GB of physical memory |
228 |
> is mapped. One apparently controls the actual memory addresses, whether |
229 |
> they skip the 3.5-4 GB range or not, the other controls the MTRRs, |
230 |
> continuous or not. If the two don't match, it could mean all of memory is |
231 |
> seen but not all of it is marked as cacheable, slowing access to the |
232 |
> uncached memory range. |
233 |
|
234 |
I have not investigated BIOS on this machine. Good point. |
235 |
|
236 |
> |
237 |
> ... Another leap of understanding... Remember those ranges need to be in |
238 |
> power-of-2 sizes and on matching power-of-2 boundaries? I happen to have |
239 |
> a gig of memory, an even power of two, so one MTR covers it exactly. |
240 |
> That wouldn't work for those with 3/4 gig or 1.5 or 3 gig or some such. |
241 |
> OTOH, I noticed a count=1 at the end of the two ranges I have mapped in my |
242 |
> /proc/mtrr, so it would appear that could be remedied by using say 3 |
243 |
> half-gig MTRs (count=3) stacked end-to-end, or a 1 gig and a half gig |
244 |
> (thus two), to map that 1.5 gig area. I'm not sure if the count= would |
245 |
> mean it's using additional MTRRs or not, tho I'd expect it would indeed |
246 |
> mean that. Thus, non-power-of-two memory layouts will probably mean more |
247 |
> MTRRs used to map the full memory MTR. |
248 |
> |
249 |
> (If there's anyone with an odd amount of memory that could verify, it'd be |
250 |
> nice...) |
251 |
|
252 |
I have an older Athlon XP 1600+ machine with 756MB. I don't pretend to |
253 |
understand this exactly but here's the results: |
254 |
|
255 |
gigastudio ~ # cat /proc/mtrr |
256 |
reg00: base=0x00000000 ( 0MB), size= 512MB: write-back, count=1 |
257 |
reg01: base=0x20000000 ( 512MB), size= 256MB: write-back, count=1 |
258 |
reg02: base=0xe8000000 (3712MB), size= 128MB: write-combining, count=1 |
259 |
reg07: base=0xf0000000 (3840MB), size= 128MB: write-combining, count=1 |
260 |
gigastudio ~ # |
261 |
|
262 |
It looks like reg00 and reg01 make up the 756MB of main memory. The |
263 |
video card is an NVidia with 128MB (I think....I'll have to go check) |
264 |
so I assume that this is reg02. I do not understand reg07. |
265 |
|
266 |
Somewhat disappointing, from this conversation, is my AMD64 machine: |
267 |
|
268 |
mark@lightning ~/.wine/drive_c/VST $ cat /proc/mtrr |
269 |
reg00: base=0x00000000 ( 0MB), size= 512MB: write-back, count=1 |
270 |
mark@lightning ~/.wine/drive_c/VST $ |
271 |
|
272 |
Why no mtrr for the video adapter? Note that this machine is running |
273 |
2.6.14-rc4-rt6. That's new today. There are no advanced drivers for |
274 |
this kernel so possibly this gets turned on when the VGA driver is |
275 |
loaded? |
276 |
> |
277 |
> My second MTRR is set to cover the 128 MB range dedicated to my video card |
278 |
> (it's got 256 MB but on 128 MB is being used, it would appear, but it's no |
279 |
> big deal since I don't do much 3D anyway because I'm running dual head |
280 |
> video @ 2048x1536 each). Here, it's the 128 MB beginning at exactly 3.5 |
281 |
> MB base address (0x e000 0000), write combining. |
282 |
> |
283 |
> OK... 0x e000 0000 is the 3.5 GB boundary, so your 0x e802 0000 is indeed |
284 |
> in the PCI device area... 0x 800 0000 is 128 MB, 0x 2 0000 is 8 KB, so the |
285 |
> base address it's attempting to use is 3.5 GB + 128 MB + 8 KB. The |
286 |
> requested size is 0x 40 0000 or 4 MB, so the closest boundary would be the |
287 |
> 0x e800 0000 you see it trying for and getting the type mismatch. |
288 |
|
289 |
And this is the address that my Via machine with 768MB shows. |
290 |
<SNIP> |
291 |
> |
292 |
> That's about all I have ATM... Hope this is as informative for others as |
293 |
> it just was for me! <g> (And, if anyone's an expert on this stuff and I |
294 |
> got it wrong, please inform me where! Better to correct any mistakes I |
295 |
> have now while it's new than after I build a bunch more suppositions on a |
296 |
> faulty foundation!) |
297 |
> |
298 |
|
299 |
That was a huge contribution to my knowledge and I appreciate it |
300 |
greatly. I think it's more than likely that there is something here |
301 |
that should be reported to the LKML but I want to understand it just a |
302 |
bit more first. |
303 |
|
304 |
Thanks, |
305 |
Mark |
306 |
|
307 |
-- |
308 |
gentoo-amd64@g.o mailing list |