Re: [gentoo-user] Re: Issues with AMD_IOMMU

[Corbin Bird]

On 05/19/2017 01:39 AM, Adam Carter wrote:
> On Wed, May 17, 2017 at 12:25 AM, Corbin Bird <corbinbird@×××××××.net
> <mailto:corbinbird@×××××××.net>> wrote:
>
>     On 05/16/2017 01:34 AM, Adam Carter wrote:
>     > Answer #1 :
>     >
>     >
>     >     Asrock, Extreme6, 990FX, UEFI, AMD FX-9590
>     >     Gigabyte, GA-MA790FX-UD5P, 790FX, BIOS, AMD PhenomII x4 980
>     >
>     >
>     > Hi Corbin,
>     >
>     > I noticed i didnt have x2apic enabled on my 990FX / FX-8350 system, so
>     > i've rebuilt/rebooted etc, but;
>     >
>     > # zgrep X2APIC /proc/config.gz
>     > CONFIG_X86_X2APIC=y
>     > # dmesg | grep -i x2apic
>     > #
>     >
>     > On my skylake box
>     > # dmesg | grep -i x2apic
>     > [    0.044148] DMAR-IR: Queued invalidation will be enabled to support
>     > x2apic and Intr-remapping.
>     > [    0.045802] DMAR-IR: Enabled IRQ remapping in x2apic mode
>     > [    0.045908] x2apic enabled
>     > [    0.046011] Switched APIC routing to cluster x2apic.
>     >
>     > What x2apic messages do you get on your 990FX / FX-9590 system?
>
>
>     Interesting kernel parameter for you to try :
>     ( from '/usr/src/linux/Documentation/kernel-parameters.txt' )
>
>     x2apic_phys     [X86-64,APIC] Use x2apic physical mode instead of
>                             default x2apic cluster mode on platforms
>                             supporting x2apic.
>
>     With that switch I have not been getting any output in 'dmesg' about
>     this at all.
>
>
> I tried adding x2apic_phys, but dmesg output did not change.
>
> From wikip;
> "In 2012, AMD announced their /Advanced Virtual Interrupt Controller/
> (/AVIC/) targeting interrupt overhead reduction in virtualization
> environments.^[33]
> <https://en.wikipedia.org/wiki/X86_virtualization#cite_note-33> This
> technology has materialize in hardware and (as announced) does not
> support x2APIC <https://en.wikipedia.org/wiki/X2APIC>.^[34]
> <https://en.wikipedia.org/wiki/X86_virtualization#cite_note-34> In 2016,
> AVIC is available on the AMD family 15h models 6Xh (Carrizo) processors
> and newer.^[35]"
> <https://en.wikipedia.org/wiki/X86_virtualization#cite_note-35>
>
> So;
> FX-8350 processors (bulldozer gen2) dont have AVIC
> Carrizo processors (bulldozer gen4), have AVIC, but AVIC doesnt support
> x2apic
> seems reasonable to conclude that there's no x2apic on bulldozer gen2,
> and that's why there's nothing in dmesg about it.
>
>
>     Question :
>     Do you have this enabled in your kernel ...
>     >   [ ]   Support for Intel IOMMU using DMA Remapping Devices
>
>
> On the Skylake box, yes, FX box, no.

Thanks for the info. Will kill the kernel parameter on my box.

---

Tried an experiment, recompiled the kernel, no GART IOMMU support.
This was not enabled either.
 < >     AMD IOMMU Version 2 driver

CPU FX-9590, 990FX Chipset

Now getting this instead :

> [    0.988027] PCI: CLS 64 bytes, default 64
> [    0.988362] iommu: Adding device 0000:00:00.0 to group 0
> [    0.988530] iommu: Adding device 0000:00:02.0 to group 1
> [    0.988689] iommu: Adding device 0000:00:0a.0 to group 2
> [    0.988872] iommu: Adding device 0000:00:0d.0 to group 3
> [    0.989029] iommu: Adding device 0000:00:11.0 to group 4
> [    0.989194] iommu: Adding device 0000:00:12.0 to group 5
> [    0.989318] iommu: Adding device 0000:00:12.2 to group 5
> [    0.989484] iommu: Adding device 0000:00:13.0 to group 6
> [    0.989615] iommu: Adding device 0000:00:13.2 to group 6
> [    0.989780] iommu: Adding device 0000:00:14.0 to group 7
> [    0.989943] iommu: Adding device 0000:00:14.1 to group 8
> [    0.990128] iommu: Adding device 0000:00:14.3 to group 9
> [    0.990292] iommu: Adding device 0000:00:14.4 to group 10
> [    0.990458] iommu: Adding device 0000:00:14.5 to group 11
> [    0.990637] iommu: Adding device 0000:00:15.0 to group 12
> [    0.990763] iommu: Adding device 0000:00:15.1 to group 12
> [    0.990935] iommu: Adding device 0000:00:16.0 to group 13
> [    0.991058] iommu: Adding device 0000:00:16.2 to group 13
> [    0.991252] iommu: Adding device 0000:01:00.0 to group 14
> [    0.991387] iommu: Adding device 0000:01:00.1 to group 14
> [    0.991560] iommu: Adding device 0000:02:00.0 to group 15
> [    0.991742] iommu: Adding device 0000:03:00.0 to group 16
> [    0.991863] iommu: Adding device 0000:06:00.0 to group 12
> [    0.991982] iommu: Adding device 0000:07:04.0 to group 12
> [    1.063849] AMD-Vi: Found IOMMU at 0000:00:00.2 cap 0x40
> [    1.063962] AMD-Vi: Interrupt remapping enabled
> [    1.064145] AMD-Vi: Lazy IO/TLB flushing enabled
> [    1.065331] perf: AMD NB counters detected
> [    1.065622] LVT offset 0 assigned for vector 0x400
> [    1.065836] perf: AMD IBS detected (0x000000ff)

And the Linux AGP Driver ( in-kernel ) is working now.

Now this is showing properly with lspci :
00:00.2 IOMMU: Advanced Micro Devices, Inc. [AMD/ATI] RD890S/RD990 I/O
Memory Management Unit (IOMMU)

And the GART IOMMU is no longer showing as an SMbus entry.

Corbin