x86: enable GART-IOMMU only after setting up protection methods

commit fe2245c905631a3a353504fc04388ce3dfaf9d9e upstream.

The current code to set up the GART as an IOMMU enables GART
translations before it removes the aperture from the kernel memory
map, sets the GART PTEs to UC, sets up the guard and scratch
pages, or does a wbinvd().  This leaves the possibility of cache
aliasing open and can cause system crashes.

Re-order the code so as to enable the GART translations only
after all safeguards are in place and the tlb has been flushed.

AMD has tested this patch on both Istanbul systems and 1st
generation Opteron systems with APG enabled and seen no adverse
effects.  Istanbul systems with HT Assist enabled sometimes
see MCE errors due to cache artifacts with the unmodified
code.

Signed-off-by: Mark Langsdorf <mark.langsdorf@amd.com>
Cc: Joerg Roedel <joerg.roedel@amd.com>
Cc: akpm@linux-foundation.org
Cc: jbarnes@virtuousgeek.org
Signed-off-by: Ingo Molnar <mingo@elte.hu>

diff --git a/arch/x86/kernel/pci-gart_64.c b/arch/x86/kernel/pci-gart_64.c
index be33a5442d8205f958a0eb2ed5121b020b3ff32b..eed133ed4037b132af0bd68acc8cfa7256146f0a 100644 (file)
--- a/arch/x86/kernel/pci-gart_64.c
+++ b/arch/x86/kernel/pci-gart_64.c
@@ -658,8 +658,6 @@ static __init int init_k8_gatt(struct agp_kern_info *info)
        memset(gatt, 0, gatt_size);
        agp_gatt_table = gatt;
 
-       enable_gart_translations();
-
        error = sysdev_class_register(&gart_sysdev_class);
        if (!error)
                error = sysdev_register(&device_gart);
@@ -827,6 +825,14 @@ void __init gart_iommu_init(void)
         */
        wbinvd();
 
+       /*
+        * Now all caches are flushed and we can safely enable
+        * GART hardware.  Doing it early leaves the possibility
+        * of stale cache entries that can lead to GART PTE
+        * errors.
+        */
+       enable_gart_translations();
+
        /*
         * Try to workaround a bug (thanks to BenH):
         * Set unmapped entries to a scratch page instead of 0.