Linus Torvalds
2b047252d0
Fix TLB gather virtual address range invalidation corner cases
...
Ben Tebulin reported:
"Since v3.7.2 on two independent machines a very specific Git
repository fails in 9/10 cases on git-fsck due to an SHA1/memory
failures. This only occurs on a very specific repository and can be
reproduced stably on two independent laptops. Git mailing list ran
out of ideas and for me this looks like some very exotic kernel issue"
and bisected the failure to the backport of commit 53a59fc67f ("mm:
limit mmu_gather batching to fix soft lockups on !CONFIG_PREEMPT").
That commit itself is not actually buggy, but what it does is to make it
much more likely to hit the partial TLB invalidation case, since it
introduces a new case in tlb_next_batch() that previously only ever
happened when running out of memory.
The real bug is that the TLB gather virtual memory range setup is subtly
buggered. It was introduced in commit 597e1c3580 ("mm/mmu_gather:
enable tlb flush range in generic mmu_gather"), and the range handling
was already fixed at least once in commit e6c495a96c ("mm: fix the TLB
range flushed when __tlb_remove_page() runs out of slots"), but that fix
was not complete.
The problem with the TLB gather virtual address range is that it isn't
set up by the initial tlb_gather_mmu() initialization (which didn't get
the TLB range information), but it is set up ad-hoc later by the
functions that actually flush the TLB. And so any such case that forgot
to update the TLB range entries would potentially miss TLB invalidates.
Rather than try to figure out exactly which particular ad-hoc range
setup was missing (I personally suspect it's the hugetlb case in
zap_huge_pmd(), which didn't have the same logic as zap_pte_range()
did), this patch just gets rid of the problem at the source: make the
TLB range information available to tlb_gather_mmu(), and initialize it
when initializing all the other tlb gather fields.
This makes the patch larger, but conceptually much simpler. And the end
result is much more understandable; even if you want to play games with
partial ranges when invalidating the TLB contents in chunks, now the
range information is always there, and anybody who doesn't want to
bother with it won't introduce subtle bugs.
Ben verified that this fixes his problem.
Reported-bisected-and-tested-by: Ben Tebulin <tebulin@googlemail.com>
Build-testing-by: Stephen Rothwell <sfr@canb.auug.org.au>
Build-testing-by: Richard Weinberger <richard.weinberger@gmail.com>
Reviewed-by: Michal Hocko <mhocko@suse.cz>
Acked-by: Peter Zijlstra <peterz@infradead.org>
Cc: stable@vger.kernel.org
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2013-08-16 08:52:46 -07:00
..
bitops
Merge branch 'modules-next' of git://git.kernel.org/pub/scm/linux/kernel/git/rusty/linux
2012-10-14 13:39:34 -07:00
4level-fixup.h
atomic-long.h
atomic.h
asm-generic: move cmpxchg*_local defs to cmpxchg.h
2013-03-13 06:11:05 +01:00
atomic64.h
audit_change_attr.h
audit: support the "standard" <asm-generic/unistd.h>
2011-05-04 14:41:28 -04:00
audit_dir_write.h
audit: support the "standard" <asm-generic/unistd.h>
2011-05-04 14:41:28 -04:00
audit_read.h
audit: support the "standard" <asm-generic/unistd.h>
2011-05-04 14:41:28 -04:00
audit_signal.h
audit_write.h
audit: support the "standard" <asm-generic/unistd.h>
2011-05-04 14:41:28 -04:00
barrier.h
Create asm-generic/barrier.h
2012-03-28 18:30:03 +01:00
bitops.h
bitops: remove minix bitops from asm/bitops.h
2011-03-23 19:46:22 -07:00
bitsperlong.h
UAPI: (Scripted) Disintegrate include/asm-generic
2012-10-04 18:20:15 +01:00
bug.h
bug.h: Fix up CONFIG_BUG=n implicit function declarations.
2012-06-25 10:32:49 -07:00
bugs.h
cache.h
cacheflush.h
asm-generic/cacheflush.h: flush icache when copying to user pages
2011-05-25 08:39:37 -07:00
checksum.h
asm-generic headers: Allow yet more arch overrides in checksum.h
2013-02-11 20:00:33 +05:30
clkdev.h
asm-generic: Add default clkdev.h
2012-10-03 21:33:53 +02:00
cmpxchg-local.h
generic: Use raw local irq variant for generic cmpxchg
2013-02-19 08:43:37 +01:00
cmpxchg.h
asm-generic: move cmpxchg*_local defs to cmpxchg.h
2013-03-13 06:11:05 +01:00
cputime.h
cputime: Generic on-demand virtual cputime accounting
2013-01-27 19:23:27 +01:00
cputime_jiffies.h
cputime: Move default nsecs_to_cputime() to jiffies based cputime file
2013-01-27 19:23:25 +01:00
cputime_nsecs.h
cputime_nsecs: use math64.h for nsec resolution conversion helpers
2013-04-26 18:58:12 +02:00
current.h
delay.h
asm-generic: delay.h fix udelay and ndelay for 8 bit args
2011-07-22 18:45:33 +02:00
device.h
div64.h
dma-coherent.h
common: dma-mapping: add support for generic dma_mmap_* calls
2012-07-30 12:25:46 +02:00
dma-contiguous.h
mm: cma: fix condition check when setting global cma area
2012-07-06 12:02:04 +02:00
dma-mapping-broken.h
asm-generic/dma-mapping-broken.h: Provide dma_alloc_attrs()/dma_free_attrs()
2012-12-25 20:14:54 +01:00
dma-mapping-common.h
common: dma-mapping: introduce dma_get_sgtable() function
2012-07-30 12:25:46 +02:00
dma.h
emergency-restart.h
exec.h
Split arch_align_stack() out from asm-generic/system.h
2012-03-28 18:30:03 +01:00
fb.h
ftrace.h
asm-generic headers: add ftrace.h
2011-03-17 09:19:04 +08:00
futex.h
futex: Sanitize futex ops argument types
2011-03-11 12:23:31 +01:00
getorder.h
bitops: Add missing parentheses to new get_order macro
2012-02-24 10:39:27 -08:00
gpio.h
gpiolib: let gpio_chip reference its descriptors
2013-02-11 22:21:28 +00:00
hardirq.h
Fix IRQ flag handling naming
2010-10-07 14:08:55 +01:00
hugetlb.h
mm/hugetlb: add more arch-defined huge_pte functions
2013-04-29 15:54:33 -07:00
hw_irq.h
ide_iops.h
int-l64.h
UAPI: (Scripted) Disintegrate include/asm-generic
2012-10-04 18:20:15 +01:00
int-ll64.h
UAPI: (Scripted) Disintegrate include/asm-generic
2012-10-04 18:20:15 +01:00
io-64-nonatomic-hi-lo.h
asm-generic: architecture independent readq/writeq for 32bit environment
2012-02-21 16:47:28 -08:00
io-64-nonatomic-lo-hi.h
asm-generic: architecture independent readq/writeq for 32bit environment
2012-02-21 16:47:28 -08:00
io.h
kernel: Fix s390 absolute memory access for /dev/mem
2013-05-22 09:45:57 +02:00
ioctl.h
UAPI: (Scripted) Disintegrate include/asm-generic
2012-10-04 18:20:15 +01:00
iomap.h
[PARISC] fix compile break caused by iomap: make IOPORT/PCI mapping functions conditional
2012-02-27 09:43:30 -06:00
irq.h
irq_regs.h
core: Replace __get_cpu_var with __this_cpu_read if not used for an address.
2010-12-17 15:07:19 +01:00
irqflags.h
Fix IRQ flag handling naming
2010-10-07 14:08:55 +01:00
Kbuild.asm
UAPI: Set up uapi/asm/Kbuild.asm
2012-10-02 18:01:56 +01:00
kdebug.h
asm-generic: kdebug.h: Checkpatch cleanup
2010-10-09 21:51:44 +02:00
kmap_types.h
asm-generic: remove km_type definitions
2012-07-24 15:27:30 +08:00
kvm_para.h
KVM: add kvm_para_available to asm-generic/kvm_para.h
2013-06-05 13:21:29 +03:00
libata-portmap.h
linkage.h
local.h
atomic: use <linux/atomic.h>
2011-07-26 16:49:47 -07:00
local64.h
atomic: use <linux/atomic.h>
2011-07-26 16:49:47 -07:00
memory_model.h
mm: fix __page_to_pfn for a const struct page argument
2011-08-17 13:00:20 -07:00
mm_hooks.h
mmu.h
asm-generic/mmu.h: Add support for FDPIC
2012-12-09 23:14:14 +01:00
mmu_context.h
module.h
Make most arch asm/module.h files use asm-generic/module.h
2012-09-28 14:31:03 +09:30
mutex-dec.h
arch: Make __mutex_fastpath_lock_retval return whether fastpath succeeded or not
2013-06-26 12:10:55 +02:00
mutex-null.h
arch: Make __mutex_fastpath_lock_retval return whether fastpath succeeded or not
2013-06-26 12:10:55 +02:00
mutex-xchg.h
arch: Make __mutex_fastpath_lock_retval return whether fastpath succeeded or not
2013-06-26 12:10:55 +02:00
mutex.h
page.h
The following changes since commit 3ee72ca992
2012-01-10 17:39:40 -08:00
param.h
UAPI: (Scripted) Disintegrate include/asm-generic
2012-10-04 18:20:15 +01:00
parport.h
include: remove __dev* attributes.
2013-01-03 15:57:16 -08:00
pci-bridge.h
PCI: work around Stratus ftServer broken PCIe hierarchy
2012-04-30 15:21:02 -06:00
pci-dma-compat.h
dma-mapping: pci: move pci_set_dma_mask and pci_set_consistent_dma_mask to pci-dma-compat.h
2010-03-12 15:52:42 -08:00
pci.h
PCI: collapse pcibios_resource_to_bus
2012-02-23 20:19:04 -07:00
pci_iomap.h
[PARISC] fix compile break caused by iomap: make IOPORT/PCI mapping functions conditional
2012-02-27 09:43:30 -06:00
percpu.h
percpu: Optimize __get_cpu_var()
2010-09-10 10:56:51 +02:00
pgalloc.h
pgtable-nopmd.h
pgtable-nopud.h
pgtable.h
mm: save soft-dirty bits on file pages
2013-08-13 17:57:48 -07:00
ptrace.h
asm-generic/ptrace.h: start a common low level ptrace helper
2011-05-26 17:12:36 -07:00
resource.h
UAPI: (Scripted) Disintegrate include/asm-generic
2012-10-04 18:20:15 +01:00
rtc.h
rwsem.h
Hexagon: Add locking types and functions
2011-11-01 07:34:20 -07:00
scatterlist.h
asm-generic: remove ARCH_HAS_SG_CHAIN in scatterlist.h
2010-05-27 09:12:54 -07:00
sections.h
vmlinux.lds: add comments for global variables and clean up useless declarations
2013-07-03 16:07:34 -07:00
segment.h
serial.h
siginfo.h
UAPI: (Scripted) Disintegrate include/asm-generic
2012-10-04 18:20:15 +01:00
signal.h
unify default ptrace_signal_deliver
2012-11-29 00:01:23 -05:00
sizes.h
ARM: 7430/1: sizes.h: move from asm-generic to <linux/sizes.h>
2012-06-28 17:14:34 +01:00
spinlock.h
statfs.h
UAPI: (Scripted) Disintegrate include/asm-generic
2012-10-04 18:20:15 +01:00
string.h
switch_to.h
Split the switch_to() wrapper out of asm-generic/system.h
2012-03-28 18:30:03 +01:00
syscall.h
asm/syscall.h: add syscall_get_arch
2012-04-14 11:13:19 +10:00
syscalls.h
burying unused conditionals
2013-02-14 09:21:15 -05:00
termios-base.h
termios.h
UAPI: (Scripted) Disintegrate include/asm-generic
2012-10-04 18:20:15 +01:00
timex.h
tlb.h
Fix TLB gather virtual address range invalidation corner cases
2013-08-16 08:52:46 -07:00
tlbflush.h
BUG: headers with BUG/BUG_ON etc. need linux/bug.h
2012-03-04 17:54:34 -05:00
topology.h
topology: alternate fix for ia64 tiger_defconfig build breakage
2010-08-09 20:44:57 -07:00
trace_clock.h
tracing,x86: Add a TSC trace_clock
2012-11-13 15:48:27 -05:00
uaccess-unaligned.h
uaccess.h
asm-generic: uaccess s/might_sleep/might_fault/
2013-05-28 09:41:05 +02:00
unaligned.h
unistd.h
We get rid of the general module prefix confusion with a binary config option,
2013-05-05 10:58:06 -07:00
user.h
asm-generic/user.h: Fix spelling in comment
2011-03-01 15:49:39 +01:00
vga.h
vmlinux.lds.h
Merge branch 'cpuinit-delete' of git://git.kernel.org/pub/scm/linux/kernel/git/paulg/linux
2013-07-07 11:01:19 -07:00
word-at-a-time.h
word-at-a-time: make the interfaces truly generic
2012-05-26 11:33:40 -07:00
xor.h
asm-generic: xor: mark static functions as __maybe_unused
2012-10-03 21:21:06 +02:00