Jann Horn
275c626c13
mm/khugepaged: invoke MMU notifiers in shmem/file collapse paths
...
commit f268f6cf875f3220afc77bdd0bf1bb136eb54db9 upstream.
Any codepath that zaps page table entries must invoke MMU notifiers to
ensure that secondary MMUs (like KVM) don't keep accessing pages which
aren't mapped anymore. Secondary MMUs don't hold their own references to
pages that are mirrored over, so failing to notify them can lead to page
use-after-free.
I'm marking this as addressing an issue introduced in commit f3f0e1d215
("khugepaged: add support of collapse for tmpfs/shmem pages"), but most of
the security impact of this only came in commit 27e1f8273113 ("khugepaged:
enable collapse pmd for pte-mapped THP"), which actually omitted flushes
for the removal of present PTEs, not just for the removal of empty page
tables.
Link: https://lkml.kernel.org/r/20221129154730.2274278-3-jannh@google.com
Link: https://lkml.kernel.org/r/20221128180252.1684965-3-jannh@google.com
Link: https://lkml.kernel.org/r/20221125213714.4115729-3-jannh@google.com
Fixes: f3f0e1d215 ("khugepaged: add support of collapse for tmpfs/shmem pages")
Signed-off-by: Jann Horn <jannh@google.com>
Acked-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Yang Shi <shy828301@gmail.com>
Cc: John Hubbard <jhubbard@nvidia.com>
Cc: Peter Xu <peterx@redhat.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
[manual backport: this code was refactored from two copies into a common
helper between 5.15 and 6.0;
pmd collapse for PTE-mapped THP was only added in 5.4;
MMU notifier API changed between 4.19 and 5.4]
Signed-off-by: Jann Horn <jannh@google.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2023-01-07 12:07:10 +01:00
..
kasan
kasan: avoid -Wmaybe-uninitialized warning
2019-05-08 07:19:07 +02:00
backing-dev.c
mm: bdi: initialize bdi_min_ratio when bdi is unregistered
2021-12-14 10:04:47 +01:00
balloon_compaction.c
mm: balloon: use general non-lru movable page feature
2016-07-26 16:19:19 -07:00
bootmem.c
mm: kmemleak: avoid using __va() on addresses that don't have a lowmem mapping
2016-10-11 15:06:33 -07:00
cleancache.c
cma.c
mm/cma.c: fail if fixed declaration can't be honored
2019-08-06 18:29:37 +02:00
cma.h
cma_debug.c
mm/cma_debug.c: fix the break condition in cma_maxchunk_get()
2019-06-22 08:17:12 +02:00
compaction.c
mm, compaction: fix NR_ISOLATED_* stats for pfn based migration
2017-01-12 11:39:32 +01:00
debug.c
mm: get rid of vmacache_flush_all() entirely
2018-09-19 22:47:17 +02:00
debug_page_ref.c
mm/page_ref: add tracepoint to track down page reference manipulation
2016-03-17 15:09:34 -07:00
dmapool.c
mm: convert printk(KERN_<LEVEL> to pr_<level>
2016-03-17 15:09:34 -07:00
early_ioremap.c
mm/early_ioremap: Fix boot hang with earlyprintk=efi,keep
2018-02-25 11:05:49 +01:00
fadvise.c
mm/fadvise.c: fix signed overflow UBSAN complaint
2018-09-15 09:42:57 +02:00
failslab.c
mm: fault-inject take over bootstrap kmem_cache check
2016-03-15 16:55:16 -07:00
filemap.c
mm: fs: initialize fsdata passed to write_begin/write_end interface
2022-11-25 17:35:43 +01:00
frame_vector.c
v4l2: don't fall back to follow_pfn() if pin_user_pages_fast() fails
2022-12-08 11:15:42 +01:00
frontswap.c
mm, frontswap: convert frontswap_enabled to static key
2016-07-26 16:19:19 -07:00
gup.c
gup: document and work around "COW can break either way" issue
2022-01-27 08:47:42 +01:00
highmem.c
mm/highmem: make nr_free_highpages() handles all highmem zones by itself
2016-05-19 19:12:14 -07:00
huge_memory.c
gup: document and work around "COW can break either way" issue
2022-01-27 08:47:42 +01:00
hugetlb.c
mm,hugetlb: take hugetlb_lock before decrementing h->resv_huge_pages
2022-11-03 23:49:17 +09:00
hugetlb_cgroup.c
mm: hugetlb: switch to css_tryget() in hugetlb_cgroup_charge_cgroup()
2019-11-25 09:51:58 +01:00
hwpoison-inject.c
init-mm.c
mm: Add a user_ns owner to mm_struct and fix ptrace permission checks
2017-01-06 10:40:13 +01:00
internal.h
vmscan: return NODE_RECLAIM_NOSCAN in node_reclaim() when CONFIG_NUMA is n
2019-12-05 15:35:02 +01:00
interval_tree.c
Kconfig
mm: don't allow deferred pages with NEED_PER_CPU_KM
2018-05-22 16:57:57 +02:00
Kconfig.debug
PM / Hibernate: allow hibernation with PAGE_POISONING_ZERO
2016-09-13 02:35:27 +02:00
khugepaged.c
mm/khugepaged: invoke MMU notifiers in shmem/file collapse paths
2023-01-07 12:07:10 +01:00
kmemcheck.c
mm: convert printk(KERN_<LEVEL> to pr_<level>
2016-03-17 15:09:34 -07:00
kmemleak-test.c
mm: convert printk(KERN_<LEVEL> to pr_<level>
2016-03-17 15:09:34 -07:00
kmemleak.c
Revert "mm: kmemleak: take a full lowmem check in kmemleak_*_phys()"
2022-09-15 12:39:45 +02:00
ksm.c
ksm: fix potential missing rmap_item for stable_node
2021-05-22 10:40:31 +02:00
list_lru.c
mm/list_lru.c: fix memory leak in __memcg_init_list_lru_node
2019-06-22 08:17:18 +02:00
maccess.c
uaccess: Add non-pagefault user-space write function
2020-09-12 11:47:35 +02:00
madvise.c
mm: madvise(MADV_DODUMP): allow hugetlbfs pages
2018-10-10 08:53:20 +02:00
Makefile
Disable the __builtin_return_address() warning globally after all
2016-10-12 10:23:41 -07:00
memblock.c
memblock: use kfree() to release kmalloced memblock regions
2022-03-02 11:32:06 +01:00
memcontrol.c
mm/memcontrol: return 1 from cgroup.memory __setup() handler
2022-04-20 09:06:41 +02:00
memory-failure.c
mm: hwpoison: change PageHWPoison behavior on hugetlb pages
2021-06-30 08:49:13 -04:00
memory.c
mm/khugepaged: fix GUP-fast interaction by sending IPI
2023-01-07 12:07:10 +01:00
memory_hotplug.c
mm: Avoid calling build_all_zonelists_init under hotplug context
2020-08-21 11:02:11 +02:00
mempolicy.c
mm/mempolicy: fix uninit-value in mpol_rebind_policy()
2022-07-29 17:05:46 +02:00
mempool.c
Revert "mm, mempool: only set __GFP_NOMEMALLOC if there are free elements"
2016-07-28 16:07:41 -07:00
memtest.c
migrate.c
hugetlbfs: fix races and page leaks during migration
2019-03-13 14:04:54 -07:00
mincore.c
mm/mincore.c: make mincore() more conservative
2019-05-21 18:48:58 +02:00
mlock.c
mm/mlock.c: change count_mm_mlocked_page_nr return type
2019-07-10 09:55:43 +02:00
mm_init.c
mm: convert printk(KERN_<LEVEL> to pr_<level>
2016-03-17 15:09:34 -07:00
mmap.c
mm: Fix TLB flush for not-first PFNMAP mappings in unmap_region()
2022-09-20 11:50:16 +02:00
mmu_context.c
mm/mmu_context, sched/core: Fix mmu_context.h assumption
2016-04-28 11:44:19 +02:00
mmu_notifier.c
mm/mmu_notifier: use hlist_add_head_rcu()
2019-08-04 09:33:42 +02:00
mmzone.c
mm, page_alloc: inline the fast path of the zonelist iterator
2016-05-19 19:12:14 -07:00
mprotect.c
x86/speculation/l1tf: Disallow non privileged high MMIO PROT_NONE mappings
2018-08-15 18:14:45 +02:00
mremap.c
mmmremap.c: avoid pointless invalidate_range_start/end on mremap(old_size=0)
2022-04-20 09:06:44 +02:00
msync.c
nobootmem.c
mm: discard memblock data later
2017-08-24 17:12:19 -07:00
nommu.c
x86/mm: split vmalloc_sync_all()
2020-04-02 17:20:26 +02:00
oom_kill.c
mm, oom: do not trigger out_of_memory from the #PF
2021-11-26 11:48:40 +01:00
page-writeback.c
mm: memcontrol: fix NULL pointer crash in test_clear_page_writeback()
2021-02-23 13:59:14 +01:00
page_alloc.c
mm: prevent page_frag_alloc() from corrupting the memory
2022-10-26 13:15:32 +02:00
page_counter.c
page_ext.c
mm/page_ext.c: fix an imbalance with kmemleak
2019-04-05 22:29:06 +02:00
page_idle.c
mm/page_idle.c: fix oops because end_pfn is larger than max_pfn
2019-07-10 09:55:38 +02:00
page_io.c
swap: fix swapfile read/write offset
2021-03-07 11:25:59 +01:00
page_isolation.c
mm/page_isolation: fix typo: "paes" -> "pages"
2016-10-07 18:46:29 -07:00
page_owner.c
mm/page_owner: don't define fields on struct page_ext by hard-coding
2016-10-07 18:46:27 -07:00
page_poison.c
mm: check the return value of lookup_page_ext for all call sites
2016-06-03 15:06:22 -07:00
pagewalk.c
mm: pagewalk: fix termination condition in walk_pte_range()
2020-10-01 20:40:06 +02:00
percpu-km.c
mm: percpu: use pr_fmt to prefix output
2016-03-17 15:09:34 -07:00
percpu-vm.c
percpu.c
percpu: stop printing kernel addresses
2019-04-27 09:34:47 +02:00
pgtable-generic.c
mm/thp/migration: switch from flush_tlb_range to flush_pmd_tlb_range
2016-03-17 15:09:34 -07:00
process_vm_access.c
mm: remove write/force parameters from __get_user_pages_unlocked()
2016-10-18 14:13:37 -07:00
quicklist.c
fix Christoph's email addresses
2016-03-17 15:09:34 -07:00
readahead.c
mm: silently skip readahead for DAX inodes
2016-08-26 17:39:35 -07:00
rmap.c
mm/rmap: Fix anon_vma->degree ambiguity leading to double-reuse
2022-09-05 10:23:58 +02:00
shmem.c
memfd: fix F_SEAL_WRITE after shmem huge page allocated
2022-03-08 19:00:59 +01:00
slab.c
mm/slab.c: fix an infinite loop in leaks_show()
2019-06-22 08:17:13 +02:00
slab.h
mm: kmemleak: slob: respect SLAB_NOLEAKTRACE flag
2021-11-26 11:48:42 +01:00
slab_common.c
mm/slab: use memzero_explicit() in kzfree()
2020-06-30 15:38:44 -04:00
slob.c
slub: move synchronize_sched out of slab_mutex on shrink
2017-03-22 12:43:38 +01:00
slub.c
mm/slub: fix to return errno if kmalloc() fails
2022-09-28 10:55:45 +02:00
sparse-vmemmap.c
treewide: replace obsolete _refok by __ref
2016-08-02 17:31:41 -04:00
sparse.c
mm/memory_hotplug: set magic number to page->freelist instead of page->lru.next
2017-10-21 17:21:36 +02:00
swap.c
thp: reduce usage of huge zero page's atomic counter
2016-10-07 18:46:28 -07:00
swap_cgroup.c
mm, swap_cgroup: reschedule when neeed in swap_cgroup_swapoff()
2017-07-05 14:40:17 +02:00
swap_state.c
mm: fix swap cache node allocation mask
2020-07-09 09:35:54 +02:00
swapfile.c
swap: fix swapfile read/write offset
2021-03-07 11:25:59 +01:00
truncate.c
mm: cleancache: fix corruption on missed inode invalidation
2018-12-08 13:05:09 +01:00
usercopy.c
usercopy: Avoid HIGHMEM pfn warning
2019-10-17 13:42:06 -07:00
userfaultfd.c
mm: userfaultfd: fix missing cache flush in mcopy_atomic_pte() and __mcopy_atomic()
2022-05-15 19:39:17 +02:00
util.c
random: move randomize_page() into mm where it belongs
2022-06-25 11:45:15 +02:00
vmacache.c
mm: get rid of vmacache_flush_all() entirely
2018-09-19 22:47:17 +02:00
vmalloc.c
mm/vmalloc.c: don't dereference possible NULL pointer in __vunmap()
2020-06-03 08:16:47 +02:00
vmpressure.c
mm: vmpressure: fix sending wrong events on underflow
2017-03-12 06:41:43 +01:00
vmscan.c
mm: remove seemingly spurious reclaimability check from laptop_mode gating
2018-09-19 22:47:12 +02:00
vmstat.c
mm, vmstat: drop zone->lock in /proc/pagetypeinfo
2021-06-03 08:23:27 +02:00
workingset.c
mm: workingset: fix premature shadow node shrinking with cgroups
2017-04-08 09:30:36 +02:00
z3fold.c
mm/z3fold.c: avoid modifying HEADLESS page and minor cleanup
2016-06-03 16:02:55 -07:00
zbud.c
zpool.c
zsmalloc.c
mm/zsmalloc.c: close race window between zs_pool_dec_isolated() and zs_unregister_migration()
2021-11-26 11:48:39 +01:00
zswap.c
zswap: re-check zswap_is_full() after do zswap_shrink()
2018-09-05 09:20:02 +02:00