Matt Fleming
478273e115
sched/loadavg: Avoid loadavg spikes caused by delayed NO_HZ accounting
...
commit 6e5f32f7a43f45ee55c401c0b9585eb01f9629a8 upstream.
If we crossed a sample window while in NO_HZ we will add LOAD_FREQ to
the pending sample window time on exit, setting the next update not
one window into the future, but two.
This situation on exiting NO_HZ is described by:
this_rq->calc_load_update < jiffies < calc_load_update
In this scenario, what we should be doing is:
this_rq->calc_load_update = calc_load_update [ next window ]
But what we actually do is:
this_rq->calc_load_update = calc_load_update + LOAD_FREQ [ next+1 window ]
This has the effect of delaying load average updates for potentially
up to ~9seconds.
This can result in huge spikes in the load average values due to
per-cpu uninterruptible task counts being out of sync when accumulated
across all CPUs.
It's safe to update the per-cpu active count if we wake between sample
windows because any load that we left in 'calc_load_idle' will have
been zero'd when the idle load was folded in calc_global_load().
This issue is easy to reproduce before,
commit 9d89c257df ("sched/fair: Rewrite runnable load and utilization average tracking")
just by forking short-lived process pipelines built from ps(1) and
grep(1) in a loop. I'm unable to reproduce the spikes after that
commit, but the bug still seems to be present from code review.
Signed-off-by: Matt Fleming <matt@codeblueprint.co.uk>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Mike Galbraith <umgwanakikbuti@gmail.com>
Cc: Morten Rasmussen <morten.rasmussen@arm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Vincent Guittot <vincent.guittot@linaro.org>
Fixes: commit 5167e8d ("sched/nohz: Rewrite and fix load-avg computation -- again")
Link: http://lkml.kernel.org/r/20170217120731.11868-2-matt@codeblueprint.co.uk
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2017-07-05 14:40:28 +02:00
..
bpf
bpf: don't trigger OOM killer under pressure with map alloc
2017-07-05 14:40:21 +02:00
configs
config: android: enable CONFIG_SECCOMP
2016-10-11 15:06:32 -07:00
debug
kernel/debug/debug_core.c: more properly delay for secondary CPUs
2017-01-06 10:40:17 +01:00
events
perf/core: Fix sys_perf_event_open() vs. hotplug
2017-07-05 14:40:26 +02:00
gcov
gcov: add support for gcc version >= 6
2016-07-15 14:54:27 +09:00
irq
genirq: Release resources in __setup_irq() error path
2017-06-24 07:11:17 +02:00
livepatch
livepatch/module: make TAINT_LIVEPATCH module-specific
2016-08-26 14:42:08 +02:00
locking
locking/rwsem: Fix down_write_killable() for CONFIG_RWSEM_GENERIC_SPINLOCK=y
2017-03-22 12:43:40 +01:00
power
PM / sleep: fix device reference leak in test_suspend
2016-11-02 05:10:04 +01:00
printk
printk: use rcuidle console tracepoint
2017-02-23 17:44:36 +01:00
rcu
rcu: Narrow early boot window of illegal synchronous grace periods
2017-01-26 08:24:37 +01:00
sched
sched/loadavg: Avoid loadavg spikes caused by delayed NO_HZ accounting
2017-07-05 14:40:28 +02:00
time
time: Fix CLOCK_MONOTONIC_RAW sub-nanosecond accounting
2017-06-29 13:00:30 +02:00
trace
tracing/kprobes: Enforce kprobes teardown after testing
2017-05-25 15:44:47 +02:00
.gitignore
acct.c
async.c
audit.c
Merge branch 'stable-4.9' of git://git.infradead.org/users/pcmoore/audit
2016-10-04 14:21:41 -07:00
audit.h
Merge branch 'stable-4.8' of git://git.infradead.org/users/pcmoore/audit
2016-07-29 17:54:17 -07:00
audit_fsnotify.c
audit_tree.c
audit: cleanup prune_tree_thread
2016-04-04 09:46:47 -04:00
audit_watch.c
Merge branch 'stable-4.8' of git://git.infradead.org/users/pcmoore/audit
2016-09-01 15:55:56 -07:00
auditfilter.c
audit: add fields to exclude filter by reusing user filter
2016-06-27 11:01:00 -04:00
auditsc.c
Merge branch 'stable-4.9' of git://git.infradead.org/users/pcmoore/audit
2016-10-04 14:21:41 -07:00
backtracetest.c
bounds.c
capability.c
ptrace: Capture the ptracer's creds not PT_PTRACE_CAP
2017-01-06 10:40:13 +01:00
cgroup.c
cgroup: Prevent kill_css() from being called more than once
2017-06-14 15:06:00 +02:00
cgroup_freezer.c
cgroup_pids.c
cgroup/pids: remove spurious suspicious RCU usage warning
2017-03-26 13:05:58 +02:00
compat.c
configs.c
context_tracking.c
cpu.c
cpu/hotplug: Drop the device lock on error
2017-06-14 15:06:04 +02:00
cpu_pm.c
cpuset.c
cpuset: consider dying css as offline
2017-06-14 15:06:00 +02:00
crash_dump.c
cred.c
cred: Reject inodes with invalid ids in set_create_file_as()
2016-06-30 18:05:09 -05:00
delayacct.c
dma.c
elfcore.c
exec_domain.c
exit.c
sched/autogroup: Do not use autogroup->tg in zombie threads
2016-11-22 12:33:43 +01:00
extable.c
fork.c
stackprotector: Increase the per-task stack canary's random range from 32 bits to 64 bits on 64-bit platforms
2017-05-25 15:44:46 +02:00
freezer.c
freezer, oom: check TIF_MEMDIE on the correct task
2016-07-28 16:07:41 -07:00
futex.c
futex: Add missing error handling to FUTEX_REQUEUE_PI
2017-03-22 12:43:40 +01:00
futex_compat.c
groups.c
cred: simpler, 1D supplementary groups
2016-10-07 18:46:30 -07:00
hung_task.c
hung_task: allow hung_task_panic when hung_task_warnings is 0
2016-10-11 15:06:33 -07:00
irq_work.c
jump_label.c
jump_labels: API for flushing deferred jump label updates
2017-01-19 20:17:59 +01:00
kallsyms.c
kallsyms: add support for relative offsets in kallsyms address table
2016-03-15 16:55:16 -07:00
kcmp.c
Kconfig.freezer
Kconfig.hz
Kconfig.locks
Kconfig.preempt
kcov.c
kcov: add missing #include <linux/sched.h>
2016-12-07 17:10:00 -08:00
kexec.c
kexec: allow architectures to override boot mapping
2016-08-02 19:35:27 -04:00
kexec_core.c
kexec: add restriction on kexec_load() segment sizes
2016-08-02 19:35:31 -04:00
kexec_file.c
kexec: fix double-free when failing to relocate the purgatory
2016-09-01 17:52:01 -07:00
kexec_internal.h
kmod.c
kprobes.c
tracing/kprobes: Enforce kprobes teardown after testing
2017-05-25 15:44:47 +02:00
ksysfs.c
kexec: add a kexec_crash_loaded() function
2016-08-02 19:35:30 -04:00
kthread.c
cgroup, kthread: close race window where new kthreads can be migrated to non-root cgroups
2017-04-21 09:31:18 +02:00
latencytop.c
Makefile
kernel/watchdog.c: move hardlockup detector to separate file
2017-06-17 06:41:57 +02:00
membarrier.c
Fix: Disable sys_membarrier when nohz_full is enabled
2017-03-12 06:41:45 +01:00
memremap.c
mm, devm_memremap_pages: hold device_hotplug lock over mem_hotplug_{begin, done}
2017-03-12 06:41:43 +01:00
module-internal.h
module.c
Re-enable CONFIG_MODVERSIONS in a slightly weaker form
2016-11-29 16:01:30 -08:00
module_signing.c
KEYS: Move the point of trust determination to __key_link()
2016-04-11 22:43:43 +01:00
notifier.c
nsproxy.c
padata.c
padata: free correct variable
2017-05-20 14:28:40 +02:00
panic.c
kernel/panic.c: add missing \n
2017-07-05 14:40:24 +02:00
params.c
pid.c
remove lots of IS_ERR_VALUE abuses
2016-05-27 15:26:11 -07:00
pid_namespace.c
pid_ns: Sleep in TASK_INTERRUPTIBLE in zap_pid_ns_processes
2017-05-25 15:44:38 +02:00
profile.c
profile: Convert to hotplug state machine
2016-07-15 10:41:42 +02:00
ptrace.c
ptrace: Properly initialize ptracer_cred on fork
2017-06-14 15:05:54 +02:00
range.c
reboot.c
relay.c
relay: check array offset before using it
2017-01-12 11:39:30 +01:00
resource.c
/proc/iomem: only expose physical resource addresses to privileged users
2016-04-14 12:56:09 -07:00
seccomp.c
seccomp: Fix tracer exit notifications during fatal signals
2016-08-30 16:12:46 -07:00
signal.c
signal: Only reschedule timers on signals timers have sent
2017-06-29 13:00:29 +02:00
smp.c
smp: Allocate smp_call_on_cpu() workqueue on stack too
2016-09-22 14:49:10 +02:00
smpboot.c
kthread/smpboot: do not park in kthread_create_on_cpu()
2016-10-11 15:06:33 -07:00
smpboot.h
softirq.c
softirq: Display IRQ_POLL for irq-poll statistics
2016-10-21 15:45:47 -06:00
stacktrace.c
stacktrace, lockdep: Fix address, newline ugliness
2017-02-14 15:25:42 -08:00
stop_machine.c
Merge branch 'sched-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2016-10-03 13:39:00 -07:00
sys.c
prctl: make PR_SET_THP_DISABLE wait for mmap_sem killable
2016-05-23 17:04:14 -07:00
sys_ni.c
x86/pkeys: Fix pkeys build breakage for some non-x86 arches
2016-09-13 14:41:36 +02:00
sysctl.c
sysctl: fix proc_doulongvec_ms_jiffies_minmax()
2017-02-01 08:33:05 +01:00
sysctl_binary.c
kernel/sysctl_binary.c: use generic UUID library
2016-05-20 17:58:30 -07:00
task_work.c
task_work: use READ_ONCE/lockless_dereference, avoid pi_lock if !task_works
2016-08-02 19:35:02 -04:00
taskstats.c
taskstats: fix the length of cgroupstats_cmd_get_policy
2016-11-03 16:55:58 -04:00
test_kprobes.c
torture.c
torture: Convert torture_shutdown() to hrtimer
2016-08-22 10:01:49 -07:00
tracepoint.c
kernel/...: convert pr_warning to pr_warn
2016-03-22 15:36:02 -07:00
tsacct.c
ucount.c
kernel/ucount.c: mark user_header with kmemleak_ignore()
2017-06-17 06:41:51 +02:00
uid16.c
cred: simpler, 1D supplementary groups
2016-10-07 18:46:30 -07:00
up.c
smp: Add function to execute a function synchronously on a CPU
2016-09-05 13:52:39 +02:00
user-return-notifier.c
user.c
user_namespace.c
Merge branch 'nsfs-ioctls' into HEAD
2016-09-22 20:00:36 -05:00
utsname.c
Merge branch 'nsfs-ioctls' into HEAD
2016-09-22 20:00:36 -05:00
utsname_sysctl.c
watchdog.c
kernel/watchdog: prevent false hardlockup on overloaded system
2017-06-17 06:41:57 +02:00
watchdog_hld.c
kernel/watchdog: prevent false hardlockup on overloaded system
2017-06-17 06:41:57 +02:00
workqueue.c
kthread: rename probe_kthread_data() to kthread_probe_data()
2016-10-11 15:06:33 -07:00
workqueue_internal.h