Tejun Heo
dbf2576e37
workqueue: make all workqueues non-reentrant ...
By default, each per-cpu part of a bound workqueue operates separately
and a work item may be executing concurrently on different CPUs. The
behavior avoids some cross-cpu traffic but leads to subtle weirdities
and not-so-subtle contortions in the API.
* There's no sane usefulness in allowing a single work item to be
executed concurrently on multiple CPUs. People just get the
behavior unintentionally and get surprised after learning about it.
Most either explicitly synchronize or use non-reentrant/ordered
workqueue but this is error-prone.
* flush_work() can't wait for multiple instances of the same work item
on different CPUs. If a work item is executing on cpu0 and then
queued on cpu1, flush_work() can only wait for the one on cpu1.
Unfortunately, work items can easily cross CPU boundaries
unintentionally when the queueing thread gets migrated. This means
that if multiple queuers compete, flush_work() can't even guarantee
that the instance queued right before it is finished before
returning.
* flush_work_sync() was added to work around some of the deficiencies
of flush_work(). In addition to the usual flushing, it ensures that
all currently executing instances are finished before returning.
This operation is expensive as it has to walk all CPUs and at the
same time fails to address competing queuer case.
Incorrectly using flush_work() when flush_work_sync() is necessary
is an easy error to make and can lead to bugs which are difficult to
reproduce.
* Similar problems exist for flush_delayed_work[_sync]().
Other than the cross-cpu access concern, there's no benefit in
allowing parallel execution and it's plain silly to have this level of
contortion for workqueue which is widely used from core code to
extremely obscure drivers.
This patch makes all workqueues non-reentrant. If a work item is
executing on a different CPU when queueing is requested, it is always
queued to that CPU. This guarantees that any given work item can be
executing on one CPU at maximum and if a work item is queued and
executing, both are on the same CPU.
The only behavior change which may affect workqueue users negatively
is that non-reentrancy overrides the affinity specified by
queue_work_on(). On a reentrant workqueue, the affinity specified by
queue_work_on() is always followed. Now, if the work item is
executing on one of the CPUs, the work item will be queued there
regardless of the requested affinity. I've reviewed all workqueue
users which request explicit affinity, and, fortunately, none seems to
be crazy enough to exploit parallel execution of the same work item.
This adds an additional busy_hash lookup if the work item was
previously queued on a different CPU. This shouldn't be noticeable
under any sane workload. Work item queueing isn't a very
high-frequency operation and they don't jump across CPUs all the time.
In a micro benchmark to exaggerate this difference - measuring the
time it takes for two work items to repeatedly jump between two CPUs a
number (10M) of times with busy_hash table densely populated, the
difference was around 3%.
While the overhead is measureable, it is only visible in pathological
cases and the difference isn't huge. This change brings much needed
sanity to workqueue and makes its behavior consistent with timer. I
think this is the right tradeoff to make.
This enables significant simplification of workqueue API.
Simplification patches will follow.
Signed-off-by: Tejun Heo <tj@kernel.org>
2012-08-20 14:51:23 -07:00
kdb: Switch to nolock variants of kmsg_dump functions
2012-07-21 10:34:00 -07:00
uprobes: __replace_page() needs munlock_vma_page()
2012-07-30 11:27:25 +02:00
gcov: disable CONSTRUCTORS for UML
2011-07-26 16:49:45 -07:00
irqdomain changes for Linux v3.6
2012-07-31 20:44:03 -07:00
NMI watchdog: fix for lockup detector breakage on resume
2012-07-30 17:25:13 -07:00
Merge branch 'perf-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2012-07-31 15:34:13 -07:00
Merge branch 'timers-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2012-07-22 11:35:46 -07:00
Staging tree patches for 3.6-rc1
2012-07-26 11:14:49 -07:00
Merge branch 'for-linus2' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs
2012-01-08 12:19:57 -08:00
[SCSI] async: make async_synchronize_full() flush all work regardless of domain
2012-07-20 09:07:37 +01:00
Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs
2012-08-01 10:26:23 -07:00
audit: remove AUDIT_SETUP_CONTEXT as it isn't used
2012-01-17 16:16:57 -05:00
VFS: Make clone_mnt()/copy_tree()/collect_mounts() return errors
2012-07-14 16:37:27 +04:00
get rid of kern_path_parent()
2012-07-14 16:35:02 +04:00
audit: allow interfield comparison in audit rules
2012-01-17 16:17:01 -05:00
seccomp: remove duplicated failure logging
2012-04-14 11:13:20 +10:00
userns: Teach inode_capable to understand inodes whose uids map to other namespaces.
2012-05-15 14:59:24 -07:00
Merge branch 'for-3.6' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup
2012-07-24 17:47:44 -07:00
cgroup: convert all non-memcg controllers to the new cftype interface
2012-04-01 12:09:55 -07:00
new helper: sigsuspend()
2012-05-21 23:52:30 -04:00
kernel/configs.c: include MODULE_*() when CONFIG_IKCONFIG_PROC=n
2011-07-25 20:57:15 -07:00
mm/hotplug: correctly setup fallback zonelists when creating new pgdat
2012-07-31 18:42:44 -07:00
kernel/cpu_pm.c: fix various typos
2012-05-31 17:49:27 -07:00
cpusets: Remove/update outdated comments
2012-07-24 13:53:28 +02:00
Merge branch 'modsplit-Oct31_2011' of git://git.kernel.org/pub/scm/linux/kernel/git/paulg/linux
2011-11-06 19:44:47 -08:00
keys: kill task_struct->replacement_session_keyring
2012-05-23 22:11:41 -04:00
KVM: Steal time implementation
2011-07-14 12:59:14 +03:00
Remove all #inclusions of asm/system.h
2012-03-28 18:30:03 +01:00
posix_types.h: Cleanup stale __NFDBITS and related definitions
2012-07-26 13:36:43 -07:00
extable: Skip sorting if sorted at build time.
2012-04-19 15:06:55 -07:00
memcg: rename config variables
2012-07-31 18:42:43 -07:00
PM / Freezer: Remove references to TIF_FREEZE in comments
2012-03-04 23:08:54 +01:00
futex: Mark get_robust_list as deprecated
2012-03-29 11:37:17 +02:00
futex: Mark get_robust_list as deprecated
2012-03-29 11:37:17 +02:00
userns: Convert in_group_p and in_egroup_p to use kgid_t
2012-05-03 03:29:33 -07:00
hrtimer: Update hrtimer base offsets each hrtimer_interrupt
2012-07-11 23:34:39 +02:00
hung task debugging: Inject NMI when hung and going to panic
2012-04-25 12:39:25 +02:00
irq_work: fix compile failure on tile from missing include
2012-04-13 13:15:16 -04:00
itimer: Use printk_once instead of WARN_ONCE
2012-04-10 11:00:30 +02:00
static keys: Inline the static_key_enabled() function
2012-02-28 20:01:08 +01:00
vsprintf: fix %ps on non symbols when using kallsyms
2012-05-29 16:22:32 -07:00
syscalls, x86: add __NR_kcmp syscall
2012-05-31 17:49:32 -07:00
locking/kconfig: Simplify INLINE_SPIN_UNLOCK usage
2012-03-23 13:18:57 +01:00
locking/kconfig: Simplify INLINE_SPIN_UNLOCK usage
2012-03-23 13:18:57 +01:00
kdump: append newline to the last lien of vmcoreinfo note
2012-07-30 17:25:20 -07:00
[media] kernel:kfifo: export __kfifo_max_r symbol
2012-04-11 18:24:37 -03:00
kmod: avoid deadlock from recursive kmod call
2012-07-30 17:25:20 -07:00
kprobes: return proper error code from register_kprobe()
2012-03-05 15:49:42 -08:00
kernel: ksysfs.c is implicitly using stat.h
2011-10-31 09:20:13 -04:00
kthread_worker: reimplement flush_kthread_work() to allow freeing the work item being executed
2012-07-22 10:15:28 -07:00
kernel: Map most files to use export.h instead of module.h
2011-10-31 09:20:12 -04:00
brlocks/lglocks: turn into functions
2012-05-29 23:28:41 -04:00
lockdep: Add CPU-idle/offline warning to lockdep-RCU splat
2012-02-21 09:06:06 -08:00
kernel: Map most files to use export.h instead of module.h
2011-10-31 09:20:12 -04:00
Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs
2012-06-01 10:34:35 -07:00
Guard check in module loader against integer overflow
2012-05-23 22:28:53 +09:30
kernel: Map most files to use export.h instead of module.h
2011-10-31 09:20:12 -04:00
sched/rt: Use schedule_preempt_disabled()
2012-03-01 10:28:03 +01:00
kernel: Map most files to use export.h instead of module.h
2011-10-31 09:20:12 -04:00
kernel: Map most files to use export.h instead of module.h
2011-10-31 09:20:12 -04:00
padata: Fix cpu hotplug
2012-03-29 19:52:46 +08:00
panic: fix a possible deadlock in panic()
2012-07-30 17:25:13 -07:00
params: replace printk(KERN_<LVL>...) with pr_<lvl>(...)
2012-05-04 17:28:18 -07:00
mm: add a low limit to alloc_large_system_hash
2012-05-24 00:28:21 -04:00
pidns: guarantee that the pidns init will be the last pidns process reaped
2012-06-20 14:39:36 -07:00
[S390] cputime: add sparse checking and cleanup
2011-12-15 14:56:19 +01:00
kernel: Map most files to use export.h instead of module.h
2011-10-31 09:20:12 -04:00
printk: only look for prefix levels in kernel messages
2012-07-30 17:25:14 -07:00
kernel: Map most files to use export.h instead of module.h
2011-10-31 09:20:12 -04:00
userns: Convert ptrace, kill, set_priority permission checks to work with kuids and kgids
2012-05-03 03:28:51 -07:00
range: fix bogus misuse of module.h to get printk()
2011-10-31 09:20:11 -04:00
rcu: Allow nesting of rcu_idle_enter() and rcu_idle_exit()
2012-02-21 09:06:12 -08:00
rcu: Consolidate tree/tiny __rcu_read_{,un}lock() implementations
2012-07-02 12:34:23 -07:00
rcu: Fix rcu_is_cpu_idle() #ifdef in TINY_RCU
2012-07-02 12:34:25 -07:00
rcu: Fix code-style issues involving "else"
2012-07-06 06:01:48 -07:00
rcu: Fix broken strings in RCU's source code.
2012-07-06 06:01:49 -07:00
rcu: Fix code-style issues involving "else"
2012-07-06 06:01:48 -07:00
Merge branches 'bigrtm.2012.07.04a', 'doctorture.2012.07.02a', 'fixes.2012.07.06a' and 'fnh.2012.07.02a' into HEAD
2012-07-06 05:59:30 -07:00
rcu: Fix code-style issues involving "else"
2012-07-06 06:01:48 -07:00
rcu: Fix broken strings in RCU's source code.
2012-07-06 06:01:49 -07:00
splice: fix racy pipe->buffers uses
2012-06-13 21:16:42 +02:00
rescounters: add res_counter_uncharge_until()
2012-05-29 16:22:27 -07:00
resource: make sure requested range is included in the root range
2012-07-30 17:25:21 -07:00
lockdep, rtmutex, bug: Show taint flags on error
2011-12-06 08:16:49 +01:00
rtmutex-tester: convert sysdev_class to a regular subsystem
2011-12-14 14:54:22 -08:00
Revert "rcu: Permit rt_mutex_unlock() with irqs disabled"
2011-12-11 10:33:18 -08:00
Remove all #inclusions of asm/system.h
2012-03-28 18:30:03 +01:00
seccomp: fix build warnings when there is no CONFIG_SECCOMP_FILTER
2012-04-18 12:24:52 +10:00
semaphore: fix improper comment reference to mutex
2012-04-05 17:15:55 -07:00
signal: make sure we don't get stopped with pending task_work
2012-07-22 23:57:54 +04:00
smp: Remove ipi_call_lock[_irq]()/ipi_call_unlock[_irq]()
2012-06-05 17:27:14 +02:00
smpboot, idle: Fix comment mismatch over idle_threads_init()
2012-05-24 22:58:08 +02:00
smpboot: Remove leftover declaration
2012-06-11 15:07:52 +02:00
mm: allow PF_MEMALLOC from softirq context
2012-07-31 18:42:45 -07:00
locking/kconfig: Simplify INLINE_SPIN_UNLOCK usage
2012-03-23 13:18:57 +01:00
rcu: Implement per-domain single-threaded call_srcu() state machine
2012-04-30 10:48:25 -07:00
kernel: Map most files to use export.h instead of module.h
2011-10-31 09:20:12 -04:00
Merge branch 'modsplit-Oct31_2011' of git://git.kernel.org/pub/scm/linux/kernel/git/paulg/linux
2011-11-06 19:44:47 -08:00
kernel/sys.c: avoid argv_free(NULL)
2012-07-30 17:25:13 -07:00
syscalls, x86: add __NR_kcmp syscall
2012-05-31 17:49:32 -07:00
Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs
2012-08-01 10:26:23 -07:00
mm: prepare for removal of obsolete /proc/sys/vm/nr_pdflush_threads
2012-07-31 18:42:40 -07:00
deal with task_work callbacks adding more work
2012-07-22 23:57:57 +04:00
taskstats: check nla_reserve() return
2012-07-30 17:25:21 -07:00
time: Remove bogus comments
2012-03-15 18:17:55 -07:00
timers: Improve get_next_timer_interrupt()
2012-06-06 13:49:02 +02:00
static keys: Introduce 'struct static_key', static_key_true()/false() and static_key_slow_[inc|dec]()
2012-02-24 10:05:59 +01:00
[S390] cputime: add sparse checking and cleanup
2011-12-15 14:56:19 +01:00
userns: Convert setting and getting uid and gid system calls to use kuid and kgid
2012-05-03 03:28:41 -07:00
kernel: Map most files to use export.h instead of module.h
2011-10-31 09:20:12 -04:00
kernel: Map most files to use export.h instead of module.h
2011-10-31 09:20:12 -04:00
userns: Silence silly gcc warning.
2012-05-19 15:44:40 -06:00
userns: Store uid and gid values in struct cred with kuid_t and kgid_t types
2012-05-03 03:28:38 -07:00
userns: Use cred->user_ns instead of cred->user->user_ns
2012-04-07 16:55:51 -07:00
Merge branch 'modsplit-Oct31_2011' of git://git.kernel.org/pub/scm/linux/kernel/git/paulg/linux
2011-11-06 19:44:47 -08:00
lockdep/waitqueues: Add better annotation
2011-12-21 10:07:39 +01:00
NMI watchdog: fix for lockup detector breakage on resume
2012-07-30 17:25:13 -07:00
workqueue: make all workqueues non-reentrant
2012-08-20 14:51:23 -07:00
dbf2576e37