Peter Zijlstra
ff77e46853
sched/rt: Fix PI handling vs. sched_setscheduler()
...
Andrea Parri reported:
> I found that the following scenario (with CONFIG_RT_GROUP_SCHED=y) is not
> handled correctly:
>
> T1 (prio = 20)
> lock(rtmutex);
>
> T2 (prio = 20)
> blocks on rtmutex (rt_nr_boosted = 0 on T1's rq)
>
> T1 (prio = 20)
> sys_set_scheduler(prio = 0)
> [new_effective_prio == oldprio]
> T1 prio = 20 (rt_nr_boosted = 0 on T1's rq)
>
> The last step is incorrect as T1 is now boosted (c.f., rt_se_boosted());
> in particular, if we continue with
>
> T1 (prio = 20)
> unlock(rtmutex)
> wakeup(T2)
> adjust_prio(T1)
> [prio != rt_mutex_getprio(T1)]
> dequeue(T1)
> rt_nr_boosted = (unsigned long)(-1)
> ...
> T1 prio = 0
>
> then we end up leaving rt_nr_boosted in an "inconsistent" state.
>
> The simple program attached could reproduce the previous scenario; note
> that, as a consequence of the presence of this state, the "assertion"
>
> WARN_ON(!rt_nr_running && rt_nr_boosted)
>
> from dec_rt_group() may trigger.
So normally we dequeue/enqueue tasks in sched_setscheduler(), which
would ensure the accounting stays correct. However in the early PI path
we fail to do so.
So this was introduced at around v3.14, by:
c365c292d0 ("sched: Consider pi boosting in setscheduler()")
which fixed another problem exactly because that dequeue/enqueue, joy.
Fix this by teaching rt about DEQUEUE_SAVE/ENQUEUE_RESTORE and have it
preserve runqueue location with that option. This requires decoupling
the on_rt_rq() state from being on the list.
In order to allow for explicit movement during the SAVE/RESTORE,
introduce {DE,EN}QUEUE_MOVE. We still must use SAVE/RESTORE in these
cases to preserve other invariants.
Respecting the SAVE/RESTORE flags also has the (nice) side-effect that
things like sys_nice()/sys_sched_setaffinity() also do not reorder
FIFO tasks (whereas they used to before this patch).
Reported-by: Andrea Parri <parri.andrea@gmail.com >
Tested-by: Andrea Parri <parri.andrea@gmail.com >
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org >
Cc: Juri Lelli <juri.lelli@arm.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Mike Galbraith <efault@gmx.de >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Steven Rostedt <rostedt@goodmis.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2016-02-29 09:53:05 +01:00
..
2015-12-13 19:59:48 -08:00
2015-12-14 10:11:09 -08:00
2016-02-04 18:26:08 +01:00
2016-01-20 18:03:56 -08:00
2016-02-04 18:25:55 +01:00
2015-12-11 16:56:16 +01:00
2016-01-13 16:00:32 +01:00
2015-11-21 18:42:00 +01:00
2015-12-22 17:04:56 +00:00
2015-12-29 10:08:45 +01:00
2016-01-07 10:10:50 -05:00
2016-01-18 12:10:45 -08:00
2016-02-17 10:29:27 -05:00
2016-02-16 15:21:47 -05:00
2016-01-05 18:04:58 +01:00
2016-01-07 09:55:39 -08:00
2015-12-28 18:43:15 +01:00
2015-12-21 14:26:28 +05:30
2016-01-24 12:50:56 -08:00
2016-02-21 20:49:34 +01:00
2015-12-17 22:08:28 -05:00
2016-01-21 14:47:08 -08:00
2016-01-12 18:26:10 +00:00
2016-02-09 11:54:23 +01:00
2016-02-21 22:03:15 -05:00
2016-01-11 16:48:35 +00:00
2015-12-16 16:36:25 +02:00
2016-01-23 18:45:06 -08:00
2016-01-13 09:26:40 -08:00
2016-01-12 01:10:19 +01:00
2015-12-23 08:37:10 -07:00
2016-01-04 16:19:57 +00:00
2016-01-14 13:40:30 +01:00
2016-01-17 19:13:15 -08:00
2016-01-09 08:39:04 -08:00
2015-12-13 15:55:55 +01:00
2016-01-24 03:47:37 +01:00
2016-01-24 03:49:03 +01:00
2015-12-03 09:32:21 -07:00
2015-12-09 10:35:16 -08:00
2015-12-22 09:38:34 -07:00
2015-12-01 10:53:59 -07:00
2016-02-04 21:37:52 -08:00
2015-12-06 12:46:31 +01:00
2015-11-25 12:14:09 -05:00
2015-11-25 11:35:14 -05:00
2015-12-03 10:24:08 -05:00
2016-01-22 10:42:57 -05:00
2016-01-12 19:20:32 -08:00
2016-01-27 09:10:29 -05:00
2015-12-23 13:08:56 -08:00
2015-11-25 22:28:30 +01:00
2016-02-15 13:06:00 -05:00
2015-12-07 00:02:05 +00:00
2016-01-20 17:20:53 -08:00
2016-01-16 11:17:25 -08:00
2015-11-24 09:56:43 +01:00
2015-12-04 10:34:45 +01:00
2016-01-01 03:49:51 +01:00
2016-01-20 17:09:18 -08:00
2016-01-22 10:22:46 -05:00
2016-02-27 10:28:52 -08:00
2016-01-14 16:00:49 -08:00
2015-12-30 13:01:03 -05:00
2016-01-13 17:30:32 +09:00
2016-01-08 01:12:06 +01:00
2016-02-06 23:45:46 -08:00
2016-01-20 17:09:18 -08:00
2016-01-20 17:09:18 -08:00
2016-01-06 15:17:47 +05:30
2016-01-15 22:08:45 +01:00
2016-01-04 10:58:35 +01:00
2015-11-25 09:22:00 -07:00
2015-11-25 09:22:02 -07:00
2015-12-11 16:56:40 +01:00
2016-02-10 16:25:52 +00:00
2015-12-11 11:05:57 -08:00
2016-01-16 11:17:24 -08:00
2016-01-06 16:31:56 -05:00
2015-12-15 08:53:36 -05:00
2015-12-14 18:58:42 -08:00
2015-12-15 15:41:24 +02:00
2016-01-08 21:40:34 -05:00
2016-01-30 13:35:31 -08:00
2016-02-18 16:23:24 -08:00
2016-02-18 16:34:15 -08:00
2015-12-22 17:10:17 -06:00
2016-01-09 22:42:31 -08:00
2016-02-05 18:10:40 -08:00
2015-12-05 17:41:42 -05:00
2015-12-28 13:41:50 +01:00
2016-01-17 11:13:55 +01:00
2016-01-21 17:20:51 -08:00
2016-01-15 17:56:32 -08:00
2015-12-21 13:07:56 -08:00
2016-01-13 11:06:03 +01:00
2015-11-25 09:22:00 -07:00
2015-11-20 11:31:26 -05:00
2015-12-03 11:49:26 -05:00
2015-12-15 16:50:20 -05:00
2015-12-15 23:26:51 -05:00
2015-12-04 10:34:46 +01:00
2016-02-15 12:42:38 +00:00
2016-01-23 18:45:06 -08:00
2016-01-20 17:09:18 -08:00
2016-01-29 12:34:29 +01:00
2015-12-02 23:37:16 -05:00
2015-12-11 11:52:29 -08:00
2015-12-14 10:03:46 +01:00
2016-01-26 16:00:14 +01:00
2015-11-23 09:44:58 +01:00
2016-01-16 11:17:23 -08:00
2016-01-16 11:17:22 -08:00
2015-11-20 15:55:52 -05:00
2016-01-20 17:09:18 -08:00
2015-12-15 10:01:43 -05:00
2015-12-12 10:15:34 -08:00
2015-11-28 19:33:29 -08:00
2016-02-25 11:27:16 +01:00
2015-11-25 17:24:23 +01:00
2016-01-15 17:56:32 -08:00
2016-02-09 11:54:23 +01:00
2016-01-04 09:57:40 +01:00
2016-01-25 15:20:44 -05:00
2016-02-23 17:17:20 -08:00
2016-02-04 09:19:45 -07:00
2015-11-23 10:37:35 -08:00
2016-01-20 17:09:18 -08:00
2015-11-23 10:37:35 -08:00
2016-01-15 17:56:32 -08:00
2015-12-03 23:01:27 +01:00
2016-02-11 18:35:48 -08:00
2015-11-25 09:22:02 -07:00
2015-12-24 11:09:40 -05:00
2016-01-20 17:09:18 -08:00
2016-01-07 14:31:27 -05:00
2016-01-15 17:56:32 -08:00
2016-02-03 08:28:43 -08:00
2016-01-15 17:56:32 -08:00
2016-01-14 16:00:49 -08:00
2016-01-15 17:56:32 -08:00
2016-01-14 16:00:49 -08:00
2016-02-03 08:28:43 -08:00
2016-02-03 08:28:43 -08:00
2016-01-15 17:56:32 -08:00
2016-02-03 08:28:43 -08:00
2015-12-14 19:15:05 -08:00
2016-02-03 16:58:15 +10:30
2015-11-30 15:26:22 -05:00
2016-01-21 11:52:16 -08:00
2016-01-09 03:07:52 -05:00
2015-12-01 15:45:05 -05:00
2015-12-15 16:50:20 -05:00
2016-02-19 15:29:26 -05:00
2015-12-15 23:25:20 -05:00
2016-01-14 16:08:23 -08:00
2015-12-28 09:57:15 -05:00
2016-02-08 15:20:01 -05:00
2016-02-17 11:44:45 -05:00
2015-12-01 10:59:38 -07:00
2015-12-17 10:43:06 -06:00
2016-01-05 11:20:12 +01:00
2015-12-09 09:23:28 -06:00
2016-01-15 12:30:35 -06:00
2016-01-25 08:35:50 -06:00
2015-12-18 11:17:26 +05:30
2016-01-15 17:56:32 -08:00
2016-01-22 17:02:18 -08:00
2016-01-15 17:29:37 -06:00
2016-02-17 17:23:36 -06:00
2016-02-25 08:42:34 +01:00
2016-02-11 18:35:48 -08:00
2016-02-11 18:35:48 -08:00
2016-01-07 14:31:27 -05:00
2015-11-30 15:26:22 -05:00
2016-01-19 19:25:21 -05:00
2016-01-12 20:25:09 -08:00
2016-01-08 01:12:06 +01:00
2015-12-10 23:08:51 +01:00
2015-12-21 03:11:12 +01:00
2016-01-08 01:12:06 +01:00
2016-01-22 17:02:18 -08:00
2016-01-16 11:17:22 -08:00
2015-12-06 21:25:17 -05:00
2016-01-02 00:29:35 +01:00
2016-01-16 11:17:29 -08:00
2016-01-01 02:09:51 +01:00
2015-11-23 09:44:58 +01:00
2016-01-20 17:09:18 -08:00
2016-02-05 18:10:40 -08:00
2016-02-27 10:28:52 -08:00
2016-01-20 17:09:18 -08:00
2015-12-07 17:02:54 -08:00
2015-12-07 17:02:54 -08:00
2015-12-07 17:01:31 -08:00
2015-12-07 17:01:31 -08:00
2016-01-05 19:07:17 +00:00
2015-12-17 22:08:28 -05:00
2016-02-05 18:10:40 -08:00
2016-01-10 22:13:15 -05:00
2015-12-15 09:41:09 +01:00
2016-02-29 09:53:05 +01:00
2015-11-19 16:22:43 +01:00
2015-12-24 11:09:40 -05:00
2015-12-13 19:59:48 -08:00
2015-12-17 11:18:44 +01:00
2015-11-24 11:48:35 +09:00
2016-01-04 16:11:11 -05:00
2016-01-20 17:09:18 -08:00
2016-01-22 18:08:52 -05:00
2015-11-20 16:17:32 -08:00
2016-02-09 04:28:06 -05:00
2016-01-20 17:09:18 -08:00
2016-01-20 17:09:18 -08:00
2016-01-20 17:09:18 -08:00
2015-12-15 23:26:51 -05:00
2016-01-06 11:02:29 +01:00
2016-01-04 10:20:19 -05:00
2016-02-25 11:27:16 +01:00
2016-01-20 17:09:18 -08:00
2016-01-20 17:29:52 -05:00
2015-12-01 14:00:53 -05:00
2015-12-29 16:00:00 +08:00
2016-01-14 16:00:49 -08:00
2016-01-15 22:34:39 +01:00
2015-12-10 22:41:06 -08:00
2016-01-18 14:49:33 -05:00
2015-12-06 12:56:06 +01:00
2016-02-15 13:04:46 -05:00
2016-01-26 23:17:54 -08:00
2015-11-25 15:49:13 +00:00
2015-12-17 09:57:27 -08:00
2016-02-10 13:19:03 +00:00
2015-12-18 17:48:50 -08:00
2015-12-06 20:42:15 -05:00
2015-11-23 09:44:58 +01:00
2015-12-01 14:58:18 -08:00
2015-12-21 15:28:11 -07:00
2015-12-04 10:38:59 -02:00
2016-01-12 20:47:06 +02:00
2016-01-12 20:47:02 +02:00
2016-01-15 17:56:32 -08:00
2016-01-14 16:00:49 -08:00
2016-01-14 16:00:49 -08:00
2016-01-14 16:00:49 -08:00
2015-12-04 10:34:45 +01:00
2016-02-24 09:09:45 +01:00
2016-01-11 21:53:59 +01:00
2015-12-03 07:24:29 -08:00
2016-01-29 13:31:10 -05:00
2015-12-13 19:46:12 -05:00