Nathan Zimmer 4a8c7bb59a
mm/mempolicy.c: convert the shared_policy lock to a rwlock ...
When running the SPECint_rate gcc on some very large boxes it was
noticed that the system was spending lots of time in
mpol_shared_policy_lookup(). The gamess benchmark can also show it and
is what I mostly used to chase down the issue since the setup for that I
found to be easier.
To be clear the binaries were on tmpfs because of disk I/O requirements.
We then used text replication to avoid icache misses and having all the
copies banging on the memory where the instruction code resides. This
results in us hitting a bottleneck in mpol_shared_policy_lookup() since
lookup is serialised by the shared_policy lock.
I have only reproduced this on very large (3k+ cores) boxes. The
problem starts showing up at just a few hundred ranks getting worse
until it threatens to livelock once it gets large enough. For example
on the gamess benchmark at 128 ranks this area consumes only ~1% of
time, at 512 ranks it consumes nearly 13%, and at 2k ranks it is over
90%.
To alleviate the contention in this area I converted the spinlock to an
rwlock. This allows a large number of lookups to happen simultaneously.
The results were quite good reducing this consumtion at max ranks to
around 2%.
[akpm@linux-foundation.org: tidy up code comments]
Signed-off-by: Nathan Zimmer <nzimmer@sgi.com>
Acked-by: David Rientjes <rientjes@google.com>
Acked-by: Vlastimil Babka <vbabka@suse.cz>
Cc: Nadia Yvette Chambers <nyc@holomorphy.com>
Cc: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
Cc: Mel Gorman <mgorman@suse.de>
Cc: "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-01-14 16:00:49 -08:00
2015-12-13 19:59:48 -08:00
2015-12-14 10:11:09 -08:00
2015-11-02 23:37:46 +01:00
2015-12-11 16:56:16 +01:00
2015-12-22 17:04:56 +00:00
2015-12-29 10:08:45 +01:00
2016-01-13 10:13:36 -08:00
2015-12-17 22:08:28 -05:00
2016-01-11 17:48:53 -05:00
2016-01-05 18:04:58 +01:00
2016-01-07 09:55:39 -08:00
2015-12-28 18:43:15 +01:00
2015-11-16 15:41:49 +00:00
2015-12-21 14:26:28 +05:30
2015-10-27 11:24:23 +01:00
2016-01-13 11:38:27 -08:00
2015-12-17 22:08:28 -05:00
2015-12-11 19:34:39 -05:00
2016-01-11 16:48:35 +00:00
2015-12-16 16:36:25 +02:00
2015-11-11 20:11:28 -08:00
2016-01-13 09:26:40 -08:00
2016-01-12 01:10:19 +01:00
2016-01-04 16:19:57 +00:00
2015-11-03 17:22:17 -08:00
2015-11-04 08:23:51 -05:00
2015-11-05 14:51:32 -08:00
2016-01-09 08:39:04 -08:00
2015-12-09 10:35:16 -08:00
2015-11-11 09:37:34 -07:00
2015-10-27 15:04:56 +09:00
2015-11-07 10:40:47 -07:00
2015-12-28 13:07:07 -07:00
2015-12-06 12:46:31 +01:00
2015-11-25 12:14:09 -05:00
2015-11-25 11:35:14 -05:00
2015-11-11 02:19:33 -05:00
2015-12-03 10:24:08 -05:00
2016-01-12 19:20:32 -08:00
2016-01-12 19:20:32 -08:00
2015-10-26 12:36:55 -07:00
2015-11-25 22:28:30 +01:00
2015-11-05 19:34:48 -08:00
2015-11-06 17:50:42 -08:00
2015-12-04 10:33:41 +01:00
2015-12-07 00:02:05 +00:00
2016-01-04 12:31:46 +01:00
2015-11-24 09:56:43 +01:00
2015-12-04 10:34:45 +01:00
2016-01-01 03:49:51 +01:00
2015-11-06 17:50:42 -08:00
2015-11-16 09:27:32 +05:30
2016-01-14 16:00:49 -08:00
2015-12-30 13:01:03 -05:00
2015-11-02 13:14:44 -08:00
2015-10-31 19:05:59 -04:00
2015-12-09 01:25:01 +01:00
2015-11-06 17:50:42 -08:00
2016-01-06 15:17:47 +05:30
2015-11-18 16:27:46 -05:00
2015-12-11 16:56:40 +01:00
2015-10-28 12:28:06 +01:00
2015-12-11 11:05:57 -08:00
2016-01-06 16:31:56 -05:00
2015-12-14 18:58:42 -08:00
2015-11-10 09:33:06 -08:00
2015-10-31 16:12:10 -07:00
2015-10-30 01:47:27 -04:00
2016-01-08 21:40:34 -05:00
2016-01-13 19:15:14 -08:00
2016-01-14 16:00:49 -08:00
2016-01-12 20:04:15 -08:00
2015-11-04 18:10:13 -08:00
2016-01-09 22:42:31 -08:00
2016-01-14 16:00:49 -08:00
2015-11-18 16:17:42 -05:00
2015-12-05 17:41:42 -05:00
2015-11-05 09:51:50 -08:00
2015-11-09 15:11:24 -08:00
2015-11-06 17:50:42 -08:00
2015-12-21 17:26:00 +00:00
2015-12-21 13:07:56 -08:00
2015-11-20 11:31:26 -05:00
2015-12-03 11:49:26 -05:00
2015-12-15 16:50:20 -05:00
2015-12-15 23:26:51 -05:00
2015-12-04 10:34:46 +01:00
2015-10-27 08:36:08 +09:00
2015-12-08 12:53:29 +01:00
2015-11-04 11:30:57 -08:00
2015-12-02 23:37:16 -05:00
2015-12-14 10:03:46 +01:00
2015-12-29 10:08:45 +01:00
2015-10-30 10:13:26 +01:00
2015-11-23 09:44:58 +01:00
2015-11-09 15:11:24 -08:00
2015-12-19 11:07:01 +01:00
2015-11-20 15:55:52 -05:00
2015-12-19 11:07:01 +01:00
2015-12-12 10:15:34 -08:00
2015-11-28 19:33:29 -08:00
2016-01-08 19:04:36 +01:00
2015-11-25 17:24:23 +01:00
2016-01-04 09:57:40 +01:00
2016-01-11 19:33:59 -08:00
2016-01-09 08:39:03 -08:00
2015-12-07 09:14:19 -07:00
2015-11-23 10:37:35 -08:00
2015-11-23 10:37:35 -08:00
2015-12-04 12:34:33 -08:00
2015-11-23 09:44:58 +01:00
2015-11-17 15:25:43 -05:00
2015-11-10 12:06:16 +01:00
2016-01-07 14:31:27 -05:00
2015-12-09 16:56:58 +00:00
2016-01-14 16:00:49 -08:00
2016-01-14 16:00:49 -08:00
2015-10-27 18:55:31 -07:00
2015-11-06 17:50:42 -08:00
2015-11-06 17:50:42 -08:00
2015-12-18 14:25:40 -08:00
2015-11-06 17:50:42 -08:00
2015-12-14 19:15:05 -08:00
2015-11-06 17:50:42 -08:00
2015-11-30 15:26:22 -05:00
2015-12-16 15:29:44 +00:00
2016-01-09 03:07:52 -05:00
2015-12-01 15:45:05 -05:00
2015-12-15 16:50:20 -05:00
2016-01-11 23:28:44 -05:00
2015-11-06 19:33:12 +01:00
2015-12-15 23:25:20 -05:00
2015-12-07 23:12:00 -05:00
2015-12-08 22:41:55 -05:00
2015-11-23 21:57:44 -05:00
2015-11-05 19:34:48 -08:00
2015-11-10 17:27:12 -06:00
2015-11-16 09:03:21 +05:30
2015-12-09 09:23:28 -06:00
2015-11-12 11:50:33 -08:00
2015-11-10 17:27:12 -06:00
2015-12-18 11:17:26 +05:30
2015-11-05 19:34:48 -08:00
2015-11-06 17:50:42 -08:00
2015-11-06 17:50:42 -08:00
2015-11-06 17:50:42 -08:00
2015-12-03 14:17:51 -05:00
2015-12-21 13:49:32 +00:00
2015-11-16 10:40:50 -05:00
2015-11-16 10:50:25 -05:00
2015-11-23 09:21:03 +01:00
2016-01-14 16:00:49 -08:00
2016-01-07 14:31:27 -05:00
2015-11-30 15:26:22 -05:00
2016-01-12 20:25:09 -08:00
2015-10-28 04:33:04 +01:00
2015-12-10 23:08:51 +01:00
2015-12-21 03:11:12 +01:00
2015-12-06 21:25:17 -05:00
2016-01-02 00:29:35 +01:00
2016-01-01 02:09:51 +01:00
2015-11-23 09:44:58 +01:00
2015-11-03 11:08:22 -05:00
2015-11-10 13:06:16 +01:00
2015-10-30 11:18:05 +09:00
2015-11-06 17:50:42 -08:00
2015-12-07 17:02:54 -08:00
2015-12-07 17:02:54 -08:00
2015-12-07 17:01:31 -08:00
2015-12-07 17:01:31 -08:00
2016-01-05 19:07:17 +00:00
2015-12-17 22:08:28 -05:00
2015-11-02 14:23:38 -05:00
2016-01-10 22:13:15 -05:00
2015-12-15 09:41:09 +01:00
2016-01-12 18:57:02 -08:00
2015-11-19 16:22:43 +01:00
2015-10-27 19:55:13 -07:00
2015-12-13 19:59:48 -08:00
2015-12-17 11:18:44 +01:00
2016-01-04 16:11:11 -05:00
2015-11-20 16:17:32 -08:00
2016-01-10 17:54:28 -05:00
2016-01-14 16:00:49 -08:00
2015-12-15 23:26:51 -05:00
2016-01-06 11:02:29 +01:00
2016-01-04 10:20:19 -05:00
2015-12-01 14:00:53 -05:00
2015-11-13 21:53:18 -08:00
2015-11-05 14:36:09 -05:00
2015-11-23 15:34:34 -08:00
2016-01-14 16:00:49 -08:00
2015-12-10 22:41:06 -08:00
2015-11-05 19:34:48 -08:00
2015-12-06 12:56:06 +01:00
2016-01-12 20:04:15 -08:00
2015-12-13 19:59:48 -08:00
2015-11-25 15:49:13 +00:00
2015-11-05 19:34:48 -08:00
2015-12-18 17:48:50 -08:00
2015-12-06 20:42:15 -05:00
2015-11-23 09:44:58 +01:00
2015-12-01 14:58:18 -08:00
2015-12-04 08:38:42 -07:00
2015-11-05 11:07:36 +10:00
2015-12-04 10:38:59 -02:00
2015-11-05 19:34:48 -08:00
2015-11-01 17:09:15 -08:00
2015-12-29 17:45:49 -08:00
2015-12-04 10:34:45 +01:00
2016-01-12 18:57:02 -08:00
2015-11-03 20:46:26 +01:00
2015-12-08 11:29:47 -05:00
2015-12-13 19:46:12 -05:00
2015-11-06 17:50:42 -08:00
2015-11-06 17:50:42 -08:00
2015-11-06 17:50:42 -08:00
4a8c7bb59a