Eric Dumazet 46d3ceabd8
tcp: TCP Small Queues ...
This introduce TSQ (TCP Small Queues)
TSQ goal is to reduce number of TCP packets in xmit queues (qdisc &
device queues), to reduce RTT and cwnd bias, part of the bufferbloat
problem.
sk->sk_wmem_alloc not allowed to grow above a given limit,
allowing no more than ~128KB [1] per tcp socket in qdisc/dev layers at a
given time.
TSO packets are sized/capped to half the limit, so that we have two
TSO packets in flight, allowing better bandwidth use.
As a side effect, setting the limit to 40000 automatically reduces the
standard gso max limit (65536) to 40000/2 : It can help to reduce
latencies of high prio packets, having smaller TSO packets.
This means we divert sock_wfree() to a tcp_wfree() handler, to
queue/send following frames when skb_orphan() [2] is called for the
already queued skbs.
Results on my dev machines (tg3/ixgbe nics) are really impressive,
using standard pfifo_fast, and with or without TSO/GSO.
Without reduction of nominal bandwidth, we have reduction of buffering
per bulk sender :
< 1ms on Gbit (instead of 50ms with TSO)
< 8ms on 100Mbit (instead of 132 ms)
I no longer have 4 MBytes backlogged in qdisc by a single netperf
session, and both side socket autotuning no longer use 4 Mbytes.
As skb destructor cannot restart xmit itself ( as qdisc lock might be
taken at this point ), we delegate the work to a tasklet. We use one
tasklest per cpu for performance reasons.
If tasklet finds a socket owned by the user, it sets TSQ_OWNED flag.
This flag is tested in a new protocol method called from release_sock(),
to eventually send new segments.
[1] New /proc/sys/net/ipv4/tcp_limit_output_bytes tunable
[2] skb_orphan() is usually called at TX completion time,
but some drivers call it in their start_xmit() handler.
These drivers should at least use BQL, or else a single TCP
session can still fill the whole NIC TX ring, since TSQ will
have no effect.
Signed-off-by: Eric Dumazet <edumazet@google.com >
Cc: Dave Taht <dave.taht@bufferbloat.net >
Cc: Tom Herbert <therbert@google.com >
Cc: Matt Mathis <mattmathis@google.com >
Cc: Yuchung Cheng <ycheng@google.com >
Cc: Nandita Dukkipati <nanditad@google.com >
Signed-off-by: David S. Miller <davem@davemloft.net >
2012-07-11 18:12:59 -07:00
2012-05-25 09:31:59 -07:00
2012-06-27 15:23:16 -04:00
2012-06-19 21:40:14 +02:00
2012-05-30 11:17:19 -07:00
2012-05-07 15:39:35 -07:00
2012-05-20 10:06:08 -03:00
2012-04-20 09:22:02 -07:00
2012-05-12 13:32:17 +08:00
2012-04-23 14:23:32 +03:00
2012-05-24 10:34:29 -07:00
2012-05-14 13:34:31 -07:00
2012-05-10 22:39:43 -07:00
2012-05-29 11:53:11 -07:00
2012-07-07 16:23:06 -07:00
2012-06-20 14:41:57 -07:00
2012-06-01 16:55:42 -07:00
2012-07-04 19:51:50 +02:00
2012-06-07 14:58:38 +02:00
2012-05-09 12:53:47 +02:00
2012-05-15 17:30:30 -04:00
2012-05-31 18:18:11 -07:00
2012-05-12 19:57:34 -07:00
2012-05-23 14:15:09 -07:00
2012-05-05 20:28:11 -07:00
2012-05-22 13:54:18 +10:00
2012-05-14 18:03:04 +01:00
2012-05-29 16:22:33 -07:00
2012-06-28 17:37:00 -07:00
2012-06-06 15:20:22 -04:00
2012-05-31 20:29:58 -04:00
2012-06-28 17:37:00 -07:00
2012-04-20 14:56:36 -07:00
2012-05-29 16:22:29 -07:00
2012-05-15 20:14:35 -04:00
2012-05-19 12:17:35 -06:00
2012-06-15 08:46:23 +02:00
2012-05-29 16:22:22 -07:00
2012-05-29 16:22:20 -07:00
2012-07-04 13:07:05 +02:00
2012-05-23 17:42:39 -07:00
2012-05-08 14:13:25 -07:00
2012-05-15 12:02:39 -07:00
2012-05-11 17:13:00 +02:00
2012-05-02 09:30:32 +01:00
2012-05-25 01:44:51 +02:00
2012-06-03 20:05:57 -07:00
2012-05-31 17:49:32 -07:00
2012-06-18 09:47:00 -04:00
2012-05-31 17:49:29 -07:00
2012-05-31 17:49:28 -07:00
2012-05-07 15:02:26 +02:00
2012-05-10 19:54:35 -07:00
2012-05-25 09:18:59 -07:00
2012-05-02 14:15:27 -05:00
2012-05-25 12:46:23 +05:30
2012-05-21 15:09:37 +02:00
2012-05-31 17:49:31 -07:00
2012-05-07 14:35:00 +02:00
2012-05-09 15:10:45 +02:00
2012-05-09 15:10:45 +02:00
2012-05-10 12:00:56 +02:00
2012-05-01 08:58:06 -04:00
2012-05-28 19:13:50 -03:00
2012-04-30 15:30:18 -07:00
2012-06-01 12:12:01 -04:00
2012-07-10 18:06:44 -07:00
2012-06-27 15:42:24 -07:00
2012-05-31 17:49:32 -07:00
2012-05-05 21:50:41 +02:00
2012-05-29 23:28:33 -04:00
2012-05-04 16:25:29 -07:00
2012-06-01 16:57:51 -07:00
2012-05-21 20:27:36 -07:00
2012-05-24 12:57:47 -07:00
2012-05-14 09:47:23 -03:00
2012-05-15 11:34:08 -04:00
2012-06-03 14:50:19 -07:00
2012-04-18 13:46:42 -07:00
2012-05-30 21:04:53 -04:00
2012-05-10 15:55:43 -04:00
2012-05-16 20:00:27 -04:00
2012-04-25 12:25:05 +02:00
2012-05-29 22:33:56 -04:00
2012-05-15 08:22:04 +02:00
2012-05-21 15:09:33 +02:00
2012-04-27 10:46:45 +08:00
2012-05-18 16:48:35 -06:00
2012-05-22 11:32:31 +02:00
2012-04-30 10:39:17 +02:00
2012-05-29 16:22:21 -07:00
2012-05-25 12:48:26 -04:00
2012-05-22 16:02:13 -07:00
2012-05-12 14:28:18 +02:00
2012-06-04 16:49:43 +02:00
2012-05-12 14:28:15 +02:00
2012-05-12 14:28:14 +02:00
2012-05-30 10:55:34 +02:00
2012-05-17 19:06:13 -04:00
2012-06-21 16:43:28 +02:00
2012-05-22 17:13:11 -04:00
2012-06-19 21:39:50 +02:00
2012-04-28 22:21:51 -04:00
2012-05-01 09:30:55 -04:00
2012-06-26 21:09:39 -07:00
2012-06-30 01:08:00 -07:00
2012-05-17 15:49:51 -04:00
2012-06-12 15:25:46 -07:00
2012-05-30 14:02:25 +02:00
2012-05-23 22:11:12 -04:00
2012-05-22 18:08:08 +02:00
2012-05-07 10:58:57 -06:00
2012-05-08 19:38:31 +02:00
2012-05-31 17:49:31 -07:00
2012-07-11 02:39:24 -07:00
2012-05-23 13:49:23 -04:00
2012-06-30 15:56:40 -07:00
2012-05-19 12:41:56 -06:00
2012-05-27 08:12:42 -04:00
2012-04-29 18:21:10 -04:00
2012-05-15 23:34:37 +02:00
2012-05-29 16:22:32 -07:00
2012-05-31 17:49:30 -07:00
2012-05-31 17:49:32 -07:00
2012-05-29 16:22:21 -07:00
2012-05-31 17:49:26 -07:00
2012-05-31 17:49:30 -07:00
2012-05-23 22:11:31 -04:00
2012-05-11 10:56:56 +01:00
2012-04-19 19:17:30 -07:00
2012-05-31 17:49:28 -07:00
2012-06-18 15:55:21 -07:00
2012-06-04 11:43:00 -04:00
2012-05-16 16:08:28 -03:00
2012-05-17 21:06:01 -03:00
2012-05-01 09:30:55 -04:00
2012-05-29 16:22:29 -07:00
2012-05-29 16:22:32 -07:00
2012-05-29 16:22:31 -07:00
2012-05-29 23:28:41 -04:00
2012-05-07 15:32:23 -04:00
2012-05-15 08:08:31 -07:00
2012-05-07 22:58:09 -04:00
2012-07-01 03:34:50 -07:00
2012-05-09 13:58:06 -07:00
2012-05-29 16:22:28 -07:00
2012-05-29 16:22:20 -07:00
2012-05-22 15:20:28 -04:00
2012-07-01 03:34:50 -07:00
2012-05-16 15:24:36 -04:00
2012-05-16 15:23:46 -04:00
2012-05-29 16:22:28 -07:00
2012-06-20 14:39:35 -07:00
2012-06-01 10:34:35 -07:00
2012-05-29 16:22:20 -07:00
2012-05-29 16:22:26 -07:00
2012-05-01 12:54:54 +02:00
2012-06-08 14:58:13 +09:30
2012-05-31 17:49:27 -07:00
2012-05-08 16:33:56 -07:00
2012-05-29 22:33:55 -04:00
2012-07-05 03:06:44 -07:00
2012-04-24 00:16:24 -04:00
2012-06-16 15:40:02 +02:00
2012-06-16 15:40:02 +02:00
2012-06-22 02:49:52 +02:00
2012-06-29 16:46:06 -07:00
2012-06-04 21:34:30 +02:00
2012-05-26 14:17:30 -04:00
2012-06-18 08:45:16 -04:00
2012-05-24 16:15:48 -04:00
2012-05-09 15:17:43 -04:00
2012-06-05 18:38:47 -04:00
2012-06-26 09:32:28 +02:00
2012-06-26 21:06:11 -07:00
2012-05-18 16:48:36 -06:00
2012-05-12 14:28:17 +02:00
2012-05-07 22:58:09 -04:00
2012-05-21 14:31:48 +01:00
2012-04-18 15:07:53 -07:00
2012-05-02 01:14:40 +02:00
2012-05-29 16:22:24 -07:00
2012-05-21 15:09:33 +02:00
2012-05-29 16:22:18 -07:00
2012-06-20 14:39:36 -07:00
2012-05-09 15:57:01 +02:00
2012-06-13 13:11:39 -07:00
2012-05-14 14:15:32 -07:00
2012-06-06 17:08:00 +02:00
2012-07-09 00:10:56 -07:00
2012-05-15 14:59:28 -07:00
2012-04-29 13:12:42 -07:00
2012-07-04 13:07:05 +02:00
2012-05-12 15:53:42 -04:00
2012-05-18 20:45:26 +02:00
2012-05-01 21:28:38 +02:00
2012-05-01 21:25:49 +02:00
2012-05-01 21:28:38 +02:00
2012-05-04 21:15:52 -07:00
2012-06-07 14:43:55 -07:00
2012-05-07 17:03:27 -07:00
2012-05-15 14:59:28 -07:00
2012-06-13 16:52:40 -07:00
2012-06-11 12:26:15 +08:00
2012-05-03 03:29:34 -07:00
2012-06-05 10:46:40 -07:00
2012-05-17 15:18:37 +02:00
2012-04-24 20:54:49 -07:00
2012-05-11 10:14:21 -07:00
2012-06-06 20:43:28 -07:00
2012-06-06 20:43:28 -07:00
2012-05-14 17:40:05 +01:00
2012-05-29 16:22:27 -07:00
2012-04-23 21:17:51 -04:00
2012-05-31 17:49:31 -07:00
2012-05-31 17:49:31 -07:00
2012-05-29 16:22:21 -07:00
2012-05-12 15:15:20 -04:00
2012-05-29 16:22:33 -07:00
2012-07-10 22:40:13 -07:00
2012-06-08 14:59:29 -07:00
2012-06-01 10:37:01 -04:00
2012-05-04 15:13:54 -07:00
2012-05-23 17:12:06 -07:00
2012-04-18 15:57:31 -07:00
2012-05-17 19:02:14 -04:00
2012-05-22 19:07:55 +09:00
2012-05-15 14:59:29 -07:00
2012-06-01 12:58:52 -04:00
2012-06-15 15:30:15 -07:00
2012-05-31 17:49:26 -07:00
2012-05-08 12:35:06 +02:00
2012-06-04 11:27:40 -04:00
2012-06-13 21:16:42 +02:00
2012-04-30 10:48:25 -07:00
2012-05-15 14:08:35 -07:00
2012-05-14 18:53:19 -04:00
2012-04-20 23:27:08 +02:00
2012-05-01 21:25:49 +02:00
2012-06-04 12:28:45 -07:00
2012-05-15 11:33:58 -04:00
2012-06-15 21:48:14 -07:00
2012-05-31 17:49:32 -07:00
2012-06-22 23:02:22 -07:00
2012-05-14 12:19:56 -07:00
2012-05-23 22:09:21 -04:00
2012-07-11 18:12:59 -07:00
2012-06-01 12:58:50 -04:00
2012-05-09 14:19:13 -07:00
2012-05-21 16:16:58 -07:00
2012-05-22 18:27:32 -07:00
2012-05-23 22:11:31 -04:00
2012-05-10 11:24:03 -07:00
2012-06-02 15:21:43 -07:00
2012-05-30 21:04:50 -04:00
2012-04-26 02:01:39 -07:00
2012-05-21 09:00:03 -07:00
2012-05-03 03:28:39 -07:00
2012-05-09 13:57:30 -07:00
2012-05-20 08:27:44 -03:00
2012-06-21 09:32:16 +01:00
2012-05-03 09:30:55 +01:00
2012-06-11 15:47:21 -03:00
2012-05-22 12:16:16 +09:30
2012-04-25 21:26:33 -07:00
2012-04-26 12:34:58 -07:00
2012-04-24 16:14:14 -07:00
2012-05-30 07:55:31 +02:00
2012-05-06 13:43:41 +08:00
46d3ceabd8