Having read more GRO kernel code and NAPI, I believe the previous design was
overly complex and resulted in poor performance.
- Netstack GRO uses a `time.Timer` to periodically clear GRO'd packets. It's
got... several problems.
1. The timer can be configured via CLI flag with arbitrary granularity, but
(IIUC) `time.Timer` relies on the netpoller, which [has millisecond
granularity].
2. The timer creates new goroutines when it fires.
3. There's a complex atomic value song and dance to setting up the timer,
canceling it when no packets are pending, and resuming it for incoming
packets.
- Linux GRO doesn't quite work this way.
- Typically, [Linux flushes GRO whenever it can] (unless running with high
HZ; Go preempts goroutines every 10ms so this does not apply). IIUC,
receiving packets triggers the scheduling of a softirq that batch reads as
many packets from a device as possible. softirqs are run as kernel threads,
so this is getting scheduled with jiffy-ish granularity.
- There's no special scheduling of a timer to flush GRO.
In Netstack our "interrupt" is that we return from a poll, then we read
multiple packets at once via recvmmsg/readv/XDP. So we've already got a delay
analogous to "trigger a ksoftirq and wait for the thread to get scheduled."
Thus, we should use zero-timeout GRO that does the following:
- When recvmmsg/poll/etc returns a single packet, just pass it directly and
immediately up the stack.
- When multiple packets are returned, coalesce them with GRO and flush them
without waiting. We never have 1000+ HZ situation.
- We can remove all atomics and locking from GRO, as there will be one
`groDispatcher` per dispatcher goroutine and no timer-spawned goroutines to
synchronize with.
**Performance**: The previous GRO implementation yielded a few percentage points
increase in performance. This is markedly better.
The following is from tcp_benchmark. It is running with host GRO/GSO disabled,
as there's nothing to GRO when the host does it for us. The RecvMMsg dispatcher
is used, as the PacketMMap dispatcher does not return multiple packets at once
(RecvMMsg averages 8 per syscall in these benchmarks).
```
│ /tmp/old.log │ /tmp/new.log │
│ Mb/s │ Mb/s vs base │
TCP/role=server/host-gso=false/host-gro=false 1.764k ± 2% 2.139k ± 2% +21.29% (p=0.000 n=20)
```
PiperOrigin-RevId: 622366744
Kevin KrakauerandgVisor bot
Lucas ManningandgVisor bot
Nayana BidariandgVisor bot