By default in runsc, we have both IPv4 and IPv6 addresses enabled on all the
interfaces. However, in runc this is based on the sysctl
net.ipv6.conf.all.disable_ipv6. This CL will make runsc behave similar to runc.
- If net.ipv6.conf.all.disable_ipv6 is > 0, then only IPv4 addresses will be
enabled on the interfaces including loopback when network mode is "sandbox".
PiperOrigin-RevId: 737656607
After 19249c0724f2 ("net: make net.core.{r,w}mem_{default,max} namespaced")
/proc/sys/net/core/rmem_default is present in non-initial netns as well.
/proc/sys/net/core/dev_weight has existed from the beginning of Linux git repo
in 1da177e4c3f4 ("Linux-2.6.12-rc2"). So it should be safe to use this file.
Demo:
```
$ sudo ip netns add foo
$ sudo ip netns exec foo stat /proc/sys/net/core/dev_weight
stat: cannot statx '/proc/sys/net/core/dev_weight': No such file or directory
$ sudo ip netns delete foo
```
Fixes#10704
PiperOrigin-RevId: 658087447
runsc's NICs will just have an fdbased LinkEndpoint to write to unless the user
asks for packet logging. Before, a write from the NIC to its endpoint would go
something like:
- sniffer.endpoint.WritePackets
- packetsocket.endpoint.WritePackets
- nested.Endpoint.WritePackets
- fdbased.endpoint.WritePackets
In addition to the extra work done in some of these steps, and the slowness of
interface calls in a hot path, directly calling into an fdbased.endpoint opens
up additional optimizations (see other CLs in this chain). For example, if the
qdisc knows it's talking to an fdbased.endpoint, it can pass in persistent
structs (iovecs, mmsghdrs) rather than re-allocating them in every call to
WritePackets.
PiperOrigin-RevId: 615499448
The existing redirect mode uses XDP to maximize performance but is not suitably
secure: packets are copied directly from the driver into a userspace buffer
(UMEM). But because the UMEM is shared among all processes with a socket open
on a particular NIC queue, sandboxes using redirect mode all map the same UMEM
and thus can read each other's packets.
Tunnel mode instead installs an eBPF program that copies packets from the
host's NIC driver into a per-sandbox NIC driver. This incurs an additional
copy, but there is no longer memory shared between sandboxes.
Benchmarking tunnel mode with redis-benchmark shows a performance gain over
standard Docker networking of 20%. Redirect mode showed a 30% improvement.
PiperOrigin-RevId: 595746162
A single flag now controls XDP. More modes will be added as we test XDP, and we
don't want to proliferate confusing flags.
PiperOrigin-RevId: 595502921
In combination with `xdp_loader redirect`, this allows `runsc` to receive
packets directly from any NIC. It is intended to be used with a machine's NIC
to avoid the Linux networking stack entirely.
PiperOrigin-RevId: 591090544
This feature is flag-gated because it can increase runsc startup time by
spawning several child processes.
Testing is done via shell script rather than the usually-preferred Go Docker
API because `docker network create` installs a bunch of iptables rules, whereas
the analagous API calls do not.
PiperOrigin-RevId: 582848436
... instead of depending on a flag.
In most cases the NAT table is in an all-ACCEPT default state:
```
root@5a7ce3a6c623:/# iptables -t nat -S
-P PREROUTING ACCEPT
-P INPUT ACCEPT
-P OUTPUT ACCEPT
-P POSTROUTING ACCEPT
```
In such cases we leave iptables off in the sandbox as a performance
optimization.
But when the table contains anything else, we now install those rules inside
the sandbox.
Turns out packetdrill installes a rule, so support for that rules is also added
in this CL.
PiperOrigin-RevId: 580051079
This just swaps out the fdbased endpoint for an XDP one. Some related
changes in here:
- Moved the AF_XDP BPF program into its own Go package so it can be
shared.
- Added a flag to tcp_benchmark to disable user namespaces. This helps
when running as root, which is required to install BPF programs.
In practice, we found that createSocket() could cost up to 40ms,
which is an unacceptable overhead in FaaS(Function as a Service)
scenario. The root cause is the bind() syscall runs into the slow
path in kernel and invokes synchronize_net(), which is an expensive
call.
From kernel commit[1] we learned that we could bypass this slow
path by adjusting the socket() syscall before we call bind(). By
giving the empty protocal number, the packet_create() will not
call __register_prot_hook(), in which the kernel turn on the flag
packet_sock.running. So when it comes to packet_do_bind(), it
won't call synchronize_net() for there is no prot_hook to unregister.
Worst cost for bind() tested on Intel Xeon E5-2682 with Linux
kernel 5.4 over 100 times:
* before: 39633.274 us
* after: 25.098 us
Reference:
[1] https://github.com/torvalds/linux/commit/902fefb82ef72a50c78cb4a20cc954b037a98d1c
Signed-off-by: Tianyu Zhou <albert.zty@antgroup.com>
Also enables GRO in syscall tests. GRO should be totally transparent and should
not affect behavior. This ensures it's tested and fixes the bug it uncovered.
GRO would not immediately flush the final ACK in the SYN-SYN/ACK-ACK handshake.
This could lead to a situation where
- Client A calls connect(), which returns once the final ACK of the handshake
is sent and A reaches state ESTABLISHED
- The ACK gets GRO'd, delaying it from reaching the server
- Client B calls non-blocking connect()
- Client B's ACK gets GRO'd as well
- Client B is marked as ESTABLISHED
- The server, with accept queue size 1, is only going to accept one connection,
but two clients are ESTABLISHED.
It now immediately flushes packets with no payload, as they are important to
TCP connection state and management.
PiperOrigin-RevId: 516870932
Nayana BidariandgVisor bot
Kevin KrakauerandgVisor bot
Ayush RanjanandgVisor bot
Anqi Shen
Lucas ManningandgVisor bot
Tianyu Zhou