kubernetes-operator/docs/usage.md

Usage

Provision pods with NetBird access using side-cars

1. Create a Setup Key in your NetBird console.
2. Create a Secret object in the namespace where you need to provision NetBird access (secret name and field can be anything).

apiVersion: v1
stringData:
  setupkey: EEEEEEEE-EEEE-EEEE-EEEE-EEEEEEEEEEEE
kind: Secret
metadata:
  name: test

1. Create an NBSetupKey object referring to your secret.

apiVersion: netbird.io/v1
kind: NBSetupKey
metadata:
  name: test
spec:
  # Optional, overrides management URL for this setupkey only
  # defaults to https://api.netbird.io
  managementURL: https://netbird.example.com 
  secretKeyRef:
    name: test # Required
    key: setupkey # Required

1. Annotate the pods you need to inject NetBird into with netbird.io/setup-key.

kind: Deployment
...
spec:
...
  template:
    metadata:
      annotations:
        netbird.io/setup-key: test # Must match the name of an NBSetupKey object in the same namespace
...
    spec:
      containers:
...

Since v0.27.0, NetBird supports extra DNS labels, which extends the DNS names that you can link to peers by grouping them and load balancing access using DNS round-robin. To enable this feature, add the following annotation to the pod:

    netbird.io/extra-dns-labels: "label1,label2"

With this setup, all peers with the same extra label would be used in a DNS round-robin fashion.

Provisioning Networks (Ingress Functionality)

Granting controller access to NetBird Management

Important

The NetBird Kubernetes operator generates configurations using NetBird API; editing or deleting these configurations in the NetBird console may cause temporary network disconnection until the operator reconciles the configuration.

1. Create a Service User on your NetBird dashboard (Must be Admin). Doc.
2. Create an access token for the Service User (Must be Admin). Doc.
3. Add access token to your helm values file under netbirdAPI.key.
   1. Alternatively, provision secret in the same namespace as the operator and set the key NB_API_KEY to the access token generated.
   2. Set netbirdAPI.keyFromSecret to the name of the secret created.
4. Set ingress.enabled to true.
   1. Optionally, to provision the network immediately, set ingress.router.enabled to true.
   2. Optionally, to provision 1 network per > The NetBird Kubernetes operator generates configurations using NetBird API; editing or deleting these configurations in the NetBird console may cause temporary network disconnection until the operator reconciles the configuration. namespace, set ingress.namespacedNetworks to true.
5. Run helm install or helm upgrade.

Minimum values.yaml example:

netbirdAPI:
  key: "nbp_XXxxxxxxXXXXxxxxXXXXXxxx"

ingress:
  enabled: true
cluster:
  name: kubernetes

Learn more about the values.yaml options here.

Exposing a Service

Important

Ingress DNS Resolution requires DNS Wildcard Routing to be enabled and at least one DNS Nameserver configured for clients.

Annotation	Description	Default	Valid Values
netbird.io/expose	Expose service using NetBird Network Resource		(null, true)
netbird.io/groups	Comma-separated list of group names to assign to Network Resource	{ClusterName}-{Namespace}-{Service}	Any comma-separated list of strings.
netbird.io/resource-name	Network Resource name	{Namespace}-{Service}	Any valid network resource name, make sure they're unique!
netbird.io/policy	Name(s) of NBPolicy to propagate service ports as destination.		Comma-separated list of names of any NBPolicy resource
netbird.io/policy-ports	Narrow down exposed ports in a policy. Leave empty for all ports.		Comma-separated integer list, integers must be between 0-65535
netbird.io/policy-protocol	Narrow down protocol for use in a policy. Leave empty for all protocols.		(tcp,udp)

Example service:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  replicas: 0
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
        - name: nginx
          image: nginx:latest
          ports:
            - containerPort: 80

---
apiVersion: v1
kind: Service
metadata:
  name: nginx-service
  annotations:
    netbird.io/expose: "true"
    netbird.io/groups: "groupA,groupB"
spec:
  selector:
    app: nginx
  ports:
    - protocol: TCP
      port: 8080
      targetPort: 80
  type: ClusterIP

Notes

• netbird.io/expose will interpret any string as a true value; the only false value is null.
• The operator does not handle duplicate resource names within the same network, it is up to you to ensure resource names are unique within the same network.
• While the NetBird console will allow group names to contain commas, this is not allowed in netbird.io/groups annotation as commas are used as separators.
• If a group already exists on NetBird console with the same name, NetBird Operator will use that group ID instead of creating a new group.
• NetBird Operator will attempt to clean up any resources created, including groups created for resources.
  • If a group is used by resources that the operator cannot clean up, the operator will eventually ignore the group in NetBird.
  • It's recommended that unique groups be used per NetBird Operator installation to remove any possible conflicts.
• The Operator does not validate service annotations on updates, as this may cause unnecessary overhead on any Service update.

Managing Policies

Simply add policies under ingress.policies, for example:

1. Add the following configuration in your values.yaml file.

ingress:
  policies:
    default:
      name: Kubernetes Default Policy # Required, name of policy in NetBird console
      description: Default # Optional
      sourceGroups: # Required, name of groups to assign as source in Policy.
      - All
      ports: # Optional, resources annotated 'netbird.io/policy=default' will append to this.
      - 443
      protocols: # Optional, restricts protocols allowed to resources, defaults to ['tcp', 'udp'].
      - tcp
      - udp
      bidirectional: true # Optional, defaults to true

2. Reference policies in Services using netbird.io/policy=default,otherpolicy,..., this will add relevant ports and destination groups to policies.
3. (Optional) Limit specific ports in exposed service by adding netbird.io/policy-ports=443.
4. (Optional) Limit specific protocol in exposed service by adding netbird.io/policy-protocol=tcp.

Notes

• Each NBPolicy will only create policies in the NetBird console when the information provided is enough to create one. If no services act as a destination or specified services do not conform to the protocol(s) defined, the policy will not be created.
• Each NBPolicy will create one policy in the NetBird console per protocol specified as long as the protocol has destinations; this ensures better-secured policies by separating ports for TCP and UDP.
• Policies currently do not support ICMP protocol, as ICMP is not supported in Kubernetes services, and there are no current plans to support it.
• NetBird currently does not support SCTP protocol.