Tetragon on Kubernetes: in-kernel runtime enforcement

Most runtime security tools tell you what just happened. Tetragon can make that decision at the hook point in the kernel, while the syscall is still in flight. That changes the conversation from “how quickly can we react?” to “can we stop this before userspace ever sees it?”. The interesting part is the caveat: killing a process is not always the same thing as preventing the operation, and Tetragon’s own docs are admirably blunt about that.

If you already have detection and alerting, that distinction is the whole reason to care. You are not replacing one category of tool with another. You are deciding whether a particular class of behaviour should be observed, triaged later, or blocked synchronously where it happens.

What Tetragon is, and what it is not

Tetragon is a runtime security observability and enforcement project in the Cilium ecosystem. More precisely, it is part of the CNCF-graduated Cilium project, rather than a graduated CNCF project in its own right. It uses eBPF programs to observe process, file, network, and kernel activity, then apply filtering and actions in-kernel.

Two details are worth clearing up early because people keep tripping over them:

• Tetragon does not require Cilium as your CNI. The project documents Kubernetes, container, and systemd-based installs, and Isovalent’s own FAQ on the point is unambiguous: yes, you can run Tetragon without Cilium.
• Kernel support matters more than the headline architecture does. The installation FAQ puts the floor at Linux 4.19 with BTF support, because Tetragon loads its programs using CO-RE. Some features raise that bar: socket tracking needs 5.3 or newer, and decoding Unix socket paths needs 5.11.

That second point is the first production filter. Before you spend time writing policies, check whether the fleet kernel can actually support the behaviour you plan to rely on.

The real difference: detection-first versus enforcement in-kernel

Detection-first tooling has a perfectly respectable model:

1. observe an event
2. ship it to userspace
3. evaluate rules
4. alert or trigger a response

That is good enough for a lot of work. If the question is “did a shell spawn in this container?” or “which process opened that connection?”, userspace detection is often exactly the right trade.

Tetragon is interesting when that loop is too late. Its selectors are evaluated in-kernel, which means the same BPF program that observes the event can also decide whether to post it, kill the process, or override the return value. There is no eBPF magic in that; you have simply removed the response-latency gap between observation and action.

The project also claims a fairly modest overhead profile when you stay in that model. Isovalent’s published worst-case benchmark, compiling a Linux kernel with Tetragon watching, put exec tracking at 1.68% overhead, rising to 2.46% with JSON export enabled. Those are their numbers, not ours, but they line up with the architectural claim: the less you bounce high-frequency events out to userspace, the less work the system does.

If you have already read our post on Kubernetes networking with Cilium: eBPF, kube-proxy replacement, and Hubble, the same broad instinct applies here. Moving policy closer to the kernel is not automatically better. It is better when you want tighter control over the hot path and you are willing to own the sharp edges that come with it.

Install Tetragon and get useful observability before you enforce anything

The nice thing about Tetragon is that it is useful before you write a single blocking policy. A basic Helm install gives you process visibility straight away:

helm repo add cilium https://helm.cilium.io
helm repo update
helm install tetragon cilium/tetragon -n kube-system
kubectl rollout status -n kube-system ds/tetragon -w

Once the DaemonSet is up, you can watch compact events from the agent. The --pods filter takes a pod name; xwing is the pod from the Star Wars demo app the Tetragon quickstart uses, so substitute whatever you have running:

kubectl exec -ti -n kube-system ds/tetragon -c tetragon -- \
  tetra getevents -o compact --pods xwing

That is enough to start seeing the shape of a workload:

• process_exec and process_exit events for what actually ran
• network events such as tcp_connect
• Kubernetes metadata on the pod, namespace, container image, and workload

This is the point where Tetragon still feels familiar. You are using it like an observability tool with unusually good kernel context.

It also exports data in the boring ways you want security plumbing to export data: JSON logs, a gRPC endpoint, and Prometheus metrics. On Kubernetes, metrics are enabled by default through the tetragon service on port 2112, with the operator on 2113. The metrics reference includes counters such as tetragon_events_total, tetragon_policy_events_total, and tetragon_bpf_missed_events_total; that last one is the first thing to graph, because missed events are the quiet failure mode of any in-kernel pipeline.

TracingPolicy is where the model clicks

Tetragon’s policy resources are TracingPolicy and TracingPolicyNamespaced, both under apiVersion: cilium.io/v1alpha1. In practice, the namespaced form is the safer place to start because cluster-wide enforcement is exactly how people accidentally learn too much about kernel hooks in one afternoon.

A policy is built from three pieces:

• hook points such as kprobes, tracepoints, uprobes, lsmhooks, and, more recently, fentries
• selectors such as matchArgs, matchBinaries, matchParentBinaries, matchNamespaces, and matchCapabilities
• actions such as Post, Sigkill, Signal, Override, NoPost, GetUrl, DnsLookup, and TrackSock

The important architectural detail is that the filtering happens in-kernel. It keeps overhead lower, because irrelevant events never have to leave the kernel, and it is what makes enforcement meaningful in the first place. A userspace engine can notice something quickly. It cannot make the kernel pretend the syscall never happened.

There is one subtlety buried in the hook choice, though. kprobes are flexible, but kernel function names are not a stable ABI, and Tetragon’s own docs warn that a kprobe policy may not port across kernel versions. If you can express the policy at a more stable hook, such as an LSM hook or a tracepoint, you generally should.

A minimal walkthrough: observe first, then enforce

The official quickstart examples are good enough that you should use them rather than inventing your own. Start with something observational and narrow.

Example 1: watch egress leaving the cluster

Tetragon ships an example policy for tcp_connect that watches connections outside the pod and service CIDRs. The quickstart has both a cluster-wide monitoring version and a namespaced _enforce version; this is the namespaced shape with the action left off, so it only posts events. The interesting part is the NotDAddr selector on the socket argument:

apiVersion: cilium.io/v1alpha1
kind: TracingPolicyNamespaced
metadata:
  name: monitor-network-activity-outside-cluster-cidr-range
spec:
  kprobes:
    - call: 'tcp_connect'
      syscall: false
      args:
        - index: 0
          type: 'sock'
      selectors:
        - matchArgs:
            - index: 0
              operator: 'NotDAddr'
              values:
                - 127.0.0.1
                - ${PODCIDR}
                - ${SERVICECIDR}

The ${PODCIDR} and ${SERVICECIDR} placeholders are meant to be filled in with envsubst before you apply it, which also forces you to go and find out what your cluster’s CIDRs actually are:

export PODCIDR=$(kubectl get nodes -o jsonpath='{.items[*].spec.podCIDR}')
export SERVICECIDR=10.96.0.0/12 # whatever your cluster was built with
envsubst < network_egress.yaml | kubectl apply -n default -f -

With that loaded, a curl to somewhere outside the cluster shows up as a 🔌 connect line in tetra getevents, and internal traffic is filtered out in-kernel and never appears. Adding matchActions: [{ action: Sigkill }] to the selector turns it into the quickstart’s enforce variant, and that is exactly the jump from “observe outbound activity” to “do not let this process talk there at all”.

In practice I would load that enforce variant in monitor mode first, not because the docs tell me to be cautious but because a security policy that kills the wrong thing inside a shared cluster tends to make itself memorable.

Example 2: block reads of sensitive files

The file-monitoring quickstart is where the enforcement caveat becomes concrete. The official example hooks security_file_permission, security_mmap_file, and security_path_truncate across a long list of paths; this is the read branch of the first hook, trimmed to /etc/shadow and /root/.ssh with mode 4 (MAY_READ):

apiVersion: cilium.io/v1alpha1
kind: TracingPolicyNamespaced
metadata:
  name: file-monitoring-filtered
spec:
  kprobes:
    - call: 'security_file_permission'
      syscall: false
      return: true
      args:
        - index: 0
          type: 'file'
        - index: 1
          type: 'int'
      returnArg:
        index: 0
        type: 'int'
      selectors:
        - matchArgs:
            - index: 0
              operator: 'Prefix'
              values:
                - '/root/.ssh'
                - '/etc/shadow'
            - index: 1
              operator: 'Equal'
              values:
                - '4'
          matchActions:
            - action: Sigkill

One small trap if you trim the official file yourself: return: true needs a matching returnArg, or the agent rejects the policy at load time with ReturnArg not specified with Return=true. Keep both or drop both.

Apply it to the namespace your test pod lives in, then try reading one of those files:

kubectl apply -n default -f file-monitoring-filtered.yaml
kubectl exec -ti xwing -- bash -c 'cat /etc/shadow'
# command terminated with exit code 137

In tetra getevents you get the matching 💥 exit … SIGKILL line for the cat. That is a much tighter loop than “shell ran, alert fired, somebody looked later”.

Start in monitor mode or you will eventually shoot the wrong thing

Tetragon gives you three policy modes:

• monitoring
• enforcement
• monitor_only for policies that have no enforcement actions at all

Those modes can be set in the policy, at load time, or at runtime, with runtime taking precedence. The safest first pass is explicit monitor mode:

spec:
  options:
    - name: 'policy-mode'
      value: 'monitor'

Or from the CLI:

tetra tracingpolicy add --mode monitor policy.yaml

And when you are ready to flip a loaded policy live:

tetra tp set-mode --namespace default file-monitoring-filtered enforce

That workflow matters because the most dangerous Tetragon policy is rarely an obviously broken one. It is a broad one that is technically valid.

A cluster-wide TracingPolicy with Sigkill and weak selectors can hit every matching process on every node, including host processes. The namespaced CRD, podSelector, containerSelector, and binary-based selectors are the difference between a precise control and a spectacularly bad afternoon.

The caveat that earns Tetragon trust: Sigkill is not the same thing as prevention

Tetragon’s enforcement docs say that sending SIGKILL does not always stop the operation being performed by the process. Their own example is a write() system call: the process may be terminated synchronously and the data may still have reached the file.

That is why the docs recommend combining Signal with Override when you need a true block rather than a hard stop after the fact. Override changes the function’s return value so the operation never completes normally.

There are catches:

• Override needs CONFIG_BPF_KPROBE_OVERRIDE
• it only works on functions that support error injection, which in practice means system calls and security check functions
• overriding uprobes can crash the target process if you are careless
• some hooks are simply a better fit for prevention than others

This is where runtime enforcement stops being a nice demo and becomes real platform engineering. You are no longer asking whether the tool can send a signal. You are asking whether the hook, action, kernel configuration, and failure mode together deliver the security guarantee you think you bought.

Where Tetragon fits next to Falco and KubeArmor

You do not need a fake rivalry here. These tools solve adjacent problems.

Falco and similar detection-first systems are strong when you want mature rules, rich userspace correlation, and broad runtime visibility without making every policy an enforcement decision. They answer “what happened?” very well.

Tetragon is stronger when the decision itself belongs in-kernel and per-process. It expects more policy design from you, but it gives you a sharper instrument.

KubeArmor sits nearby again, with enforcement built around LSM-backed controls, and is a CNCF project in its own right. If your mental model is “I want runtime policy on what a workload may do”, you will end up comparing these two sooner or later.

The honest framing is simple:

• if you mainly need detection, start with the tool whose detection model and rules you trust
• if you need synchronous prevention at the hook point, Tetragon starts to look like the only sensible option
• if you cannot explain the difference between killing a process and preventing the operation, you are not ready to turn on enforcement yet

Takeaway

Tetragon is worth your time when “we saw it happen” is no longer the bar. Its model is genuinely different: observe, filter, and, when needed, act in-kernel before userspace gets a vote.

The part to remember is the unfashionable one. Enforcement is only as good as the hook, selector scope, and action semantics behind it. Start with namespaced policies, run them in monitor mode first, and treat Sigkill as a blunt instrument unless you have confirmed that Override gives you the prevention guarantee you actually want.