Agent Runtime

Kubernetes SRE Helper Agent for Multi-Cluster Toil

An LLM agent with read-only kubectl tools that answers 'which cluster is this workload in?' across dozens of clusters, turning a daily manual hunt across 30+ contexts into a single natural-language query, with results delivered to chat.

What This Builds

Platform teams that run many clusters hit a recurring, tedious task: a workload is misbehaving and you do not remember which of your 30+ clusters it lives in, so you kubectl config use-context your way through them one at a time. This recipe builds a small SRE helper agent that automates exactly that — you ask in plain language, the agent searches the clusters for you and reports back where the workload is and its status.

It is deliberately scoped. Rather than a general “do anything in Kubernetes” agent, it starts with one painful, well-defined task and wraps it in tools the LLM can call. This is the approach in Tanat Lokejaroenlarb’s SRE helper agent, built with LangChainGo to locate workloads across clusters.

The Stack

  • LangChainGo is the Go port of LangChain. It runs the agent loop and tool-calling — a natural fit since most Kubernetes tooling is already Go.
  • Kubernetes / kubectl: the agent’s tools wrap read-only commands (list contexts, get deployments/pods in a namespace, describe a workload) across your configured clusters. Keep the tools strictly read-only so the agent can investigate but never mutate state.
  • Chat / notification surface: results go back to the engineer in Slack, Telegram, or a terminal, so the answer arrives where the question was asked.

Step-by-Step Outline

  1. Enumerate the clusters/contexts the agent is allowed to touch (from kubeconfig) and confirm read-only credentials for each.
  2. Define a small set of tools, each wrapping a read-only kubectl operation: list-contexts, find-workload(name) that scans contexts, get-status(context, namespace, workload).
  3. Build the agent in LangChainGo with those tools and a system prompt that constrains it to investigation, never mutation.
  4. Accept a natural-language query (“where is the checkout deployment and is it healthy?”), let the agent iterate over the find/status tools across clusters.
  5. Return a concise answer — cluster/context, namespace, replica and pod health — to the chat surface that asked.
  6. Expand carefully: add describe-style detail or recent-events tools next, and keep every new tool read-only.

Source