Skip to content

Pentesting Kubernetes

Useful Links:

Kubernetes Outline

Kubernetes or k8s is a "container" orchestration platform, designed to automate deployment, scaling and management of containerized applications (Docker images and such)

There are two main parts to the k8s architecture:

  1. Master Node
  2. Worker Node

A cluster is made up of a Master node and N Worker nodes.

The job of the Master node is to act as a command and control (c2) server for the cluster and provide a control for all of the worker nodes. The Master node has many different containers running on it, but the main one of interest is the kube-apiserver which as it's name suggests, is the API server that k8s uses to orchestrate the cluster.

All the traffic in the cluster is passed through this container. It also exposes an API that allows communication using a client (kubectl or customized one)

The kube-apiserver is the only container that communicates with the etcd, which is the database of the cluster and contains all the sensitive data (tokens, passwords, etc.)

That bit is important for our upcoming testing as it will mostly be the goal to be able to gain access to this API so that we can do things we're not supposed to be able to do.

insert image here

On the worker nodes, the primary agent is the kubelet. It receives instructions from the kube-apisever and is responsible for executing the instruction such as deploying pods, which are logical groups of one or more containers, or downloading the image for the containers.

There are two main "users" of a k8s cluster:

  • Human users
  • Service accounts

Human users are exactly that, humans that are using the system. Service accounts are accounts that are configured for a particular pod to communicate with the kube-apiserver.

Connecting and Using a k8s Cluster

There are three stages that a user or service account must go through to connect to the cluster in order to receive or carry out instructions:

  • Authentication
  • Authorization
  • Admission Control

This loop is performed whenever an account sends a request to the kube_apiserver, the request could be something as simple as "list all pods" or something a bit more restricted such as creating a new service account.

Authentication

This is usually done with either a certificate, a token or using Basic Authentication (i.e. username & password), there are also ways that it can be configured to be used with things like LDAP, but that's just an abstraction of the authentication process.

Authorization

Once the users token, certificate, or username and password combination has been verified by the cluster k8s then checks that user or service accounts permissions to see if they are allowed to perform the requested action (i.e. download a new image)

Admission Control

These are pieces of software that are able to acccess the content that is being requested by the users or service accounts, there are lots of these "Admission Controllers":

k8s Docs on Admission Controllers

RBAC (Role Based Access Control)

The RBAC uses specific “resources” and “verbs” to grant access:

  • Resources are the information that can be accessed (e.g., information about pods, read secrets, etc.’).
  • Verbs are how the resources can be accessed (e.g., list secrets, get pods, watch, etc.’).

Tools

Attack Methods

  • Listing Secrets

An attacker that gains access to list secrets in the cluster can use the following curl commands to get all secrets in “kube-system” namespace:

curl -v -H “Authorization: Bearer <jwt_token>” https://<master_ip>:<port>/api/v1/namespaces/kube-system/secrets/
  • Access Any Resource or Verb

If you have access to see the RBAC config file then you can check to see the privileges for the cluster:

  • Access Any Resource
  • Pod Creation
  • Create/Update Deployment, Daemonsets, Statefulsets, Replicationcontrollers, Replicasets, Jobs and Cronjobs
  • Privilege to Use Pods/Exec
  • Privilege to Get/Patch Rolebindings
  • Impersonating a Privileged Account

We're now looking at the TypeScript to see if we can find an XSS because the application is using websockets to speak back to the pods, so if we can find an XSS then we can send a proper request to the pods and potentially get RCE or information leakage