Kubernetes

The Kubernetes check performs requests on Kubernetes resources such as Pods to get the desired information.

kubernetes.yaml
apiVersion: canaries.flanksource.com/v1
kind: Canary
metadata:
  name: kube-system-checks
spec:
  schedule: "@every 5m"
  kubernetes:
    - namespace: kube-system
      name: kube-system
      kind: Pod
      # ready: true
      # resource:
      #   labelSelector: k8s-app=kube-dns
      namespaceSelector:
        name: default
      display:
        expr: |
          dyn(results).
          map(i, i.Object).
          filter(i, !k8s.isHealthy(i)).
          map(i, "%s/%s -> %s".format([i.metadata.namespace, i.metadata.name, k8s.getHealth(i).message])).join('\n')
      test:
        expr: dyn(results).all(x, k8s.isHealthy(x))

Field	Description	Scheme
kind*	Kubernetes object kind	string
name*	Name of the check, must be unique within the canary	string
healthy	Fail the check if any resources are unhealthy	boolean
ignore	Ignore the specified resources from the fetched resources. Can be a glob pattern.	[]glob
namespace	Failing checks are placed in this namespace, useful if you have shared namespaces
namespaceSelector	Namespace of the Kubernetes object	ResourceSelector
ready	Fail the check if any resources are not ready	boolean
resource	Queries resources related to specified Kubernetes object	ResourceSelector
description	Description for the check	string
display	Expression to change the formatting of the display	Expression
icon	Icon for overwriting default icon on the dashboard	Icon
labels	Labels for check	map[string]string
metrics	Metrics to export from	[]Metrics
test	Evaluate whether a check is healthy	Expression
transform	Transform data from a check into multiple individual checks	Expression
kubeconfig	Path to a kubeconfig on disk, or a reference to an existing secret	EnvVar

Healthy

Using healthy: true is functionally equivalent to the test expression above:

apiVersion: canaries.flanksource.com/v1
kind: Canary
metadata:
  name: kube-system-checks
spec:
  interval: 30
  kubernetes:
    - namespace: kube-system
      name: kube-system
      kind: Pod
      healthy: true
      resource:
        labelSelector: k8s-app=kube-dns
      namespaceSelector:
        name: kube-system
      display:
        expr: |
          dyn(results).
          map(i, i.Object).
          filter(i, !k8s.isHealthy(i)).
          map(i, "%s/%s -> %s".format([i.metadata.namespace, i.metadata.name, k8s.getHealth(i).message])).join('\n')

See the CEL Kubernetes docs for more details on the k8s.isHealthy and other functions available

Ready

Similar to the healthy flag, there's also a ready flag which is functionally equivalent to having the following test expression

dyn(results).all(x, k8s.isReady(x))

---

Checking for certificate readiness

junit.yaml
apiVersion: canaries.flanksource.com/v1
kind: Canary
metadata:
  name: cert-manager
spec:
  schedule: "@every 15m"
  kubernetes:
    - name: cert-manager-check
      kind: Certificate
      test:
        expr: |
          dyn(results).
          map(i, i.Object).
          filter(i, i.status.conditions[0].status != "True").size() == 0
      display:
        expr: |
          dyn(results).
          map(i, i.Object).
          filter(i, i.status.conditions[0].status != "True").
          map(i, "%s/%s -> %s".format([i.metadata.namespace, i.metadata.name, i.status.conditions[0].message])).join('\n')

Remote clusters

A single canary-checker instance can connect to any number of remote clusters via custom kubeconfig. Either the kubeconfig itself or the path to the kubeconfig can be provided.

kubeconfig from kubernetes secret

remote-cluster.yaml
---
apiVersion: canaries.flanksource.com/v1
kind: Canary
metadata:
  name: pod-access-check
spec:
  schedule: "@every 5m"
  kubernetes:
    - name: pod access on aws cluster
      namespace: default
      description: "deploy httpbin"
      kubeconfig:
        valueFrom:
          secretKeyRef:
            name: aws-kubeconfig
            key: kubeconfig
      kind: Pod
      ready: true
      namespaceSelector:
        name: default

Kubeconfig inline

remote-cluster.yaml
apiVersion: canaries.flanksource.com/v1
kind: Canary
metadata:
  name: pod-access-check
spec:
  schedule: "@every 5m"
  kubernetes:
    - name: pod access on aws cluster
      namespace: default
      kubeconfig:
        value: |
          apiVersion: v1
          clusters:
              - cluster:
                  certificate-authority-data: xxxxx
                  server: https://xxxxx.sk1.eu-west-1.eks.amazonaws.com
                name: arn:aws:eks:eu-west-1:765618022540:cluster/aws-cluster
          contexts:
              - context:
                  cluster: arn:aws:eks:eu-west-1:765618022540:cluster/aws-cluster
                  namespace: mission-control
                  user: arn:aws:eks:eu-west-1:765618022540:cluster/aws-cluster
                name: arn:aws:eks:eu-west-1:765618022540:cluster/aws-cluster
          current-context: arn:aws:eks:eu-west-1:765618022540:cluster/aws-cluster
          kind: Config
          preferences: {}
          users:
              - name: arn:aws:eks:eu-west-1:765618022540:cluster/aws-cluster
                user:
                  exec:
                      ....
      kind: Pod
      ready: true
      namespaceSelector:
        name: default

Kubeconfig from local filesystem

remote-cluster.yaml
---
apiVersion: canaries.flanksource.com/v1
kind: Canary
metadata:
  name: pod-access-check
spec:
  schedule: "@every 5m"
  kubernetes:
    - name: pod access on aws cluster
      namespace: default
      kubeconfig:
        value: /root/.kube/aws-kubeconfig
      kind: Pod
      ready: true
      namespaceSelector:
        name: default