HTTP

This check performs queries on HTTP to monitor their activity.

apiVersion: canaries.flanksource.com/v1
kind: Canary
metadata:
  name: http-single
  labels:
    canary: http
spec:
  schedule: "@every 5m"
  http:
    - name: http pass response 200 status code
      url: https://httpbin.flanksource.com/status/200
      thresholdMillis: 3000
      responseCodes: [200]
      maxSSLExpiry: 7

Field	Description	Scheme
`name*`	Name of the check, must be unique within the canary	`string`
`url*`	HTTP URL	`string`
`body`	Request Body Contents	`string`
`connection`	Connection name e.g. connection://http/google	`string`
`crawl`	Crawl configuration for following links	Crawl
`endpoint`	Deprecated: Use url instead	`string`
`envVars`	Setup environment variables that are accessible while templating	[]EnvVar
`headers`	Header fields	[]EnvVar
`maxSSLExpiry`	Maximum number of days until the SSL Certificate expires Equivalent to `test.expr: sslAge < Age("7d")`	`integer`
`method`	HTTP Request method. Defaults to `GET`	`string`
`ntlm`	When set to true will authenticate using NTLM v1 protocol	`boolean`
`ntlmv2`	When set to true will authenticate using NTLM v2 protocol	`boolean`
`oauth2`	OAuth2 configuration	OAuth2Config
`password`	Password for authentication	EnvVar
`password`		EnvVar
`responseCodes`	Expected HTTP response codes Equivalent to `test.expr: code in [200, 201]`	[]int
`responseContent`	Expected response body content Equivalent to `test.expr: content.contains("value")`	`string`
`responseJSONContent`	Deprecated: Use expr and jsonpath function	JSONCheck
`templateBody`	If true body is templated	`boolean`
`thresholdMillis`	Maximum duration in milliseconds for the HTTP request. It will fail the check if it takes longer. Defaults to `5000`	`integer`
`tlsConfig`	TLS config	TLSConfig
`username`	Username for authentication	EnvVar
`username`		EnvVar
`description`	Description for the check	`string`
`display`	Expression to change the formatting of the display	`Expression`
`icon`	Icon for overwriting default icon on the dashboard	Icon
`labels`	Labels for check	`[map[string]string]`
`markFailOnEmpty`	If a transformation or datasource returns empty results, the check should fail	`boolean`
`metrics`	Metrics to export from	`[]Metrics`
`test`	Evaluate whether a check is healthy	`Expression`
`transform`	Transform data from a check into multiple individual checks	`Expression`

TLS Config

Field	Description	Scheme
`ca`	PEM encoded certificate of the CA to verify the server certificate	EnvVar
`cert`	PEM encoded client certificate	EnvVar
`handshakeTimeout`	Timeout for SSL handshake (defaults to 10 seconds)	`integer`
`insecureSkipVerify`	Controls whether a client verifies the server's certificate chain and host name	`boolean`
`key`	PEM encoded client private key	EnvVar

Verifying against an external CA

apiVersion: canaries.flanksource.com/v1
kind: Canary
metadata:
  name: http-check
spec:
  schedule: "@every 30s"
  http:
    - name: http pass response 200 status code
      url: https://httpbin.flanksource.com/status/200
      tlsConfig:
        ca:
          valueFrom:
            secretKeyRef:
              name: ca-cert
              key: ca.pem

OAuth

Field	Description	Scheme
`params`	Parameters for OAuth	`[map[string]string]`
`scope`	Scope for OAuth token request	`[]string`
`tokenURL`	URL for OAuth Token	`string`

Crawl Configuration

Field	Description	Scheme
`allowedDomains`	List of domains to crawl	`[]string`
`allowedURLFilters`	List of URL regex patterns to include	`[]string`
`delay`	Duration to wait before creating a new request to the matching domains. Defaults to `500ms`	Duration
`depth`	Maximum number of links to follow	`integer`
`disallowedDomains`	List of domains to exclude from crawling	`[]string`
`disallowedURLFilters`	List of URL regex patterns to exclude	`[]string`
`filters`	List of regex filters to apply to the crawled links	`[]string`
`parallelism`	Number of the maximum allowed concurrent requests. Defaults to `2`	`integer`
`randomDelay`	Extra randomized duration to wait added to Delay before creating a new request. Defaults to `100ms`	Duration

Result Variables

Result variables can be used in test, display and transform expressions

Name	Description	Scheme
`code`	HTTP response code	`int`
`headers`	HTTP response headers	`map[string]string`
`elapsed`	HTTP Request duration	`time.Duration`
`sslAge`	Time until SSL certificate expires	`time.Duration`
`content`	HTTP Response body	string
`json`	If `Content-Type=application/json` response body converted into JSON object	`JSON`

The above canary (http-check.yaml) is functionally equivalent to http-check-expr.yaml below

apiVersion: canaries.flanksource.com/v1
kind: Canary
metadata:
  name: http-tls-duration
spec:
  schedule: "@every 5m"
  http:
    - name: http pass response 200 status code
      endpoint: https://httpbin.flanksource.com/status/200
      test:
        expr: "code in [200,201,301] && sslAge > Duration('7d')"

Authentication

Basic Authentication

http_auth.yaml
apiVersion: canaries.flanksource.com/v1
kind: Canary
metadata:
  name: http-basic-auth
spec:
  schedule: "@every 1m"
  http:
    - name: "basic auth fail"
      url: https://httpbin.flanksource.com/basic-auth/hello/world
      responseCodes: [401]
    - name: "basic auth pass"
      url: https://httpbin.flanksource.com/basic-auth/hello/world
      responseCodes: [200]
      username:
        valueFrom:
          secretKeyRef:
            name: httpbin-secret
            key: username
      password:
        valueFrom:
          secretKeyRef:
            name: httpbin-secret
            key: password

Template Body Variables

Name	Scheme
`metadata.name`	string
`metadata.namespace`	string
`metadata.labels`	map[string]string
`{fields from []envVars}`	any

Variables defined in envVars are available to template with the name that's configured on the spec. Eg: In the following spec, the vars my_secret_path and my_secret_var, defined in envVars, are available during templating.

Templating request body from env variables

http_template.yaml
apiVersion: canaries.flanksource.com/v1
kind: Canary
metadata:
  name: templated-http
spec:
  schedule: "@every 5m"
  http:
    - name: templated-http
      url: https://webhook.site/#!/9f1392a6-718a-4ef5-a8e2-bfb55b08afca/f93d307b-0aaf-4a38-b9b3-db5daaae5657/1
      responseCodes: [200]
      templateBody: true

      env:
        - name: db
          valueFrom:
            secretKeyRef:
              name: db-user-pass
              key: username
      body: |
        {
          "canary": "{{.canary.name}}",
          "secret": "{{.db}}"
        }

Templating URL from env variables

http_user_pass_template.yaml
apiVersion: canaries.flanksource.com/v1
kind: Canary
metadata:
  name: http-basic-auth-url
spec:
  http:
    - name: test-url-via-env
      # The URL can be templated from arbritrary values using the env field and $(.) syntax
      url: $(.url)
      env:
        - name: url
          value: https://hello:world2@httpbin.org/basic-auth/hello/world2
    - name: test-basic-via-env
      # the url can be constructed from multiple variables
      url: https://$(.user):$(.pass)@httpbin.org/basic-auth/hello/world
      templateBody: true
      body: |
        {{. | toJSONPretty " " }}
      responseCodes: [200]
      env:
        - name: user
          value: hello
        - name: pass
          value: world

See Escaping variables

Metrics

HTTP Check Metrics
`canary_check_http_response_status{status, statusClass, url}`	Counter	Response code counter for each endpoint
`canary_check_http_ssl_expiry{url}`	Gauge

Status class is one of 1xx, 2xx, 3xx, 4xx, 5xx

Adding custom metrics

metrics.yaml
apiVersion: canaries.flanksource.com/v1
kind: Canary
metadata:
  name: http-pass-single
spec:
  schedule: "@every 5m"
  http:
    - name: http-minimal-check
      url: https://httpbin.flanksource.com/status/200
      metrics:
        - name: httpbin_count
          type: counter
          value: "1"
          labels:
            - name: check_name
              valueExpr: check.name
            - name: code
              valueExpr: code
        - name: httpbin_2xx_duration
          type: counter
          value: elapsed.getMilliseconds()
          labels:
            - name: check_name
              valueExpr: check.name

Transforming metrics into individual checks

metrics.yaml
apiVersion: canaries.flanksource.com/v1
kind: Canary
metadata:
  name: exchange-rates
spec:
  schedule: "every 30 @minutes"
  http:
    - name: exchange-rates
      url: https://api.frankfurter.app/latest?from=USD&to=GBP,EUR,ILS
      transform:
        expr: |
          {
            'metrics': json.rates.keys().map(k,  {
              'name': "exchange_rate",
              'type': "gauge",
              'value': json.rates[k],
              'labels': {
                "from": json.base,
                "to": k
              }
            })
          }.toJSON()
      metrics:
        - name: exchange_rate_api
          type: histogram
          value: elapsed.getMilliseconds()

TLS Config​

OAuth​

Crawl Configuration​

Result Variables​

Authentication​

Template Body Variables​

Metrics​