Observability
Everything is under Observability in the site, per controller and time range.
Metrics
Section titled “Metrics”Reconciles, errors, reconcile time and queue depth, plus the controller’s CPU, memory and
network. Scraped every 15 seconds from where your controller serves them (:8080/metrics unless
you set it) and kept 15 days.
Changes on the charts
Section titled “Changes on the charts”Lines on the charts mark when a controller changed, so you can match a shift in its metrics to a
new version. Point at a line, or move to it with the arrow keys, to see what changed, for example
image ghcr.io/acme/op:v1 → ghcr.io/acme/op:v2.
- Solid lines are changes to the controller: deploying it, or a new image, CPU or memory limit, args, env or probes. Args and env show that they changed (env by name), never their values.
- Dotted lines are restarts, stops and starts, and rollouts by the platform itself, such as a new tunnel or a renewed certificate.
A line marks when the change was made. Metrics follow a little later: the new pod has to start, and rates are averaged over a few minutes. Changes only show from when ctrlplane started recording them, and an image pushed again under the same tag shows as a restart, not a new version.
The same records are in your logs store, under the stream {ctrlplane_source="deploys"}.
Your controllers’ output, with a live tail and search in
LogsQL, for example "failed to sync".
Events
Section titled “Events”What your controllers record about objects, newest first; warnings also show on the overview.
Your API server drops events after an hour, but ctrlplane keeps them as long as logs, under the
stream {ctrlplane_source="events"}.
Your own Grafana
Section titled “Your own Grafana”Your API server serves this data, pinned to your control plane:
| Data source | URL |
|---|---|
| Prometheus | https://<id>.api.ctrlplane.run/platform/metrics |
| VictoriaLogs | https://<id>.api.ctrlplane.run/platform/logs |
-
Create a token for a service account (it may only read metrics and logs):
Terminal window kubectl create serviceaccount grafanakubectl create token grafana --duration=8760h -
In each data source, add the header
Authorization: Bearer <token>. -
Under TLS, add your control plane’s CA from your kubeconfig:
Terminal window kubectl config view --raw --minify \-o jsonpath='{.clusters[0].cluster.certificate-authority-data}' | base64 -d