Perses

[!WARNING] This plugin is in beta and please report any bugs by creating an issue here.

Table of Contents

Learn more about the Perses Plugin. Use it to visualize Prometheus/Thanos metrics for your Greenhouse remote cluster.

The main terminologies used in this document can be found in core-concepts.

Overview

Observability is often required for the operation and automation of service offerings. Perses is a CNCF project and it aims to become an open-standard for dashboards and visualization. It provides you with tools to display Prometheus metrics on live dashboards with insightful charts and visualizations. In the Greenhouse context, this complements the kube-monitoring plugin, which automatically acts as a Perses data source which is recognized by Perses. In addition, the Plugin provides a mechanism that automates the lifecycle of datasources and dashboards without having to restart Perses.

Perses Architecture

Disclaimer

This is not meant to be a comprehensive package that covers all scenarios. If you are an expert, feel free to configure the Plugin according to your needs.

Contribution is highly appreciated. If you discover bugs or want to add functionality to the plugin, then pull requests are always welcome.

Quick Start

This guide provides a quick and straightforward way how to use Perses as a Greenhouse Plugin on your Kubernetes cluster.

Prerequisites

  • A running and Greenhouse-managed Kubernetes remote cluster
  • kube-monitoring Plugin should be installed with .spec.kubeMonitoring.prometheus.persesDatasource: true and it should have at least one Prometheus instance running in the cluster

The plugin works by default with anonymous access enabled. This plugin comes with some default dashboards and the kube-monitoring datasource will be automatically discovered by the plugin.

Step 1: Add your dashboards and datasources

Dashboards are selected from ConfigMaps across namespaces. The plugin searches for ConfigMaps with the label perses.dev/resource: "true" and imports them into Perses. The ConfigMap must contain a key like my-dashboard.json with the dashboard JSON content. Please refer this section for more information.

A guide on how to create custom dashboards on the UI can be found here.

Configuration

ParameterDescriptionDefault
perses.additionalLabelsAdditional labels to add to all resources{}
perses.annotationsStatefulset annotations{}
perses.config.annotationsAnnotations for config{}
perses.config.database.file.extensionDatabase file extensionjson
perses.config.database.file.folderDatabase file folder path/perses
perses.config.database.sqlSQL database configuration{}
perses.config.important_dashboardsList of important dashboards[]
perses.config.provisioning.folders.0Provisioning folder path/etc/perses/provisioning
perses.config.provisioning.intervalProvisioning check interval10s
perses.config.schemas.datasources_pathDatasource schemas path/etc/perses/cue/schemas/datasources
perses.config.schemas.intervalSchema check interval5m
perses.config.schemas.panels_pathPanel schemas path/etc/perses/cue/schemas/panels
perses.config.schemas.queries_pathQuery schemas path/etc/perses/cue/schemas/queries
perses.config.schemas.variables_pathVariable schemas path/etc/perses/cue/schemas/variables
perses.config.security.cookie.same_siteCookie SameSite attributelax
perses.config.security.cookie.secureEnable secure cookiesfalse
perses.config.security.enableAuthEnable authenticationfalse
perses.config.security.readOnlyConfigure Perses instance as readonly modefalse
perses.datasourcesConfigure datasources (DEPRECATED). Please use the ‘sidecar’ configuration to provision datasources[]
perses.fullnameOverrideOverride fully qualified app name""
perses.image.nameContainer image namepersesdev/perses
perses.image.pullPolicyImage pull policyIfNotPresent
perses.image.versionOverride default image tag""
perses.ingress.annotationsAdditional annotations for the Ingress resource. To enable certificate autogeneration, place here your cert-manager annotations.{}
perses.ingress.enabledConfigure the ingress resource that allows you to access Thanos Query Frontendfalse
perses.ingress.hostsIngress hostnames["perses.local"]
perses.ingress.ingressClassNameIngressClass that will be be used to implement the Ingress (Kubernetes 1.18+)""
perses.ingress.pathIngress path/
perses.ingress.pathTypeIngress path typePrefix
perses.ingress.tlsIngress TLS configuration[]
perses.livenessProbe.enabledEnable liveness probetrue
perses.livenessProbe.failureThresholdLiveness probe failure threshold5
perses.livenessProbe.initialDelaySecondsLiveness probe initial delay10
perses.livenessProbe.periodSecondsLiveness probe period60
perses.livenessProbe.successThresholdLiveness probe success threshold1
perses.livenessProbe.timeoutSecondsLiveness probe timeout5
perses.logLevelLogging level - available options “panic”, “error”, “warning”, “info”, “debug”, “trace” levelinfo
perses.nameOverrideOverride chart name""
perses.persistence.accessModesPVC access modes for data volume["ReadWriteOnce"]
perses.persistence.annotationsPVC annotations{}
perses.persistence.enabledEnable persistence. If disabled, it will use a emptydir volumefalse
perses.persistence.labelsPVC labels{}
perses.persistence.securityContext.fsGroupSecurity context for the PVC when persistence is enabled2000
perses.persistence.sizePVC storage size8Gi
perses.readinessProbe.enabledEnable readiness probetrue
perses.readinessProbe.failureThresholdReadiness probe failure threshold5
perses.readinessProbe.initialDelaySecondsReadiness probe initial delay5
perses.readinessProbe.periodSecondsReadiness probe period10
perses.readinessProbe.successThresholdReadiness probe success threshold1
perses.readinessProbe.timeoutSecondsReadiness probe timeout5
perses.replicasNumber of replicas1
perses.resourcesResource limits and requests{}
perses.service.annotationsService annotations{}
perses.service.labelsService labels{}
perses.service.portService port8080
perses.service.portNameService port namehttp
perses.service.targetPortContainer target port8080
perses.service.typeService typeClusterIP
perses.serviceAccount.annotationsService account annotations{}
perses.serviceAccount.createCreate service accounttrue
perses.serviceAccount.nameService account name""
perses.sidecar.enabledEnable sidecar to auto discover the configmaps holding perses dashboards and datasourcesfalse
perses.sidecar.image.repositoryContainer image repository for the sidecarkiwigrid/k8s-sidecar
perses.sidecar.image.tagContainer image tag for the sidecar1.28.0
perses.sidecar.labelLabel key to watch for ConfigMaps containing Perses resourcesperses.dev/resource
perses.sidecar.labelValueLabel value to watch for ConfigMaps containing Perses resources"true"
perses.volumeMountsAdditional volume mounts[]
perses.volumesAdditional volumes[]

Create a custom dashboard

  1. Add a new Project by clicking on ADD PROJECT in the top right corner. Give it a name and click Add.
  2. Add a new dashboard by clicking on ADD DASHBOARD. Give it a name and click Add.
  3. Now you can add variables, panels to your dashboard.
  4. You can group your panels by adding the panels to a Panel Group.
  5. Move and resize the panels as needed.
  6. Watch this gif to learn more.
  7. You do not need to add the kube-monitoring datasource manually. It will be automatically discovered by Perses.
  8. Click Save after you have made changes.
  9. Export the dashboard.
    • Click on the {} icon in the top right corner of the dashboard.
    • Copy the entire JSON model.
    • See the next section for detailed instructions on how and where to paste the copied dashboard JSON model.

Add Dashboards as ConfigMaps

By default, a sidecar container is deployed in the Perses pod. This container watches all configmaps in the cluster and filters out the ones with a label perses.dev/resource: "true". The files defined in those configmaps are written to a folder and this folder is accessed by Perses. Changes to the configmaps are continuously monitored and are reflected in Perses within 10 seconds.

A recommendation is to use one configmap per dashboard. This way, you can easily manage the dashboards in your git repository.

Folder structure:

dashboards/
├── dashboard1.json
├── dashboard2.json
├── prometheusdatasource1.json
├── prometheusdatasource2.json
templates/
├──dashboard-json-configmap.yaml

Helm template to create a configmap for each dashboard:

{{- range $path, $bytes := .Files.Glob "dashboards/*.json" }}
---
apiVersion: v1
kind: ConfigMap

metadata:
  name: {{ printf "%s-%s" $.Release.Name $path | replace "/" "-" | trunc 63 }}
  labels:
    perses.dev/resource: "true"

data:
{{ printf "%s: |-" $path | replace "/" "-" | indent 2 }}
{{ printf "%s" $bytes | indent 4 }}

{{- end }}