Getting started

• 1: Core Concepts
• 1.1: Organizations
• 1.2: Teams
• 1.3: Clusters
• 1.4: PluginDefinitions, Plugins and PluginPresets
• 2: Overview
• 3: Installation
• 4: Operational Processes
• 5: Ownership

1 - Core Concepts

1.1 - Organizations

What are Organizations?

Organizations are the top-level entities in Greenhouse. Each Organization gets a dedicated Namespace, that contains all resources bound to the Organization. Greenhouse expects an Organization to provide it’s own Identity Provider and currently supports OIDC Identity Providers. Greenhouse also supports SCIM for syncing users and groups from an Identity Provider.

See creating an Organization for more details.

Organization Namespace and Permissions

The Organization’s Namespace in the Greenhouse cluster contains all resources bound to the Organization. This Namespace is automatically provisioned when a new Organization is created and shares the Organization’s name. Once the Namespace is created, Greenhouse will automatically seed RBAC Roles and ClusterRoles for the Organization. These are used to grant permissions for the Organization’s resources to Teams.

• The Administrators of an Organization are specified via a identity provider (IDP) group during the creation of the Organization. Greenhouse automatically creates a Team called <org-name>-admin. This Team is also a support group and all alerts created by the Greenhouse controller are routed to this admin Team, if no other ownership is provided. See operational processes and ownership for details.
• The Administrators for Plugins and Clusters need to be defined by the Organization Admins via RoleBindings for the seeded Roles role:<org-name>:cluster-admin and role:<org-name>:plugin-admin.
• All authenticated users are considered members of the Organization and are granted the organization:<org-name> Role.

The following roles are seeded for each Organization:

OIDC

Each Organization must specify the OIDC configuration for the Organization’s IDP. This configuration is used together with DEXIDP to authenticate users in the Organization.

SCIM

Each Organization can specify SCIM credentials which are used to syncronize users and groups from an Identity Provider. This makes it possible to view the members of a Team in the Greenhouse dashboard.

1.2 - Teams

What are Teams?

Teams in a Greenhouse Organization are used to group users by group claims on the token provided by the upstream identity provider (IdP).

This can be used, for example, for

• organizational management
• access and permission management
• identifying Ownership of resources

The Greenhouse UI is showing the members of a Team.

Team RBAC

TeamRoles and TeamRoleBindings provide a mechanism to control the permissions of Teams to onboarded Clusters of an Organization.

Team role-based access control (RBAC) wraps the concept of Kubernetes RBAC in TeamRoles and TeamRoleBindings . TeamRoles are used to define a set of RBAC permissions. These permissions can be granted to Teams with TeamRoleBindings . A TeamRoleBinding refers to a Team, a TeamRole , Cluster(s) and optional Namespaces. Depending on the latter, Greenhouse will create the appropriate rbacv1 resources on the targeted cluster(s) in either Cluster or Namespace scope. More information about how this can be configured is mentioned in this user guide.

Example of a TeamRoleBinding for a observability-admin which grants the cluster-admin role on the observability cluster in the logs and metrics namespaces. The TeamRoleBinding contains a list of namespaces and a label selector to select the cluster(s) to target. If no Namespaces are provided, then Greenhouse will create a ClusterRoleBinding instead of a RoleBinding.

Support Groups

Support Groups in Greenhouse are a subset of Teams in an Organization. These Teams are used to identify response groups for operational tasks and to prefilter UI content.

Since a user can be part of many Teams the expectation is that they are only part of one Support Group.

To identify a Team as a Support Group in Greenhouse it needs to be labeled with greenhouse.sap/support-group: "true".

1.3 - Clusters

What are Clusters?

In the context of Greenhouse a Cluster represents a Kubernetes cluster that is onboarded to Greenhouse. Onboarded in this context means that Greenhouse can handle the management of role-based access control (RBAC) and the provisioning of operating tools (e.g. logging, monitoring, ingress etc.). The Greenhouse dashboard provides an overview of all onboarded clusters. Throughout Greenhouse the reference to a Cluster is used to target it for configuration and deployments.

Cluster access

During the initial onboarding of a cluster, Greenhouse will create a dedicated ServiceAccount inside the onboarded cluster. This ServiceAccount’s token is rotated automatically by Greenhouse.

Cluster registry (coming soon)

Once a Cluster is onboarded to Greenhouse a ClusterKubeConfig is generated for the Cluster based on the OIDC configuration of the Organization. This enables members of an Organization to access the fleet of onboarded Clusters via the common Identity Provider. on the respective Clusters can be managed via Greenhouse Team RBAC.

In order to make it convenient to use these ClusterKubeConfigs and to easily switch between multiple context locally there will be a CLI provided by Greenhouse.

1.4 - PluginDefinitions, Plugins and PluginPresets

What are PluginDefinitions and Plugins?

PluginDefinitons and Plugins are the Greenhouse way to extend the core functionality with domain specific features. PluginDefinitions, as the name suggests, are the definition of a Plugin, whereas a Plugin is a concrete instance of a PluginDefinition that is deployed to a Cluster.

The PluginDefinitions are shared between all Organizations in Greenhouse. A PluginDefinition can include a frontend, that is displayed in the Greenhouse dashboard and/or a backend component. The frontend is expected to be a standalone microfrontend created with the Juno framework. The backend components of a PluginDefinition are packaged as a Helm Chart and provide sane and opinionated default values. This allows Greenhouse to package and distribute tools such as Prometheus with a sensible default configuration, as well as giving the user a list of configurable values.

A Plugin is used to deploy the Helm Chart referenced in a PluginDefinition to a Cluster. The Plugin can be considered as an instance of a PluginDefinition, this instance specifies the PluginDefinition, the desired Cluster and additional values to set. Depending on the PluginDefinition, it may be necessary to specify required values (e.g. credentials, endpoints, etc.), but in general the PluginDefinition provides well-established default values.

[!NOTE] In this example the Plugin ‘openTelemetry-cluster-a’ is used to deploy the PluginDefinition ‘openTelemetry’ to the cluster ‘cluster-a’.

PluginPresets

PluginPresets are a mechanism to configure Plugins for multiple clusters at once. They are used to define a common configuration for a PluginDefinition that can be applied to multiple clusters, while allowing to override the configuration for individual clusters.

[!NOTE] In this example the PluginPreset ’example-obs’ references the PluginDefinition ’example’ and contains a clusterSelector that matches the clusters ‘cluster-a’ and ‘cluster-b’. The PluginPreset creates two Plugins ’example-obs-cluster-a’ and ’example-obs-cluster-b’ for the respective clusters.

2 - Overview

What is Greenhouse?

Greenhouse is a cloud operations platform designed to streamline and simplify the management of a large-scale, distributed infrastructure.

It offers a unified interface for organizations to manage various operational aspects efficiently and transparently and operate their cloud infrastructure in compliance with industry standards.
The platform addresses common challenges such as the fragmentation of tools, visibility of application-specific permission concepts and the management of organizational groups. It also emphasizes the harmonization and standardization of authorization concepts to enhance security and scalability. With its operator-friendly dashboard, features and extensive automation capabilities, Greenhouse empowers organizations to optimize their cloud operations, reduce manual efforts, and achieve greater operational efficiency.

Value Propositions

1. Holistic dashboard Unified dashboard for infrastructure, alert, security, compliance, and organizational management. (Juno)
2. Organization management Greenhouse allows to manage organizational groups as Teams. Teams can be provided with fine-grained access control to resources and tools. (e.g. Github Repositories, Kubernetes RBAC, etc.)
3. Automation Greenhouse allows to configure tools and access control in a declarative way, that is auto-deployed across a fleet of Kubernetes clusters.
4. Security & Compliance With Heureka, Greenhouse integrates a Security Posture Management tool that focuses on remediation of security issues (vulnerabilities, security events, policy violations), while ensuring compliance and auditability.
5. Extensibility Greenhouse provides a plugin system that provides a curated catalog of plugins with sane defaults. Furthermore, it is possible to extend the platform with self-developed plugins.

Roadmap

The Roadmap Kanban board provides an overview of ongoing and planned efforts.

Architecture & Design

The Greenhouse design and architecture document describes the various use-cases and user stories.

3 - Installation

This section provides a step-by-step guide to install Greenhouse on a Gardner shoot cluster.

Prerequisites

Before you start the installation, make sure you have the following prerequisites:

• Helm & Kubernetes CLI
• OAuth2/OpenID provider (see Authentik)
• Gardener Shoot Cluster configured to use the OIDC provider
• nginx-ingress deployed in the cluster

Installation

To install Greenhouse on your Gardener shoot cluster, follow these steps:

1. Create a values file called values.yaml with the following content:

     global:
       dnsDomain: tld.domain # Shoot.spec.dns.domain
       kubeAPISubDomain: myapi # api is already used by Gardener
       oidc:
         enabled: true
         issuer: <issuer-url>
         clientID: <client-ID>
         clientSecret: <top-secret>
   
     organization:
       enabled: false # disable, because the greenhouse webhook is not running yet
   
     teams:
       admin:
         mappedIdPGroup: greenhouse-admins
   
     # gardener specifics
     dashboard:
       ingress:
         annotations:
           dns.gardener.cloud/dnsnames: "*"
           dns.gardener.cloud/ttl: "600"
           dns.gardener.cloud/class: garden
           cert.gardener.cloud/purpose: managed
   
     idproxy:
       enabled: false # disable because no organization is created yet
       ingress:
         annotations:
           dns.gardener.cloud/dnsnames: "*"
           dns.gardener.cloud/ttl: "600"
           dns.gardener.cloud/class: garden
           cert.gardener.cloud/purpose: managed
   
     cors-proxy:
       ingress:
         annotations:
           dns.gardener.cloud/dnsnames: "*"
           dns.gardener.cloud/ttl: "600"
           dns.gardener.cloud/class: garden
           cert.gardener.cloud/purpose: managed
   
     # disable Plugins for the greenhouse organization, PluginDefinitions are missing
     plugins:
       enabled: false
   
     # disable, Prometheus CRDs are missing
     manager:
       alerts:
         enabled: false
   

2. Install the Greenhouse Helm chart:

   helm install greenhouse oci://ghcr.io/cloudoperators/greenhouse/charts/greenhouse --version <greenhouse-release-version> -f values.yaml
   

3. Enable Greenhouse OIDC

   Now set organization.enabled and idproxy.enabled to true in the values.yaml file and upgrade the Helm release:

   helm upgrade greenhouse oci://ghcr.io/cloudoperators/greenhouse/charts/greenhouse --version <greenhouse-release-version> -f values.yaml
   

   This will create the initial Greenhouse Organization and the Greenhouse Admin Team. This Organization will receive the greenhouse namespace, which is used to manage the Greenhouse installation and allows to administer other organizations. Enabling the idproxy will deploy the idproxy service which handles the OIDC authentication.

4 - Operational Processes

Operational Processes in Greenhouse

Greenhouse provides a couple of predefined operational processes.

Operational processes facilitated via Greenhouse heavily rely on the Ownership principle. It is used to route operational tasks to Support Groups.

Examples for these operational tasks are:

• Alert routing based on metrics
• Lifecycle management of k8s Clusters
• Security posture and vulnerability patch management
• Secret rotation and management

Labels Used

Greenhouse focuses on labels in three different places:

• On resources (e.g. PluginPresets, Clusters but also k8s Deployments, Pods, etc.)
• On metrics exposed by those resources
• On Prometheus alerts based on metrics

The following labels are used by Greenhouse automation:

Alert Routing

Monitoring and alert routing is achieved through a combination of Plugins running on the remote Clusters and the Greenhouse central cluster.

All alerts processed with Greenhouse need the support_group label that can be extracted from the greenhouse.sap/owned-by.

With the Alerts Plugin a holistic alerts dashboard is integrated to the Greenhouse UI. This dashboard is prefiltered on the support group a user is member of. It directly displays alerts by region and severity. Also service is prominently displayed.

It is good practice to also route alerts by support_group and/or severity to specific Alertmanager receivers (e.g. Slack channels).

Lifecycle management of k8s Clusters

All Cluster related alerts, including version expiration and other maintenance tasks are routed to the owning support_group of the Cluster.

Security Management

Security posture and vulnerability management is achieved through the heureka Plugin. It scans for security violations in running k8s containers and displays these by support_group and service.

Secret Management

With secret management Greenhouse wants to have alerts on expiring Secrets in need of rotation. These alerts will be routed to the respective support_groups. See roadmap item for further information.

5 - Ownership

What is Ownership within Greenhouse

Ownership in Greenhouse is the combination of two of the core features:

• User and Organization management via Teams
• Deployment of resources (Plugins, TeamRoleBindings) to remote Clusters

Greenhouse provides a 1:1 relationship between a Team and

• PluginPresets
• Plugins
• Clusters
• TeamRoleBindings
• Secrets

Within the context of Greenhouse this relationship is called Ownership.

Why Ownership of Resources

Operational processes facilitated via Greenhouse rely on Ownership:

By identifying the owner of a resource it is possible to route operational tasks on the resource to the owner.

How is Ownership achieved

Greenhouse expects a label with the key greenhouse.sap/owned-by with a value matching an existing Team on the following resources in the Greenhouse central cluster:

• PluginPresets
• Plugins
• Clusters
• TeamRoleBindings
• Secrets

Missing greenhouse.sap/owned-by label results in a StatusCondition called OwnerLabelSetCondition set to false. A greenhouse_owned_by_label_missing metric on missing owner labels is exposed and alerted on.

The owner label is also expected on k8s resources (e.g. Deployments, Pods, …) exposing metrics on the remote clusters.

Label Transport

On Greenhouse central cluster

The Greenhouse controller transports labels from a source resource to a target resource on the Greenhouse cluster. This is currently active for:

• Secrets that are used to bootstrap a Cluster
• PluginPresets creating Plugins

The transport works via an metadata.annotation on the source:

metadata:
  ...
  labels:
    foo: bar
    qux: baz
    greenhouse.sap/owned_by: foo-team
    ...
  annotations:
    greenhouse.sap/propagate-labels: "foo, greenhouse.sap/owned_by"
  ...

which results in metadata.labels and a state in metadata.annotations added to the target:

metadata:
  annotations:
   ...
    greenhouse.sap/last-applied-propagator: '{"labelKeys":["foo","greenhouse.sap/owned_by"]}'
  labels:
    foo: bar
    greenhouse.sap/owned_by: foo-team
   ...

On Resources on Remote Clusters

Greenhouse will provide the automation to label all resources created by a Plugin on the remote Cluster in the future: https://github.com/cloudoperators/greenhouse-extensions/issues/704

Currently Greenhouse provides the owned-by label as a OptionValue to be consumed by the underlying helm chart of the Plugin.