Environments

Variables

kustomization/vars

The vars Kustomization loads the folder vars/<your namespace name>/ into your tenant namespace. Everything in it — ConfigMaps with per-environment values, SOPS-encrypted secrets — becomes available in the cluster before your applications reconcile (apps should dependsOn: [vars]).

graph LR
    A["<b style='color: #FF8C00;'>tenant-apps</b>"]:::mainEntry -->|Loading| B["<b style='color: #1E90FF;'>vars/kustomization</b>"]:::intermediate
    B -->|Loading| C["<b style='color: #32CD32;'>/vars/<namespace>/</b>"]:::finalStep

    classDef mainEntry fill:#FFFACD,stroke:#FF8C00,stroke-width:2px,stroke-dasharray: 5 5;
    classDef intermediate fill:#E0FFFF,stroke:#1E90FF,stroke-width:2px,stroke-dasharray: 5 5;
    classDef finalStep fill:#E0FFD1,stroke:#32CD32,stroke-width:2px,stroke-dasharray: 5 5;

Once the ConfigMaps are loaded you can reference their values in any manifest managed by a Flux Kustomization:

$ oc get configmap -n tenant-tpl
NAME                       DATA   AGE
echo-server-settings       2      16s
...

Using variables in your manifests

Values that change between environments/namespaces are ${VARIABLES} in the manifests, substituted by Flux at deploy time (postBuild.substituteFrom). Two sources are available:

• tenant-settings — injected by the platform bootstrap into every Kustomization. Provides TENANT_NAMESPACE, TENANT_DOMAIN, TECHNICAL_ACCOUNT, cluster domains, ...
• your own ConfigMaps/Secrets in vars/<namespace>/, e.g. echo-server-settings. To keep them next to tenant-settings, the Kustomization must carry the label substitution.flux.use.json/enabled: "true" — otherwise your substituteFrom is replaced by tenant-settings.

From the echo-server HelmRelease:

spec:
  serviceAccountName: ${TECHNICAL_ACCOUNT}      # from tenant-settings
  values:
    controllers:
      echo-server:
        replicas: ${ECHO_SERVER_REPLICAS:=2}    # from vars/<namespace>/echo-server-settings, default 2

Variables support defaults (${VAR:=default}) and your own ConfigMap/Secret references are marked optional: true, so a missing settings file falls back to the defaults instead of failing the Kustomization.

Quote numbers in the ConfigMap ("2"): substitution is textual. The full walkthrough is in Repository Handling Multiple Environments.

Reloader

We have deployed on our cluster https://github.com/stakater/Reloader which allow your application to automatically be redeployed if a secret or confimaps change is detected.

Simply annotate your deployment,cronjob,statefullset etc....

annotations:
    reloader.stakater.com/auto: "true"

In the context of the App template bwj-s since we are using a helmRelease you should annotate the controller. As an example for the echo-server.

values:
    controllers:
      echo-server:
        annotations:
          reloader.stakater.com/auto: "true"
        strategy: RollingUpdate

Secret

Encryption

In regard of GitOps there is multiple way to handle encryption of secret that live within a git repository. We recommending you to use SOPS as you encryption engine.

We have already Setup a key Private key dedicated to your namespace that will be able to decrypt any secret that you is in your git repository.

.sops.yaml is the configuration file that will handle how you secret will be encrypted while using sops.

Quick Start

To work with secret you'll need:

• SOPS stable release
• age

Create your own key

Linux

mkdir -p $XDG_CONFIG_HOME/sops/age
age-keygen -o $XDG_CONFIG_HOME/sops/age/keys.txt

MacOS

mkdir -p "$HOME/Library/Application Support/sops/age"
age-keygen -o "$HOME/Library/Application Support/sops/age/keys.txt"

Propagate your Public key

Edit the .sops.yaml file (one per environment folder under vars/) and add your public key that you have generated previously.

Please notice that you can copy this file into any subfolder of your project in case you need to have different keys depending on your secrets file. This is useful to limit who has access to the production secrets while all developers might have access to the dev secrets.

$ cat vars/tenant-tpl/.sops.yaml
  ---

  # This example uses YAML anchors which allows reuse of multiple keys
  # without having to repeat yourself.
  # Also see https://github.com/Mic92/dotfiles/blob/master/nixos/.sops.yaml
  # for a more complex example.
  keys:
    age:
    - &cluster_age_key age13jnzxrtrghlh8zvc9q3d8yd2a9xdp8jset72l8dwz6pept3j3c0qkmxd47
    - &YOUR_KEY_NAME <PUBLIC_KEY>
  creation_rules:
    - path_regex: .+secret(\.sops)?\.ya?ml
      input_type: yaml
      encrypted_regex: ^(data|stringData)$
      key_groups:
        - age: &key_groups
            - *cluster_age_key
            - *YOUR_KEY_NAME
    - path_regex: .+secret(\.sops)?\.env
      input_type: env
      key_groups:
        - age: *key_groups
  stores:
    yaml:
      indent: 2

Create your first secret

$ sops name_of_you_file.secret.sops.yaml

You can then deploy it the cluster will be able to Decrypt it using it's public key

Edit secrets

You can use the same command used to create new secrets to edit existing files

You can also use this VSCode plugin

Rewrapping secret

In case add/remove a key secret generated previously will need to be reencrypted with the appropriate key. We have place a shell script that will do that for you.

It will reencrypt all the secret that it will find in folder and subfolder following the .sops.yaml rules files of your directory.

💥 INFOS
You can have as many .sops.yaml file as you want in your repository