diff --git a/Guides/Repository-Handling-Multiple-Environments.md b/Guides/Repository-Handling-Multiple-Environments.md new file mode 100644 index 0000000..b57165e --- /dev/null +++ b/Guides/Repository-Handling-Multiple-Environments.md @@ -0,0 +1,257 @@ +# GitOps-Ready Namespaces with Flux +## Overview + +In our OpenShift clusters, namespaces are provisioned GitOps-ready by default. +Each namespace is linked to a Git repository where configurations are stored and managed declaratively. + +This setup enables: + +* Consistent GitOps workflow across tenants + +* Multi-environment support from a single repo + +* Overlay approach with minimal duplication + +Each namespace is link to a git repository this combination is what we call a +*Tenant* for this example we are going to take the example of 3 environments +dev, qa, prod: + +We forward the `TENANT_NAMESPACE` variable holding the value of you tenant +name. This allows Kustomizations to dynamically target the right folder for +each environment. + +Flux Kustomization e.g + +```yaml +apiVersion: kustomize.toolkit.fluxcd.io/v1 +kind: Kustomization +metadata: + name: my-app + namespace: ${TENANT_NAMESPACE} +spec: + interval: 5m + path: ./overlays/${TENANT_NAMESPACE} + prune: true + sourceRef: + kind: GitRepository + name: my-app-repo +``` +If the tenant is my-app-dev then it will target the folder `./overlays/my-app-dev` + +## Overlays Structure and Patterns + +### Simple Patterns +We use the Kustomize overlays pattern to separate environment-specific +differences from shared configuration. Given our vanilla tenant folder. + +```shell +(venv) euler@HAL:~/.../tenants/tenant-tpl(main)$ tree +. +├── config +│   ├── app +│   │   └── limits-ranges.yaml +│   └── ks.yaml +├── echo-server +│   ├── app +│   │   └── helmrelease.yaml +│   └── ks.yaml +├── kustomization.yaml +├── README.md +├── renovate.json5 +├── repos +│   ├── helm +│   │   └── bjw-s.yaml +│   └── ks.yaml +├── scripts +│   └── rewrap-secrets.sh +└── vars + ├── ks.yaml + └── tenant-tpl + ├── example.yaml + └── README.md +``` +I will in this example spawn two namespaces link to that git repository +*test-tpl* & *test-tpl-dev* + +Query to check resources from flux perspective. + +```bash +$ flux get all -n tenant-tpl + NAME REVISION SUSPENDED READY MESSAGE + gitrepository/tenant-repos main@sha1:0fcffb6b False True stored artifact for revision 'main@sha1:0fcffb6b' + + NAME REVISION SUSPENDED READY MESSAGE + helmrepository/bjw-s False True Helm repository is Ready + + NAME REVISION SUSPENDED READY MESSAGE + helmchart/tenant-tpl-echo-server 3.2.1 False True pulled 'app-template' chart with version '3.2.1' + + NAME REVISION SUSPENDED READY MESSAGE + helmrelease/echo-server 3.2.1 False True Helm install succeeded for release tenant-tpl/echo-server.v1 with chart app-template@3.2.1 + + NAME REVISION SUSPENDED READY MESSAGE + kustomization/echo-server main@sha1:0fcffb6b False True Applied revision: main@sha1:0fcffb6b + kustomization/repos-sync main@sha1:0fcffb6b False True Applied revision: main@sha1:0fcffb6b + kustomization/tenant-apps main@sha1:0fcffb6b False True Applied revision: main@sha1:0fcffb6b + kustomization/tenant-config main@sha1:0fcffb6b False True Applied revision: main@sha1:0fcffb6b + kustomization/vars main@sha1:0fcffb6b False True Applied revision: main@sha1:0fcffb6b + +$ flux get all -n tenant-tpl-dev + NAME REVISION SUSPENDED READY MESSAGE + gitrepository/tenant-repos main@sha1:0fcffb6b False True stored artifact for revision 'main@sha1:0fcffb6b' + + NAME REVISION SUSPENDED READY MESSAGE + helmrepository/bjw-s False True Helm repository is Ready + + NAME REVISION SUSPENDED READY MESSAGE + helmchart/tenant-tpl-dev-echo-server 3.2.1 False True pulled 'app-template' chart with version '3.2.1' + + NAME REVISION SUSPENDED READY MESSAGE + helmrelease/echo-server 3.2.1 False True Helm install succeeded for release tenant-tpl-dev/echo-server.v1 with chart app-template@3.2.1 + + NAME REVISION SUSPENDED READY MESSAGE + kustomization/echo-server main@sha1:0fcffb6b False True Applied revision: main@sha1:0fcffb6b + kustomization/repos-sync main@sha1:0fcffb6b False True Applied revision: main@sha1:0fcffb6b + kustomization/tenant-apps main@sha1:0fcffb6b False True Applied revision: main@sha1:0fcffb6b + kustomization/tenant-config main@sha1:0fcffb6b False True Applied revision: main@sha1:0fcffb6b + kustomization/vars False False kustomization path not found: stat /tmp/kustomization-1496015889/vars/tenant-tpl-dev: no such file or directory +``` + +As you can see they are similar and have both deployed the echo server. However +as you can see there is a slight difference with the `kustomzation/vars`. The +`dev` environements is complaining about some missing directory. Let's inspect +the vars kustomization + +```yaml +$ cat vars/ks.yaml +--- +# yaml-language-server: $schema=https://kubernetes-schemas.pages.dev/kustomize.toolkit.fluxcd.io/kustomization_v1.json +apiVersion: kustomize.toolkit.fluxcd.io/v1 +kind: Kustomization +metadata: + name: &app vars + namespace: ${TENANT_NAMESPACE} +spec: + targetNamespace: ${TENANT_NAMESPACE} + commonMetadata: + labels: + app.kubernetes.io/name: *app + path: ./vars/${TENANT_NAMESPACE} + prune: true + sourceRef: + kind: GitRepository + name: tenant-repos + wait: false + interval: 10m + retryInterval: 1m + timeout: 5m + +``` +As you can see this kustomization is defining a lot of field with the value +`${TENANT_NAMESPACE}`. That's because the `/vars` folder is already +preconfigured to handle overlays patterns. It will target the right directory +according to the namespace name he's running on. If we check our git repository +structure: + +```shell +(venv) euler@HAL:~/.../tenants/tenant-tpl(main)$ tree vars/ +vars/ +├── ks.yaml +└── tenant-tpl + ├── example.yaml + └── README.md + +2 directories, 3 files +``` + +According to the kustomization definition the path: +`./vars/${TENANT_NAMESPACE}` will not be found for the tenant `tenant-tpl-dev` +We are going to create the appropriate folder. + +```shell +$ mkdir vars/tenant-tpl-dev +$ cp -r vars/tenant-tpl/example.yaml vars/tenant-tpl + tenant-tpl/ tenant-tpl-dev/ +$ cp -r vars/tenant-tpl/example.yaml vars/tenant-tpl-dev/dev-example.yaml +$ vim vars/tenant-tpl-dev/dev-example.yaml +$ cat vars/tenant-tpl-dev/dev-example.yaml +--- +apiVersion: v1 +kind: ConfigMap +metadata: + name: example-vars-dev +data: + EXAMPLE: foo-dev +$ git commit -am 'Adding dev vars folder' + [main 07a29a5] Adding dev vars folder + 1 file changed, 7 insertions(+) + create mode 100644 vars/tenant-tpl-dev/dev-example.yaml +$ git push + Enumerating objects: 7, done. + Counting objects: 100% (7/7), done. + Delta compression using up to 22 threads + Compressing objects: 100% (4/4), done. + Writing objects: 100% (5/5), 1.16 KiB | 1.16 MiB/s, done. + Total 5 (delta 1), reused 0 (delta 0), pack-reused 0 + To ssh://git-ssh.kvant.cloud:2222/phoenix-oss/tenant-tpl.git + 0fcffb6..07a29a5 main -> main +``` +Verification + +```shell +$ flux reconcile ks vars -n tenant-tpl-dev --with-source #One liner to force source update and reconciliation + ► annotating GitRepository tenant-repos in tenant-tpl-dev namespace + ✔ GitRepository annotated + ◎ waiting for GitRepository reconciliation + ✔ fetched revision main@sha1:07a29a5d22eae9be68fd9d9da141b3fa83f9f5d2 + ► annotating Kustomization vars in tenant-tpl-dev namespace + ✔ Kustomization annotated + ◎ waiting for Kustomization reconciliation + ✔ applied revision main@sha1:07a29a5d22eae9be68fd9d9da141b3fa83f9f5d2 + +$ flux get all -n tenant-tpl-dev + NAME REVISION SUSPENDED READY MESSAGE + gitrepository/tenant-repos main@sha1:07a29a5d False True stored artifact for revision 'main@sha1:07a29a5d' + + NAME REVISION SUSPENDED READY MESSAGE + helmrepository/bjw-s False True Helm repository is Ready + + NAME REVISION SUSPENDED READY MESSAGE + helmchart/tenant-tpl-dev-echo-server 3.2.1 False True pulled 'app-template' chart with version '3.2.1' + + NAME REVISION SUSPENDED READY MESSAGE + helmrelease/echo-server 3.2.1 False True Helm install succeeded for release tenant-tpl-dev/echo-server.v1 with chart app-template@3.2.1 + + NAME REVISION SUSPENDED READY MESSAGE + kustomization/echo-server main@sha1:07a29a5d False True Applied revision: main@sha1:07a29a5d + kustomization/repos-sync main@sha1:07a29a5d False True Applied revision: main@sha1:07a29a5d + kustomization/tenant-apps main@sha1:07a29a5d False True Applied revision: main@sha1:07a29a5d + kustomization/tenant-config main@sha1:07a29a5d False True Applied revision: main@sha1:07a29a5d + kustomization/vars main@sha1:07a29a5d False True Applied revision: main@sha1:07a29a5d + +$ oc get configmap -n tenant-tpl-dev + NAME DATA AGE + example-vars-dev 1 3m32s + kube-root-ca.crt 1 36m + openshift-service-ca.crt 1 36m + tenant-settings 10 36m +``` + +Congratulations you now manage to handle simple overlays patterns and have the +possibility to handle multiple environments using one git repository. + + +### Advanced Patterns + +#### Base + +In the previous example we showed to you how to handle configmaps or secrets +to be loaded for a given environments. In this example we are going to create a +directory structure that allow us to *Have shared resources definition and +environments specific one. + +Taking back our echo-server example. + +XXX + +