Initiate documentation for gitops patterns
parent
22f9a8280a
commit
efcff670d5
GPG key ID:
2681291F9206E471
1 changed files with 257 additions and 0 deletions
257
Guides/Repository-Handling-Multiple-Environments.md
Normal file
257
Guides/Repository-Handling-Multiple-Environments.md
Normal file
|
|
@ -0,0 +1,257 @@
|
||||||
|
# GitOps-Ready Namespaces with Flux
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
In our OpenShift clusters, namespaces are provisioned GitOps-ready by default.
|
||||||
|
Each namespace is linked to a Git repository where configurations are stored and managed declaratively.
|
||||||
|
|
||||||
|
This setup enables:
|
||||||
|
|
||||||
|
* Consistent GitOps workflow across tenants
|
||||||
|
|
||||||
|
* Multi-environment support from a single repo
|
||||||
|
|
||||||
|
* Overlay approach with minimal duplication
|
||||||
|
|
||||||
|
Each namespace is link to a git repository this combination is what we call a
|
||||||
|
*Tenant* for this example we are going to take the example of 3 environments
|
||||||
|
dev, qa, prod:
|
||||||
|
|
||||||
|
We forward the `TENANT_NAMESPACE` variable holding the value of you tenant
|
||||||
|
name. This allows Kustomizations to dynamically target the right folder for
|
||||||
|
each environment.
|
||||||
|
|
||||||
|
Flux Kustomization e.g
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
||||||
|
kind: Kustomization
|
||||||
|
metadata:
|
||||||
|
name: my-app
|
||||||
|
namespace: ${TENANT_NAMESPACE}
|
||||||
|
spec:
|
||||||
|
interval: 5m
|
||||||
|
path: ./overlays/${TENANT_NAMESPACE}
|
||||||
|
prune: true
|
||||||
|
sourceRef:
|
||||||
|
kind: GitRepository
|
||||||
|
name: my-app-repo
|
||||||
|
```
|
||||||
|
If the tenant is my-app-dev then it will target the folder `./overlays/my-app-dev`
|
||||||
|
|
||||||
|
## Overlays Structure and Patterns
|
||||||
|
|
||||||
|
### Simple Patterns
|
||||||
|
We use the Kustomize overlays pattern to separate environment-specific
|
||||||
|
differences from shared configuration. Given our vanilla tenant folder.
|
||||||
|
|
||||||
|
```shell
|
||||||
|
(venv) euler@HAL:~/.../tenants/tenant-tpl(main)$ tree
|
||||||
|
.
|
||||||
|
├── config
|
||||||
|
│ ├── app
|
||||||
|
│ │ └── limits-ranges.yaml
|
||||||
|
│ └── ks.yaml
|
||||||
|
├── echo-server
|
||||||
|
│ ├── app
|
||||||
|
│ │ └── helmrelease.yaml
|
||||||
|
│ └── ks.yaml
|
||||||
|
├── kustomization.yaml
|
||||||
|
├── README.md
|
||||||
|
├── renovate.json5
|
||||||
|
├── repos
|
||||||
|
│ ├── helm
|
||||||
|
│ │ └── bjw-s.yaml
|
||||||
|
│ └── ks.yaml
|
||||||
|
├── scripts
|
||||||
|
│ └── rewrap-secrets.sh
|
||||||
|
└── vars
|
||||||
|
├── ks.yaml
|
||||||
|
└── tenant-tpl
|
||||||
|
├── example.yaml
|
||||||
|
└── README.md
|
||||||
|
```
|
||||||
|
I will in this example spawn two namespaces link to that git repository
|
||||||
|
*test-tpl* & *test-tpl-dev*
|
||||||
|
|
||||||
|
Query to check resources from flux perspective.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
$ flux get all -n tenant-tpl
|
||||||
|
NAME REVISION SUSPENDED READY MESSAGE
|
||||||
|
gitrepository/tenant-repos main@sha1:0fcffb6b False True stored artifact for revision 'main@sha1:0fcffb6b'
|
||||||
|
|
||||||
|
NAME REVISION SUSPENDED READY MESSAGE
|
||||||
|
helmrepository/bjw-s False True Helm repository is Ready
|
||||||
|
|
||||||
|
NAME REVISION SUSPENDED READY MESSAGE
|
||||||
|
helmchart/tenant-tpl-echo-server 3.2.1 False True pulled 'app-template' chart with version '3.2.1'
|
||||||
|
|
||||||
|
NAME REVISION SUSPENDED READY MESSAGE
|
||||||
|
helmrelease/echo-server 3.2.1 False True Helm install succeeded for release tenant-tpl/echo-server.v1 with chart app-template@3.2.1
|
||||||
|
|
||||||
|
NAME REVISION SUSPENDED READY MESSAGE
|
||||||
|
kustomization/echo-server main@sha1:0fcffb6b False True Applied revision: main@sha1:0fcffb6b
|
||||||
|
kustomization/repos-sync main@sha1:0fcffb6b False True Applied revision: main@sha1:0fcffb6b
|
||||||
|
kustomization/tenant-apps main@sha1:0fcffb6b False True Applied revision: main@sha1:0fcffb6b
|
||||||
|
kustomization/tenant-config main@sha1:0fcffb6b False True Applied revision: main@sha1:0fcffb6b
|
||||||
|
kustomization/vars main@sha1:0fcffb6b False True Applied revision: main@sha1:0fcffb6b
|
||||||
|
|
||||||
|
$ flux get all -n tenant-tpl-dev
|
||||||
|
NAME REVISION SUSPENDED READY MESSAGE
|
||||||
|
gitrepository/tenant-repos main@sha1:0fcffb6b False True stored artifact for revision 'main@sha1:0fcffb6b'
|
||||||
|
|
||||||
|
NAME REVISION SUSPENDED READY MESSAGE
|
||||||
|
helmrepository/bjw-s False True Helm repository is Ready
|
||||||
|
|
||||||
|
NAME REVISION SUSPENDED READY MESSAGE
|
||||||
|
helmchart/tenant-tpl-dev-echo-server 3.2.1 False True pulled 'app-template' chart with version '3.2.1'
|
||||||
|
|
||||||
|
NAME REVISION SUSPENDED READY MESSAGE
|
||||||
|
helmrelease/echo-server 3.2.1 False True Helm install succeeded for release tenant-tpl-dev/echo-server.v1 with chart app-template@3.2.1
|
||||||
|
|
||||||
|
NAME REVISION SUSPENDED READY MESSAGE
|
||||||
|
kustomization/echo-server main@sha1:0fcffb6b False True Applied revision: main@sha1:0fcffb6b
|
||||||
|
kustomization/repos-sync main@sha1:0fcffb6b False True Applied revision: main@sha1:0fcffb6b
|
||||||
|
kustomization/tenant-apps main@sha1:0fcffb6b False True Applied revision: main@sha1:0fcffb6b
|
||||||
|
kustomization/tenant-config main@sha1:0fcffb6b False True Applied revision: main@sha1:0fcffb6b
|
||||||
|
kustomization/vars False False kustomization path not found: stat /tmp/kustomization-1496015889/vars/tenant-tpl-dev: no such file or directory
|
||||||
|
```
|
||||||
|
|
||||||
|
As you can see they are similar and have both deployed the echo server. However
|
||||||
|
as you can see there is a slight difference with the `kustomzation/vars`. The
|
||||||
|
`dev` environements is complaining about some missing directory. Let's inspect
|
||||||
|
the vars kustomization
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
$ cat vars/ks.yaml
|
||||||
|
---
|
||||||
|
# yaml-language-server: $schema=https://kubernetes-schemas.pages.dev/kustomize.toolkit.fluxcd.io/kustomization_v1.json
|
||||||
|
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
||||||
|
kind: Kustomization
|
||||||
|
metadata:
|
||||||
|
name: &app vars
|
||||||
|
namespace: ${TENANT_NAMESPACE}
|
||||||
|
spec:
|
||||||
|
targetNamespace: ${TENANT_NAMESPACE}
|
||||||
|
commonMetadata:
|
||||||
|
labels:
|
||||||
|
app.kubernetes.io/name: *app
|
||||||
|
path: ./vars/${TENANT_NAMESPACE}
|
||||||
|
prune: true
|
||||||
|
sourceRef:
|
||||||
|
kind: GitRepository
|
||||||
|
name: tenant-repos
|
||||||
|
wait: false
|
||||||
|
interval: 10m
|
||||||
|
retryInterval: 1m
|
||||||
|
timeout: 5m
|
||||||
|
|
||||||
|
```
|
||||||
|
As you can see this kustomization is defining a lot of field with the value
|
||||||
|
`${TENANT_NAMESPACE}`. That's because the `/vars` folder is already
|
||||||
|
preconfigured to handle overlays patterns. It will target the right directory
|
||||||
|
according to the namespace name he's running on. If we check our git repository
|
||||||
|
structure:
|
||||||
|
|
||||||
|
```shell
|
||||||
|
(venv) euler@HAL:~/.../tenants/tenant-tpl(main)$ tree vars/
|
||||||
|
vars/
|
||||||
|
├── ks.yaml
|
||||||
|
└── tenant-tpl
|
||||||
|
├── example.yaml
|
||||||
|
└── README.md
|
||||||
|
|
||||||
|
2 directories, 3 files
|
||||||
|
```
|
||||||
|
|
||||||
|
According to the kustomization definition the path:
|
||||||
|
`./vars/${TENANT_NAMESPACE}` will not be found for the tenant `tenant-tpl-dev`
|
||||||
|
We are going to create the appropriate folder.
|
||||||
|
|
||||||
|
```shell
|
||||||
|
$ mkdir vars/tenant-tpl-dev
|
||||||
|
$ cp -r vars/tenant-tpl/example.yaml vars/tenant-tpl
|
||||||
|
tenant-tpl/ tenant-tpl-dev/
|
||||||
|
$ cp -r vars/tenant-tpl/example.yaml vars/tenant-tpl-dev/dev-example.yaml
|
||||||
|
$ vim vars/tenant-tpl-dev/dev-example.yaml
|
||||||
|
$ cat vars/tenant-tpl-dev/dev-example.yaml
|
||||||
|
---
|
||||||
|
apiVersion: v1
|
||||||
|
kind: ConfigMap
|
||||||
|
metadata:
|
||||||
|
name: example-vars-dev
|
||||||
|
data:
|
||||||
|
EXAMPLE: foo-dev
|
||||||
|
$ git commit -am 'Adding dev vars folder'
|
||||||
|
[main 07a29a5] Adding dev vars folder
|
||||||
|
1 file changed, 7 insertions(+)
|
||||||
|
create mode 100644 vars/tenant-tpl-dev/dev-example.yaml
|
||||||
|
$ git push
|
||||||
|
Enumerating objects: 7, done.
|
||||||
|
Counting objects: 100% (7/7), done.
|
||||||
|
Delta compression using up to 22 threads
|
||||||
|
Compressing objects: 100% (4/4), done.
|
||||||
|
Writing objects: 100% (5/5), 1.16 KiB | 1.16 MiB/s, done.
|
||||||
|
Total 5 (delta 1), reused 0 (delta 0), pack-reused 0
|
||||||
|
To ssh://git-ssh.kvant.cloud:2222/phoenix-oss/tenant-tpl.git
|
||||||
|
0fcffb6..07a29a5 main -> main
|
||||||
|
```
|
||||||
|
Verification
|
||||||
|
|
||||||
|
```shell
|
||||||
|
$ flux reconcile ks vars -n tenant-tpl-dev --with-source #One liner to force source update and reconciliation
|
||||||
|
► annotating GitRepository tenant-repos in tenant-tpl-dev namespace
|
||||||
|
✔ GitRepository annotated
|
||||||
|
◎ waiting for GitRepository reconciliation
|
||||||
|
✔ fetched revision main@sha1:07a29a5d22eae9be68fd9d9da141b3fa83f9f5d2
|
||||||
|
► annotating Kustomization vars in tenant-tpl-dev namespace
|
||||||
|
✔ Kustomization annotated
|
||||||
|
◎ waiting for Kustomization reconciliation
|
||||||
|
✔ applied revision main@sha1:07a29a5d22eae9be68fd9d9da141b3fa83f9f5d2
|
||||||
|
|
||||||
|
$ flux get all -n tenant-tpl-dev
|
||||||
|
NAME REVISION SUSPENDED READY MESSAGE
|
||||||
|
gitrepository/tenant-repos main@sha1:07a29a5d False True stored artifact for revision 'main@sha1:07a29a5d'
|
||||||
|
|
||||||
|
NAME REVISION SUSPENDED READY MESSAGE
|
||||||
|
helmrepository/bjw-s False True Helm repository is Ready
|
||||||
|
|
||||||
|
NAME REVISION SUSPENDED READY MESSAGE
|
||||||
|
helmchart/tenant-tpl-dev-echo-server 3.2.1 False True pulled 'app-template' chart with version '3.2.1'
|
||||||
|
|
||||||
|
NAME REVISION SUSPENDED READY MESSAGE
|
||||||
|
helmrelease/echo-server 3.2.1 False True Helm install succeeded for release tenant-tpl-dev/echo-server.v1 with chart app-template@3.2.1
|
||||||
|
|
||||||
|
NAME REVISION SUSPENDED READY MESSAGE
|
||||||
|
kustomization/echo-server main@sha1:07a29a5d False True Applied revision: main@sha1:07a29a5d
|
||||||
|
kustomization/repos-sync main@sha1:07a29a5d False True Applied revision: main@sha1:07a29a5d
|
||||||
|
kustomization/tenant-apps main@sha1:07a29a5d False True Applied revision: main@sha1:07a29a5d
|
||||||
|
kustomization/tenant-config main@sha1:07a29a5d False True Applied revision: main@sha1:07a29a5d
|
||||||
|
kustomization/vars main@sha1:07a29a5d False True Applied revision: main@sha1:07a29a5d
|
||||||
|
|
||||||
|
$ oc get configmap -n tenant-tpl-dev
|
||||||
|
NAME DATA AGE
|
||||||
|
example-vars-dev 1 3m32s
|
||||||
|
kube-root-ca.crt 1 36m
|
||||||
|
openshift-service-ca.crt 1 36m
|
||||||
|
tenant-settings 10 36m
|
||||||
|
```
|
||||||
|
|
||||||
|
Congratulations you now manage to handle simple overlays patterns and have the
|
||||||
|
possibility to handle multiple environments using one git repository.
|
||||||
|
|
||||||
|
|
||||||
|
### Advanced Patterns
|
||||||
|
|
||||||
|
#### Base
|
||||||
|
|
||||||
|
In the previous example we showed to you how to handle configmaps or secrets
|
||||||
|
to be loaded for a given environments. In this example we are going to create a
|
||||||
|
directory structure that allow us to *Have shared resources definition and
|
||||||
|
environments specific one.
|
||||||
|
|
||||||
|
Taking back our echo-server example.
|
||||||
|
|
||||||
|
XXX
|
||||||
|
|
||||||
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue