Initiate documentation for gitops patterns
parent
22f9a8280a
commit
efcff670d5
GPG key ID:
2681291F9206E471
1 changed files with 257 additions and 0 deletions
257
Guides/Repository-Handling-Multiple-Environments.md
Normal file
257
Guides/Repository-Handling-Multiple-Environments.md
Normal file
|
|
@ -0,0 +1,257 @@
|
|||
# GitOps-Ready Namespaces with Flux
|
||||
## Overview
|
||||
|
||||
In our OpenShift clusters, namespaces are provisioned GitOps-ready by default.
|
||||
Each namespace is linked to a Git repository where configurations are stored and managed declaratively.
|
||||
|
||||
This setup enables:
|
||||
|
||||
* Consistent GitOps workflow across tenants
|
||||
|
||||
* Multi-environment support from a single repo
|
||||
|
||||
* Overlay approach with minimal duplication
|
||||
|
||||
Each namespace is link to a git repository this combination is what we call a
|
||||
*Tenant* for this example we are going to take the example of 3 environments
|
||||
dev, qa, prod:
|
||||
|
||||
We forward the `TENANT_NAMESPACE` variable holding the value of you tenant
|
||||
name. This allows Kustomizations to dynamically target the right folder for
|
||||
each environment.
|
||||
|
||||
Flux Kustomization e.g
|
||||
|
||||
```yaml
|
||||
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
||||
kind: Kustomization
|
||||
metadata:
|
||||
name: my-app
|
||||
namespace: ${TENANT_NAMESPACE}
|
||||
spec:
|
||||
interval: 5m
|
||||
path: ./overlays/${TENANT_NAMESPACE}
|
||||
prune: true
|
||||
sourceRef:
|
||||
kind: GitRepository
|
||||
name: my-app-repo
|
||||
```
|
||||
If the tenant is my-app-dev then it will target the folder `./overlays/my-app-dev`
|
||||
|
||||
## Overlays Structure and Patterns
|
||||
|
||||
### Simple Patterns
|
||||
We use the Kustomize overlays pattern to separate environment-specific
|
||||
differences from shared configuration. Given our vanilla tenant folder.
|
||||
|
||||
```shell
|
||||
(venv) euler@HAL:~/.../tenants/tenant-tpl(main)$ tree
|
||||
.
|
||||
├── config
|
||||
│ ├── app
|
||||
│ │ └── limits-ranges.yaml
|
||||
│ └── ks.yaml
|
||||
├── echo-server
|
||||
│ ├── app
|
||||
│ │ └── helmrelease.yaml
|
||||
│ └── ks.yaml
|
||||
├── kustomization.yaml
|
||||
├── README.md
|
||||
├── renovate.json5
|
||||
├── repos
|
||||
│ ├── helm
|
||||
│ │ └── bjw-s.yaml
|
||||
│ └── ks.yaml
|
||||
├── scripts
|
||||
│ └── rewrap-secrets.sh
|
||||
└── vars
|
||||
├── ks.yaml
|
||||
└── tenant-tpl
|
||||
├── example.yaml
|
||||
└── README.md
|
||||
```
|
||||
I will in this example spawn two namespaces link to that git repository
|
||||
*test-tpl* & *test-tpl-dev*
|
||||
|
||||
Query to check resources from flux perspective.
|
||||
|
||||
```bash
|
||||
$ flux get all -n tenant-tpl
|
||||
NAME REVISION SUSPENDED READY MESSAGE
|
||||
gitrepository/tenant-repos main@sha1:0fcffb6b False True stored artifact for revision 'main@sha1:0fcffb6b'
|
||||
|
||||
NAME REVISION SUSPENDED READY MESSAGE
|
||||
helmrepository/bjw-s False True Helm repository is Ready
|
||||
|
||||
NAME REVISION SUSPENDED READY MESSAGE
|
||||
helmchart/tenant-tpl-echo-server 3.2.1 False True pulled 'app-template' chart with version '3.2.1'
|
||||
|
||||
NAME REVISION SUSPENDED READY MESSAGE
|
||||
helmrelease/echo-server 3.2.1 False True Helm install succeeded for release tenant-tpl/echo-server.v1 with chart app-template@3.2.1
|
||||
|
||||
NAME REVISION SUSPENDED READY MESSAGE
|
||||
kustomization/echo-server main@sha1:0fcffb6b False True Applied revision: main@sha1:0fcffb6b
|
||||
kustomization/repos-sync main@sha1:0fcffb6b False True Applied revision: main@sha1:0fcffb6b
|
||||
kustomization/tenant-apps main@sha1:0fcffb6b False True Applied revision: main@sha1:0fcffb6b
|
||||
kustomization/tenant-config main@sha1:0fcffb6b False True Applied revision: main@sha1:0fcffb6b
|
||||
kustomization/vars main@sha1:0fcffb6b False True Applied revision: main@sha1:0fcffb6b
|
||||
|
||||
$ flux get all -n tenant-tpl-dev
|
||||
NAME REVISION SUSPENDED READY MESSAGE
|
||||
gitrepository/tenant-repos main@sha1:0fcffb6b False True stored artifact for revision 'main@sha1:0fcffb6b'
|
||||
|
||||
NAME REVISION SUSPENDED READY MESSAGE
|
||||
helmrepository/bjw-s False True Helm repository is Ready
|
||||
|
||||
NAME REVISION SUSPENDED READY MESSAGE
|
||||
helmchart/tenant-tpl-dev-echo-server 3.2.1 False True pulled 'app-template' chart with version '3.2.1'
|
||||
|
||||
NAME REVISION SUSPENDED READY MESSAGE
|
||||
helmrelease/echo-server 3.2.1 False True Helm install succeeded for release tenant-tpl-dev/echo-server.v1 with chart app-template@3.2.1
|
||||
|
||||
NAME REVISION SUSPENDED READY MESSAGE
|
||||
kustomization/echo-server main@sha1:0fcffb6b False True Applied revision: main@sha1:0fcffb6b
|
||||
kustomization/repos-sync main@sha1:0fcffb6b False True Applied revision: main@sha1:0fcffb6b
|
||||
kustomization/tenant-apps main@sha1:0fcffb6b False True Applied revision: main@sha1:0fcffb6b
|
||||
kustomization/tenant-config main@sha1:0fcffb6b False True Applied revision: main@sha1:0fcffb6b
|
||||
kustomization/vars False False kustomization path not found: stat /tmp/kustomization-1496015889/vars/tenant-tpl-dev: no such file or directory
|
||||
```
|
||||
|
||||
As you can see they are similar and have both deployed the echo server. However
|
||||
as you can see there is a slight difference with the `kustomzation/vars`. The
|
||||
`dev` environements is complaining about some missing directory. Let's inspect
|
||||
the vars kustomization
|
||||
|
||||
```yaml
|
||||
$ cat vars/ks.yaml
|
||||
---
|
||||
# yaml-language-server: $schema=https://kubernetes-schemas.pages.dev/kustomize.toolkit.fluxcd.io/kustomization_v1.json
|
||||
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
||||
kind: Kustomization
|
||||
metadata:
|
||||
name: &app vars
|
||||
namespace: ${TENANT_NAMESPACE}
|
||||
spec:
|
||||
targetNamespace: ${TENANT_NAMESPACE}
|
||||
commonMetadata:
|
||||
labels:
|
||||
app.kubernetes.io/name: *app
|
||||
path: ./vars/${TENANT_NAMESPACE}
|
||||
prune: true
|
||||
sourceRef:
|
||||
kind: GitRepository
|
||||
name: tenant-repos
|
||||
wait: false
|
||||
interval: 10m
|
||||
retryInterval: 1m
|
||||
timeout: 5m
|
||||
|
||||
```
|
||||
As you can see this kustomization is defining a lot of field with the value
|
||||
`${TENANT_NAMESPACE}`. That's because the `/vars` folder is already
|
||||
preconfigured to handle overlays patterns. It will target the right directory
|
||||
according to the namespace name he's running on. If we check our git repository
|
||||
structure:
|
||||
|
||||
```shell
|
||||
(venv) euler@HAL:~/.../tenants/tenant-tpl(main)$ tree vars/
|
||||
vars/
|
||||
├── ks.yaml
|
||||
└── tenant-tpl
|
||||
├── example.yaml
|
||||
└── README.md
|
||||
|
||||
2 directories, 3 files
|
||||
```
|
||||
|
||||
According to the kustomization definition the path:
|
||||
`./vars/${TENANT_NAMESPACE}` will not be found for the tenant `tenant-tpl-dev`
|
||||
We are going to create the appropriate folder.
|
||||
|
||||
```shell
|
||||
$ mkdir vars/tenant-tpl-dev
|
||||
$ cp -r vars/tenant-tpl/example.yaml vars/tenant-tpl
|
||||
tenant-tpl/ tenant-tpl-dev/
|
||||
$ cp -r vars/tenant-tpl/example.yaml vars/tenant-tpl-dev/dev-example.yaml
|
||||
$ vim vars/tenant-tpl-dev/dev-example.yaml
|
||||
$ cat vars/tenant-tpl-dev/dev-example.yaml
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: example-vars-dev
|
||||
data:
|
||||
EXAMPLE: foo-dev
|
||||
$ git commit -am 'Adding dev vars folder'
|
||||
[main 07a29a5] Adding dev vars folder
|
||||
1 file changed, 7 insertions(+)
|
||||
create mode 100644 vars/tenant-tpl-dev/dev-example.yaml
|
||||
$ git push
|
||||
Enumerating objects: 7, done.
|
||||
Counting objects: 100% (7/7), done.
|
||||
Delta compression using up to 22 threads
|
||||
Compressing objects: 100% (4/4), done.
|
||||
Writing objects: 100% (5/5), 1.16 KiB | 1.16 MiB/s, done.
|
||||
Total 5 (delta 1), reused 0 (delta 0), pack-reused 0
|
||||
To ssh://git-ssh.kvant.cloud:2222/phoenix-oss/tenant-tpl.git
|
||||
0fcffb6..07a29a5 main -> main
|
||||
```
|
||||
Verification
|
||||
|
||||
```shell
|
||||
$ flux reconcile ks vars -n tenant-tpl-dev --with-source #One liner to force source update and reconciliation
|
||||
► annotating GitRepository tenant-repos in tenant-tpl-dev namespace
|
||||
✔ GitRepository annotated
|
||||
◎ waiting for GitRepository reconciliation
|
||||
✔ fetched revision main@sha1:07a29a5d22eae9be68fd9d9da141b3fa83f9f5d2
|
||||
► annotating Kustomization vars in tenant-tpl-dev namespace
|
||||
✔ Kustomization annotated
|
||||
◎ waiting for Kustomization reconciliation
|
||||
✔ applied revision main@sha1:07a29a5d22eae9be68fd9d9da141b3fa83f9f5d2
|
||||
|
||||
$ flux get all -n tenant-tpl-dev
|
||||
NAME REVISION SUSPENDED READY MESSAGE
|
||||
gitrepository/tenant-repos main@sha1:07a29a5d False True stored artifact for revision 'main@sha1:07a29a5d'
|
||||
|
||||
NAME REVISION SUSPENDED READY MESSAGE
|
||||
helmrepository/bjw-s False True Helm repository is Ready
|
||||
|
||||
NAME REVISION SUSPENDED READY MESSAGE
|
||||
helmchart/tenant-tpl-dev-echo-server 3.2.1 False True pulled 'app-template' chart with version '3.2.1'
|
||||
|
||||
NAME REVISION SUSPENDED READY MESSAGE
|
||||
helmrelease/echo-server 3.2.1 False True Helm install succeeded for release tenant-tpl-dev/echo-server.v1 with chart app-template@3.2.1
|
||||
|
||||
NAME REVISION SUSPENDED READY MESSAGE
|
||||
kustomization/echo-server main@sha1:07a29a5d False True Applied revision: main@sha1:07a29a5d
|
||||
kustomization/repos-sync main@sha1:07a29a5d False True Applied revision: main@sha1:07a29a5d
|
||||
kustomization/tenant-apps main@sha1:07a29a5d False True Applied revision: main@sha1:07a29a5d
|
||||
kustomization/tenant-config main@sha1:07a29a5d False True Applied revision: main@sha1:07a29a5d
|
||||
kustomization/vars main@sha1:07a29a5d False True Applied revision: main@sha1:07a29a5d
|
||||
|
||||
$ oc get configmap -n tenant-tpl-dev
|
||||
NAME DATA AGE
|
||||
example-vars-dev 1 3m32s
|
||||
kube-root-ca.crt 1 36m
|
||||
openshift-service-ca.crt 1 36m
|
||||
tenant-settings 10 36m
|
||||
```
|
||||
|
||||
Congratulations you now manage to handle simple overlays patterns and have the
|
||||
possibility to handle multiple environments using one git repository.
|
||||
|
||||
|
||||
### Advanced Patterns
|
||||
|
||||
#### Base
|
||||
|
||||
In the previous example we showed to you how to handle configmaps or secrets
|
||||
to be loaded for a given environments. In this example we are going to create a
|
||||
directory structure that allow us to *Have shared resources definition and
|
||||
environments specific one.
|
||||
|
||||
Taking back our echo-server example.
|
||||
|
||||
XXX
|
||||
|
||||
|
||||
Loading…
Add table
Add a link
Reference in a new issue