Adding base overlays patterns
parent
327de5c69b
commit
d010a5595f
GPG key ID:
2681291F9206E471
1 changed files with 248 additions and 2 deletions
|
|
@ -250,8 +250,254 @@ to be loaded for a given environments. In this example we are going to create a
|
|||
directory structure that allow us to *Have shared resources definition and
|
||||
environments specific one.
|
||||
|
||||
Taking back our echo-server example.
|
||||
Taking back our echo-server example we are going to show how spawn different
|
||||
resources definitions base on the environments. Here the diagram of what we are
|
||||
trying to achieve.
|
||||
|
||||
XXX
|
||||
```mermaid
|
||||
flowchart TD
|
||||
%% Node styles
|
||||
classDef file fill:#ffffff,stroke:#9CA3AF,stroke-width:1px,color:#374151,rounded:10px;
|
||||
classDef var fill:#FEF9C3,stroke:#F59E0B,stroke-width:1px,color:#78350F,rounded:8px;
|
||||
classDef overlay fill:#DCFCE7,stroke:#22C55E,stroke-width:2px,color:#166534,rounded:10px;
|
||||
classDef base fill:#E0F2FE,stroke:#3B82F6,stroke-width:2px,color:#1E40AF,rounded:10px;
|
||||
classDef flux fill:#F3E8FF,stroke:#7C3AED,stroke-width:2px,color:#4C1D95,rounded:10px;
|
||||
|
||||
%% Git repository
|
||||
subgraph GitRepo["📂 Git Repository"]
|
||||
subgraph Base["Base (shared resources)"]
|
||||
D1["Deployment.yaml"]:::file
|
||||
HR["HelmRelease.yaml"]:::file
|
||||
KB["kustomization.yaml"]:::base
|
||||
end
|
||||
|
||||
subgraph Overlays["Overlays (tenant-specific)"]
|
||||
subgraph Dev["🟢 Dev Overlay"]
|
||||
P1["patch-replicas.yaml"]:::file
|
||||
KD["kustomization.yaml"]:::overlay
|
||||
end
|
||||
subgraph Prod["🔴 Prod Overlay"]
|
||||
P2["patch-resources.yaml"]:::file
|
||||
KP["kustomization.yaml"]:::overlay
|
||||
end
|
||||
end
|
||||
end
|
||||
|
||||
%% Flux Kustomization
|
||||
subgraph Flux["⚡ FluxCD"]
|
||||
KOverlay["Kustomization CR → overlays/${TENANT_NAMESPACE}"]:::flux
|
||||
end
|
||||
|
||||
%% Kubernetes Cluster
|
||||
subgraph Cluster["🖥️ Kubernetes Cluster"]
|
||||
NS["${TENANT_NAMESPACE} namespace"]:::var
|
||||
APP["myapp resources"]:::file
|
||||
end
|
||||
|
||||
%% Connections
|
||||
KOverlay --> Dev
|
||||
KOverlay --> Prod
|
||||
Dev --> Base
|
||||
Prod --> Base
|
||||
Dev --> APP
|
||||
Prod --> APP
|
||||
APP --> NS
|
||||
```
|
||||
In the given example we are going on the tenant-tpl-dev increase the number of
|
||||
replicas for the echo server. While we change the resources value for the
|
||||
production. This structure allow us to avoid duplicating the helmrelease.yaml
|
||||
base definition and only modify the value we want for base on each environment.
|
||||
|
||||
We end up with that structure for our echo-server.
|
||||
|
||||
```shell
|
||||
tenant-tpl/echo-server(main)$ tree
|
||||
.
|
||||
├── base
|
||||
│ ├── helmrelease.yaml
|
||||
│ └── kustomization.yaml
|
||||
├── ks.yaml
|
||||
└── overlays
|
||||
├── tenant-tpl
|
||||
│ ├── kustomization.yaml
|
||||
│ └── patch-resources.yaml
|
||||
└── tenant-tpl-dev
|
||||
├── kustomization.yaml
|
||||
└── patch-replicas.yaml
|
||||
```
|
||||
##### Analysis and proof
|
||||
Our goal was to define a main helmrelease for the echo server and change some
|
||||
parameters according to the environments. We increase the number of replicas
|
||||
for dev and change the amount of resources for prod.
|
||||
|
||||
First look to the flux side to proof reconciliation and versioning.
|
||||
|
||||
*tenant-tpl*
|
||||
```shell
|
||||
(venv) euler@HAL:~/.../tenant-tpl/echo-server(main)$ flux get all -n tenant-tpl
|
||||
NAME REVISION SUSPENDED READY MESSAGE
|
||||
gitrepository/tenant-repos main@sha1:3f1afa0f False True stored artifact for revision 'main@sha1:3f1afa0f'
|
||||
|
||||
NAME REVISION SUSPENDED READY MESSAGE
|
||||
helmrepository/bjw-s False True Helm repository is Ready
|
||||
|
||||
NAME REVISION SUSPENDED READY MESSAGE
|
||||
helmchart/tenant-tpl-echo-server 3.2.1 False True pulled 'app-template' chart with version '3.2.1'
|
||||
|
||||
NAME REVISION SUSPENDED READY MESSAGE
|
||||
helmrelease/echo-server 3.2.1 False True Helm upgrade succeeded for release tenant-tpl/echo-server.v2 with chart app-template@3.2.1
|
||||
|
||||
NAME REVISION SUSPENDED READY MESSAGE
|
||||
kustomization/echo-server main@sha1:3f1afa0f False True Applied revision: main@sha1:3f1afa0f
|
||||
kustomization/repos-sync main@sha1:3f1afa0f False True Applied revision: main@sha1:3f1afa0f
|
||||
kustomization/tenant-apps main@sha1:3f1afa0f False True Applied revision: main@sha1:3f1afa0f
|
||||
kustomization/tenant-config main@sha1:3f1afa0f False True Applied revision: main@sha1:3f1afa0f
|
||||
kustomization/vars main@sha1:3f1afa0f False True Applied revision: main@sha1:3f1afa0f
|
||||
```
|
||||
|
||||
*tenant-tpl-dev*
|
||||
|
||||
```shell
|
||||
(venv) euler@HAL:~/.../tenant-tpl/echo-server(main)$ flux get all -n tenant-tpl-dev
|
||||
NAME REVISION SUSPENDED READY MESSAGE
|
||||
gitrepository/tenant-repos main@sha1:3f1afa0f False True stored artifact for revision 'main@sha1:3f1afa0f'
|
||||
|
||||
NAME REVISION SUSPENDED READY MESSAGE
|
||||
helmrepository/bjw-s False True Helm repository is Ready
|
||||
|
||||
NAME REVISION SUSPENDED READY MESSAGE
|
||||
helmchart/tenant-tpl-dev-echo-server 3.2.1 False True pulled 'app-template' chart with version '3.2.1'
|
||||
|
||||
NAME REVISION SUSPENDED READY MESSAGE
|
||||
helmrelease/echo-server 3.2.1 False True Helm upgrade succeeded for release tenant-tpl-dev/echo-server.v2 with chart app-template@3.2.1
|
||||
|
||||
NAME REVISION SUSPENDED READY MESSAGE
|
||||
kustomization/echo-server main@sha1:3f1afa0f False True Applied revision: main@sha1:3f1afa0f
|
||||
kustomization/repos-sync main@sha1:3f1afa0f False True Applied revision: main@sha1:3f1afa0f
|
||||
kustomization/tenant-apps main@sha1:3f1afa0f False True Applied revision: main@sha1:3f1afa0f
|
||||
kustomization/tenant-config main@sha1:3f1afa0f False True Applied revision: main@sha1:3f1afa0f
|
||||
kustomization/vars main@sha1:3f1afa0f False True Applied revision: main@sha1:3f1afa0f
|
||||
```
|
||||
|
||||
Both are at the same versioning on `main@sha1:3f1afa0f'` Now inspecting the
|
||||
echo-server helmrelease.
|
||||
|
||||
```shell
|
||||
$ flux trace hr echo-server -n tenant-tpl-dev
|
||||
|
||||
Object: HelmRelease/echo-server
|
||||
Namespace: tenant-tpl-dev
|
||||
Status: Managed by Flux
|
||||
---
|
||||
Kustomization: echo-server
|
||||
Namespace: tenant-tpl-dev
|
||||
Target: tenant-tpl-dev
|
||||
Path: ./echo-server/overlays/tenant-tpl-dev
|
||||
Revision: main@sha1:3f1afa0f19882e2c6acdf318b6a6d0195e24d046
|
||||
Status: Last reconciled at 2025-09-15 13:13:04 +0200 CEST
|
||||
Message: Applied revision: main@sha1:3f1afa0f19882e2c6acdf318b6a6d0195e24d046
|
||||
---
|
||||
GitRepository: tenant-repos
|
||||
Namespace: tenant-tpl-dev
|
||||
URL: https://git.kvant.cloud/phoenix-oss/tenant-tpl
|
||||
Branch: main
|
||||
Revision: main@sha1:3f1afa0f19882e2c6acdf318b6a6d0195e24d046
|
||||
Status: Last reconciled at 2025-09-15 13:12:46 +0200 CEST
|
||||
Message: stored artifact for revision 'main@sha1:3f1afa0f19882e2c6acdf318b6a6d0195e24d046'
|
||||
|
||||
$ flux trace hr echo-server -n tenant-tpl
|
||||
|
||||
Object: HelmRelease/echo-server
|
||||
Namespace: tenant-tpl
|
||||
Status: Managed by Flux
|
||||
---
|
||||
Kustomization: echo-server
|
||||
Namespace: tenant-tpl
|
||||
Target: tenant-tpl
|
||||
Path: ./echo-server/overlays/tenant-tpl
|
||||
Revision: main@sha1:3f1afa0f19882e2c6acdf318b6a6d0195e24d046
|
||||
Status: Last reconciled at 2025-09-15 13:12:21 +0200 CEST
|
||||
Message: Applied revision: main@sha1:3f1afa0f19882e2c6acdf318b6a6d0195e24d046
|
||||
---
|
||||
GitRepository: tenant-repos
|
||||
Namespace: tenant-tpl
|
||||
URL: https://git.kvant.cloud/phoenix-oss/tenant-tpl
|
||||
Branch: main
|
||||
Revision: main@sha1:3f1afa0f19882e2c6acdf318b6a6d0195e24d046
|
||||
Status: Last reconciled at 2025-09-15 13:12:07 +0200 CEST
|
||||
Message: stored artifact for revision 'main@sha1:3f1afa0f19882e2c6acdf318b6a6d0195e24d046'
|
||||
|
||||
```
|
||||
As you can see in the `path` hold a different value based on the environments.
|
||||
Thanks to our main kustomization that defined the path using
|
||||
`${TENANT_NAMESPACE}` value. Each overlays include the base but applied a patch
|
||||
on it before sending it to kube.
|
||||
|
||||
Let's verify that our patch are correctly applied. On dev we wanted 4 replicas.
|
||||
|
||||
|
||||
```shell
|
||||
$ oc get pods -n tenant-tpl-dev
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
echo-server-5bd6b558d6-4nngj 1/1 Running 0 10m
|
||||
echo-server-5bd6b558d6-65zlb 1/1 Running 0 10m
|
||||
echo-server-5bd6b558d6-ks2fq 1/1 Running 0 3d19h
|
||||
echo-server-5bd6b558d6-mz2wf 1/1 Running 0 3d19h
|
||||
echo-server-5bd6b558d6-rm64f 1/1 Running 0 10m
|
||||
```
|
||||
4 replicas <span style="color:green;">✔</span>
|
||||
|
||||
On tenant-tpl we wanted to increase the resources.
|
||||
|
||||
*DEV*
|
||||
```shell
|
||||
|
||||
$ oc get pods -n tenant-tpl-dev -o jsonpath='{range .items[*]}{.metadata.name}{"\n"}{range .spec.containers[*]}{.name}{" CPU Requests: "}{.resources.requests.cpu}{" Memory Requests: "}{.resources.requests.memory}{"\n"}{end}{end}'
|
||||
echo-server-5bd6b558d6-4nngj
|
||||
app CPU Requests: 10m Memory Requests: 64Mi
|
||||
echo-server-5bd6b558d6-65zlb
|
||||
app CPU Requests: 10m Memory Requests: 64Mi
|
||||
echo-server-5bd6b558d6-ks2fq
|
||||
app CPU Requests: 10m Memory Requests: 64Mi
|
||||
echo-server-5bd6b558d6-mz2wf
|
||||
app CPU Requests: 10m Memory Requests: 64Mi
|
||||
echo-server-5bd6b558d6-rm64f
|
||||
app CPU Requests: 10m Memory Requests: 64Mi
|
||||
```
|
||||
*PROD*
|
||||
|
||||
```shell
|
||||
$ oc get pods -n tenant-tpl -o jsonpath='{range .items[*]}{.metadata.name}{"\n"}{range .spec.containers[*]}{.name}{" CPU Requests: "}{.resources.requests.cpu}{" Memory Requests: "}{.resources.requests.memory}{"\n"}{end}{end}'
|
||||
echo-server-6cc5465c7d-5kg5r
|
||||
app CPU Requests: 100m Memory Requests: 128Mi
|
||||
echo-server-6cc5465c7d-dsmb6
|
||||
app CPU Requests: 100m Memory Requests: 128Mi
|
||||
|
||||
```
|
||||
|
||||
We confirm that we have different resources. Our patch properly work and we now
|
||||
achieve to have a comon base and patching fields according to the environments
|
||||
<span style="color:green;">✔</span>
|
||||
|
||||
### Find the file reference:
|
||||
|
||||
## Base
|
||||
- [helmrelease.yaml](https://git.kvant.cloud/phoenix-oss/tenant-tpl/-/blob/main/echo-server/base/helmrelease.yaml)
|
||||
- [kustomization.yaml](https://git.kvant.cloud/phoenix-oss/tenant-tpl/-/blob/main/echo-server/base/kustomization.yaml)
|
||||
|
||||
## Main Kustomization
|
||||
- [ks.yaml](https://git.kvant.cloud/phoenix-oss/tenant-tpl/-/blob/main/echo-server/ks.yaml)
|
||||
|
||||
## Overlays
|
||||
|
||||
### tenant-tpl
|
||||
- [kustomization.yaml](https://git.kvant.cloud/phoenix-oss/tenant-tpl/-/blob/main/echo-server/overlays/tenant-tpl/kustomization.yaml)
|
||||
- [patch-resources.yaml](https://git.kvant.cloud/phoenix-oss/tenant-tpl/-/blob/main/echo-server/overlays/tenant-tpl/patch-resources.yaml)
|
||||
|
||||
### tenant-tpl-dev
|
||||
- [kustomization.yaml](https://git.kvant.cloud/phoenix-oss/tenant-tpl/-/blob/main/echo-server/overlays/tenant-tpl-dev/kustomization.yaml)
|
||||
- [patch-replicas.yaml](https://git.kvant.cloud/phoenix-oss/tenant-tpl/-/blob/main/echo-server/overlays/tenant-tpl-dev/patch-replicas.yaml)
|
||||
|
||||
### Link Reference
|
||||
- [FLUX Repository Structure ](https://fluxcd.io/flux/guides/repository-structure/#repository-structure-2)
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue