docs: add guide for deploying app repos to the cluster
Document the recommended two-repo model for application delivery, including Flux attachment objects, Doppler/ExternalSecret wiring, Tailscale service exposure, and the steps for enabling the suspended apps layer.
This commit is contained in:
@@ -0,0 +1,287 @@
|
||||
# App Repo Deployment Guide
|
||||
|
||||
This guide explains the recommended way to deploy an application to this cluster.
|
||||
|
||||
## Recommended Model
|
||||
|
||||
Use two repos:
|
||||
|
||||
- `HetznerTerra` (this repo): cluster, addons, shared infrastructure, Flux wiring
|
||||
- `your-app-repo`: application source, Dockerfile, CI, Kubernetes manifests or Helm chart
|
||||
|
||||
Why:
|
||||
|
||||
- cluster lifecycle stays separate from app code
|
||||
- app CI can build and tag images independently
|
||||
- this repo remains the source of truth for what the cluster is allowed to deploy
|
||||
|
||||
## Current Cluster Assumptions
|
||||
|
||||
- Flux is already installed and reconciles this repo from `main`
|
||||
- `clusters/prod/flux-system/kustomization-apps.yaml` points at `./apps`
|
||||
- `apps` is suspended by default
|
||||
- private access is through Tailscale
|
||||
- runtime secrets should come from Doppler via External Secrets
|
||||
|
||||
## Deployment Options
|
||||
|
||||
### Option A: Separate app repo
|
||||
|
||||
Recommended for most real applications.
|
||||
|
||||
Flow:
|
||||
|
||||
1. App repo builds and pushes an image.
|
||||
2. This repo defines a `GitRepository` pointing at the app repo.
|
||||
3. This repo defines a `Kustomization` pointing at a path in the app repo.
|
||||
4. Flux pulls the app repo and applies the manifests.
|
||||
|
||||
### Option B: In-repo app manifests
|
||||
|
||||
Only use this when the application is tiny or tightly coupled to the platform.
|
||||
|
||||
Flow:
|
||||
|
||||
1. Put Kubernetes manifests directly under `apps/` in this repo.
|
||||
2. Unsuspend the top-level `apps` Kustomization.
|
||||
|
||||
This is simpler, but mixes platform and app changes together.
|
||||
|
||||
## App Repo Structure
|
||||
|
||||
Suggested layout:
|
||||
|
||||
```text
|
||||
your-app-repo/
|
||||
├── src/
|
||||
├── Dockerfile
|
||||
├── .gitea/workflows/
|
||||
└── deploy/
|
||||
├── base/
|
||||
│ ├── namespace.yaml
|
||||
│ ├── deployment.yaml
|
||||
│ ├── service.yaml
|
||||
│ ├── externalsecret.yaml
|
||||
│ └── kustomization.yaml
|
||||
└── prod/
|
||||
├── kustomization.yaml
|
||||
└── patch-*.yaml
|
||||
```
|
||||
|
||||
If you prefer Helm, replace `deploy/base` and `deploy/prod` with a chart path and point Flux at that instead.
|
||||
|
||||
## What the App Repo Should Own
|
||||
|
||||
- application source code
|
||||
- image build pipeline
|
||||
- image tag strategy
|
||||
- Deployment / Service / Ingress or Tailscale-facing Service manifests
|
||||
- app-specific `ExternalSecret` manifests
|
||||
- app-specific namespace
|
||||
|
||||
## What This Repo Should Own
|
||||
|
||||
- cluster-level permission to deploy the app
|
||||
- the `GitRepository` and top-level `Kustomization` that attach the app repo to the cluster
|
||||
- whether the `apps` layer is suspended or active
|
||||
|
||||
## Recommended First App Integration
|
||||
|
||||
In this repo, add Flux objects under `apps/` that point to the app repo.
|
||||
|
||||
Example files to add:
|
||||
|
||||
- `apps/gitrepository-my-app.yaml`
|
||||
- `apps/kustomization-my-app.yaml`
|
||||
- update `apps/kustomization.yaml`
|
||||
|
||||
Example `apps/gitrepository-my-app.yaml`:
|
||||
|
||||
```yaml
|
||||
apiVersion: source.toolkit.fluxcd.io/v1
|
||||
kind: GitRepository
|
||||
metadata:
|
||||
name: my-app
|
||||
namespace: flux-system
|
||||
spec:
|
||||
interval: 1m
|
||||
ref:
|
||||
branch: main
|
||||
secretRef:
|
||||
name: flux-system
|
||||
url: ssh://git@<your-git-host>:<port>/<org>/<your-app-repo>.git
|
||||
```
|
||||
|
||||
Example `apps/kustomization-my-app.yaml`:
|
||||
|
||||
```yaml
|
||||
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
||||
kind: Kustomization
|
||||
metadata:
|
||||
name: my-app
|
||||
namespace: flux-system
|
||||
spec:
|
||||
interval: 10m
|
||||
prune: true
|
||||
sourceRef:
|
||||
kind: GitRepository
|
||||
name: my-app
|
||||
path: ./deploy/prod
|
||||
wait: true
|
||||
timeout: 5m
|
||||
dependsOn:
|
||||
- name: infrastructure
|
||||
```
|
||||
|
||||
Then update `apps/kustomization.yaml`:
|
||||
|
||||
```yaml
|
||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
resources:
|
||||
- gitrepository-my-app.yaml
|
||||
- kustomization-my-app.yaml
|
||||
```
|
||||
|
||||
## App Secrets
|
||||
|
||||
Recommended path:
|
||||
|
||||
1. Put runtime values in Doppler.
|
||||
2. In the app manifests, create an `ExternalSecret` that reads from `doppler-hetznerterra`.
|
||||
3. Reference the resulting Kubernetes Secret from the Deployment.
|
||||
|
||||
Example app-side `ExternalSecret`:
|
||||
|
||||
```yaml
|
||||
apiVersion: external-secrets.io/v1
|
||||
kind: ExternalSecret
|
||||
metadata:
|
||||
name: my-app-env
|
||||
namespace: my-app
|
||||
spec:
|
||||
refreshInterval: 1h
|
||||
secretStoreRef:
|
||||
name: doppler-hetznerterra
|
||||
kind: ClusterSecretStore
|
||||
target:
|
||||
name: my-app-env
|
||||
creationPolicy: Owner
|
||||
data:
|
||||
- secretKey: DATABASE_URL
|
||||
remoteRef:
|
||||
key: MY_APP_DATABASE_URL
|
||||
```
|
||||
|
||||
## Image Delivery
|
||||
|
||||
Recommended flow:
|
||||
|
||||
1. App repo CI builds a container image.
|
||||
2. CI pushes it to a registry.
|
||||
3. The app repo updates the Kubernetes image tag in `deploy/prod`.
|
||||
4. Flux notices the Git change and deploys it.
|
||||
|
||||
Keep the first version simple. Do not add image automation until the basic deploy path is proven.
|
||||
|
||||
## Exposing the App
|
||||
|
||||
Pick one:
|
||||
|
||||
### Private app over Tailscale
|
||||
|
||||
Best fit for this cluster right now.
|
||||
|
||||
Create a Service like the existing Rancher/Grafana/Prometheus pattern:
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: my-app-tailscale
|
||||
namespace: my-app
|
||||
annotations:
|
||||
tailscale.com/hostname: my-app
|
||||
tailscale.com/tags: "tag:prod"
|
||||
tailscale.com/proxy-class: infra-stable
|
||||
spec:
|
||||
type: LoadBalancer
|
||||
loadBalancerClass: tailscale
|
||||
selector:
|
||||
app.kubernetes.io/name: my-app
|
||||
ports:
|
||||
- name: http
|
||||
port: 80
|
||||
protocol: TCP
|
||||
targetPort: 3000
|
||||
```
|
||||
|
||||
Use `http://my-app.<your-tailnet>` or your chosen hostname.
|
||||
|
||||
### Cluster-internal only
|
||||
|
||||
Create only a `ClusterIP` Service.
|
||||
|
||||
### Public ingress
|
||||
|
||||
Not recommended as the first app path in this repo. Get the private path working first.
|
||||
|
||||
## Enabling the Apps Layer
|
||||
|
||||
The cluster-wide `apps` Kustomization is suspended by default.
|
||||
|
||||
When you are ready to let Flux deploy app attachments from `apps/`, unsuspend it:
|
||||
|
||||
```bash
|
||||
kubectl -n flux-system patch kustomization apps --type=merge -p '{"spec":{"suspend":false}}'
|
||||
```
|
||||
|
||||
Or commit a change to `clusters/prod/flux-system/kustomization-apps.yaml` changing:
|
||||
|
||||
```yaml
|
||||
suspend: true
|
||||
```
|
||||
|
||||
to:
|
||||
|
||||
```yaml
|
||||
suspend: false
|
||||
```
|
||||
|
||||
## First Deploy Checklist
|
||||
|
||||
Before deploying the first app, make sure:
|
||||
|
||||
1. app image builds successfully
|
||||
2. app repo contains valid `deploy/prod` manifests
|
||||
3. this repo contains the `GitRepository` + `Kustomization` attachment objects
|
||||
4. required Doppler secrets exist
|
||||
5. `apps` is unsuspended if you are using the top-level `apps` layer
|
||||
|
||||
## Verification Commands
|
||||
|
||||
From a machine with cluster access:
|
||||
|
||||
```bash
|
||||
kubectl -n flux-system get gitrepositories,kustomizations
|
||||
kubectl get ns
|
||||
kubectl -n my-app get deploy,svc,pods,externalsecret,secret
|
||||
```
|
||||
|
||||
If private over Tailscale:
|
||||
|
||||
```bash
|
||||
kubectl -n my-app get svc my-app-tailscale -o wide
|
||||
```
|
||||
|
||||
## Minimal Recommendation
|
||||
|
||||
If you want the simplest, lowest-risk first deploy:
|
||||
|
||||
1. create a separate app repo
|
||||
2. add `deploy/base` + `deploy/prod`
|
||||
3. add a `GitRepository` + `Kustomization` in this repo under `apps/`
|
||||
4. keep the app private with a Tailscale `LoadBalancer` Service
|
||||
5. use Doppler + `ExternalSecret` for runtime config
|
||||
|
||||
That matches the current cluster design with the least surprise.
|
||||
Reference in New Issue
Block a user