HetznerTerra/APP_REPO_DEPLOYMENT_GUIDE.md

App Repo Deployment Guide

This guide explains the recommended way to deploy an application to this cluster.

Recommended Model

Use two repos:

• HetznerTerra (this repo): cluster, addons, shared infrastructure, Flux wiring
• your-app-repo: application source, Dockerfile, CI, Kubernetes manifests or Helm chart

Why:

• cluster lifecycle stays separate from app code
• app CI can build and tag images independently
• this repo remains the source of truth for what the cluster is allowed to deploy

Current Cluster Assumptions

• Flux is already installed and reconciles this repo from main
• clusters/prod/flux-system/kustomization-apps.yaml points at ./apps
• apps is suspended by default
• private access is through Tailscale
• runtime secrets should come from Doppler via External Secrets

Deployment Options

Option A: Separate app repo

Recommended for most real applications.

Flow:

1. App repo builds and pushes an image.
2. This repo defines a GitRepository pointing at the app repo.
3. This repo defines a Kustomization pointing at a path in the app repo.
4. Flux pulls the app repo and applies the manifests.

Option B: In-repo app manifests

Only use this when the application is tiny or tightly coupled to the platform.

Flow:

1. Put Kubernetes manifests directly under apps/ in this repo.
2. Unsuspend the top-level apps Kustomization.

This is simpler, but mixes platform and app changes together.

App Repo Structure

Suggested layout:

your-app-repo/
├── src/
├── Dockerfile
├── .gitea/workflows/
└── deploy/
    ├── base/
    │   ├── namespace.yaml
    │   ├── deployment.yaml
    │   ├── service.yaml
    │   ├── externalsecret.yaml
    │   └── kustomization.yaml
    └── prod/
        ├── kustomization.yaml
        └── patch-*.yaml

If you prefer Helm, replace deploy/base and deploy/prod with a chart path and point Flux at that instead.

What the App Repo Should Own

• application source code
• image build pipeline
• image tag strategy
• Deployment / Service / Ingress or Tailscale-facing Service manifests
• app-specific ExternalSecret manifests
• app-specific namespace

What This Repo Should Own

• cluster-level permission to deploy the app
• the GitRepository and top-level Kustomization that attach the app repo to the cluster
• whether the apps layer is suspended or active

Recommended First App Integration

In this repo, add Flux objects under apps/ that point to the app repo.

Example files to add:

• apps/gitrepository-my-app.yaml
• apps/kustomization-my-app.yaml
• update apps/kustomization.yaml

Example apps/gitrepository-my-app.yaml:

apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
  name: my-app
  namespace: flux-system
spec:
  interval: 1m
  ref:
    branch: main
  secretRef:
    name: flux-system
  url: ssh://git@<your-git-host>:<port>/<org>/<your-app-repo>.git

Example apps/kustomization-my-app.yaml:

apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: my-app
  namespace: flux-system
spec:
  interval: 10m
  prune: true
  sourceRef:
    kind: GitRepository
    name: my-app
  path: ./deploy/prod
  wait: true
  timeout: 5m
  dependsOn:
    - name: infrastructure

Then update apps/kustomization.yaml:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - gitrepository-my-app.yaml
  - kustomization-my-app.yaml

App Secrets

Recommended path:

1. Put runtime values in Doppler.
2. In the app manifests, create an ExternalSecret that reads from doppler-hetznerterra.
3. Reference the resulting Kubernetes Secret from the Deployment.

Example app-side ExternalSecret:

apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
  name: my-app-env
  namespace: my-app
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: doppler-hetznerterra
    kind: ClusterSecretStore
  target:
    name: my-app-env
    creationPolicy: Owner
  data:
    - secretKey: DATABASE_URL
      remoteRef:
        key: MY_APP_DATABASE_URL

Image Delivery

Recommended flow:

1. App repo CI builds a container image.
2. CI pushes it to a registry.
3. The app repo updates the Kubernetes image tag in deploy/prod.
4. Flux notices the Git change and deploys it.

Keep the first version simple. Do not add image automation until the basic deploy path is proven.

Exposing the App

Pick one:

Private app over Tailscale

Best fit for this cluster right now.

Create a Service like the existing Rancher/Grafana/Prometheus pattern:

apiVersion: v1
kind: Service
metadata:
  name: my-app-tailscale
  namespace: my-app
  annotations:
    tailscale.com/hostname: my-app
    tailscale.com/tags: "tag:prod"
    tailscale.com/proxy-class: infra-stable
spec:
  type: LoadBalancer
  loadBalancerClass: tailscale
  selector:
    app.kubernetes.io/name: my-app
  ports:
    - name: http
      port: 80
      protocol: TCP
      targetPort: 3000

Use http://my-app.<your-tailnet> or your chosen hostname.

Cluster-internal only

Create only a ClusterIP Service.

Public ingress

Not recommended as the first app path in this repo. Get the private path working first.

Enabling the Apps Layer

The cluster-wide apps Kustomization is suspended by default.

When you are ready to let Flux deploy app attachments from apps/, unsuspend it:

kubectl -n flux-system patch kustomization apps --type=merge -p '{"spec":{"suspend":false}}'

Or commit a change to clusters/prod/flux-system/kustomization-apps.yaml changing:

suspend: true

to:

suspend: false

First Deploy Checklist

Before deploying the first app, make sure:

1. app image builds successfully
2. app repo contains valid deploy/prod manifests
3. this repo contains the GitRepository + Kustomization attachment objects
4. required Doppler secrets exist
5. apps is unsuspended if you are using the top-level apps layer

Verification Commands

From a machine with cluster access:

kubectl -n flux-system get gitrepositories,kustomizations
kubectl get ns
kubectl -n my-app get deploy,svc,pods,externalsecret,secret

If private over Tailscale:

kubectl -n my-app get svc my-app-tailscale -o wide

Minimal Recommendation

If you want the simplest, lowest-risk first deploy:

1. create a separate app repo
2. add deploy/base + deploy/prod
3. add a GitRepository + Kustomization in this repo under apps/
4. keep the app private with a Tailscale LoadBalancer Service
5. use Doppler + ExternalSecret for runtime config

That matches the current cluster design with the least surprise.