HetznerTerra/AGENTS.md

AGENTS.md

Repository guide for agentic contributors working in this repo.

Scope

• This is an infrastructure repository for a Hetzner + k3s + Flux stack.
• Primary areas: terraform/, ansible/, clusters/, infrastructure/, .gitea/workflows/.
• Treat README.md and STABLE_BASELINE.md as user-facing context, but prefer the repo's current manifests and workflows as the source of truth.
• Keep changes small and reviewable; prefer the narrowest file set that solves the task.

Current Tooling

• Terraform for cloud infra and state-backed provisioning.
• Ansible for bootstrap, OS prep, k3s install, and pre-Flux prerequisites.
• Flux/Kustomize for cluster and addon reconciliation.
• Python for inventory generation (ansible/generate_inventory.py).

Important Files

• terraform/main.tf - provider and version pins.
• terraform/variables.tf - input surface and defaults.
• terraform/*.tf - Hetzner network, firewall, servers, SSH, outputs.
• ansible/site.yml - ordered bootstrap playbook.
• ansible/generate_inventory.py - renders ansible/inventory.ini from Terraform outputs.
• clusters/prod/flux-system/ - Flux source and top-level reconciliation graph.
• infrastructure/addons/<addon>/ - Flux-managed addon manifests.
• .gitea/workflows/*.yml - CI/CD entry points and the best reference for expected commands.

Build / Validate / Test

Terraform

• Format all Terraform: terraform -chdir=terraform fmt -recursive
• Check formatting: terraform -chdir=terraform fmt -check -recursive
• Validate config: terraform -chdir=terraform validate
• Full plan: terraform -chdir=terraform plan -var-file=../terraform.tfvars
• Apply: terraform -chdir=terraform apply -var-file=../terraform.tfvars
• Destroy: terraform -chdir=terraform destroy -var-file=../terraform.tfvars

Terraform, single-target / focused checks

• Plan one resource: terraform -chdir=terraform plan -var-file=../terraform.tfvars -target=hcloud_server.control_plane[0]
• Import/check existing state: use terraform state list and terraform state show <address> before editing imports.
• If you touch only Terraform formatting, run terraform fmt -check -recursive first.

Ansible

• Install collections: ansible-galaxy collection install -r ansible/requirements.yml
• Generate inventory: cd ansible && python3 generate_inventory.py
• Syntax check: ansible-playbook -i ansible/inventory.ini ansible/site.yml --syntax-check
• Dry-run one host: ansible-playbook -i ansible/inventory.ini ansible/site.yml --check --diff -l control_plane[0]
• Run the bootstrap playbook: ansible-playbook ansible/site.yml
• Targeted maintenance: ansible-playbook ansible/site.yml -t upgrade or -t reset
• Dashboards only: ansible-playbook ansible/dashboards.yml

Python

• Syntax check the inventory generator: python3 -m py_compile ansible/generate_inventory.py
• If you modify the script, run it after Terraform outputs exist: cd ansible && python3 generate_inventory.py.

Kubernetes / Flux manifests

• Render a single addon: kubectl kustomize infrastructure/addons/<addon>
• Render cluster bootstrap objects: kubectl kustomize clusters/prod/flux-system
• Prefer validating the exact directory you edited, not the whole repo, unless the change is cross-cutting.
• For Flux changes, verify the relevant Kustomization/HelmRelease/ExternalSecret manifests render cleanly before committing.

Kubeconfig refresh

After a full cluster rebuild, the kubeconfig goes stale (new certs, new IPs). Refresh it with:

• scripts/refresh-kubeconfig.sh <cp1-public-ip> (preferred)
• Or manually: ssh -i ~/.ssh/infra root@<cp1-ip> "cat /etc/rancher/k3s/k3s.yaml" | sed 's/127.0.0.1/<cp1-ip>/g' > outputs/kubeconfig
• The Ansible site.yml Finalize step also rewrites the server address to the public IP during bootstrap.

Code Style

General

• Match the existing style in adjacent files.
• Prefer ASCII unless the file already uses Unicode or a Unicode character is necessary.
• Do not introduce new tools, frameworks, or abstractions unless the repo already uses them.
• Keep diffs minimal and avoid unrelated cleanup.

Terraform / HCL

• Use 2-space indentation.
• Keep terraform {} blocks first, then providers, locals, variables, resources, and outputs in a logical order.
• Name variables, locals, and resources in snake_case.
• Keep descriptions on variables and outputs.
• Mark sensitive values with sensitive = true.
• Use aligned = formatting when practical; run terraform fmt instead of hand-formatting.
• Prefer explicit depends_on only when required.
• Keep logic in locals if it is reused or non-trivial.

Ansible / YAML

• Use 2-space YAML indentation.
• Use descriptive task names in sentence case (e.g. Install k3s server).
• Keep tasks idempotent; use changed_when: false and failed_when: false for probes and checks.
• Use command/shell only when a dedicated module is not a better fit.
• Use shell only when you need pipes, redirection, heredocs, or shell expansion.
• Prefer when guards and default(...) filters over duplicating tasks.
• Keep role names and file names kebab-case; keep variables snake_case.
• For multi-line shell snippets in workflows or tasks, use set -e or set -euo pipefail when the command sequence should fail fast.

Kubernetes / Flux YAML

• Keep one Kubernetes object per file unless the repo already groups a small set of tightly related objects.
• Use kebab-case filenames that match the repo pattern (helmrelease-*.yaml, kustomization-*.yaml, *-externalsecret.yaml).
• Keep addon manifests under infrastructure/addons/<addon>/ with a nested kustomization.yaml.
• Keep Flux graph objects in clusters/prod/flux-system/.
• Quote strings that contain :, *, cron expressions, or shell-sensitive characters.
• Preserve existing labels/annotations unless the change specifically needs them.

Python

• Follow PEP 8 style and keep imports ordered: stdlib, third-party, local.
• Use snake_case for functions and variables.
• Keep scripts small and explicit; exit non-zero on failure.
• Prefer clear subprocess error handling over silent failures.

Editing Practices

• Read the target file and adjacent patterns before editing.
• Preserve user changes; do not overwrite unrelated diffs.
• Prefer apply_patch for small single-file edits.
• Use scripting only when it is cleaner than repeated manual edits.
• Keep comments minimal and only add them for non-obvious logic.

Secrets / Security

• Never commit tokens, passwords, kubeconfigs, private keys, or generated secrets.
• Use Gitea secrets, Doppler, or External Secrets for runtime secrets.
• Avoid printing secret values in logs, comments, or commit messages.
• If you must inspect a secret locally, only verify shape/length or compare values indirectly.

Workflow Expectations

• Read the target file and nearby patterns before editing.
• Check git status before and after your changes.
• Run the narrowest relevant validation command after edits.
• If you make a live-cluster workaround, also update the declarative manifests so Flux can own it.
• Do not overwrite user changes you did not make.
• If a change spans Terraform + Ansible + Flux, update and verify each layer separately.

CI / Workflow Notes

• CI currently uses .gitea/workflows/deploy.yml, .gitea/workflows/destroy.yml, and .gitea/workflows/dashboards.yml as the canonical automation references.
• The workflows run terraform fmt -check -recursive, terraform validate, Terraform plan/apply, Ansible bootstrap, and targeted Flux bootstrap steps.
• If you change workflow behavior, keep the repo docs and the workflow commands in sync.

Cursor / Copilot Rules

• No .cursor/rules/, .cursorrules, or .github/copilot-instructions.md files were present when this file was created.
• If those files are added later, mirror their guidance here and treat them as authoritative.