262 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
MichaelFisher1997	ceefcc3b29	cleanup: Remove obsolete port-forwarding, deferred Traefik files, and CI workaround ... - Remove ansible/roles/private-access/ (replaced by Tailscale LB services) - Remove deferred observability ingress/traefik files (replaced by direct Tailscale LBs) - Remove orphaned kustomization-traefik-config.yaml (no backing directory) - Simplify CI: remove SA patch + job deletion workaround for rancher-backup (now handled by postRenderer in HelmRelease) - Update AGENTS.md to reflect current architecture	2026-04-02 01:21:23 +00:00
MichaelFisher1997	0d339b3163	fix: Use rancher/kubectl image for rancher-backup hook ... bitnami/kubectl:1.34 tag doesn't exist. rancher/kubectl is already available in the cluster's image cache.	2026-04-02 01:00:27 +00:00
MichaelFisher1997	30ccf13c82	fix: Use postRenderer to replace broken kuberlr-kubectl image in rancher-backup hook ... The chart's post-install hook hardcodes rancher/kuberlr-kubectl which can't download kubectl. Use Flux postRenderers to patch the job image to bitnami/kubectl at render time.	2026-04-02 00:51:50 +00:00
MichaelFisher1997	75e3604f30	fix: Skip post-install hooks for rancher-backup HelmRelease ... The chart's post-install hook uses rancher/kuberlr-kubectl which fails to download kubectl. The SA automountServiceAccountToken is managed manually, so the hook is unnecessary.	2026-04-02 00:45:03 +00:00
MichaelFisher1997	e4235a6e58	fix: Correct Flux UI pod selector labels to match deployed weave-gitops labels ... Actual labels are app.kubernetes.io/name=weave-gitops and app.kubernetes.io/instance=flux-system-weave-gitops.	2026-04-01 02:08:12 +00:00
MichaelFisher1997	ea2d534171	fix: Use admin.existingSecret for Grafana creds from Doppler ... Revert to idiomatic Grafana chart approach. ExternalSecret creates the secret with admin-user/admin-password keys before Grafana's first start on fresh cluster creation.	2026-04-01 01:41:49 +00:00
MichaelFisher1997	a1b9fe6aa6	fix: Use Flux valuesFrom to inject Doppler Grafana creds as Helm values ... Switch from admin.existingSecret to valuesFrom so Flux reads the Doppler-managed secret and injects credentials as standard Helm values.	2026-03-31 23:40:54 +00:00
MichaelFisher1997	33765657ec	fix: Correct pod selectors for Prometheus and Flux Tailscale services, use Doppler for Grafana creds ... Prometheus needs operator.prometheus.io/name label selector. Flux UI pods are labeled gitops-server not weave-gitops. Grafana now reads admin creds from Doppler via ExternalSecret instead of hardcoded values.	2026-03-31 22:54:57 +00:00
MichaelFisher1997	b8f64fa952	feat: Expose Grafana, Prometheus, and Flux UI via Tailscale LoadBalancer services ... Replace Ansible port-forwarding + tailscale serve with direct Tailscale LB services matching the existing Rancher pattern. Each service gets its own tailnet hostname (grafana/prometheus/flux.silverside-gopher.ts.net).	2026-03-31 08:53:28 +00:00
MichaelFisher1997	569d741751	push	2026-03-31 02:46:55 +00:00
MichaelFisher1997	89e53d9ec9	fix: Handle restricted B2 keys and safe JSON parsing in restore step	2026-03-31 01:43:04 +00:00
MichaelFisher1997	5a2551f40a	fix: Fix flux CLI download URL - use correct GitHub URL with v prefix on version	2026-03-30 03:11:40 +00:00
MichaelFisher1997	8c7b62c024	feat: Automate Rancher backup restore in CI pipeline ... - Wait for Rancher and rancher-backup operator to be ready - Patch default SA in cattle-resources-system (fixes post-install hook failure) - Clean up failed patch-sa jobs - Force reconcile rancher-backup HelmRelease - Find latest backup from B2 using Backblaze API - Create Restore CR to restore Rancher state from latest backup - Wait for restore to complete before continuing	2026-03-30 01:56:29 +00:00
MichaelFisher1997	a1f07f863a	docs: Update restore template with real Backup CR format ... Include actual restore CR spec and note the latest backup filename for reference.	2026-03-30 00:09:53 +00:00
MichaelFisher1997	2c3a49c2e0	fix: Rename B2 secret keys to match rancher-backup operator expectations ... The operator expects accessKey/secretKey, not aws_access_key_id/aws_secret_access_key.	2026-03-30 00:05:13 +00:00
MichaelFisher1997	a7ce3dcc1a	fix: Remove s3 block from rancher-backup HelmRelease values ... The S3 config caused the operator to try downloading kubectl, which fails in the container. S3 credentials are correctly configured in the Backup CR and ExternalSecret instead.	2026-03-29 23:47:21 +00:00
MichaelFisher1997	0ab9418458	fix: Re-add HTTPS port to Tailscale LB for Rancher ... Rancher now manages its own TLS (no longer tls:external), so it serves HTTPS on port 443. The Tailscale LoadBalancer needs to expose both HTTP (80) and HTTPS (443) targeting the corresponding container ports.	2026-03-29 23:04:49 +00:00
MichaelFisher1997	c251672618	fix: Configure S3 bucketName for rancher-backup operator	2026-03-29 23:01:18 +00:00
MichaelFisher1997	89364e8f37	fix: Add dependsOn for rancher-backup operator to wait for CRDs	2026-03-29 22:57:22 +00:00
MichaelFisher1997	20d7a6f777	fix: Install rancher-backup CRD chart before operator ... The rancher-backup operator requires CRDs from the rancher-backup-crd chart to be installed first.	2026-03-29 22:51:34 +00:00
MichaelFisher1997	22ce5fd6f4	feat: Add cert-manager as dependency for Rancher ... Rancher requires cert-manager when managing its own TLS (not tls:external). Added cert-manager HelmRelease with CRDs enabled.	2026-03-29 22:36:30 +00:00
MichaelFisher1997	afb1782d38	fix: Separate Backup CRs into their own kustomization ... The Backup and Restore CRs need the rancher-backup CRDs to exist first. Moved them to a separate kustomization that depends on the operator being ready.	2026-03-29 22:22:29 +00:00
MichaelFisher1997	48870433bf	fix: Remove tls:external from Rancher HelmRelease ... With Tailscale LoadBalancer, TLS is not actually terminated at the edge. The Tailscale proxy does TCP passthrough, so Rancher must serve its own TLS certs. Setting tls: external caused Rancher to listen HTTP-only, which broke HTTPS access through Tailscale.	2026-03-29 22:19:23 +00:00
MichaelFisher1997	f2c506b350	refactor: Replace CNPG external DB with rancher-backup operator ... Rancher 2.x uses embedded etcd, not an external PostgreSQL database. The CATTLE_DB_CATTLE_* env vars are Rancher v1 only and were ignored. - Remove all CNPG (CloudNativePG) cluster, operator, and related configs - Remove external DB env vars from Rancher HelmRelease - Remove rancher-db-password ExternalSecret - Add rancher-backup operator HelmRelease (v106.0.2+up8.1.0) - Add B2 credentials ExternalSecret for backup storage - Add recurring Backup CR (daily at 03:00, 7 day retention) - Add commented-out Restore CR for rebuild recovery - Update Flux dependency graph accordingly	2026-03-29 21:53:16 +00:00
MichaelFisher1997	efdf13976a	fix: Handle missing 'online' field in Tailscale API response	2026-03-29 13:52:23 +00:00
MichaelFisher1997	5269884408	feat: Auto-cleanup stale Tailscale devices before cluster boot ... Adds tailscale-cleanup Ansible role that uses the Tailscale API to delete offline devices matching reserved hostnames (e.g. rancher). Runs during site.yml before Finalize to prevent hostname collisions like rancher-1 on rebuild. Requires TAILSCALE_API_KEY (API access token) passed as extra var.	2026-03-29 11:47:53 +00:00
MichaelFisher1997	6e5b0518be	feat: Add kubeconfig refresh script and fix Ansible Finalize to use public IP ... - scripts/refresh-kubeconfig.sh fetches a fresh kubeconfig from CP1 - Ansible site.yml Finalize step now uses public IP instead of Tailscale hostname for the kubeconfig server address - Updated AGENTS.md with kubeconfig refresh instructions	2026-03-29 03:31:36 +00:00
MichaelFisher1997	905d069e91	fix: Add serverName to CNPG externalClusters for B2 recovery ... CNPG uses the external cluster name (b2-backup) as the barman server name by default, but the backups were stored under server name rancher-db.	2026-03-29 03:22:19 +00:00
MichaelFisher1997	25ba4b7115	fix: Add skipEmptyWalArchiveCheck annotation and B2 secret healthcheck to CNPG ... - Skip WAL archive emptiness check so recovery works when restoring over an existing backup archive in B2 - Add healthCheck for b2-credentials secret in CNPG kustomization to prevent recovery from starting before ExternalSecret has synced	2026-03-29 03:15:23 +00:00
MichaelFisher1997	6a593fd559	feat: Add B2 recovery bootstrap to CNPG cluster	2026-03-29 00:22:24 +00:00
MichaelFisher1997	936f54a1b5	fix: Restore canonical Rancher tailnet hostname	2026-03-29 00:00:39 +00:00
MichaelFisher1997	c9df11e65f	fix: Align Rancher tailnet hostname with live proxy	2026-03-28 23:47:09 +00:00
MichaelFisher1997	a3c238fda9	fix: Apply Rancher server URL after chart install	2026-03-28 23:12:59 +00:00
MichaelFisher1997	a15fa50302	fix: Use Doppler-backed Rancher bootstrap password	2026-03-28 22:51:38 +00:00
MichaelFisher1997	0f4f0b09fb	fix: Add Rancher DB password ExternalSecret	2026-03-28 22:42:05 +00:00
MichaelFisher1997	4c002a870c	fix: Remove invalid Rancher server-url manifest	2026-03-28 22:39:31 +00:00
MichaelFisher1997	43d11ac7e6	docs: Add agent guidance and sync Rancher docs	2026-03-28 22:13:37 +00:00
MichaelFisher1997	8c5edcf0a1	fix: Set Rancher server URL to tailnet hostname	2026-03-28 04:07:44 +00:00
MichaelFisher1997	a81da0d178	feat: Expose Rancher via Tailscale hostname	2026-03-28 03:59:02 +00:00
MichaelFisher1997	2a72527c79	fix: Switch Traefik from LoadBalancer to NodePort, remove unused Hetzner LB	2026-03-28 03:21:19 +00:00
MichaelFisher1997	7cb3b84ecb	feat: Replace custom pgdump job with CNPG ScheduledBackup	2026-03-28 03:15:39 +00:00
MichaelFisher1997	d4930235fa	fix: Point CNPG backups at the existing B2 bucket	2026-03-26 23:35:19 +00:00
MichaelFisher1997	ee8dc4b451	fix: Add Role for B2 credentials access	2026-03-26 23:04:40 +00:00
MichaelFisher1997	144d40e7ac	feat: Add RBAC for CNP to read B2 credentials secret	2026-03-26 22:56:00 +00:00
MichaelFisher1997	cc14e32572	fix: Use gzip instead of lzop for backup compression	2026-03-26 22:51:10 +00:00
MichaelFisher1997	a207a5a7fd	fix: Remove invalid encryption field from CNP backup config	2026-03-26 22:49:29 +00:00
MichaelFisher1997	4e1772c175	feat: Add B2 backup configuration to CNP Cluster	2026-03-26 22:47:31 +00:00
MichaelFisher1997	ff70b12084	chore: Add HTTP/HTTPS firewall rules for Load Balancer	2026-03-26 22:36:13 +00:00
MichaelFisher1997	a3963c56e6	cleanup: Remove traefik-config, simplify traefik helmrelease	2026-03-26 03:16:56 +00:00
MichaelFisher1997	612435c42c	fix: Add Hetzner LB health check config to Traefik	2026-03-26 03:11:10 +00:00