lhc backup / lhc restore

Trigger a full backup or restore of the namespace secrets, Postgres, ClickHouse, and object storage. These commands are thin wrappers around the built-in job definitions — they resolve the relevant job by type, trigger a run, and wait for it to finish.

Commands

backup

Trigger backup job runs for the namespace secrets, Postgres, ClickHouse, and (optionally) object storage:

lhc backup

The jobs run in a fixed order — BACKUP_SECRETS first (the encryption key must never be older than the data it decrypts), then Postgres and ClickHouse, then the object storage mirror last. The command waits for all triggered jobs to reach a terminal state and prints a summary table with job_type, run_id, and status for each. Exits with a non-zero status code if a database or object storage job did not succeed (failed, cancelled or skipped) or the wait timed out.

Flag	Description
--skip-secrets	Skip the encrypted secret backup. The command prints a warning that the resulting backup contains no secrets and is not restorable after a cluster loss
--skip-object-storage	Skip the object storage mirror backup (e.g. if the external target is not configured yet)

Secret backup requires an age public key on spec.backup.secretsAgePublicKey in the Custom Resource. BACKUP_SECRETS runs first; if it fails — most often because the key is not set — lhc backup aborts before any database or object storage backup is started, prints how to configure the key, and exits non-zero. No partial backup is taken. To back up the databases without secrets on purpose, use --skip-secrets. See Backup — Kubernetes Secrets Backup for the key setup.

Object storage backup requires an external, customer-configured backup target (S3 or Google Cloud Storage). If it is not configured, the object storage backup job fails with a clear error — use --skip-object-storage until it is set up. The secret, Postgres, and ClickHouse backups always write to the internal object storage first; the mirror is what carries them off-site.

---

restore

Trigger restore job runs for Postgres, ClickHouse, and (optionally) object storage:

lhc restore --yes

Flag	Description
--yes	Required. Confirms the restore — it overwrites current data
--postgres-timestamp <value>	Postgres backup to restore from (default: latest)
--clickhouse-backup-path <value>	ClickHouse backup timestamp folder to restore from (e.g. 20260711210000). Required unless --skip-clickhouse is set — ClickHouse backups have no latest alias
--skip-postgres	Skip the Postgres restore
--skip-clickhouse	Skip the ClickHouse restore
--skip-object-storage	Skip the object storage restore

lhc restore --yes \
  --postgres-timestamp latest \
  --clickhouse-backup-path 20260711210000

The components run one after the other — Postgres, ClickHouse, object storage. The first one that does not complete successfully (failed, cancelled, or still running when the wait ends) stops the restore: the later components are listed as NOT_STARTED and are not triggered, and the command exits non-zero. The instance may then be in a mixed state; check the stopped run with lhc jobs status <run-id> and decide whether to restore the remaining components with the --skip-* flags.

Only one backup or restore runs at a time across the instance. Triggering any backup or restore job while another one is active answers 423 with the active run IDs.

Restore does not stop services for you

lhc restore does not pause any Lakehousecat services, and it must not be run with deployments scaled down — it runs as a job through lhc and Airflow. Run it in a maintenance window: changes users make meanwhile are lost. See Backup & Restore for the full sequence.

What is not covered

• Restoring the namespace secrets is not part of lhc restore — it is a manual step run from your own machine before the instance exists (in a disaster-recovery rebuild, the secrets are the precondition for Airflow, which lhc restore runs through). lhc backup does back the secrets up. See Restore — Step 1.
• Maintenance mode is not automated — see the caution above.
• Object storage backup destinations currently support S3 and Google Cloud Storage.