Complete walkthrough¶

Getting started is the 15-minute first run. This page is the long version: the same journey — install → repository → policy → schedule → snapshot → restore — but with the reasoning behind every choice and value, a second track for NAS users, and the kubectl kopiur plugin woven in as the day-2 interface. Read it when you're past "does it work" and into "what should my setup look like, and why".

One paragraph of mental model (the Concepts page has the full story): a Repository is where snapshots live; a SnapshotPolicy is the recipe — what to back up, it runs nothing on its own; a Snapshot is one invocation of a recipe, as a Kubernetes object; a SnapshotSchedule is the cron that creates those invocations; a Restore reads a snapshot back into a PVC. Everything below is those five pieces, in order.

Pick your track

Every manifest step below has two tabs: S3 (AWS, MinIO, RustFS, Ceph RGW, …) and Filesystem (NAS) (an NFS export on your NAS). Pick one in any tab and the whole page follows — the selection is linked and persists. The two tracks are deliberately near-identical: only the Repository backend and one restore detail change. Using Azure, GCS, B2, SFTP, WebDAV, or rclone instead? Follow the S3 track and swap the Repository from your backend's page.

What you'll build¶

Stage	Resource	Day-2 CLI verb
Where snapshots live	`Repository` + a Secret	`kubectl kopiur status` / `doctor`
What to back up	`SnapshotPolicy`	`kubectl kopiur snapshot now`
When it runs	`SnapshotSchedule`	`kubectl kopiur suspend` / `resume`
One backup	`Snapshot` (created for you)	`kubectl kopiur snapshots list` / `logs`
Proof it works	`Restore`	`kubectl kopiur restore` / `ls` / `browse`

Both tracks ship as one apply-ready bundle each — deploy/examples/walkthrough/s3.yaml and deploy/examples/walkthrough/nas.yaml — and every YAML block below is pulled from them at build time, stage by stage. You can follow along step-wise, or fill in the REPLACE_MEs and apply a whole bundle at once; the operator resolves the ordering.

You need a cluster (≥ 1.24), Helm, kubectl, and a PVC with data in it — the walkthrough assumes one named app-data in namespace demo (the Getting started prerequisites show how to create a throwaway one).

Step 0 — Choices before you install¶

The Helm install itself is two commands. The decisions worth making consciously first:

Install scope. The default installScope=cluster lets one operator watch every namespace (RBAC is a ClusterRole), and it's required for ClusterRepository — the shared-repo pattern where a platform team owns the storage and tenant namespaces reference it without ever seeing credentials. It's the default because a namespace-scoped Role silently disables that cluster-scoped kind. Choose installScope=namespaced as the explicit least-privilege opt-down when you want the operator confined to objects in its own release namespace (a Role, not a ClusterRole); ClusterRepository is then not reconciled. Moving between scopes later is a Helm upgrade, not a migration. Details: Installation → Install scope.

Webhook TLS. Kopiur validates and defaults your resources through an admission webhook, which needs a serving certificate. The default webhook.tls.mode=self means the operator mints and rotates it itself — zero dependencies, the right answer unless you already run cert-manager (then cert-manager keeps all your certs in one system). manual is for clusters where certificates must come from your own PKI. Details: Installation → Webhook TLS.

CRD lifecycle. The CRDs ship in the chart's special crds/ directory: helm install installs them, but helm upgrade never touches them. On a helm-CLI upgrade that carries a schema change you apply them yourself (kubectl apply --server-side -f deploy/crds/); a GitOps CRD pipeline with a CreateReplace sync handles it automatically. Details: Installation → CRD lifecycle.

With the defaults chosen deliberately, install:

$ helm install kopiur oci://ghcr.io/home-operations/charts/kopiur --namespace kopiur-system --create-namespace
$ kubectl -n kopiur-system rollout status deploy/kopiur-controller
$ kubectl -n kopiur-system rollout status deploy/kopiur-webhook

Every other knob (images by digest, resources, replicas for HA, observability) is in Helm chart values — none of them block a first run.

Step 1 — Install the kubectl plugin¶

Everything in this walkthrough can be done with raw YAML and kubectl get -w. The plugin exists because day-2 operations are imperative by nature — "back up now", "what failed?", "show me the files in that snapshot" — and deserve verbs with --wait, streamed logs, and meaningful exit codes instead of hand-rolled watch loops. Via krew (the kopiur repo doubles as its own index):

$ kubectl krew index add kopiur https://github.com/home-operations/kopiur.git
$ kubectl krew install kopiur/kopiur

Smoke-test it — on a fresh install this prints an empty-but-healthy overview:

$ kubectl kopiur status -n demo

No krew? Each GitHub release attaches per-platform binaries — see the plugin page.

Step 2 — Credentials¶

The mover Job that actually runs kopia reads its secrets from a Secret in the same namespace as the data (demo here) — it's loaded with envFrom, which is namespace-local, so credentials never transit the operator. What goes in it differs by track, and this is the first place the two tracks teach different lessons:

S3Filesystem (NAS)

apiVersion: v1
kind: Secret
metadata:
  name: walkthrough-creds
  namespace: demo
type: Opaque
stringData:
  # Backend auth keys are read by well-known names from the referenced Secret.
  AWS_ACCESS_KEY_ID: "REPLACE_ME"
  AWS_SECRET_ACCESS_KEY: "REPLACE_ME"
  # Repository encryption password. Generate it (e.g. `openssl rand -base64 24`)
  # and store it OUTSIDE the cluster too — lose it and the backups are
  # unrecoverable; kopia cannot decrypt without it.
  KOPIA_PASSWORD: "choose-something-long-and-random"

apiVersion: v1
kind: Secret
metadata:
  name: walkthrough-creds
  namespace: demo
type: Opaque
stringData:
  # Filesystem/NFS backends need ONLY the repository encryption password — no
  # object-store keys. The data sits on your NAS, but kopia still encrypts it,
  # so a stolen disk or a curious houseguest reads ciphertext.
  # Generate the password (e.g. `openssl rand -base64 24`) and store it OUTSIDE
  # the cluster too — lose it and the backups are unrecoverable.
  KOPIA_PASSWORD: "choose-something-long-and-random"

The S3 track carries the object-store keys (AWS_*, read by well-known names — the per-backend key table covers the other backends). The NAS track needs only KOPIA_PASSWORD: there is no storage account to authenticate to, but kopia still encrypts everything it writes — your NAS holds ciphertext either way.

Lose the password, lose the backups

KOPIA_PASSWORD encrypts the repository. If you lose it, the backups are unrecoverable — kopia cannot decrypt without it. Generate something long and random and store it in your password manager or secret store, not only in the cluster (a backup password that only exists in the cluster it's backing up defeats the purpose).

Step 3 — The Repository (where)¶

S3Filesystem (NAS)

apiVersion: kopiur.home-operations.com/v1alpha1
kind: Repository
metadata:
  name: primary
  namespace: demo
spec:
  backend:
    s3:
      bucket: my-backups
      # One bucket can host many repositories — give each its own prefix. A repo
      # is also an encryption + blast-radius boundary: a leaked password or a bad
      # actor in `demo/` can't touch `prod/`.
      prefix: demo/
      # Omit for AWS (derived from region); REQUIRED for MinIO / RustFS / Ceph
      # RGW and any other S3-compatible endpoint.
      endpoint: s3.us-east-1.amazonaws.com
      region: us-east-1
      auth:
        secretRef:
          name: walkthrough-creds # the AWS_* keys above
  encryption:
    passwordSecretRef:
      name: walkthrough-creds
      key: KOPIA_PASSWORD # which key in the Secret holds the password
  # Initialize a brand-new kopia repository if the prefix is empty. Set false
  # (or omit the block) when adopting a repository that already exists — then a
  # missing repo is an error, not something to paper over.
  create:
    enabled: true
  # Maintenance is default-managed: omit this block entirely and the operator
  # creates and owns a Maintenance with exactly these values (quick every 6h,
  # full daily at 03:00). It is spelled out here so you can see the knobs;
  # `enabled: false` opts out.
  maintenance:
    enabled: true
    schedule:
      quick: { cron: "0 */6 * * *", jitter: 30m }
      full: { cron: "0 3 * * *", jitter: 1h }

apiVersion: kopiur.home-operations.com/v1alpha1
kind: Repository
metadata:
  name: primary
  namespace: demo
spec:
  backend:
    filesystem:
      path: /repo # mount path INSIDE the mover pod where kopia writes the repo
      volume:
        nfs:
          server: nas.lan # NFS server hostname or IP
          # The export on the server (absolute path). One subdirectory per repo
          # plays the same isolation role as an S3 prefix.
          path: /export/kopia/demo
  encryption:
    passwordSecretRef:
      name: walkthrough-creds
      key: KOPIA_PASSWORD # which key in the Secret holds the password
  # Initialize a brand-new kopia repository if the directory is empty. Set false
  # (or omit the block) when adopting a repository that already exists — then a
  # missing repo is an error, not something to paper over.
  #
  # PERMISSIONS: the export must be WRITABLE by the UID the mover runs as
  # (default 65532). If create/connect fails with "permission denied", the
  # operator's Warning Event names the exact UID and the `chown -R <uid> <path>`
  # to run on the NAS. See the Permissions guide.
  create:
    enabled: true
  # Maintenance is default-managed: omit this block entirely and the operator
  # creates and owns a Maintenance with exactly these values (quick every 6h,
  # full daily at 03:00). It is spelled out here so you can see the knobs;
  # `enabled: false` opts out.
  maintenance:
    enabled: true
    schedule:
      quick: { cron: "0 */6 * * *", jitter: 30m }
      full: { cron: "0 3 * * *", jitter: 1h }

The values worth pausing on:

One bucket, many repositories (S3: prefix; NAS: a subdirectory per repo). A repository is an encryption boundary and a blast-radius boundary — a leaked password for demo/ reads nothing from prod/. The trade-off: kopia deduplicates within a repository, so two namespaces backing up similar data into separate repos store it twice. Guidance on drawing that line: Repositories & backends.
create.enabled: true initializes a brand-new kopia repository when the target is empty. When you're adopting an existing repository (e.g. one written by another cluster, or by VolSync — see the migration guide), set it false so a typo'd bucket name fails loudly instead of quietly initializing an empty repo.
maintenance is default-managed. Omit the block entirely and the operator creates an owned Maintenance with exactly the values shown (quick every 6 h, full daily). It's spelled out in the bundle so you can see the knobs — most users should delete the block and take the default.
NAS only — permissions: the export must be writable by the mover's UID (default 65532). If it isn't, the operator's Warning Event names the exact chown to run. See Permissions, UID & GID.

Apply, then wait for Ready — this is the gate everything else waits on, and every Kopiur CRD exposes a standard Ready condition for exactly this:

$ kubectl -n demo wait --for=condition=Ready repository/primary --timeout=2m

Stuck? kubectl -n demo describe repository primary shows conditions and events with the actual cause (wrong keys, unreachable endpoint/NAS, missing repo with create.enabled: false), or jump ahead to kubectl kopiur doctor.

Step 4 — The SnapshotPolicy (what, and for how long)¶

S3Filesystem (NAS)

apiVersion: kopiur.home-operations.com/v1alpha1
kind: SnapshotPolicy
metadata:
  name: app-data
  namespace: demo
spec:
  # repository.kind defaults to Repository; shown explicitly for clarity.
  repository:
    kind: Repository
    name: primary
  sources:
    - pvc:
        name: app-data # the PVC to back up
  # Direct (opt-in; pinned explicitly here since this walkthrough targets
  # arbitrary storage with no assumed CSI snapshot support) reads the live
  # volume — zero CSI requirements, fine for files that don't rewrite in
  # place. `Snapshot` (CSI VolumeSnapshot) is the CRD default and is the
  # right choice for databases and RWOP volumes once your CSI driver supports
  # it; `Clone` where your driver clones faster than it snapshots. See the
  # copy-methods guide.
  copyMethod: Direct
  # GFS retention is the ONLY thing that prunes successful backups. Each tier
  # answers a different recovery question:
  retention:
    keepDaily: 14 # "restore any day from the last two weeks"
    keepWeekly: 8 # "...any week from the last two months"
    keepMonthly: 6 # "...any month from the last half year"
  # What deleting a produced Snapshot CR does to the kopia snapshot behind it.
  # Delete (the default, explicit here) keeps cluster state and repository in
  # lock-step. Retain/Orphan keep the data when the CR goes — see the backups
  # guide before changing this.
  defaultDeletionPolicy: Delete

apiVersion: kopiur.home-operations.com/v1alpha1
kind: SnapshotPolicy
metadata:
  name: app-data
  namespace: demo
spec:
  # repository.kind defaults to Repository; shown explicitly for clarity.
  repository:
    kind: Repository
    name: primary
  sources:
    - pvc:
        name: app-data # the PVC to back up
  # Direct (opt-in; pinned explicitly here since this walkthrough targets
  # arbitrary storage with no assumed CSI snapshot support) reads the live
  # volume — zero CSI requirements, fine for files that don't rewrite in
  # place. `Snapshot` (CSI VolumeSnapshot) is the CRD default and is the
  # right choice for databases and RWOP volumes once your CSI driver supports
  # it; `Clone` where your driver clones faster than it snapshots. See the
  # copy-methods guide.
  copyMethod: Direct
  # GFS retention is the ONLY thing that prunes successful backups. Each tier
  # answers a different recovery question:
  retention:
    keepDaily: 14 # "restore any day from the last two weeks"
    keepWeekly: 8 # "...any week from the last two months"
    keepMonthly: 6 # "...any month from the last half year"
  # What deleting a produced Snapshot CR does to the kopia snapshot behind it.
  # Delete (the default, explicit here) keeps cluster state and repository in
  # lock-step. Retain/Orphan keep the data when the CR goes — see the backups
  # guide before changing this.
  defaultDeletionPolicy: Delete

Identical in both tracks — the policy doesn't care where bytes land. The reasoning:

Retention is GFS (grandfather–father–son) and is the only thing that prunes successful backups. Don't pick numbers; answer recovery questions. keepDaily: 14 = "I can restore any day from the last two weeks" (covers "we noticed the corruption a week later"). keepWeekly: 8 = "any week from the last two months" (covers slow-burn mistakes). keepMonthly: 6 = "any month from the last half year" (compliance / archaeology). Thanks to deduplication, the marginal cost of the older tiers is small — they share unchanged data with newer snapshots.
copyMethod: Direct reads the live volume — no CSI requirements, and a reasonable choice for data that doesn't rewrite in place when you'd rather not depend on the CSI snapshot stack (pinned explicitly here since it's no longer the CRD default). The moment a database is involved, reach for Snapshot (the default: a CSI VolumeSnapshot taken first, so kopia reads a crash-consistent point in time) — it's also the only way to back up ReadWriteOncePod volumes. Clone suits drivers that clone faster than they snapshot. Decision table: Copy methods; for application-consistent backups (quiesce first), see hooks.
defaultDeletionPolicy: Delete ties each produced Snapshot CR to its kopia snapshot: delete the CR, the data goes too, and cluster state stays truthful. Retain/Orphan decouple them — useful, but read the deletionPolicy section before choosing, because "I deleted the CR but the repo kept growing" and "I deleted the CR and lost the snapshot" are both surprises with the wrong setting.

This policy backs up one PVC. Label-selector sources (every PVC matching app=web, grouped consistently) and NFS sources are the same resource with a different sources entry — Backups & schedules covers them.

Step 5 — The SnapshotSchedule (when)¶

S3Filesystem (NAS)

apiVersion: kopiur.home-operations.com/v1alpha1
kind: SnapshotSchedule
metadata:
  name: app-data-nightly
  namespace: demo
spec:
  # Which recipe (SnapshotPolicy) to invoke each firing.
  policyRef:
    name: app-data
  schedule:
    # `H` = a deterministic per-schedule minute (hashed from the schedule's UID),
    # so fifty schedules saying "nightly at 2" don't all fire at 02:00 sharp.
    cron: "H 2 * * *"
    # On top of H, spread each firing over a window — kinder to the repository
    # and the storage backend when many policies share them.
    jitter: 30m
    # GitOps-friendly: applying this manifest does NOT take an immediate backup.
    # Set true if "create schedule" should also mean "back up right now".
    runOnCreate: false
    # If last night's backup is somehow still running, skip this firing rather
    # than stack a second one (the default, explicit here). `Replace` cancels
    # the old run; `Allow` runs both.
    concurrencyPolicy: Forbid

apiVersion: kopiur.home-operations.com/v1alpha1
kind: SnapshotSchedule
metadata:
  name: app-data-nightly
  namespace: demo
spec:
  # Which recipe (SnapshotPolicy) to invoke each firing.
  policyRef:
    name: app-data
  schedule:
    # `H` = a deterministic per-schedule minute (hashed from the schedule's UID),
    # so fifty schedules saying "nightly at 2" don't all fire at 02:00 sharp.
    cron: "H 2 * * *"
    # On top of H, spread each firing over a window — kinder to the repository
    # and the storage backend when many policies share them.
    jitter: 30m
    # GitOps-friendly: applying this manifest does NOT take an immediate backup.
    # Set true if "create schedule" should also mean "back up right now".
    runOnCreate: false
    # If last night's backup is somehow still running, skip this firing rather
    # than stack a second one (the default, explicit here). `Replace` cancels
    # the old run; `Allow` runs both.
    concurrencyPolicy: Forbid

Also identical in both tracks. Why these values:

cron: "H 2 * * *" — H is a Jenkins-style placeholder: a deterministic minute derived from the schedule's identity, so this fires at, say, 02:17 every night. Fifty teams writing "nightly at 2" stop stampeding the repository at 02:00:00 without anyone coordinating. The resolved next firing is pinned to status.nextSchedule.at.
jitter: 30m spreads the start within a window on top of H — kinder to the backend when many policies share it, and it costs you nothing for a nightly backup.
runOnCreate: false (the default) means applying this manifest does not immediately fire a backup — exactly what you want under GitOps, where a re-applied manifest shouldn't mean a surprise snapshot at 3 pm. The trade-off: your first backup waits for tonight, which is why the next step triggers one manually.
concurrencyPolicy: Forbid (the default) skips a firing if the previous one is somehow still running, rather than stacking movers on the same PVC.

$ kubectl -n demo get snapshotschedule app-data-nightly \
    -o jsonpath='{.status.nextSchedule.at}'
2026-06-13T02:17:00Z

Step 6 — First snapshot, the day-2 way¶

The schedule will produce Snapshot CRs nightly. Don't wait for it — trigger the recipe now, watch it run, and stream the mover's logs, in one command:

$ kubectl kopiur snapshot now --policy app-data -n demo --wait --logs

This creates a Snapshot CR with origin: manual — exactly the object the schedule creates nightly with origin: scheduled, so what you just verified is what runs unattended from now on. (--tag reason=walkthrough attaches searchable kopia tags; --pin exempts a snapshot from GFS retention — handy before risky migrations.) --wait exits 0 on Succeeded and 1 on Failed, so the same command drops into CI and scripts.

Then look at what exists:

$ kubectl kopiur snapshots list -n demo
NAME                            POLICY    ORIGIN  PHASE      SNAPSHOT-ID   SIZE     FILES  START                 AGE
app-data-manual-20260612140012  app-data  manual  Succeeded  a1b2c3d4e5f6  148 MiB  412    2026-06-12T14:00:12Z  1m

Richer than kubectl get snapshots, but the CRs are still ordinary objects — both views work. For a failed run, kubectl kopiur logs snapshot <name> -n demo replays the mover's logs even after the Job is gone (details).

Step 7 — Look inside the repository¶

Before trusting a restore to a 2 a.m. incident, look at what's actually in a snapshot — read-only, without restoring anything:

$ kubectl kopiur ls app-data-manual-20260612140012 -n demo
$ kubectl kopiur cat app-data-manual-20260612140012 etc/config.yaml -n demo
$ kubectl kopiur browse app-data-manual-20260612140012 -n demo   # interactive: ls/cd/cat/get

These run through a short-lived in-cluster session pod that mounts nothing from your workloads and can only read the repository — the session-pod model explains the security boundary. Browsing is gated behind its own opt-in RBAC (rbac.browse=true in the chart, the default) so platform teams can switch it off wholesale.

Step 8 — Restore (prove the round trip)¶

A backup you've never restored is a hope, not a backup. Restore into a fresh PVC and compare, so the original is never touched. Here the tracks differ for the second and last time:

S3Filesystem (NAS)

Object-store backends name an explicit snapshot. As a one-liner — pick the snapshot from snapshots list, restore it, follow until done:

$ kubectl kopiur restore --from-snapshot app-data-manual-20260612140012 \
    --create-pvc app-data-restored --size 1Gi -n demo --wait

Or declaratively, the same thing as a manifest (paste the Snapshot name into snapshotRef):

apiVersion: kopiur.home-operations.com/v1alpha1
kind: Restore
metadata:
  name: walkthrough-verify
  namespace: demo
spec:
  source:
    # Object-store backends name an explicit snapshot: paste the Snapshot CR
    # name from `kubectl kopiur snapshots list`. (Latest-for-a-policy resolution
    # — `fromPolicy` — needs a locally mountable repo and is filesystem-only;
    # see the NAS track.)
    snapshotRef:
      name: REPLACE_ME
  target:
    # A fresh PVC, so the verification never touches the original volume. The
    # operator creates it; capacity is REQUIRED — it won't guess a size.
    pvc:
      name: app-data-restored
      capacity: 1Gi
      accessModes:
        - ReadWriteOnce

Filesystem repositories can resolve "the latest snapshot for this policy" at restore time — no snapshot name to paste:

apiVersion: kopiur.home-operations.com/v1alpha1
kind: Restore
metadata:
  name: walkthrough-verify
  namespace: demo
spec:
  source:
    # The filesystem-backend superpower: resolve "the latest snapshot for this
    # policy's identity" at restore time — no Snapshot CR name to paste. This
    # is what powers deploy-or-restore on a fresh cluster (example 05).
    fromPolicy:
      name: app-data
      offset: 0 # 0 = latest, 1 = previous, ...
  target:
    # A fresh PVC, so the verification never touches the original volume. The
    # operator creates it; capacity is REQUIRED — it won't guess a size.
    pvc:
      name: app-data-restored
      capacity: 1Gi
      accessModes:
        - ReadWriteOnce

This fromPolicy source is what powers deploy-or-restore: the same manifests restore data on a fresh cluster and back it up everywhere else (example 05, GitOps guide). The CLI equivalent: kubectl kopiur restore --from-policy app-data --create-pvc app-data-restored --size 1Gi -n demo --wait.

fromPolicy resolves "latest" on every backend

Resolving "latest for a policy" lists the repository's snapshots inside the restore Job, so it works on S3 and the other object-store backends just as it does on a filesystem repo — no controller-side repo mount needed. You can still name the snapshot explicitly (snapshotRef / --from-snapshot) or pin an exact ID via the identity source when you don't want "latest".

$ kubectl -n demo wait --for=jsonpath='{.status.phase}'=Completed restore/walkthrough-verify --timeout=5m

Completed means the data is in app-data-restored — mount it in a pod and diff against the original; that's the real proof. The restored PVC is deliberately not owned by the Restore, so deleting the CR afterwards keeps the data. If your app runs with an fsGroup and the restored files come out unreadable, see restore-side permissions.

Step 9 — Day-2 operations¶

The verbs you'll actually use after today, each one line here and detailed in CLI → Operations:

kubectl kopiur status -n demo — one screen: repositories, policies, schedules, in-flight work, last/next runs. The morning-coffee view.
kubectl kopiur doctor -n demo — when something is red: checks CRDs, operator, webhook, repository connectivity, credentials, and stuck work, and exits 1 if any check fails (CI-friendly).
kubectl kopiur suspend schedule app-data-nightly -n demo / resume — pause firings around upgrades or maintenance windows, declaratively (it sets spec.suspend, so GitOps sees the change).
kubectl kopiur maintenance run --repository primary -n demo --wait — out-of-band compaction/pruning. You normally never need it: maintenance is default-managed (Step 3). It exists for "I just deleted a terabyte and want the space back now".

Teardown¶

$ kubectl -n demo delete snapshotschedule app-data-nightly
$ kubectl -n demo delete restore walkthrough-verify     # the restored PVC stays
$ kubectl -n demo delete snapshot --all
$ kubectl -n demo delete snapshotpolicy app-data
$ kubectl -n demo delete repository primary
$ helm uninstall kopiur -n kopiur-system

Deleting a Snapshot deletes its snapshot

With deletionPolicy: Delete (the produced default — chosen in Step 4), removing a Snapshot CR runs kopia snapshot delete via a finalizer. That's the lock-step behavior you opted into; use Retain/Orphan per-snapshot if a CR must go but the data must stay — see deletionPolicy.

Where to go next¶

Scenarios — the same machinery aimed at specific problems: protect a database (hooks), recover deleted data, disaster recovery, cross-cluster migration, restore drills.
Repositories & backends — ClusterRepository for one shared repo across namespaces, and the other six backends.
Repository replication — mirror the repository to a second backend; the "2" in 3-2-1.
Examples — the per-capability manifest ladder when you need one specific pattern.
Troubleshooting — when a step above doesn't go green.