Maintenance¶
Kopia repositories need periodic maintenance to stay healthy: compacting indexes, advancing epochs, and — most importantly — reclaiming storage by garbage-collecting content that deleted snapshots no longer reference. Without it, a repository keeps growing even as you expire old backups.
Kopiur makes maintenance a first-class, default-managed concern. You don't have to remember to schedule it: every Repository and ClusterRepository gets a Maintenance resource automatically, and the operator runs kopia maintenance on a schedule for every backend — filesystem and object stores (S3, Azure, GCS, B2, …) alike.
How it runs
Each scheduled run executes in a short-lived mover Job (the same mechanism used for backups and restores), so maintenance works identically whether your repository lives on a PVC or in an object store the operator can't reach directly.
Quick vs. full¶
kopia has two maintenance passes, and Kopiur schedules them independently:
| Pass | kopia command | What it does | Default schedule |
|---|---|---|---|
| Quick | kopia maintenance run --no-full |
Cheap, frequent: index compaction, epoch advance. | every 6h (0 */6 * * *), 30m jitter |
| Full | kopia maintenance run --full |
Heavier: content garbage-collection + rewrite — this is what reclaims storage. | daily at 03:00 (0 3 * * *), 1h jitter |
A full run subsumes a quick run, so when both are due at once the operator runs full and advances both clocks.
The default-managed model¶
Maintenance is on by default. For every Repository/ClusterRepository, the operator projects a Maintenance resource (named after the repository) with the default schedule above. You can see it with:
$ kubectl get maintenance -A
NAMESPACE NAME REPOSITORY OWNER AGE
billing nas-primary nas-primary kopiur/billing/nas-primary 4h44m
There are three ways to control it, in increasing order of explicitness.
Try it end-to-end¶
Prove the default-managed model — a Maintenance appears with nothing but a Repository — and then force a run on demand. The bundle deploy/examples/tryit/maintenance.yaml is deliberately minimal: namespace, a repo PVC, a KOPIA_PASSWORD Secret, and a filesystem Repository — and no Maintenance resource. The operator projects one for you.
The Repository below is the only CR in the bundle — note it carries no spec.maintenance block and there is no standalone Maintenance; the managed one is auto-projected the moment this repository is Ready:
# A bare Repository. NOTE there is no `spec.maintenance` block and no standalone
# Maintenance CR: maintenance is default-on, so the operator projects an owned
# `Maintenance` named `primary` the moment this repository is Ready.
apiVersion: kopiur.home-operations.com/v1alpha1
kind: Repository
metadata:
name: primary
namespace: kopiur-tryit
spec:
backend:
filesystem:
path: /repo
volume:
pvc:
name: kopia-repo
encryption:
passwordSecretRef:
name: repo-creds
key: KOPIA_PASSWORD
create:
enabled: true
Fill in the single REPLACE_ME (KOPIA_PASSWORD) and apply once:
$ kubectl apply -f deploy/examples/tryit/maintenance.yaml
$ kubectl -n kopiur-tryit wait --for=condition=Ready repository/primary --timeout=2m
1. The managed Maintenance auto-appears. You authored none — the operator created primary (named after the repository) with the default quick-6h / full-daily schedule:
$ kubectl -n kopiur-tryit get maintenance
NAME REPOSITORY OWNER AGE
primary primary kopiur/kopiur-tryit/primary 20s
2. Request a full run NOW. Stamp the two on-demand annotations (--overwrite because the timestamp changes each time):
$ kubectl annotate maintenance primary -n kopiur-tryit --overwrite \
kopiur.home-operations.com/run-requested="$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
kopiur.home-operations.com/run-mode=full
3. Prove the run completed (deep). The outcome lands in status.manualRun:
$ kubectl -n kopiur-tryit wait \
--for=jsonpath='{.status.manualRun.phase}'=Succeeded \
maintenance/primary --timeout=5m
$ kubectl -n kopiur-tryit get maintenance primary \
-o jsonpath='{.status.manualRun}'
{"requestedAt":"2026-06-17T14:05:00Z","mode":"full","phase":"Succeeded","completedAt":"2026-06-17T14:05:42Z"}
(Illustrative timestamps.)
4. Confirm the full clock advanced. A full run also stamps status.full.lastRunAt:
$ kubectl -n kopiur-tryit get maintenance primary \
-o jsonpath='{.status.full.lastRunAt}'
2026-06-17T14:05:42Z # illustrative
lastContentReclaimedBytes reads 0 even on a successful run
kopia maintenance run does not emit a machine-readable reclaimed-bytes figure, so status.full.lastContentReclaimedBytes is reported as 0 today even though the run does reclaim space. The field exists and round-trips; populating it precisely is a planned enhancement.
To tear down: kubectl delete namespace kopiur-tryit.
1. Tune it inline on the repository¶
Set spec.maintenance on the Repository/ClusterRepository to override the schedule (or other knobs) while keeping it operator-managed:
# Maintenance — tuned inline on the Repository (docs/maintenance.md)
#
# The simplest way to control maintenance: set `spec.maintenance` on the
# Repository/ClusterRepository itself. Maintenance stays operator-managed (the
# controller projects and owns a Maintenance for you), you just override the
# schedule (or other knobs) instead of authoring a standalone Maintenance.
#
# This shows a filesystem (PVC-backed) repository, but the `spec.maintenance`
# block is identical for every backend. Omit the block entirely and you get the
# defaults (quick every 6h, full daily at 03:00); set `enabled: false` to opt out.
#
# Field shapes verified against crates/api: externally-tagged backend
# (`backend.filesystem`); `schedule.quick`/`schedule.full` are CronSpec
# { cron, jitter }.
---
apiVersion: v1
kind: Secret
metadata:
name: nas-primary-kopia
namespace: billing
type: Opaque
stringData:
# Filesystem backends need ONLY the repository encryption password.
KOPIA_PASSWORD: "choose-something-long-and-random"
---
apiVersion: kopiur.home-operations.com/v1alpha1
kind: Repository
metadata:
name: nas-primary
namespace: billing
spec:
backend:
filesystem:
path: /repo # mount path INSIDE the mover pod
volume:
pvc:
name: nas-primary # the PVC mounted read-write at `path`
encryption:
passwordSecretRef:
name: nas-primary-kopia
key: KOPIA_PASSWORD
# Tune maintenance while keeping it operator-managed. Absent ⇒ the defaults.
maintenance:
enabled: true
schedule:
quick: { cron: "0 */6 * * *", jitter: 30m } # cheap, frequent: index compaction
full: { cron: "0 3 * * *", jitter: 1h } # heavy: GC + rewrite — reclaims storage
spec.maintenance fields:
| Field | Purpose |
|---|---|
enabled |
Default true. Set false to opt out (see Disabling). |
schedule |
Override quick/full cron + jitter (and timezone). Absent ⇒ the defaults above. |
mover |
Pod overrides for the maintenance Job (resources, scheduling, security context). |
failurePolicy |
backoffLimit / activeDeadlineSeconds for the Job. |
takeoverPolicy |
Ownership-lease policy (see Ownership). |
namespace |
ClusterRepository only — which namespace the managed Maintenance lives in (defaults to the operator's namespace). |
Timezone cascades three levels
quick.timezone/full.timezone (per-cron) wins; else the shared schedule.timezone; else the target repository's scheduleDefaults.timezone (Repositories → scheduleDefaults); else UTC. Set it once on the repository instead of repeating it on every Maintenance.
2. Author a standalone Maintenance¶
For fine-grained control — a custom ownership identity or takeover policy — author a Maintenance directly (example 08). When one references a repository, the operator defers to it and never creates a duplicate, even if spec.maintenance is otherwise default-on.
apiVersion: kopiur.home-operations.com/v1alpha1
kind: Maintenance
metadata:
name: nas-primary-maintenance
namespace: billing
annotations:
# Request an immediate FULL run on next apply. Bump the timestamp to request
# another; remove these to stop requesting on-demand runs (the crons keep
# firing regardless).
kopiur.home-operations.com/run-requested: "2026-01-01T00:00:00Z"
kopiur.home-operations.com/run-mode: full
spec:
repository:
kind: Repository
name: nas-primary
schedule:
quick:
cron: "0 */6 * * *"
jitter: 30m
full:
cron: "0 3 * * 0"
jitter: 1h
timezone: UTC
ownership:
owner: "kopia-operator/nas-primary"
takeoverPolicy: PromptCondition
mover:
resources:
requests:
cpu: 250m
memory: 1Gi
limits:
cpu: "2"
memory: 4Gi
failurePolicy:
backoffLimit: 1
activeDeadlineSeconds: 14400 # cap a RUNNING maintenance Job (4h)
podStartupDeadlineSeconds: 300 # fail fast if the maintenance mover can't START in 5m (default 300)
Disabling maintenance¶
Set spec.maintenance.enabled: false on the repository. The operator stops managing a Maintenance for it.
Disabling stops space reclamation
With no maintenance, the repository never garbage-collects, so storage grows without bound even as you expire backups. Only disable it if something else runs kopia maintenance against the repository. Note that enabled: false only tells the operator not to create its own Maintenance — an externally-authored one referencing the repository is always honored.
Ownership and shared repositories¶
kopia tracks a single maintenance owner per repository. When several clusters (or operators) share one repository, only one should run maintenance at a time. spec.ownership encodes who that is and what to do on conflict:
owner— a stable identity string for thisMaintenance(the operator derives it for a managedMaintenance; see the lease format below).ownerAliases— previous lease strings still recognized as self (the lease-format migration path, below).takeoverPolicy— a closed enum:
| Policy | Behavior when another owner holds the lease |
|---|---|
Never (default, safest) |
Do nothing; surface that the lease is held elsewhere and wait. |
PromptCondition |
Set a condition asking an operator to decide; don't seize it. |
Force |
Forcibly claim the lease and run. |
The lease is read inside the maintenance Job (which is the only place with repository access for object stores). If the policy declines to take over, the run is a successful no-op that records why on the resource's conditions.
The lease string and kopia's recorded owner¶
Every default-managed Maintenance derives its lease string (spec.ownership.owner) from the repository it covers. The format depends on whether the repository has identityDefaults.cluster set — a repository shared across clusters needs a distinct lease per cluster, or every cluster would derive the identical lease and fight over it:
| Repository kind | Cluster identity | Lease string (spec.ownership.owner) |
kopia's recorded owner (user@hostname) |
|---|---|---|---|
Repository |
unset | kopiur/<namespace>/<name> |
kopiur@kopiur-<namespace>-<name> |
Repository |
east |
kopiur/east/<namespace>/<name> |
kopiur@kopiur.east.<namespace>.<name> |
ClusterRepository |
unset | kopiur/clusterrepository/<name> |
kopiur@kopiur-clusterrepository-<name> |
ClusterRepository |
east |
kopiur/east/clusterrepository/<name> |
kopiur@kopiur.east.clusterrepository.<name> |
Why the cluster-qualified owner is dot-joined, not dash-joined
A single-cluster lease sanitizes to one dash-joined DNS label (kopiur-media-nas), unchanged from before multi-cluster support. A cluster-qualified lease is instead dot-joined per segment (kopiur.east.media.nas). This dot-join only ever applies to a lease the operator itself generated — its first path segment is the literal, reserved kopiur — so a hand-authored spec.ownership.owner you write yourself always falls back to the single dash-joined form; its derivation can never change across an upgrade merely because it happens to also split into four segments.
ownerAliases — carrying ownership across a lease-format change¶
spec.ownership.ownerAliases lists previous lease strings kopiur should still recognize as itself. This is the migration path for turning identityDefaults.cluster on for the first time on a repository that already has a managed Maintenance: the operator automatically records the pre-cluster lease as an alias on every managed Maintenance for a cluster-identified repository, so a run recognizes kopia's already-recorded owner as its own and claims and re-stamps it to the new cluster-qualified format, instead of yielding to what would otherwise look like a foreign owner. You don't set this by hand for a managed Maintenance — only a standalone one needs it authored explicitly if you hand-roll the same migration.
Sharing one repository's maintenance across clusters: pick ONE owner¶
kopia has no cross-host lock beyond this lease, so when several clusters' Repository/ClusterRepository objects all point at the same physical repository, each cluster's independently-managed Maintenance tries to claim the same lease. The default (takeoverPolicy: Never) makes that safe — every cluster but the current holder yields — but it isn't the recommended steady state once you know which cluster should own it:
- Pick the one cluster that runs maintenance for the shared repository (kopia doesn't care which).
- On every other cluster, set
spec.maintenance.enabled: falseon that repository — the operator stops creating/reconciling aMaintenancethere at all, instead of one that exists only to yield forever. - Remove
takeoverPolicy: Forcefrom every cluster except the owner.Forceunconditionally seizes the lease on its next run; leave it set on more than one cluster and they fight over it every reconcile, each re-seizing it from the other. SetForceonly on the one cluster you intend to own it, and only long enough to claim the lease once — then revert it toNever.
Yielding is the safety net, not the recommended posture: a non-owner cluster left with maintenance enabled just yields loudly (Ready=False, reason MaintenanceYielding) rather than fighting for the lease or touching data — but the loud yielding (and a mover Job that runs for nothing every cron slot) is exactly why the explicit enabled: false posture above is preferred once you know the owner. See Share one repository across clusters for the full walkthrough, with this step placed in the correct order relative to the identity flip.
Self-healing a stale owner¶
kopia stamps a maintenance owner when a repository is created. kopiur stamps its own stable, lease-derived owner (the table above) so that every maintenance Job — each from a fresh, throwaway pod — recognizes itself as the owner and runs. On every bootstrap (initial connect and each catalog refresh), if the recorded owner doesn't match, kopiur may re-stamp it (kopia maintenance set --owner is not owner-gated) — how readily depends on whether the repository has a cluster identity:
- No
identityDefaults.cluster(single-cluster repository). Self-healing is unchanged from before multi-cluster support: any stale owner is re-stamped unconditionally. Safe, because at most one cluster's operator ever bootstraps this repository — a stale owner can only be kopia's own ephemeral create-time identity (e.g.nonroot@nas-bootstrap-5trlr) or an older-format stamp from this same operator, never another cluster's. identityDefaults.clusterset (shared repository). Self-heal restamps only an owner that is empty, already the desired owner, or matches a recognizedownerAliasesentry — i.e. only this cluster's own current or legacy formats. A foreign cluster's owner is always honored and left completely alone: with the unconditional rule, every cluster sharing the repository would see the OTHER's owner as "stale" and re-claim it on its own next bootstrap, ping-ponging the lease back and forth forever.- The tradeoff: an ancient owner this operator has never recognized before — a workstation
kopiaCLI, or any owner string that isn't this cluster's current lease or a registered alias — is left alone rather than auto-clobbered, even though it may genuinely be stale. Move it with a one-timespec.ownership.takeoverPolicy: Force(orkopia maintenance set --ownerby hand); once claimed, this operator's own lease-derived owner is recognized on every subsequent run, so no further manual steps are needed. RevertForcetoNeverright after — see the "pick ONE owner" section above for why leaving it set is unsafe on a shared repository.
Index-blob health¶
kopia stores its content index as a set of index blobs. Each backup adds one; maintenance compacts them back down. So in a healthy repository the count rises during the day and falls after the next full-maintenance run. If maintenance stops keeping up — most often a stale owner (above), but also a disabled/failing Maintenance — the count climbs without bound, and once it gets high enough kopia warns "Found too many index blobs (N)" and backup/restore performance degrades.
kopiur observes the count on every bootstrap and surfaces it three ways, without blocking the repository (it stays Ready; this is a degradation warning, not an outage):
- a print column —
kubectl get repository/kubectl get clusterrepositoryshows anIndexBlobscolumn (wide output); status.storageStats.indexBlobCount;- when the count crosses the threshold, an
IndexBlobHealth=Falsecondition (reasonTooManyIndexBlobs) and a Kubernetes Warning event with the remediation in its message:
$ kubectl describe clusterrepository nas-shared | grep -A2 TooManyIndexBlobs
$ kubectl get events --field-selector reason=TooManyIndexBlobs -A
The threshold knob¶
spec.health.indexBlobWarnThreshold sets the count above which the warning fires. It's optional on both Repository and ClusterRepository:
| Value | Meaning |
|---|---|
| absent | Use the built-in default of 1000 (well above a healthy repo). |
| a positive number | Warn when the count exceeds it (lower = earlier warning). |
0 |
Disable the warning entirely. |
When maintenance is running and the count still won't fall¶
Raising the threshold hides the warning; it doesn't compact anything, and it has no effect on kopia's own hard-coded "Found too many index blobs" message or on how long your movers take to connect. If maintenance is demonstrably running and the count is still stuck in the thousands, the cause is usually kopia's epoch-advance gate, and no maintenance schedule can fix it.
kopia cannot compact an index blob until its epoch closes. An epoch may only close once it is older than MinEpochDuration — 24h by default — no matter how many blobs accumulate inside it, and compaction then trails two epochs behind. So a fleet producing, say, ~60 index blobs an hour is forced to ~1700 blobs before an epoch may close, and permanently carries 1700–3400 uncompacted blobs. Compaction pace was never the bottleneck; the 24h floor is.
spec.parameters.epoch exposes that gate on both Repository and ClusterRepository:
Every field is optional and kopiur has no defaults of its own — absent means "leave kopia's current value alone", so a repository that declares nothing here is completely unaffected. Applying a change rewrites the repository's format blob, which invalidates other kopia clients' cached copy of it (they re-read within ~15 minutes), so kopiur calls set-parameters only when the declared values actually differ from what the repository reports.
Check what landed under status.parameters.epoch — it reports what the repository says, not what you asked for, so a value that failed to apply shows as a mismatch rather than as silence:
$ kubectl get repository primary -o jsonpath='{.status.parameters.epoch}' | jq
{
"enabled": true,
"minDuration": "6h",
"refreshFrequency": "20m",
"cleanupSafetyMargin": "4h",
"advanceOnCount": 20,
"advanceOnSizeMiB": 10,
"checkpointFrequency": 7,
"deleteParallelism": 4
}
The full field set, plus the sharp edges:
# Example 33 — tune kopia's epoch parameters (spec.parameters.epoch)
#
# THE SYMPTOM: `IndexBlobHealth=False` / `TooManyIndexBlobs`, thousands of index
# blobs that never go down however often maintenance runs, and mover connects that
# take tens of seconds because every one of them reads those blobs.
#
# THE CAUSE, if maintenance is demonstrably running: kopia cannot compact an index
# blob until its EPOCH closes. An epoch may only close once it is older than
# MinEpochDuration — 24h by default — no matter how many blobs accumulate inside it.
# Compaction then trails two epochs behind. So a fleet producing ~60 index blobs an
# hour is held at ~1700 blobs per epoch and carries 1700–3400 uncompacted blobs
# permanently. No maintenance schedule can fix that: compaction pace is not the
# bottleneck, the 24h advance gate is.
#
# THE FIX: lower minDuration so epochs close (and blobs become compactable) sooner.
# 6h is a reasonable starting point for a busy repository.
#
# WHY THIS IS A CR FIELD: `kopia repository set-parameters --epoch-min-duration=6h`
# run by hand works — the value persists in the format blob and movers pick it up on
# connect — but it is invisible to GitOps, survives in no manifest, and a repository
# re-bootstrap silently reverts it. Declared here, kopiur re-applies it whenever it
# drifts.
#
# ABSENT MEANS "DON'T TOUCH". Every field below is optional and kopiur has no
# defaults of its own: omit one and kopia's current value is left exactly as it is.
# The flip side is that REMOVING a value you previously set does not restore kopia's
# default — it leaves the repository at whatever you last applied. Set it back
# explicitly if you want the default again.
#
# NOTE: applying these rewrites the repository's format blob, which invalidates every
# other kopia client's cached copy (they re-read within ~15 minutes on their own).
# kopiur therefore only calls set-parameters when the declared values actually differ
# from what the repository reports — a converged spec costs nothing.
#
# NOT AVAILABLE on a `mode: ReadOnly` repository: set-parameters is a repository-wide
# write and kopia refuses it on a read-only connection, so declaring it there is
# rejected at admission. In a multi-cluster layout, declare the parameters on the
# cluster that owns the repository — they are a property of the repository, not of
# each consumer.
#
# See docs/repository-health.md
---
apiVersion: kopiur.home-operations.com/v1alpha1
kind: Repository
metadata:
name: primary
namespace: backups
spec:
backend:
s3:
bucket: kopia-backups
endpoint: s3.example.com
region: us-east-1
auth:
secretRef:
name: s3-creds
encryption:
passwordSecretRef:
name: kopia-repo
key: KOPIA_PASSWORD
parameters:
epoch:
# The gate. kopia's default is 24h; an epoch cannot close before this,
# regardless of how many index blobs it holds.
minDuration: 6h
# The rest are shown for reference — omit them unless you have a reason.
# Each comment is kopia's default, which is what you get by leaving it out.
#
# refreshFrequency: 20m # how often clients re-read epoch state
# advanceOnCount: 20 # index blobs that trigger an advance (once past minDuration)
# advanceOnSizeMiB: 10 # index size that triggers an advance (MEBIbytes: 10 = 10485760 bytes,
# # despite kopia's own log rendering it as "10 MB")
# checkpointFrequency: 7 # epochs between full index checkpoints
# deleteParallelism: 4 # parallelism for epoch cleanup deletions
# Observe the result under `status.parameters.epoch`:
#
# kubectl get repository primary -n backups -o jsonpath='{.status.parameters.epoch}'
#
# It reports what the REPOSITORY says, not what you declared, so a value that
# failed to apply shows up as a mismatch rather than as silence.
#
# `cleanupSafetyMargin` appears there too but is deliberately not settable: it is
# the grace window that stops kopia deleting index blobs a concurrent writer still
# needs, and there is no safe generic advice for lowering it.
Three things worth knowing:
- Removing a value does not restore kopia's default. Under "absent means don't touch", deleting
minDurationfrom your manifest leaves the repository at 6h forever. Set it back explicitly if you want 24h again. mode: ReadOnlyrepositories cannot declare parameters.set-parametersis a repository-wide write and kopia refuses it on a read-only connection, so the combination is rejected at admission. In a multi-cluster layout, declare them on the cluster that owns the repository — they describe the repository, not each consumer. Two clusters declaring different values for one repository will fight over it.cleanupSafetyMarginis reported but not settable. It is the grace window that stops kopia deleting index blobs a concurrent writer still needs, and there is no safe generic advice for lowering it.
A high index-blob count means maintenance isn't running
The warning is a symptom; the fix is to get maintenance compacting again. Check that a Maintenance exists and is Ready, that it isn't yielding (LeaseOwned=False, reason LeaseHeldByOther), and that its owner is the stable lease owner — not an ephemeral …-bootstrap-… identity. The operator self-heals a stale owner on the next bootstrap; to recover now, set spec.maintenance.takeoverPolicy: Force once. Raising or zeroing indexBlobWarnThreshold only silences the warning — it does not compact the index.
Running maintenance on demand¶
Maintenance normally fires on its quick/full crons, but you can request an
out-of-band run at any time by stamping two annotations on the Maintenance —
the operator routes it through the same mover, ownership-lease, and
single-flight path as the scheduled slots, so a manual run can never violate the
one-job-per-repository guarantee.
Declaratively (GitOps-friendly — set them in Git on the managed Maintenance,
as in example 08):
metadata:
annotations:
# A NEW value requests a new run; re-applying the same value is a no-op.
kopiur.home-operations.com/run-requested: "2026-01-01T00:00:00Z"
kopiur.home-operations.com/run-mode: full # quick (default when absent) | full
run-requestedis an RFC3339 timestamp. A new timestamp requests a new run; re-applying the same value is a no-op once that request was handled.run-modeisquick(the default when absent) orfull.
Imperative equivalent
For a one-off run without editing Git, stamp the same two annotations with
kubectl (--overwrite because the timestamp changes each time):
The outcome lands in status.manualRun (requestedAt, mode, phase:
Running/Succeeded/Failed, completedAt). The
kubectl plugin wraps this as
kubectl kopiur maintenance run [NAME | --repository NAME] [--full] [--wait].
Repositories bootstrapped by kopiur ≤ 0.3.x: claim the lease once
kopia records the repository CREATOR as the maintenance owner. Older kopiur
releases left that as the (ephemeral) bootstrap pod's identity, so every
maintenance run saw a "foreign" owner and takeoverPolicy: Never yielded
forever — maintenance silently never ran. New bootstraps stamp a stable,
lease-derived owner (kopiur@kopiur-<ns>-<repo>); for repositories created
before that, set spec.ownership.takeoverPolicy: Force once (or run
kopia maintenance set --owner kopiur@<lease> by hand) — the next run claims
the lease and subsequent runs proceed normally.
Inspecting status¶
Key status fields:
| Field | Meaning |
|---|---|
ownership.owner / ownership.claimedAt |
Current lease holder and when it was claimed. |
quick.lastRunAt / full.lastRunAt |
Timestamp of the most recent run of each pass. |
quick.lastHandledAt / full.lastHandledAt |
The most recent cron slot whose Job finished — including a yield (which doesn't move lastRunAt), so a handled slot never re-fires. |
quick.lastContentReclaimedBytes / full.lastContentReclaimedBytes |
Storage reclaimed — the only place this is surfaced. |
conditions[type=LeaseOwned] |
True when this resource holds the lease and is running; False (with a reason) when waiting on the repository, a held lease, or a failure. |
The running mover Jobs are labeled, so you can watch them directly:
Reclaimed bytes currently reports 0
kopia maintenance run does not emit a machine-readable reclaimed-bytes figure, so lastContentReclaimedBytes is reported as 0 today even though the run does reclaim space. The field exists and round-trips; populating it precisely is a planned enhancement.
Behavior you can rely on¶
- Runs at the scheduled time. Spawning is gated on the same cron + jitter logic as
SnapshotSchedule, seeded deterministically per resource, so two replicas agree and the run lands in its window — it is not "every reconcile". - Waits for the repository. Maintenance only runs once the target repository reports
Ready(an object-store repository must finish connecting or being created first). Until then the resource showsLeaseOwned=False, reason=WaitingForRepository. - One run at a time. The operator never starts a second maintenance Job for a repository while one is in flight.
- Catches up after downtime — once. If the operator is down across several scheduled slots, it runs a single catch-up pass on recovery, not a storm of missed runs.
- Self-cleaning Jobs. Finished maintenance Jobs are removed automatically (
ttlSecondsAfterFinished); a failed run is retried with backoff. - A handled slot never re-fires. Each scheduled slot runs once — its outcome (a real run or a deliberate yield to a foreign lease holder) is recorded durably in
status.<quick|full>.lastHandledAt, so the Job self-cleanup above cannot make the same slot run again. Only a failed slot is retried. - Yielding is loud, not silent. When every run yields (a foreign owner holds the lease and
takeoverPolicy: Never), kopia GC/compaction is not happening — the resource reportsReady=False, reasonMaintenanceYielding, with thetakeoverPolicy: Forceremediation in the message, instead of a misleadingReady=True.
See also¶
deploy/examples/08-maintenance.yaml— a standaloneMaintenance.deploy/examples/01-single-pvc-scheduled.yaml— inlinespec.maintenance.- Share one repository across clusters — the full multi-cluster walkthrough, including the maintenance-ownership steps above in their correct order.
- Troubleshooting → Maintenance isn't running —
LeaseHeldByOtheron a shared repository: expected vs. stale.