When a container that a volume backup stopped no longer existed (for
example after a redeploy), recovery kept it as remaining. The execution
needed attention and every later run of the schedule was skipped.
- Treat "no such container" or "no such object" from docker inspect as
recovered; other failures still need attention
Require deploy or root token access when creating managed mounts outside
resource directories or changing their file contents through the API.
Synchronize KeyDB and Dragonfly passwords with local and shared
REDIS_PASSWORD values while preserving secret manager resolution.
Recovery attempts now back off exponentially, stop after 8 attempts, and
mark the execution as needing attention with a failure category. The
scheduler dispatches at most 100 recovery jobs per run, and a pending S3
cleanup no longer blocks new backups. S3 delete errors now raise with
their cause.
The scheduler dispatched VolumeBackupRecoveryJob every minute for each
pending execution, and each job retried 20 times with a 60 second backoff.
A stuck execution kept about 20 failing jobs active at a time.
- Run each recovery job once; the scheduler is the only retry path.
- Make the job unique per execution so duplicates do not pile up.
- Dispatch recovery for an execution at most once every 5 minutes.
- Clear the pending flag and record the reason when recovery cannot
succeed (server, resource, S3 storage, or filename no longer exists),
so it does not fail forever and block new backups.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Store s3_storage_id on scheduled volume backup executions so retention
and recovery use the S3 that received the upload. Extract
DeleteScheduledVolumeBackup for UI and resource deletion, and isolate
database backup retention failures so cleanup errors do not fail the job.
Add polymorphic volume backup scheduling for persistent volumes and directories, expose schedule management via API, and reorganize backup configuration and execution views.
Andras BacsaiandClaude Opus 5.5