- Restart a running Sentinel that has not pushed for 10 minutes, at most
once per hour. The restart applies the current URL and token.
- Restart an unhealthy Sentinel. Do not restart while its health check
starts or while its API does not answer yet, which caused restart loops.
- Remember why pushes fail: the reason Coolify rejects a push, and the last
push error from Sentinel's /api/push-status. Show it on the Sentinel page
while Sentinel is out of sync.
- When the instance URL or public IP changes, move remote servers that use
the generated Sentinel URL to the new URL. Custom URLs stay.
The missed daily backups on Coolify Cloud came from Redis evicting queue keys
(allkeys-lfu on a full instance), not from the scan of all schedules. The
next_run_at design (452259aac, 5c5740533) did not prevent that, and it added
stored state that must be kept in sync on every frequency, timezone, and server
change. It was not released, so remove it:
- revert next_run_at, its model hooks, the "Next run" UI, and the diagnostics
changes; restore scheduled_job_states and the queue overlap lock
- a new migration removes the next_run_at columns and creates
scheduled_job_states again where the removed migration already ran
Keep the changes that fix real problems:
- run the manager every minute from the scheduler with `scheduled:dispatch`,
so a busy queue cannot delay it past the catch-up window
- load the server chain of scheduled backups with each chunk, which removes
about 7,000 queries per run on Coolify Cloud
- catch Throwable for each schedule, so one broken schedule does not stop the
others
- publish database and volume backups again when their queued job was not
started after 60 minutes (for example, lost in Redis); log lost tasks and
Docker cleanups as missed. A lost backup is not run again when a newer
occurrence of the same schedule exists.
- database import: add keep_owners (restore PostgreSQL owners and
privileges) and sqlite_database (pick the target SQLite file) to the
API, OpenAPI spec, and import form
- oauth: require password confirmation and support team selection on
OAuth login
- remote process: send team broadcast events to the team of the server
that ran the process instead of the causer's user id
- servers API: handle the legacy build server setting
- compose: support parent bind paths and volume driver options
- dns: improve record hints and provider management
- notifications: fix throttling and missing backup notifications
- ui: rework the confirmation modal
- add feature and unit tests for the new behavior
High disk usage notifications were sent every ~10 minutes on Coolify Cloud
because the cache rate limiter did not hold across Sentinel-triggered checks.
- Add a notification_throttles table and NotificationThrottle helper that
atomically records when a notification was sent for a resource.
- Add the ThrottledNotification contract; Team::notify() skips a throttled
notification that was already sent within its interval.
- High disk usage: configurable per-server interval
(server_disk_usage_notification_interval_hours, default 24h) in
Server > Advanced and the servers API.
- Container restarted (coolify-proxy, database TCP proxy): at most one
notification per hour per server/database during a restart loop.
- Move the unreachable and missing-backup markers into the new table and
migrate existing data (servers.unreachable_notification_sent and
scheduled_database_backups.missing_backup_notification_sent_at).
- Regenerate OpenAPI files.
The scheduled job manager checked every enabled schedule each minute and
evaluated its cron expression in PHP. On Coolify Cloud one run took more than
60 seconds, so runs overlapped and minutes were skipped.
Each schedule now stores its next due time:
- next_run_at on scheduled database backups, scheduled tasks, and volume
backups, and docker_cleanup_next_run_at on server settings (UTC, indexed)
- model hooks calculate it when a schedule is created or its frequency or
enabled flag changes; a server timezone change or a move to another server
calculates it again for the affected schedules
The `scheduled:dispatch` command runs the manager every minute from the
scheduler, not from a queue. It selects only due rows and claims each
occurrence with an atomic update of next_run_at, together with its delivery
record, so parallel runs on several nodes never dispatch it twice.
- database and volume backups run once when they are late; tasks and Docker
cleanups more than 10 minutes late are logged as missed
- a local time that occurs two times when the clocks go back runs once
- the migration continues each schedule after its last recorded occurrence and
drops scheduled_job_states
Show the next run, in the server timezone, on the database backup, scheduled
task, volume backup, and Docker cleanup pages, or "Invalid frequency" when the
frequency cannot be calculated. scheduled:diagnostics now lists overdue
schedules and open deliveries.
Remove workarounds that are no longer needed: the queue overlap lock and its
stale-lock repair, the catch-up window, and debug-only logs.
- ServerSetting: default is_traffic_analytics_enabled to false on the
model, so a new instance does not show null before a refresh.
- Update navbar, mobile menu, and proxy button tests to the current
redesigned UI (split action menus, settings rail, neutral icons).
- Traffic nudge test: set server_role to mark a build server, the same
as the product code.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Mark Analytics, Dashboard\TrafficAnalytics and Project\Application\Analytics
Livewire components #[Lazy] with skeleton placeholder() views so pages render
instantly while Sentinel data loads in a deferred request
- Add SentinelTrafficClient::prefetchServerWide() to warm all server-wide
endpoints in a single batched docker exec instead of ~14-15 serial SSH
round-trips, and refactor overview/paths/breakdown URL building into shared
helpers
- Cap per-app leaderboard overviews fetched via MAX_LEADERBOARD_APPS to bound
requests on servers with many apps
- Add hasRequestSeries() to detect empty request charts and render a no-data
state instead of a blank chart
- Gate ServerSetting's automatic Sentinel restart on is_sentinel_enabled so
tuning geoip/traffic settings no longer silently enables Sentinel
- Add reusable skeleton UI components (skeleton, skeleton/table, skeleton/tiles)
for the new placeholder views
Store each server's Docker version and build stop commands via
dockerStopCommand() so newer Docker uses --timeout instead of
deprecated --time/-t. Also show Docker version in server details.
- Add live 24h polling toggle to server/application analytics views
- Add geo visualization (world map, country flags/names) for traffic
- Add dashboard nudge for servers eligible but not yet analytics-enabled
- Add lazy-loaded TrafficOverview widget to application General page
- Default-enable traffic analytics on new eligible server settings
- Rotate Caddy access logs via lumberjack roll options
- Add Traefik logrotate sidecar for copytruncate access-log rotation
Use the current request scheme/host when no FQDN or public IP is set,
skipping loopback hosts. Polish boarding/server create cards, optional
listbox tooltips, and cover the layout and Sentinel URL behavior in tests.
Ensure a Sentinel endpoint from instance FQDN/IP when enabling, use process dialogs for validation with install state, and tidy server create, boarding, metrics empty state, and log timestamps.
Models now declare $hidden for passwords, tokens, db URLs, and compose
fields. API controllers flip from makeHidden-on-deny to makeVisible-on-
allow (can_read_sensitive=true), fixing fields that were never hidden.
Also adds missing fields (mysql/mariadb passwords, logdrain keys, etc.)
to privileged disclosure lists.
Tests added: Feature/Security/ApiSensitiveFieldsTest and
Unit/Models/SensitiveFieldsHiddenTest cover all affected models and
controllers.
Replace random string with encrypted JSON payload containing
server_uuid, binding token to its server for validation.
Remove double-encrypt test no longer relevant to new token format.
Replace hard validation error with self-healing token logic. Tokens that
are null, empty, or fail decryption are now regenerated automatically
rather than crashing sentinel startup or metrics reads.
Token format changed from encrypted JSON payload to a plain 64-char
random string (Str::random), eliminating double-encryption issues and
simplifying the validation regex to cover the new character set.
New `ensureValidSentinelToken()` method on ServerSetting centralises
the get-or-regenerate contract; both StartSentinel and HasMetrics now
delegate to it. HasMetrics logs a warning when regeneration occurs so
operators know a sentinel container restart is required.
`isValidSentinelToken()` now accepts `?string` (null → false).
Adds feature tests covering: null/empty/undecryptable stored values,
idempotent return of valid tokens, RuntimeException only when
regeneration itself produces an invalid token, no double-encryption of
newly generated tokens, and cast round-trip consistency.
Add `connection_timeout` field to server settings, allowing per-server
override of the global SSH connection timeout constant.
- Migration adds `connection_timeout` integer column (default 10s)
- `ServerSetting` model exposes and casts the new field
- `SshMultiplexingHelper::getConnectionTimeout()` resolves per-server
value with fallback to `constants.ssh.connection_timeout`
- All SSH/SCP command builders use the new resolver instead of the
global config directly
- Livewire `Show` component binds `connectionTimeout` with validation
(1–300 seconds) and syncs to/from the model
- UI input added to server settings form with helper text
- Feature tests cover default, persistence, resolver, and fallback
Replace all uses of `forceFill`, `forceCreate`, and `forceFill` with their
non-force equivalents across models, actions, controllers, and Livewire
components. Add explicit `$fillable` arrays to all affected Eloquent models
to enforce mass assignment protection.
Add ModelFillableCreationTest and ModelFillableRegressionTest to verify that
model creation respects fillable constraints and prevent regressions.
Replace $guarded usage with explicit $fillable arrays across all models.
Sync fillable definitions with current database schema and add tests.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Restrict mass-assignable attributes across user/team/redis models and
switch privileged root/team creation paths to forceFill/forceCreate.
Encrypt legacy ClickHouse admin passwords via migration and cast the
correct ClickHouse password field as encrypted.
Tighten API and runtime exposure by removing sensitive team fields from
responses and sanitizing Git/compose error messages.
Expand security-focused feature coverage for command-injection and mass
assignment protections.
Adds a new server-level setting that allows administrators to disable
per-application image retention globally for all applications on a server.
When enabled, Docker cleanup will only keep the currently running image
regardless of individual application retention settings.
Changes:
- Add migration for disable_application_image_retention boolean field
- Update ServerSetting model with cast
- Add checkbox in DockerCleanup page (Advanced section)
- Modify CleanupDocker action to check server-level setting
- Update Rollback page to show warning and disable inputs when server
retention is disabled
- Add helper text noting server-level override capability
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add configurable deployment_queue_limit server setting (default: 25)
- Check queue size before accepting new deployments
- Return 429 status for webhooks/API when queue is full (allows retry)
- Show error toast in UI when queue limit reached
- Add UI control in Server Advanced settings
Fixes#6708🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Andras Bacsai
Aditya Tripathi
peaklabs-dev
SierraJC
Andras Bacsai