diff --git a/.ai/lessons.md b/.ai/lessons.md index fadfc3fae0..31af053617 100644 --- a/.ai/lessons.md +++ b/.ai/lessons.md @@ -1,5 +1,9 @@ # Lessons +## Keep imported data exact +- The global `TrimStrings` and `ConvertEmptyStringsToNull` middleware change every string of a JSON request. An endpoint that receives stored records (for example the server transfer import) must skip them with `skipWhen()`; otherwise text loses its edge whitespace and an empty value becomes null. +- Code that creates models from stored data with model events on also runs their `created` hooks and set-mutators (default destinations, settings, variables, parser versions, ports). Compare the result with the source record by record, and restore or remove what the hooks changed. + ## Prove regressions before changing code - Reproduce the reported failure on the unchanged baseline before adding a fix. - When a symptom matches an earlier fix, inspect that fix and prove why it no longer works before adding another workaround. @@ -23,9 +27,12 @@ - Do not symlink `vendor` into another worktree. Composer resolves the base path from the real `vendor` path, so the tests load the app classes of the other checkout. Run `composer install --ignore-platform-req=ext-zip` in the worktree. - To assert that the unsaved-changes bar shows, check `el.classList.contains('is-dirty')`. Playwright treats the hidden bar (`opacity-0`) as visible, and its class attribute always contains the text `is-dirty` from Tailwind variants. - In browser tests, address `x-forms.input` fields by `name` (`type('fieldName', ...)`); its HTML `id` is not always the Livewire property, so `#property` waits forever. +- For load tests against a dev VM, use your own instance (`COOLIFY_DEV_INSTANCE= COOLIFY_DEV_SIDE_BY_SIDE=1 ./scripts/dev start`); other sessions can reseed the localhost VM of a shared instance. The 1 GiB VM drops packets when its conntrack table is full (`dmesg`: `nf_conntrack: table full`), so raise `net.netfilter.nf_conntrack_max` on it and run an idle load first to prove a clean baseline. - Do not call `sleep()` in a browser test. The app server runs in the test process, so `sleep()` stops it; use `$page->wait()`. +- `Queue::fake()` also captures `dispatchSync()` of a `ShouldQueue` job, so the job never runs. Fake only the jobs to block (`Queue::fake([SomeJob::class])`). - In tests, turning on `is_metrics_enabled` (and other Sentinel settings) with `save()`/`update()` runs the settings hook, which starts Sentinel over a real SSH connection on the sync queue. Use `saveQuietly()` or `Queue::fake()`. - Call `visit()` directly in each `tests/v4/Browser` test body; Pest does not mark a test that only uses helper-wrapped `visit()` as a browser test, so it fails with `sendText() on null`. +- The root team (id 0) has `personal_team = true`, so `personal_team` does not identify a user's own team. To clean up test users, detach them from shared teams and then delete the user; never delete a team selected by a filter. A Team delete revokes the team's API tokens and deletes its private keys before any guard can stop it. - SQLite returns `0`/`1` for boolean columns without a `boolean` cast, while Postgres returns real booleans. A strict `=== false` check on such a column never matches in tests, so tests can pass on behavior that Postgres blocks. Add the cast or compare loosely. ## Verify the complete user flow @@ -104,6 +111,9 @@ - Host test runs share `storage/` with the dev container. Fake the `ssh-keys`/`ssh-mux` disks and `Process` in tests that run seeders or write SSH files, or the test deletes/chowns the dev instance's SSH keys and SSH breaks for `www-data`. - Dev QEMU servers from `dev:qemu` are seeded, not validated: they have no `coolify` Docker network, and Alpine has no bash until `InstallPrerequisites` runs. +## Report failures of queued actions +- A value that an action returns when it runs as a queued job (`::dispatch()`) goes nowhere. Do not report a failure only with a return string or `throwError: false`; log it and broadcast it to the page that started the action. + ## Format only your own files - `pint --dirty` also rewrites uncommitted files that belong to other work in the tree. When the tree has unrelated changes, pass your changed paths to Pint. @@ -133,3 +143,9 @@ ## Session writes are last-writer-wins - Laravel saves the whole session payload at the end of every request, and pages run concurrent `wire:poll` requests. A value that one request changes can be overwritten by a request that started earlier. `KeepConcurrentTeamSwitch` protects `currentTeam`; protect any new session value that changes mid-session the same way, or keep it out of the session. + +## Keep `:has()` off page-wide ancestors +- A `:has()` on `html`, `body`, `.app-shell`, or a sibling of `
` (for example `html:has(.x) .y`) makes Chrome restyle the page on every DOM change. Chart libraries read layout many times per render, so one ApexCharts render took ~400 ms and blocked navigation. Use a class set by the Blade layout instead, and keep `:has()` on small, local subjects. + +## Validate input safely +- In PCRE, `$` also matches before a final `\n`. In validation regexes, use `\z` or reject control characters first, or a trailing newline passes. diff --git a/.codex/config.toml b/.codex/config.toml index 300ea316fd..864e1fbd98 100644 --- a/.codex/config.toml +++ b/.codex/config.toml @@ -1,4 +1,3 @@ [mcp_servers.laravel-boost] command = "php" args = ["artisan", "boost:mcp"] -cwd = "/Users/heyandras/devel/coolify" diff --git a/.cursor/rules/coolify-ai-docs.mdc b/.cursor/rules/coolify-ai-docs.mdc deleted file mode 100644 index f3080de0e3..0000000000 --- a/.cursor/rules/coolify-ai-docs.mdc +++ /dev/null @@ -1,156 +0,0 @@ ---- -title: Coolify AI Documentation -description: Master reference to all Coolify AI documentation in .ai/ directory -globs: **/* -alwaysApply: true ---- - -# Coolify AI Documentation - -All Coolify AI documentation has been consolidated in the **`.ai/`** directory for better organization and single source of truth. - -## Quick Start - -- **For Claude Code**: Start with `AGENTS.md` in the root directory -- **For Cursor IDE**: Start with `.ai/README.md` for navigation -- **For All AI Tools**: Browse `.ai/` directory by topic - -## Documentation Structure - -All detailed documentation lives in `.ai/` with the following organization: - -### ๐Ÿ“š Core Documentation -- **[Technology Stack](.ai/core/technology-stack.md)** - All versions, packages, dependencies (SINGLE SOURCE OF TRUTH for versions) -- **[Project Overview](.ai/core/project-overview.md)** - What Coolify is, high-level architecture -- **[Application Architecture](.ai/core/application-architecture.md)** - System design, components, relationships -- **[Deployment Architecture](.ai/core/deployment-architecture.md)** - Deployment flows, Docker, proxies - -### ๐Ÿ’ป Development -- **[Development Workflow](.ai/development/development-workflow.md)** - Dev setup, commands, daily workflows -- **[Testing Patterns](.ai/development/testing-patterns.md)** - How to write/run tests, Docker requirements -- **[Laravel Boost](.ai/development/laravel-boost.md)** - Laravel-specific guidelines (SINGLE SOURCE for Laravel Boost) - -### ๐ŸŽจ Code Patterns -- **[Database Patterns](.ai/patterns/database-patterns.md)** - Eloquent, migrations, relationships -- **[Frontend Patterns](.ai/patterns/frontend-patterns.md)** - Livewire, Alpine.js, Tailwind CSS -- **[Security Patterns](.ai/patterns/security-patterns.md)** - Auth, authorization, security -- **[Form Components](.ai/patterns/form-components.md)** - Enhanced forms with authorization -- **[API & Routing](.ai/patterns/api-and-routing.md)** - API design, routing conventions - -### ๐Ÿ“– Meta -- **[Maintaining Docs](.ai/meta/maintaining-docs.md)** - How to update/improve documentation -- **[Sync Guide](.ai/meta/sync-guide.md)** - Keeping docs synchronized - -## Quick Decision Tree - -**What are you working on?** - -### Running Commands -โ†’ `.ai/development/development-workflow.md` -- `npm run dev` / `npm run build` - Frontend -- `php artisan serve` / `php artisan migrate` - Backend -- `docker exec coolify php artisan test` - Feature tests (requires Docker) -- `./vendor/bin/pest tests/Unit` - Unit tests (no Docker needed) -- `./vendor/bin/pint` - Code formatting - -### Writing Tests -โ†’ `.ai/development/testing-patterns.md` -- **Unit tests**: No database, use mocking, run outside Docker -- **Feature tests**: Can use database, MUST run inside Docker -- Critical: Docker execution requirements prevent database connection errors - -### Building UI -โ†’ `.ai/patterns/frontend-patterns.md` + `.ai/patterns/form-components.md` -- Livewire 3.5.20 with server-side state -- Alpine.js for client interactions -- Tailwind CSS 4.1.4 styling -- Form components with `canGate` authorization - -### Database Work -โ†’ `.ai/patterns/database-patterns.md` -- Eloquent ORM patterns -- Migration best practices -- Relationship definitions -- Query optimization - -### Security & Authorization -โ†’ `.ai/patterns/security-patterns.md` + `.ai/patterns/form-components.md` -- Team-based access control -- Policy and gate patterns -- Form authorization (`canGate`, `canResource`) -- API security with Sanctum - -### Laravel-Specific -โ†’ `.ai/development/laravel-boost.md` -- Laravel 12.4.1 patterns -- Livewire 3 best practices -- Pest testing patterns -- Laravel conventions - -### Version Numbers -โ†’ `.ai/core/technology-stack.md` -- **SINGLE SOURCE OF TRUTH** for all version numbers -- Laravel 12.4.1, PHP 8.4.7, Tailwind 4.1.4, etc. -- Never duplicate versions - always reference this file - -## Critical Patterns (Always Follow) - -### Testing Commands -```bash -# Unit tests (no database, outside Docker) -./vendor/bin/pest tests/Unit - -# Feature tests (requires database, inside Docker) -docker exec coolify php artisan test -``` - -**NEVER** run Feature tests outside Docker - they will fail with database connection errors. - -### Form Authorization -ALWAYS include authorization on form components: -```blade - -``` - -### Livewire Components -MUST have exactly ONE root element. No exceptions. - -### Version Numbers -Use exact versions from `technology-stack.md`: -- โœ… Laravel 12.4.1 -- โŒ Laravel 12 or "v12" - -### Code Style -```bash -# Always run before committing -./vendor/bin/pint -``` - -## For AI Assistants - -### Important Notes -1. **Single Source of Truth**: Each piece of information exists in ONE location only -2. **Cross-Reference, Don't Duplicate**: Link to other files instead of copying content -3. **Version Precision**: Always use exact versions from `technology-stack.md` -4. **Docker for Feature Tests**: This is non-negotiable for database-dependent tests -5. **Form Authorization**: Security requirement, not optional - -### When to Use Which File -- **Quick commands**: `AGENTS.md` or `development-workflow.md` -- **Detailed patterns**: Topic-specific files in `.ai/patterns/` -- **Testing**: `.ai/development/testing-patterns.md` -- **Laravel specifics**: `.ai/development/laravel-boost.md` -- **Versions**: `.ai/core/technology-stack.md` - -## Maintaining Documentation - -When updating documentation: -1. Read `.ai/meta/maintaining-docs.md` first -2. Follow single source of truth principle -3. Update cross-references when moving content -4. Test all links work -5. See `.ai/meta/sync-guide.md` for sync guidelines - -## Migration Note - -This file replaces all previous `.cursor/rules/*.mdc` files. All content has been migrated to `.ai/` directory for better organization and to serve as single source of truth for all AI tools (Claude Code, Cursor IDE, etc.). diff --git a/AGENTS.md b/AGENTS.md index 661a920c56..1ab68ce6b8 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -29,7 +29,7 @@ Docker Compose-based dev setup with services: coolify (app, which also runs Reve docker compose -f docker-compose.yml -f docker-compose.dev.yml up -d ``` -The main checkout serves its branch at `localhost:8000` (Reverb `6001`, terminal `6002`, db `5432`, redis `6379`, vite `5173`). Worktrees get a port block at `20000 + slot*10` (app `+0`, Reverb `+1`, terminal `+2`, db `+3`, redis `+4`, vite `+5`). Each instance has its own libvirt network `coolify-dev-` (`10.221..0/24`) and VMs `coolify-dev---`; VMs are reused, use `php artisan dev:qemu --fresh` to rebuild one. On macOS with Lima >= 2.0 (`brew install lima`), the localhost VM is the Lima instance `coolify-dev--` (same profiles, users, and SSH key); Coolify reaches it through Lima's forwarded SSH port on `host.docker.internal`, and only guest ports 80/443 are forwarded to the Mac. Rebuild it with `limactl delete -f `. Force it with `COOLIFY_DEV_SERVER_BACKEND=lima`. If `APP_URL` in `.env` is a `*.ts.net` host, the browser ports are published with `tailscale serve`. Set `COOLIFY_DEV_INSTANCE=` to run another instance from the same checkout. +The main checkout serves its branch at `localhost:8000` (Reverb `6001`, terminal `6002`, db `5432`, redis `6379`, vite `5173`). Worktrees get a port block at `20000 + slot*10` (app `+0`, Reverb `+1`, terminal `+2`, db `+3`, redis `+4`, vite `+5`). Each instance has its own libvirt network `coolify-dev-` (`10.221..0/24`) and VMs `coolify-dev---`; VMs are reused, use `php artisan dev:qemu --fresh` to rebuild one. On macOS with Lima >= 2.0 (`brew install lima`), the localhost VM is the Lima instance `coolify-dev--` (same profiles, users, and SSH key); Coolify reaches it through Lima's forwarded SSH port on `host.docker.internal`, and only guest ports 80/443 are forwarded to the Mac. Rebuild it with `limactl delete -f `. Force it with `COOLIFY_DEV_SERVER_BACKEND=lima`. To pass the host NVIDIA GPU through to the KVM localhost VM (`ubuntu-root` only), start with `COOLIFY_DEV_GPU=1 ./scripts/dev start` (or `php artisan dev:qemu ubuntu-root --as-localhost --gpu`). This attaches every PCI function of the GPU, gives the VM 4 GiB, stops `nvidia-persistenced` on the host, and installs the NVIDIA driver and container toolkit in the VM. The host cannot use the GPU while the VM runs. Remove it with `--fresh` without `--gpu`. If `APP_URL` in `.env` is a `*.ts.net` host, the browser ports are published with `tailscale serve`. Set `COOLIFY_DEV_INSTANCE=` to run another instance from the same checkout. ## Testing the Self-Hosted Upgrade Process @@ -262,6 +262,7 @@ The persistent-storage implementation is the reference pattern: `Project\Service - Run `vendor/bin/pint --dirty --format agent` before finalizing changes - Every change must have tests โ€” write or update tests, then run them. For bug fixes, follow TDD: write a failing test first, then fix the bug (see Test Enforcement below) - Check sibling files for conventions before creating new files +- Nixpacks is deprecated. Never recommend it to users (issue replies, docs, UI copy, workarounds). Recommend the Dockerfile build pack, or Railpack where it fits. - When adding remote shell commands, account for servers using non-root SSH users: commands pass through `parseCommandsByLineForSudo()`, so test pipelines, redirects, substitutions, and `sh -c`/`bash -c` scripts with the non-root sudo parser. ## Git Workflow diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index d93150c6bf..5fca2d5408 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -230,6 +230,8 @@ To build and run Coolify locally, see: [Development](./DEVELOPMENT.md) ### Testing the Coolify Helper Locally +When you change a tool version in `docker/coolify-helper/Dockerfile`, run `./scripts/helper-checksums` to update the SHA256 checksums of the downloads. + Use `scripts/dev-helper` to build a local helper image and test it with the running development instance. The script requires the standard local Coolify container and the seeded Dockerfile, Docker Compose, and Nixpacks applications. Run the complete workflow: diff --git a/DESIGN.md b/DESIGN.md index ef61480091..633d9e0256 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -25,15 +25,21 @@ change. - **Layers, not boxes.** Depth comes from a few surface steps (canvas, card, well, control), 1px borders, and very soft shadows. Never use thick borders or heavy floating shadows. -- **Card and well.** Content sits in a darker inset panel (the well) inside a - soft outer box with a small title row. +- **Card and well.** Content sits in an inset panel (the well) inside a soft + outer box with a small title row. In light mode the well is lighter than the + outer box. In dark mode a top-level card is a near-canvas outer box with a + lighter `#121212` well. - **Tactile buttons.** Buttons and navigation icon tiles have a 2px bottom edge. They lift 1px on hover and press down on click. -- **Dark gray, not black.** Dark mode is a pure neutral dark gray (page - canvas about sRGB 10,10,10). Light mode is pure neutral - gray. -- **One accent per theme.** Purple in light mode, yellow in dark mode (purple - has too little contrast on dark surfaces). Everything else is neutral. +- **Dark gray, not black.** Dark mode page canvas and the top bar are neutral + sRGB 10,10,10 (`#0a0a0a`). Card wells sit one step up (`#121212`). Light mode is + pure neutral gray. +- **One accent.** Coolify purple in both themes (`#6b16ed`, hover `#7317ff`, + pressed `#5a12c7`, depth `#4a0fa3`, light text and icons `#a391ff`). Yellow + is only for real warnings. In the default dark theme the sidebar, including + the logo cell, is purple-black (`#120e1a`) with a faint brand wash. Users + who want a neutral sidebar can pick a custom theme. The dark command palette + has a light purple tint. The page canvas stays neutral. - **Quiet text.** Sentence case. A clear text ladder: primary, dim, faint. No em dashes in UI copy. @@ -94,8 +100,8 @@ follow the theme. .dark { /* Surfaces (dark: pure neutral dark gray) */ - --ui-canvas: oklch(14.48% 0 0); /* sRGB 10,10,10 */ - --ui-panel: oklch(19.13% 0 0); /* sRGB 20,20,20: header bar, card and table headers */ + --ui-canvas: #0a0a0a; /* page and top bar */ + --ui-panel: #121212; /* cards and table headers, not the top bar */ --ui-card-bg: rgb(255 255 255 / 6%); --ui-card-border: rgb(255 255 255 / 8%); --ui-well-bg: rgb(0 0 0 / 34%); @@ -114,8 +120,15 @@ follow the theme. --ui-fg-dim: oklch(76.99% 0 0); --ui-fg-faint: oklch(65% 0 0); - /* Accent and status: yellow accent in dark mode */ - --ui-accent: oklch(88.13% 0.1507 91.7); + /* Accent stays purple. Yellow is the warning color, not the accent. */ + --ui-accent: #6b16ed; + --ui-accent-top: #7317ff; + --ui-accent-bottom: #5a12c7; + --ui-accent-edge: #4a0fa3; + --ui-accent-fg: #ffffff; + --ui-accent-light: #a391ff; + --ui-sidebar: #120e1a; + --ui-card: #121212; --ui-success: oklch(72.27% 0.192 149.58); --ui-warning: oklch(88.13% 0.1507 91.7); --ui-error: oklch(66% 0.2 27.33); @@ -131,16 +144,40 @@ Rules for the tokens: - Floating surfaces (menus, popovers, tooltips, modals) must be opaque. Use `--ui-menu-bg`, never a translucent value, or the content behind shows through. -- Keep the dark surfaces pure neutral (chroma 0). A brand tint on the - surfaces was tried and removed. +- Keep the page canvas, cards, menus, and logs pure neutral (chroma 0). The + default dark sidebar, the logo cell above it, and the dark command palette + are the tinted surfaces. - **Dark lightness level.** The canvas is at 14.48% and the header panel at 19.13%. To change the level, move every dark surface and border token by the same amount (canvas, panel, menu, hover, borders, and the Coolify surface ladder), so the steps between layers stay the same. If you move only some of them, a hover fill can become darker than its menu. -- The primary buttons use purple in both themes. Only focus rings, active - navigation, and other accent states change to yellow in dark mode. +- Primary buttons, focus rings, and active navigation use purple in both + themes. Warning yellow is not an accent. +- Do not hard-code the brand purple (`#6b16ed`, `rgb(107 22 237 / โ€ฆ)`, + `rgb(163 145 255 / โ€ฆ)`, `#8a4dff`), the dark canvas (`#0a0a0a`), or the + card color (`#121212`) in component CSS. Use the tokens: `--color-brand`, + `-hover`, `-press`, `-depth`, `-light`, `-ring`, `-foreground`, + `--color-sidebar`, `--color-app`, and `--color-card`. For a translucent tint, use + `color-mix(in srgb, var(--color-brand) 14%, transparent)`. A custom theme + (`html[data-theme="custom"]`) overrides only these tokens, so a hard-coded + value stays purple when the user picks another color. +- The custom theme has four colors: Sidebar, Selected (the current page in + the sidebar), Accent, and Background. The accent gives the brand fill and its + hover, press, depth, and ring shades, and `--color-brand-foreground` (the + computed `--theme-accent-foreground`, so text stays readable on a light color + such as yellow). The surface, border, and text tokens come from one ladder + in `app.css` that mixes the readable text color into the background. The + sidebar uses the same ladder with the sidebar color, and it can have the + other light/dark scheme than the page (`data-sidebar-scheme`; `dark:` + utilities follow it). When you add a color token, give it a value in that + ladder. +- A translucent fill on a control that sits directly on the page (not in a + card or table) shows the floating-navigation dots through it. Mix the tint + with the canvas instead: + `color-mix(in srgb, var(--color-brand) 6%, var(--color-app))`. Controls in + cards keep the translucent tint, so they blend with the card. ## P3. Scales @@ -179,15 +216,56 @@ Class names below are examples. Keep the values and structure. - Three layers: the canvas (page and sidebar, `--ui-canvas`), the top header bar (`--ui-panel`, 48px high), and the cards on the canvas. -- In dark mode, the header bar, the outer box of a top-level card, and the - header row of a table use the same color (`--ui-panel`). The card wells and - table rows are the darker content layer below them. -- The sidebar is the canvas color and has a 1px `--ui-chrome-border` right - edge. The brand block (product name and version) at the top of the sidebar - column also uses the canvas color, with a 1px bottom border. +- In dark mode, the outer box of a top-level card is + `color-mix(in srgb, var(--ui-panel) 66%, black)` (about `#0c0c0c`), and its + well is `--ui-panel` (`#121212`). Tables use the same pair: the darker mix + for the header row and `--ui-panel` for the rows. +- In light mode the sidebar is the canvas color and has a 1px + `--ui-chrome-border` right edge. In the default dark theme the sidebar + column, including the logo cell, is `#120e1a` with a faint lavender right + edge (`rgb(163 145 255 / 0.10)`). The nav has a brand-purple wash that + starts at 9% and fades out; the logo cell has a flat 9% wash, so the two + read as one surface. A custom theme gets the neutral sidebar rules and its + own sidebar color. +- The top bar matches the canvas. It has no bottom border at rest. After the + page scrolls under it, dark mode draws a neutral hairline + (`rgb(255 255 255 / 0.06)`), a translucent bar, blur, and a soft shadow. + The logo cell stays the solid sidebar color. - Keep the chrome borders faint. A strong border on near-black looks like a line drawn on the page. +### App header (logo cell) + +- The logo cell is the top of the sidebar column, in the 48px top bar. It is + as wide as the sidebar: 224px expanded, 64px collapsed. +- Expanded: the wordmark (15px semibold, tight tracking, `--ui-fg`), then the + version on the same baseline (10.5px medium, `--ui-fg-faint`), with 6px + between them and 16px side padding. An update badge sits at the right end. +- Collapsed: only the 20px logo mark, centered. The cell gets a bottom border + so that it lines up with the top bar edge. +- The update badge is a small pill: 18px high, 6px side padding, 9.5px + semibold text, fully rounded, a 10% fill, and a 25% inset ring of the same + color. It is purple in light mode and warning yellow in dark mode. While an + update runs, the same pill says "Updating". It hides when the sidebar is + collapsed. +- The mobile drawer starts with the same wordmark and version in a 48px row + with a bottom border, and a 32px close button on the right. It has no update + badge. + +### Mobile header + +- Below the desktop breakpoint (1024px) a sticky top bar replaces the desktop + top bar and sidebar. It is about 56px high (12px vertical padding, 16px side + padding, 24px from 640px), translucent (95% of `--ui-panel`) with a small + blur, and has a faint bottom border. +- Left: the logo mark (18px) in a 32px tile with an 8px radius and a neutral + fill (`rgb(255 255 255 / 6%)` in dark mode), then the team switcher. The + team name truncates. +- Right, with 4px gaps: status indicators (configuration warnings, active + deployments), the update badge, the user menu, and a 36px sidebar button + (the same sidebar glyph as the desktop collapse toggle). The sidebar button + opens the drawer from the right. + ### Sidebar navigation ```css @@ -205,21 +283,36 @@ Class names below are examples. Keep the values and structure. box-shadow: 0 2px 0 var(--ui-depth-color); transition: transform 80ms, box-shadow 80ms, border-color 120ms, background-color 120ms; } -.dark .nav-tile { border-color: rgb(255 255 255 / 9%); background: rgb(255 255 255 / 4%); } +.dark .nav-tile { + border-color: rgb(255 255 255 / 0.07); + background: rgb(255 255 255 / 0.03); + color: oklch(72% 0 0); +} .nav-item:hover .nav-tile { transform: translateY(-1px); box-shadow: 0 3px 0 var(--ui-depth-color); } .nav-item:active .nav-tile { transform: translateY(2px); box-shadow: none; } .nav-item.is-active .nav-tile { - color: var(--ui-accent); - border-color: color-mix(in oklab, var(--ui-accent) 45%, transparent); - --ui-depth-color: color-mix(in oklab, var(--ui-accent) 45%, transparent); + color: #fff; + border-color: #8a4dff; + background: linear-gradient(180deg, var(--ui-accent-top), var(--ui-accent-bottom)); + --ui-depth-color: var(--ui-accent-edge); + box-shadow: 0 2px 0 var(--ui-accent-edge); /* no glow: the row clips it */ } ``` - Use outline icons (1.5px stroke) at 16px. A brand item can use a one-color brand mark when it is clearer than an outline icon. -- Rows with a submenu end in a 12px chevron: accent in light mode, faint gray - in dark mode. -- Group rows under 11px section labels, and separate groups with a 1px divider. +- Rows with a submenu end in a 12px chevron. In dark mode the chevron is + hidden until the row is hovered, focused, or current. +- Group rows under section labels (11px in light mode; 10.5px uppercase muted + gray in dark mode), and separate groups with a 1px divider. +- Do not put a glow (a blurred `box-shadow`) on a tile. The current row clips + it, so it shows as a dim square behind the tile. +- In the collapsed sidebar, Search is a tile like the other items: no row fill + and no frame around it. Only the expanded sidebar shows Search as an input. +- A menu that replaces another menu in place starts with a back row: a + centered label and a 12px left chevron (`โ€น Dashboard`). +- The main menu has no row above its items. Dashboard is the first item, right + under Search, on every page. ### Card (outer box, title row, well) @@ -230,7 +323,8 @@ Class names below are examples. Keep the values and structure. background: var(--ui-card-bg); box-shadow: var(--ui-card-shadow); overflow: visible; /* menus and tooltips must escape */ } -.dark .card { background: var(--ui-panel); } /* top-level card: same as the header bar */ +.dark .card { background: color-mix(in srgb, var(--ui-panel) 66%, black); box-shadow: none; } /* top-level card */ +.dark .card > .card-well { background: var(--ui-panel); box-shadow: none; } /* top-level well */ .card-header { display: flex; flex-wrap: wrap; align-items: center; justify-content: space-between; gap: 16px; padding: 4px 8px 10px; @@ -296,6 +390,7 @@ Class names below are examples. Keep the values and structure. - Use one primary button per area. Other actions are neutral buttons. - Destructive actions use the error color for text and focus ring. +- With Appearance โ†’ Style โ†’ Flat, all buttons lose the raised edge. ### Menus, listboxes, and popovers @@ -321,12 +416,11 @@ Class names below are examples. Keep the values and structure. ### Tables - Header row: 40px, 13px medium, `--ui-fg-faint`, and a 1px `--ui-divider` - bottom border. Fill: `rgb(0 0 0 / 2%)` in light mode, `--ui-panel` in dark - mode (the same as the header bar and the card outer box). -- Dark mode rows use the card well color - (`color-mix(in srgb, var(--ui-panel) 66%, black)`, the same as - `--ui-well-bg` over the panel), so a table looks like a card: header on - top, content below. + bottom border. Fill: `rgb(0 0 0 / 2%)` in light mode, + `color-mix(in srgb, var(--ui-panel) 66%, black)` in dark mode (the same as + the card outer box). +- Dark mode rows use the card well color (`--ui-panel`, `#121212`), so a + table looks like a card: header on top, content below. - Data rows: about 48px, 10px 16px padding, a 1px divider, and a faint hover fill. Put the row action at the far right. - Toolbar on top (search left; filter, sort, and add right), pagination in the @@ -335,6 +429,17 @@ Class names below are examples. Keep the values and structure. - Grid view tiles of a collection are cards without a well. In dark mode they are `--ui-panel`, the same as the header bar. +### View switcher + +- A segmented control (`.view-toggle`) on the right of a collection toolbar. +- Table and grid buttons are icon-only (30px square, 14px icon). The name is in + `title` and `aria-label`. +- Show only the view that is not current. In table view the control shows the + grid icon; in grid view, the list icon. Do not show a selected pill for the + current view. +- Text links stay labelled: the `Canvas` link with its "Try it" badge, and the + `List view` link that is the only way back from a canvas. + ### Status badge ```css @@ -362,10 +467,27 @@ Class names below are examples. Keep the values and structure. - Modals use the card anatomy (outer box, title row, well), size to their content on desktop, and put the actions on the right below a divider. Use - `--ui-modal-shadow`. + `--ui-modal-shadow`. In dark mode the edge is only the 1px card border, with + no extra hairline ring outside it. - Toasts are compact cards (max width 26rem) with a status icon tile, a title, an optional description, and a dismiss button. They close after 4 seconds and pause on hover. +- The toast edge blends into the toast background. In dark mode it is + `rgb(255 255 255 / 0.05)`, not the control border. +- The status icon tile is 3D, like a navigation tile: a 1px border and a 2px + bottom edge in the status color (`currentColor` at 22% and 30%). + +### Command palette + +- In dark mode the palette has a light brand tint: the shell is + `color-mix(in oklab, #6b16ed 10%, )` with a + `rgb(163 145 255 / 0.22)` ring, and the results body is 6% purple with a + `rgb(163 145 255 / 0.12)` ring. The search icon is `#a391ff`. The + placeholder is muted lavender. +- On the selected (focused) row in dark mode, the parts on a purple fill are + white: the quick-command and type badges, the icon tile glyph (for example + the plus of a create row), and the chevron. Purple text on a purple fill is + not readable. ### Logs and code @@ -439,15 +561,15 @@ The interface is compact and product-focused: - near-neutral layered surfaces instead of large bordered boxes; - 13โ€“14px UI typography and 32px controls; -- layer cards are a soft outer box around a darker inset well (see section 5), +- layer cards are a soft outer box around an inset well (see section 5), with 1px borders and a restrained `0 1px 2px rgb(0 0 0 / 0.05)` lift, never heavy borders or a strong floating shadow; - full-width data tables for dense collections; - outline Reicon glyphs through ``; -- the Coolify purple brand accent in light mode; -- the readable Coolify yellow accent in dark mode; -- solid active-item fills (neutral black/white opacity), not accent gradients; - active state is a flat selected surface with no accent rail; +- the Coolify purple brand accent in both themes (`#6b16ed`); yellow only for + real warnings, such as "Update available"; +- the active sidebar item is a purple gradient tile with a 2px depth edge and + no glow. The row itself stays transparent; - sentence-case labels and headings; - never use the em dash (`โ€”`) in UI copy. Prefer a period, colon, comma, or ASCII hyphen (`-`) for empty cells and separators. @@ -527,19 +649,18 @@ the card well (`.dark .application-settings-section-body :is(.listbox-panel, ...)`). | `--coollabs-subtle` | 50% neutral | 70.8% neutral | labels and muted titles (light darkened for WCAG AA 4.5:1) | -Accent behavior is intentionally theme-aware: +Accent behavior uses Coolify purple in both themes. Brand tokens live in +`@theme` as `--color-brand` (`#6b16ed`), `--color-brand-hover` (`#7317ff`), +`--color-brand-press` (`#5a12c7`), `--color-brand-depth` (`#4a0fa3`), and +`--color-brand-light` (`#a391ff`). Dark mode sets `--color-accent` to +`--color-brand`. `--color-warning` stays yellow and is only for real warnings. -- **Light mode:** Coolify purple (`coollabs`) for active controls, focus, - primary actions, and navigation accents. -- **Dark mode:** Coolify yellow (`warning`) for the same states because the - original purple did not provide sufficient text and ring contrast. - -Do not hard-code blue focus rings or leave yellow accent utilities active in -light mode. Primary action patterns should normally follow: +Do not hard-code blue focus rings, and do not use `dark:text-warning` as a +stand-in for the accent. Selected chips follow: ```html bg-coollabs/10 text-coollabs ring-coollabs/25 -dark:bg-warning/15 dark:text-warning dark:ring-warning/25 +dark:bg-brand/15 dark:text-brand-light dark:ring-brand-light/25 ``` The filled top-level action/tab treatment uses the same palette at a restrained @@ -554,16 +675,22 @@ lift off the content: - **Content canvas** is the base layer: `bg-app` in dark (deepest, `--color-app` `oklch(14.48% 0 0)`, sRGB 10,10,10), `bg-neutral-50` in light. The `
` content area and page body use it. -- **Topbar chrome** uses `bg-panel` in dark (`--color-panel` - `oklch(19.13% 0 0)`, sRGB 20,20,20, a clear step lighter than the content canvas) - and `bg-white` in light, so the header reads as a separate panel. In dark - mode, top-level layer cards and table header rows use the same `--color-panel`. +- **Topbar chrome** uses the canvas color in dark (`#0a0a0a`) and `bg-white` + in light. It has no bottom border until the page scrolls under it. Scrolling + adds `is-scrolled` on `` (see `layouts/app.blade.php`) and the dark bar + becomes `rgb(10 10 10 / 0.72)` with blur and a neutral hairline. In dark + mode, top-level card wells and table rows use `--color-card` (`#121212`), + not the top bar. - **Desktop sidebar and the topbar brand block** (Coolify name and version) - use the content canvas color (`bg-neutral-50` / `dark:bg-app`). The sidebar - has a 1px right border, and the brand block has a 1px bottom border. These - chrome borders stay faint (`border-black/[0.06]`, `dark:border-white/[0.04]`); - do not use `dark:border-white/[0.06]`, which a global rule forces to the - solid hairline grey. The mobile drawer keeps `bg-panel` / `bg-white`. + use the canvas color in light mode. In the default dark theme they use + `--color-sidebar` (`#120e1a`): a vertical brand wash on the nav, a flat 9% + wash on the logo cell, and a lavender right edge. Inactive icon tiles, + Search, the account button, and section labels (10.5px uppercase mono) are + lavender. The Admin icon stays neutral. These purple rules use + `html.dark:not([data-theme="custom"])`; a custom theme keeps neutral tiles, + Search, and account button. Chevrons are hidden until the row is hovered, + focused, or current. Do not use `dark:border-white/[0.06]` on these chrome + edges; a global rule forces that utility to the solid hairline grey. Dark surface tokens are exact oklch equivalents of chosen sRGB steps. oklch lightness compresses toward pure black below ~15%, so check the rendered sRGB @@ -571,23 +698,31 @@ value. The dark ladder is `--color-app` 10, `--color-panel` 20, `--coollabs-elevated` 22, `--coollabs-base` 28, `--coollabs-recessed` 34 (sRGB, r=g=b), which reads as distinct surfaces. -Temperature: in both modes every panel is **pure neutral gray** (r=g=b). -In dark mode, every surface token (`--color-app`, `--color-panel`, -`--coollabs-*`, `--color-surface`, `--color-raised`, `--color-selected`, -`--color-log`) has chroma 0, so menus, listbox panels, modals, logs, and the -command palette match the canvas. Do not give one surface a cast (blue, warm, -or purple). The light page canvas uses `bg-neutral-50` (not +Temperature: the page canvas, cards, menus, listboxes, modals, and logs stay +**pure neutral gray** (r=g=b). Do not tint those. The exceptions are the +default dark sidebar (`--color-sidebar`, `#120e1a`, with a brand-purple wash) +and the dark command palette (a faint brand-purple wash). A custom theme derives all +of these surfaces from its background and sidebar colors (see the token rules in P2). The light page canvas uses `bg-neutral-50` (not `bg-gray-50`, which is faintly cool) so it matches the neutral cards and chrome. -The only intentional color is the purple/yellow brand accent. +The only intentional color is the purple brand accent. Yellow is for warnings. - **Layer cards** use the outer box and inset well from section 5. +- **Top-level pages** (Projects, Servers, and the other section roots) do not + repeat the page title in the breadcrumb when that title is the visible `h1`. + Nested pages keep the full trail (project, environment, resource). +- **Pagination** is hidden when there is only one page (both arrows disabled). +- **Project cards** spell out their counts: "1 environment ยท 35 resources". +- **The view switcher** (`x-canvas-view-toggle`, + `x-shared-variables.view-controls`) shows only the other view, as an icon + button: the grid icon in table view, the list icon in grid view. The labelled + `Canvas` link and the `List view` link on canvas pages keep their text. - **Tables and collection tiles** outside a layer card use the `rounded-xl border โ€ฆ dark:bg-white/[0.05]` shell plus the crisp `--coollabs-hairline` ring; in light they are `bg-white` with the ring and the restrained card lift. In dark mode these shells are painted like layer cards. A tile, a separate pagination bar, or an empty box is a card without a - well, so it is `--color-panel`. A table (a shell whose first child is a - `border-b` header row) has a `--color-panel` header row and rows in the well - color (`color-mix(in srgb, var(--color-panel) 66%, black)`). + well, so it is `--color-card` (`#121212`). A table (a shell whose first child is a + `border-b` header row) has a header row in the card outer color + (`color-mix(in srgb, var(--color-card) 66%, black)`) and `--color-card` rows. These rules match the Tailwind classes in `resources/css/app.css`, because the table shells are copied markup without a shared class. @@ -606,15 +741,44 @@ color. Row-hover states keep the lighter `dark:hover:bg-white/[0.025]`. - With the default tile items (not Appearance โ†’ Navigation items โ†’ Flat), each icon sits in a 30px 3D tile (16px icon, 6px padding, 1px border) in a 36px row. - With tile items, the row has no fill: the current item shows through its - accent tile (purple light, yellow dark) and its brighter label. + purple accent tile and its brighter label. The dark active tile has a 2px + depth edge and no glow, because the row (`overflow: hidden`) clips a glow + into a square. +- In the collapsed desktop sidebar, Search is a tile like the other items. The + row fill is removed in `@layer utilities` (the button has a layered + `!important` fill utility that an unlayered rule cannot beat), and the dark + tile uses the same colors as the other tiles. - With Flat items, active rows are rounded pills (`rounded-md`) with a solid neutral selected fill (`bg-black/5` light, `bg-white/6` dark) and no accent rail. Hover rows use the same radius. Do not use accent-tinted gradients on nav rows; yellow washes look muddy on dark UI. - Nested items sit behind a thin 1px guide line (`.nav-children`) and use the same selected pill, not a thick box border. -- The update badge sits on the version row and uses a tiny fully rounded - primary-action pill. +- The logo cell (`.app-topbar-brand` in `layouts/app.blade.php`) is `w-56` + expanded and `w-16` collapsed. Expanded, it shows the `Coolify` wordmark + (`text-[15px] font-semibold tracking-tight`), `` + (`text-[10.5px]`, `text-fg-faint` in dark), and the update badge + (`livewire:upgrade`, instance admins on self-hosted only) at `ml-auto`. + Collapsed, it shows only `/coolify-logo.svg` at `size-5` and adds + `border-b`. +- The update badge sits on the version row: an 18px fully rounded pill with + 9.5px semibold text. It is `bg-coollabs/10 text-coollabs ring-coollabs/25` in + light mode and the warning pill (`dark:bg-warning/15 dark:text-warning + dark:ring-warning/25`) in dark mode, because an update is a real warning. + While an upgrade runs it says "Updating". Both open the upgrade modal. +- The mobile drawer header (`data-mobile-sidebar-brand`) repeats the wordmark + and version in an `h-12` row with a bottom border and a close button. It + does not show the update badge. +- The mobile top bar (`MOBILE TOP BAR` in `layouts/app.blade.php`, `lg:hidden`) + is `sticky top-0 z-40`, `px-4 py-3 sm:px-6`, `bg-white/95 dark:bg-panel/95 + backdrop-blur-sm` with `border-neutral-200/60 dark:border-white/[0.06]`. + Left: `/coolify-logo.svg` (18px) in a `size-8 rounded-lg bg-neutral-100 + dark:bg-white/[0.06]` tile, then `livewire:switch-team`. Right (`gap-1`): + `#configuration-warning-hud-slot-mobile`, `deployments-indicator + variant="mobile"`, the update badge (`livewire:upgrade key="mobile-upgrade"`, + instance admins on self-hosted only), `x-top-user-menu`, and a `size-9` + button that toggles the drawer. Keep these in the same order as the desktop + top bar. - Submenu chevrons (the right chevron on rows that open a submenu and the back chevron) are `text-coollabs` in light mode and grey in dark mode (`dark:text-fg-faint` right, `dark:text-fg-dim` back). @@ -633,6 +797,13 @@ in-flow resource heading, or legacy `.navbar-main` tabs to one resource type. Keep route-derived active state in Blade/Livewire. Do not rely only on Alpine state because it can disappear after polling or a Livewire morph. +From `xl`, the resource menu replaces the main menu in the main sidebar +(`x-main-nav-menu`, `registerMainNavContext()`, `components/navbar.blade.php`). +Its back row (`โ€น Dashboard`, `โ€น Servers`, `โ€น Service`) shows the menu one depth +lower, without a page load. The main menu has no row that opens the resource +menu again; to get back to it, open the resource again. Dashboard is always the +first item under Search. + The global topbar owns the current resource identity, its compact status badges, configuration warnings (`#configuration-warning-hud-slot`), and the resource actions (`#resource-action-hud-slot`). If a resource is missing from @@ -808,12 +979,14 @@ Anatomy: - soft outer box: 12px radius, 1px border, 6px padding, faint fill (`rgb(0 0 0 / 0.75%)` light). In dark mode a top-level card is - `--color-panel`, the same as the topbar; a nested card keeps + `color-mix(in srgb, var(--color-card) 66%, black)` and its well is + `--color-card` (`#121212`); a nested card keeps `rgb(255 255 255 / 6%)` over its parent; - small title row on the outer box (13px semibold title, 11px description and header links), no divider below it; -- body is an inset well: 8px radius, 1px border, white in light and - `rgb(0 0 0 / 34%)` in dark; +- body is an inset well: 8px radius, 1px border, white in light, + `--color-card` in a dark top-level card, and `rgb(0 0 0 / 34%)` in a dark + nested card; - 16px body padding; - optional `flush` mode for full-bleed tables; - card-level actions belong in the header slot; @@ -831,9 +1004,9 @@ Some pages put a well (`.application-settings-section-body`) directly on the page without a card, for example the domains and environment variable tables. The well colors are defined on the well itself as well as on the card, so a standalone well keeps its background and border. In dark mode a standalone well -uses the card outer-box surface (`--card-bg`, `--card-border`) instead of the -black well fill, because the black fill directly on the canvas is darker than -the page. An `x-empty` inside a +matches a top-level card: its content is `--color-card` (`#121212`), and a +table header row in it is the darker card outer color +(`color-mix(in srgb, var(--color-card) 66%, black)`). An `x-empty` inside a standalone well shows only its centered text, as it does inside a card. Inside the dark well, controls use a darker translucent fill and border @@ -1140,7 +1313,11 @@ top-anchored overlay: - create rows use a neutral plus tile that only picks up the accent when the row is focused; - type pills and quickcommand chips stay recessed; they tint with the accent - only on the focused row; + only on the focused row. In dark mode the text of the focused row's chips, + its icon tile glyph, and its chevron are white; +- in dark mode the shell, header, and results body carry the brand-purple + tint from Part 1 (rules after the dark `.view-toggle` rules in + `resources/css/app.css`). Light mode stays neutral; - neutral thin scrollbar inside the results body (not brand-colored); - create-resource modals opened from the palette reuse the standard `application-settings-section` layer-card shell. @@ -1157,7 +1334,10 @@ server โ†’ destination โ†’ project โ†’ environment create flow. Current toast behavior: - compact layered card, maximum width 26rem; -- Reicon status tile for success, info, warning, danger, or default; +- Reicon status tile for success, info, warning, danger, or default. The tile + (`.toast-icon`) is 3D: a border and a 2px bottom edge in the status color; +- a faint edge (`.toast-card`, dark `rgb(255 255 255 / 0.05)`) that blends into + the toast background; - title plus optional description; - dismiss and copy-details actions; - normally up to four stacked notifications, without evicting persistent notices; diff --git a/README.md b/README.md index 66114586e8..de6b80ab95 100644 --- a/README.md +++ b/README.md @@ -71,7 +71,6 @@ Thank you so much! ### Huge Sponsors -* [CubePath](https://cubepath.com/coolify) - Premium dedicated servers and cloud VPS hosting * [Context.dev](https://www.context.dev/) - Web scraping API for AI agents * [Ginernet](https://ginernet.com/) - Hosting powerful servers in Spain * [SerpAPI](https://serpapi.com) - Google Search API โ€” Scrape Google and other search engines from our fast, easy, and complete API. @@ -116,7 +115,6 @@ Thank you so much! * [VPSDime](https://vpsdime.com/) - Cheap VPS Hosting - 4GB for $5/month * [dataforest Cloud](https://cloud.dataforest.net/en) - Deploy cloud servers as seeds independently in seconds. Enterprise hardware, premium network, 100% made in Germany. * [ISHosting](https://ishosting.com/) - Hosting and VPS solutions -* [QuickSrv](https://quicksrv.io/) - Fast and reliable server hosting * [NodePlus.ai](https://nodeplus.ai/) - AI that reads every system your business runs on ### Small Sponsors @@ -159,7 +157,6 @@ Thank you so much! Web3 Jobs LinkDr Arvensis Systems -Reshot RunPod Gravity Wiz UXWizz diff --git a/app/Actions/Application/GetRollbackImages.php b/app/Actions/Application/GetRollbackImages.php index 73ab3931fa..4378fe54cf 100644 --- a/app/Actions/Application/GetRollbackImages.php +++ b/app/Actions/Application/GetRollbackImages.php @@ -4,6 +4,7 @@ namespace App\Actions\Application; use App\Enums\ApplicationDeploymentStatus; use App\Models\Application; +use App\Models\Server; use Lorisleiva\Actions\Concerns\AsAction; /** @@ -27,10 +28,7 @@ class GetRollbackImages $server = $application->destination?->server; if ($server?->isFunctional()) { - $output = instant_remote_process([ - "docker inspect --format='{{.Config.Image}}' {$application->uuid}", - ], $server, throwError: false); - $current = self::imageTag(str($output)->trim()->toString()); + $current = self::currentTag($server, $application); $repository = $application->docker_registry_image_name ?: $application->uuid; $output = instant_remote_process([ @@ -61,6 +59,21 @@ class GetRollbackImages ]; } + /** + * The tag of the running production containers. Container names can have a timestamp or a + * custom name, so find them by their owner labels. Null when no container runs, or when the + * running containers use different tags (for example during a rolling update). + */ + private static function currentTag(Server $server, Application $application): ?string + { + $tags = getCurrentApplicationContainerStatus($server, $application, 0) + ->filter(fn ($container) => data_get($container, 'State') === 'running') + ->map(fn ($container) => self::imageTag((string) data_get($container, 'Image'))) + ->unique(); + + return $tags->count() === 1 ? $tags->first() : null; + } + /** * Each successful deployment pushes `:` to the registry, so the commits of * recent deployments are rollback targets. Docker image applications do not push, and diff --git a/app/Actions/Application/StopApplication.php b/app/Actions/Application/StopApplication.php index 9ac26b243c..8f8b4bd4f2 100644 --- a/app/Actions/Application/StopApplication.php +++ b/app/Actions/Application/StopApplication.php @@ -3,8 +3,10 @@ namespace App\Actions\Application; use App\Actions\Server\CleanupDocker; +use App\Events\ApplicationStopFailed; use App\Events\ServiceStatusChanged; use App\Models\Application; +use Illuminate\Support\Facades\Log; use Lorisleiva\Actions\Concerns\AsAction; class StopApplication @@ -50,7 +52,11 @@ class StopApplication $commands[] = "docker rm -f {$escapedContainerName}"; } - instant_remote_process(command: $commands, server: $server, throwError: false); + try { + instant_remote_process(command: $commands, server: $server); + } catch (\Exception $e) { + $errors[] = $e->getMessage(); + } } if ($removeContainers && $application->build_pack === 'dockercompose') { @@ -64,8 +70,13 @@ class StopApplication $errors[] = $e->getMessage(); } } + $teamId = $application->environment->project->team->id; if ($errors !== []) { - return implode(' ', $errors); + $message = implode(' ', $errors); + Log::warning('Failed to stop application', ['application_uuid' => $application->uuid, 'error' => $message]); + ApplicationStopFailed::dispatch($teamId, $application->uuid, $message); + + return $message; } $status = [ @@ -82,7 +93,7 @@ class StopApplication } $application->update($status); - ServiceStatusChanged::dispatch($application->environment->project->team->id); + ServiceStatusChanged::dispatch($teamId); return null; } diff --git a/app/Actions/Database/StartDatabaseImport.php b/app/Actions/Database/StartDatabaseImport.php index 5689882c2b..daa74c98bd 100644 --- a/app/Actions/Database/StartDatabaseImport.php +++ b/app/Actions/Database/StartDatabaseImport.php @@ -301,6 +301,7 @@ class StartDatabaseImport .' -e S3_ENDPOINT='.escapeshellarg((string) $storage->endpoint) .' -e S3_ACCESS_KEY='.escapeshellarg((string) $storage->key) .' -e S3_SECRET_KEY='.escapeshellarg((string) $storage->secret) + .' '.$storage->minioClientRegionOption() .' '.$image.' sleep 86400', ], $server); } catch (Throwable) { diff --git a/app/Actions/Development/InstallDevelopmentGpuDriver.php b/app/Actions/Development/InstallDevelopmentGpuDriver.php new file mode 100644 index 0000000000..ffe62f497c --- /dev/null +++ b/app/Actions/Development/InstallDevelopmentGpuDriver.php @@ -0,0 +1,47 @@ +/dev/null 2>&1; then', + 'apt-get update -qq', + 'apt-get install -y -qq ubuntu-drivers-common', + 'ubuntu-drivers install --gpgpu', + // For example 580-server-open. The utilities package has no -open suffix. + 'flavor=$(dpkg-query -W -f=\'${Package}\n\' \'nvidia-headless-no-dkms-*\' | sed -n \'s/^nvidia-headless-no-dkms-//p\' | head -n1)', + // ubuntu-drivers installs the modules for the newest kernel, which can differ from the running one. + 'apt-get install -y -qq "nvidia-utils-${flavor%-open}" "linux-modules-nvidia-${flavor}-$(uname -r)"', + 'modprobe nvidia', + 'nvidia-smi', + 'fi', + 'if ! command -v nvidia-ctk >/dev/null 2>&1; then', + 'curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | gpg --batch --yes --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg', + 'curl -fsSL https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | sed \'s#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g\' > /etc/apt/sources.list.d/nvidia-container-toolkit.list', + 'apt-get update -qq', + 'apt-get install -y -qq nvidia-container-toolkit', + 'nvidia-ctk runtime configure --runtime=docker', + 'systemctl restart docker', + 'fi', + ], $server, timeout: 1800, disableMultiplexing: true, maxAttempts: 1); + + $server->gatherServerMetadata(); + } +} diff --git a/app/Actions/Development/ManageDevelopmentQemuVm.php b/app/Actions/Development/ManageDevelopmentQemuVm.php index 9b1a1c791f..9fab0050a4 100644 --- a/app/Actions/Development/ManageDevelopmentQemuVm.php +++ b/app/Actions/Development/ManageDevelopmentQemuVm.php @@ -2,6 +2,7 @@ namespace App\Actions\Development; +use Illuminate\Contracts\Process\ProcessResult; use Illuminate\Database\QueryException; use Illuminate\Support\Facades\Process; use Lorisleiva\Actions\Concerns\AsAction; @@ -12,28 +13,44 @@ class ManageDevelopmentQemuVm /** * Start (or create) the selected VMs and seed their servers. Existing VMs are reused unless $fresh is set. + * With $gpu, the host GPU is passed through and the container seeds the server, because it installs the + * driver over Coolify's SSH connection. * * @param string|array $profileNames */ - public function handle(string|array $profileNames, bool $asLocalhost = false, bool $fresh = false): void + public function handle(string|array $profileNames, bool $asLocalhost = false, bool $fresh = false, bool $gpu = false): void { $profileNames = is_array($profileNames) ? array_values(array_unique($profileNames)) : [$profileNames]; foreach ($profileNames as $index => $profileName) { - StartDevelopmentQemuVm::run($profileName, $fresh); + StartDevelopmentQemuVm::run($profileName, $fresh, $gpu); + + if ($gpu) { + $this->seedInContainer($profileName, $index > 0, $asLocalhost, true)->throw(); + + continue; + } try { SeedDevelopmentQemuServer::run($profileName, $index === 0, $asLocalhost); } catch (QueryException $exception) { - $keepOthers = $index === 0 ? '' : ' --keep-others'; - $localhostOption = $asLocalhost ? ' --as-localhost' : ''; - $container = escapeshellarg(config('development-qemu.coolify_container')); - $result = Process::run("docker exec {$container} php artisan dev:qemu:seed ".escapeshellarg($profileName).$keepOthers.$localhostOption); - - if ($result->failed()) { + if ($this->seedInContainer($profileName, $index > 0, $asLocalhost, false)->failed()) { throw $exception; } } } } + + private function seedInContainer(string $profileName, bool $keepOthers, bool $asLocalhost, bool $gpu): ProcessResult + { + $container = escapeshellarg(config('development-qemu.coolify_container')); + $command = "docker exec {$container} php artisan dev:qemu:seed ".escapeshellarg($profileName) + .($keepOthers ? ' --keep-others' : '') + .($asLocalhost ? ' --as-localhost' : '') + .($gpu ? ' --gpu' : ''); + + return $gpu + ? Process::forever()->run($command, fn (string $type, string $output) => fwrite(STDOUT, $output)) + : Process::run($command); + } } diff --git a/app/Actions/Development/SeedDevelopmentQemuServer.php b/app/Actions/Development/SeedDevelopmentQemuServer.php index ad385f878e..288e1ab8c2 100644 --- a/app/Actions/Development/SeedDevelopmentQemuServer.php +++ b/app/Actions/Development/SeedDevelopmentQemuServer.php @@ -2,6 +2,8 @@ namespace App\Actions\Development; +use App\Enums\ProxyStatus; +use App\Enums\ProxyTypes; use App\Models\PrivateKey; use App\Models\Server; use InvalidArgumentException; @@ -50,6 +52,10 @@ class SeedDevelopmentQemuServer 'team_id' => 0, 'private_key_id' => $privateKey->id, ]); + if (! $server->proxyType()) { + $server->proxy->set('type', ProxyTypes::CADDY->value); + $server->proxy->set('status', ProxyStatus::EXITED->value); + } $server->deleted_at = null; $server->save(); diff --git a/app/Actions/Development/StartDevelopmentQemuVm.php b/app/Actions/Development/StartDevelopmentQemuVm.php index 5815185508..8e52a6dcc4 100644 --- a/app/Actions/Development/StartDevelopmentQemuVm.php +++ b/app/Actions/Development/StartDevelopmentQemuVm.php @@ -15,8 +15,9 @@ class StartDevelopmentQemuVm /** * Start the VM of a profile, creating it only when its libvirt domain does not exist yet. * With $fresh, only this profile's VM is destroyed, undefined, and deleted before it is created again. + * With $gpu, the NVIDIA GPU of the host is passed through to the VM. An existing VM is stopped to add it. */ - public function handle(string $profileName, bool $fresh = false): void + public function handle(string $profileName, bool $fresh = false, bool $gpu = false): void { $this->ensureDevelopmentEnvironment(); $profile = config("development-qemu.profiles.{$profileName}"); @@ -33,9 +34,12 @@ class StartDevelopmentQemuVm } if (! $fresh && $this->domainExists($profile['domain'])) { + if ($gpu) { + $this->attachGpu($profile['domain']); + } $this->startExistingVm($profile['domain']); } else { - $this->createPreparedVm($profile); + $this->createPreparedVm($profile, $gpu); } ConfigureDevelopmentQemuHost::run(); @@ -43,13 +47,13 @@ class StartDevelopmentQemuVm } /** @param array{domain: string, template: string, ip: string, user: string, mac: string, image: string, image_url: string, os_variant: string, provisioner: string} $profile */ - private function createPreparedVm(array $profile): void + private function createPreparedVm(array $profile, bool $gpu): void { $preparedImage = $this->preparedImage($profile['template']); if (! File::exists($preparedImage)) { $this->createVm($profile, false); - $this->waitForPreparation($profile['domain']); + $this->waitForShutOff($profile['domain'], 900); $this->runOrFail('virsh undefine '.escapeshellarg($profile['domain'])); $this->runOrFail('mv '.escapeshellarg($this->vmDisk($profile['domain'])).' '.escapeshellarg($preparedImage)); if (File::exists($preparedImage)) { @@ -57,7 +61,7 @@ class StartDevelopmentQemuVm } } - $this->createVm($profile, true); + $this->createVm($profile, true, $gpu); } private function domainExists(string $domain): bool @@ -79,20 +83,84 @@ class StartDevelopmentQemuVm return; } - $this->applyMemory($domain); + $hasGpu = $this->hasGpu($domain); + $this->applyMemory($domain, $hasGpu); + + if ($hasGpu) { + $this->releaseHostGpu(); + } + $this->runOrFail('virsh start '.escapeshellarg($domain)); } /** * Sets the configured memory on a stopped VM, so VMs created with a larger size shrink on their next start. */ - private function applyMemory(string $domain): void + private function applyMemory(string $domain, bool $hasGpu): void { - $memory = (int) config('development-qemu.memory').'M'; + $memory = (int) config($hasGpu ? 'development-qemu.gpu_memory' : 'development-qemu.memory').'M'; $this->runOrFail(sprintf('virsh setmaxmem %s %s --config', escapeshellarg($domain), $memory)); $this->runOrFail(sprintf('virsh setmem %s %s --config', escapeshellarg($domain), $memory)); } + private function hasGpu(string $domain): bool + { + return str_contains(Process::run('virsh dumpxml --inactive '.escapeshellarg($domain))->output(), 'hasGpu($domain)) { + return; + } + + if (trim(Process::run('virsh domstate '.escapeshellarg($domain))->output()) !== 'shut off') { + $this->runOrFail('virsh shutdown '.escapeshellarg($domain)); + $this->waitForShutOff($domain, 120); + } + + foreach ($this->hostGpuDevices() as $device) { + $this->runOrFail(sprintf('virt-xml %s --add-device --hostdev %s', escapeshellarg($domain), escapeshellarg($device))); + } + } + + /** + * Returns the libvirt node device names of every PCI function of the first NVIDIA display controller, + * for example pci_0000_01_00_0 (GPU) and pci_0000_01_00_1 (its audio device). All functions must move together. + * + * @return array + */ + private function hostGpuDevices(): array + { + $devices = collect(explode("\n", trim(Process::run('lspci -Dnmm -d 10de:')->output()))) + ->filter() + ->map(fn (string $line): array => str_getcsv($line, ' ')); + $gpu = $devices->first(fn (array $device): bool => in_array($device[1] ?? null, ['0300', '0302'], true)); + + if ($gpu === null) { + throw new RuntimeException('No NVIDIA GPU found on this host.'); + } + + $slot = substr($gpu[0], 0, strrpos($gpu[0], '.')); + + return $devices + ->filter(fn (array $device): bool => str_starts_with($device[0], "{$slot}.")) + ->map(fn (array $device): string => 'pci_'.str_replace([':', '.'], '_', $device[0])) + ->values() + ->all(); + } + + /** + * nvidia-persistenced keeps the GPU open, which blocks libvirt from moving it to vfio-pci. + */ + private function releaseHostGpu(): void + { + Process::run('systemctl stop nvidia-persistenced'); + } + private function destroyVm(string $domain): void { Process::run('virsh destroy '.escapeshellarg($domain)); @@ -101,7 +169,7 @@ class StartDevelopmentQemuVm } /** @param array{domain: string, template: string, ip: string, user: string, mac: string, image: string, image_url: string, os_variant: string, provisioner: string} $profile */ - private function createVm(array $profile, bool $prepared): void + private function createVm(array $profile, bool $prepared, bool $gpu = false): void { $directory = config('development-qemu.storage_path'); File::ensureDirectoryExists($directory); @@ -155,11 +223,17 @@ class StartDevelopmentQemuVm $seedDisk = $prepared ? '' : ' --disk path='.escapeshellarg($seedImage).',format=raw,bus=virtio,readonly=on'; // Instance networks are isolated, so VMs of different instances may share the profile MAC. $macCheck = config('development-qemu.instance') === null ? '' : ' --check mac_in_use=off'; + $hostDevices = ''; + + if ($gpu) { + $hostDevices = collect($this->hostGpuDevices())->map(fn (string $device): string => ' --hostdev '.escapeshellarg($device))->implode(''); + $this->releaseHostGpu(); + } $this->runOrFail(sprintf( - 'virt-install --connect qemu:///system --name %s --memory %d --vcpus %d --import --os-variant %s --disk path=%s,format=qcow2,bus=virtio%s --network network=%s,model=virtio,mac=%s --noautoconsole'.$macCheck, + 'virt-install --connect qemu:///system --name %s --memory %d --vcpus %d --import --os-variant %s --disk path=%s,format=qcow2,bus=virtio%s --network network=%s,model=virtio,mac=%s --noautoconsole'.$macCheck.$hostDevices, escapeshellarg($profile['domain']), - config('development-qemu.memory'), + config($gpu ? 'development-qemu.gpu_memory' : 'development-qemu.memory'), config('development-qemu.vcpus'), escapeshellarg($profile['os_variant']), escapeshellarg($disk), @@ -182,10 +256,10 @@ class StartDevelopmentQemuVm return config('development-qemu.storage_path')."/{$template}-prepared.qcow2"; } - private function waitForPreparation(string $domain): void + private function waitForShutOff(string $domain, int $timeoutSeconds): void { $check = 'while [ "$(virsh domstate '.escapeshellarg($domain).')" != "shut off" ]; do sleep 2; done'; - $this->runOrFail('timeout 900 bash -c '.escapeshellarg($check)); + $this->runOrFail("timeout {$timeoutSeconds} bash -c ".escapeshellarg($check)); } private function moveLegacyFiles(string $directory): void diff --git a/app/Actions/Fortify/CreateNewUser.php b/app/Actions/Fortify/CreateNewUser.php index f24b9699f2..730b0b0873 100644 --- a/app/Actions/Fortify/CreateNewUser.php +++ b/app/Actions/Fortify/CreateNewUser.php @@ -5,11 +5,13 @@ namespace App\Actions\Fortify; use App\Jobs\SendVerificationEmailJob; use App\Models\Team; use App\Models\User; +use App\Support\RootSetupWindow; use Illuminate\Support\Facades\Hash; use Illuminate\Support\Facades\RateLimiter; use Illuminate\Support\Facades\Validator; use Illuminate\Validation\Rule; use Illuminate\Validation\Rules\Password; +use Illuminate\Validation\ValidationException; use Laravel\Fortify\Contracts\CreatesNewUsers; class CreateNewUser implements CreatesNewUsers @@ -34,6 +36,12 @@ class CreateNewUser implements CreatesNewUsers abort(403); } + if (! User::query()->exists() && ! RootSetupWindow::isOpen()) { + throw ValidationException::withMessages([ + 'email' => 'The setup window for the root account is closed. Restart Coolify on the server (docker restart coolify) to open it again.', + ]); + } + $this->ensureRegistrationIsNotRateLimited($input); Validator::make($input, [ @@ -93,7 +101,7 @@ class CreateNewUser implements CreatesNewUsers { $keys = [ [ - 'key' => 'registration:ip:'.sha1(auth_rate_limit_ip(request())), + 'key' => 'registration:ip:'.sha1(client_ip(request())), 'max' => self::REGISTRATION_IP_MAX_ATTEMPTS, 'decay' => self::REGISTRATION_IP_DECAY_SECONDS, ], diff --git a/app/Actions/Proxy/StartProxy.php b/app/Actions/Proxy/StartProxy.php index 98c0e0bc64..5eaf8d7655 100644 --- a/app/Actions/Proxy/StartProxy.php +++ b/app/Actions/Proxy/StartProxy.php @@ -7,6 +7,7 @@ use App\Events\ProxyStatusChanged; use App\Events\ProxyStatusChangedUI; use App\Models\Server; use App\Services\ProxyPortParser; +use Illuminate\Support\Collection; use Lorisleiva\Actions\Concerns\AsAction; use Lorisleiva\Actions\Decorators\JobDecorator; use Spatie\Activitylog\Models\Activity; @@ -40,12 +41,35 @@ class StartProxy ProxyStatusChangedUI::dispatch($server->team_id); } + SaveProxyConfiguration::run($server, $configuration); + $server->markProxyConfigurationApplied($configuration); + $commands = self::startCommands($server); + + if ($async) { + $server->markProxyTransitionStarted(); + + return remote_process($commands, $server, callEventOnFinish: 'ProxyStatusChanged', callEventData: $server->id, queue: deployment_queue()); + } else { + instant_remote_process($commands, $server); + + $server->proxy->set('type', $proxyType); + $server->save(); + ProxyStatusChanged::dispatch($server->id); + + return 'OK'; + } + } + + /** + * Commands that start the proxy. They first remove an existing proxy container. + * RestartProxyJob uses the same commands after its stop phase. + */ + public static function startCommands(Server $server): Collection + { $commands = collect([]); $proxy_path = $server->proxyPath(); // Absolute paths: a non-root SSH user may not be able to enter the proxy directory (#4255). $compose_file = rtrim($proxy_path, '/').'/docker-compose.yml'; - SaveProxyConfiguration::run($server, $configuration); - $server->markProxyConfigurationApplied($configuration); if ($server->isSwarmManager()) { $commands = $commands->merge([ @@ -79,7 +103,7 @@ class StartProxy " echo 'Successfully stopped and removed existing coolify-proxy.'", 'fi', ]); - if ($proxyType !== ProxyTypes::TRAEFIK->value) { + if ($server->proxyType() !== ProxyTypes::TRAEFIK->value) { // The sidecar belongs to the Traefik compose project, so --remove-orphans of another proxy keeps it. $commands->push('docker rm -f '.TRAEFIK_LOGROTATE_CONTAINER.' 2>/dev/null || true'); } @@ -93,16 +117,6 @@ class StartProxy $commands = $commands->merge(connectProxyToNetworks($server)); } - if ($async) { - return remote_process($commands, $server, callEventOnFinish: 'ProxyStatusChanged', callEventData: $server->id, queue: deployment_queue()); - } else { - instant_remote_process($commands, $server); - - $server->proxy->set('type', $proxyType); - $server->save(); - ProxyStatusChanged::dispatch($server->id); - - return 'OK'; - } + return $commands; } } diff --git a/app/Actions/Server/CheckUpdates.php b/app/Actions/Server/CheckUpdates.php index d0586f2d16..08775f9e6d 100644 --- a/app/Actions/Server/CheckUpdates.php +++ b/app/Actions/Server/CheckUpdates.php @@ -24,6 +24,28 @@ class CheckUpdates */ public const DNF_UPGRADES_COMMAND = 'LANG=C dnf repoquery -q --refresh --upgrades --latest-limit 1 --qf "%{name} %{arch} %{evr} %{repoid}\n"'; + /** + * Simulations of `apt-get upgrade` without and with the phased updates. apt holds back an update in a phased rollout + * until this machine is in the rollout, but `apt list --upgradable` still shows it. The difference of the two runs + * gives the phased updates. Text matching is not possible: apt 2.4 (Ubuntu 22.04) prints them as "kept back". + * + * @see https://documentation.ubuntu.com/server/explanation/software/about-apt-upgrade-and-phased-updates/ + */ + public const APT_SIMULATE_UPGRADE_COMMAND = 'LANG=C apt-get -s upgrade 2>/dev/null'; + + public const APT_SIMULATE_PHASED_UPGRADE_COMMAND = 'LANG=C apt-get -s -o APT::Get::Always-Include-Phased-Updates=true upgrade 2>/dev/null'; + + /** + * Packages held with `apt-mark hold`. `apt list --upgradable` shows their updates, but apt does not install them. + */ + public const APT_HELD_COMMAND = 'apt-mark showhold 2>/dev/null'; + + /** + * Simulation of `apk upgrade`. A version pin in /etc/apk/world (`name=1.2-r0`, `name<2`, `name~1.2`) blocks an + * update that `apk list --upgradable` still shows, and also the updates of packages that depend on that version. + */ + public const APK_SIMULATE_UPGRADE_COMMAND = 'LANG=C apk upgrade -s 2>/dev/null'; + public const RPM_INSTALLED_COMMAND = 'rpm -qa --qf "%{NAME} %{ARCH} %{VERSION}-%{RELEASE}\n"'; /** @@ -103,6 +125,11 @@ SH; 'system' => 'Restarts core system services', ]; + public const HELD_REASON_LABELS = [ + 'phased' => 'Phased rollout', + 'held' => 'Held', + ]; + public function handle(Server $server) { $osId = 'unknown'; @@ -179,7 +206,14 @@ SH; break; case 'apt': instant_remote_process(['apt-get update -qq'], $server); - $out = $this->parseAptOutput(instant_remote_process(['LANG=C apt list --upgradable 2>/dev/null'], $server)); + $out = $this->separateHeldUpdates( + $this->parseAptOutput(instant_remote_process(['LANG=C apt list --upgradable 2>/dev/null'], $server)), + $this->parseAptHeldPackages( + (string) instant_remote_process([self::APT_HELD_COMMAND], $server, false), + (string) instant_remote_process([self::APT_SIMULATE_UPGRADE_COMMAND], $server, false), + (string) instant_remote_process([self::APT_SIMULATE_PHASED_UPGRADE_COMMAND], $server, false), + ), + ); break; case 'pacman': $out = $this->parsePacmanOutput(instant_remote_process([self::PACMAN_CHECK_COMMAND], $server)); @@ -187,6 +221,7 @@ SH; case 'apk': instant_remote_process(['apk update -q'], $server); $out = $this->parseApkOutput(instant_remote_process(['LANG=C apk list --upgradable 2>/dev/null'], $server)); + $out = $this->separateHeldUpdates($out, $this->parseApkHeldPackages($out, (string) instant_remote_process([self::APK_SIMULATE_UPGRADE_COMMAND], $server, false))); break; default: return [ @@ -414,6 +449,62 @@ SH; ]; } + /** + * Moves the updates that the package manager does not install to `held_updates`, with the reason in `held_reason`. + * + * @param array $heldPackages Package name => reason. + */ + public function separateHeldUpdates(array $out, array $heldPackages): array + { + $updates = collect($out['updates'] ?? [])->partition(fn (array $update) => isset($heldPackages[$update['package']])); + $out['updates'] = $updates[1]->values()->all(); + $out['held_updates'] = $updates[0]->map(fn (array $update) => [...$update, 'held_reason' => $heldPackages[$update['package']]])->values()->all(); + $out['total_updates'] = count($out['updates']); + + return $out; + } + + /** + * @param string $held Output of APT_HELD_COMMAND. + * @param string $upgrade Output of APT_SIMULATE_UPGRADE_COMMAND. + * @param string $upgradeWithPhased Output of APT_SIMULATE_PHASED_UPGRADE_COMMAND. + * @return array + */ + public function parseAptHeldPackages(string $held, string $upgrade, string $upgradeWithPhased): array + { + $installs = function (string $output): ?array { + // Without the summary line, the simulation failed and tells nothing. + if (! preg_match('/^\d+ upgraded,/m', $output)) { + return null; + } + preg_match_all('/^Inst ([^\s:]+)/m', $output, $matches); + + return $matches[1]; + }; + $installed = $installs($upgrade); + $installedWithPhased = $installs($upgradeWithPhased); + $phased = $installed === null || $installedWithPhased === null ? [] : array_diff($installedWithPhased, $installed); + preg_match_all('/^([^\s:]+)/m', $held, $heldMatches); + + return [...array_fill_keys($phased, 'phased'), ...array_fill_keys($heldMatches[1], 'held')]; + } + + /** + * @param string $simulation Output of APK_SIMULATE_UPGRADE_COMMAND. + * @return array + */ + public function parseApkHeldPackages(array $out, string $simulation): array + { + // Without the summary line, the simulation failed and tells nothing. + if (! preg_match('/^OK: /m', $simulation)) { + return []; + } + preg_match_all('/^\(\s*\d+\/\d+\) Upgrading (\S+) \(/m', $simulation, $matches); + $held = array_diff(array_column($out['updates'] ?? [], 'package'), $matches[1]); + + return array_fill_keys($held, 'held'); + } + public function parsePacmanOutput(string $output): array { $updates = []; diff --git a/app/Actions/Server/CleanupDocker.php b/app/Actions/Server/CleanupDocker.php index f439e01af5..f132ebd17d 100644 --- a/app/Actions/Server/CleanupDocker.php +++ b/app/Actions/Server/CleanupDocker.php @@ -2,6 +2,8 @@ namespace App\Actions\Server; +use App\Enums\ApplicationDeploymentStatus; +use App\Models\ApplicationDeploymentQueue; use App\Models\Server; use App\Support\Actions\UniqueUntilProcessingJobDecorator; use Illuminate\Contracts\Queue\ShouldBeUnique; @@ -83,9 +85,24 @@ class CleanupDocker implements ShouldBeUnique return 'docker-cleanup-last-run-'.$server->uuid; } + /** + * The deployment that is queued or running on the server (as target or build server), if any. + * The cleanup prunes the build caches, which fails a build that uses them. A deployment that + * wrote no log for longer than the deployment timeout is stuck and does not count. + */ + public static function activeDeployment(Server $server): ?ApplicationDeploymentQueue + { + return ApplicationDeploymentQueue::query() + ->where(fn ($query) => $query->where('server_id', $server->id)->orWhere('build_server_id', $server->id)) + ->whereIn('status', [ApplicationDeploymentStatus::QUEUED->value, ApplicationDeploymentStatus::IN_PROGRESS->value]) + ->where('updated_at', '>=', now()->subSeconds($server->settings->dynamic_timeout ?? 3600)) + ->first(['id', 'deployment_uuid']); + } + /** * Cleanup after a resource stop: skipped when a cleanup ran recently, and the queued run - * only cleans when the disk usage is at or above the server cleanup threshold. + * only cleans when the disk usage is at or above the server cleanup threshold and no + * deployment is active on the server. */ public static function dispatchAfterStop(Server $server): void { @@ -125,6 +142,9 @@ class CleanupDocker implements ShouldBeUnique $this->deadline = now()->getTimestamp() + self::REMOTE_COMMANDS_DEADLINE; if ($onlyAboveThreshold) { + if (self::activeDeployment($server)) { + return []; + } $diskUsage = $server->getDiskUsage(); if (is_numeric($diskUsage) && (int) $diskUsage < $server->settings->docker_cleanup_threshold) { return []; @@ -304,14 +324,11 @@ class CleanupDocker implements ShouldBeUnique $disableRetention = $server->settings->disable_application_image_retention ?? false; - // Image of each container, keyed by container name + // Images that a container uses, running or stopped $containerImages = collect(explode("\n", $this->runRemoteCommand("docker ps -a --format '{{.Names}}#{{.Image}}' 2>/dev/null || true", $server) ?? '')) ->filter() - ->mapWithKeys(function (string $line) { - [$name, $image] = array_pad(explode('#', $line, 2), 2, ''); - - return [$name => $image]; - }); + ->map(fn (string $line) => array_pad(explode('#', $line, 2), 2, '')[1]) + ->unique(); $serverImages = collect(explode("\n", $this->runRemoteCommand("docker images --format '{{.Repository}}#{{.Tag}}#{{.CreatedAt}}' 2>/dev/null || true", $server) ?? '')) ->filter() @@ -327,49 +344,58 @@ class CleanupDocker implements ShouldBeUnique }) ->filter(fn ($image) => $image['tag'] !== '' && $image['tag'] !== ''); + // A deployment starts the containers of its new images only after the build or pull, + // so the images of an application with a queued or running deployment all stay. + $deployingApplicationIds = ApplicationDeploymentQueue::query() + ->whereIn('application_id', $applications->map(fn ($application) => (string) $application->id)) + ->whereIn('status', [ApplicationDeploymentStatus::QUEUED->value, ApplicationDeploymentStatus::IN_PROGRESS->value]) + ->where('updated_at', '>=', now()->subSeconds($server->settings->dynamic_timeout ?? 3600)) + ->pluck('application_id'); + $imagesToDelete = collect(); foreach ($applications as $application) { - $imagesToKeep = $disableRetention ? 0 : ($application->settings->docker_images_to_keep ?? 2); - $imageRepository = $application->docker_registry_image_name ?? $application->uuid; - - // Tag of the image of the running container (named after the application uuid) - $currentTag = preg_match('/:([^:]+)$/', $containerImages->get($application->uuid, ''), $matches) ? $matches[1] : ''; - - // Matches both uuid:tag and uuid_servicename:tag (Docker Compose with build), like - // the docker images --filter reference='*' filter - $images = $serverImages->filter(fn ($image) => str_starts_with($image['repository'], $imageRepository) - && ! str_contains(substr($image['repository'], strlen($imageRepository)), '/')); - - if ($images->isEmpty()) { + if ($deployingApplicationIds->contains((string) $application->id)) { continue; } - // Separate images into categories - // PR images (pr-*) are always deleted - // Build images (*-build) are cleaned up to match retained regular images - $prImages = $images->filter(fn ($image) => str_starts_with($image['tag'], 'pr-')); - $buildImages = $images->filter(fn ($image) => ! str_starts_with($image['tag'], 'pr-') && str_ends_with($image['tag'], '-build')); - $regularImages = $images->filter(fn ($image) => ! str_starts_with($image['tag'], 'pr-') && ! str_ends_with($image['tag'], '-build')); + $imagesToKeep = $disableRetention ? 0 : ($application->settings->docker_images_to_keep ?? 2); + $imageRepository = $application->docker_registry_image_name ?? $application->uuid; - // Filter out current running image from regular images and sort by creation date - $sortedRegularImages = $regularImages - ->filter(fn ($image) => $image['tag'] !== $currentTag) - ->sortByDesc('created_at') - ->values(); + // Matches both uuid:tag and uuid_servicename:tag (Docker Compose with build), like + // the docker images --filter reference='*' filter. Each repository (the + // application, or one Docker Compose service) has its own retention. + $repositories = $serverImages + ->filter(fn ($image) => str_starts_with($image['repository'], $imageRepository) + && ! str_contains(substr($image['repository'], strlen($imageRepository)), '/')) + ->groupBy('repository'); - // Clean up build images (-build suffix) that don't correspond to retained regular images - // Build images are intermediate artifacts (e.g. Nixpacks) not used by running containers. - // If a build is in progress, docker rmi will fail silently since the image is in use. - $keptTags = $sortedRegularImages->take($imagesToKeep)->pluck('tag'); - if ($currentTag !== '') { - $keptTags->push($currentTag); + foreach ($repositories as $images) { + // Tags that a container uses + $currentTags = $images->filter(fn ($image) => $containerImages->contains($image['image_ref']))->pluck('tag'); + + // Separate images into categories + // PR images (pr-*) are always deleted + // Build images (*-build) are cleaned up to match retained regular images + $prImages = $images->filter(fn ($image) => str_starts_with($image['tag'], 'pr-')); + $buildImages = $images->filter(fn ($image) => ! str_starts_with($image['tag'], 'pr-') && str_ends_with($image['tag'], '-build')); + $regularImages = $images->filter(fn ($image) => ! str_starts_with($image['tag'], 'pr-') && ! str_ends_with($image['tag'], '-build')); + + // Filter out the images in use from regular images and sort by creation date + $sortedRegularImages = $regularImages + ->reject(fn ($image) => $currentTags->contains($image['tag'])) + ->sortByDesc('created_at') + ->values(); + + // Clean up build images (-build suffix) that don't correspond to retained regular images + // Build images are intermediate artifacts (e.g. Nixpacks) not used by running containers. + $keptTags = $sortedRegularImages->take($imagesToKeep)->pluck('tag')->concat($currentTags); + + $imagesToDelete = $imagesToDelete + ->concat($prImages) + ->concat($sortedRegularImages->skip($imagesToKeep)) + ->concat($buildImages->reject(fn ($image) => $keptTags->contains(preg_replace('/-build$/', '', $image['tag'])))); } - - $imagesToDelete = $imagesToDelete - ->concat($prImages) - ->concat($sortedRegularImages->skip($imagesToKeep)) - ->concat($buildImages->reject(fn ($image) => $keptTags->contains(preg_replace('/-build$/', '', $image['tag'])))); } // Images in use fail silently; docker rmi continues with the next image diff --git a/app/Actions/Server/DeleteServer.php b/app/Actions/Server/DeleteServer.php index bcffee2a50..92e66e6dfc 100644 --- a/app/Actions/Server/DeleteServer.php +++ b/app/Actions/Server/DeleteServer.php @@ -19,6 +19,10 @@ class DeleteServer public function handle(int $serverId, bool $deleteFromHetzner = false, ?int $hetznerServerId = null, ?int $cloudProviderTokenId = null, ?int $teamId = null, bool $deleteFromVultr = false, ?string $vultrInstanceId = null, bool $deleteFromDigitalOcean = false, ?int $digitalOceanDropletId = null, bool $deleteFromHostinger = false, ?int $hostingerVirtualMachineId = null) { $server = Server::withTrashed()->find($serverId); + // Another Coolify instance manages the cloud server of a transferred server. + if ($server?->isTransferredAway()) { + $deleteFromHetzner = $deleteFromVultr = $deleteFromDigitalOcean = $deleteFromHostinger = false; + } // Delete from Hetzner even if server is already gone from Coolify if ($deleteFromHetzner && ($hetznerServerId || ($server && $server->hetzner_server_id))) { diff --git a/app/Actions/Server/RebootServer.php b/app/Actions/Server/RebootServer.php index 0ca6a25f6b..d2022012ee 100644 --- a/app/Actions/Server/RebootServer.php +++ b/app/Actions/Server/RebootServer.php @@ -20,9 +20,7 @@ SH; public function handle(Server $server): void { - if ($server->isTransferredAway()) { - throw new \RuntimeException('This server is managed by another Coolify instance.'); - } + $server->assertNotTransferredAway(); if (! $server->isFunctional()) { throw new \RuntimeException('Server is not reachable or not ready.'); } diff --git a/app/Actions/Server/ValidateServer.php b/app/Actions/Server/ValidateServer.php index 7c973130c4..f4855b9ffa 100644 --- a/app/Actions/Server/ValidateServer.php +++ b/app/Actions/Server/ValidateServer.php @@ -114,7 +114,7 @@ class ValidateServer if ($this->docker_version) { return 'OK'; } else { - $this->error = 'Docker Engine is not installed. Please install Docker manually before continuing: documentation.'; + $this->error = $server->dockerEngineVersionError(); $server->update([ 'validation_logs' => $this->error, ]); diff --git a/app/Actions/Service/UpdateServiceApplicationFromApi.php b/app/Actions/Service/UpdateServiceApplicationFromApi.php index 9e7657984e..dafbc80f30 100644 --- a/app/Actions/Service/UpdateServiceApplicationFromApi.php +++ b/app/Actions/Service/UpdateServiceApplicationFromApi.php @@ -67,6 +67,16 @@ class UpdateServiceApplicationFromApi $serviceApplication->setNoindexDomains($payload['noindex_domains'] ?? []); } + if (array_key_exists('redirect', $payload)) { + if (in_array($payload['redirect'], ['www', 'non-www'], true) && ! $this->hasWwwAndNonWwwDomains($serviceApplication->fqdn)) { + return response()->json([ + 'message' => 'Validation failed.', + 'errors' => ['redirect' => ['The www and non-www redirects need both a www and a non-www domain in url.']], + ], 422); + } + $serviceApplication->redirect = $payload['redirect']; + } + if (array_key_exists('human_name', $payload)) { $serviceApplication->human_name = $payload['human_name']; } @@ -130,4 +140,14 @@ class UpdateServiceApplicationFromApi return null; } + + private function hasWwwAndNonWwwDomains(?string $fqdn): bool + { + $hosts = str($fqdn ?? '')->explode(',') + ->map(fn (string $url) => strtolower((string) parse_url(trim($url), PHP_URL_HOST))) + ->filter(); + + return $hosts->contains(fn (string $host) => str_starts_with($host, 'www.')) + && $hosts->contains(fn (string $host) => ! str_starts_with($host, 'www.')); + } } diff --git a/app/Actions/Shared/MigrateResourceToDestination.php b/app/Actions/Shared/MigrateResourceToDestination.php index b0979e1677..3650b68624 100644 --- a/app/Actions/Shared/MigrateResourceToDestination.php +++ b/app/Actions/Shared/MigrateResourceToDestination.php @@ -86,6 +86,12 @@ class MigrateResourceToDestination ]); } + if ($this->hasDomains($resource) && $targetServer->isProxyShouldRun() && $targetServer->proxy->get('status') !== 'running') { + throw ValidationException::withMessages([ + 'destination_id' => 'The proxy of the target server is not running. Start it on the server page before you migrate a resource with domains.', + ]); + } + $crossServer = $sourceServer && (int) $sourceServer->id !== (int) $targetServer->id; if (! $crossServer) { @@ -169,6 +175,20 @@ class MigrateResourceToDestination $this->resaveFileStorages($resource->fresh()); } + protected function hasDomains( + Application|Service|StandalonePostgresql|StandaloneRedis|StandaloneMongodb|StandaloneMysql|StandaloneMariadb|StandaloneKeydb|StandaloneDragonfly|StandaloneClickhouse|StandaloneSqlite $resource, + ): bool { + if ($resource instanceof Application) { + return $resource->maintenanceDomains()->isNotEmpty(); + } + + if ($resource instanceof Service) { + return $resource->applications()->get()->contains(fn ($application) => filled($application->fqdn)); + } + + return false; + } + protected function stopResource( Application|Service|StandalonePostgresql|StandaloneRedis|StandaloneMongodb|StandaloneMysql|StandaloneMariadb|StandaloneKeydb|StandaloneDragonfly|StandaloneClickhouse|StandaloneSqlite $resource, ): void { diff --git a/app/Console/Commands/Cloud/RestoreDatabase.php b/app/Console/Commands/Cloud/RestoreDatabase.php index 7c6c0d4c66..d2d5e9b246 100644 --- a/app/Console/Commands/Cloud/RestoreDatabase.php +++ b/app/Console/Commands/Cloud/RestoreDatabase.php @@ -144,7 +144,7 @@ class RestoreDatabase extends Command // Use pg_restore for custom format dumps $command = sprintf( - 'PGPASSWORD=%s pg_restore -h %s -p %s -U %s -d %s -v %s', + 'PGPASSWORD=%s pg_restore --no-owner --no-acl -h %s -p %s -U %s -d %s -v %s', escapeshellarg($password), escapeshellarg($host), escapeshellarg($port), diff --git a/app/Console/Commands/Init.php b/app/Console/Commands/Init.php index e21a2f1c81..6cd914fd9c 100644 --- a/app/Console/Commands/Init.php +++ b/app/Console/Commands/Init.php @@ -16,6 +16,7 @@ use App\Models\Server; use App\Models\StandalonePostgresql; use App\Models\User; use App\Support\ResourceStartActivity; +use App\Support\RootSetupWindow; use Carbon\Carbon; use Illuminate\Console\Command; use Illuminate\Support\Facades\Artisan; @@ -36,6 +37,12 @@ class Init extends Command Artisan::call('optimize:clear'); Artisan::call('optimize'); + try { + RootSetupWindow::openIfNoUserExists(); + } catch (\Throwable $e) { + echo "Could not open the root setup window: {$e->getMessage()}\n"; + } + try { $this->pullTemplatesFromCDN(); } catch (\Throwable $e) { diff --git a/app/Console/Commands/ManageDevelopmentQemuVmCommand.php b/app/Console/Commands/ManageDevelopmentQemuVmCommand.php index 1dc235d0d0..bf27d1747e 100644 --- a/app/Console/Commands/ManageDevelopmentQemuVmCommand.php +++ b/app/Console/Commands/ManageDevelopmentQemuVmCommand.php @@ -9,7 +9,7 @@ use function Laravel\Prompts\multiselect; class ManageDevelopmentQemuVmCommand extends Command { - protected $signature = 'dev:qemu {profiles?* : Profile keys from config/development-qemu.php} {--as-localhost : Use the VM for the localhost server (id 0)} {--fresh : Destroy and recreate the selected VMs instead of reusing them}'; + protected $signature = 'dev:qemu {profiles?* : Profile keys from config/development-qemu.php} {--as-localhost : Use the VM for the localhost server (id 0)} {--fresh : Destroy and recreate the selected VMs instead of reusing them} {--gpu : Pass the NVIDIA GPU of the host through to the VM and install its driver}'; protected $description = 'Start (or create) selected development QEMU VMs and seed their Coolify servers'; @@ -34,7 +34,17 @@ class ManageDevelopmentQemuVmCommand extends Command return self::FAILURE; } - ManageDevelopmentQemuVm::run($profileNames, (bool) $this->option('as-localhost'), (bool) $this->option('fresh')); + if ($this->option('gpu')) { + $profile = $profiles[$profileNames[0]] ?? null; + + if (count($profileNames) !== 1 || $profile === null || $profile['user'] !== 'root' || ! str_starts_with($profile['os_variant'], 'ubuntu')) { + $this->error('GPU passthrough needs exactly one Ubuntu root profile (ubuntu-root).'); + + return self::FAILURE; + } + } + + ManageDevelopmentQemuVm::run($profileNames, (bool) $this->option('as-localhost'), (bool) $this->option('fresh'), (bool) $this->option('gpu')); foreach ($profileNames as $profileName) { $profile = $profiles[$profileName]; diff --git a/app/Console/Commands/SeedDevelopmentQemuServerCommand.php b/app/Console/Commands/SeedDevelopmentQemuServerCommand.php index 26fc463ff7..aee9925163 100644 --- a/app/Console/Commands/SeedDevelopmentQemuServerCommand.php +++ b/app/Console/Commands/SeedDevelopmentQemuServerCommand.php @@ -2,6 +2,7 @@ namespace App\Console\Commands; +use App\Actions\Development\InstallDevelopmentGpuDriver; use App\Actions\Development\SeedDevelopmentQemuServer; use App\Actions\Development\StartDevelopmentInstanceDatabase; use Illuminate\Console\Command; @@ -9,7 +10,7 @@ use Throwable; class SeedDevelopmentQemuServerCommand extends Command { - protected $signature = 'dev:qemu:seed {profile : Profile key from config/development-qemu.php} {--keep-others} {--as-localhost : Use the VM for the localhost server (id 0)} {--ip= : SSH host instead of the profile IP} {--port=22 : SSH port}'; + protected $signature = 'dev:qemu:seed {profile : Profile key from config/development-qemu.php} {--keep-others} {--as-localhost : Use the VM for the localhost server (id 0)} {--ip= : SSH host instead of the profile IP} {--port=22 : SSH port} {--gpu : Install the NVIDIA driver and container toolkit in the VM}'; protected $description = 'Seed one development QEMU server in the Coolify database'; @@ -33,6 +34,12 @@ class SeedDevelopmentQemuServerCommand extends Command } } + if ($this->option('gpu')) { + $this->info('Installing the NVIDIA driver and container toolkit. This can take some minutes.'); + InstallDevelopmentGpuDriver::run($server); + $this->info('The GPU is ready.'); + } + return self::SUCCESS; } } diff --git a/app/Events/ApplicationStopFailed.php b/app/Events/ApplicationStopFailed.php new file mode 100644 index 0000000000..37884978c0 --- /dev/null +++ b/app/Events/ApplicationStopFailed.php @@ -0,0 +1,27 @@ +teamId}")]; + } +} diff --git a/app/Exceptions/ContainerShellUnavailableException.php b/app/Exceptions/ContainerShellUnavailableException.php new file mode 100644 index 0000000000..7c158b86d8 --- /dev/null +++ b/app/Exceptions/ContainerShellUnavailableException.php @@ -0,0 +1,11 @@ +successful()) { return self::refreshMultiplexedConnection($server); } + // A wedged master accepts on its socket but never answers, and a new + // `ssh -fN` would hang on it too. Remove the socket so a fresh master can bind. + if ($check === null) { + File::delete(self::muxSocket($server)); + } + return self::establishNewMultiplexedConnection($server); }); } catch (LockTimeoutException) { @@ -64,8 +74,20 @@ class SshMultiplexingHelper } } + /** + * A disabled server, such as the archive of a transferred server, gets no SSH or scp connection. + */ + private static function assertNotDisabled(Server $server): void + { + // Read the saved flag too: a job that loaded the server before a transfer finished must stop. + if ($server->settings->force_disabled || ($server->exists && $server->settings()->value('force_disabled'))) { + throw new \RuntimeException('Server is disabled.'); + } + } + public static function establishNewMultiplexedConnection(Server $server): bool { + self::assertNotDisabled($server); $sshConfig = self::serverSshConfiguration($server); $sshKeyLocation = $sshConfig['sshKeyLocation']; $muxSocket = $sshConfig['muxFilename']; @@ -73,7 +95,7 @@ class SshMultiplexingHelper $serverInterval = config('constants.ssh.server_interval'); $muxPersistTime = config('constants.ssh.mux_persist_time'); - $establishCommand = "ssh -fN -o ControlMaster=auto -o ControlPath=$muxSocket -o ControlPersist={$muxPersistTime} "; + $establishCommand = 'ssh -fN -o ControlMaster=auto -o ControlPath='.escapeshellarg($muxSocket)." -o ControlPersist={$muxPersistTime} "; if (data_get($server, 'settings.is_cloudflare_tunnel')) { $establishCommand .= ' -o ProxyCommand="cloudflared access ssh --hostname %h" '; @@ -92,8 +114,8 @@ class SshMultiplexingHelper public static function removeMuxFile(Server $server): void { - $checkProcess = Process::run(self::muxControlCommand($server, 'check')); - $pid = preg_match('/pid=(\d+)/', $checkProcess->output().$checkProcess->errorOutput(), $matches) + $checkProcess = self::runMuxControlCommand($server, 'check'); + $pid = $checkProcess && preg_match('/pid=(\d+)/', $checkProcess->output().$checkProcess->errorOutput(), $matches) ? $matches[1] : null; @@ -101,11 +123,19 @@ class SshMultiplexingHelper self::markMuxProcessAsRetiring($pid, self::muxSocket($server)); } - $stopProcess = Process::run(self::muxControlCommand($server, 'stop')); + $stopProcess = self::runMuxControlCommand($server, 'stop'); - if ($pid !== null && ! $stopProcess->successful()) { + if ($stopProcess?->successful()) { + return; + } + + if ($pid !== null) { self::unmarkMuxProcessAsRetiring($pid, self::muxSocket($server)); } + + // A master that does not stop must not stay reusable under the old user, IP, or port. + // Without its socket it is an orphan, and the cleanup job reaps it. + File::delete(self::muxSocket($server)); } public static function markMuxProcessAsRetiring(string $pid, string $muxSocket, ?string $processStartTime = null): void @@ -133,6 +163,7 @@ class SshMultiplexingHelper public static function generateScpCommand(Server $server, string $source, string $dest): string { + self::assertNotDisabled($server); $sshConfig = self::serverSshConfiguration($server); $sshKeyLocation = $sshConfig['sshKeyLocation']; $scpCommand = 'timeout '.config('constants.ssh.command_timeout').' scp '; @@ -173,6 +204,7 @@ class SshMultiplexingHelper */ public static function generateScpDownloadCommand(Server $server, string $remoteSource, string $localDest): string { + self::assertNotDisabled($server); $sshConfig = self::serverSshConfiguration($server); $sshKeyLocation = $sshConfig['sshKeyLocation']; $scpCommand = 'timeout '.config('constants.ssh.command_timeout').' scp '; @@ -234,9 +266,7 @@ class SshMultiplexingHelper private static function sshCommandPrefix(Server $server, bool $disableMultiplexing, ?int $commandTimeout): string { - if ($server->settings->force_disabled) { - throw new \RuntimeException('Server is disabled.'); - } + self::assertNotDisabled($server); $sshConfig = self::serverSshConfiguration($server); $sshKeyLocation = $sshConfig['sshKeyLocation']; @@ -288,17 +318,17 @@ class SshMultiplexingHelper { $sshConfig = self::serverSshConfiguration($server); $muxSocket = $sshConfig['muxFilename']; - $healthCheckTimeout = config('constants.ssh.mux_health_check_timeout'); - - $healthCommand = "timeout $healthCheckTimeout ssh -o ControlMaster=auto -o ControlPath=$muxSocket "; + $healthCommand = 'ssh -o ControlMaster=auto -o ControlPath='.escapeshellarg($muxSocket).' '; if (data_get($server, 'settings.is_cloudflare_tunnel')) { $healthCommand .= '-o ProxyCommand="cloudflared access ssh --hostname %h" '; } $healthCommand .= self::escapedUserAtHost($server)." 'echo \"health_check_ok\"'"; - $process = Process::run($healthCommand); + // The health check goes over the network, so a slow link gets at least the server's connection timeout. + $timeout = max((int) config('constants.ssh.mux_health_check_timeout'), self::getConnectionTimeout($server)); + $process = self::runWithTimeout($healthCommand, $timeout); - return $process->exitCode() === 0 && str_contains($process->output(), 'health_check_ok'); + return $process?->exitCode() === 0 && str_contains($process->output(), 'health_check_ok'); } public static function refreshMultiplexedConnection(Server $server): bool @@ -337,7 +367,36 @@ class SshMultiplexingHelper private static function masterConnectionExists(Server $server): bool { - return Process::run(self::muxControlCommand($server, 'check'))->exitCode() === 0; + return self::runMuxControlCommand($server, 'check')?->successful() === true; + } + + /** + * Run `ssh -O ` against a mux socket file. Returns null when the master does not answer in time. + */ + public static function runMuxSocketCommand(string $muxSocket, string $operation): ?ProcessResult + { + return self::runWithTimeout( + "ssh -O {$operation} -o ControlPath=".escapeshellarg($muxSocket).' localhost', + (int) config('constants.ssh.mux_health_check_timeout'), + ); + } + + private static function runMuxControlCommand(Server $server, string $operation): ?ProcessResult + { + return self::runWithTimeout(self::muxControlCommand($server, $operation), (int) config('constants.ssh.mux_health_check_timeout')); + } + + /** + * Use a PHP-side timeout, not the `timeout` binary: BusyBox `timeout` in the Coolify image + * kills ssh with SIGTERM, and Symfony turns that into an exception. Returns null on timeout. + */ + private static function runWithTimeout(string $command, int $timeout): ?ProcessResult + { + try { + return Process::timeout($timeout)->run($command); + } catch (ProcessTimedOutException) { + return null; + } } private static function connectionIsReusable(Server $server): bool @@ -355,7 +414,7 @@ class SshMultiplexingHelper private static function muxControlCommand(Server $server, string $operation): string { - $command = "ssh -O {$operation} -o ControlPath=".self::muxSocket($server).' '; + $command = "ssh -O {$operation} -o ControlPath=".escapeshellarg(self::muxSocket($server)).' '; if (data_get($server, 'settings.is_cloudflare_tunnel')) { $command .= '-o ProxyCommand="cloudflared access ssh --hostname %h" '; } @@ -366,7 +425,7 @@ class SshMultiplexingHelper private static function multiplexingOptions(Server $server): string { return '-o ControlMaster=auto ' - .'-o ControlPath='.self::muxSocket($server).' ' + .'-o ControlPath='.escapeshellarg(self::muxSocket($server)).' ' .'-o ControlPersist='.config('constants.ssh.mux_persist_time').' '; } @@ -422,7 +481,7 @@ class SshMultiplexingHelper private static function getCommonSshOptions(Server $server, string $sshKeyLocation, int $connectionTimeout, int $serverInterval, bool $isScp = false): string { - $options = "-i {$sshKeyLocation} " + $options = '-i '.escapeshellarg($sshKeyLocation).' ' .'-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null ' .'-o PasswordAuthentication=no ' ."-o ConnectTimeout=$connectionTimeout " diff --git a/app/Helpers/SshRetryHandler.php b/app/Helpers/SshRetryHandler.php index aaaf4252a8..45aaf6f9fa 100644 --- a/app/Helpers/SshRetryHandler.php +++ b/app/Helpers/SshRetryHandler.php @@ -27,8 +27,8 @@ class SshRetryHandler /** * Convenience static method for retry execution */ - public static function retry(callable $callback, array $context = [], bool $throwError = true) + public static function retry(callable $callback, array $context = [], bool $throwError = true, ?int $maxAttempts = null) { - return self::instance()->executeWithSshRetry($callback, $context, $throwError); + return self::instance()->executeWithSshRetry($callback, $context, $throwError, $maxAttempts); } } diff --git a/app/Http/Controllers/Api/ApplicationsController.php b/app/Http/Controllers/Api/ApplicationsController.php index 136b7c5992..6e6e3018f0 100644 --- a/app/Http/Controllers/Api/ApplicationsController.php +++ b/app/Http/Controllers/Api/ApplicationsController.php @@ -53,13 +53,16 @@ class ApplicationsController extends Controller 'include_source_commit_in_build', 'is_env_sorting_enabled', 'redact_all_env_values_in_logs', + 'include_coolify_default_variables', 'is_pr_deployments_public_enabled', 'stop_grace_period', + 'rolling_update_drain_seconds', 'docker_images_to_keep', 'is_gzip_enabled', 'is_stripprefix_enabled', 'is_permanent_redirect_enabled', 'traefik_certificate_resolver', + 'is_container_label_readonly_enabled', 'is_raw_compose_deployment_enabled', 'is_log_drain_enabled', 'is_gpu_enabled', @@ -81,10 +84,12 @@ class ApplicationsController extends Controller 'include_source_commit_in_build', 'is_env_sorting_enabled', 'redact_all_env_values_in_logs', + 'include_coolify_default_variables', 'is_pr_deployments_public_enabled', 'is_gzip_enabled', 'is_stripprefix_enabled', 'is_permanent_redirect_enabled', + 'is_container_label_readonly_enabled', 'is_raw_compose_deployment_enabled', 'is_log_drain_enabled', 'is_gpu_enabled', @@ -147,7 +152,12 @@ class ApplicationsController extends Controller $application->settings?->makeHidden(['id', 'application_id', 'created_at', 'updated_at']); } - return serializeApiResponse($application); + $payload = serializeApiResponse($application); + if ($application instanceof Application && filled(data_get($payload, 'docker_compose'))) { + $payload['docker_compose'] = redactLockedValuesInGeneratedCompose($payload['docker_compose'], $application); + } + + return $payload; } /** @@ -231,15 +241,18 @@ class ApplicationsController extends Controller return; } + $wasLabelManagementEnabled = $application->settings->is_container_label_readonly_enabled; + + $application->settings->fill($settings)->save(); + $regenerateLabels = ! $application->wasRecentlyCreated && $application->settings->is_container_label_readonly_enabled - && (array_key_exists('is_gzip_enabled', $settings) + && (! $wasLabelManagementEnabled + || array_key_exists('is_gzip_enabled', $settings) || array_key_exists('is_stripprefix_enabled', $settings) || array_key_exists('is_permanent_redirect_enabled', $settings) || array_key_exists('traefik_certificate_resolver', $settings)); - $application->settings->fill($settings)->save(); - if ($regenerateLabels) { $application->custom_labels = str(implode('|coolify|', generateLabelsApplication($application)))->replace('|coolify|', "\n"); $application->save(); @@ -433,6 +446,7 @@ class ApplicationsController extends Controller 'instant_deploy' => ['type' => 'boolean', 'description' => 'The flag to indicate if the application should be deployed instantly.'], 'dockerfile' => ['type' => 'string', 'description' => 'The Dockerfile content.'], 'dockerfile_location' => ['type' => 'string', 'description' => 'The Dockerfile location in the repository.'], + 'dockerfile_target_build' => ['type' => 'string', 'nullable' => true, 'description' => 'The Docker build target (stage) in the Dockerfile.'], 'docker_compose_location' => ['type' => 'string', 'description' => 'The Docker Compose location.'], 'docker_compose_custom_start_command' => ['type' => 'string', 'description' => 'The Docker Compose custom start command.'], 'docker_compose_custom_build_command' => ['type' => 'string', 'description' => 'The Docker Compose custom build command.'], @@ -459,13 +473,16 @@ class ApplicationsController extends Controller 'include_source_commit_in_build' => ['type' => 'boolean', 'description' => 'Include the source commit in the build.'], 'is_env_sorting_enabled' => ['type' => 'boolean', 'description' => 'Sort environment variables.'], 'redact_all_env_values_in_logs' => ['type' => 'boolean', 'description' => 'Redact all environment variable values in logs. When false, only locked (shown once) values are redacted. Requires team admin permissions.'], + 'include_coolify_default_variables' => ['type' => 'boolean', 'default' => true, 'description' => 'Add the predefined COOLIFY_* variables (COOLIFY_URL, COOLIFY_FQDN, ...) to the environment. Requires team admin permissions. Redeploy to apply it.'], 'is_pr_deployments_public_enabled' => ['type' => 'boolean', 'description' => 'Make pull request deployments public.'], 'stop_grace_period' => ['type' => 'integer', 'nullable' => true, 'minimum' => 1, 'maximum' => 3600, 'description' => 'Container stop grace period in seconds.'], + 'rolling_update_drain_seconds' => ['type' => 'integer', 'minimum' => 0, 'maximum' => 300, 'description' => 'Seconds to wait after the new container is healthy before old containers stop during a rolling update. 0 turns the wait off. Default: 5.'], 'docker_images_to_keep' => ['type' => 'integer', 'minimum' => 0, 'maximum' => 100, 'description' => 'Number of Docker images to retain.'], 'is_gzip_enabled' => ['type' => 'boolean', 'description' => 'Enable gzip compression.'], 'is_stripprefix_enabled' => ['type' => 'boolean', 'description' => 'Enable path prefix stripping.'], 'is_permanent_redirect_enabled' => ['type' => 'boolean', 'description' => 'Use permanent redirects (301/308) instead of temporary (302/307) for the HTTP to HTTPS and www redirects. Defaults to false.'], 'traefik_certificate_resolver' => ['type' => 'string', 'nullable' => true, 'description' => 'Traefik certificate resolver for this application. Null uses the server setting, "none" uses no resolver.'], + 'is_container_label_readonly_enabled' => ['type' => 'boolean', 'default' => true, 'description' => 'Let Coolify manage the container labels. True regenerates custom_labels from domains, redirect, and proxy settings. False keeps custom_labels as you set them: domains, redirect, and basic auth changes then do not change the labels.'], 'is_raw_compose_deployment_enabled' => ['type' => 'boolean', 'description' => 'Deploy the raw Docker Compose definition.'], 'is_log_drain_enabled' => ['type' => 'boolean', 'description' => 'Enable log drain for the application.'], 'is_gpu_enabled' => ['type' => 'boolean', 'description' => 'Enable GPU support.'], @@ -639,6 +656,7 @@ class ApplicationsController extends Controller 'instant_deploy' => ['type' => 'boolean', 'description' => 'The flag to indicate if the application should be deployed instantly.'], 'dockerfile' => ['type' => 'string', 'description' => 'The Dockerfile content.'], 'dockerfile_location' => ['type' => 'string', 'description' => 'The Dockerfile location in the repository'], + 'dockerfile_target_build' => ['type' => 'string', 'nullable' => true, 'description' => 'The Docker build target (stage) in the Dockerfile.'], 'docker_compose_location' => ['type' => 'string', 'description' => 'The Docker Compose location.'], 'docker_compose_custom_start_command' => ['type' => 'string', 'description' => 'The Docker Compose custom start command.'], 'docker_compose_custom_build_command' => ['type' => 'string', 'description' => 'The Docker Compose custom build command.'], @@ -665,13 +683,16 @@ class ApplicationsController extends Controller 'include_source_commit_in_build' => ['type' => 'boolean', 'description' => 'Include the source commit in the build.'], 'is_env_sorting_enabled' => ['type' => 'boolean', 'description' => 'Sort environment variables.'], 'redact_all_env_values_in_logs' => ['type' => 'boolean', 'description' => 'Redact all environment variable values in logs. When false, only locked (shown once) values are redacted. Requires team admin permissions.'], + 'include_coolify_default_variables' => ['type' => 'boolean', 'default' => true, 'description' => 'Add the predefined COOLIFY_* variables (COOLIFY_URL, COOLIFY_FQDN, ...) to the environment. Requires team admin permissions. Redeploy to apply it.'], 'is_pr_deployments_public_enabled' => ['type' => 'boolean', 'description' => 'Make pull request deployments public.'], 'stop_grace_period' => ['type' => 'integer', 'nullable' => true, 'minimum' => 1, 'maximum' => 3600, 'description' => 'Container stop grace period in seconds.'], + 'rolling_update_drain_seconds' => ['type' => 'integer', 'minimum' => 0, 'maximum' => 300, 'description' => 'Seconds to wait after the new container is healthy before old containers stop during a rolling update. 0 turns the wait off. Default: 5.'], 'docker_images_to_keep' => ['type' => 'integer', 'minimum' => 0, 'maximum' => 100, 'description' => 'Number of Docker images to retain.'], 'is_gzip_enabled' => ['type' => 'boolean', 'description' => 'Enable gzip compression.'], 'is_stripprefix_enabled' => ['type' => 'boolean', 'description' => 'Enable path prefix stripping.'], 'is_permanent_redirect_enabled' => ['type' => 'boolean', 'description' => 'Use permanent redirects (301/308) instead of temporary (302/307) for the HTTP to HTTPS and www redirects. Defaults to false.'], 'traefik_certificate_resolver' => ['type' => 'string', 'nullable' => true, 'description' => 'Traefik certificate resolver for this application. Null uses the server setting, "none" uses no resolver.'], + 'is_container_label_readonly_enabled' => ['type' => 'boolean', 'default' => true, 'description' => 'Let Coolify manage the container labels. True regenerates custom_labels from domains, redirect, and proxy settings. False keeps custom_labels as you set them: domains, redirect, and basic auth changes then do not change the labels.'], 'is_raw_compose_deployment_enabled' => ['type' => 'boolean', 'description' => 'Deploy the raw Docker Compose definition.'], 'is_log_drain_enabled' => ['type' => 'boolean', 'description' => 'Enable log drain for the application.'], 'is_gpu_enabled' => ['type' => 'boolean', 'description' => 'Enable GPU support.'], @@ -845,6 +866,7 @@ class ApplicationsController extends Controller 'instant_deploy' => ['type' => 'boolean', 'description' => 'The flag to indicate if the application should be deployed instantly.'], 'dockerfile' => ['type' => 'string', 'description' => 'The Dockerfile content.'], 'dockerfile_location' => ['type' => 'string', 'description' => 'The Dockerfile location in the repository.'], + 'dockerfile_target_build' => ['type' => 'string', 'nullable' => true, 'description' => 'The Docker build target (stage) in the Dockerfile.'], 'docker_compose_location' => ['type' => 'string', 'description' => 'The Docker Compose location.'], 'docker_compose_custom_start_command' => ['type' => 'string', 'description' => 'The Docker Compose custom start command.'], 'docker_compose_custom_build_command' => ['type' => 'string', 'description' => 'The Docker Compose custom build command.'], @@ -871,13 +893,16 @@ class ApplicationsController extends Controller 'include_source_commit_in_build' => ['type' => 'boolean', 'description' => 'Include the source commit in the build.'], 'is_env_sorting_enabled' => ['type' => 'boolean', 'description' => 'Sort environment variables.'], 'redact_all_env_values_in_logs' => ['type' => 'boolean', 'description' => 'Redact all environment variable values in logs. When false, only locked (shown once) values are redacted. Requires team admin permissions.'], + 'include_coolify_default_variables' => ['type' => 'boolean', 'default' => true, 'description' => 'Add the predefined COOLIFY_* variables (COOLIFY_URL, COOLIFY_FQDN, ...) to the environment. Requires team admin permissions. Redeploy to apply it.'], 'is_pr_deployments_public_enabled' => ['type' => 'boolean', 'description' => 'Make pull request deployments public.'], 'stop_grace_period' => ['type' => 'integer', 'nullable' => true, 'minimum' => 1, 'maximum' => 3600, 'description' => 'Container stop grace period in seconds.'], + 'rolling_update_drain_seconds' => ['type' => 'integer', 'minimum' => 0, 'maximum' => 300, 'description' => 'Seconds to wait after the new container is healthy before old containers stop during a rolling update. 0 turns the wait off. Default: 5.'], 'docker_images_to_keep' => ['type' => 'integer', 'minimum' => 0, 'maximum' => 100, 'description' => 'Number of Docker images to retain.'], 'is_gzip_enabled' => ['type' => 'boolean', 'description' => 'Enable gzip compression.'], 'is_stripprefix_enabled' => ['type' => 'boolean', 'description' => 'Enable path prefix stripping.'], 'is_permanent_redirect_enabled' => ['type' => 'boolean', 'description' => 'Use permanent redirects (301/308) instead of temporary (302/307) for the HTTP to HTTPS and www redirects. Defaults to false.'], 'traefik_certificate_resolver' => ['type' => 'string', 'nullable' => true, 'description' => 'Traefik certificate resolver for this application. Null uses the server setting, "none" uses no resolver.'], + 'is_container_label_readonly_enabled' => ['type' => 'boolean', 'default' => true, 'description' => 'Let Coolify manage the container labels. True regenerates custom_labels from domains, redirect, and proxy settings. False keeps custom_labels as you set them: domains, redirect, and basic auth changes then do not change the labels.'], 'is_raw_compose_deployment_enabled' => ['type' => 'boolean', 'description' => 'Deploy the raw Docker Compose definition.'], 'is_log_drain_enabled' => ['type' => 'boolean', 'description' => 'Enable log drain for the application.'], 'is_gpu_enabled' => ['type' => 'boolean', 'description' => 'Enable GPU support.'], @@ -1048,13 +1073,16 @@ class ApplicationsController extends Controller 'include_source_commit_in_build' => ['type' => 'boolean', 'description' => 'Include the source commit in the build.'], 'is_env_sorting_enabled' => ['type' => 'boolean', 'description' => 'Sort environment variables.'], 'redact_all_env_values_in_logs' => ['type' => 'boolean', 'description' => 'Redact all environment variable values in logs. When false, only locked (shown once) values are redacted. Requires team admin permissions.'], + 'include_coolify_default_variables' => ['type' => 'boolean', 'default' => true, 'description' => 'Add the predefined COOLIFY_* variables (COOLIFY_URL, COOLIFY_FQDN, ...) to the environment. Requires team admin permissions. Redeploy to apply it.'], 'is_pr_deployments_public_enabled' => ['type' => 'boolean', 'description' => 'Make pull request deployments public.'], 'stop_grace_period' => ['type' => 'integer', 'nullable' => true, 'minimum' => 1, 'maximum' => 3600, 'description' => 'Container stop grace period in seconds.'], + 'rolling_update_drain_seconds' => ['type' => 'integer', 'minimum' => 0, 'maximum' => 300, 'description' => 'Seconds to wait after the new container is healthy before old containers stop during a rolling update. 0 turns the wait off. Default: 5.'], 'docker_images_to_keep' => ['type' => 'integer', 'minimum' => 0, 'maximum' => 100, 'description' => 'Number of Docker images to retain.'], 'is_gzip_enabled' => ['type' => 'boolean', 'description' => 'Enable gzip compression.'], 'is_stripprefix_enabled' => ['type' => 'boolean', 'description' => 'Enable path prefix stripping.'], 'is_permanent_redirect_enabled' => ['type' => 'boolean', 'description' => 'Use permanent redirects (301/308) instead of temporary (302/307) for the HTTP to HTTPS and www redirects. Defaults to false.'], 'traefik_certificate_resolver' => ['type' => 'string', 'nullable' => true, 'description' => 'Traefik certificate resolver for this application. Null uses the server setting, "none" uses no resolver.'], + 'is_container_label_readonly_enabled' => ['type' => 'boolean', 'default' => true, 'description' => 'Let Coolify manage the container labels. True regenerates custom_labels from domains, redirect, and proxy settings. False keeps custom_labels as you set them: domains, redirect, and basic auth changes then do not change the labels.'], 'is_raw_compose_deployment_enabled' => ['type' => 'boolean', 'description' => 'Deploy the raw Docker Compose definition.'], 'is_log_drain_enabled' => ['type' => 'boolean', 'description' => 'Enable log drain for the application.'], 'is_gpu_enabled' => ['type' => 'boolean', 'description' => 'Enable GPU support.'], @@ -1221,13 +1249,16 @@ class ApplicationsController extends Controller 'include_source_commit_in_build' => ['type' => 'boolean', 'description' => 'Include the source commit in the build.'], 'is_env_sorting_enabled' => ['type' => 'boolean', 'description' => 'Sort environment variables.'], 'redact_all_env_values_in_logs' => ['type' => 'boolean', 'description' => 'Redact all environment variable values in logs. When false, only locked (shown once) values are redacted. Requires team admin permissions.'], + 'include_coolify_default_variables' => ['type' => 'boolean', 'default' => true, 'description' => 'Add the predefined COOLIFY_* variables (COOLIFY_URL, COOLIFY_FQDN, ...) to the environment. Requires team admin permissions. Redeploy to apply it.'], 'is_pr_deployments_public_enabled' => ['type' => 'boolean', 'description' => 'Make pull request deployments public.'], 'stop_grace_period' => ['type' => 'integer', 'nullable' => true, 'minimum' => 1, 'maximum' => 3600, 'description' => 'Container stop grace period in seconds.'], + 'rolling_update_drain_seconds' => ['type' => 'integer', 'minimum' => 0, 'maximum' => 300, 'description' => 'Seconds to wait after the new container is healthy before old containers stop during a rolling update. 0 turns the wait off. Default: 5.'], 'docker_images_to_keep' => ['type' => 'integer', 'minimum' => 0, 'maximum' => 100, 'description' => 'Number of Docker images to retain.'], 'is_gzip_enabled' => ['type' => 'boolean', 'description' => 'Enable gzip compression.'], 'is_stripprefix_enabled' => ['type' => 'boolean', 'description' => 'Enable path prefix stripping.'], 'is_permanent_redirect_enabled' => ['type' => 'boolean', 'description' => 'Use permanent redirects (301/308) instead of temporary (302/307) for the HTTP to HTTPS and www redirects. Defaults to false.'], 'traefik_certificate_resolver' => ['type' => 'string', 'nullable' => true, 'description' => 'Traefik certificate resolver for this application. Null uses the server setting, "none" uses no resolver.'], + 'is_container_label_readonly_enabled' => ['type' => 'boolean', 'default' => true, 'description' => 'Let Coolify manage the container labels. True regenerates custom_labels from domains, redirect, and proxy settings. False keeps custom_labels as you set them: domains, redirect, and basic auth changes then do not change the labels.'], 'is_raw_compose_deployment_enabled' => ['type' => 'boolean', 'description' => 'Deploy the raw Docker Compose definition.'], 'is_log_drain_enabled' => ['type' => 'boolean', 'description' => 'Enable log drain for the application.'], 'is_gpu_enabled' => ['type' => 'boolean', 'description' => 'Enable GPU support.'], @@ -1337,7 +1368,7 @@ class ApplicationsController extends Controller if ($return instanceof JsonResponse) { return $return; } - $allowedFields = ['project_uuid', 'environment_name', 'environment_uuid', 'server_uuid', 'destination_uuid', 'type', 'name', 'description', 'is_static', 'is_spa', 'is_auto_deploy_enabled', 'is_force_https_enabled', 'is_preview_deployments_enabled', 'domains', 'noindex_domains', 'git_repository', 'git_branch', 'git_commit_sha', 'git_https_username', 'git_https_password', 'private_key_uuid', 'docker_registry_image_name', 'docker_registry_image_tag', 'build_pack', 'install_command', 'build_command', 'start_command', 'ports_exposes', 'ports_mappings', 'custom_network_aliases', 'base_directory', 'publish_directory', 'health_check_enabled', 'health_check_type', 'health_check_command', 'health_check_path', 'health_check_port', 'health_check_host', 'health_check_method', 'health_check_return_code', 'health_check_scheme', 'health_check_response_text', 'health_check_interval', 'health_check_timeout', 'health_check_retries', 'health_check_start_period', 'limits_memory', 'limits_memory_swap', 'limits_memory_swappiness', 'limits_memory_reservation', 'limits_cpus', 'limits_cpuset', 'limits_cpu_shares', 'custom_labels', 'custom_docker_run_options', 'post_deployment_command', 'post_deployment_command_container', 'pre_deployment_command', 'pre_deployment_command_container', 'manual_webhook_secret_github', 'manual_webhook_secret_gitlab', 'manual_webhook_secret_bitbucket', 'manual_webhook_secret_gitea', 'redirect', 'github_app_uuid', 'instant_deploy', 'dockerfile', 'dockerfile_location', 'docker_compose_location', 'docker_compose_raw', 'docker_compose_custom_start_command', 'docker_compose_custom_build_command', 'docker_compose_domains', 'watch_paths', 'use_build_server', 'use_build_secrets', 'static_image', 'custom_nginx_configuration', 'is_http_basic_auth_enabled', 'http_basic_auth_username', 'http_basic_auth_password', 'http_basic_auth_hash_algorithm', 'http_basic_auth_bcrypt_cost', 'http_basic_auth_argon2id_memory_cost', 'http_basic_auth_argon2id_time_cost', 'connect_to_docker_network', 'force_domain_override', 'autogenerate_domain', 'is_container_label_escape_enabled', 'tags', 'is_preserve_repository_enabled', 'preview_url_template', 'max_restart_count', ...self::APPLICATION_SETTING_FIELDS]; + $allowedFields = ['project_uuid', 'environment_name', 'environment_uuid', 'server_uuid', 'destination_uuid', 'type', 'name', 'description', 'is_static', 'is_spa', 'is_auto_deploy_enabled', 'is_force_https_enabled', 'is_preview_deployments_enabled', 'domains', 'noindex_domains', 'git_repository', 'git_branch', 'git_commit_sha', 'git_https_username', 'git_https_password', 'private_key_uuid', 'docker_registry_image_name', 'docker_registry_image_tag', 'build_pack', 'install_command', 'build_command', 'start_command', 'ports_exposes', 'ports_mappings', 'custom_network_aliases', 'base_directory', 'publish_directory', 'health_check_enabled', 'health_check_type', 'health_check_command', 'health_check_path', 'health_check_port', 'health_check_host', 'health_check_method', 'health_check_return_code', 'health_check_scheme', 'health_check_response_text', 'health_check_interval', 'health_check_timeout', 'health_check_retries', 'health_check_start_period', 'limits_memory', 'limits_memory_swap', 'limits_memory_swappiness', 'limits_memory_reservation', 'limits_cpus', 'limits_cpuset', 'limits_cpu_shares', 'custom_labels', 'custom_docker_run_options', 'post_deployment_command', 'post_deployment_command_container', 'pre_deployment_command', 'pre_deployment_command_container', 'manual_webhook_secret_github', 'manual_webhook_secret_gitlab', 'manual_webhook_secret_bitbucket', 'manual_webhook_secret_gitea', 'redirect', 'github_app_uuid', 'instant_deploy', 'dockerfile', 'dockerfile_location', 'dockerfile_target_build', 'docker_compose_location', 'docker_compose_raw', 'docker_compose_custom_start_command', 'docker_compose_custom_build_command', 'docker_compose_domains', 'watch_paths', 'use_build_server', 'use_build_secrets', 'static_image', 'custom_nginx_configuration', 'is_http_basic_auth_enabled', 'http_basic_auth_username', 'http_basic_auth_password', 'http_basic_auth_hash_algorithm', 'http_basic_auth_bcrypt_cost', 'http_basic_auth_argon2id_memory_cost', 'http_basic_auth_argon2id_time_cost', 'connect_to_docker_network', 'force_domain_override', 'autogenerate_domain', 'is_container_label_escape_enabled', 'tags', 'is_preserve_repository_enabled', 'preview_url_template', 'max_restart_count', ...self::APPLICATION_SETTING_FIELDS]; $validator = customApiValidator($request->all(), [ 'name' => 'string|max:255', @@ -3231,6 +3262,7 @@ class ApplicationsController extends Controller 'instant_deploy' => ['type' => 'boolean', 'description' => 'The flag to indicate if the application should be deployed instantly.'], 'dockerfile' => ['type' => 'string', 'description' => 'The Dockerfile content.'], 'dockerfile_location' => ['type' => 'string', 'description' => 'The Dockerfile location in the repository.'], + 'dockerfile_target_build' => ['type' => 'string', 'nullable' => true, 'description' => 'The Docker build target (stage) in the Dockerfile.'], 'docker_compose_location' => ['type' => 'string', 'description' => 'The Docker Compose location.'], 'docker_compose_custom_start_command' => ['type' => 'string', 'description' => 'The Docker Compose custom start command.'], 'docker_compose_custom_build_command' => ['type' => 'string', 'description' => 'The Docker Compose custom build command.'], @@ -3261,13 +3293,16 @@ class ApplicationsController extends Controller 'include_source_commit_in_build' => ['type' => 'boolean', 'description' => 'Include the source commit in the build.'], 'is_env_sorting_enabled' => ['type' => 'boolean', 'description' => 'Sort environment variables.'], 'redact_all_env_values_in_logs' => ['type' => 'boolean', 'description' => 'Redact all environment variable values in logs. When false, only locked (shown once) values are redacted. Requires team admin permissions.'], + 'include_coolify_default_variables' => ['type' => 'boolean', 'default' => true, 'description' => 'Add the predefined COOLIFY_* variables (COOLIFY_URL, COOLIFY_FQDN, ...) to the environment. Requires team admin permissions. Redeploy to apply it.'], 'is_pr_deployments_public_enabled' => ['type' => 'boolean', 'description' => 'Make pull request deployments public.'], 'stop_grace_period' => ['type' => 'integer', 'nullable' => true, 'minimum' => 1, 'maximum' => 3600, 'description' => 'Container stop grace period in seconds.'], + 'rolling_update_drain_seconds' => ['type' => 'integer', 'minimum' => 0, 'maximum' => 300, 'description' => 'Seconds to wait after the new container is healthy before old containers stop during a rolling update. 0 turns the wait off. Default: 5.'], 'docker_images_to_keep' => ['type' => 'integer', 'minimum' => 0, 'maximum' => 100, 'description' => 'Number of Docker images to retain.'], 'is_gzip_enabled' => ['type' => 'boolean', 'description' => 'Enable gzip compression.'], 'is_stripprefix_enabled' => ['type' => 'boolean', 'description' => 'Enable path prefix stripping.'], 'is_permanent_redirect_enabled' => ['type' => 'boolean', 'description' => 'Use permanent redirects (301/308) instead of temporary (302/307) for the HTTP to HTTPS and www redirects. Defaults to false.'], 'traefik_certificate_resolver' => ['type' => 'string', 'nullable' => true, 'description' => 'Traefik certificate resolver for this application. Null uses the server setting, "none" uses no resolver.'], + 'is_container_label_readonly_enabled' => ['type' => 'boolean', 'default' => true, 'description' => 'Let Coolify manage the container labels. True regenerates custom_labels from domains, redirect, and proxy settings. False keeps custom_labels as you set them: domains, redirect, and basic auth changes then do not change the labels.'], 'is_raw_compose_deployment_enabled' => ['type' => 'boolean', 'description' => 'Deploy the raw Docker Compose definition.'], 'is_log_drain_enabled' => ['type' => 'boolean', 'description' => 'Enable log drain for the application.'], 'is_gpu_enabled' => ['type' => 'boolean', 'description' => 'Enable GPU support.'], @@ -3372,7 +3407,7 @@ class ApplicationsController extends Controller $this->authorize('update', $application); - if ($request->has('redact_all_env_values_in_logs')) { + if ($request->has('redact_all_env_values_in_logs') || $request->has('include_coolify_default_variables')) { $this->authorize('manageEnvironment', $application); } @@ -4062,95 +4097,28 @@ class ApplicationsController extends Controller ], 422); } $is_preview = $request->is_preview ?? false; - $is_literal = $request->is_literal ?? false; $key = str($request->key)->trim()->replace(' ', '_')->value; - if ($is_preview) { - $env = $application->environment_variables_preview->where('key', $key)->first(); - if ($env) { - $env->value = $request->value; - if ($env->is_literal != $is_literal) { - $env->is_literal = $is_literal; - } - if ($env->is_preview != $is_preview) { - $env->is_preview = $is_preview; - } - if ($env->is_multiline != $request->is_multiline) { - $env->is_multiline = $request->is_multiline; - } - if ($env->is_shown_once != $request->is_shown_once) { - $env->is_shown_once = $request->is_shown_once; - } - if ($request->has('is_runtime') && $env->is_runtime != $request->is_runtime) { - $env->is_runtime = $request->is_runtime; - } - if ($request->has('is_buildtime') && $env->is_buildtime != $request->is_buildtime) { - $env->is_buildtime = $request->is_buildtime; - } - if ($request->has('comment') && $env->comment != $request->comment) { - $env->comment = $request->comment; - } - $env->save(); - - auditLog('api.application.env_updated', [ - 'team_id' => $teamId, - 'application_uuid' => $application->uuid, - 'env_uuid' => $env->uuid, - 'env_key' => $env->key, - 'is_preview' => (bool) $is_preview, - ]); - - return response()->json($this->removeSensitiveData($env))->setStatusCode(201); - } else { - return response()->json([ - 'message' => 'Environment variable not found.', - ], 404); - } - } else { - $env = $application->environment_variables->where('key', $key)->first(); - if ($env) { - $env->value = $request->value; - if ($env->is_literal != $is_literal) { - $env->is_literal = $is_literal; - } - if ($env->is_preview != $is_preview) { - $env->is_preview = $is_preview; - } - if ($env->is_multiline != $request->is_multiline) { - $env->is_multiline = $request->is_multiline; - } - if ($env->is_shown_once != $request->is_shown_once) { - $env->is_shown_once = $request->is_shown_once; - } - if ($request->has('is_runtime') && $env->is_runtime != $request->is_runtime) { - $env->is_runtime = $request->is_runtime; - } - if ($request->has('is_buildtime') && $env->is_buildtime != $request->is_buildtime) { - $env->is_buildtime = $request->is_buildtime; - } - if ($request->has('comment') && $env->comment != $request->comment) { - $env->comment = $request->comment; - } - $env->save(); - - auditLog('api.application.env_updated', [ - 'team_id' => $teamId, - 'application_uuid' => $application->uuid, - 'env_uuid' => $env->uuid, - 'env_key' => $env->key, - 'is_preview' => (bool) $is_preview, - ]); - - return response()->json($this->removeSensitiveData($env))->setStatusCode(201); - } else { - return response()->json([ - 'message' => 'Environment variable not found.', - ], 404); - } + $environmentVariables = $is_preview ? $application->environment_variables_preview : $application->environment_variables; + $env = $environmentVariables->where('key', $key)->first(); + if (! $env) { + return response()->json([ + 'message' => 'Environment variable not found.', + ], 404); } - return response()->json([ - 'message' => 'Something is not okay. Are you okay?', - ], 500); + $env->value = $request->value; + $env->fill($request->only(['is_literal', 'is_multiline', 'is_shown_once', 'is_runtime', 'is_buildtime', 'comment'])); + $env->save(); + + auditLog('api.application.env_updated', [ + 'team_id' => $teamId, + 'application_uuid' => $application->uuid, + 'env_uuid' => $env->uuid, + 'env_key' => $env->key, + 'is_preview' => (bool) $is_preview, + ]); + + return response()->json($this->removeSensitiveData($env))->setStatusCode(201); } #[OA\Patch( @@ -4295,25 +4263,7 @@ class ApplicationsController extends Controller $env = $application->environment_variables_preview()->where('key', $key)->first(); if ($env) { $env->value = $item->get('value'); - - if ($env->is_literal != $is_literal) { - $env->is_literal = $is_literal; - } - if ($env->is_multiline != $item->get('is_multiline')) { - $env->is_multiline = $item->get('is_multiline'); - } - if ($env->is_shown_once != $item->get('is_shown_once')) { - $env->is_shown_once = $item->get('is_shown_once'); - } - if ($item->has('is_runtime') && $env->is_runtime != $item->get('is_runtime')) { - $env->is_runtime = $item->get('is_runtime'); - } - if ($item->has('is_buildtime') && $env->is_buildtime != $item->get('is_buildtime')) { - $env->is_buildtime = $item->get('is_buildtime'); - } - if ($item->has('comment') && $env->comment != $item->get('comment')) { - $env->comment = $item->get('comment'); - } + $env->fill($item->only(['is_literal', 'is_multiline', 'is_shown_once', 'is_runtime', 'is_buildtime', 'comment'])->all()); $env->save(); } else { $env = $application->environment_variables()->create([ @@ -4334,24 +4284,7 @@ class ApplicationsController extends Controller $env = $application->environment_variables()->where('key', $key)->first(); if ($env) { $env->value = $item->get('value'); - if ($env->is_literal != $is_literal) { - $env->is_literal = $is_literal; - } - if ($env->is_multiline != $item->get('is_multiline')) { - $env->is_multiline = $item->get('is_multiline'); - } - if ($env->is_shown_once != $item->get('is_shown_once')) { - $env->is_shown_once = $item->get('is_shown_once'); - } - if ($item->has('is_runtime') && $env->is_runtime != $item->get('is_runtime')) { - $env->is_runtime = $item->get('is_runtime'); - } - if ($item->has('is_buildtime') && $env->is_buildtime != $item->get('is_buildtime')) { - $env->is_buildtime = $item->get('is_buildtime'); - } - if ($item->has('comment') && $env->comment != $item->get('comment')) { - $env->comment = $item->get('comment'); - } + $env->fill($item->only(['is_literal', 'is_multiline', 'is_shown_once', 'is_runtime', 'is_buildtime', 'comment'])->all()); $env->save(); } else { $env = $application->environment_variables()->create([ diff --git a/app/Http/Controllers/Api/DatabasesController.php b/app/Http/Controllers/Api/DatabasesController.php index eef4cb477a..33f326fd29 100644 --- a/app/Http/Controllers/Api/DatabasesController.php +++ b/app/Http/Controllers/Api/DatabasesController.php @@ -210,6 +210,25 @@ class DatabasesController extends Controller return $backup->unsetRelation('s3Storages')->unsetRelation('s3'); } + /** + * Build the create response; the connection URLs contain credentials, so they need sensitive read access. + * + * @return array{uuid: string, internal_db_url?: string, external_db_url?: string} + */ + private function createdDatabasePayload(Request $request, $database): array + { + $payload = ['uuid' => $database->uuid]; + if ($request->attributes->get('can_read_sensitive', false) !== true) { + return $payload; + } + $payload['internal_db_url'] = $database->internal_db_url; + if ($database->is_public && $database->public_port) { + $payload['external_db_url'] = $database->external_db_url; + } + + return $payload; + } + private function removeSensitiveData($database, bool $loadNestedServerSecrets = false) { $database->makeHidden([ @@ -237,7 +256,19 @@ class DatabasesController extends Controller $this->hideNestedServerSecrets($database, $loadNestedServerSecrets); } - return serializeApiResponse($database); + $payload = serializeApiResponse($database); + + // A locked password variable is never shown again, also not inside the connection URLs. + if (request()->attributes->get('can_read_sensitive', false) === true + && method_exists($database, 'isCredentialLocked') && $database->isCredentialLocked()) { + foreach (['internal_db_url', 'external_db_url'] as $key) { + if ($payload->has($key)) { + $payload->put($key, null); + } + } + } + + return $payload; } private function hideNestedServerSecrets(Model $model, bool $loadRelations = false): void @@ -444,7 +475,7 @@ class DatabasesController extends Controller $this->authorize('view', $database); $messageSecrets = []; - $backupConfig = ScheduledDatabaseBackup::ownedByCurrentTeamAPI($teamId)->with(['executions', 's3', 's3Storages'])->where('database_id', $database->id)->get() + $backupConfig = ScheduledDatabaseBackup::ownedByCurrentTeamAPI($teamId)->with(['executions', 's3', 's3Storages'])->where('database_id', $database->id)->where('database_type', $database->getMorphClass())->get() ->map(function (ScheduledDatabaseBackup $backup) use ($database, &$messageSecrets): ScheduledDatabaseBackup { $backup->executions->each(function (ScheduledDatabaseBackupExecution $execution) use ($database, &$messageSecrets): void { $this->redactBackupExecutionMessage($execution, $database, $messageSecrets); @@ -594,6 +625,8 @@ class DatabasesController extends Controller 'health_check_retries' => ['type' => 'integer', 'description' => 'Healthcheck retries count.', 'minimum' => 1, 'default' => 5], 'health_check_start_period' => ['type' => 'integer', 'description' => 'Healthcheck start period in seconds.', 'minimum' => 0, 'default' => 5], 'redact_all_env_values_in_logs' => ['type' => 'boolean', 'description' => 'Redact all environment variable values in logs. When false, only locked (shown once) values, database passwords, and remote secrets are redacted. Requires team admin permissions.'], + 'include_coolify_default_variables' => ['type' => 'boolean', 'description' => 'Add the predefined COOLIFY_* variables (COOLIFY_URL, COOLIFY_FQDN, ...) to the environment. Requires team admin permissions. Restart the database to apply it.'], + 'is_log_drain_enabled' => ['type' => 'boolean', 'description' => 'Send the database logs to the log drain of the server. Log drain must be enabled on the server. Restart the database to apply it.'], ], ), ) @@ -654,6 +687,8 @@ class DatabasesController extends Controller 'limits_cpu_shares' => 'numeric', 'instant_deploy' => 'boolean', 'redact_all_env_values_in_logs' => 'boolean', + 'include_coolify_default_variables' => 'boolean', + 'is_log_drain_enabled' => 'boolean', ]); if ($validator->fails()) { @@ -665,6 +700,8 @@ class DatabasesController extends Controller $uuid = $request->uuid; $instantDeploy = $request->boolean('instant_deploy'); $redactAllEnvValuesInLogs = $request->has('redact_all_env_values_in_logs') ? $request->boolean('redact_all_env_values_in_logs') : null; + $isLogDrainEnabled = $request->has('is_log_drain_enabled') ? $request->boolean('is_log_drain_enabled') : null; + $includeCoolifyDefaultVariables = $request->has('include_coolify_default_variables') ? $request->boolean('include_coolify_default_variables') : null; removeUnnecessaryFieldsFromRequest($request); $database = queryDatabaseByUuidWithinTeam($uuid, $teamId); if (! $database) { @@ -678,10 +715,17 @@ class DatabasesController extends Controller $this->authorize('manage', $database); } - if ($redactAllEnvValuesInLogs !== null) { + if ($redactAllEnvValuesInLogs !== null || $includeCoolifyDefaultVariables !== null) { $this->authorize('manageEnvironment', $database); } + if ($isLogDrainEnabled && ! $database->destination->server->isLogDrainEnabled()) { + return response()->json([ + 'message' => 'Validation failed.', + 'errors' => ['is_log_drain_enabled' => ['Log drain is not enabled on the server. Enable it on the server first.']], + ], 422); + } + if ($request->has('is_public') && (bool) $request->is_public !== (bool) $database->is_public) { $forbidden = $this->deployAbilityForbiddenResponse($request, 'Making a database public or private starts or stops its proxy on the server, so it needs a token with the deploy permission.'); if ($forbidden) { @@ -935,6 +979,12 @@ class DatabasesController extends Controller if ($redactAllEnvValuesInLogs !== null) { $database->redact_all_env_values_in_logs = $redactAllEnvValuesInLogs; } + if ($isLogDrainEnabled !== null) { + $database->is_log_drain_enabled = $isLogDrainEnabled; + } + if ($includeCoolifyDefaultVariables !== null) { + $database->include_coolify_default_variables = $includeCoolifyDefaultVariables; + } $database->update($request->only($allowedFields)); // The Redis password is stored in the REDIS_PASSWORD variable, not in a column. if ($database instanceof StandaloneRedis && filled($request->redis_password)) { @@ -961,6 +1011,8 @@ class DatabasesController extends Controller 'changed_fields' => [ ...array_values(array_intersect($allowedFields, array_keys($request->all()))), ...($redactAllEnvValuesInLogs === null ? [] : ['redact_all_env_values_in_logs']), + ...($isLogDrainEnabled === null ? [] : ['is_log_drain_enabled']), + ...($includeCoolifyDefaultVariables === null ? [] : ['include_coolify_default_variables']), ], ]); @@ -969,6 +1021,14 @@ class DatabasesController extends Controller ]); } + private function disableLocalBackupWithoutS3Response(): JsonResponse + { + return response()->json([ + 'message' => 'Validation failed.', + 'errors' => ['disable_local_backup' => ['Local backups can only be disabled when save_s3 is true.']], + ], 422); + } + #[OA\Post( summary: 'Create Backup', description: 'Create a new scheduled backup configuration for a database', @@ -1001,6 +1061,7 @@ class DatabasesController extends Controller 'frequency' => ['type' => 'string', 'description' => 'Backup frequency (cron expression or: every_minute, hourly, daily, weekly, monthly, yearly)'], 'enabled' => ['type' => 'boolean', 'description' => 'Whether the backup is enabled', 'default' => true], 'save_s3' => ['type' => 'boolean', 'description' => 'Whether to save backups to S3', 'default' => false], + 'disable_local_backup' => ['type' => 'boolean', 'description' => 'Delete the local backup file after the upload to S3. Only allowed when save_s3 is true.', 'default' => false], 's3_storage_uuid' => ['type' => 'string', 'description' => 'Primary S3 storage UUID. Sent without s3_storage_uuids, it selects this storage as the only destination. Either field is required if save_s3 is true.'], 's3_storage_uuids' => ['type' => 'array', 'items' => ['type' => 'string'], 'description' => 'UUIDs of every S3 storage the backup is uploaded to. Replaces the current destinations. When s3_storage_uuid is also sent, it must be in this list and becomes the primary destination; otherwise the first entry is the primary destination.'], 'databases_to_backup' => ['type' => 'string', 'description' => 'Comma separated list of databases to backup'], @@ -1054,7 +1115,7 @@ class DatabasesController extends Controller )] public function create_backup(Request $request) { - $backupConfigFields = ['save_s3', 'enabled', 'dump_all', 'frequency', 'databases_to_backup', 'database_backup_retention_amount_locally', 'database_backup_retention_days_locally', 'database_backup_retention_max_storage_locally', 'database_backup_retention_amount_s3', 'database_backup_retention_days_s3', 'database_backup_retention_max_storage_s3', 's3_storage_uuid', 's3_storage_uuids', 'timeout', 'missing_backup_notification_days']; + $backupConfigFields = ['save_s3', 'enabled', 'dump_all', 'frequency', 'databases_to_backup', 'database_backup_retention_amount_locally', 'database_backup_retention_days_locally', 'database_backup_retention_max_storage_locally', 'database_backup_retention_amount_s3', 'database_backup_retention_days_s3', 'database_backup_retention_max_storage_s3', 's3_storage_uuid', 's3_storage_uuids', 'timeout', 'missing_backup_notification_days', 'disable_local_backup']; $teamId = getTeamIdFromToken(); if (is_null($teamId)) { @@ -1071,6 +1132,7 @@ class DatabasesController extends Controller 'frequency' => 'required|string', 'enabled' => 'boolean', 'save_s3' => 'boolean', + 'disable_local_backup' => 'boolean', 'dump_all' => 'boolean', 'backup_now' => 'boolean|nullable', 's3_storage_uuid' => 'string|exists:s3_storages,uuid|nullable', @@ -1138,6 +1200,9 @@ class DatabasesController extends Controller 'errors' => ['s3_storage_uuid' => ['The s3_storage_uuid or s3_storage_uuids field is required when save_s3 is true.']], ], 422); } + if ($request->boolean('disable_local_backup') && ! $request->boolean('save_s3')) { + return $this->disableLocalBackupWithoutS3Response(); + } // Check for extra fields $extraFields = array_diff(array_keys($request->all()), $backupConfigFields, ['backup_now']); @@ -1256,6 +1321,7 @@ class DatabasesController extends Controller type: 'object', properties: [ 'save_s3' => ['type' => 'boolean', 'description' => 'Whether data is saved in s3 or not'], + 'disable_local_backup' => ['type' => 'boolean', 'description' => 'Delete the local backup file after the upload to S3. Only allowed when save_s3 is true. Turning save_s3 off also turns this off.'], 's3_storage_uuid' => ['type' => 'string', 'description' => 'Primary S3 storage UUID. Sent without s3_storage_uuids, it selects this storage as the only destination; when it is already a destination, it becomes the primary destination and the other destinations stay.'], 's3_storage_uuids' => ['type' => 'array', 'items' => ['type' => 'string'], 'description' => 'UUIDs of every S3 storage the backup is uploaded to. Replaces the current destinations. When s3_storage_uuid is also sent, it must be in this list and becomes the primary destination; otherwise the first entry is the primary destination.'], 'backup_now' => ['type' => 'boolean', 'description' => 'Whether to take a backup now or not. Needs a token with the deploy permission.'], @@ -1304,7 +1370,7 @@ class DatabasesController extends Controller )] public function update_backup(Request $request) { - $backupConfigFields = ['save_s3', 'enabled', 'dump_all', 'frequency', 'databases_to_backup', 'database_backup_retention_amount_locally', 'database_backup_retention_days_locally', 'database_backup_retention_max_storage_locally', 'database_backup_retention_amount_s3', 'database_backup_retention_days_s3', 'database_backup_retention_max_storage_s3', 's3_storage_uuid', 's3_storage_uuids', 'timeout', 'missing_backup_notification_days']; + $backupConfigFields = ['save_s3', 'enabled', 'dump_all', 'frequency', 'databases_to_backup', 'database_backup_retention_amount_locally', 'database_backup_retention_days_locally', 'database_backup_retention_max_storage_locally', 'database_backup_retention_amount_s3', 'database_backup_retention_days_s3', 'database_backup_retention_max_storage_s3', 's3_storage_uuid', 's3_storage_uuids', 'timeout', 'missing_backup_notification_days', 'disable_local_backup']; $teamId = getTeamIdFromToken(); if (is_null($teamId)) { @@ -1317,6 +1383,7 @@ class DatabasesController extends Controller } $validator = customApiValidator($request->all(), [ 'save_s3' => 'boolean', + 'disable_local_backup' => 'boolean', 'backup_now' => 'boolean|nullable', 'enabled' => 'boolean', 'dump_all' => 'boolean', @@ -1382,7 +1449,7 @@ class DatabasesController extends Controller return $s3Storages; } - $backupConfig = ScheduledDatabaseBackup::ownedByCurrentTeamAPI($teamId)->where('database_id', $database->id) + $backupConfig = ScheduledDatabaseBackup::ownedByCurrentTeamAPI($teamId)->where('database_id', $database->id)->where('database_type', $database->getMorphClass()) ->where('uuid', $request->scheduled_backup_uuid) ->first(); if (! $backupConfig) { @@ -1409,11 +1476,18 @@ class DatabasesController extends Controller 'errors' => $errors, ], 422); } + if ($request->boolean('disable_local_backup') && ! $saveToS3) { + return $this->disableLocalBackupWithoutS3Response(); + } $backupData = Arr::except($request->only($backupConfigFields), ['s3_storage_uuid', 's3_storage_uuids']); if ($s3Storages !== null) { $backupData['s3_storage_id'] = $s3Storages->first()?->id; } + // Without S3 the backup must stay on the server, as in the UI. + if (! $saveToS3) { + $backupData['disable_local_backup'] = false; + } // Validate databases_to_backup input if (! empty($backupData['databases_to_backup'])) { @@ -2274,13 +2348,7 @@ class DatabasesController extends Controller $this->attachTagsToResource($database, $tagNames, $teamId); } $database->refresh(); - $payload = [ - 'uuid' => $database->uuid, - 'internal_db_url' => $database->internal_db_url, - ]; - if ($database->is_public && $database->public_port) { - $payload['external_db_url'] = $database->external_db_url; - } + $payload = $this->createdDatabasePayload($request, $database); auditLog('api.database.created', [ 'team_id' => $teamId, @@ -2346,13 +2414,7 @@ class DatabasesController extends Controller } $database->refresh(); - $payload = [ - 'uuid' => $database->uuid, - 'internal_db_url' => $database->internal_db_url, - ]; - if ($database->is_public && $database->public_port) { - $payload['external_db_url'] = $database->external_db_url; - } + $payload = $this->createdDatabasePayload($request, $database); auditLog('api.database.created', [ 'team_id' => $teamId, @@ -2418,13 +2480,7 @@ class DatabasesController extends Controller } $database->refresh(); - $payload = [ - 'uuid' => $database->uuid, - 'internal_db_url' => $database->internal_db_url, - ]; - if ($database->is_public && $database->public_port) { - $payload['external_db_url'] = $database->external_db_url; - } + $payload = $this->createdDatabasePayload($request, $database); auditLog('api.database.created', [ 'team_id' => $teamId, @@ -2487,13 +2543,7 @@ class DatabasesController extends Controller } $database->refresh(); - $payload = [ - 'uuid' => $database->uuid, - 'internal_db_url' => $database->internal_db_url, - ]; - if ($database->is_public && $database->public_port) { - $payload['external_db_url'] = $database->external_db_url; - } + $payload = $this->createdDatabasePayload($request, $database); auditLog('api.database.created', [ 'team_id' => $teamId, @@ -2589,13 +2639,7 @@ class DatabasesController extends Controller } $database->refresh(); - $payload = [ - 'uuid' => $database->uuid, - 'internal_db_url' => $database->internal_db_url, - ]; - if ($database->is_public && $database->public_port) { - $payload['external_db_url'] = $database->external_db_url; - } + $payload = $this->createdDatabasePayload($request, $database); auditLog('api.database.created', [ 'team_id' => $teamId, @@ -2638,13 +2682,7 @@ class DatabasesController extends Controller } $database->refresh(); - $payload = [ - 'uuid' => $database->uuid, - 'internal_db_url' => $database->internal_db_url, - ]; - if ($database->is_public && $database->public_port) { - $payload['external_db_url'] = $database->external_db_url; - } + $payload = $this->createdDatabasePayload($request, $database); auditLog('api.database.created', [ 'team_id' => $teamId, @@ -2709,13 +2747,7 @@ class DatabasesController extends Controller } $database->refresh(); - $payload = [ - 'uuid' => $database->uuid, - 'internal_db_url' => $database->internal_db_url, - ]; - if ($database->is_public && $database->public_port) { - $payload['external_db_url'] = $database->external_db_url; - } + $payload = $this->createdDatabasePayload($request, $database); auditLog('api.database.created', [ 'team_id' => $teamId, @@ -3060,7 +3092,7 @@ class DatabasesController extends Controller $this->authorize('update', $database); // Find the backup configuration by its UUID - $backup = ScheduledDatabaseBackup::ownedByCurrentTeamAPI($teamId)->where('database_id', $database->id) + $backup = ScheduledDatabaseBackup::ownedByCurrentTeamAPI($teamId)->where('database_id', $database->id)->where('database_type', $database->getMorphClass()) ->where('uuid', $request->scheduled_backup_uuid) ->first(); @@ -3196,7 +3228,7 @@ class DatabasesController extends Controller $this->authorize('update', $database); // Find the backup configuration by its UUID - $backup = ScheduledDatabaseBackup::ownedByCurrentTeamAPI($teamId)->where('database_id', $database->id) + $backup = ScheduledDatabaseBackup::ownedByCurrentTeamAPI($teamId)->where('database_id', $database->id)->where('database_type', $database->getMorphClass()) ->where('uuid', $request->scheduled_backup_uuid) ->first(); @@ -3314,7 +3346,7 @@ class DatabasesController extends Controller } // Find the backup configuration by its UUID - $backup = ScheduledDatabaseBackup::ownedByCurrentTeamAPI($teamId)->where('database_id', $database->id) + $backup = ScheduledDatabaseBackup::ownedByCurrentTeamAPI($teamId)->where('database_id', $database->id)->where('database_type', $database->getMorphClass()) ->where('uuid', $request->scheduled_backup_uuid) ->first(); diff --git a/app/Http/Controllers/Api/DestinationsController.php b/app/Http/Controllers/Api/DestinationsController.php index 9991df2793..a7d0c3c45e 100644 --- a/app/Http/Controllers/Api/DestinationsController.php +++ b/app/Http/Controllers/Api/DestinationsController.php @@ -284,6 +284,10 @@ class DestinationsController extends Controller private function existingServerNetworkResponse(Server $server, string $network): ?JsonResponse { + if (! $server->isFunctional()) { + return response()->json(['message' => 'Server is not functional.'], 400); + } + try { $output = instant_remote_process(["docker network ls --format '{{.Name}}'"], $server); } catch (\Throwable) { diff --git a/app/Http/Controllers/Api/ProjectCanvasController.php b/app/Http/Controllers/Api/ProjectCanvasController.php index 534cd7f56a..7dd5e02b63 100644 --- a/app/Http/Controllers/Api/ProjectCanvasController.php +++ b/app/Http/Controllers/Api/ProjectCanvasController.php @@ -4,16 +4,20 @@ namespace App\Http\Controllers\Api; use App\Http\Controllers\Controller; use App\Models\Project; +use App\Services\EnvironmentCanvas; use App\Services\ProjectCanvas; use Illuminate\Http\JsonResponse; use Illuminate\Http\Request; +use Illuminate\Support\Facades\Validator; /** * Project canvas: the resources of every environment of one project (or of every project). - * Moves and links are saved through EnvironmentCanvasController. Called by resources/js/environment-canvas.js through the /canvas/api session bridge. + * Moves of cards and links are saved through EnvironmentCanvasController; the projects canvas saves the place of a project here. Called by resources/js/environment-canvas.js through the /canvas/api session bridge. */ class ProjectCanvasController extends Controller { + private const MAX_POSITIONS = 1000; + public function __construct(private ProjectCanvas $canvas) {} public function index(Request $request): JsonResponse @@ -41,6 +45,50 @@ class ProjectCanvasController extends Controller return response()->json($this->canvas->payload($project, (bool) $request->attributes->get('can_read_sensitive', false), $this->canEdit($request, (int) $project->team_id))); } + /** + * Saves the place of project zones on the projects canvas (top-left of each zone), keyed by project UUID. + */ + public function updatePositions(Request $request): JsonResponse + { + $teamId = getTeamIdFromToken(); + if (is_null($teamId)) { + return invalidTokenResponse(); + } + + $return = validateIncomingRequest($request); + if ($return instanceof JsonResponse) { + return $return; + } + $coordinateRule = ['required', 'integer', 'between:-'.EnvironmentCanvas::MAX_COORDINATE.','.EnvironmentCanvas::MAX_COORDINATE]; + $validator = Validator::make($request->all(), [ + 'positions' => ['required', 'array', 'max:'.self::MAX_POSITIONS], + 'positions.*' => ['required', 'array'], + 'positions.*.x' => $coordinateRule, + 'positions.*.y' => $coordinateRule, + ]); + if ($validator->fails()) { + return response()->json([ + 'message' => 'Validation failed.', + 'errors' => $validator->errors(), + ], 422); + } + + // Projects of other teams are left out, like unknown UUIDs. + $projects = Project::query()->whereTeamId($teamId)->whereIn('uuid', array_keys($request->input('positions')))->get(); + foreach ($projects as $project) { + $this->authorize('update', $project); + } + + $saved = []; + foreach ($projects as $project) { + $position = $request->input("positions.{$project->uuid}"); + $project->canvas_position = $saved[$project->uuid] = ['x' => (int) $position['x'], 'y' => (int) $position['y']]; + $project->save(); + } + + return response()->json(['positions' => (object) $saved]); + } + /** * Same rule as the environment canvas: team admins with a write token move cards and draw links. */ diff --git a/app/Http/Controllers/Api/ServerCanvasController.php b/app/Http/Controllers/Api/ServerCanvasController.php index 79c33553ad..7f1d152af8 100644 --- a/app/Http/Controllers/Api/ServerCanvasController.php +++ b/app/Http/Controllers/Api/ServerCanvasController.php @@ -4,16 +4,20 @@ namespace App\Http\Controllers\Api; use App\Http\Controllers\Controller; use App\Models\Server; +use App\Services\EnvironmentCanvas; use App\Services\ServerCanvas; use Illuminate\Http\JsonResponse; use Illuminate\Http\Request; +use Illuminate\Support\Facades\Validator; /** * Server canvas: every server of the team (or one server) with the resources on it. - * Moves and links are saved through EnvironmentCanvasController. Called by resources/js/environment-canvas.js through the /canvas/api session bridge. + * Moves of cards and links are saved through EnvironmentCanvasController; the servers and dashboard canvases save the place of a server here. Called by resources/js/environment-canvas.js through the /canvas/api session bridge. */ class ServerCanvasController extends Controller { + private const MAX_POSITIONS = 1000; + public function __construct(private ServerCanvas $canvas) {} public function index(Request $request): JsonResponse @@ -60,6 +64,50 @@ class ServerCanvasController extends Controller return response()->json($this->canvas->payload($teamId, collect([$server]), $server, $this->canReadSensitive($request), $this->canEdit($request, $teamId))); } + /** + * Saves the place of server zones on the servers and dashboard canvases (top-left of each zone), keyed by server UUID. + */ + public function updatePositions(Request $request): JsonResponse + { + $teamId = getTeamIdFromToken(); + if (is_null($teamId)) { + return invalidTokenResponse(); + } + + $return = validateIncomingRequest($request); + if ($return instanceof JsonResponse) { + return $return; + } + $coordinateRule = ['required', 'integer', 'between:-'.EnvironmentCanvas::MAX_COORDINATE.','.EnvironmentCanvas::MAX_COORDINATE]; + $validator = Validator::make($request->all(), [ + 'positions' => ['required', 'array', 'max:'.self::MAX_POSITIONS], + 'positions.*' => ['required', 'array'], + 'positions.*.x' => $coordinateRule, + 'positions.*.y' => $coordinateRule, + ]); + if ($validator->fails()) { + return response()->json([ + 'message' => 'Validation failed.', + 'errors' => $validator->errors(), + ], 422); + } + + // Servers of other teams are left out, like unknown UUIDs (for example the zone of resources without a server). + $servers = Server::query()->whereTeamId($teamId)->whereIn('uuid', array_keys($request->input('positions')))->get(); + foreach ($servers as $server) { + $this->authorize('update', $server); + } + + $saved = []; + foreach ($servers as $server) { + $position = $request->input("positions.{$server->uuid}"); + $server->canvas_position = $saved[$server->uuid] = ['x' => (int) $position['x'], 'y' => (int) $position['y']]; + $server->save(); + } + + return response()->json(['positions' => (object) $saved]); + } + /** * Same rule as the environment canvas: team admins with a write token move cards and draw links. */ diff --git a/app/Http/Controllers/Api/ServerLogDrainsController.php b/app/Http/Controllers/Api/ServerLogDrainsController.php index af5aaa1dfd..a5da9edb89 100644 --- a/app/Http/Controllers/Api/ServerLogDrainsController.php +++ b/app/Http/Controllers/Api/ServerLogDrainsController.php @@ -274,10 +274,12 @@ class ServerLogDrainsController extends Controller $server->refresh(); // Match Livewire instantSave: start or stop the drain service after settings change. - if ($server->isLogDrainEnabled()) { - StartLogDrain::dispatch($server); - } else { - StopLogDrain::dispatch($server); + if ($server->isFunctional()) { + if ($server->isLogDrainEnabled()) { + StartLogDrain::dispatch($server); + } else { + StopLogDrain::dispatch($server); + } } auditLog('api.server.log_drains.updated', [ diff --git a/app/Http/Controllers/Api/ServerProxyController.php b/app/Http/Controllers/Api/ServerProxyController.php index 9861831123..e7964f68e1 100644 --- a/app/Http/Controllers/Api/ServerProxyController.php +++ b/app/Http/Controllers/Api/ServerProxyController.php @@ -56,6 +56,7 @@ class ServerProxyController extends Controller 'status' => data_get($server->proxy, 'status'), 'redirect_enabled' => (bool) data_get($server->proxy, 'redirect_enabled', true), 'redirect_url' => data_get($server->proxy, 'redirect_url'), + 'proxy_error_page' => $server->proxy_error_page, 'generate_exact_labels' => (bool) ($server->settings->generate_exact_labels ?? false), 'traefik_certificate_resolver' => $server->traefikCertificateResolver(), ]; @@ -91,6 +92,7 @@ class ServerProxyController extends Controller new OA\Property(property: 'status', type: 'string', nullable: true, example: 'running'), new OA\Property(property: 'redirect_enabled', type: 'boolean', example: true), new OA\Property(property: 'redirect_url', type: 'string', nullable: true, example: 'https://example.com'), + new OA\Property(property: 'proxy_error_page', type: 'string', nullable: true), new OA\Property(property: 'generate_exact_labels', type: 'boolean', example: false), new OA\Property(property: 'traefik_certificate_resolver', type: 'string', nullable: true, example: 'letsencrypt', description: 'Traefik certificate resolver for generated HTTPS routers. Null when the proxy uses its own certificates.'), new OA\Property(property: 'configuration', type: 'string', nullable: true, description: 'Docker Compose proxy configuration when stored in the database. Only present with read:sensitive.'), @@ -136,6 +138,7 @@ class ServerProxyController extends Controller properties: [ new OA\Property(property: 'redirect_enabled', type: 'boolean'), new OA\Property(property: 'redirect_url', type: 'string', nullable: true, description: 'Public http(s) redirect URL, or null to clear.'), + new OA\Property(property: 'proxy_error_page', type: 'string', nullable: true, description: 'Custom HTML error page that the proxy shows for unknown hosts (when redirect_url is empty) and for resources in maintenance mode. Null restores the default page.'), new OA\Property(property: 'generate_exact_labels', type: 'boolean'), new OA\Property(property: 'traefik_certificate_resolver', type: 'string', nullable: true, description: 'Traefik certificate resolver for generated HTTPS routers (letters, digits, _ and -). Null or empty to use your own certificates. Redeploy resources to apply.'), new OA\Property(property: 'proxy_type', type: 'string', enum: ['traefik', 'caddy', 'nginx', 'none'], description: 'Proxy type (case-insensitive). Changing it starts the new proxy on the server and needs the deploy permission.'), @@ -153,6 +156,7 @@ class ServerProxyController extends Controller new OA\Property(property: 'status', type: 'string', nullable: true), new OA\Property(property: 'redirect_enabled', type: 'boolean'), new OA\Property(property: 'redirect_url', type: 'string', nullable: true), + new OA\Property(property: 'proxy_error_page', type: 'string', nullable: true), new OA\Property(property: 'generate_exact_labels', type: 'boolean'), new OA\Property(property: 'traefik_certificate_resolver', type: 'string', nullable: true), new OA\Property(property: 'configuration', type: 'string', nullable: true), @@ -178,10 +182,11 @@ class ServerProxyController extends Controller return $return; } - $allowedFields = ['redirect_enabled', 'redirect_url', 'generate_exact_labels', 'traefik_certificate_resolver', 'proxy_type']; + $allowedFields = ['redirect_enabled', 'redirect_url', 'proxy_error_page', 'generate_exact_labels', 'traefik_certificate_resolver', 'proxy_type']; $validator = customApiValidator($request->all(), [ 'redirect_enabled' => 'boolean', 'redirect_url' => ['nullable', 'string', new SafeExternalUrl], + 'proxy_error_page' => ['nullable', 'string', proxyPageSizeRule('custom error page')], 'generate_exact_labels' => 'boolean', 'traefik_certificate_resolver' => ['nullable', 'string', 'max:255', 'regex:'.ServerSetting::TRAEFIK_CERTIFICATE_RESOLVER_PATTERN], 'proxy_type' => 'string|nullable', @@ -236,6 +241,11 @@ class ServerProxyController extends Controller $redirectChanged = true; } + if ($request->exists('proxy_error_page')) { + $server->proxy_error_page = $request->input('proxy_error_page') ?: null; + $redirectChanged = true; + } + if ($redirectChanged) { $server->save(); } @@ -314,6 +324,7 @@ class ServerProxyController extends Controller new OA\Property(property: 'status', type: 'string', nullable: true), new OA\Property(property: 'redirect_enabled', type: 'boolean'), new OA\Property(property: 'redirect_url', type: 'string', nullable: true), + new OA\Property(property: 'proxy_error_page', type: 'string', nullable: true), new OA\Property(property: 'generate_exact_labels', type: 'boolean'), new OA\Property(property: 'configuration', type: 'string', nullable: true), ] @@ -389,6 +400,10 @@ class ServerProxyController extends Controller ], 422); } + if (! $server->isFunctional()) { + return response()->json(['message' => 'Server is not functional.'], 400); + } + SaveProxyConfiguration::run($server, $configuration); auditLog('api.server.proxy.configuration_saved', [ @@ -443,6 +458,10 @@ class ServerProxyController extends Controller $this->authorize('manageProxy', $server); + if (! $server->isFunctional()) { + return response()->json(['message' => 'Server is not functional.'], 400); + } + RestartProxyJob::dispatch($server); auditLog('api.server.proxy.restarted', [ diff --git a/app/Http/Controllers/Api/ServerTransferController.php b/app/Http/Controllers/Api/ServerTransferController.php index 6179cb350a..efd8f428b7 100644 --- a/app/Http/Controllers/Api/ServerTransferController.php +++ b/app/Http/Controllers/Api/ServerTransferController.php @@ -6,13 +6,16 @@ use App\Http\Controllers\Controller; use App\Models\GithubApp; use App\Models\GitlabApp; use App\Models\Server; +use App\Rules\SafeWebhookUrl; use App\Rules\ValidServerIp; use App\Services\ServerTransfer\ServerTransferBundle; use App\Services\ServerTransfer\ServerTransferClaimer; use App\Services\ServerTransfer\ServerTransferImporter; use App\Services\ServerTransfer\ServerTransferMigrator; +use Illuminate\Http\Exceptions\ThrottleRequestsException; use Illuminate\Http\JsonResponse; use Illuminate\Http\Request; +use Illuminate\Support\Facades\Validator; use Illuminate\Validation\ValidationException; use OpenApi\Attributes as OA; use Throwable; @@ -27,7 +30,7 @@ class ServerTransferController extends Controller #[OA\Post( summary: 'Migrate server to another Coolify instance', - description: 'Start a transfer in the background: export this server, import and claim it on the target instance (with the given token), then disable automations here. Follow the progress with GET /servers/{uuid}/transfer. The target URL must use https. Requires read:sensitive and write.', + description: 'Start a transfer in the background: export this server, import and claim it on the target instance (with the given token), then disable automations here. All secrets of the server, also locked environment variables, are sent to the target. Follow the progress with GET /servers/{uuid}/transfer. The target URL must use https. Requires read:sensitive and write, and the owner role of the server team.', path: '/servers/{uuid}/migrate', operationId: 'migrate-server-between-instances', security: [['bearerAuth' => []]], @@ -47,7 +50,7 @@ class ServerTransferController extends Controller ), responses: [ new OA\Response(response: 202, description: 'Transfer started'), - new OA\Response(response: 403, description: 'Missing sensitive permission'), + new OA\Response(response: 403, description: 'Missing sensitive permission or not an owner of the server team'), new OA\Response(response: 404, ref: '#/components/responses/404'), new OA\Response(response: 422, description: 'Validation or remote import failed'), ] @@ -61,7 +64,7 @@ class ServerTransferController extends Controller if (! $this->canReadSensitive($request)) { return response()->json([ - 'message' => 'Migrating a server requires a token with read:sensitive (or root) ability and an admin/owner team role.', + 'message' => 'Migrating a server requires a token with read:sensitive (or root) ability and the owner team role.', ], 403); } @@ -70,7 +73,7 @@ class ServerTransferController extends Controller return response()->json(['message' => 'Server not found.'], 404); } - $this->authorize('update', $server); + $this->authorize('transfer', $server); $return = validateIncomingRequest($request); if ($return instanceof JsonResponse) { @@ -135,7 +138,7 @@ class ServerTransferController extends Controller new OA\Response(response: 404, ref: '#/components/responses/404'), ] )] - public function status(string $uuid): JsonResponse + public function status(Request $request, string $uuid): JsonResponse { $teamId = getTeamIdFromToken(); if (is_null($teamId)) { @@ -149,12 +152,18 @@ class ServerTransferController extends Controller $this->authorize('view', $server); + // The log and the error of a transfer can name hosts and credentials problems, so members see only the state. + $migration = data_get($server->server_metadata, 'migration'); + if (is_array($migration) && ! $request->user()->can('update', $server)) { + $migration = collect($migration)->except(['log', 'error'])->all(); + } + return response()->json([ 'server_uuid' => $server->uuid, 'transfer' => collect((array) data_get($server->server_metadata, 'transfer', [])) ->only(['status', 'export_id', 'imported_at', 'claimed_at', 'transferred_at', 'target_instance_url']) ->all(), - 'migration' => data_get($server->server_metadata, 'migration'), + 'migration' => $migration, ]); } @@ -217,7 +226,18 @@ class ServerTransferController extends Controller // The target connects to the server over SSH when it takes over management, so an unreachable // server would leave an imported server here that this instance cannot manage. - $connection = @fsockopen($ip, $port, timeout: 5); + // On Cloud the probe is rate limited and cannot reach private, loopback, or link-local addresses, + // so users cannot scan ports of Cloud's own network. + // A host name is resolved once and the probe connects to that address, so DNS cannot change it in between. + Server::throttleValidation($teamId, $ip); + $probeIp = Server::normalizeIp($ip); + if (isCloud() && ! filter_var($probeIp, FILTER_VALIDATE_IP)) { + $probeIp = gethostbyname($probeIp); + } + $probeHost = filter_var($probeIp, FILTER_VALIDATE_IP, FILTER_FLAG_IPV6) ? "[{$probeIp}]" : $probeIp; + $probeAllowed = ! isCloud() || (filter_var($probeIp, FILTER_VALIDATE_IP) + && Validator::make(['ip' => "http://{$probeHost}:{$port}"], ['ip' => [new SafeWebhookUrl]])->passes()); + $connection = $probeAllowed ? @fsockopen($probeHost, $port, timeout: 5) : false; if ($connection !== false) { fclose($connection); } @@ -235,8 +255,11 @@ class ServerTransferController extends Controller [ 'label' => "the target can reach the SSH port of the server ({$ip}:{$port})", 'ok' => $connection !== false, - 'message' => $connection !== false ? null - : "This instance cannot reach the server at {$ip}:{$port}. Allow SSH from this instance to the server, then transfer again.", + 'message' => match (true) { + $connection !== false => null, + ! $probeAllowed => "This instance cannot connect to {$ip}: it is a private, loopback, or link-local address, or it does not resolve to a public IPv4 address. Use the public IP address of the server.", + default => "This instance cannot reach the server at {$ip}:{$port}. Allow SSH from this instance to the server, then transfer again.", + }, ], ]; @@ -319,6 +342,8 @@ class ServerTransferController extends Controller preserveUuids: $request->boolean('preserve_uuids', true), adoptMode: $request->boolean('adopt_mode', true), ); + } catch (ThrottleRequestsException $e) { + throw $e; } catch (Throwable $e) { $payload = ['message' => $e->getMessage()]; if ($e instanceof ValidationException) { @@ -386,11 +411,23 @@ class ServerTransferController extends Controller ], 422); } + // The UI prefills this URL for a transfer back, which sends a team token to it. + $targetInstanceUrl = $request->input('target_instance_url'); + if (filled($targetInstanceUrl)) { + try { + $targetInstanceUrl = ServerTransferMigrator::validateInstanceUrl($targetInstanceUrl); + } catch (ValidationException $e) { + return response()->json(['message' => 'Validation failed.', 'errors' => ['target_instance_url' => $e->validator->errors()->all()]], 422); + } catch (Throwable $e) { + return response()->json(['message' => 'Validation failed.', 'errors' => ['target_instance_url' => [$e->getMessage()]]], 422); + } + } + try { $result = $this->claimer->markTransferred( $server, exportId: $request->input('export_id'), - targetInstanceUrl: $request->input('target_instance_url'), + targetInstanceUrl: filled($targetInstanceUrl) ? $targetInstanceUrl : null, ); } catch (Throwable $e) { return response()->json(['message' => $e->getMessage()], 422); diff --git a/app/Http/Controllers/Api/ServersController.php b/app/Http/Controllers/Api/ServersController.php index fec376109b..febd83a9ac 100644 --- a/app/Http/Controllers/Api/ServersController.php +++ b/app/Http/Controllers/Api/ServersController.php @@ -18,6 +18,7 @@ use App\Models\Project; use App\Models\Server as ModelsServer; use App\Models\Team; use App\Rules\ValidServerIp; +use App\Services\ServerTransfer\ServerTransferClaimer; use App\Support\ValidationPatterns; use Illuminate\Http\JsonResponse; use Illuminate\Http\Request; @@ -52,6 +53,27 @@ class ServersController extends Controller ->put('is_build_server', $settings->effectiveServerRole() === ServerRole::BUILD); } + /** + * Another instance manages a transferred server. Changing its host or credentials would let + * "Enable management" connect to a different host, so these fields stay as they are. + * + * @return array> + */ + private function transferredServerConnectionChanges(Request $request, ModelsServer $server): array + { + $changed = [ + 'ip' => $request->filled('ip') && ModelsServer::normalizeIp($request->input('ip')) !== ModelsServer::normalizeIp($server->getRawOriginal('ip')), + 'user' => $request->filled('user') && (string) $request->input('user') !== (string) $server->user, + 'port' => $request->filled('port') && (int) $request->input('port') !== (int) $server->port, + 'private_key_uuid' => $request->filled('private_key_uuid') && $request->input('private_key_uuid') !== $server->privateKey?->uuid, + ]; + + return collect($changed) + ->filter() + ->map(fn () => ['This field cannot be changed on a transferred server.']) + ->all(); + } + /** * Rejects requests where the deprecated is_build_server flag disagrees with server_role. */ @@ -602,6 +624,9 @@ class ServersController extends Controller if ($foundServer) { return response()->json(['message' => 'A server with this IP/Domain is already in use.'], 400); } + if ($request->instant_validate) { + ModelsServer::throttleValidation($teamId, $request->ip); + } $proxyType = $request->proxy_type ? str($request->proxy_type)->upper() : ProxyTypes::TRAEFIK->value; @@ -680,6 +705,9 @@ class ServersController extends Controller 'server_disk_usage_check_frequency' => ['type' => 'string', 'description' => 'Cron expression for disk usage check frequency.'], 'server_disk_usage_notification_interval_hours' => ['type' => 'integer', 'description' => 'Minimum hours between high disk usage notifications (1-720). Default: 24.'], 'connection_timeout' => ['type' => 'integer', 'description' => 'SSH connection timeout in seconds (1-300). Default: 10.'], + 'wildcard_domain' => ['type' => 'string', 'nullable' => true, 'description' => 'Wildcard domain (URL) used to generate domains for new resources on this server. Null removes it.'], + 'server_timezone' => ['type' => 'string', 'description' => 'Timezone of the server, for example Europe/Berlin. Scheduled tasks and backups use it.'], + 'backup_compression_cpu_percentage' => ['type' => 'integer', 'enum' => [25, 50, 75, 100], 'description' => 'Maximum CPU percentage that backup compression can use. Default: 25.'], ], ), ), @@ -717,7 +745,7 @@ class ServersController extends Controller )] public function update_server(Request $request) { - $allowedFields = ['name', 'description', 'ip', 'port', 'user', 'private_key_uuid', 'server_role', 'is_build_server', 'instant_validate', 'proxy_type', 'concurrent_builds', 'dynamic_timeout', 'deployment_queue_limit', 'server_disk_usage_notification_threshold', 'server_disk_usage_check_frequency', 'server_disk_usage_notification_interval_hours', 'connection_timeout', 'is_terminal_enabled']; + $allowedFields = ['name', 'description', 'ip', 'port', 'user', 'private_key_uuid', 'server_role', 'is_build_server', 'instant_validate', 'proxy_type', 'concurrent_builds', 'dynamic_timeout', 'deployment_queue_limit', 'server_disk_usage_notification_threshold', 'server_disk_usage_check_frequency', 'server_disk_usage_notification_interval_hours', 'connection_timeout', 'is_terminal_enabled', 'wildcard_domain', 'server_timezone', 'backup_compression_cpu_percentage']; $teamId = getTeamIdFromToken(); if (is_null($teamId)) { @@ -747,6 +775,13 @@ class ServersController extends Controller 'server_disk_usage_notification_interval_hours' => 'integer|min:1|max:720', 'connection_timeout' => 'integer|min:1|max:300', 'is_terminal_enabled' => 'boolean|nullable', + 'wildcard_domain' => 'nullable|url', + 'server_timezone' => ['string', function (string $attribute, mixed $value, \Closure $fail): void { + if (! validate_timezone($value)) { + $fail('The server_timezone must be a valid timezone.'); + } + }], + 'backup_compression_cpu_percentage' => 'integer|in:25,50,75,100', ], [ ...ValidationPatterns::serverUsernameMessages(), 'name.not_regex' => 'The name must not contain control characters such as new lines.', @@ -771,6 +806,15 @@ class ServersController extends Controller return response()->json(['message' => 'Server not found.'], 404); } $this->authorize('update', $server); + if ($server->isTransferredAway() && ($connectionErrors = $this->transferredServerConnectionChanges($request, $server))) { + return response()->json([ + 'message' => 'Validation failed.', + 'errors' => $connectionErrors, + ], 422); + } + if ($request->instant_validate) { + ModelsServer::throttleValidation($teamId, $request->ip ?? $server->ip); + } if ($request->proxy_type) { $validProxyTypes = collect(ProxyTypes::cases())->map(function ($proxyType) { return str($proxyType->value)->lower(); @@ -852,10 +896,13 @@ class ServersController extends Controller ]); } - $advancedSettings = $request->only(['concurrent_builds', 'dynamic_timeout', 'deployment_queue_limit', 'server_disk_usage_notification_threshold', 'server_disk_usage_check_frequency', 'server_disk_usage_notification_interval_hours', 'connection_timeout']); + $advancedSettings = $request->only(['concurrent_builds', 'dynamic_timeout', 'deployment_queue_limit', 'server_disk_usage_notification_threshold', 'server_disk_usage_check_frequency', 'server_disk_usage_notification_interval_hours', 'connection_timeout', 'server_timezone', 'backup_compression_cpu_percentage']); if (! empty($advancedSettings)) { $server->settings()->update(array_filter($advancedSettings, fn ($value) => ! is_null($value))); } + if ($request->has('wildcard_domain')) { + $server->settings()->update(['wildcard_domain' => $request->input('wildcard_domain')]); + } if ($proxyTypeChanged) { $server->changeProxy($request->proxy_type, async: true); @@ -967,6 +1014,17 @@ class ServersController extends Controller if (is_null($deleteFromProvider)) { return response()->json(['message' => 'delete_from_provider must be a boolean.'], 422); } + // Another Coolify instance manages a transferred server. This instance deletes only its own records: + // never the cloud server, the containers, or the DNS records that the other instance uses. + if ($server->isTransferredAway()) { + if ($deleteFromProvider) { + return response()->json(['message' => 'This server is managed by another Coolify instance. Delete its cloud server from that instance.'], 422); + } + app(ServerTransferClaimer::class)->deleteArchive($server); + auditLog('api.server.deleted', ['team_id' => $teamId, 'server_uuid' => $server->uuid, 'server_name' => $server->name, 'ip' => $server->ip, 'archive' => true]); + + return response()->json(['message' => 'Server deleted.']); + } if ($server->definedResources()->count() > 0 && ! $force) { return response()->json(['message' => 'Server has resources. Use ?force=true to delete all resources and the server, or delete resources manually first.'], 400); @@ -1125,6 +1183,7 @@ class ServersController extends Controller if ($install && ($response = $this->deployAbilityForbiddenResponse($request, 'Installing the server runs commands on it and starts the proxy, so it needs a token with the deploy permission.'))) { return $response; } + ModelsServer::throttleValidation($teamId, $server->ip); if ($install) { ValidateAndInstallServerJob::dispatch($server); } else { diff --git a/app/Http/Controllers/Api/ServiceApplicationsController.php b/app/Http/Controllers/Api/ServiceApplicationsController.php index 14d8b97924..f6147855aa 100644 --- a/app/Http/Controllers/Api/ServiceApplicationsController.php +++ b/app/Http/Controllers/Api/ServiceApplicationsController.php @@ -6,6 +6,7 @@ use App\Actions\Service\DeployServiceApplication; use App\Actions\Service\RestartServiceApplication; use App\Actions\Service\StopServiceApplication; use App\Actions\Service\UpdateServiceApplicationFromApi; +use App\Enums\RedirectTypes; use App\Http\Controllers\Controller; use App\Models\EnvironmentVariable; use App\Models\ServerSetting; @@ -16,6 +17,7 @@ use Illuminate\Http\JsonResponse; use Illuminate\Http\Request; use Illuminate\Support\Collection; use Illuminate\Support\Facades\Validator; +use Illuminate\Validation\Rule; use OpenApi\Attributes as OA; class ServiceApplicationsController extends Controller @@ -263,6 +265,7 @@ class ServiceApplicationsController extends Controller 'is_stripprefix_enabled' => new OA\Property(property: 'is_stripprefix_enabled', type: 'boolean', nullable: true), 'is_force_https_enabled' => new OA\Property(property: 'is_force_https_enabled', type: 'boolean', nullable: true), 'is_permanent_redirect_enabled' => new OA\Property(property: 'is_permanent_redirect_enabled', type: 'boolean', nullable: true, description: 'Use permanent redirects (301/308) instead of temporary (302/307) for the HTTP to HTTPS and www redirects. Defaults to false.'), + 'redirect' => new OA\Property(property: 'redirect', type: 'string', enum: ['both', 'www', 'non-www'], description: 'How to handle www and non-www domains. "www" redirects non-www to www, "non-www" redirects www to non-www. Both need a www and a non-www domain in url.'), 'max_restart_count' => new OA\Property(property: 'max_restart_count', type: 'integer', minimum: 0, nullable: true, description: 'Maximum Docker restart count before Coolify stops the container. Set to 0 to disable the limit.'), ] ) @@ -339,6 +342,7 @@ class ServiceApplicationsController extends Controller 'is_stripprefix_enabled', 'is_force_https_enabled', 'is_permanent_redirect_enabled', + 'redirect', 'max_restart_count', ]; @@ -356,6 +360,7 @@ class ServiceApplicationsController extends Controller 'is_stripprefix_enabled' => 'sometimes|boolean', 'is_force_https_enabled' => 'sometimes|boolean', 'is_permanent_redirect_enabled' => 'sometimes|boolean', + 'redirect' => ['sometimes', Rule::enum(RedirectTypes::class)], 'max_restart_count' => 'sometimes|integer|min:0', ]; diff --git a/app/Http/Controllers/Api/ServicesController.php b/app/Http/Controllers/Api/ServicesController.php index 005f0e4d43..3bbf9879a1 100644 --- a/app/Http/Controllers/Api/ServicesController.php +++ b/app/Http/Controllers/Api/ServicesController.php @@ -7,7 +7,6 @@ use App\Actions\Service\StartService; use App\Actions\Service\StopService; use App\Http\Controllers\Controller; use App\Jobs\DeleteResourceJob; -use App\Jobs\VolumeCloneJob; use App\Models\EnvironmentVariable; use App\Models\LocalFileVolume; use App\Models\LocalPersistentVolume; @@ -23,7 +22,6 @@ use Illuminate\Database\Eloquent\Model; use Illuminate\Http\JsonResponse; use Illuminate\Http\Request; use Illuminate\Support\Collection; -use Illuminate\Support\Facades\Bus; use Illuminate\Support\Facades\Validator; use OpenApi\Attributes as OA; @@ -78,7 +76,12 @@ class ServicesController extends Controller $service->makeHidden(['value', 'real_value']); } - return serializeApiResponse($service); + $payload = serializeApiResponse($service); + if ($service instanceof Service && filled(data_get($payload, 'docker_compose'))) { + $payload['docker_compose'] = redactLockedValuesInGeneratedCompose($payload['docker_compose'], $service); + } + + return $payload; } /** @@ -1023,7 +1026,6 @@ class ServicesController extends Controller new OA\Parameter(name: 'delete_volumes', in: 'query', required: false, description: 'Delete volumes.', schema: new OA\Schema(type: 'boolean', default: true)), new OA\Parameter(name: 'docker_cleanup', in: 'query', required: false, description: 'Run docker cleanup when the server disk usage is at or above its cleanup threshold. Skipped when a cleanup ran on the server in the last hour.', schema: new OA\Schema(type: 'boolean', default: true)), new OA\Parameter(name: 'delete_connected_networks', in: 'query', required: false, description: 'Delete connected networks.', schema: new OA\Schema(type: 'boolean', default: true)), - new OA\Parameter(name: 'delete_from_coolify_only', in: 'query', required: false, description: 'Remove only Coolify metadata without deleting Docker resources.', schema: new OA\Schema(type: 'boolean', default: false)), ], responses: [ new OA\Response( @@ -1054,6 +1056,7 @@ class ServicesController extends Controller ref: '#/components/responses/404', ), new OA\Response(response: 403, description: 'The token needs the write and deploy permissions.'), + new OA\Response(response: 422, ref: '#/components/responses/422'), ] )] public function delete_by_uuid(Request $request) @@ -1072,10 +1075,14 @@ class ServicesController extends Controller $this->authorize('delete', $service); - $service->delete(); + // The parameter was removed. Reject it instead of ignoring it, so old clients do not delete Docker resources they asked to keep. + if ($request->boolean('delete_from_coolify_only')) { + return response()->json([ + 'message' => 'The delete_from_coolify_only parameter is no longer supported. Nothing was deleted. Send the request without it: Coolify removes the service also when the server is not reachable.', + ], 422); + } - $deleteFromCoolifyOnlyRequested = $request->boolean('delete_from_coolify_only'); - $deleteFromCoolifyOnly = $deleteFromCoolifyOnlyRequested || ! $service->server?->isFunctional(); + $service->delete(); DeleteResourceJob::dispatch( resource: $service, @@ -1083,23 +1090,15 @@ class ServicesController extends Controller deleteConnectedNetworks: $request->boolean('delete_connected_networks', true), deleteConfigurations: $request->boolean('delete_configurations', true), dockerCleanup: $request->boolean('docker_cleanup', true), - deleteFromCoolifyOnly: $deleteFromCoolifyOnly, ); auditLog('api.service.deleted', [ 'team_id' => $teamId, 'service_uuid' => $service->uuid, 'service_name' => $service->name, - 'delete_from_coolify_only' => $deleteFromCoolifyOnly, ]); - $message = match (true) { - $deleteFromCoolifyOnlyRequested => 'The service will be removed from Coolify only; Docker resources will remain.', - $deleteFromCoolifyOnly => 'Server is not reachable. The service will be removed from Coolify only; Docker resources may remain.', - default => 'Service deletion request queued.', - }; - - return response()->json(['message' => $message]); + return response()->json(['message' => 'Service deletion request queued.']); } #[OA\Patch( @@ -1150,6 +1149,7 @@ class ServicesController extends Controller 'force_domain_override' => ['type' => 'boolean', 'default' => false, 'description' => 'Force domain override even if conflicts are detected.'], 'is_container_label_escape_enabled' => ['type' => 'boolean', 'default' => true, 'description' => 'Escape special characters in labels. By default, $ (and other chars) is escaped. If you want to use env variables inside the labels, turn this off.'], 'redact_all_env_values_in_logs' => ['type' => 'boolean', 'default' => true, 'description' => 'Redact all environment variable values in logs. When false, only locked (shown once) values are redacted. Requires team admin permissions.'], + 'include_coolify_default_variables' => ['type' => 'boolean', 'default' => true, 'description' => 'Add the predefined COOLIFY_* variables (COOLIFY_URL, COOLIFY_FQDN, ...) to the environment. Requires team admin permissions. Redeploy to apply it.'], ], ) ), @@ -1238,7 +1238,7 @@ class ServicesController extends Controller $this->authorize('update', $service); - if ($request->has('redact_all_env_values_in_logs')) { + if ($request->has('redact_all_env_values_in_logs') || $request->has('include_coolify_default_variables')) { $this->authorize('manageEnvironment', $service); } @@ -1247,7 +1247,7 @@ class ServicesController extends Controller $this->authorize('deploy', $service); } - $allowedFields = ['name', 'description', 'instant_deploy', 'docker_compose_raw', 'connect_to_docker_network', 'urls', 'force_domain_override', 'is_container_label_escape_enabled', 'redact_all_env_values_in_logs']; + $allowedFields = ['name', 'description', 'instant_deploy', 'docker_compose_raw', 'connect_to_docker_network', 'urls', 'force_domain_override', 'is_container_label_escape_enabled', 'redact_all_env_values_in_logs', 'include_coolify_default_variables']; $validationRules = [ 'name' => 'string|max:255', @@ -1262,6 +1262,7 @@ class ServicesController extends Controller 'force_domain_override' => 'boolean', 'is_container_label_escape_enabled' => 'boolean', 'redact_all_env_values_in_logs' => 'boolean', + 'include_coolify_default_variables' => 'boolean', ]; $validationMessages = [ 'urls.*.array' => 'An item in the urls array has invalid fields. Only name and url fields are supported.', @@ -1332,6 +1333,9 @@ class ServicesController extends Controller if ($request->has('redact_all_env_values_in_logs')) { $service->redact_all_env_values_in_logs = $request->boolean('redact_all_env_values_in_logs'); } + if ($request->has('include_coolify_default_variables')) { + $service->include_coolify_default_variables = $request->boolean('include_coolify_default_variables'); + } $previousDnsHostnames = $this->serviceApplicationDnsHostnames($service); $service->save(); @@ -3337,149 +3341,10 @@ class ServicesController extends Controller : $service->name.'-clone-'.$uuid; $cloneVolumeData = $request->boolean('clone_volumes', false); - $newService = $service->replicate([ - 'id', - 'created_at', - 'updated_at', - ])->fill([ + $newService = clone_service($service, $destination, [ 'uuid' => $uuid, 'name' => $name, - 'destination_id' => $destination->id, - 'destination_type' => $destination->getMorphClass(), - 'server_id' => $destination->server_id, - ]); - $newService->save(); - - foreach ($service->tags as $tag) { - $newService->tags()->attach($tag->id); - } - - foreach ($service->scheduled_tasks()->get() as $task) { - $task->replicate([ - 'id', - 'created_at', - 'updated_at', - ])->fill([ - 'uuid' => new_public_id(), - 'service_id' => $newService->id, - 'team_id' => $teamId, - ])->save(); - } - - foreach ($service->environment_variables()->get() as $environmentVariable) { - $environmentVariable->replicate([ - 'id', - 'created_at', - 'updated_at', - ])->fill([ - 'resourceable_id' => $newService->id, - 'resourceable_type' => $newService->getMorphClass(), - ])->save(); - } - - // Create applications/databases (and their volumes) for the clone first. - // Child rows are not copied by Service::replicate(). - $newService->parse(); - $newService->refresh(); - - $sourceApplicationsByName = $service->applications()->get()->keyBy('name'); - $sourceDatabasesByName = $service->databases()->get()->keyBy('name'); - $pendingVolumeClones = []; - $sourceServer = $service->destination?->server; - $targetServer = $newService->destination?->server; - - foreach ($newService->applications()->get() as $application) { - $application->fill(['status' => 'exited'])->save(); - - $sourceApplication = $sourceApplicationsByName->get($application->name); - if (! $sourceApplication) { - continue; - } - - if ($cloneVolumeData) { - $targetVolumesByMount = $application->persistentStorages()->get()->keyBy('mount_path'); - foreach ($sourceApplication->persistentStorages()->get() as $sourceVolume) { - $targetVolume = $targetVolumesByMount->get($sourceVolume->mount_path); - if (! $targetVolume) { - continue; - } - - $pendingVolumeClones[] = [ - 'source' => $sourceVolume->name, - 'target' => $targetVolume->name, - 'model' => $targetVolume, - ]; - } - } - } - - foreach ($newService->databases()->get() as $database) { - $database->fill(['status' => 'exited'])->save(); - - $sourceDatabase = $sourceDatabasesByName->get($database->name); - if (! $sourceDatabase) { - continue; - } - - if ($cloneVolumeData) { - $targetVolumesByMount = $database->persistentStorages()->get()->keyBy('mount_path'); - foreach ($sourceDatabase->persistentStorages()->get() as $sourceVolume) { - $targetVolume = $targetVolumesByMount->get($sourceVolume->mount_path); - if (! $targetVolume) { - continue; - } - - $pendingVolumeClones[] = [ - 'source' => $sourceVolume->name, - 'target' => $targetVolume->name, - 'model' => $targetVolume, - ]; - } - } - - foreach ($sourceDatabase->scheduledBackups()->get() as $backup) { - $newBackup = $backup->replicate([ - 'id', - 'created_at', - 'updated_at', - ])->fill([ - 'uuid' => new_public_id(), - 'database_id' => $database->id, - 'database_type' => $database->getMorphClass(), - 'team_id' => $teamId, - ]); - $newBackup->save(); - $backup->copyS3StoragesTo($newBackup); - } - } - - if ($cloneVolumeData && $pendingVolumeClones !== [] && $sourceServer && $targetServer) { - try { - $chain = [ - function () use ($service) { - StopService::run($service); - }, - ]; - - foreach ($pendingVolumeClones as $clone) { - $chain[] = new VolumeCloneJob( - $clone['source'], - $clone['target'], - $sourceServer, - $targetServer, - $clone['model'], - ); - } - - $chain[] = function () use ($service) { - StartService::run($service); - }; - - Bus::chain($chain)->onQueue('high')->dispatch(); - } catch (\Exception $e) { - \Log::error('Failed to queue service volume clone for '.$service->uuid.': '.$e->getMessage()); - } - } + ], $cloneVolumeData); auditLog('api.service.cloned', [ 'team_id' => $teamId, diff --git a/app/Http/Controllers/FileBrowserDownloadController.php b/app/Http/Controllers/FileBrowserDownloadController.php index 17bed2392e..c0aeb76cd3 100644 --- a/app/Http/Controllers/FileBrowserDownloadController.php +++ b/app/Http/Controllers/FileBrowserDownloadController.php @@ -24,7 +24,7 @@ class FileBrowserDownloadController extends Controller public function __invoke(string $token): StreamedResponse { - /** @var array{user_id:int,resource_type:class-string,resource_id:int,server_id:int,container:string,path:string,name:string,is_dir:bool}|null $download */ + /** @var array{user_id:int,resource_type:class-string,resource_id:int,server_id:int,container:string,path:string,name:string,is_dir:bool,use_helper?:bool}|null $download */ $download = Cache::pull(self::cacheKey($token)); abort_unless(is_array($download) && $download['user_id'] === auth()->id(), 404); @@ -33,7 +33,7 @@ class FileBrowserDownloadController extends Controller Gate::authorize('view', $resource); abort_unless((int) $resource->environment->project->team_id === (int) auth()->user()->currentTeam()->id, 403); - $filesystem = new ContainerFilesystemService(Server::findOrFail($download['server_id']), $download['container']); + $filesystem = new ContainerFilesystemService(Server::findOrFail($download['server_id']), $download['container'], $download['use_helper'] ?? false); return response()->streamDownload(function () use ($filesystem, $download) { set_time_limit(0); diff --git a/app/Http/Controllers/Webhook/Concerns/MatchesManualWebhookApplications.php b/app/Http/Controllers/Webhook/Concerns/MatchesManualWebhookApplications.php index 7f2feb1e04..decb6705fc 100644 --- a/app/Http/Controllers/Webhook/Concerns/MatchesManualWebhookApplications.php +++ b/app/Http/Controllers/Webhook/Concerns/MatchesManualWebhookApplications.php @@ -50,7 +50,13 @@ trait MatchesManualWebhookApplications // Git hosts (GitHub, GitLab, Gitea, Bitbucket) treat owner/repo names // case-insensitively, so compare the canonical paths case-insensitively. - return hash_equals(mb_strtolower($fullName), mb_strtolower($repositoryPath)); + $fullName = mb_strtolower($fullName); + $repositoryPath = mb_strtolower($repositoryPath); + + // A self-hosted Git host can serve repositories under a sub-path + // (https://example.com/gitea/owner/repo), which is not part of the + // repository name in the webhook payload. + return $repositoryPath === $fullName || str_ends_with($repositoryPath, '/'.$fullName); } /** diff --git a/app/Http/Controllers/Webhook/Concerns/ThrottlesManualWebhookFailures.php b/app/Http/Controllers/Webhook/Concerns/ThrottlesManualWebhookFailures.php index 7e48d1a637..f02e0af13e 100644 --- a/app/Http/Controllers/Webhook/Concerns/ThrottlesManualWebhookFailures.php +++ b/app/Http/Controllers/Webhook/Concerns/ThrottlesManualWebhookFailures.php @@ -51,7 +51,7 @@ trait ThrottlesManualWebhookFailures /** * The connection address of the client. X-Forwarded-For is ignored, since * any client can set it and rotate it to get a new bucket. On cloud, a - * valid CF-Connecting-IP is used, as in auth_rate_limit_ip(). + * valid CF-Connecting-IP is used. */ protected function manualWebhookFailureClientAddress(Request $request): string { diff --git a/app/Http/Controllers/Webhook/Github.php b/app/Http/Controllers/Webhook/Github.php index 77e2979ccf..e644a68c26 100644 --- a/app/Http/Controllers/Webhook/Github.php +++ b/app/Http/Controllers/Webhook/Github.php @@ -421,17 +421,15 @@ class Github extends Controller return response('Invalid signature.'); } $hmac = hash_hmac('sha256', $request->getContent(), $webhook_secret); - if (config('app.env') !== 'local') { - if (! hash_equals($x_hub_signature_256, $hmac)) { - auditLogWebhookFailure('github', 'invalid_signature', [ - 'mode' => 'app', - 'github_app_id' => $github_app->id, - 'github_app_name' => $github_app->name, - 'installation_target_id' => $x_github_hook_installation_target_id, - ]); + if (! hash_equals($x_hub_signature_256, $hmac)) { + auditLogWebhookFailure('github', 'invalid_signature', [ + 'mode' => 'app', + 'github_app_id' => $github_app->id, + 'github_app_name' => $github_app->name, + 'installation_target_id' => $x_github_hook_installation_target_id, + ]); - return response('Invalid signature.'); - } + return response('Invalid signature.'); } TransferFinishChecklist::recordWebhook('github_app', $github_app->uuid); if ($x_github_event === 'installation' || $x_github_event === 'installation_repositories') { diff --git a/app/Http/Kernel.php b/app/Http/Kernel.php index 26cbf3ac45..e897919e71 100644 --- a/app/Http/Kernel.php +++ b/app/Http/Kernel.php @@ -21,6 +21,7 @@ use App\Http\Middleware\KeepConcurrentTeamSwitch; use App\Http\Middleware\PreventRequestsDuringMaintenance; use App\Http\Middleware\RedirectIfAuthenticated; use App\Http\Middleware\RequirePassword; +use App\Http\Middleware\RequireTeamTwoFactor; use App\Http\Middleware\TrimStrings; use App\Http\Middleware\TrustProxies; use App\Http\Middleware\ValidateSignature; @@ -81,6 +82,7 @@ class Kernel extends HttpKernel SubstituteBindings::class, CheckForcePasswordReset::class, DecideWhatToDoWithUser::class, + RequireTeamTwoFactor::class, ], diff --git a/app/Http/Middleware/ApiIpAllowed.php b/app/Http/Middleware/ApiIpAllowed.php index ede32567d0..72aaa4b7c8 100644 --- a/app/Http/Middleware/ApiIpAllowed.php +++ b/app/Http/Middleware/ApiIpAllowed.php @@ -22,7 +22,7 @@ class ApiIpAllowed $allowedIps = array_filter(array_map('trim', explode(',', $allowedIps))); - if (! empty($allowedIps) && ! checkIPAgainstAllowlist($request->ip(), $allowedIps)) { + if (! empty($allowedIps) && ! checkIPAgainstAllowlist(client_ip($request), $allowedIps)) { return response()->json(['success' => true, 'message' => 'You are not allowed to access the API.'], 403); } diff --git a/app/Http/Middleware/RequireTeamTwoFactor.php b/app/Http/Middleware/RequireTeamTwoFactor.php new file mode 100644 index 0000000000..bb5421458e --- /dev/null +++ b/app/Http/Middleware/RequireTeamTwoFactor.php @@ -0,0 +1,40 @@ +user(); + if (! $user || filled($user->two_factor_confirmed_at)) { + return $next($request); + } + + $teamId = data_get(session('currentTeam'), 'id'); + if (is_null($teamId) || in_array($request->path(), allowedPathsForTwoFactorSetup(), true)) { + return $next($request); + } + + // Read the flag from the database: the session and cache can hold a team from before the change. + if (! Team::query()->whereKey($teamId)->value('is_two_factor_required')) { + return $next($request); + } + + if ($request->expectsJson() || $request->routeIs('*livewire.update')) { + abort(403, 'Your team requires two-factor authentication. Configure it to continue.'); + } + + return redirect()->route('two-factor.required'); + } +} diff --git a/app/Jobs/ApplicationDeploymentJob.php b/app/Jobs/ApplicationDeploymentJob.php index 7b7871d763..ae31fcfe56 100644 --- a/app/Jobs/ApplicationDeploymentJob.php +++ b/app/Jobs/ApplicationDeploymentJob.php @@ -87,8 +87,6 @@ class ApplicationDeploymentJob implements ShouldBeEncrypted, ShouldQueue public static int $batch_counter = 0; - private bool $newVersionIsHealthy = false; - private ApplicationDeploymentQueue $application_deployment_queue; private Application $application; @@ -158,6 +156,9 @@ class ApplicationDeploymentJob implements ShouldBeEncrypted, ShouldQueue /** @var array|null */ private ?array $remote_secrets_cache = null; + /** @var array|null */ + private ?array $previewVariableIdsWithProductionValues = null; + private ?bool $has_secret_manager_source = null; private $env_nixpacks_args; @@ -294,9 +295,9 @@ class ApplicationDeploymentJob implements ShouldBeEncrypted, ShouldQueue } if ($this->preview) { if ($this->application->build_pack === 'dockercompose') { - $this->preview->generate_preview_fqdn_compose(); + $this->preview->generate_preview_fqdn_compose(keepExistingDomains: true); } else { - $this->preview->generate_preview_fqdn(); + $this->preview->generate_preview_fqdn(keepExistingDomains: true); } } if ($this->application->isGithubAppSource()) { @@ -856,6 +857,9 @@ class ApplicationDeploymentJob implements ShouldBeEncrypted, ShouldQueue } $this->generated_docker_compose = $yaml; $this->execute_remote_command([ + "docker exec {$this->deployment_uuid} rm -f {$this->workdir}{$this->docker_compose_location}", + 'hidden' => true, + ], [ "docker exec -i {$this->deployment_uuid} tee {$this->workdir}{$this->docker_compose_location} > /dev/null", 'input' => $yaml, 'hidden' => true, @@ -1329,10 +1333,16 @@ class ApplicationDeploymentJob implements ShouldBeEncrypted, ShouldQueue $composeFileName = "$mainDir/".addPreviewDeploymentSuffix('docker-compose', $this->pull_request_id).'.yaml'; $this->docker_compose_location = '/'.addPreviewDeploymentSuffix('docker-compose', $this->pull_request_id).'.yaml'; } + // `docker cp` above can copy symlinks from the repository to these names. Remove them first, so that + // Coolify writes regular files. $this->execute_remote_command( [ "mkdir -p $mainDir", ], + [ + "rm -f $composeFileName $mainDir/README.md", + 'hidden' => true, + ], [ "tee $composeFileName > /dev/null", 'input' => $this->generated_docker_compose, @@ -1538,6 +1548,8 @@ class ApplicationDeploymentJob implements ShouldBeEncrypted, ShouldQueue $this->rolling_update(); return true; + } elseif ($configurationDiff->isLegacyFallback()) { + $this->application_deployment_queue->addLogEntry('The last deployment stored no configuration snapshot. Rebuilding image.'); } else { $this->application_deployment_queue->addLogEntry('Build configuration changed. Rebuilding image.'); } @@ -1741,10 +1753,10 @@ class ApplicationDeploymentJob implements ShouldBeEncrypted, ShouldQueue $sort = $this->application->settings->is_env_sorting_enabled; if ($sort) { $sorted_environment_variables = $this->application->runtime_environment_variables->sortBy('key'); - $sorted_environment_variables_preview = $this->application->runtime_environment_variables_preview->sortBy('key'); + $sorted_environment_variables_preview = $this->application->runtime_environment_variables_preview()->whereNotIn('id', $this->previewVariableIdsWithProductionValues())->get()->sortBy('key'); } else { $sorted_environment_variables = $this->application->runtime_environment_variables->sortBy('id'); - $sorted_environment_variables_preview = $this->application->runtime_environment_variables_preview->sortBy('id'); + $sorted_environment_variables_preview = $this->application->runtime_environment_variables_preview()->whereNotIn('id', $this->previewVariableIdsWithProductionValues())->get()->sortBy('id'); } if ($this->build_pack === 'dockercompose') { $sorted_environment_variables = $sorted_environment_variables->reject(fn (EnvironmentVariable $env) => $this->isGeneratedDockerComposeEnvironmentVariable($env)); @@ -1881,33 +1893,14 @@ class ApplicationDeploymentJob implements ShouldBeEncrypted, ShouldQueue $envs->push($this->runtimeEnvironmentAssignment($env)); } - // Fall back to production env vars for keys not overridden by preview vars, - // but only when preview vars are configured. This ensures variables like - // DB_PASSWORD that are only set for production will be available in the - // preview .env file (fixing ${VAR} interpolation in docker-compose YAML), - // while avoiding leaking production values when previews aren't configured. - if ($runtime_environment_variables_preview->isNotEmpty()) { - $previewKeys = $runtime_environment_variables_preview->pluck('key')->toArray(); - $fallback_production_vars = $sorted_environment_variables->filter(function ($env) use ($previewKeys) { - return $env->is_runtime && ! in_array($env->key, $previewKeys); - }); - foreach ($fallback_production_vars as $env) { - if ($this->shouldOmitBlankComposeEnvironmentVariable($env)) { - continue; - } - - $envs->push($this->runtimeEnvironmentAssignment($env)); - } - } - // Add PORT if not exists, use the first port as default if ($this->build_pack !== 'dockercompose') { - if ($this->application->environment_variables_preview->where('key', 'PORT')->isEmpty()) { + if ($this->deployedPreviewVariables()->where('key', 'PORT')->isEmpty()) { $envs->push("PORT={$ports[0]}"); } } // Add HOST if not exists - if ($this->application->environment_variables_preview->where('key', 'HOST')->isEmpty()) { + if ($this->deployedPreviewVariables()->where('key', 'HOST')->isEmpty()) { $envs->push('HOST=0.0.0.0'); } } @@ -1968,6 +1961,9 @@ class ApplicationDeploymentJob implements ShouldBeEncrypted, ShouldQueue // Generate runtime environment variables locally $environment_variables = $this->generate_runtime_environment_variables(); + // Each .env write first removes the file: with "Preserve repository", `docker cp` can copy a symlink + // from the repository to this name. Coolify writes a regular file instead of the link target. + // Handle empty environment variables if ($environment_variables->isEmpty()) { // For Docker Compose and Docker Image, we need to create an empty .env file @@ -1978,7 +1974,7 @@ class ApplicationDeploymentJob implements ShouldBeEncrypted, ShouldQueue // Create empty .env file $this->execute_remote_command( [ - executeInDocker($this->deployment_uuid, "touch $this->workdir/.env"), + executeInDocker($this->deployment_uuid, "rm -f $this->workdir/.env && touch $this->workdir/.env"), ] ); @@ -1986,6 +1982,10 @@ class ApplicationDeploymentJob implements ShouldBeEncrypted, ShouldQueue if ($this->use_build_server) { $this->server = $this->mainServer; $this->execute_remote_command( + [ + "rm -f $this->configuration_dir/.env", + 'hidden' => true, + ], [ "touch $this->configuration_dir/.env", ] @@ -1993,6 +1993,10 @@ class ApplicationDeploymentJob implements ShouldBeEncrypted, ShouldQueue $this->server = $this->build_server; } else { $this->execute_remote_command( + [ + "rm -f $this->configuration_dir/.env", + 'hidden' => true, + ], [ "touch $this->configuration_dir/.env", ] @@ -2038,7 +2042,7 @@ class ApplicationDeploymentJob implements ShouldBeEncrypted, ShouldQueue $this->application_deployment_queue->addLogEntry('Creating .env file with runtime variables for container.', hidden: true); $this->execute_remote_command( [ - executeInDocker($this->deployment_uuid, "echo '$envs_base64' | base64 -d | tee $this->workdir/.env > /dev/null"), + executeInDocker($this->deployment_uuid, "rm -f $this->workdir/.env && echo '$envs_base64' | base64 -d | tee $this->workdir/.env > /dev/null"), 'skip_command_log' => true, ] ); @@ -2051,6 +2055,10 @@ class ApplicationDeploymentJob implements ShouldBeEncrypted, ShouldQueue if ($this->use_build_server) { $this->server = $this->mainServer; $this->execute_remote_command( + [ + "rm -f $this->configuration_dir/.env", + 'hidden' => true, + ], [ "echo '$envs_base64' | base64 -d | tee $this->configuration_dir/.env > /dev/null", 'skip_command_log' => true, @@ -2059,6 +2067,10 @@ class ApplicationDeploymentJob implements ShouldBeEncrypted, ShouldQueue $this->server = $this->build_server; } else { $this->execute_remote_command( + [ + "rm -f $this->configuration_dir/.env", + 'hidden' => true, + ], [ "echo '$envs_base64' | base64 -d | tee $this->configuration_dir/.env > /dev/null", 'skip_command_log' => true, @@ -2250,6 +2262,7 @@ class ApplicationDeploymentJob implements ShouldBeEncrypted, ShouldQueue } } else { $sorted_environment_variables = $this->application->environment_variables_preview() + ->whereNotIn('id', $this->previewVariableIdsWithProductionValues()) ->withoutBuildpackControlVariables() ->where('is_buildtime', true) // ONLY build-time variables ->orderBy($this->application->settings->is_env_sorting_enabled ? 'key' : 'id') @@ -2468,6 +2481,39 @@ class ApplicationDeploymentJob implements ShouldBeEncrypted, ShouldQueue $this->application_deployment_queue->addLogEntry(' Suggested name: '.ValidationPatterns::displayShellEnvironmentVariableKey($suggestedKey), type: 'info'); } + /** + * IDs of preview variables that have the same value as the production variable with the same key. + * Coolify copies each new production variable to the previews. With public PR deployments, a + * preview can run code from a fork, so it gets only the preview values that the user changed. + * + * @return array + */ + private function previewVariableIdsWithProductionValues(): array + { + if ($this->previewVariableIdsWithProductionValues !== null) { + return $this->previewVariableIdsWithProductionValues; + } + + if ($this->pull_request_id === 0 || ! $this->application->settings->is_pr_deployments_public_enabled) { + return $this->previewVariableIdsWithProductionValues = []; + } + + $productionValues = $this->application->environment_variables()->get()->pluck('value', 'key'); + + return $this->previewVariableIdsWithProductionValues = $this->application->environment_variables_preview()->get() + ->filter(fn (EnvironmentVariable $env) => $productionValues->has($env->key) && $productionValues->get($env->key) === $env->value) + ->pluck('id') + ->all(); + } + + /** + * The preview variables that this preview deployment uses. + */ + private function deployedPreviewVariables(): Collection + { + return $this->application->environment_variables_preview->whereNotIn('id', $this->previewVariableIdsWithProductionValues()); + } + /** * Skip build-time variables with names that new variables can no longer use (such as my-var). * validateDeploymentEnvironmentVariableKeys() logs a warning for them before the deployment starts. @@ -2709,6 +2755,7 @@ class ApplicationDeploymentJob implements ShouldBeEncrypted, ShouldQueue $this->application_deployment_queue->addLogEntry('Rolling update started.'); $this->start_by_compose_file(); $this->health_check(); + $this->wait_for_proxy_to_route_new_container(); $this->stop_running_container(); $this->application_deployment_queue->addLogEntry('Rolling update completed.'); } @@ -2718,6 +2765,46 @@ class ApplicationDeploymentJob implements ShouldBeEncrypted, ShouldQueue } } + /** + * The proxy adds the new container some seconds after it is healthy. Stopping the old + * container before that sends requests to a closed port (502), see #8627. + */ + private function wait_for_proxy_to_route_new_container(): void + { + $seconds = $this->application->settings->rollingUpdateDrainSeconds(); + if ($seconds === 0 || ! $this->newVersionIsHealthy()) { + return; + } + $oldContainerNames = getCurrentApplicationContainerStatus($this->server, $this->application, $this->pull_request_id) + ->filter(fn ($container) => data_get($container, 'State') === 'running' && data_get($container, 'Names') !== $this->container_name) + ->pluck('Names'); + if ($oldContainerNames->isEmpty()) { + return; + } + if ($this->proxyLabelsChanged($oldContainerNames)) { + // The proxy drops a route that two containers define differently until the old one stops. + $this->application_deployment_queue->addLogEntry('The proxy labels changed, so the old containers stop without waiting.'); + + return; + } + $this->application_deployment_queue->addLogEntry("Waiting {$seconds} seconds for the proxy to send traffic to the new container."); + Sleep::for($seconds)->seconds(); + } + + private function proxyLabelsChanged(Collection $oldContainerNames): bool + { + $containerNames = $oldContainerNames->prepend($this->container_name)->map(fn (string $name) => escapeshellarg($name))->implode(' '); + $output = instant_remote_process(["docker inspect --format '{{json .Config.Labels}}' {$containerNames}"], $this->server, false); + + return collect(preg_split('/\R/', trim((string) $output))) + ->map(fn (string $labels) => collect(json_decode($labels, true) ?: []) + ->filter(fn ($value, string $key) => str_starts_with($key, 'traefik.') || str_starts_with($key, 'caddy')) + ->sortKeys() + ->all()) + ->unique() + ->count() > 1; + } + private function health_check() { try { @@ -2725,7 +2812,7 @@ class ApplicationDeploymentJob implements ShouldBeEncrypted, ShouldQueue // Implement healthcheck for swarm } else { if ($this->application->isHealthcheckDisabled() && $this->application->custom_healthcheck_found === false) { - $this->newVersionIsHealthy = true; + $this->markNewVersionHealthy(true); return; } @@ -2773,12 +2860,12 @@ class ApplicationDeploymentJob implements ShouldBeEncrypted, ShouldQueue $healthCheckStatus = $this->trimmedSavedOutput('health_check')->replace('"', '')->value(); if ($healthCheckStatus === 'healthy') { - $this->newVersionIsHealthy = true; + $this->markNewVersionHealthy(true); $this->application->update(['status' => 'running']); $this->application_deployment_queue->addLogEntry('New container is healthy.'); break; } elseif ($healthCheckStatus === 'unhealthy') { - $this->newVersionIsHealthy = false; + $this->markNewVersionHealthy(false); $this->application_deployment_queue->addLogEntry('New container is unhealthy.', type: 'error'); $this->query_logs(); break; @@ -2800,6 +2887,24 @@ class ApplicationDeploymentJob implements ShouldBeEncrypted, ShouldQueue } } + /** + * Laravel runs failed() on a fresh copy of the job, so the health of the new version is kept outside the job instance. + */ + private function markNewVersionHealthy(bool $healthy): void + { + $key = "deployment-new-version-healthy:{$this->deployment_uuid}"; + if ($healthy) { + Cache::put($key, true, now()->addDay()); + } else { + Cache::forget($key); + } + } + + private function newVersionIsHealthy(): bool + { + return Cache::has("deployment-new-version-healthy:{$this->deployment_uuid}"); + } + private function query_logs() { $escapedContainerName = escapeshellarg($this->container_name); @@ -2830,7 +2935,7 @@ class ApplicationDeploymentJob implements ShouldBeEncrypted, ShouldQueue if ($this->use_build_server) { $this->server = $this->build_server; } - $this->newVersionIsHealthy = true; + $this->markNewVersionHealthy(true); $this->generate_image_names(); $this->application_deployment_queue->addLogEntry("Starting pull request (#{$this->pull_request_id}) deployment of {$this->customRepository}:{$this->application->git_branch}."); $this->prepare_builder_image(); @@ -3057,13 +3162,10 @@ class ApplicationDeploymentJob implements ShouldBeEncrypted, ShouldQueue $fqdn = $this->preview->fqdn; } if (isset($fqdn)) { - $domains = str($fqdn)->explode(',') - ->map(fn (string $domain) => trim($domain)) - ->filter() - ->filter(fn (string $domain) => isValidDomainUrl($domain)); + $domains = $this->validDomainsFromFqdn($fqdn); if ($domains->isNotEmpty()) { $url = $domains->map(fn (string $domain) => Url::fromString($domain)->withPort(null)->__toString())->implode(','); - $fqdn = $domains->map(fn (string $domain) => Url::fromString($domain)->getHost())->implode(','); + $fqdn = $this->hostsFromFqdn($fqdn); if ((int) $this->application->compose_parsing_version >= 3) { $this->coolify_variables .= 'COOLIFY_URL='.escapeShellValue($url).' '; $this->coolify_variables .= 'COOLIFY_FQDN='.escapeShellValue($fqdn).' '; @@ -3079,6 +3181,24 @@ class ApplicationDeploymentJob implements ShouldBeEncrypted, ShouldQueue $this->coolify_variables .= 'COOLIFY_RESOURCE_UUID='.escapeShellValue($this->application->uuid).' '; } + private function validDomainsFromFqdn(?string $fqdn): Collection + { + return str($fqdn ?? '')->explode(',') + ->map(fn (string $domain) => trim($domain)) + ->filter() + ->filter(fn (string $domain) => isValidDomainUrl($domain)); + } + + /** + * Comma-separated hosts without scheme, port, or path (the COOLIFY_FQDN value). + */ + private function hostsFromFqdn(?string $fqdn): string + { + return $this->validDomainsFromFqdn($fqdn) + ->map(fn (string $domain) => Url::fromString($domain)->getHost()) + ->implode(','); + } + private function shellAssignmentForDockerfileArg(string $assignment): string { [$key, $value] = array_pad(explode('=', $assignment, 2), 2, null); @@ -3097,11 +3217,7 @@ class ApplicationDeploymentJob implements ShouldBeEncrypted, ShouldQueue private function gitLsRemoteCommand(string $lsRemoteRef, ?string $identityFile = null): string { - $sshCommand = "ssh -o ConnectTimeout=30 -p {$this->customPort} -o Port={$this->customPort} -o LogLevel=ERROR -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null"; - - if ($identityFile !== null) { - $sshCommand .= " -i {$identityFile} -o IdentitiesOnly=yes"; - } + $sshCommand = gitSshCommand($this->customPort, $identityFile); $gitConfigOption = $this->application->gitHttpsCredentialConfigOption(); $gitCommand = $gitConfigOption ? "git {$gitConfigOption}" : 'git'; @@ -3453,7 +3569,7 @@ class ApplicationDeploymentJob implements ShouldBeEncrypted, ShouldQueue } } } else { - foreach ($this->application->nixpacks_environment_variables_preview as $env) { + foreach ($this->application->nixpacks_environment_variables_preview()->whereNotIn('id', $this->previewVariableIdsWithProductionValues())->get() as $env) { $resolvedValue = $env->getResolvedValueWithServer($this->mainServer); if (! is_null($resolvedValue) && $resolvedValue !== '') { $value = ($env->is_literal || $env->is_multiline) ? trim($resolvedValue, "'") : $resolvedValue; @@ -3535,11 +3651,11 @@ class ApplicationDeploymentJob implements ShouldBeEncrypted, ShouldQueue { $genericBuildVariables = $this->pull_request_id === 0 ? $this->application->environment_variables()->withoutBuildpackControlVariables()->where('is_buildtime', true)->get() - : $this->application->environment_variables_preview()->withoutBuildpackControlVariables()->where('is_buildtime', true)->get(); + : $this->application->environment_variables_preview()->whereNotIn('id', $this->previewVariableIdsWithProductionValues())->withoutBuildpackControlVariables()->where('is_buildtime', true)->get(); $railpackVariables = $this->pull_request_id === 0 ? $this->application->railpack_environment_variables()->get() - : $this->application->railpack_environment_variables_preview()->get(); + : $this->application->railpack_environment_variables_preview()->whereNotIn('id', $this->previewVariableIdsWithProductionValues())->get(); $variables = $this->withoutLegacyEnvironmentVariableKeys($genericBuildVariables->merge($railpackVariables)) ->mapWithKeys(function (EnvironmentVariable $environmentVariable) { @@ -3949,7 +4065,7 @@ COPY ./nginx.conf /etc/nginx/conf.d/default.conf"); // Only add SOURCE_COMMIT for runtime OR when explicitly enabled for build-time // SOURCE_COMMIT changes with each commit and breaks Docker cache if included in build if (! $forBuildTime || $this->application->settings->include_source_commit_in_build) { - if ($this->application->environment_variables_preview->where('key', 'SOURCE_COMMIT')->isEmpty()) { + if ($this->deployedPreviewVariables()->where('key', 'SOURCE_COMMIT')->isEmpty()) { if (! is_null($this->commit)) { $coolify_envs->put('SOURCE_COMMIT', $this->commit); } else { @@ -3957,15 +4073,15 @@ COPY ./nginx.conf /etc/nginx/conf.d/default.conf"); } } } - if ($this->application->environment_variables_preview->where('key', 'COOLIFY_FQDN')->isEmpty()) { + if ($this->deployedPreviewVariables()->where('key', 'COOLIFY_FQDN')->isEmpty()) { if ((int) $this->application->compose_parsing_version >= 3) { $coolify_envs->put('COOLIFY_URL', $this->preview->fqdn); } else { $coolify_envs->put('COOLIFY_FQDN', $this->preview->fqdn); } } - if ($this->application->environment_variables_preview->where('key', 'COOLIFY_URL')->isEmpty()) { - $url = str($this->preview->fqdn)->replace('http://', '')->replace('https://', ''); + if ($this->deployedPreviewVariables()->where('key', 'COOLIFY_URL')->isEmpty()) { + $url = $this->hostsFromFqdn($this->preview->fqdn); if ((int) $this->application->compose_parsing_version >= 3) { $coolify_envs->put('COOLIFY_FQDN', $url); } else { @@ -3973,15 +4089,15 @@ COPY ./nginx.conf /etc/nginx/conf.d/default.conf"); } } if ($this->application->build_pack !== 'dockercompose' || $this->application->compose_parsing_version === '1' || $this->application->compose_parsing_version === '2') { - if ($this->application->environment_variables_preview->where('key', 'COOLIFY_BRANCH')->isEmpty()) { + if ($this->deployedPreviewVariables()->where('key', 'COOLIFY_BRANCH')->isEmpty()) { $coolify_envs->put('COOLIFY_BRANCH', $local_branch); } - if ($this->application->environment_variables_preview->where('key', 'COOLIFY_RESOURCE_UUID')->isEmpty()) { + if ($this->deployedPreviewVariables()->where('key', 'COOLIFY_RESOURCE_UUID')->isEmpty()) { $coolify_envs->put('COOLIFY_RESOURCE_UUID', $this->application->uuid); } // Only add COOLIFY_CONTAINER_NAME for runtime (not build-time) - it changes every deployment and breaks Docker cache if (! $forBuildTime) { - if ($this->application->environment_variables_preview->where('key', 'COOLIFY_CONTAINER_NAME')->isEmpty()) { + if ($this->deployedPreviewVariables()->where('key', 'COOLIFY_CONTAINER_NAME')->isEmpty()) { $coolify_envs->put('COOLIFY_CONTAINER_NAME', $this->container_name); } } @@ -3989,7 +4105,7 @@ COPY ./nginx.conf /etc/nginx/conf.d/default.conf"); // Runtime only, so a project or server change does not break the Docker build cache if (! $forBuildTime) { - $coolify_envs = $coolify_envs->merge(coolifyDefaultEnvironmentVariables($this->application, $this->application->environment_variables_preview->pluck('key'), $this->mainServer)); + $coolify_envs = $coolify_envs->merge(coolifyDefaultEnvironmentVariables($this->application, $this->deployedPreviewVariables()->pluck('key'), $this->mainServer)); } } else { @@ -4012,7 +4128,7 @@ COPY ./nginx.conf /etc/nginx/conf.d/default.conf"); } } if ($this->application->environment_variables->where('key', 'COOLIFY_URL')->isEmpty()) { - $url = str($this->application->fqdn)->replace('http://', '')->replace('https://', ''); + $url = $this->hostsFromFqdn($this->application->fqdn); if ((int) $this->application->compose_parsing_version >= 3) { $coolify_envs->put('COOLIFY_FQDN', $url); } else { @@ -4086,6 +4202,7 @@ COPY ./nginx.conf /etc/nginx/conf.d/default.conf"); } } else { $envs = $this->application->environment_variables_preview() + ->whereNotIn('id', $this->previewVariableIdsWithProductionValues()) ->withoutBuildpackControlVariables() ->where('is_buildtime', true) ->get(); @@ -4354,6 +4471,9 @@ COPY ./nginx.conf /etc/nginx/conf.d/default.conf"); $this->docker_compose = Yaml::dump($docker_compose, 10); $this->generated_docker_compose = $this->docker_compose; $this->execute_remote_command([ + "docker exec {$this->deployment_uuid} rm -f {$this->workdir}/docker-compose.yaml", + 'hidden' => true, + ], [ "docker exec -i {$this->deployment_uuid} tee {$this->workdir}/docker-compose.yaml > /dev/null", 'input' => $this->docker_compose, 'hidden' => true, @@ -4947,7 +5067,7 @@ COPY ./nginx.conf /etc/nginx/conf.d/default.conf"); { try { $this->application_deployment_queue->addLogEntry('Removing old containers.'); - if ($this->newVersionIsHealthy || $force) { + if ($this->newVersionIsHealthy() || $force) { if ($this->application->settings->is_consistent_container_name_enabled) { $containers = getCurrentApplicationContainerStatus($this->server, $this->application, $this->pull_request_id); $this->containerNamesToRemove($containers)->each(function (string $containerName) { @@ -4976,7 +5096,7 @@ COPY ./nginx.conf /etc/nginx/conf.d/default.conf"); } } catch (Exception $e) { // If new version is healthy, this is just cleanup - don't fail the deployment - if ($this->newVersionIsHealthy || $force) { + if ($this->newVersionIsHealthy() || $force) { $this->application_deployment_queue->addLogEntry( "Warning: Could not remove old container: {$e->getMessage()}", 'stderr', @@ -5297,6 +5417,7 @@ COPY ./nginx.conf /etc/nginx/conf.d/default.conf"); } else { // Only add preview environment variables that are available during build $envs = $this->application->environment_variables_preview() + ->whereNotIn('id', $this->previewVariableIdsWithProductionValues()) ->withoutBuildpackControlVariables() ->where('is_buildtime', true) ->get(); @@ -6147,7 +6268,7 @@ COPY ./nginx.conf /etc/nginx/conf.d/default.conf"); // 69420 means failed to push the image to the registry, so we don't need to remove the new version as it is the currently running one if ($this->application->settings->is_consistent_container_name_enabled || $this->pull_request_id !== 0) { // do not remove already running container for PR deployments - } elseif ($this->newVersionIsHealthy) { + } elseif ($this->newVersionIsHealthy()) { // The new container passed its health check, so the old container may already be removed. // A later failure (additional destinations, queue advancement, ...) must not remove the only running version. $this->application_deployment_queue->addLogEntry('Deployment failed after the new version became healthy. Keeping the new version of your application running.', 'stderr'); diff --git a/app/Jobs/CheckForUpdatesJob.php b/app/Jobs/CheckForUpdatesJob.php index 2c16b4475e..2414238b86 100644 --- a/app/Jobs/CheckForUpdatesJob.php +++ b/app/Jobs/CheckForUpdatesJob.php @@ -8,6 +8,7 @@ use Illuminate\Contracts\Queue\ShouldQueue; use Illuminate\Foundation\Bus\Dispatchable; use Illuminate\Queue\InteractsWithQueue; use Illuminate\Queue\SerializesModels; +use Illuminate\Support\Facades\Log; class CheckForUpdatesJob implements ShouldBeEncrypted, ShouldQueue { @@ -21,15 +22,18 @@ class CheckForUpdatesJob implements ShouldBeEncrypted, ShouldQueue $this->cdnTries = $cdnTries; } - public function handle(): void + /** + * @return bool Whether the check finished and new_version_available is up to date + */ + public function handle(): bool { try { if (isDev() || isCloud()) { - return; + return false; } $versions = refresh_versions_file($this->cdnTries); if ($versions === null) { - return; + return false; } CheckTraefikVersionJob::dispatch(); @@ -37,8 +41,12 @@ class CheckForUpdatesJob implements ShouldBeEncrypted, ShouldQueue $latestVersion = data_get($versions, 'coolify.v4.version'); $newVersionAvailable = version_compare($latestVersion, config('constants.coolify.version'), '>'); instanceSettings()->update(['new_version_available' => $newVersionAvailable]); + + return true; } catch (\Throwable $e) { - // Consider implementing a notification to administrators + Log::warning('Could not check for Coolify updates', ['error' => $e->getMessage()]); + + return false; } } } diff --git a/app/Jobs/CheckMissingDatabaseBackupsJob.php b/app/Jobs/CheckMissingDatabaseBackupsJob.php index ca91156574..ae517c4591 100644 --- a/app/Jobs/CheckMissingDatabaseBackupsJob.php +++ b/app/Jobs/CheckMissingDatabaseBackupsJob.php @@ -58,6 +58,11 @@ class CheckMissingDatabaseBackupsJob implements ShouldBeEncrypted, ShouldBeUniqu return; } + // Another Coolify instance backs up the databases of a transferred server. + if ($backup->server()?->isTransferredAway()) { + return; + } + // Send once for each period without backup activity. NotificationThrottle::sendOnce( $backup, diff --git a/app/Jobs/CheckMissingVolumeBackupsJob.php b/app/Jobs/CheckMissingVolumeBackupsJob.php index 1f8ab30b4f..3c6dccfbf5 100644 --- a/app/Jobs/CheckMissingVolumeBackupsJob.php +++ b/app/Jobs/CheckMissingVolumeBackupsJob.php @@ -58,6 +58,11 @@ class CheckMissingVolumeBackupsJob implements ShouldBeEncrypted, ShouldBeUnique, return; } + // Another Coolify instance backs up the volumes of a transferred server. + if ($backup->server()?->isTransferredAway()) { + return; + } + // Send once for each period without backup activity. NotificationThrottle::sendOnce( $backup, diff --git a/app/Jobs/CleanupStaleMultiplexedConnections.php b/app/Jobs/CleanupStaleMultiplexedConnections.php index 3162ae9c41..d52790e8c4 100644 --- a/app/Jobs/CleanupStaleMultiplexedConnections.php +++ b/app/Jobs/CleanupStaleMultiplexedConnections.php @@ -81,9 +81,9 @@ class CleanupStaleMultiplexedConnections implements ShouldQueue return null; } - $check = Process::run('timeout 10 ssh -O check -o ControlPath='.escapeshellarg($muxSocket).' localhost'); + $check = SshMultiplexingHelper::runMuxSocketCommand($muxSocket, 'check'); - return preg_match('/pid=(\d+)/', $check->output().$check->errorOutput(), $matches) + return $check && preg_match('/pid=(\d+)/', $check->output().$check->errorOutput(), $matches) ? $matches[1] : false; } @@ -227,10 +227,7 @@ class CleanupStaleMultiplexedConnections implements ShouldQueue } $muxSocket = "/var/www/html/storage/app/ssh/mux/{$muxFile}"; - $checkCommand = "ssh -O check -o ControlPath={$muxSocket} {$server->user}@{$server->ip} 2>/dev/null"; - $checkProcess = Process::run($checkCommand); - - if ($checkProcess->exitCode() !== 0) { + if (! SshMultiplexingHelper::runMuxSocketCommand($muxSocket, 'check')?->successful()) { $this->removeMultiplexFile($muxFile, 'connection_check_failed'); } } @@ -272,8 +269,8 @@ class CleanupStaleMultiplexedConnections implements ShouldQueue } $muxSocket = "/var/www/html/storage/app/ssh/mux/{$muxFile}"; - $checkProcess = Process::run("ssh -O check -o ControlPath={$muxSocket} localhost"); - $pid = preg_match('/pid=(\d+)/', $checkProcess->output().$checkProcess->errorOutput(), $matches) + $checkProcess = SshMultiplexingHelper::runMuxSocketCommand($muxSocket, 'check'); + $pid = $checkProcess && preg_match('/pid=(\d+)/', $checkProcess->output().$checkProcess->errorOutput(), $matches) ? $matches[1] : null; @@ -281,9 +278,8 @@ class CleanupStaleMultiplexedConnections implements ShouldQueue SshMultiplexingHelper::markMuxProcessAsRetiring($pid, $muxSocket); } - $closeCommand = "ssh -O stop -o ControlPath={$muxSocket} localhost 2>/dev/null"; - $stopProcess = Process::run($closeCommand); - if ($pid !== null && ! $stopProcess->successful()) { + $stopProcess = SshMultiplexingHelper::runMuxSocketCommand($muxSocket, 'stop'); + if ($pid !== null && ! $stopProcess?->successful()) { SshMultiplexingHelper::unmarkMuxProcessAsRetiring($pid, $muxSocket); } Storage::disk('ssh-mux')->delete($muxFile); diff --git a/app/Jobs/DatabaseBackupJob.php b/app/Jobs/DatabaseBackupJob.php index eb63bdee94..26337ebe70 100644 --- a/app/Jobs/DatabaseBackupJob.php +++ b/app/Jobs/DatabaseBackupJob.php @@ -317,7 +317,7 @@ class DatabaseBackupJob implements ShouldBeEncrypted, ShouldQueue return; } $this->backup_dir = backup_dir().'/databases/'.str($this->team->name)->slug().'-'.$this->team->id.'/'.$this->directory_name; - if ($this->database->name === 'coolify-db') { + if ($this->database instanceof StandalonePostgresql && $this->database->isInstanceDatabase()) { $databasesToBackup = ['coolify']; $this->directory_name = $this->container_name = 'coolify-db'; $ip = Str::slug($this->server->ip); @@ -460,6 +460,10 @@ class DatabaseBackupJob implements ShouldBeEncrypted, ShouldQueue // Local backup failed: the partial file is not tracked, so retention would never remove it. deleteBackupsLocally($this->backup_location, $this->server); $failureMessage = $this->redact($this->error_output ?? $this->backup_output ?? $e->getMessage()); + $exitCodeReason = $this->exitCodeReason($e->getCode()); + if ($exitCodeReason !== null) { + $failureMessage = $exitCodeReason."\n\n".$failureMessage; + } if ($this->backup_log) { $this->backup_log->update([ 'status' => 'failed', @@ -698,7 +702,9 @@ class DatabaseBackupJob implements ShouldBeEncrypted, ShouldQueue // Validate and escape database name to prevent command injection validateShellSafePath($database, 'database name'); $escapedDatabase = escapeshellarg($database); - $backupCommand .= " {$escapedContainerName} pg_dump --format=custom --no-acl --no-owner --username $escapedUsername $escapedDatabase > {$escapedBackupLocation}"; + // Keep owners and privileges in the archive: services such as Supabase need them, + // and the restore skips them unless "Keep owners and privileges" is selected. + $backupCommand .= " {$escapedContainerName} pg_dump --format=custom --username $escapedUsername $escapedDatabase > {$escapedBackupLocation}"; } $commands[] = $backupCommand; @@ -814,6 +820,24 @@ class DatabaseBackupJob implements ShouldBeEncrypted, ShouldQueue } } + /** + * Explains exit codes of a dump that was stopped from outside. The dump tool cannot report these + * itself, so its output ends after the last progress line without an error. + */ + private function exitCodeReason(int|string $exitCode): ?string + { + return match ($exitCode) { + 137 => 'The backup stopped with exit code 137: the dump process or the database container was killed (SIGKILL). ' + .'This is usually the out-of-memory killer: the dump runs inside the database container and shares its memory limit. ' + .'Check the memory limit of the database and the kernel log (dmesg) on the server.', + 143 => 'The backup stopped with exit code 143: the dump process or the database container was terminated (SIGTERM), ' + .'for example because the database was stopped or restarted during the backup.', + 255 => 'The backup stopped with exit code 255: the SSH connection to the server closed. ' + .'Check the network connection and the SSH server log on the server.', + default => null, + }; + } + private function add_to_backup_output($output): void { if ($this->backup_output) { @@ -914,7 +938,7 @@ class DatabaseBackupJob implements ShouldBeEncrypted, ShouldQueue // Remove a leftover helper in the same batch, so a replayed batch does not fail with a name conflict. $commands[] = "docker rm -f backup-of-{$this->backup_log_uuid} >/dev/null 2>&1 || true"; $mount = escapeshellarg($this->backupMountSource().':'.$this->backup_location.':ro'); - $commands[] = "docker run -d --network {$safeNetwork} --name backup-of-{$this->backup_log_uuid} --rm -v {$mount} {$fullImageName}"; + $commands[] = "docker run -d --network {$safeNetwork} --name backup-of-{$this->backup_log_uuid} --rm {$s3->minioClientRegionOption()} -v {$mount} {$fullImageName}"; // Escape S3 credentials to prevent command injection $escapedEndpoint = escapeshellarg($endpoint); diff --git a/app/Jobs/DockerCleanupJob.php b/app/Jobs/DockerCleanupJob.php index 146f751b23..9daee0c82c 100644 --- a/app/Jobs/DockerCleanupJob.php +++ b/app/Jobs/DockerCleanupJob.php @@ -110,6 +110,17 @@ class DockerCleanupJob implements ShouldBeEncrypted, ShouldQueue return; } + $activeDeployment = $this->manualCleanup ? null : CleanupDocker::activeDeployment($this->server); + if ($activeDeployment) { + $this->execution_log->update([ + 'status' => 'success', + 'message' => 'Docker cleanup skipped: deployment '.$activeDeployment->deployment_uuid.' is queued or running on this server. The next scheduled cleanup runs as usual.', + ]); + event(new DockerCleanupDone($this->execution_log)); + + return; + } + $this->usageBefore = $this->server->getDiskUsage(); if ($this->manualCleanup || $this->server->settings->force_docker_cleanup) { diff --git a/app/Jobs/PrepareFileDownloadJob.php b/app/Jobs/PrepareFileDownloadJob.php index b36d5ccd96..d121d5e648 100644 --- a/app/Jobs/PrepareFileDownloadJob.php +++ b/app/Jobs/PrepareFileDownloadJob.php @@ -36,6 +36,7 @@ class PrepareFileDownloadJob implements ShouldQueue public bool $isDir, public string $downloadName, public int $s3StorageId, + public bool $useHelper = false, ) {} public static function cacheKey(string $downloadId): string @@ -60,7 +61,7 @@ class PrepareFileDownloadJob implements ShouldQueue if ($this->container === null) { instant_remote_process([FileDownloadStorageService::buildHostFileUploadCommand($this->path, $upload['url'], $upload['headers'])], $server, timeout: 3600); } else { - (new ContainerFilesystemService($server, $this->container))->uploadToS3($this->path, $this->isDir, $upload['url'], $upload['headers']); + (new ContainerFilesystemService($server, $this->container, $this->useHelper))->uploadToS3($this->path, $this->isDir, $upload['url'], $upload['headers']); } $this->store(['status' => 'ready', 'url' => $downloads->downloadUrl($storage, $key)]); diff --git a/app/Jobs/PushServerUpdateJob.php b/app/Jobs/PushServerUpdateJob.php index 7405b1346c..9a9fd8a56c 100644 --- a/app/Jobs/PushServerUpdateJob.php +++ b/app/Jobs/PushServerUpdateJob.php @@ -551,7 +551,7 @@ class PushServerUpdateJob implements ShouldBeEncrypted, ShouldQueue, Silenced ->select($databaseColumns) ->where(fn ($query) => $this->scopeDestination($query, $standaloneDockerIds, $swarmDockerIds)) ->get(); - })->filter(fn ($database) => data_get($database, 'name') !== 'coolify-db')->values(); + })->reject(fn ($database) => $database instanceof StandalonePostgresql && $database->isInstanceDatabase())->values(); } private function serverDestinationIds(): array @@ -817,7 +817,7 @@ class PushServerUpdateJob implements ShouldBeEncrypted, ShouldQueue, Silenced private function updateProxyStatus() { // If proxy is not found, start it - if ($this->server->isProxyShouldRun()) { + if ($this->server->isProxyShouldRun() && ! $this->server->isProxyTransitionInProgress()) { if ($this->foundProxy === false) { try { if (CheckProxy::run($this->server)) { diff --git a/app/Jobs/RestartProxyJob.php b/app/Jobs/RestartProxyJob.php index c99cf9d9eb..5008ff4ce9 100644 --- a/app/Jobs/RestartProxyJob.php +++ b/app/Jobs/RestartProxyJob.php @@ -4,6 +4,7 @@ namespace App\Jobs; use App\Actions\Proxy\GetProxyConfiguration; use App\Actions\Proxy\SaveProxyConfiguration; +use App\Actions\Proxy\StartProxy; use App\Events\ProxyStatusChangedUI; use App\Models\Server; use App\Services\ProxyDashboardCacheService; @@ -45,6 +46,8 @@ class RestartProxyJob implements ShouldBeEncrypted, ShouldQueue } ProxyPortParser::fromConfiguration($configuration); + $this->server->markProxyTransitionStarted(); + // Set status to restarting $this->server->proxy->status = 'restarting'; $this->server->proxy->force_stop = false; @@ -68,8 +71,12 @@ class RestartProxyJob implements ShouldBeEncrypted, ShouldQueue ProxyStatusChangedUI::dispatch($this->server->team_id, $this->activity_id); } catch (\InvalidArgumentException $e) { + $this->server->markProxyTransitionFinished(); + return handleError($e); } catch (\Throwable $e) { + $this->server->markProxyTransitionFinished(); + // Set error status $this->server->proxy->status = 'error'; $this->server->save(); @@ -91,9 +98,6 @@ class RestartProxyJob implements ShouldBeEncrypted, ShouldQueue private function buildRestartCommands(string $configuration): array { $containerName = $this->server->isSwarm() ? 'coolify-proxy_traefik' : 'coolify-proxy'; - $proxy_path = $this->server->proxyPath(); - // Absolute paths: a non-root SSH user may not be able to enter the proxy directory (#4255). - $compose_file = rtrim($proxy_path, '/').'/docker-compose.yml'; $stopTimeout = 30; SaveProxyConfiguration::run($this->server, $configuration); @@ -127,34 +131,7 @@ class RestartProxyJob implements ShouldBeEncrypted, ShouldQueue ]); // === START PHASE === - if ($this->server->isSwarmManager()) { - $commands = $commands->merge([ - "echo 'Starting proxy (Swarm mode)...'", - "mkdir -p $proxy_path/dynamic", - "echo 'Creating required Docker Compose file.'", - "echo 'Starting coolify-proxy.'", - "docker stack deploy --detach=true -c $compose_file coolify-proxy", - "echo 'Successfully started coolify-proxy.'", - ]); - } else { - $caddyfile = 'import /dynamic/*.caddy'; - $commands = $commands->merge([ - "echo 'Starting proxy...'", - "mkdir -p $proxy_path/dynamic", - "echo '$caddyfile' | tee $proxy_path/dynamic/Caddyfile > /dev/null", - "echo 'Creating required Docker Compose file.'", - "echo 'Pulling docker image.'", - "docker compose -f $compose_file pull", - ]); - // Ensure required networks exist BEFORE docker compose up - $commands = $commands->merge(ensureProxyNetworksExist($this->server)); - $commands = $commands->merge([ - "echo 'Starting coolify-proxy.'", - "docker compose -f $compose_file up -d --wait --remove-orphans", - "echo 'Successfully started coolify-proxy.'", - ]); - $commands = $commands->merge(connectProxyToNetworks($this->server)); - } + $commands = $commands->merge(StartProxy::startCommands($this->server)); return $commands->toArray(); } diff --git a/app/Jobs/ServerCheckJob.php b/app/Jobs/ServerCheckJob.php index 0ec1ce606f..1d449cc23e 100644 --- a/app/Jobs/ServerCheckJob.php +++ b/app/Jobs/ServerCheckJob.php @@ -77,7 +77,7 @@ class ServerCheckJob implements ShouldBeEncrypted, ShouldQueue $this->checkLogDrainContainer(); } - if ($this->server->proxySet() && ! $this->server->proxy->force_stop) { + if ($this->server->proxySet() && ! $this->server->proxy->force_stop && ! $this->server->isProxyTransitionInProgress()) { $this->server->proxyType(); $foundProxyContainer = $this->containers->filter(function ($value, $key) { if ($this->server->isSwarm()) { diff --git a/app/Jobs/ServerLimitCheckJob.php b/app/Jobs/ServerLimitCheckJob.php index ff4b7531c3..82ad38d6c8 100644 --- a/app/Jobs/ServerLimitCheckJob.php +++ b/app/Jobs/ServerLimitCheckJob.php @@ -31,7 +31,7 @@ class ServerLimitCheckJob implements ShouldBeEncrypted, ShouldQueue public function handle() { try { - $servers = $this->team->servers; + $servers = $this->team->managedServers; $servers_count = $servers->count(); $number_of_servers_to_disable = $servers_count - $this->team->limits; if ($number_of_servers_to_disable > 0) { diff --git a/app/Jobs/ServerManagerJob.php b/app/Jobs/ServerManagerJob.php index e1ab5f94ee..815bb09361 100644 --- a/app/Jobs/ServerManagerJob.php +++ b/app/Jobs/ServerManagerJob.php @@ -102,7 +102,7 @@ class ServerManagerJob implements ShouldBeEncrypted, ShouldQueue || $server->digitalocean_droplet_id || $server->hostinger_virtual_machine_id; - if ($hasCloudResource && $server->cloudProviderToken) { + if ($hasCloudResource && $server->cloudProviderToken && ! $server->isTransferredAway()) { ServerCloudProviderStatusCheckJob::dispatch($server); } }); diff --git a/app/Jobs/ValidateAndInstallServerJob.php b/app/Jobs/ValidateAndInstallServerJob.php index 987b53e7f6..c6e8f53957 100644 --- a/app/Jobs/ValidateAndInstallServerJob.php +++ b/app/Jobs/ValidateAndInstallServerJob.php @@ -160,14 +160,14 @@ class ValidateAndInstallServerJob implements ShouldBeEncrypted, ShouldQueue // Validate Docker version $dockerVersion = $this->server->validateDockerEngineVersion(); if (! $dockerVersion) { - $requiredDockerVersion = str(config('constants.docker.minimum_required_version'))->before('.'); - $errorMessage = 'Minimum Docker Engine version '.$requiredDockerVersion.' is not installed. Please install Docker manually before continuing: documentation.'; + $errorMessage = $this->server->dockerEngineVersionError(); $this->server->update([ 'validation_logs' => $errorMessage, 'is_validating' => false, ]); Log::error('ValidateAndInstallServer: Docker version not sufficient', [ 'server_id' => $this->server->id, + 'docker_version' => $this->server->dockerVersion(), ]); return; @@ -180,7 +180,7 @@ class ValidateAndInstallServerJob implements ShouldBeEncrypted, ShouldQueue ]); // Start proxy if needed - if (! $this->server->isBuildServer()) { + if ($this->server->isProxyShouldRun()) { $proxyShouldRun = CheckProxy::run($this->server, true); if ($proxyShouldRun) { // Ensure networks exist BEFORE dispatching async proxy startup diff --git a/app/Jobs/VolumeBackupJob.php b/app/Jobs/VolumeBackupJob.php index 8fc34ca252..d3079fe096 100644 --- a/app/Jobs/VolumeBackupJob.php +++ b/app/Jobs/VolumeBackupJob.php @@ -437,7 +437,7 @@ class VolumeBackupJob implements ShouldBeEncrypted, ShouldQueue try { instant_remote_process([ 'docker rm -f '.escapeshellarg($containerName).' >/dev/null 2>&1 || true', - 'docker run -d --name '.escapeshellarg($containerName).' --rm -v ' + 'docker run -d --name '.escapeshellarg($containerName).' --rm '.$s3->minioClientRegionOption().' -v ' .escapeshellarg($backupLocation.':'.$backupLocation.':ro').' '.escapeshellarg($image), 'docker exec '.escapeshellarg($containerName).' mc alias set'.$resolveOptions.' temporary ' .escapeshellarg($s3->endpoint).' '.escapeshellarg($s3->key).' '.escapeshellarg($s3->secret), @@ -476,6 +476,7 @@ class VolumeBackupJob implements ShouldBeEncrypted, ShouldQueue .' | sed -n '.escapeshellarg('s/.*"size":\([0-9][0-9]*\).*/\1/p'); return 'docker run --rm --name '.escapeshellarg($containerName) + .' '.$s3->minioClientRegionOption() .' -v '.escapeshellarg($source.':/volume:ro') .' '.escapeshellarg($image) .' sh -c '.escapeshellarg($streamScript); diff --git a/app/Jobs/VolumeCloneJob.php b/app/Jobs/VolumeCloneJob.php index 677a01a35b..20ad2ff8a8 100644 --- a/app/Jobs/VolumeCloneJob.php +++ b/app/Jobs/VolumeCloneJob.php @@ -69,7 +69,7 @@ class VolumeCloneJob implements ShouldBeEncrypted, ShouldQueue instant_remote_process([ $createTargetVolume, - "docker run --rm -v {$srcVol}:/source -v {$tgtVol}:/target alpine sh -c 'cp -a /source/. /target/ && chown -R 1000:1000 /target'", + "docker run --rm -v {$srcVol}:/source -v {$tgtVol}:/target alpine sh -c 'cp -a /source/. /target/'", ], $this->sourceServer); $this->rememberTargetVolumeWithoutDriverOptions($sourceConfiguration); } @@ -123,7 +123,7 @@ class VolumeCloneJob implements ShouldBeEncrypted, ShouldQueue instant_remote_process([ $createTargetVolume, - "docker run --rm -v {$tgtVol}:/target -v {$tgtDir}:/clone alpine sh -c 'cd /target && tar xzf /clone/volume-data.tar.gz && chown -R 1000:1000 /target'", + "docker run --rm -v {$tgtVol}:/target -v {$tgtDir}:/clone alpine sh -c 'cd /target && tar xzf /clone/volume-data.tar.gz'", ], $this->targetServer); $this->rememberTargetVolumeWithoutDriverOptions($sourceConfiguration); } catch (\Exception $e) { diff --git a/app/Listeners/CloudflareTunnelChangedNotification.php b/app/Listeners/CloudflareTunnelChangedNotification.php index 3cc44c2f82..c1804f0955 100644 --- a/app/Listeners/CloudflareTunnelChangedNotification.php +++ b/app/Listeners/CloudflareTunnelChangedNotification.php @@ -19,6 +19,9 @@ class CloudflareTunnelChangedNotification $ssh_domain = data_get($event, 'data.ssh_domain'); $this->server = Server::where('id', $server_id)->firstOrFail(); + if ($this->server->isTransferredAway()) { + return; + } // Check if cloudflare tunnel is running (container is healthy) - try 3 times with 5 second intervals $cloudflareHealthy = false; diff --git a/app/Listeners/ProxyStatusChangedNotification.php b/app/Listeners/ProxyStatusChangedNotification.php index 74851cee52..73b43d551c 100644 --- a/app/Listeners/ProxyStatusChangedNotification.php +++ b/app/Listeners/ProxyStatusChangedNotification.php @@ -24,6 +24,7 @@ class ProxyStatusChangedNotification implements ShouldQueueAfterCommit if (is_null($server)) { return; } + $server->markProxyTransitionFinished(); $proxyContainerName = 'coolify-proxy'; $status = getContainerStatus($server, $proxyContainerName); $server->proxy->set('status', $status); diff --git a/app/Livewire/ActivityMonitor.php b/app/Livewire/ActivityMonitor.php index 25935d88e2..12095942c0 100644 --- a/app/Livewire/ActivityMonitor.php +++ b/app/Livewire/ActivityMonitor.php @@ -3,7 +3,9 @@ namespace App\Livewire; use App\Models\Server; +use App\Models\Service; use App\Models\User; +use Illuminate\Database\Eloquent\Model; use Livewire\Attributes\Locked; use Livewire\Component; use Spatie\Activitylog\Models\Activity; @@ -79,41 +81,37 @@ class ActivityMonitor extends Component return; } - $currentTeamId = currentTeam()?->id; + $this->activity = $this->canReadActivity($activity) ? $activity : null; + } - // Check team_id stored directly in activity properties - $activityTeamId = data_get($activity, 'properties.team_id'); - if ($activityTeamId !== null) { - if ((int) $activityTeamId !== (int) $currentTeamId) { - $this->activity = null; + /** + * Activity output can contain secrets, so only users who can edit the resource that the + * activity ran for (or its server) may read it. The policy uses the team of that resource, + * not the session team. + */ + private function canReadActivity(Activity $activity): bool + { + $user = auth()->user(); + if (! $user) { + return false; + } - return; - } + $subject = $activity->subject; + if ($subject instanceof Model) { + return $user->can('update', $subject); + } - $this->activity = $activity; - - return; + // Service starts store the service uuid instead of a subject. + $typeUuid = data_get($activity, 'properties.type_uuid'); + if (filled($typeUuid) && ($service = Service::where('uuid', $typeUuid)->first())) { + return $user->can('update', $service); } - // Fallback: verify ownership via the server that ran the command $serverUuid = data_get($activity, 'properties.server_uuid'); - if ($serverUuid) { - $server = Server::where('uuid', $serverUuid)->first(); - if ($server && (int) $server->team_id !== (int) $currentTeamId) { - $this->activity = null; + $server = filled($serverUuid) ? Server::where('uuid', $serverUuid)->first() : null; - return; - } - - if ($server) { - $this->activity = $activity; - - return; - } - } - - // Fail closed: no team_id and no server_uuid means we cannot verify ownership - $this->activity = null; + // Fail closed: an activity without a resource or server cannot be authorized. + return $server !== null && $user->can('update', $server); } public function polling() diff --git a/app/Livewire/Analytics.php b/app/Livewire/Analytics.php index 007618501c..62c3aab048 100644 --- a/app/Livewire/Analytics.php +++ b/app/Livewire/Analytics.php @@ -56,8 +56,7 @@ class Analytics extends Component #[Url(as: 'app')] public string $appUuid = ''; - // Realtime refresh; off by default (click "Live" to arm it). Only meaningful on the - // 24h range, which matches the 60s Sentinel cache TTL. + // Realtime refresh every 60s (the Sentinel cache TTL); off by default, works with every range. public bool $live = false; public ?array $overview = null; @@ -186,12 +185,11 @@ class Analytics extends Component $this->loadData(); } - public function toggleLive(): void + public function updatedLive(): void { - if ($this->range !== '24h') { - return; + if ($this->live) { + $this->loadData(); } - $this->live = ! $this->live; } #[On('trafficAnalyticsStateChanged')] @@ -211,11 +209,6 @@ class Analytics extends Component } } - public function isLivePollable(): bool - { - return $this->live && $this->range === '24h'; - } - public function updatedServerUuid(): void { // Scope the app options to the newly selected server and drop an app filter diff --git a/app/Livewire/Dashboard/ServerMetricsChart.php b/app/Livewire/Dashboard/ServerMetricsChart.php index 83748052ce..f2ba0df17c 100644 --- a/app/Livewire/Dashboard/ServerMetricsChart.php +++ b/app/Livewire/Dashboard/ServerMetricsChart.php @@ -22,6 +22,7 @@ class ServerMetricsChart extends Component } try { + $this->server->prefetchMetrics(['cpu', 'memory'], 10); $this->dispatch("dashboard-server-metrics-{$this->server->uuid}", [ 'cpu' => $this->server->getCpuMetrics(10), 'memory' => $this->server->getMemoryMetrics(10), diff --git a/app/Livewire/Fleet/Metrics.php b/app/Livewire/Fleet/Metrics.php index bcf6917499..a8e11ee0fe 100644 --- a/app/Livewire/Fleet/Metrics.php +++ b/app/Livewire/Fleet/Metrics.php @@ -11,6 +11,7 @@ use Illuminate\Contracts\View\View; use Illuminate\Foundation\Auth\Access\AuthorizesRequests; use Illuminate\Support\Collection; use Illuminate\Support\Facades\Concurrency; +use Illuminate\Support\Facades\Log; use Livewire\Attributes\Lazy; use Livewire\Attributes\Url; use Livewire\Component; @@ -196,9 +197,11 @@ class Metrics extends Component $this->loadData(); } - public function isLivePollable(): bool + public function updatedLive(): void { - return $this->live && $this->range === '24h'; + if ($this->live) { + $this->loadData(); + } } /** @@ -226,6 +229,10 @@ class Metrics extends Component $clients = []; $summaryTasks = []; foreach ($targets as $server) { + // Coolify does not connect to an unreachable server, and its SSH retries would hold a worker past the process timeout. + if (! $server->isFunctional()) { + continue; + } $client = $clients[$server->uuid] = $this->metricsClient($server); $summaryTasks[$server->uuid] = static fn () => $client->summary(); } @@ -245,21 +252,21 @@ class Metrics extends Component $results = $this->runMetricTasks($tasks); foreach ($targets as $server) { - $summary = $summaries[$server->uuid]; + $summary = $summaries[$server->uuid] ?? null; foreach (['cpu', 'memory', 'disk', 'load'] as $metric) { $series[$metric][] = $results[$server->uuid.':'.$metric] ?? []; } if ($summary === null) { - $rows[] = $this->offlineRow($server); + $rows[] = $this->rowWithoutMetrics($server); continue; } $current = $results[$server->uuid.':containers'] ?? []; $rows[] = array_merge( - ['uuid' => $server->uuid, 'name' => $server->name, 'online' => true, 'containers' => count($current)], - $summary + $summary, + ['uuid' => $server->uuid, 'name' => $server->name, 'online' => true, 'reporting' => true, 'containers' => count($current)] ); foreach ($current as $c) { @@ -301,19 +308,28 @@ class Metrics extends Component } $results = []; foreach (array_chunk($tasks, 8, true) as $batch) { - $results += Concurrency::run($batch); + try { + $results += Concurrency::run($batch); + } catch (\Throwable $e) { + // A worker that runs past the process timeout fails its batch. Show those metrics as missing, not a server error. + Log::warning('Fleet metrics batch failed', ['error' => $e->getMessage()]); + $results += array_fill_keys(array_keys($batch), null); + } } return $results; } /** + * A server whose Sentinel returned no current metrics. It is online when Coolify can reach it, + * because a failed metrics request does not mean that the server is down. + * * @return array */ - protected function offlineRow(Server $server): array + protected function rowWithoutMetrics(Server $server): array { return [ - 'uuid' => $server->uuid, 'name' => $server->name, 'online' => false, + 'uuid' => $server->uuid, 'name' => $server->name, 'online' => (bool) $server->isFunctional(), 'reporting' => false, 'cpu' => null, 'memUsed' => null, 'memTotal' => null, 'diskUsed' => null, 'diskTotal' => null, 'diskPercent' => null, 'load1' => null, 'netRx' => null, 'netTx' => null, 'containers' => null, diff --git a/app/Livewire/GlobalSearch.php b/app/Livewire/GlobalSearch.php index df1840d615..cd2b9b0134 100644 --- a/app/Livewire/GlobalSearch.php +++ b/app/Livewire/GlobalSearch.php @@ -688,7 +688,7 @@ class GlobalSearch extends Component 'search_text' => 'destinations docker networks', ], [ - 'name' => 'Security', + 'name' => 'Keys & Tokens', 'type' => 'navigation', 'description' => 'Manage private keys and API tokens', 'link' => route('security.private-key.index'), diff --git a/app/Livewire/Profile/Index.php b/app/Livewire/Profile/Index.php index e782de176a..825d6a27c5 100644 --- a/app/Livewire/Profile/Index.php +++ b/app/Livewire/Profile/Index.php @@ -151,7 +151,7 @@ class Index extends Component } // Additional rate limit by IP address (5 requests per hour) - $ipKey = 'email-change:ip:'.request()->ip(); + $ipKey = 'email-change:ip:'.client_ip(request()); if (! RateLimiter::attempt($ipKey, 5, function () {}, 3600)) { $this->dispatch('error', 'Too many requests from your IP address. Please try again later.'); diff --git a/app/Livewire/Project/Application/Advanced.php b/app/Livewire/Project/Application/Advanced.php index 0075bdddd4..3f3d95573a 100644 --- a/app/Livewire/Project/Application/Advanced.php +++ b/app/Livewire/Project/Application/Advanced.php @@ -68,6 +68,8 @@ class Advanced extends Component #[Validate(['string', 'nullable'])] public ?string $stopGracePeriod = null; + public ?int $rollingUpdateDrainSeconds = DEFAULT_ROLLING_UPDATE_DRAIN_SECONDS; + #[Validate(['boolean'])] public bool $isBuildServerEnabled = false; @@ -170,6 +172,7 @@ class Advanced extends Component // Load stop_grace_period separately since it has its own save handler // Convert null to empty string to prevent dirty detection issues $this->stopGracePeriod = $this->application->settings->stop_grace_period ?? ''; + $this->rollingUpdateDrainSeconds = $this->application->settings->rollingUpdateDrainSeconds(); } private function syncCertificateResolverFromModel(): void @@ -432,6 +435,30 @@ class Advanced extends Component } } + public function saveRollingUpdateDrainSeconds() + { + try { + $this->authorize('update', $this->application); + + $validated = $this->validate( + ['rollingUpdateDrainSeconds' => ['required', 'integer', 'min:0', 'max:'.MAX_ROLLING_UPDATE_DRAIN_SECONDS]], + [], + ['rollingUpdateDrainSeconds' => 'rolling update drain time'] + ); + + $this->application->settings->rolling_update_drain_seconds = $validated['rollingUpdateDrainSeconds']; + $changedFields = array_keys($this->application->settings->getDirty()); + $this->application->settings->save(); + $this->auditSettingsUpdate($changedFields); + + $this->dispatch('success', 'Rolling update drain time updated.'); + } catch (ValidationException $e) { + throw $e; + } catch (\Throwable $e) { + return handleError($e, $this); + } + } + public function saveMaxRestartCount() { try { diff --git a/app/Livewire/Project/Application/Deployment/Index.php b/app/Livewire/Project/Application/Deployment/Index.php index 1c8bc27ca3..08194f6bfc 100644 --- a/app/Livewire/Project/Application/Deployment/Index.php +++ b/app/Livewire/Project/Application/Deployment/Index.php @@ -7,6 +7,7 @@ use App\Models\Application; use App\Models\ApplicationDeploymentQueue; use App\Traits\ListensToTeamChannel; use Illuminate\Support\Collection; +use Livewire\Attributes\On; use Livewire\Component; class Index extends Component @@ -68,18 +69,24 @@ class Index extends Component ]); } - public function mount() + #[On('deploymentSelected')] + public function setSelectedDeployment(string $deploymentUuid): void { - $project = currentTeam()->load(['projects'])->projects->where('uuid', request()->route('project_uuid'))->first(); + $this->selectedDeploymentUuid = $deploymentUuid; + } + + public function mount(string $project_uuid, string $environment_uuid, string $application_uuid) + { + $project = currentTeam()->load(['projects'])->projects->where('uuid', $project_uuid)->first(); if (! $project) { return redirect()->route('dashboard'); } - $environment = $project->load(['environments'])->environments->where('uuid', request()->route('environment_uuid'))->first(); + $environment = $project->load(['environments'])->environments->where('uuid', $environment_uuid)->first(); if (! $environment) { abort(404); } $environment->load(['applications']); - $application = $environment->applications->where('uuid', request()->route('application_uuid'))->first(); + $application = $environment->applications->where('uuid', $application_uuid)->first(); if (! $application) { return redirect()->route('dashboard'); } diff --git a/app/Livewire/Project/Application/Deployment/Show.php b/app/Livewire/Project/Application/Deployment/Show.php index 338a15d023..5227784b6d 100644 --- a/app/Livewire/Project/Application/Deployment/Show.php +++ b/app/Livewire/Project/Application/Deployment/Show.php @@ -4,6 +4,7 @@ namespace App\Livewire\Project\Application\Deployment; use App\Models\Application; use App\Models\ApplicationDeploymentQueue; +use App\Models\EnvironmentVariable; use App\Traits\AuditsApplicationSettings; use Livewire\Component; @@ -34,20 +35,20 @@ class Show extends Component ]; } - public function mount() + public function mount(string $project_uuid, string $environment_uuid, string $application_uuid, string $deployment_uuid) { - $deploymentUuid = request()->route('deployment_uuid'); + $deploymentUuid = $deployment_uuid; - $project = currentTeam()->load(['projects'])->projects->where('uuid', request()->route('project_uuid'))->first(); + $project = currentTeam()->load(['projects'])->projects->where('uuid', $project_uuid)->first(); if (! $project) { return redirect()->route('dashboard'); } - $environment = $project->load(['environments'])->environments->where('uuid', request()->route('environment_uuid'))->first(); + $environment = $project->load(['environments'])->environments->where('uuid', $environment_uuid)->first(); if (! $environment) { abort(404); } $environment->load(['applications']); - $application = $environment->applications->where('uuid', request()->route('application_uuid'))->first(); + $application = $environment->applications->where('uuid', $application_uuid)->first(); if (! $application) { return redirect()->route('dashboard'); } @@ -71,6 +72,34 @@ class Show extends Component $this->isKeepAliveOn(); } + /** + * Swap the logs viewer to another deployment of this application without a page load. + */ + public function selectDeployment(string $deploymentUuid): void + { + $deployment = ApplicationDeploymentQueue::where('deployment_uuid', $deploymentUuid) + ->where('application_id', $this->application->id) + ->first(); + if (! $deployment) { + return; + } + + $this->application_deployment_queue = $deployment; + $this->deployment_uuid = $deploymentUuid; + $this->horizon_job_status = $deployment->getHorizonJobStatus(); + $this->isKeepAliveOn(); + + $this->dispatch('deploymentSelected', deploymentUuid: $deploymentUuid)->to(Index::class); + + $url = route('project.application.deployment.show', [ + 'project_uuid' => $this->application->environment->project->uuid, + 'environment_uuid' => $this->application->environment->uuid, + 'application_uuid' => $this->application->uuid, + 'deployment_uuid' => $deploymentUuid, + ]); + $this->js('history.replaceState(history.state, "", '.json_encode($url).')'); + } + public function toggleDebug() { try { @@ -116,25 +145,57 @@ class Show extends Component return decode_remote_command_output($this->application_deployment_queue); } + /** + * The displayed log lines that match the search, with secrets hidden. + */ + public function exportLogs(bool $withTimestamps = true, string $search = ''): string + { + $this->authorize('view', $this->application); + + $search = mb_strtolower(trim($search)); + $lines = $this->logLines + ->map(fn (array $line): array => [ + 'timestamp' => $line['timestamp'], + 'text' => $this->exportLineText($line), + ]) + ->filter(fn (array $line): bool => $search === '' || str_contains(mb_strtolower($line['timestamp'].' '.$line['text']), $search)) + ->map(fn (array $line): string => $withTimestamps ? $line['timestamp'].' '.$line['text'] : $line['text']) + ->join("\n"); + + return sanitizeLogsForExport($lines, $this->logRedactionSecrets()); + } + + /** + * Every log line, also the debug lines, with secrets hidden. + */ public function downloadAllLogs(): string { $this->authorize('update', $this->application); $logs = decode_remote_command_output($this->application_deployment_queue, includeAll: true) - ->map(function ($line) { - $prefix = ''; - if ($line['hidden']) { - $prefix = '[DEBUG] '; - } - if (isset($line['command']) && $line['command']) { - $prefix .= '[CMD]: '; - } - - return $line['timestamp'].' '.$prefix.trim($line['line']); - }) + ->map(fn (array $line): string => $line['timestamp'].' '.($line['hidden'] ? '[DEBUG] ' : '').$this->exportLineText($line)) ->join("\n"); - return sanitizeLogsForExport($logs); + return sanitizeLogsForExport($logs, $this->logRedactionSecrets()); + } + + /** + * @param array{line: string, command?: bool|null} $line + */ + private function exportLineText(array $line): string + { + return (! empty($line['command']) ? '[CMD]: ' : '').trim($line['line']); + } + + /** + * @return array{known: list, locked: list} + */ + private function logRedactionSecrets(): array + { + return EnvironmentVariable::logRedactionSecretsFor( + $this->application, + includePreview: (int) $this->application_deployment_queue->pull_request_id !== 0, + ); } public function render() diff --git a/app/Livewire/Project/Application/Domains.php b/app/Livewire/Project/Application/Domains.php index bd4768e260..118ba4461c 100644 --- a/app/Livewire/Project/Application/Domains.php +++ b/app/Livewire/Project/Application/Domains.php @@ -78,7 +78,10 @@ class Domains extends Component public ?string $editingGeneratedHost = null; - /** @var array */ + /** The internal port the edited domain uses when its Port field is empty. */ + public ?int $editingInheritedPort = null; + + /** @var array */ #[Locked] public array $domainRows = []; @@ -561,7 +564,7 @@ class Domains extends Component /** * @param array $stored - * @return array{url: string, service: ?string, dns_status: string, dns_message: string, expected_ip: ?string, checked_at: ?string, is_suggested: bool, suggested_for: ?string, suggestion_label: ?string, needs_force_add: bool, internal_port: ?int, has_port_override: bool} + * @return array{url: string, service: ?string, dns_status: string, dns_message: string, expected_ip: ?string, checked_at: ?string, is_suggested: bool, suggested_for: ?string, suggestion_label: ?string, needs_force_add: bool, internal_port: ?int, has_port_override: bool, inherited_port?: ?int} */ protected function domainRowFromStored(string $url, ?string $service, array $stored): array { @@ -574,6 +577,7 @@ class Domains extends Component 'service' => $service, 'internal_port' => $port['internal_port'], 'has_port_override' => $port['has_port_override'], + 'inherited_port' => $this->inheritedDomainInternalPort($service), 'dns_status' => 'pending', 'dns_message' => 'Not checked yet.', 'expected_ip' => $this->serverIp, @@ -621,37 +625,35 @@ class Domains extends Component ]; } + return [ + 'internal_port' => $this->inheritedDomainInternalPort($service), + 'has_port_override' => false, + ]; + } + + /** + * The internal port a domain uses when it has no per-domain override. + */ + protected function inheritedDomainInternalPort(?string $service = null): ?int + { $composePort = dockerComposeServicePort($this->application->docker_compose_raw, $service); if ($composePort !== null) { - return [ - 'internal_port' => $composePort, - 'has_port_override' => false, - ]; + return $composePort; } if ($this->isCompose && $service !== null) { - return [ - 'internal_port' => null, - 'has_port_override' => false, - ]; + return null; } if ($this->application->settings?->is_static) { - return [ - 'internal_port' => 80, - 'has_port_override' => false, - ]; + return 80; } $exposed = $this->application->ports_exposes_array; - $defaultPort = isset($exposed[0]) && is_numeric($exposed[0]) && (int) $exposed[0] > 0 + + return isset($exposed[0]) && is_numeric($exposed[0]) && (int) $exposed[0] > 0 ? (int) $exposed[0] : null; - - return [ - 'internal_port' => $defaultPort, - 'has_port_override' => false, - ]; } /** @@ -1342,6 +1344,7 @@ class Domains extends Component $this->editingOriginalRedirect = $this->editingRedirect; $this->editingDomainWasRegenerated = false; $this->editingGeneratedHost = null; + $this->editingInheritedPort = $this->domainRows[$index]['inherited_port'] ?? null; $this->resetEditDomainDnsGate(); $this->resetErrorBag('editingDomain'); $this->showEditDomainModal = true; @@ -1432,6 +1435,7 @@ class Domains extends Component $this->editingOriginalRedirect = 'both'; $this->editingDomainWasRegenerated = false; $this->editingGeneratedHost = null; + $this->editingInheritedPort = null; $this->resetEditDomainDnsGate(); $this->resetErrorBag('editingDomain'); if ($this->pendingAction === 'update') { diff --git a/app/Livewire/Project/Application/General.php b/app/Livewire/Project/Application/General.php index 0a1fcdfc3d..09a634c681 100644 --- a/app/Livewire/Project/Application/General.php +++ b/app/Livewire/Project/Application/General.php @@ -137,6 +137,8 @@ class General extends Component public bool $initLoadingCompose = false; + public bool $networkingModalOpen = false; + public ?string $initialDockerComposeLocation = null; public ?Collection $parsedServices; @@ -153,6 +155,7 @@ class General extends Component 'resetDefaultLabels', 'configurationChanged' => '$refresh', 'confirmDomainUsage', + 'composeFileLoaded' => 'syncLoadedComposeFile', ]; protected function rules(): array @@ -320,6 +323,8 @@ class General extends Component $this->parsedServices = collect([ 'services' => collect(data_get($this->parsedServices, 'services', []))->map(fn () => []), ]); + } elseif (isset($this->parsedServices)) { + $this->parsedServices = redactLockedValuesInGeneratedCompose($this->parsedServices, $this->application); } if ($this->application->build_pack === 'dockercompose') { // Only update if user has permission @@ -409,7 +414,7 @@ class General extends Component $this->application->docker_registry_image_name = $this->dockerRegistryImageName; $this->application->docker_registry_image_tag = $this->dockerRegistryImageTag; $this->application->docker_compose_location = $this->dockerComposeLocation; - $this->application->docker_compose = $this->dockerCompose; + // docker_compose is generated by the parser; the property holds a redacted display copy. $this->application->docker_compose_raw = $this->dockerComposeRaw; $this->application->docker_compose_custom_start_command = $this->dockerComposeCustomStartCommand; $this->application->docker_compose_custom_build_command = $this->dockerComposeCustomBuildCommand; @@ -466,7 +471,7 @@ class General extends Component $this->dockerRegistryImageTag = $this->application->docker_registry_image_tag; $this->dockerComposeLocation = $this->application->docker_compose_location; $canViewCompose = auth()->user()?->can('update', $this->application) ?? false; - $this->dockerCompose = $canViewCompose ? $this->application->docker_compose : null; + $this->dockerCompose = $canViewCompose ? redactLockedValuesInGeneratedCompose($this->application->docker_compose, $this->application) : null; $this->dockerComposeRaw = $canViewCompose ? $this->application->docker_compose_raw : null; $this->dockerComposeCustomStartCommand = $this->application->docker_compose_custom_start_command; $this->dockerComposeCustomBuildCommand = $this->application->docker_compose_custom_build_command; @@ -571,17 +576,7 @@ class General extends Component return; } - - // Refresh parsedServiceDomains to reflect any changes in docker_compose_domains - $this->application->refresh(); - - // Sync the docker_compose_raw from the model to the component property - // This ensures the Monaco editor displays the loaded compose file - $this->syncData(); - - $this->parsedServiceDomains = $this->application->docker_compose_domains ? json_decode($this->application->docker_compose_domains, true) : []; - // Convert service names with dots and dashes to use underscores for HTML form binding - $this->parsedServiceDomains = $this->sanitizeParsedServiceDomainsForForm($this->parsedServiceDomains); + $this->showLoadedComposeFile(); $showToast && $this->dispatch('success', 'Docker compose file loaded.'); $this->dispatch('compose_loaded'); @@ -600,6 +595,34 @@ class General extends Component } } + /** + * Shows a Compose file that the heading loaded while this form is open. + */ + public function syncLoadedComposeFile(): void + { + $this->authorize('update', $this->application); + + $this->application->refresh(); + $this->parsedServices = $this->application->parse(); + $this->showLoadedComposeFile(); + } + + private function showLoadedComposeFile(): void + { + $this->parsedServices = redactLockedValuesInGeneratedCompose($this->parsedServices, $this->application); + + // Refresh parsedServiceDomains to reflect any changes in docker_compose_domains + $this->application->refresh(); + + // Sync the docker_compose_raw from the model to the component property + // This ensures the Monaco editor displays the loaded compose file + $this->syncData(); + + $this->parsedServiceDomains = $this->application->docker_compose_domains ? json_decode($this->application->docker_compose_domains, true) : []; + // Convert service names with dots and dashes to use underscores for HTML form binding + $this->parsedServiceDomains = $this->sanitizeParsedServiceDomainsForForm($this->parsedServiceDomains); + } + public function generateDomain(string $serviceName) { try { @@ -820,6 +843,7 @@ class General extends Component $this->validate(); $oldPortsExposes = $this->application->ports_exposes; + $oldCustomNetworkAliases = $this->application->custom_network_aliases; $oldIsContainerLabelEscapeEnabled = $this->application->settings->is_container_label_escape_enabled; $oldDockerComposeLocation = $this->initialDockerComposeLocation; $oldBaseDirectory = $this->application->base_directory; @@ -951,9 +975,10 @@ class General extends Component $this->application->refresh(); $dnsCleanup->queueReleaseOfRemovedHostnames($this->application, $previousDnsHostnames, $this->application->team()->id); $this->syncData(); - if ($oldPortsExposes !== $this->portsExposes) { + if ($oldPortsExposes !== $this->application->ports_exposes || $oldCustomNetworkAliases !== $this->application->custom_network_aliases) { $this->dispatch('applicationNetworkingUpdated')->to(InternalAccess::class); } + $this->networkingModalOpen = false; $showToaster && ! $warning && $this->dispatch('success', 'Application settings updated!'); } catch (\Throwable $e) { $this->application->refresh(); diff --git a/app/Livewire/Project/Application/Heading.php b/app/Livewire/Project/Application/Heading.php index 7c3aac02cd..9b40240c19 100644 --- a/app/Livewire/Project/Application/Heading.php +++ b/app/Livewire/Project/Application/Heading.php @@ -4,10 +4,13 @@ namespace App\Livewire\Project\Application; use App\Actions\Application\StopApplication; use App\Actions\Docker\GetContainersStatus; +use App\Livewire\Project\Service\Storage; +use App\Livewire\Project\Shared\EnvironmentVariable\All; use App\Models\Application; use App\Models\ApplicationDeploymentQueue; use App\Traits\ListensToTeamChannel; use Illuminate\Foundation\Auth\Access\AuthorizesRequests; +use Livewire\Attributes\Locked; use Livewire\Component; class Heading extends Component @@ -29,14 +32,23 @@ class Heading extends Component public bool $docker_cleanup = true; + /** + * Set once when the General page opens, where the user sees the Compose file location. Other pages + * only show the button, and a later switch to the Compose build pack waits for the user. + */ + #[Locked] + public bool $loadComposeOnOpen = false; + public function getListeners() { return [ 'compose_loaded' => '$refresh', + 'configurationChanged' => '$refresh', 'update_links' => '$refresh', ...$this->teamChannelListeners([ 'ServiceStatusChanged' => 'checkStatus', 'ServiceChecked' => '$refresh', + 'ApplicationStopFailed' => 'stopFailed', ]), ]; } @@ -52,6 +64,41 @@ class Heading extends Component $lastDeployment = $this->application->get_last_successful_deployment(); $this->lastDeploymentInfo = data_get_str($lastDeployment, 'commit')->limit(7).' '.data_get($lastDeployment, 'commit_message'); $this->lastDeploymentLink = $this->application->gitCommitLink(data_get($lastDeployment, 'commit')); + $this->loadComposeOnOpen = $this->activeRouteName === 'project.application.configuration' + && $this->needsComposeFile() + && auth()->user()?->can('update', $this->application); + } + + public function needsComposeFile(): bool + { + return $this->application->build_pack === 'dockercompose' && is_null($this->application->docker_compose_raw); + } + + public function loadComposeFile(bool $isInit = false) + { + try { + $this->authorize('update', $this->application); + + if ($isInit && ! $this->needsComposeFile()) { + return; + } + + $loaded = $this->application->loadComposeFile(); + if (is_null(data_get($loaded, 'parsedServices'))) { + $this->dispatch('error', 'Failed to parse your docker-compose file. Please check the syntax and try again.'); + + return; + } + + $this->dispatch('success', 'Docker compose file loaded.'); + $this->dispatch('composeFileLoaded')->to(General::class); + $this->dispatch('storageCountsChanged')->to(Storage::class); + $this->dispatch('refreshEnvs')->to(All::class); + } catch (\Throwable $e) { + return handleError($e, $this); + } finally { + $this->application->refresh(); + } } /** @@ -77,6 +124,20 @@ class Heading extends Component } } + /** + * @param array{applicationUuid?: mixed, message?: mixed} $event + */ + public function stopFailed(array $event): void + { + if (($event['applicationUuid'] ?? null) !== $this->application->uuid || ! is_string($event['message'] ?? null)) { + $this->skipRender(); + + return; + } + + $this->dispatch('error', 'Failed to stop the application.', $event['message']); + } + public function manualCheckStatus() { $this->checkStatus(); @@ -111,7 +172,7 @@ class Heading extends Component try { $this->authorize('deploy', $this->application); - if ($this->application->build_pack === 'dockercompose' && is_null($this->application->docker_compose_raw)) { + if ($this->needsComposeFile()) { $this->dispatch('error', 'Failed to deploy', 'Please load a Compose file first.'); return; diff --git a/app/Livewire/Project/Application/PreviewDomains.php b/app/Livewire/Project/Application/PreviewDomains.php index c3a287f8b9..89b9c4fb5b 100644 --- a/app/Livewire/Project/Application/PreviewDomains.php +++ b/app/Livewire/Project/Application/PreviewDomains.php @@ -28,6 +28,15 @@ class PreviewDomains extends Component public ?string $newDomainService = null; + /** + * Parsed once on mount and on save. Each parse queues a file sync job, so the + * 2-second DNS poll must not parse the compose file on every render. + * + * @var array + */ + #[Locked] + public array $composeServices = []; + public array $editingDomainParts = ['scheme' => 'https', 'host' => '', 'port' => '', 'path' => '']; public ?int $editingIndex = null; @@ -45,7 +54,8 @@ class PreviewDomains extends Component $this->authorize('view', $this->preview->application); $this->refreshDomains(); if ($this->preview->application->build_pack === 'dockercompose') { - $this->newDomainService = $this->composeServices()[0] ?? null; + $this->composeServices = $this->composeServices(); + $this->newDomainService = $this->composeServices[0] ?? null; } } @@ -53,7 +63,6 @@ class PreviewDomains extends Component { return view('livewire.project.application.preview-domains', [ 'isCompose' => $this->preview->application->build_pack === 'dockercompose', - 'composeServices' => $this->composeServices(), ]); } @@ -61,7 +70,7 @@ class PreviewDomains extends Component { $this->authorize('update', $this->preview->application); if ($this->preview->application->build_pack === 'dockercompose' - && ($this->newDomainService === null || ! in_array($this->newDomainService, $this->composeServices(), true))) { + && ($this->newDomainService === null || ! in_array($this->newDomainService, $this->composeServices, true))) { $this->addError('newDomainService', 'Select a valid Compose service.'); return; @@ -96,7 +105,7 @@ class PreviewDomains extends Component $domain = $this->domainRows[$index]['url'] ?? DomainPortOverrides::withoutPort($domain); $this->newDomainParts = DomainUrlParts::empty(); $this->newDomainService = $this->preview->application->build_pack === 'dockercompose' - ? ($this->composeServices()[0] ?? null) + ? ($this->composeServices[0] ?? null) : null; $this->forceUseUnknownPort = false; $this->dispatch('close-preview-domain-add', previewId: $this->preview->id); @@ -455,6 +464,7 @@ class PreviewDomains extends Component if ($this->preview->application->build_pack === 'dockercompose') { try { $composeServices = $this->composeServices(failOnError: true); + $this->composeServices = $composeServices; } catch (\Throwable) { $this->refreshDomains(); $this->dispatch('error', 'Compose configuration could not be parsed. Preview domains were not changed.'); diff --git a/app/Livewire/Project/Application/Previews.php b/app/Livewire/Project/Application/Previews.php index edcd09e05c..7de6552b94 100644 --- a/app/Livewire/Project/Application/Previews.php +++ b/app/Livewire/Project/Application/Previews.php @@ -80,8 +80,8 @@ class Previews extends Component private function syncDockerTags(): void { $this->previewDockerTags = []; - foreach ($this->application->previews as $key => $preview) { - $this->previewDockerTags[$key] = $preview->docker_registry_image_tag; + foreach ($this->application->previews as $preview) { + $this->previewDockerTags[$preview->id] = $preview->docker_registry_image_tag; } } @@ -118,16 +118,8 @@ class Previews extends Component throw new \Exception('Preview not found'); } - $previewKey = $this->application->previews->search(function ($item) use ($preview_id) { - return $item->id == $preview_id; - }); - - if ($previewKey === false) { - throw new \Exception('Preview not found'); - } - - $this->validateOnly("previewDockerTags.{$previewKey}"); - $preview->docker_registry_image_tag = $this->previewDockerTags[$previewKey] ?? null; + $this->validateOnly("previewDockerTags.{$preview->id}"); + $preview->docker_registry_image_tag = $this->previewDockerTags[$preview->id] ?? null; $changedFields = $preview->isDirty('docker_registry_image_tag') ? ['docker_registry_image_tag'] : []; $preview->save(); if ($changedFields !== []) { @@ -155,7 +147,7 @@ class Previews extends Component ]); $this->auditPreviewEvent('ui.application.preview_created', $pull_request_id); } - $found->generate_preview_fqdn_compose(); + $found->generate_preview_fqdn_compose(keepExistingDomains: ! $found->wasRecentlyCreated); $this->application->refresh(); $this->syncDockerTags(); } else { @@ -173,7 +165,7 @@ class Previews extends Component if ($found && $this->application->build_pack === 'dockerimage' && str($docker_registry_image_tag)->isNotEmpty()) { $this->saveDockerTagWithAudit($found, $docker_registry_image_tag); } - $found->generate_preview_fqdn(generateWithoutApplicationDomain: true); + $found->generate_preview_fqdn(generateWithoutApplicationDomain: true, keepExistingDomains: true); $this->application->refresh(); $this->syncDockerTags(); $this->dispatch('update_links'); @@ -385,6 +377,7 @@ class Previews extends Component // Refresh the application and its previews relationship to reflect the soft delete $this->application->load('previews'); + unset($this->previewDockerTags[$preview->id]); $this->dispatch('update_links'); $this->dispatch('success', 'Preview deletion started. It may take a few moments to complete.'); } catch (\Throwable $e) { diff --git a/app/Livewire/Project/CloneMe.php b/app/Livewire/Project/CloneMe.php index b310c248ee..99cacd1a93 100644 --- a/app/Livewire/Project/CloneMe.php +++ b/app/Livewire/Project/CloneMe.php @@ -4,8 +4,6 @@ namespace App\Livewire\Project; use App\Actions\Database\StartDatabase; use App\Actions\Database\StopDatabase; -use App\Actions\Service\StartService; -use App\Actions\Service\StopService; use App\Jobs\VolumeCloneJob; use App\Models\Environment; use App\Models\Project; @@ -299,182 +297,10 @@ class CloneMe extends Component } foreach ($services as $service) { - $uuid = new_public_id(); - $newService = $service->replicate([ - 'id', - 'created_at', - 'updated_at', - ])->fill([ - 'uuid' => $uuid, + clone_service($service, $selectedDestination, [ + 'name' => $service->name, 'environment_id' => $environment->id, - 'destination_id' => $selectedDestination->id, - 'destination_type' => $selectedDestination->getMorphClass(), - 'server_id' => $selectedDestination->server_id, - ]); - $newService->save(); - - $tags = $service->tags; - foreach ($tags as $tag) { - $newService->tags()->attach($tag->id); - } - - $scheduledTasks = $service->scheduled_tasks()->get(); - foreach ($scheduledTasks as $task) { - $newTask = $task->replicate([ - 'id', - 'created_at', - 'updated_at', - ])->fill([ - 'uuid' => new_public_id(), - 'service_id' => $newService->id, - 'team_id' => $teamId, - ]); - $newTask->save(); - } - - $environmentVariables = $service->environment_variables()->get(); - foreach ($environmentVariables as $environmentVariable) { - $newEnvironmentVariable = $environmentVariable->replicate([ - 'id', - 'created_at', - 'updated_at', - ])->fill([ - 'resourceable_id' => $newService->id, - 'resourceable_type' => $newService->getMorphClass(), - ]); - $newEnvironmentVariable->save(); - } - - $service->cloneSecretManagerLinkTo($newService); - - foreach ($newService->applications() as $application) { - $application->fill([ - 'status' => 'exited', - ])->save(); - - $persistentVolumes = $application->persistentStorages()->get(); - foreach ($persistentVolumes as $volume) { - $newName = ''; - if (str_starts_with($volume->name, $application->uuid)) { - $newName = str($volume->name)->replace($application->uuid, $application->uuid); - } else { - $newName = $application->uuid.'-'.$volume->name; - } - - $newPersistentVolume = $volume->replicate([ - 'id', - 'created_at', - 'updated_at', - 'uuid', - ])->fill([ - 'name' => $newName, - 'resource_id' => $application->id, - ]); - $newPersistentVolume->save(); - - if ($this->cloneVolumeData) { - try { - StopService::dispatch($application); - $sourceVolume = $volume->name; - $targetVolume = $newPersistentVolume->name; - $sourceServer = $application->service->destination->server; - $targetServer = $newService->destination->server; - - VolumeCloneJob::dispatch($sourceVolume, $targetVolume, $sourceServer, $targetServer, $newPersistentVolume); - - StartService::dispatch($application); - } catch (\Exception $e) { - \Log::error('Failed to copy volume data for '.$volume->name.': '.$e->getMessage()); - } - } - } - - $fileStorages = $application->fileStorages()->get(); - foreach ($fileStorages as $storage) { - $newStorage = $storage->replicate([ - 'id', - 'created_at', - 'updated_at', - ])->fill([ - 'resource_id' => $application->id, - ]); - $newStorage->save(); - } - } - - foreach ($newService->databases() as $database) { - $database->fill([ - 'status' => 'exited', - ])->save(); - - $persistentVolumes = $database->persistentStorages()->get(); - foreach ($persistentVolumes as $volume) { - $newName = ''; - if (str_starts_with($volume->name, $database->uuid)) { - $newName = str($volume->name)->replace($database->uuid, $database->uuid); - } else { - $newName = $database->uuid.'-'.$volume->name; - } - - $newPersistentVolume = $volume->replicate([ - 'id', - 'created_at', - 'updated_at', - 'uuid', - ])->fill([ - 'name' => $newName, - 'resource_id' => $database->id, - ]); - $newPersistentVolume->save(); - - if ($this->cloneVolumeData) { - try { - StopService::dispatch($database->service); - $sourceVolume = $volume->name; - $targetVolume = $newPersistentVolume->name; - $sourceServer = $database->service->destination->server; - $targetServer = $newService->destination->server; - - VolumeCloneJob::dispatch($sourceVolume, $targetVolume, $sourceServer, $targetServer, $newPersistentVolume); - - StartService::dispatch($database->service); - } catch (\Exception $e) { - \Log::error('Failed to copy volume data for '.$volume->name.': '.$e->getMessage()); - } - } - } - - $fileStorages = $database->fileStorages()->get(); - foreach ($fileStorages as $storage) { - $newStorage = $storage->replicate([ - 'id', - 'created_at', - 'updated_at', - ])->fill([ - 'resource_id' => $database->id, - ]); - $newStorage->save(); - } - - $scheduledBackups = $database->scheduledBackups()->get(); - foreach ($scheduledBackups as $backup) { - $uuid = new_public_id(); - $newBackup = $backup->replicate([ - 'id', - 'created_at', - 'updated_at', - ])->fill([ - 'uuid' => $uuid, - 'database_id' => $database->id, - 'database_type' => $database->getMorphClass(), - 'team_id' => $teamId, - ]); - $newBackup->save(); - $backup->copyS3StoragesTo($newBackup); - } - } - - $newService->parse(); + ], $this->cloneVolumeData); } return [$project, $environment]; diff --git a/app/Livewire/Project/Database/Clickhouse/General.php b/app/Livewire/Project/Database/Clickhouse/General.php index 2cd0bfecee..68452e78af 100644 --- a/app/Livewire/Project/Database/Clickhouse/General.php +++ b/app/Livewire/Project/Database/Clickhouse/General.php @@ -73,7 +73,7 @@ class General extends Component return handleError($e, $this); } - $this->isPasswordHiddenForMember = auth()->user()?->isMember() ?? false; + $this->isPasswordHiddenForMember = ! (auth()->user()?->can('update', $this->database) ?? false); } protected function rules(): array diff --git a/app/Livewire/Project/Database/Dragonfly/General.php b/app/Livewire/Project/Database/Dragonfly/General.php index b3c5bcc945..3c6b27e306 100644 --- a/app/Livewire/Project/Database/Dragonfly/General.php +++ b/app/Livewire/Project/Database/Dragonfly/General.php @@ -65,7 +65,7 @@ class General extends Component return handleError($e, $this); } - $this->isPasswordHiddenForMember = auth()->user()?->isMember() ?? false; + $this->isPasswordHiddenForMember = ! (auth()->user()?->can('update', $this->database) ?? false); } protected function rules(): array diff --git a/app/Livewire/Project/Database/Keydb/General.php b/app/Livewire/Project/Database/Keydb/General.php index fd84bad209..1e159d2f9b 100644 --- a/app/Livewire/Project/Database/Keydb/General.php +++ b/app/Livewire/Project/Database/Keydb/General.php @@ -67,7 +67,7 @@ class General extends Component return handleError($e, $this); } - $this->isPasswordHiddenForMember = auth()->user()?->isMember() ?? false; + $this->isPasswordHiddenForMember = ! (auth()->user()?->can('update', $this->database) ?? false); } protected function rules(): array @@ -129,8 +129,8 @@ class General extends Component } else { $this->name = $this->database->name; $this->description = $this->database->description; - $this->keydbConf = $this->database->keydb_conf; $canSeeCredentials = auth()->user()?->can('update', $this->database) ?? false; + $this->keydbConf = $canSeeCredentials ? $this->database->keydb_conf : null; $this->keydbPassword = $canSeeCredentials ? $this->database->keydb_password : ''; $this->image = $this->database->image; $this->portsMappings = $this->database->ports_mappings; diff --git a/app/Livewire/Project/Database/Mariadb/General.php b/app/Livewire/Project/Database/Mariadb/General.php index 747389522d..d65136e9f0 100644 --- a/app/Livewire/Project/Database/Mariadb/General.php +++ b/app/Livewire/Project/Database/Mariadb/General.php @@ -129,7 +129,7 @@ class General extends Component return handleError($e, $this); } - $this->isPasswordHiddenForMember = auth()->user()?->isMember() ?? false; + $this->isPasswordHiddenForMember = ! (auth()->user()?->can('update', $this->database) ?? false); } private function syncData(bool $toModel = false): void @@ -159,7 +159,7 @@ class General extends Component $this->mariadbUser = $this->database->mariadb_user; $this->mariadbPassword = $canSeeCredentials ? $this->database->mariadb_password : ''; $this->mariadbDatabase = $this->database->mariadb_database; - $this->mariadbConf = $this->database->mariadb_conf; + $this->mariadbConf = $canSeeCredentials ? $this->database->mariadb_conf : null; $this->image = $this->database->image; $this->portsMappings = $this->database->ports_mappings; $this->isPublic = $this->database->is_public; diff --git a/app/Livewire/Project/Database/Mongodb/General.php b/app/Livewire/Project/Database/Mongodb/General.php index 517a3e5a42..02357a5581 100644 --- a/app/Livewire/Project/Database/Mongodb/General.php +++ b/app/Livewire/Project/Database/Mongodb/General.php @@ -122,7 +122,7 @@ class General extends Component return handleError($e, $this); } - $this->isPasswordHiddenForMember = auth()->user()?->isMember() ?? false; + $this->isPasswordHiddenForMember = ! (auth()->user()?->can('update', $this->database) ?? false); } private function syncData(bool $toModel = false): void @@ -146,9 +146,9 @@ class General extends Component } else { $this->name = $this->database->name; $this->description = $this->database->description; - $this->mongoConf = $this->database->mongo_conf; - $this->mongoInitdbRootUsername = $this->database->mongo_initdb_root_username; $canSeeCredentials = auth()->user()?->can('update', $this->database) ?? false; + $this->mongoConf = $canSeeCredentials ? $this->database->mongo_conf : null; + $this->mongoInitdbRootUsername = $this->database->mongo_initdb_root_username; $this->mongoInitdbRootPassword = $canSeeCredentials ? $this->database->mongo_initdb_root_password : ''; $this->mongoInitdbDatabase = $this->database->mongo_initdb_database; $this->image = $this->database->image; diff --git a/app/Livewire/Project/Database/Mysql/General.php b/app/Livewire/Project/Database/Mysql/General.php index ddac8ab829..fd3abfc218 100644 --- a/app/Livewire/Project/Database/Mysql/General.php +++ b/app/Livewire/Project/Database/Mysql/General.php @@ -129,7 +129,7 @@ class General extends Component return handleError($e, $this); } - $this->isPasswordHiddenForMember = auth()->user()?->isMember() ?? false; + $this->isPasswordHiddenForMember = ! (auth()->user()?->can('update', $this->database) ?? false); } private function syncData(bool $toModel = false): void @@ -159,7 +159,7 @@ class General extends Component $this->mysqlUser = $this->database->mysql_user; $this->mysqlPassword = $canSeeCredentials ? $this->database->mysql_password : ''; $this->mysqlDatabase = $this->database->mysql_database; - $this->mysqlConf = $this->database->mysql_conf; + $this->mysqlConf = $canSeeCredentials ? $this->database->mysql_conf : null; $this->image = $this->database->image; $this->portsMappings = $this->database->ports_mappings; $this->isPublic = $this->database->is_public; diff --git a/app/Livewire/Project/Database/Postgresql/General.php b/app/Livewire/Project/Database/Postgresql/General.php index 3d11fa6b7a..30e03d7630 100644 --- a/app/Livewire/Project/Database/Postgresql/General.php +++ b/app/Livewire/Project/Database/Postgresql/General.php @@ -143,7 +143,7 @@ class General extends Component return handleError($e, $this); } - $this->isPasswordHiddenForMember = auth()->user()?->isMember() ?? false; + $this->isPasswordHiddenForMember = ! (auth()->user()?->can('update', $this->database) ?? false); } private function syncData(bool $toModel = false): void @@ -176,8 +176,8 @@ class General extends Component $this->postgresDb = $this->database->postgres_db; $this->postgresInitdbArgs = $this->database->postgres_initdb_args; $this->postgresHostAuthMethod = $this->database->postgres_host_auth_method; - $this->postgresConf = $this->database->postgres_conf; - $this->initScripts = $this->database->init_scripts; + $this->postgresConf = $canSeeCredentials ? $this->database->postgres_conf : null; + $this->initScripts = $canSeeCredentials ? $this->database->init_scripts : null; $this->image = $this->database->image; $this->portsMappings = $this->database->ports_mappings; $this->isPublic = $this->database->is_public; diff --git a/app/Livewire/Project/Database/Redis/General.php b/app/Livewire/Project/Database/Redis/General.php index 3ab0d3eda9..5c96bef77a 100644 --- a/app/Livewire/Project/Database/Redis/General.php +++ b/app/Livewire/Project/Database/Redis/General.php @@ -42,12 +42,14 @@ class General extends Component public string $redisUsername; - public string $redisPassword; + public ?string $redisPassword = null; public string $redisVersion; public bool $isPasswordHiddenForMember = false; + public bool $isPasswordLocked = false; + protected $listeners = [ 'envsUpdated' => 'refresh', ]; @@ -68,9 +70,11 @@ class General extends Component 'redisUsername' => ValidationPatterns::databaseIdentifierRules( enforcePattern: $this->redisUsername !== $this->database->redis_username, ), - 'redisPassword' => ValidationPatterns::databasePasswordRules( - enforcePattern: $this->redisPassword !== $this->database->redis_password, - ), + 'redisPassword' => $this->database->isCredentialLocked() + ? ['nullable'] + : ValidationPatterns::databasePasswordRules( + enforcePattern: $this->redisPassword !== $this->database->redis_password, + ), ]; } @@ -122,7 +126,7 @@ class General extends Component return handleError($e, $this); } - $this->isPasswordHiddenForMember = auth()->user()?->isMember() ?? false; + $this->isPasswordHiddenForMember = ! (auth()->user()?->can('update', $this->database) ?? false); } private function syncData(bool $toModel = false): void @@ -143,7 +147,6 @@ class General extends Component } else { $this->name = $this->database->name; $this->description = $this->database->description; - $this->redisConf = $this->database->redis_conf; $this->image = $this->database->image; $this->portsMappings = $this->database->ports_mappings; $this->isPublic = $this->database->is_public; @@ -154,7 +157,9 @@ class General extends Component $this->redisVersion = $this->database->getRedisVersion(); $this->redisUsername = $this->database->redis_username; $canSeeCredentials = auth()->user()?->can('update', $this->database) ?? false; - $this->redisPassword = $canSeeCredentials ? $this->database->redis_password : ''; + $this->redisConf = $canSeeCredentials ? $this->database->redis_conf : null; + $this->isPasswordLocked = $this->database->isCredentialLocked(); + $this->redisPassword = $canSeeCredentials && ! $this->isPasswordLocked ? $this->database->redis_password : null; } } @@ -193,10 +198,13 @@ class General extends Component ['value' => $this->redisUsername, 'resourceable_id' => $this->database->id] ); } - $this->database->runtime_environment_variables()->updateOrCreate( - ['key' => 'REDIS_PASSWORD'], - ['value' => $this->redisPassword, 'resourceable_id' => $this->database->id] - ); + // The form never loads a locked password, so saving its empty value would overwrite it. + if (! $this->database->isCredentialLocked()) { + $this->database->runtime_environment_variables()->updateOrCreate( + ['key' => 'REDIS_PASSWORD'], + ['value' => $this->redisPassword, 'resourceable_id' => $this->database->id] + ); + } $this->dispatch('success', 'Database updated.'); $this->dispatch('databaseUpdated'); diff --git a/app/Livewire/Project/New/DockerCompose.php b/app/Livewire/Project/New/DockerCompose.php index 2fb883d6b2..524d04173e 100644 --- a/app/Livewire/Project/New/DockerCompose.php +++ b/app/Livewire/Project/New/DockerCompose.php @@ -2,10 +2,13 @@ namespace App\Livewire\Project\New; +use App\Models\Environment; use App\Models\EnvironmentVariable; use App\Models\Project; use App\Models\Service; use Illuminate\Foundation\Auth\Access\AuthorizesRequests; +use Illuminate\Support\Facades\DB; +use Livewire\Attributes\Locked; use Livewire\Component; class DockerCompose extends Component @@ -20,6 +23,9 @@ class DockerCompose extends Component public array $query; + #[Locked] + public ?string $createdServiceUuid = null; + public function mount() { $this->parameters = get_route_parameters(); @@ -35,7 +41,10 @@ class DockerCompose extends Component $this->validate([ 'dockerComposeRaw' => 'required', ]); - parseDockerComposeYaml($this->dockerComposeRaw); + $services = data_get(parseDockerComposeYaml($this->dockerComposeRaw), 'services'); + if (! is_array($services) || $services === [] || array_is_list($services)) { + throw new \Exception('The Docker Compose file must define at least one service under "services".'); + } // Validate for command injection BEFORE saving to database validateDockerComposeForInjection($this->dockerComposeRaw); @@ -51,43 +60,57 @@ class DockerCompose extends Component } $destination_class = $destination->getMorphClass(); - $service = new Service([ - 'docker_compose_raw' => $this->dockerComposeRaw, - 'environment_id' => $environment->id, - 'server_id' => $destination->server_id, - 'destination_id' => $destination->id, - 'destination_type' => $destination_class, - ]); - $service->save(); - - $variables = parseEnvFormatToArray($this->envFile); - foreach ($variables as $key => $data) { - // Extract value and comment from parsed data - // Handle both array format ['value' => ..., 'comment' => ...] and plain string values - $value = is_array($data) ? ($data['value'] ?? '') : $data; - $comment = is_array($data) ? ($data['comment'] ?? null) : null; - - EnvironmentVariable::create([ - 'key' => $key, - 'value' => $value, - 'comment' => $comment, - 'is_preview' => false, - 'resourceable_id' => $service->id, - 'resourceable_type' => $service->getMorphClass(), - ]); + if ($this->createdServiceUuid) { + return $this->redirectToService($this->createdServiceUuid, $project, $environment); } - $service->parse(isNew: true); - // Apply service-specific application prerequisites - applyServiceApplicationPrerequisites($service); + $service = DB::transaction(function () use ($environment, $destination, $destination_class) { + $service = new Service([ + 'docker_compose_raw' => $this->dockerComposeRaw, + 'environment_id' => $environment->id, + 'server_id' => $destination->server_id, + 'destination_id' => $destination->id, + 'destination_type' => $destination_class, + ]); + $service->save(); - return redirect()->route('project.service.configuration', [ - 'service_uuid' => $service->uuid, - 'environment_uuid' => $environment->uuid, - 'project_uuid' => $project->uuid, - ]); + $variables = parseEnvFormatToArray($this->envFile); + foreach ($variables as $key => $data) { + // Extract value and comment from parsed data + // Handle both array format ['value' => ..., 'comment' => ...] and plain string values + $value = is_array($data) ? ($data['value'] ?? '') : $data; + $comment = is_array($data) ? ($data['comment'] ?? null) : null; + + EnvironmentVariable::create([ + 'key' => $key, + 'value' => $value, + 'comment' => $comment, + 'is_preview' => false, + 'resourceable_id' => $service->id, + 'resourceable_type' => $service->getMorphClass(), + ]); + } + $service->parse(isNew: true); + + // Apply service-specific application prerequisites + applyServiceApplicationPrerequisites($service); + + return $service; + }); + $this->createdServiceUuid = $service->uuid; + + return $this->redirectToService($service->uuid, $project, $environment); } catch (\Throwable $e) { return handleError($e, $this); } } + + private function redirectToService(string $serviceUuid, Project $project, Environment $environment): mixed + { + return redirect()->route('project.service.configuration', [ + 'service_uuid' => $serviceUuid, + 'environment_uuid' => $environment->uuid, + 'project_uuid' => $project->uuid, + ]); + } } diff --git a/app/Livewire/Project/New/Select.php b/app/Livewire/Project/New/Select.php index 9ada4fd0fb..342cb1c942 100644 --- a/app/Livewire/Project/New/Select.php +++ b/app/Livewire/Project/New/Select.php @@ -28,9 +28,9 @@ class Select extends Component public bool $onlyBuildServerAvailable = false; - public ?Collection $standaloneDockers; + public ?Collection $standaloneDockers = null; - public ?Collection $swarmDockers; + public ?Collection $swarmDockers = null; public array $parameters; @@ -79,11 +79,14 @@ class Select extends Component $queryDestination = request()->query('destination'); if ($queryType === 'postgresql' && $queryServerId !== null && $queryDestination) { - $this->type = $queryType; - $this->server_id = $queryServerId; - $this->destination_uuid = $queryDestination; - $this->server = Server::ownedByCurrentTeam()->find($queryServerId); - $this->current_step = 'select-postgresql-type'; + $server = Server::isUsable()->whereKey($queryServerId)->first(); + if ($server && $this->findDestination($server, $queryDestination)) { + $this->type = $queryType; + $this->server_id = $server->id; + $this->destination_uuid = $queryDestination; + $this->server = $server; + $this->current_step = 'select-postgresql-type'; + } } } catch (\Exception $e) { return handleError($e, $this); @@ -433,20 +436,23 @@ class Select extends Component if (count($this->servers) === 1 && $this->buildServers?->isEmpty()) { $server = $this->servers->first(); if ($server instanceof Server) { - $this->setServer($server); + $this->setServer($server->id); } } if (! is_null($this->server)) { $foundServer = $this->servers->where('id', $this->server->id)->first(); if ($foundServer) { - return $this->setServer($foundServer); + return $this->setServer($foundServer->id); } } $this->current_step = 'servers'; } - public function setServer(Server $server) + public function setServer(int $serverId) { + $server = $this->servers?->firstWhere('id', $serverId); + abort_unless($server instanceof Server, 404); + $this->server_id = $server->id; $this->server = $server; $this->standaloneDockers = $server->standaloneDockers; @@ -465,11 +471,18 @@ class Select extends Component public function setDestination(string $destination_uuid) { + abort_unless($this->server && $this->findDestination($this->server, $destination_uuid), 404); + $this->destination_uuid = $destination_uuid; return $this->whatToDoNext(); } + private function findDestination(Server $server, string $destinationUuid): ?object + { + return $server->standaloneDockers->concat($server->swarmDockers)->firstWhere('uuid', $destinationUuid); + } + public function setPostgresqlType(string $type) { $this->postgresql_type = $type; diff --git a/app/Livewire/Project/Service/EditCompose.php b/app/Livewire/Project/Service/EditCompose.php index b52d5a63dc..a8652c8509 100644 --- a/app/Livewire/Project/Service/EditCompose.php +++ b/app/Livewire/Project/Service/EditCompose.php @@ -54,12 +54,11 @@ class EditCompose extends Component { if ($toModel) { $this->service->docker_compose_raw = $this->dockerComposeRaw; - $this->service->docker_compose = $this->dockerCompose; $this->service->is_container_label_escape_enabled = $this->isContainerLabelEscapeEnabled; } else { $canViewCompose = auth()->user()?->can('update', $this->service) ?? false; $this->dockerComposeRaw = $canViewCompose ? $this->service->docker_compose_raw : null; - $this->dockerCompose = $canViewCompose ? $this->service->docker_compose : null; + $this->dockerCompose = $canViewCompose ? redactLockedValuesInGeneratedCompose($this->service->docker_compose, $this->service) : null; $this->isContainerLabelEscapeEnabled = $this->service->is_container_label_escape_enabled ?? false; } } diff --git a/app/Livewire/Project/Service/StackForm.php b/app/Livewire/Project/Service/StackForm.php index 9c55156489..9247049d6d 100644 --- a/app/Livewire/Project/Service/StackForm.php +++ b/app/Livewire/Project/Service/StackForm.php @@ -45,6 +45,9 @@ class StackForm extends Component // Add dynamic field rules foreach ($this->fields ?? collect() as $key => $field) { + if (data_get($field, 'isLocked')) { + continue; + } $rules = data_get($field, 'rules', 'nullable'); $baseRules["fields.$key.value"] = $rules; } @@ -78,7 +81,7 @@ class StackForm extends Component $this->service->name = $this->name; $this->service->description = $this->description; $this->service->docker_compose_raw = $this->dockerComposeRaw; - $this->service->docker_compose = $this->dockerCompose; + // docker_compose is generated by parse(); the property holds a redacted display copy. $this->service->connect_to_docker_network = $this->connectToDockerNetwork; } else { // Sync FROM model (on load/refresh) @@ -86,7 +89,7 @@ class StackForm extends Component $this->description = $this->service->description; $canViewCompose = auth()->user()?->can('update', $this->service) ?? false; $this->dockerComposeRaw = $canViewCompose ? $this->service->docker_compose_raw : null; - $this->dockerCompose = $canViewCompose ? $this->service->docker_compose : null; + $this->dockerCompose = $canViewCompose ? redactLockedValuesInGeneratedCompose($this->service->docker_compose, $this->service) : null; $this->connectToDockerNetwork = $this->service->connect_to_docker_network; } } @@ -128,16 +131,30 @@ class StackForm extends Component return $group; }); - $this->isPasswordHiddenForMember = auth()->user()?->isMember() ?? false; - if ($this->isPasswordHiddenForMember) { - $this->fields = $this->fields->map(function ($field) { - if (data_get($field, 'isPassword')) { - $field['value'] = null; - } + $this->isPasswordHiddenForMember = ! (auth()->user()?->can('manageEnvironment', $this->service) ?? false); + $lockedKeys = $this->lockedFieldKeys(); + $this->fields = $this->fields->map(function ($field) use ($lockedKeys) { + $field['isLocked'] = $lockedKeys->contains(data_get($field, 'key')); + if ($field['isLocked'] || ($this->isPasswordHiddenForMember && data_get($field, 'isPassword'))) { + $field['value'] = null; + } - return $field; - }); - } + return $field; + }); + } + + /** + * Keys of the extra fields that are locked (shown once), directly or through a referenced variable. + */ + private function lockedFieldKeys(): Collection + { + $referencingLockedKeys = collect($this->service->extraFields()) + ->flatMap(fn ($fields) => collect($fields)) + ->filter(fn ($field) => data_get($field, 'isLocked')) + ->pluck('key'); + + return $this->service->environment_variables()->where('is_shown_once', true)->pluck('key') + ->merge($referencingLockedKeys); } public function saveCompose($raw) @@ -173,7 +190,10 @@ class StackForm extends Component // Use transaction to ensure atomicity - if parse fails, save is rolled back DB::transaction(function () { $this->service->save(); - $this->service->saveExtraFields($this->fields); + $lockedKeys = $this->lockedFieldKeys(); + $this->service->saveExtraFields( + $this->fields->reject(fn ($field) => $lockedKeys->contains(data_get($field, 'key'))) + ); $this->service->parse(); }); // Refresh and write files after a successful commit diff --git a/app/Livewire/Project/Shared/Danger.php b/app/Livewire/Project/Shared/Danger.php index 0865974828..14b2881cf3 100644 --- a/app/Livewire/Project/Shared/Danger.php +++ b/app/Livewire/Project/Shared/Danger.php @@ -115,6 +115,8 @@ class Danger extends Component try { $this->authorize('delete', $this->resource); + // Soft delete first, so the environment page after the redirect does not list the resource. + $this->resource->delete(); DeleteResourceJob::dispatch( $this->resource, $this->delete_volumes, @@ -132,32 +134,6 @@ class Danger extends Component } } - public function deleteFromCoolifyOnly(string $password): mixed - { - if (! verifyPasswordConfirmation($password, $this)) { - return 'The provided password is incorrect.'; - } - - if (! $this->resource instanceof Service) { - return 'Service not found.'; - } - - try { - $this->authorize('delete', $this->resource); - DeleteResourceJob::dispatch( - resource: $this->resource, - deleteFromCoolifyOnly: true, - )->afterResponse(); - - return redirectRoute($this, 'project.resource.index', [ - 'project_uuid' => $this->projectUuid, - 'environment_uuid' => $this->environmentUuid, - ]); - } catch (\Throwable $e) { - return handleError($e, $this); - } - } - public function flush($password) { try { diff --git a/app/Livewire/Project/Shared/EnvironmentVariable/All.php b/app/Livewire/Project/Shared/EnvironmentVariable/All.php index 55b529fd9b..1591fca861 100644 --- a/app/Livewire/Project/Shared/EnvironmentVariable/All.php +++ b/app/Livewire/Project/Shared/EnvironmentVariable/All.php @@ -69,6 +69,8 @@ class All extends Component private ?Collection $composeEnvironmentAssignments = null; + private ?bool $canViewComposeContent = null; + protected $listeners = [ 'saveKey' => 'submit', 'refreshEnvs', @@ -897,10 +899,24 @@ class All extends Component /** @return array{services: list, default: ?string, operator: ?string, required: bool}|null */ private function composeInfoFor(string $key): ?array { - return dockerComposeEnvironmentVariableInfo( + $composeInfo = dockerComposeEnvironmentVariableInfo( $this->resource->docker_compose_raw ?? $this->resource->docker_compose, $key ); + + if ($composeInfo !== null && ! $this->canViewComposeContent()) { + $composeInfo['default'] = null; + } + + return $composeInfo; + } + + /** + * Compose defaults are compose content, which only users who can update the resource may see. + */ + private function canViewComposeContent(): bool + { + return $this->canViewComposeContent ??= auth()->user()?->can('update', $this->resource) ?? false; } private function composeEnvironmentAssignments(): Collection diff --git a/app/Livewire/Project/Shared/EnvironmentVariable/Show.php b/app/Livewire/Project/Shared/EnvironmentVariable/Show.php index fb0fa4a772..a19693a4a5 100644 --- a/app/Livewire/Project/Shared/EnvironmentVariable/Show.php +++ b/app/Livewire/Project/Shared/EnvironmentVariable/Show.php @@ -129,6 +129,9 @@ class Show extends Component public function mount() { + if ($this->composeInfo !== null && ! (auth()->user()?->can('update', $this->resource) ?? false)) { + $this->composeInfo['default'] = null; + } $this->syncData(); if ($this->env->getMorphClass() === SharedEnvironmentVariable::class) { $this->isSharedVariable = true; @@ -224,11 +227,14 @@ class Show extends Component $this->env->is_shared = $this->is_shared; } $this->env->key = $this->key; - $this->env->value = $this->value; + // The form never holds a locked value, so keep the stored one and keep the lock. + if (! $this->env->is_shown_once) { + $this->env->value = $this->value; + $this->env->is_shown_once = $this->is_shown_once; + } $this->env->comment = $this->comment; $this->env->is_multiline = $this->is_multiline; $this->env->is_literal = $this->is_literal; - $this->env->is_shown_once = $this->is_shown_once; $this->env->save(); $this->valuesLoaded = true; } else { @@ -356,9 +362,11 @@ class Show extends Component } } - if (! $this->isSharedVariable && $this->is_required && str($this->value)->isEmpty()) { - $oldValue = $this->env->getOriginal('value'); - $this->value = $oldValue; + $submittedValue = $this->env->is_shown_once ? $this->env->value : $this->value; + if (! $this->isSharedVariable && $this->is_required && str($submittedValue)->isEmpty()) { + if (! $this->env->is_shown_once) { + $this->value = $this->env->getOriginal('value'); + } $this->dispatch('error', 'Required environment variables cannot be empty.'); return; diff --git a/app/Livewire/Project/Shared/FileBrowser.php b/app/Livewire/Project/Shared/FileBrowser.php index 09f8f0c6b9..e634f9755d 100644 --- a/app/Livewire/Project/Shared/FileBrowser.php +++ b/app/Livewire/Project/Shared/FileBrowser.php @@ -2,6 +2,7 @@ namespace App\Livewire\Project\Shared; +use App\Exceptions\ContainerShellUnavailableException; use App\Http\Controllers\FileBrowserDownloadController; use App\Jobs\PrepareFileDownloadJob; use App\Models\Application; @@ -14,6 +15,7 @@ use Illuminate\Contracts\View\View; use Illuminate\Database\Eloquent\Model; use Illuminate\Foundation\Auth\Access\AuthorizesRequests; use Illuminate\Support\Facades\Cache; +use Illuminate\Support\Facades\Log; use Illuminate\Support\Facades\RateLimiter; use Illuminate\Support\Str; use Livewire\Attributes\Locked; @@ -44,6 +46,17 @@ class FileBrowser extends Component #[Locked] public bool $loadFailed = false; + /** The container has no usable shell, so a helper container reaches its files. */ + #[Locked] + public bool $useHelper = false; + + #[Locked] + public bool $shellUnavailable = false; + + /** Why the helper container failed, shown when the reason is known. */ + #[Locked] + public ?string $helperError = null; + #[Locked] public ?string $container = null; @@ -103,6 +116,7 @@ class FileBrowser extends Component try { $this->availableContainers = $this->containerRunning ? $this->resolveContainers() : []; $this->container = $this->availableContainers[0] ?? null; + $this->useHelper = false; if (is_null($this->container)) { $this->containerRunning = false; } else { @@ -125,6 +139,7 @@ class FileBrowser extends Component } $this->closeEditor(); $this->container = $name; + $this->useHelper = false; $this->currentPath = $this->service()->defaultRoot(); $this->loadEntries(); } @@ -132,6 +147,9 @@ class FileBrowser extends Component public function open(string $name): void { $this->guard(); + if ($this->isHelperSymlink($name)) { + return; + } $this->currentPath = $this->childPath($name); $this->loadEntries(); } @@ -215,6 +233,9 @@ class FileBrowser extends Component public function openEditor(string $name): void { $this->guard(); + if ($this->isHelperSymlink($name)) { + return; + } $path = $this->childPath($name); try { // read() guards binary/oversized files internally and throws. @@ -276,6 +297,9 @@ class FileBrowser extends Component public function download(string $name): void { $this->guard(); + if ($this->isHelperSymlink($name)) { + return; + } try { $path = $this->childPath($name); $target = $this->service()->downloadInfo($path); @@ -297,7 +321,7 @@ class FileBrowser extends Component } $downloadId = (string) Str::uuid(); $this->pendingDownloads[] = $downloadId; - PrepareFileDownloadJob::dispatch($downloadId, auth()->id(), $this->server()->id, $this->container, $path, $target['isDir'], $downloadName, $s3Storage->id); + PrepareFileDownloadJob::dispatch($downloadId, auth()->id(), $this->server()->id, $this->container, $path, $target['isDir'], $downloadName, $s3Storage->id, $this->useHelper); $this->dispatch('info', 'Preparing '.$downloadName.'. The download starts when it is ready.'); return; @@ -313,6 +337,7 @@ class FileBrowser extends Component 'path' => $path, 'name' => $downloadName, 'is_dir' => $target['isDir'], + 'use_helper' => $this->useHelper, ], FileBrowserDownloadController::TOKEN_TTL_SECONDS); $this->redirect(route('project.files.download', $token)); @@ -442,7 +467,24 @@ class FileBrowser extends Component protected function loadEntries(): void { - $entries = $this->service()->list($this->currentPath); + $this->shellUnavailable = false; + $this->helperError = null; + try { + $entries = $this->service()->list($this->currentPath); + } catch (ContainerShellUnavailableException $e) { + if (! $this->useHelper) { + $this->useHelper = true; + $this->loadEntries(); + + return; + } + Log::warning('File browser helper container failed', ['container' => $this->container, 'error' => $e->getMessage()]); + $entries = []; + $this->shellUnavailable = true; + if (preg_match('/Unable to find image|failed to resolve reference|pull access denied|manifest unknown/i', $e->getMessage())) { + $this->helperError = 'The server could not pull the Coolify helper image ('.coolifyHelperImage().':'.getHelperVersion().'). Check that the server can reach the registry.'; + } + } $this->entries = array_map(fn ($e) => [ 'name' => $e->name, 'type' => $e->type, @@ -456,7 +498,22 @@ class FileBrowser extends Component protected function service(): ContainerFilesystemService { - return new ContainerFilesystemService($this->server(), $this->container); + return new ContainerFilesystemService($this->server(), $this->container, $this->useHelper); + } + + /** + * The helper resolves absolute symlinks in its own file system, not in the + * container, so it would show the wrong files. It cannot change them, + * because its root file system is read-only. + */ + protected function isHelperSymlink(string $name): bool + { + if (! $this->useHelper || ! collect($this->entries)->contains(fn (array $entry) => $entry['name'] === $name && $entry['type'] === 'symlink')) { + return false; + } + $this->dispatch('error', 'Symlinks cannot be opened in a container without a shell. Open the target path instead.'); + + return true; } protected function server(): Server diff --git a/app/Livewire/Project/Shared/GetLogs.php b/app/Livewire/Project/Shared/GetLogs.php index f9b15afac4..cabf0bc469 100644 --- a/app/Livewire/Project/Shared/GetLogs.php +++ b/app/Livewire/Project/Shared/GetLogs.php @@ -20,6 +20,7 @@ use App\Models\StandaloneRedis; use App\Models\StandaloneSqlite; use App\Support\ValidationPatterns; use Illuminate\Foundation\Auth\Access\AuthorizesRequests; +use Illuminate\Support\Collection; use Illuminate\Support\Facades\Process; use Livewire\Attributes\Locked; use Livewire\Attributes\Renderless; @@ -52,6 +53,14 @@ class GetLogs extends Component #[Locked] public ?string $container = null; + /** + * Containers of one service. Empty keeps the single-container log command. + * + * @var list + */ + #[Locked] + public array $sources = []; + public ?string $displayName = null; public ?string $pull_request = null; @@ -66,6 +75,8 @@ class GetLogs extends Component public bool $collapsible = true; + public bool $fill = false; + public function mount() { if (! is_null($this->resource)) { @@ -149,24 +160,17 @@ class GetLogs extends Component * holds a Docker timestamp, only lines from that moment onward are returned. */ #[Renderless] - public function getLogs(?string $since = null): string + public function getLogs(mixed $since = null): string { - if (! $this->canReadLogs()) { - return 'Unauthorized.'; - } - if (! Server::ownedByCurrentTeam()->where('id', $this->server->id)->exists()) { - return 'Unauthorized.'; - } - if (! $this->server->isFunctional()) { - return ''; - } - if ($this->container && ! ValidationPatterns::isValidContainerName($this->container)) { - return 'Invalid container name.'; - } - if (! $this->container) { - return ''; + if ($this->sources !== []) { + return $this->mergedLogs(is_array($since) ? $since : null, self::MAX_DISPLAY_SIZE_BYTES, '5MB', export: false); } + if (! $this->canReadSingleContainer()) { + return $this->singleContainerDenial(); + } + + $since = is_string($since) ? $since : null; if (is_string($since) && preg_match(self::DOCKER_TIMESTAMP_PATTERN, $since)) { $options = "--since {$since} -t"; } else { @@ -178,16 +182,13 @@ class GetLogs extends Component public function copyLogs(): string { - if (! $this->canReadLogs()) { - return ''; + if ($this->sources !== []) { + return sanitizeLogsForExport( + $this->mergedLogs(null, self::MAX_DISPLAY_SIZE_BYTES, '5MB', export: false, stripTimestamps: ! $this->showTimeStamps), + $this->logRedactionSecrets(), + ); } - if (! Server::ownedByCurrentTeam()->where('id', $this->server->id)->exists()) { - return ''; - } - if (! $this->server->isFunctional() || ! $this->container) { - return ''; - } - if (! ValidationPatterns::isValidContainerName($this->container)) { + if (! $this->canReadSingleContainer()) { return ''; } @@ -198,16 +199,10 @@ class GetLogs extends Component public function downloadAllLogs(): string { - if (! $this->canReadLogs()) { - return ''; + if ($this->sources !== []) { + return sanitizeLogsForExport($this->mergedLogs(null, self::MAX_DOWNLOAD_SIZE_BYTES, '50MB', export: true, all: true), $this->logRedactionSecrets()); } - if (! Server::ownedByCurrentTeam()->where('id', $this->server->id)->exists()) { - return ''; - } - if (! $this->server->isFunctional() || ! $this->container) { - return ''; - } - if (! ValidationPatterns::isValidContainerName($this->container)) { + if (! $this->canReadSingleContainer()) { return ''; } @@ -216,6 +211,39 @@ class GetLogs extends Component return sanitizeLogsForExport($this->runLogCommand($command, self::MAX_DOWNLOAD_SIZE_BYTES, '50MB', $this->showTimeStamps), $this->logRedactionSecrets()); } + /** + * One container keeps the previous denial text. An invalid name is distinct from a missing permission. + */ + private function singleContainerDenial(): string + { + if (! $this->canReadLogs() || ! Server::ownedByCurrentTeam()->where('id', $this->server->id)->exists()) { + return 'Unauthorized.'; + } + if (! $this->server->isFunctional() || ! $this->container) { + return ''; + } + if (! ValidationPatterns::isValidContainerName($this->container)) { + return 'Invalid container name.'; + } + + return ''; + } + + private function canReadSingleContainer(): bool + { + if (! $this->canReadLogs()) { + return false; + } + if (! Server::ownedByCurrentTeam()->where('id', $this->server->id)->exists()) { + return false; + } + if (! $this->server->isFunctional() || ! $this->container) { + return false; + } + + return ValidationPatterns::isValidContainerName($this->container); + } + /** * Environment values and database credentials of the resource whose logs are exported. * @@ -228,6 +256,191 @@ class GetLogs extends Component return EnvironmentVariable::logRedactionSecretsFor($resource, includePreview: $resource instanceof Application && filled($this->pull_request)); } + /** + * Read every container of one service and return one timestamp-ordered stream. + * A source that is not one of this resource's servers, or whose container name + * is not safe to pass to the shell, rejects the whole read before any command runs. + * + * @param array|null $since + */ + private function mergedLogs(?array $since, int $maxBytes, string $limitLabel, bool $export, bool $all = false, ?bool $stripTimestamps = null): string + { + $sources = $this->authorizedSources(); + if ($sources === null) { + return $export ? '' : 'Unauthorized.'; + } + if ($sources === []) { + return ''; + } + + $tail = $this->resolveLogTail(); + $groups = []; + $truncated = false; + foreach ($sources as $source) { + $sinceValue = $this->dockerSince($since, $source['label']); + if ($all) { + $options = '-t'; + } elseif ($sinceValue !== null) { + $options = "--since {$sinceValue} -t"; + } else { + $options = "-n {$tail} -t"; + } + $fetched = $this->executeLogCommand( + $source['server'], + $this->logCommand($options, $source['server'], $source['container']), + $maxBytes, + ); + $truncated = $truncated || $fetched['truncated']; + $groups[] = ['label' => $source['label'], 'logs' => $fetched['logs']]; + } + + $stripTimestamps ??= $export && ! $this->showTimeStamps; + $merged = $this->joinMergedLines($groups, $stripTimestamps, $export); + if (strlen($merged) > $maxBytes) { + $merged = rtrim(substr($merged, 0, $maxBytes), "\n"); + $truncated = true; + } + if ($truncated && $merged !== '') { + $merged .= "\n\n[... Output truncated at {$limitLabel} limit ...]"; + } + + return $merged; + } + + /** + * @param array|null $since + */ + private function dockerSince(?array $since, string $label): ?string + { + $value = $since[$label] ?? null; + if (! is_string($value) || preg_match(self::DOCKER_TIMESTAMP_PATTERN, $value) !== 1) { + return null; + } + + return $value; + } + + /** + * @param list $groups + */ + private function joinMergedLines(array $groups, bool $stripTimestamp, bool $export): string + { + $rows = []; + foreach ($groups as $group) { + if ($group['logs'] === '') { + continue; + } + foreach (explode("\n", $group['logs']) as $line) { + if (trim($line) === '') { + continue; + } + $rows[] = [ + 'timestamp' => explode(' ', $line, 2)[0], + 'label' => $group['label'], + 'line' => $line, + ]; + } + } + + usort($rows, function (array $left, array $right): int { + $byTime = $left['timestamp'] <=> $right['timestamp']; + + return $byTime !== 0 ? $byTime : strcmp($left['label'], $right['label']); + }); + + $lines = []; + foreach ($rows as $row) { + $text = $stripTimestamp ? $this->stripDockerTimestamp($row['line']) : $row['line']; + $lines[] = $export ? $row['label'].' | '.$text : $row['label']."\t".$text; + } + + return implode("\n", $lines); + } + + private function stripDockerTimestamp(string $line): string + { + if (preg_match('/^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(?:\.\d{1,9})?Z (.*)$/s', $line, $matches) === 1) { + return $matches[1]; + } + + return $line; + } + + /** + * @return list|null + */ + private function authorizedSources(): ?array + { + if (! $this->canReadLogs() || $this->resource === null) { + return null; + } + + $teamId = $this->resourceTeamId(); + if ($teamId === null) { + return null; + } + + $allowed = $this->serversForResource()->keyBy(fn (Server $server): string => (string) $server->uuid); + $resolved = []; + $labels = []; + foreach ($this->sources as $source) { + if (! is_array($source)) { + return null; + } + $uuid = (string) ($source['server'] ?? ''); + $container = (string) ($source['container'] ?? ''); + $label = (string) ($source['label'] ?? ''); + if ($label === '' || str_contains($label, "\t") || str_contains($label, "\n") || isset($labels[$label])) { + return null; + } + $server = $allowed->get($uuid); + if (! $server instanceof Server || (int) $server->team_id !== $teamId) { + return null; + } + if (! ValidationPatterns::isValidContainerName($container)) { + return null; + } + if (! $server->isFunctional()) { + continue; + } + $labels[$label] = true; + $resolved[] = [ + 'server' => $server, + 'container' => $container, + 'label' => $label, + ]; + } + + return $resolved; + } + + private function resourceTeamId(): ?int + { + if (! is_object($this->resource) || ! method_exists($this->resource, 'team')) { + return null; + } + $team = $this->resource->team(); + $id = is_object($team) ? ($team->id ?? null) : null; + + return is_numeric($id) ? (int) $id : null; + } + + private function serversForResource(): Collection + { + $resource = $this->resource; + if ($resource instanceof Application) { + return $resource->maintenanceServers(); + } + if ($resource instanceof Service) { + return collect([$resource->server])->filter(); + } + if (is_object($resource) && method_exists($resource, 'destination')) { + return collect([$resource->destination?->server])->filter(); + } + + return collect(); + } + private function resolveLogTail(): int|string { if ($this->numberOfLines === -1) { @@ -243,16 +456,18 @@ class GetLogs extends Component return $this->numberOfLines; } - private function logCommand(string $options): string + private function logCommand(string $options, ?Server $server = null, ?string $container = null): string { + $server ??= $this->server; + $container ??= (string) $this->container; $options = trim($options); $options = $options === '' ? '' : "{$options} "; - $command = $this->server->isSwarm() - ? "docker service logs {$options}{$this->container}" - : "docker logs {$options}{$this->container}"; + $command = $server->isSwarm() + ? "docker service logs {$options}{$container}" + : "docker logs {$options}{$container}"; - if ($this->server->isNonRoot()) { - $command = parseCommandsByLineForSudo(collect($command), $this->server)[0]; + if ($server->isNonRoot()) { + $command = parseCommandsByLineForSudo(collect($command), $server)[0]; } return $command; @@ -260,12 +475,8 @@ class GetLogs extends Component private function runLogCommand(string $command, int $maxBytes, string $limitLabel, bool $timestamped = true): string { - $sshCommand = SshMultiplexingHelper::generateSshCommand($this->server, $this->boundedLogCommand($command, $maxBytes)); - - // boundedLogCommand() caps the output at $maxBytes + 1, so one extra byte means truncation. - $output = Process::timeout(config('constants.ssh.command_timeout'))->run($sshCommand)->output(); - $truncated = strlen($output) > $maxBytes; - $logs = rtrim(removeAnsiColors($truncated ? substr($output, 0, $maxBytes) : $output), "\n"); + $fetched = $this->executeLogCommand($this->server, $command, $maxBytes); + $logs = $fetched['logs']; if ($timestamped) { // Docker writes stdout and stderr separately, so sort timestamped lines back into order. $logs = str($logs)->split('/\n/')->sort(function ($a, $b) { @@ -273,13 +484,28 @@ class GetLogs extends Component })->join("\n"); } - if ($truncated) { + if ($fetched['truncated']) { $logs .= "\n\n[... Output truncated at {$limitLabel} limit ...]"; } return $logs; } + /** + * @return array{logs: string, truncated: bool} + */ + private function executeLogCommand(Server $server, string $command, int $maxBytes): array + { + $sshCommand = SshMultiplexingHelper::generateSshCommand($server, $this->boundedLogCommand($command, $maxBytes)); + + // boundedLogCommand() caps the output at $maxBytes + 1, so one extra byte means truncation. + $output = Process::timeout(config('constants.ssh.command_timeout'))->run($sshCommand)->output(); + $truncated = strlen($output) > $maxBytes; + $logs = rtrim(removeAnsiColors($truncated ? substr($output, 0, $maxBytes) : $output), "\n"); + + return ['logs' => $logs, 'truncated' => $truncated]; + } + private function boundedLogCommand(string $command, int $maxBytes): string { return "({$command}) 2>&1 | head -c ".($maxBytes + 1); diff --git a/app/Livewire/Project/Shared/Logs.php b/app/Livewire/Project/Shared/Logs.php index 609d2efa3b..1add0bb6a0 100644 --- a/app/Livewire/Project/Shared/Logs.php +++ b/app/Livewire/Project/Shared/Logs.php @@ -13,8 +13,10 @@ use App\Models\StandaloneMysql; use App\Models\StandalonePostgresql; use App\Models\StandaloneRedis; use App\Models\StandaloneSqlite; +use App\Support\RuntimeLogTargets; use App\Traits\ListensToTeamChannel; use Illuminate\Support\Collection; +use Livewire\Attributes\Url; use Livewire\Component; class Logs extends Component @@ -48,6 +50,14 @@ class Logs extends Component public bool $containersLoaded = false; + #[Url(as: 'pull_request_id', except: '0')] + public string $pullRequestId = '0'; + + #[Url(as: 'service', except: '')] + public string $service = ''; + + private bool $syncingSelection = false; + public function getListeners() { return $this->teamChannelListeners([ @@ -63,13 +73,40 @@ class Logs extends Component $this->serverContainers[$server->id] = $this->getContainersForServer($server); } $this->containersLoaded = true; + $this->syncLogSelection(); } catch (\Exception $e) { - $this->containersLoaded = true; // Set to true to stop loading spinner + $this->containersLoaded = true; + $this->syncLogSelection(); return handleError($e, $this); } } + public function updatedPullRequestId(): void + { + $this->syncLogSelection(); + } + + public function updatedService(): void + { + $this->syncLogSelection(); + } + + public function syncLogSelection(): void + { + if ($this->syncingSelection || ! $this->containersLoaded) { + return; + } + + $this->syncingSelection = true; + + $targets = $this->selectionTargets(); + $this->pullRequestId = $targets['deployment']; + $this->service = $targets['service']; + + $this->syncingSelection = false; + } + private function getContainersForServer($server) { if (! $server->isFunctional()) { @@ -169,18 +206,55 @@ class Logs extends Component } } $this->containers = $this->containers->sort(); - if (data_get($this->query, 'pull_request_id')) { - $this->containers = $this->containers->filter(function ($container) { - return str_contains($container, $this->query['pull_request_id']); - }); - } } catch (\Exception $e) { return handleError($e, $this); } } + /** + * @return array{ + * deployments: list, + * services: list, + * deployment: string, + * service: string, + * sources: list, + * streamKey: string, + * } + */ + private function selectionTargets(): array + { + return RuntimeLogTargets::select( + $this->logServerPayload(), + $this->pullRequestId, + $this->service, + $this->type === 'application', + ); + } + + /** + * @return list>}> + */ + private function logServerPayload(): array + { + if (! isset($this->servers)) { + return []; + } + + return $this->servers + ->filter(fn ($server): bool => count($this->serverContainers[$server->id] ?? []) > 0) + ->map(fn ($server): array => [ + 'uuid' => (string) $server->uuid, + 'name' => (string) $server->name, + 'containers' => array_values($this->serverContainers[$server->id] ?? []), + ]) + ->values() + ->all(); + } + public function render() { - return view('livewire.project.shared.logs'); + return view('livewire.project.shared.logs', [ + 'targets' => $this->containersLoaded ? $this->selectionTargets() : null, + ]); } } diff --git a/app/Livewire/Project/Shared/Metrics.php b/app/Livewire/Project/Shared/Metrics.php index e5b87b48cd..8b027d7b49 100644 --- a/app/Livewire/Project/Shared/Metrics.php +++ b/app/Livewire/Project/Shared/Metrics.php @@ -16,21 +16,12 @@ class Metrics extends Component public int $interval = 5; - public bool $poll = true; - - public function pollData() - { - if ($this->poll || $this->interval <= 10) { - $this->loadData(); - if ($this->interval > 10) { - $this->poll = false; - } - } - } + public bool $live = true; public function loadData() { try { + $this->resource->prefetchMetrics(['cpu', 'memory'], $this->interval); $cpuMetrics = $this->resource->getCpuMetrics($this->interval); $memoryMetrics = $this->resource->getMemoryMetrics($this->interval); $this->dispatch("refreshChartData-{$this->chartId}-cpu", [ @@ -46,12 +37,24 @@ class Metrics extends Component public function setInterval() { - if ($this->interval <= 10) { - $this->poll = true; - } $this->loadData(); } + public function updatedLive(): void + { + if ($this->live) { + $this->loadData(); + } + } + + /** + * Short ranges refresh every 5 seconds; longer ranges change slowly, so they refresh every minute. + */ + public function pollSeconds(): int + { + return $this->interval <= 60 ? 5 : 60; + } + public function render() { return view('livewire.project.shared.metrics'); diff --git a/app/Livewire/Project/Shared/ResourceOperations.php b/app/Livewire/Project/Shared/ResourceOperations.php index a459b988e0..414af27128 100644 --- a/app/Livewire/Project/Shared/ResourceOperations.php +++ b/app/Livewire/Project/Shared/ResourceOperations.php @@ -4,8 +4,6 @@ namespace App\Livewire\Project\Shared; use App\Actions\Database\StartDatabase; use App\Actions\Database\StopDatabase; -use App\Actions\Service\StartService; -use App\Actions\Service\StopService; use App\Actions\Shared\MigrateResourceToDestination; use App\Jobs\VolumeCloneJob; use App\Models\Application; @@ -259,143 +257,9 @@ class ResourceOperations extends Component return redirect()->to($route); } elseif ($this->resource->type() === 'service') { - $uuid = new_public_id(); - $new_resource = $this->resource->replicate([ - 'id', - 'created_at', - 'updated_at', - ])->fill([ - 'uuid' => $uuid, - 'name' => $this->resource->name.'-clone-'.$uuid, + $new_resource = clone_service($this->resource, $new_destination, [ 'environment_id' => $new_environment->id, - 'destination_id' => $new_destination->id, - 'destination_type' => $new_destination->getMorphClass(), - 'server_id' => $new_destination->server_id, - ]); - - $new_resource->save(); - - $tags = $this->resource->tags; - foreach ($tags as $tag) { - $new_resource->tags()->attach($tag->id); - } - - $scheduledTasks = $this->resource->scheduled_tasks()->get(); - foreach ($scheduledTasks as $task) { - $newTask = $task->replicate([ - 'id', - 'created_at', - 'updated_at', - ])->fill([ - 'uuid' => new_public_id(), - 'service_id' => $new_resource->id, - 'team_id' => $this->resourceTeamId(), - ]); - $newTask->save(); - } - - $environmentVariables = $this->resource->environment_variables()->get(); - foreach ($environmentVariables as $environmentVariable) { - $newEnvironmentVariable = $environmentVariable->replicate([ - 'id', - 'created_at', - 'updated_at', - ])->fill([ - 'resourceable_id' => $new_resource->id, - 'resourceable_type' => $new_resource->getMorphClass(), - ]); - $newEnvironmentVariable->save(); - } - - $this->resource->cloneSecretManagerLinkTo($new_resource); - - foreach ($new_resource->applications() as $application) { - $application->fill([ - 'status' => 'exited', - ])->save(); - - $persistentVolumes = $application->persistentStorages()->get(); - foreach ($persistentVolumes as $volume) { - $newName = ''; - if (str_starts_with($volume->name, $volume->resource->uuid)) { - $newName = str($volume->name)->replace($volume->resource->uuid, $application->uuid); - } else { - $newName = $application->uuid.'-'.str($volume->name)->afterLast('-'); - } - - $newPersistentVolume = $volume->replicate([ - 'id', - 'created_at', - 'updated_at', - 'uuid', - ])->fill([ - 'name' => $newName, - 'resource_id' => $application->id, - ]); - $newPersistentVolume->save(); - - if ($this->cloneVolumeData) { - try { - StopService::dispatch($application); - $sourceVolume = $volume->name; - $targetVolume = $newPersistentVolume->name; - $sourceServer = $application->service->destination->server; - $targetServer = $new_resource->destination->server; - - VolumeCloneJob::dispatch($sourceVolume, $targetVolume, $sourceServer, $targetServer, $newPersistentVolume); - - StartService::dispatch($application); - } catch (\Exception $e) { - \Log::error('Failed to copy volume data for '.$volume->name.': '.$e->getMessage()); - } - } - } - } - - foreach ($new_resource->databases() as $database) { - $database->fill([ - 'status' => 'exited', - ])->save(); - - $persistentVolumes = $database->persistentStorages()->get(); - foreach ($persistentVolumes as $volume) { - $newName = ''; - if (str_starts_with($volume->name, $volume->resource->uuid)) { - $newName = str($volume->name)->replace($volume->resource->uuid, $database->uuid); - } else { - $newName = $database->uuid.'-'.str($volume->name)->afterLast('-'); - } - - $newPersistentVolume = $volume->replicate([ - 'id', - 'created_at', - 'updated_at', - 'uuid', - ])->fill([ - 'name' => $newName, - 'resource_id' => $database->id, - ]); - $newPersistentVolume->save(); - - if ($this->cloneVolumeData) { - try { - StopService::dispatch($database->service); - $sourceVolume = $volume->name; - $targetVolume = $newPersistentVolume->name; - $sourceServer = $database->service->destination->server; - $targetServer = $new_resource->destination->server; - - VolumeCloneJob::dispatch($sourceVolume, $targetVolume, $sourceServer, $targetServer, $newPersistentVolume); - - StartService::dispatch($database->service); - } catch (\Exception $e) { - \Log::error('Failed to copy volume data for '.$volume->name.': '.$e->getMessage()); - } - } - } - } - - $new_resource->parse(); + ], $this->cloneVolumeData); $route = route('project.service.configuration', [ 'project_uuid' => $new_environment->project->uuid, @@ -458,7 +322,7 @@ class ResourceOperations extends Component $new_destination = $this->findResourceTeamDestination($destination_uuid); if (! $new_destination) { - return $this->addError('destination_id', 'Destination not found.'); + return $this->dispatch('error', 'Destination not found.'); } $result = MigrateResourceToDestination::run( @@ -503,7 +367,7 @@ class ResourceOperations extends Component } catch (ValidationException $e) { $message = collect($e->errors())->flatten()->first() ?? $e->getMessage(); - return $this->addError('destination_id', $message); + return $this->dispatch('error', $message); } catch (\Throwable $e) { return handleError($e, $this); } diff --git a/app/Livewire/Project/Shared/ResourceTrafficAnalytics.php b/app/Livewire/Project/Shared/ResourceTrafficAnalytics.php index 02f1718ede..6357e77b6d 100644 --- a/app/Livewire/Project/Shared/ResourceTrafficAnalytics.php +++ b/app/Livewire/Project/Shared/ResourceTrafficAnalytics.php @@ -35,9 +35,8 @@ abstract class ResourceTrafficAnalytics extends Component #[Locked] public ?string $analyticsServerUuid = null; - // Realtime refresh. Off by default (click "Live" to arm it). Only meaningful on the - // 24h range; the 60s cadence matches the SentinelTrafficClient cache TTL and - // Sentinel's per-minute rollups. The control is disabled for 7d/30d. + // Realtime refresh every 60s (the SentinelTrafficClient cache TTL); off by default, + // works with every range. public bool $live = false; public ?array $overview = null; @@ -109,20 +108,11 @@ abstract class ResourceTrafficAnalytics extends Component $this->loadData(); } - public function toggleLive(): void + public function updatedLive(): void { - if ($this->range !== '24h') { - return; + if ($this->live) { + $this->loadData(); } - $this->live = ! $this->live; - } - - /** - * Realtime polling is only armed when the user has it on and the range is 24h. - */ - public function isLivePollable(): bool - { - return $this->live && $this->range === '24h'; } public function loadData(): void diff --git a/app/Livewire/Project/Show.php b/app/Livewire/Project/Show.php index 8f57d14417..16fabe6c01 100644 --- a/app/Livewire/Project/Show.php +++ b/app/Livewire/Project/Show.php @@ -63,7 +63,7 @@ class Show extends Component $user = auth()->user(); return view('livewire.project.show', [ - 'environmentsJs' => $this->project->environments->map(function (Environment $environment) use ($user): array { + 'environmentsJs' => $this->project->environments->sort(fn (Environment $first, Environment $second): int => Environment::compareNames($first->name, $second->name))->values()->map(function (Environment $environment) use ($user): array { $resourceCount = collect([ $environment->applications_count, $environment->services_count, diff --git a/app/Livewire/Server/Charts.php b/app/Livewire/Server/Charts.php index 6ff1f682fa..65ddde8a7d 100644 --- a/app/Livewire/Server/Charts.php +++ b/app/Livewire/Server/Charts.php @@ -22,7 +22,7 @@ class Charts extends Component public int $interval = 5; - public bool $poll = true; + public bool $live = true; #[Validate(['required', 'integer', 'min:1'])] public int|string $sentinelMetricsRefreshRateSeconds; @@ -89,19 +89,10 @@ class Charts extends Component } } - public function pollData() - { - if ($this->poll || $this->interval <= 10) { - $this->loadData(); - if ($this->interval > 10) { - $this->poll = false; - } - } - } - public function loadData() { try { + $this->server->prefetchMetrics(['cpu', 'memory', 'disk', 'load', 'network'], $this->interval); $cpuMetrics = $this->server->getCpuMetrics($this->interval); $memoryMetrics = $this->server->getMemoryMetrics($this->interval); // Disk/network/load arrived in a later Sentinel; fetch each optionally so an @@ -156,9 +147,21 @@ class Charts extends Component public function setInterval() { - if ($this->interval <= 10) { - $this->poll = true; - } $this->loadData(); } + + public function updatedLive(): void + { + if ($this->live) { + $this->loadData(); + } + } + + /** + * Short ranges refresh every 5 seconds; longer ranges change slowly, so they refresh every minute. + */ + public function pollSeconds(): int + { + return $this->interval <= 60 ? 5 : 60; + } } diff --git a/app/Livewire/Server/CloudflareTunnel.php b/app/Livewire/Server/CloudflareTunnel.php index 31bca417f6..29d52b5a2c 100644 --- a/app/Livewire/Server/CloudflareTunnel.php +++ b/app/Livewire/Server/CloudflareTunnel.php @@ -88,6 +88,7 @@ class CloudflareTunnel extends Component { try { $this->authorize('update', $this->server); + $this->server->assertNotTransferredAway(); if (filled($this->server->settings->cloudflare_tunnel_id)) { foreach ($tunnels->tearDown($this->server) as $warning) { $this->dispatch('warning', $warning); @@ -114,6 +115,7 @@ class CloudflareTunnel extends Component { try { $this->authorize('update', $this->server); + $this->server->assertNotTransferredAway(); $this->isCloudflareTunnelsEnabled = true; $this->server->settings->is_cloudflare_tunnel = true; $this->server->settings->save(); @@ -129,6 +131,7 @@ class CloudflareTunnel extends Component { try { $this->authorize('update', $this->server); + $this->server->assertNotTransferredAway(); $this->ssh_domain = $this->normalizeSshDomain($this->ssh_domain); $activity = ConfigureCloudflared::run($this->server, $this->cloudflare_token, $this->ssh_domain); auditLog('ui.server.cloudflare_tunnel.configuration_started', $this->auditContext([ diff --git a/app/Livewire/Server/Delete.php b/app/Livewire/Server/Delete.php index 2b5051c65e..ce9313a9f9 100644 --- a/app/Livewire/Server/Delete.php +++ b/app/Livewire/Server/Delete.php @@ -70,11 +70,16 @@ class Delete extends Component return; } + $resources = $this->server->definedResources(); if ($isArchive) { - app(ServerTransferClaimer::class)->forgetDnsRecords($this->server); + $claimer = app(ServerTransferClaimer::class); + $claimer->forgetDnsRecords($this->server); + // Applications that use the server only as an additional destination stay; they lose this server. + $claimer->detachAdditionalApplications($this->server); + $resources = $claimer->archiveResources($this->server); } if ($this->force_delete_resources) { - foreach ($this->server->definedResources() as $resource) { + foreach ($resources as $resource) { DeleteResourceJob::dispatch($resource, deleteFromCoolifyOnly: $isArchive); } } diff --git a/app/Livewire/Server/Navbar.php b/app/Livewire/Server/Navbar.php index 1a64d0a15e..4e742df4e5 100644 --- a/app/Livewire/Server/Navbar.php +++ b/app/Livewire/Server/Navbar.php @@ -251,6 +251,7 @@ class Navbar extends Component { $this->server->refresh(); $this->server->load('settings'); + $this->proxyStatus = $this->server->proxy->status ?? 'unknown'; $this->dispatchProxyConfigurationState(); } diff --git a/app/Livewire/Server/PrivateKey/Show.php b/app/Livewire/Server/PrivateKey/Show.php index 97eb760d2c..72b9d31b02 100644 --- a/app/Livewire/Server/PrivateKey/Show.php +++ b/app/Livewire/Server/PrivateKey/Show.php @@ -5,6 +5,7 @@ namespace App\Livewire\Server\PrivateKey; use App\Models\PrivateKey; use App\Models\Server; use Illuminate\Foundation\Auth\Access\AuthorizesRequests; +use Illuminate\Http\Exceptions\ThrottleRequestsException; use Illuminate\Support\Facades\DB; use Livewire\Component; @@ -44,8 +45,18 @@ class Show extends Component return; } + $this->authorize('update', $this->server); + if ($this->server->isTransferredAway()) { + $this->dispatch('error', 'The private key of a transferred server cannot be changed.'); + + return; + } + try { + Server::throttleValidation($this->server->team_id, $this->server->ip); + } catch (ThrottleRequestsException $e) { + return handleError($e, $this); + } try { - $this->authorize('update', $this->server); DB::transaction(function () use ($ownedPrivateKey) { $this->server->privateKey()->associate($ownedPrivateKey); $this->server->save(); @@ -95,6 +106,8 @@ class Show extends Component public function checkConnection() { try { + $this->authorize('update', $this->server); + Server::throttleValidation($this->server->team_id, $this->server->ip); ['uptime' => $uptime, 'error' => $error] = $this->server->validateConnection(); if ($uptime) { $this->dispatch('success', 'Server is reachable.'); diff --git a/app/Livewire/Server/Security/TerminalAccess.php b/app/Livewire/Server/Security/TerminalAccess.php index 2406bdcb33..49b7e9bc06 100644 --- a/app/Livewire/Server/Security/TerminalAccess.php +++ b/app/Livewire/Server/Security/TerminalAccess.php @@ -36,11 +36,6 @@ class TerminalAccess extends Component try { $this->authorize('update', $this->server); - // Check if user is admin or owner - if (! auth()->user()->isAdmin()) { - throw new \Exception('Only team administrators and owners can modify terminal access.'); - } - // Verify password if (! verifyPasswordConfirmation($password, $this)) { return 'The provided password is incorrect.'; diff --git a/app/Livewire/Server/Show.php b/app/Livewire/Server/Show.php index ff5751ed2a..796ddcc82c 100644 --- a/app/Livewire/Server/Show.php +++ b/app/Livewire/Server/Show.php @@ -7,6 +7,7 @@ use App\Enums\ServerRole; use App\Events\ServerReachabilityChanged; use App\Models\CloudProviderToken; use App\Models\Server; +use App\Models\Team; use App\Rules\ValidServerIp; use App\Services\DigitalOceanService; use App\Services\HetznerService; @@ -251,6 +252,12 @@ class Show extends Component if ($toModel) { $this->validate(); $this->ip = Server::normalizeIp($this->ip); + if ($this->server->isTransferredAway() && $this->connectionFieldsChanged()) { + $this->ip = $this->server->ip; + $this->user = $this->server->user; + $this->port = $this->server->port; + throw new \Exception('The IP address, SSH user, and SSH port of a transferred server cannot be changed.'); + } // Check only a changed IP, so that an old duplicate does not block saving other fields. $ipChanged = $this->ip !== Server::normalizeIp($this->server->getRawOriginal('ip')); $foundServer = $ipChanged ? Server::where('ip', $this->ip) @@ -322,6 +329,13 @@ class Show extends Component } } + private function connectionFieldsChanged(): bool + { + return $this->ip !== Server::normalizeIp($this->server->getRawOriginal('ip')) + || (string) $this->user !== (string) $this->server->user + || (int) $this->port !== (int) $this->server->port; + } + /** * Server columns are audited by the model. Settings live on ServerSetting, so record their names here. * @@ -406,13 +420,16 @@ class Show extends Component return; } - if ($this->server->team->serverOverflow()) { + // The archive does not use a server slot, so taking it back needs a free slot. + if (Team::serverLimitReached($this->server->team)) { $this->dispatch('error', 'Your team is over its server limit. Upgrade your subscription or remove a server first.'); return; } try { + // The claim connects to the host over SSH, so it uses the same Cloud limit as a validation. + Server::throttleValidation($this->server->team_id, $this->server->ip); $claimer->claim($this->server); } catch (\Throwable $e) { handleError($e, $this); @@ -442,6 +459,7 @@ class Show extends Component try { $this->authorize('update', $this->server); $this->syncData(true); + Server::throttleValidation($this->server->team_id, $this->server->ip); ['uptime' => $uptime, 'error' => $error] = $this->server->validateConnection(); if ($uptime) { $this->dispatch('success', 'Server is reachable.'); @@ -594,6 +612,9 @@ class Show extends Component { try { $this->authorize('view', $this->server); + if ($this->server->isTransferredAway()) { + return; + } if (! $this->server->hetzner_server_id || ! $this->server->cloudProviderToken) { $this->dispatch('error', 'This server is not associated with a Hetzner Cloud server or token.'); @@ -643,6 +664,9 @@ class Show extends Component { try { $this->authorize('view', $this->server); + if ($this->server->isTransferredAway()) { + return; + } if (! $this->server->vultr_instance_id || ! $this->server->cloudProviderToken) { $this->dispatch('error', 'This server is not associated with a Vultr instance or token.'); @@ -665,6 +689,9 @@ class Show extends Component { try { $this->authorize('view', $this->server); + if ($this->server->isTransferredAway()) { + return; + } if (! $this->server->digitalocean_droplet_id || ! $this->server->cloudProviderToken) { $this->dispatch('error', 'This server is not associated with a DigitalOcean droplet or token.'); @@ -687,6 +714,9 @@ class Show extends Component { try { $this->authorize('view', $this->server); + if ($this->server->isTransferredAway()) { + return null; + } if (! $this->server->hostinger_virtual_machine_id || ! $this->server->cloudProviderToken) { $this->dispatch('error', 'This server is not associated with a Hostinger VPS or token.'); @@ -738,6 +768,7 @@ class Show extends Component { try { $this->authorize('update', $this->server); + $this->server->assertNotTransferredAway(); if (! $this->server->hetzner_server_id || ! $this->server->cloudProviderToken) { $this->dispatch('error', 'This server is not associated with a Hetzner Cloud server or token.'); @@ -760,6 +791,7 @@ class Show extends Component { try { $this->authorize('update', $this->server); + $this->server->assertNotTransferredAway(); if (! $this->server->vultr_instance_id || ! $this->server->cloudProviderToken) { $this->dispatch('error', 'This server is not associated with a Vultr instance or token.'); @@ -782,6 +814,7 @@ class Show extends Component { try { $this->authorize('update', $this->server); + $this->server->assertNotTransferredAway(); if (! $this->server->digitalocean_droplet_id || ! $this->server->cloudProviderToken) { $this->dispatch('error', 'This server is not associated with a DigitalOcean droplet or token.'); @@ -804,6 +837,7 @@ class Show extends Component { try { $this->authorize('update', $this->server); + $this->server->assertNotTransferredAway(); if (! $this->server->hostinger_virtual_machine_id || ! $this->server->cloudProviderToken) { $this->dispatch('error', 'This server is not associated with a Hostinger VPS or token.'); @@ -913,6 +947,7 @@ class Show extends Component try { $this->authorize('update', $this->server); + $this->server->assertNotTransferredAway(); $token = $this->availableHetznerTokens->firstWhere('id', $this->selectedHetznerTokenId); if (! $token) { @@ -954,6 +989,7 @@ class Show extends Component try { $this->authorize('update', $this->server); + $this->server->assertNotTransferredAway(); $token = $this->availableHetznerTokens->firstWhere('id', $this->selectedHetznerTokenId); if (! $token) { @@ -985,6 +1021,7 @@ class Show extends Component try { $this->authorize('update', $this->server); + $this->server->assertNotTransferredAway(); $token = $this->availableHetznerTokens->firstWhere('id', $this->selectedHetznerTokenId); if (! $token) { @@ -1041,6 +1078,7 @@ class Show extends Component try { $this->authorize('update', $this->server); + $this->server->assertNotTransferredAway(); $token = $this->availableDigitalOceanTokens->firstWhere('id', $this->selectedDigitalOceanTokenId); if (! $token) { @@ -1082,6 +1120,7 @@ class Show extends Component try { $this->authorize('update', $this->server); + $this->server->assertNotTransferredAway(); $token = $this->availableDigitalOceanTokens->firstWhere('id', $this->selectedDigitalOceanTokenId); if (! $token) { @@ -1113,6 +1152,7 @@ class Show extends Component try { $this->authorize('update', $this->server); + $this->server->assertNotTransferredAway(); $token = $this->availableDigitalOceanTokens->firstWhere('id', $this->selectedDigitalOceanTokenId); if (! $token) { @@ -1172,6 +1212,7 @@ class Show extends Component try { $this->authorize('update', $this->server); + $this->server->assertNotTransferredAway(); $token = $this->availableHostingerTokens->firstWhere('id', $this->selectedHostingerTokenId); if (! $token) { @@ -1212,6 +1253,7 @@ class Show extends Component try { $this->authorize('update', $this->server); + $this->server->assertNotTransferredAway(); $token = $this->availableHostingerTokens->firstWhere('id', $this->selectedHostingerTokenId); if (! $token) { @@ -1243,6 +1285,7 @@ class Show extends Component try { $this->authorize('update', $this->server); + $this->server->assertNotTransferredAway(); $token = $this->availableHostingerTokens->firstWhere('id', $this->selectedHostingerTokenId); if (! $token) { @@ -1302,6 +1345,7 @@ class Show extends Component try { $this->authorize('update', $this->server); + $this->server->assertNotTransferredAway(); $token = $this->availableVultrTokens->firstWhere('id', $this->selectedVultrTokenId); if (! $token) { @@ -1343,6 +1387,7 @@ class Show extends Component try { $this->authorize('update', $this->server); + $this->server->assertNotTransferredAway(); $token = $this->availableVultrTokens->firstWhere('id', $this->selectedVultrTokenId); if (! $token) { @@ -1374,6 +1419,7 @@ class Show extends Component try { $this->authorize('update', $this->server); + $this->server->assertNotTransferredAway(); $token = $this->availableVultrTokens->firstWhere('id', $this->selectedVultrTokenId); if (! $token) { diff --git a/app/Livewire/Server/Transfer.php b/app/Livewire/Server/Transfer.php index 22bb6d0685..f6f01cc975 100644 --- a/app/Livewire/Server/Transfer.php +++ b/app/Livewire/Server/Transfer.php @@ -38,7 +38,12 @@ class Transfer extends Component // The transfer exports every credential of the server, so only admins and owners may open it. $this->authorize('update', $this->server); $this->exportId = data_get($this->server->server_metadata, 'transfer.export_id'); - $this->managerUrl = (string) data_get($this->server->server_metadata, 'transfer.target_instance_url', ''); + // Transfer back sends a new team token to this URL, so it is prefilled only when this instance chose it. + $managerUrl = data_get($this->server->server_metadata, 'transfer.target_instance_url'); + if (is_string($managerUrl) && ! data_get($this->server->server_metadata, 'transfer.claimed_by_marker') + && ServerTransferMigrator::isValidInstanceUrl($managerUrl)) { + $this->managerUrl = $managerUrl; + } $this->thisInstanceUrl = rtrim((string) (instanceSettings()->fqdn ?: config('app.url')), '/'); } catch (Throwable $e) { handleError($e, $this); @@ -77,8 +82,19 @@ class Transfer extends Component return array_values(array_filter((array) data_get($this->migration, 'checklist', []), 'is_string')); } + /** + * The link to the server on the target. The target sends it, so only an https link is shown. + */ + public function getTargetServerUrlProperty(): ?string + { + $url = data_get($this->migration, 'target_server_url'); + + return ServerTransferMigrator::isLinkableUrl($url) ? $url : null; + } + public function refreshMigration(): void { + $this->authorize('update', $this->server); $this->server->refresh(); $this->lastWarnings = array_values((array) data_get($this->migration, 'warnings', $this->lastWarnings)); } @@ -86,7 +102,7 @@ class Transfer extends Component public function migrateServer(ServerTransferMigrator $migrator): void { try { - $this->authorize('update', $this->server); + $this->authorize('transfer', $this->server); if ($this->isLocalhost) { throw new \RuntimeException('The Coolify host (localhost) cannot be transferred.'); } @@ -105,12 +121,14 @@ class Transfer extends Component $this->server->refresh(); $this->lastWarnings = []; - $this->targetToken = ''; $this->dispatch('success', 'All checks passed. The transfer started.'); } catch (Throwable $e) { // A failed check is saved in the transfer log, so show it. $this->server->refresh(); handleError($e, $this); + } finally { + // Public properties are sent to the browser with every response. + $this->targetToken = ''; } } @@ -127,10 +145,11 @@ class Transfer extends Component ]); $this->server->refresh(); - $this->managerToken = ''; $this->dispatch('success', 'The other instance started the transfer back.'); } catch (Throwable $e) { handleError($e, $this); + } finally { + $this->managerToken = ''; } } diff --git a/app/Livewire/Server/ValidateAndInstall.php b/app/Livewire/Server/ValidateAndInstall.php index 407f5f31d2..84b2ffaa32 100644 --- a/app/Livewire/Server/ValidateAndInstall.php +++ b/app/Livewire/Server/ValidateAndInstall.php @@ -8,6 +8,7 @@ use App\Events\ServerValidated; use App\Jobs\CheckAndStartSentinelJob; use App\Models\Server; use Illuminate\Foundation\Auth\Access\AuthorizesRequests; +use Livewire\Attributes\Locked; use Livewire\Component; class ValidateAndInstall extends Component @@ -22,6 +23,7 @@ class ValidateAndInstall extends Component public bool $install = true; + #[Locked] public $uptime = null; public $supported_os_type = null; @@ -113,6 +115,7 @@ class ValidateAndInstall extends Component { try { $this->authorize('update', $this->server); + Server::throttleValidation($this->server->team_id, $this->server->ip); if ($this->server->vultr_instance_id) { $status = $this->server->refreshVultrState(); $this->server->refresh(); @@ -177,6 +180,10 @@ class ValidateAndInstall extends Component { $this->authorize('update', $this->server); + if (! $this->uptime) { + return; + } + $this->supported_os_type = $this->server->validateOS(); if (! $this->supported_os_type) { $this->error = 'Server OS type is not supported. Please install Docker manually before continuing: documentation.'; @@ -193,6 +200,10 @@ class ValidateAndInstall extends Component { $this->authorize('update', $this->server); + if (! $this->uptime) { + return; + } + $validationResult = $this->server->validatePrerequisites(); $this->prerequisites_installed = $validationResult['success']; if (! $validationResult['success']) { @@ -233,6 +244,10 @@ class ValidateAndInstall extends Component { $this->authorize('update', $this->server); + if (! $this->uptime) { + return; + } + $this->docker_installed = $this->server->validateDockerEngine(); $this->docker_compose_installed = $this->server->validateDockerCompose(); if (! $this->docker_installed || ! $this->docker_compose_installed) { @@ -271,6 +286,10 @@ class ValidateAndInstall extends Component { $this->authorize('update', $this->server); + if (! $this->uptime) { + return; + } + if ($this->server->isSwarm()) { $swarmInstalled = $this->server->validateDockerSwarm(); if ($swarmInstalled) { @@ -291,6 +310,11 @@ class ValidateAndInstall extends Component if ($this->server->isSentinelEnabled()) { CheckAndStartSentinelJob::dispatch($this->server); } + if (! $this->server->isProxyShouldRun()) { + $this->dispatch('success', 'Server validated.'); + + return; + } $this->dispatch('success', 'Server validated, proxy is starting in a moment.'); $proxyShouldRun = CheckProxy::run($this->server, true); if (! $proxyShouldRun) { @@ -301,8 +325,7 @@ class ValidateAndInstall extends Component instant_remote_process(ensureProxyNetworksExist($this->server)->toArray(), $this->server, false); StartProxy::dispatch($this->server); } else { - $requiredDockerVersion = str(config('constants.docker.minimum_required_version'))->before('.'); - $this->error = 'Minimum Docker Engine version '.$requiredDockerVersion.' is not installed. Please install Docker manually before continuing: documentation.'; + $this->error = $this->server->dockerEngineVersionError(); $this->server->update([ 'validation_logs' => $this->error, ]); diff --git a/app/Livewire/Settings/Advanced.php b/app/Livewire/Settings/Advanced.php index 86060ddd3f..17f9efc36d 100644 --- a/app/Livewire/Settings/Advanced.php +++ b/app/Livewire/Settings/Advanced.php @@ -66,6 +66,12 @@ class Advanced extends Component public string $file_download_storage = 'stream'; + public int $server_validation_rate_limit_seconds = 600; + + public int $server_validation_max_failures_per_ip = 10; + + public int $server_validation_max_failures_per_team = 30; + public function rules() { return [ @@ -85,6 +91,9 @@ class Advanced extends Component 'webhook_allow_localhost' => 'boolean', 'domain_connect_private_key' => 'nullable|string', 'image_cdn_url' => 'nullable|url|max:255', + 'server_validation_rate_limit_seconds' => 'required|integer|min:60|max:86400', + 'server_validation_max_failures_per_ip' => 'required|integer|min:1|max:1000', + 'server_validation_max_failures_per_team' => 'required|integer|min:1|max:10000', ]; } @@ -114,6 +123,9 @@ class Advanced extends Component ? 's3:'.$this->settings->avatar_s3_storage_id : 'local'; $this->image_cdn_url = $this->settings->image_cdn_url; + $this->server_validation_rate_limit_seconds = $this->settings->server_validation_rate_limit_seconds; + $this->server_validation_max_failures_per_ip = $this->settings->server_validation_max_failures_per_ip; + $this->server_validation_max_failures_per_team = $this->settings->server_validation_max_failures_per_team; $this->avatar_storage_options = [ ['value' => 'local', 'label' => 'Local storage'], ...S3Storage::query() @@ -200,6 +212,12 @@ class Advanced extends Component return; } + if (isCloud()) { + $this->settings->server_validation_rate_limit_seconds = $this->server_validation_rate_limit_seconds; + $this->settings->server_validation_max_failures_per_ip = $this->server_validation_max_failures_per_ip; + $this->settings->server_validation_max_failures_per_team = $this->server_validation_max_failures_per_team; + } + if (isCloud() && filled($this->domain_connect_private_key)) { $this->settings->domain_connect_private_key = $this->normalizeDomainConnectPrivateKey($this->domain_connect_private_key); $this->domain_connect_private_key = null; diff --git a/app/Livewire/Settings/Updates.php b/app/Livewire/Settings/Updates.php index 27fe705423..8fd02e046e 100644 --- a/app/Livewire/Settings/Updates.php +++ b/app/Livewire/Settings/Updates.php @@ -159,10 +159,14 @@ class Updates extends Component public function checkManually() { $this->authorize('update', $this->settings); - CheckForUpdatesJob::dispatchSync(cdnTries: 1); + $checked = (new CheckForUpdatesJob(cdnTries: 1))->handle(); $this->dispatch('updateAvailable'); - $settings = instanceSettings(); - if ($settings->new_version_available) { + if (! $checked) { + $this->dispatch('error', 'Could not check for updates. Try again later.'); + + return; + } + if (instanceSettings()->new_version_available) { $this->dispatch('success', 'New version available!'); } else { $this->dispatch('success', 'No new version available.'); diff --git a/app/Livewire/Source/Gitea/Change.php b/app/Livewire/Source/Gitea/Change.php index e66cc355cb..5980ed3fbf 100644 --- a/app/Livewire/Source/Gitea/Change.php +++ b/app/Livewire/Source/Gitea/Change.php @@ -143,7 +143,7 @@ class Change extends Component })->whereUuid($gitea_app_uuid)->firstOrFail(); $this->privateKeys = PrivateKey::ownedByCurrentTeamCached(); - $this->applications = $this->gitea_app->applications; + $this->applications = $this->gitea_app->applications()->whereRelation('environment.project.team', 'id', currentTeam()->id)->get(); $settings = instanceSettings(); @@ -264,10 +264,18 @@ class Change extends Component } } + private function authorizeSystemWideChange(): void + { + if ($this->isSystemWide && ! $this->gitea_app->is_system_wide) { + $this->authorize('makeSystemWide', GiteaApp::class); + } + } + public function submit() { try { $this->authorize('update', $this->gitea_app); + $this->authorizeSystemWideChange(); $this->validate(); @@ -283,6 +291,7 @@ class Change extends Component { try { $this->authorize('update', $this->gitea_app); + $this->authorizeSystemWideChange(); $this->validateOnly('isSystemWide'); diff --git a/app/Livewire/Source/Gitea/Create.php b/app/Livewire/Source/Gitea/Create.php index 5b7e2a7719..a3c1247db9 100644 --- a/app/Livewire/Source/Gitea/Create.php +++ b/app/Livewire/Source/Gitea/Create.php @@ -67,6 +67,10 @@ class Create extends Component 'organization' => 'nullable|string', ]); + if ($this->is_system_wide) { + $this->authorize('makeSystemWide', GiteaApp::class); + } + $gitea_app = GiteaApp::create([ 'name' => $this->name, 'api_url' => $this->api_url, diff --git a/app/Livewire/Source/Github/Change.php b/app/Livewire/Source/Github/Change.php index ca59577ccb..5ec577f2e4 100644 --- a/app/Livewire/Source/Github/Change.php +++ b/app/Livewire/Source/Github/Change.php @@ -305,7 +305,7 @@ class Change extends Component $this->authorize('view', $this->github_app); $this->privateKeys = PrivateKey::ownedByCurrentTeamCached(); - $this->applications = $this->github_app->applications; + $this->applications = $this->github_app->applications()->whereRelation('environment.project.team', 'id', currentTeam()->id)->get(); $settings = instanceSettings(); // Sync data from model to properties diff --git a/app/Livewire/Source/Gitlab/Change.php b/app/Livewire/Source/Gitlab/Change.php index 67bcea322b..a7ff5b47e8 100644 --- a/app/Livewire/Source/Gitlab/Change.php +++ b/app/Livewire/Source/Gitlab/Change.php @@ -143,7 +143,7 @@ class Change extends Component })->whereUuid($gitlab_app_uuid)->firstOrFail(); $this->privateKeys = PrivateKey::ownedByCurrentTeamCached(); - $this->applications = $this->gitlab_app->applications; + $this->applications = $this->gitlab_app->applications()->whereRelation('environment.project.team', 'id', currentTeam()->id)->get(); $settings = instanceSettings(); diff --git a/app/Livewire/Storage/Form.php b/app/Livewire/Storage/Form.php index ebf43d5019..6cd161306e 100644 --- a/app/Livewire/Storage/Form.php +++ b/app/Livewire/Storage/Form.php @@ -121,7 +121,7 @@ class Form extends Component { $this->syncData(false); - $this->isPasswordHiddenForMember = auth()->user()?->isMember() ?? false; + $this->isPasswordHiddenForMember = ! (auth()->user()?->can('update', $this->storage) ?? false); } public function testConnection() diff --git a/app/Livewire/Team/Index.php b/app/Livewire/Team/Index.php index 58b940f932..62c03d57ed 100644 --- a/app/Livewire/Team/Index.php +++ b/app/Livewire/Team/Index.php @@ -28,6 +28,8 @@ class Index extends Component public bool $is_build_server_fallback_enabled = true; + public bool $is_two_factor_required = false; + public $avatar; protected function rules(): array @@ -37,6 +39,7 @@ class Index extends Component 'description' => ValidationPatterns::descriptionRules(), 'is_mcp_server_enabled' => 'boolean', 'is_build_server_fallback_enabled' => 'boolean', + 'is_two_factor_required' => 'boolean', ]; } @@ -68,6 +71,7 @@ class Index extends Component $this->team->description = $this->description; $this->team->is_mcp_server_enabled = $this->is_mcp_server_enabled; $this->team->is_build_server_fallback_enabled = $this->is_build_server_fallback_enabled; + $this->team->is_two_factor_required = $this->is_two_factor_required; } else { // Sync FROM model (on load/refresh) $this->name = $this->team->name; @@ -76,6 +80,7 @@ class Index extends Component // hydrated onto the in-memory model stored in session. $this->is_mcp_server_enabled = (bool) ($this->team->is_mcp_server_enabled ?? true); $this->is_build_server_fallback_enabled = (bool) ($this->team->is_build_server_fallback_enabled ?? true); + $this->is_two_factor_required = (bool) $this->team->is_two_factor_required; } } @@ -133,6 +138,12 @@ class Index extends Component $this->validate(); try { $this->authorize('update', $this->team); + if ($this->is_two_factor_required && ! $this->team->is_two_factor_required && blank(auth()->user()->two_factor_confirmed_at)) { + $this->is_two_factor_required = false; + $this->dispatch('error', 'Enable two-factor authentication on your profile before you require it for the team.'); + + return; + } $this->syncData(true); $this->team->save(); refreshSession(); diff --git a/app/Livewire/TwoFactorRequired.php b/app/Livewire/TwoFactorRequired.php new file mode 100644 index 0000000000..72d4091b10 --- /dev/null +++ b/app/Livewire/TwoFactorRequired.php @@ -0,0 +1,22 @@ +user()->two_factor_confirmed_at) && session('status') !== 'two-factor-authentication-confirmed') { + return redirect()->route('dashboard'); + } + } + + public function render() + { + return view('livewire.two-factor-required', [ + 'teamName' => data_get(session('currentTeam'), 'name'), + ])->layout('layouts.simple'); + } +} diff --git a/app/Mcp/Concerns/BuildsResponse.php b/app/Mcp/Concerns/BuildsResponse.php index 824a602355..6cde90dc07 100644 --- a/app/Mcp/Concerns/BuildsResponse.php +++ b/app/Mcp/Concerns/BuildsResponse.php @@ -2,9 +2,11 @@ namespace App\Mcp\Concerns; +use App\Mcp\Servers\CoolifyServer; use App\Services\Security\SensitiveDataRedactor; use Laravel\Mcp\Request; use Laravel\Mcp\Response; +use Laravel\Mcp\Server\Tool; trait BuildsResponse { @@ -128,6 +130,7 @@ trait BuildsResponse { $payload = ['data' => $data]; + $actions = $this->onlyVisibleTools($actions); if ($actions !== []) { $payload['_actions'] = $actions; } @@ -139,6 +142,28 @@ trait BuildsResponse return Response::json($payload); } + /** + * Removes suggestions for tools that the current token cannot see, so agents do not call hidden tools. + * + * @param array> $suggestions + * @return array> + */ + protected function onlyVisibleTools(array $suggestions): array + { + $names = array_column($suggestions, 'tool'); + if ($names === []) { + return $suggestions; + } + + $visible = collect(CoolifyServer::TOOLS) + ->map(fn (string $class): Tool => app($class)) + ->filter(fn (Tool $tool): bool => in_array($tool->name(), $names, true) && $tool->eligibleForRegistration()) + ->map(fn (Tool $tool): string => $tool->name()) + ->all(); + + return array_values(array_filter($suggestions, fn (array $suggestion): bool => in_array($suggestion['tool'] ?? null, $visible, true))); + } + /** * @return array{page:int, per_page:int, offset:int} */ diff --git a/app/Mcp/Tools/CoolifyHelp.php b/app/Mcp/Tools/CoolifyHelp.php index 27c36bdd11..c7caeb8bb2 100644 --- a/app/Mcp/Tools/CoolifyHelp.php +++ b/app/Mcp/Tools/CoolifyHelp.php @@ -120,7 +120,7 @@ class CoolifyHelp extends Tool 'topic' => $topic, 'description' => self::RECIPES[$topic]['description'], 'steps' => collect(self::RECIPES[$topic]['steps']) - ->map(fn (array $step): array => $step + ['available' => isset($tools[$step['tool']])]) + ->map(fn (array $step): array => $step + ['available' => $this->stepAvailable($step, $tools)]) ->all(), 'notes' => ['Steps with available=false need more token abilities.', 'Ask the user before every action that needs confirm=true.'], ])); @@ -153,7 +153,7 @@ class CoolifyHelp extends Tool /** * Registered tools that this token can see, keyed by name. * - * @return array + * @return array|null}> */ private function visibleTools(): array { @@ -163,12 +163,28 @@ class CoolifyHelp extends Tool ->mapWithKeys(fn (Tool $tool): array => [$tool->name() => [ 'summary' => Str::before($tool->description(), '. ').'.', 'read_only' => ($tool->annotations()['readOnlyHint'] ?? false) === true, + 'actions' => $tool instanceof RestActionTool ? $tool->availableActionNames() : null, ]]) ->all(); } /** - * @param array $tools + * A step is available when the token sees its tool and, for grouped tools, its action. + * + * @param array{tool: string, action?: string, note: string} $step + * @param array|null}> $tools + */ + private function stepAvailable(array $step, array $tools): bool + { + if (! isset($tools[$step['tool']])) { + return false; + } + + return ! isset($step['action']) || in_array($step['action'], $tools[$step['tool']]['actions'] ?? [], true); + } + + /** + * @param array|null}> $tools * @return array */ private function summaries(array $tools): array diff --git a/app/Mcp/Tools/GetLogs.php b/app/Mcp/Tools/GetLogs.php index 8bc84fb22a..3d4a431d69 100644 --- a/app/Mcp/Tools/GetLogs.php +++ b/app/Mcp/Tools/GetLogs.php @@ -206,6 +206,8 @@ class GetLogs extends Tool $next[] = ['tool' => 'control', 'args' => ['resource' => 'service', 'action' => 'start', 'uuid' => $uuid], 'hint' => 'Start if token has deploy ability']; } + $next = $this->onlyVisibleTools($next); + return [ 'ok' => false, 'reason' => $reason, diff --git a/app/Mcp/Tools/ManageService.php b/app/Mcp/Tools/ManageService.php index f1de9a151f..523b0a9939 100644 --- a/app/Mcp/Tools/ManageService.php +++ b/app/Mcp/Tools/ManageService.php @@ -17,7 +17,7 @@ class ManageService extends RestActionTool { return [ 'update' => ['method' => 'PATCH', 'path' => 'services/{uuid}', 'ability' => 'write', 'description' => 'update the service (data: name, description, docker_compose_raw base64, connect_to_docker_network, urls, force_domain_override, is_container_label_escape_enabled, redact_all_env_values_in_logs, instant_deploy; instant_deploy needs deploy ability)'], - 'delete' => ['method' => 'DELETE', 'path' => 'services/{uuid}', 'ability' => 'delete', 'confirm' => true, 'description' => 'delete the service (data: delete_volumes, delete_connected_networks, delete_configurations, docker_cleanup, delete_from_coolify_only)'], + 'delete' => ['method' => 'DELETE', 'path' => 'services/{uuid}', 'ability' => 'delete', 'confirm' => true, 'description' => 'delete the service (data: delete_volumes, delete_connected_networks, delete_configurations, docker_cleanup)'], 'clone' => ['method' => 'POST', 'path' => 'services/{uuid}/clone', 'ability' => 'write', 'confirm' => true, 'description' => 'clone the service to a destination of the team (data: destination_uuid, name, clone_volumes; clone_volumes stops the source service while volumes are copied and needs deploy ability)'], 'move' => ['method' => 'POST', 'path' => 'services/{uuid}/move', 'ability' => 'write', 'confirm' => true, 'description' => 'move the service to another environment of the team (data: environment_uuid)'], 'update_application' => ['method' => 'PATCH', 'path' => 'services/{uuid}/applications/{app_uuid}', 'ability' => 'write', 'description' => 'update a service application (data: url, noindex_domains, human_name, description, image, exclude_from_status, is_log_drain_enabled, is_gzip_enabled, traefik_certificate_resolver, is_stripprefix_enabled, is_force_https_enabled, max_restart_count)'], diff --git a/app/Mcp/Tools/RestActionTool.php b/app/Mcp/Tools/RestActionTool.php index feccc6a864..6da98d05d8 100644 --- a/app/Mcp/Tools/RestActionTool.php +++ b/app/Mcp/Tools/RestActionTool.php @@ -57,6 +57,16 @@ abstract class RestActionTool extends Tool return array_filter($this->actions(), fn (array $action): bool => $this->currentTokenCan($action['ability'])); } + /** + * Names of the actions that the current token has the ability for. + * + * @return array + */ + public function availableActionNames(): array + { + return array_keys($this->availableActions()); + } + public function shouldRegister(): bool { return $this->availableActions() !== []; diff --git a/app/Models/Application.php b/app/Models/Application.php index 036955d5eb..674f51b273 100644 --- a/app/Models/Application.php +++ b/app/Models/Application.php @@ -1497,23 +1497,13 @@ class Application extends BaseModel public function pendingDeploymentConfigurationDiff(): ConfigurationDiff { - $currentSnapshot = $this->deploymentConfigurationSnapshot(); - $lastDeployment = $this->get_last_successful_deployment(); - - $previousSnapshot = $lastDeployment?->configuration_snapshot; + $previousSnapshot = $this->get_last_successful_deployment()?->configuration_snapshot; if (! $previousSnapshot) { - $oldConfigHash = data_get($this, 'config_hash'); - $hasLegacyChange = $oldConfigHash === null || $oldConfigHash !== $this->legacyConfigurationHash(); - - if (! $hasLegacyChange) { - return ConfigurationDiff::unchanged(); - } - - $previousSnapshot = []; + return ConfigurationDiff::legacy(); } - return app(ConfigurationDiffer::class)->diff($previousSnapshot, $currentSnapshot); + return app(ConfigurationDiffer::class)->diff(ApplicationConfigurationSnapshot::withoutPreviewVolumes($previousSnapshot, $this), $this->deploymentConfigurationSnapshot()); } public function hasPendingDeploymentConfigurationChanges(): bool @@ -1536,7 +1526,7 @@ class Application extends BaseModel $this->refresh(); if (! $deployment) { - $this->forceFill(['config_hash' => $this->legacyConfigurationHash()])->save(); + $this->forceFill(['config_hash' => $this->deploymentConfigurationHash()])->save(); return; } @@ -1557,25 +1547,13 @@ class Application extends BaseModel 'configuration_hash' => $hash, 'configuration_snapshot' => $snapshot, 'configuration_diff' => $previousDeployment?->configuration_snapshot - ? app(ConfigurationDiffer::class)->diff($previousDeployment->configuration_snapshot, $snapshot)->toArray() + ? app(ConfigurationDiffer::class)->diff(ApplicationConfigurationSnapshot::withoutPreviewVolumes($previousDeployment->configuration_snapshot, $this), $snapshot)->toArray() : null, ]); $this->forceFill(['config_hash' => $hash])->save(); } - private function legacyConfigurationHash(): string - { - $newConfigHash = base64_encode($this->fqdn.json_encode($this->noindexDomains()->all()).$this->git_repository.$this->git_branch.$this->git_commit_sha.$this->build_pack.$this->static_image.$this->install_command.$this->build_command.$this->start_command.$this->ports_exposes.$this->ports_mappings.$this->custom_network_aliases.$this->base_directory.$this->publish_directory.$this->dockerfile.$this->dockerfile_location.$this->custom_labels.$this->custom_docker_run_options.$this->dockerfile_target_build.$this->redirect.$this->custom_nginx_configuration.$this->settings?->use_build_secrets.$this->settings?->inject_build_args_to_dockerfile.$this->settings?->include_source_commit_in_build); - if ($this->pull_request_id === 0 || $this->pull_request_id === null) { - $newConfigHash .= json_encode($this->environment_variables()->get(['value', 'is_multiline', 'is_literal', 'is_buildtime', 'is_runtime'])->makeVisible('value')->sort()); - } else { - $newConfigHash .= json_encode($this->environment_variables_preview()->get(['value', 'is_multiline', 'is_literal', 'is_buildtime', 'is_runtime'])->makeVisible('value')->sort()); - } - - return md5($newConfigHash); - } - public function customRepository() { return convertGitUrl($this->git_repository, $this->deploymentType(), $this->source); @@ -1615,7 +1593,7 @@ class Application extends BaseModel // If shallow clone is enabled and we need a specific commit, // we need to fetch that specific commit with depth=1 if ($isShallowCloneEnabled) { - $git_clone_command = "{$git_clone_command} && cd {$escapedBaseDir} && {$sshCommand} {$gitCommand} fetch --depth=1 origin {$escapedCommit} && {$gitCommand} -c advice.detachedHead=false checkout {$escapedCommit} >/dev/null 2>&1"; + $git_clone_command = "{$git_clone_command} && cd {$escapedBaseDir} && echo 'Fetching the commit.' && {$sshCommand} {$gitCommand} fetch --depth=1 origin {$escapedCommit} && {$gitCommand} -c advice.detachedHead=false checkout {$escapedCommit} >/dev/null 2>&1"; } else { $git_clone_command = "{$git_clone_command} && cd {$escapedBaseDir} && {$sshCommand} {$gitCommand} -c advice.detachedHead=false checkout {$escapedCommit} >/dev/null 2>&1"; } @@ -1629,10 +1607,12 @@ class Application extends BaseModel } // Add shallow submodules flag if shallow clone is enabled $submoduleFlags = $isShallowCloneEnabled ? '--depth=1' : ''; - $git_clone_command = "{$git_clone_command} {$gitCommand} submodule sync && {$sshCommand} {$gitCommand} submodule update --init --recursive {$submoduleFlags}; fi"; + $git_clone_command = "{$git_clone_command} echo 'Updating submodules.' && {$gitCommand} submodule sync && {$sshCommand} {$gitCommand} submodule update --init --recursive {$submoduleFlags}; fi"; } + // `git lfs pull` opens an SSH session even without LFS files, which can hang on hosts + // without LFS over SSH (Azure DevOps). `git lfs ls-files` is offline. if ($this->settings->is_git_lfs_enabled && ! $onlyCheckout) { - $git_clone_command = "{$git_clone_command} && cd {$escapedBaseDir} && {$sshCommand} {$gitCommand} lfs pull"; + $git_clone_command = "{$git_clone_command} && cd {$escapedBaseDir} && if {$gitCommand} lfs ls-files -n | grep -q .; then echo 'Downloading Git LFS objects.' && {$sshCommand} {$gitCommand} lfs pull; fi"; } return $git_clone_command; @@ -1742,7 +1722,8 @@ class Application extends BaseModel $private_key = base64_encode($private_key); $gitlabPort = $oauthSource->custom_port ?? 22; $escapedCustomRepository = str_replace("'", "'\\''", $customRepository); - $base_command = "GIT_SSH_COMMAND=\"ssh -o ConnectTimeout=30 -p {$gitlabPort} -o Port={$gitlabPort} -o LogLevel=ERROR -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i {$customSshKeyLocation} -o IdentitiesOnly=yes\" {$base_command} '{$escapedCustomRepository}'"; + $sshCommand = gitSshCommand($gitlabPort, $customSshKeyLocation); + $base_command = "GIT_SSH_COMMAND=\"{$sshCommand}\" {$base_command} '{$escapedCustomRepository}'"; $commands = $this->gitSshKeySetupCommands($deployment_uuid, $private_key, $exec_in_docker); @@ -1787,7 +1768,8 @@ class Application extends BaseModel // When used with executeInDocker (which uses bash -c '...'), we need to escape for bash context // Replace ' with '\'' to safely escape within single-quoted bash strings $escapedCustomRepository = str_replace("'", "'\\''", $customRepository); - $base_command = "GIT_SSH_COMMAND=\"ssh -o ConnectTimeout=30 -p {$customPort} -o Port={$customPort} -o LogLevel=ERROR -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i {$customSshKeyLocation} -o IdentitiesOnly=yes\" {$base_command} '{$escapedCustomRepository}'"; + $sshCommand = gitSshCommand($customPort, $customSshKeyLocation); + $base_command = "GIT_SSH_COMMAND=\"{$sshCommand}\" {$base_command} '{$escapedCustomRepository}'"; $commands = $this->gitSshKeySetupCommands($deployment_uuid, $private_key, $exec_in_docker); @@ -2010,6 +1992,7 @@ class Application extends BaseModel 'commands' => $this->gitCommandsAsShellCommand($commands), 'branch' => $branch, 'fullRepoUrl' => $fullRepoUrl, + 'git' => 'git'.(isset($gitConfigOption) ? " {$gitConfigOption}" : ''), ]; } @@ -2060,6 +2043,7 @@ class Application extends BaseModel 'commands' => $commands->implode(' && '), 'branch' => $branch, 'fullRepoUrl' => $fullRepoUrl, + 'git' => "git {$gitConfigOption}", ]; } @@ -2070,7 +2054,7 @@ class Application extends BaseModel $private_key = base64_encode($private_key); $gitlabPort = $oauthSource->custom_port ?? 22; $escapedCustomRepository = escapeshellarg($customRepository); - $gitlabSshCommand = "ssh -o ConnectTimeout=30 -p {$gitlabPort} -o Port={$gitlabPort} -o LogLevel=ERROR -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i {$customSshKeyLocation} -o IdentitiesOnly=yes"; + $gitlabSshCommand = gitSshCommand($gitlabPort, $customSshKeyLocation); $gitlabGitSshCommand = "GIT_SSH_COMMAND=\"{$gitlabSshCommand}\""; $git_clone_command_base = "{$gitlabGitSshCommand} {$git_clone_command} {$escapedCustomRepository} {$escapedBaseDir}"; if ($only_checkout) { @@ -2100,6 +2084,7 @@ class Application extends BaseModel 'commands' => $commands, 'branch' => $branch, 'fullRepoUrl' => $fullRepoUrl, + 'git' => "{$gitlabGitSshCommand} git", ]; } @@ -2118,6 +2103,7 @@ class Application extends BaseModel 'commands' => $this->gitCommandsAsShellCommand($commands), 'branch' => $branch, 'fullRepoUrl' => $fullRepoUrl, + 'git' => 'git', ]; } } @@ -2129,7 +2115,7 @@ class Application extends BaseModel } $private_key = base64_encode($private_key); $escapedCustomRepository = escapeshellarg($customRepository); - $deployKeySshCommand = "ssh -o ConnectTimeout=30 -p {$customPort} -o Port={$customPort} -o LogLevel=ERROR -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i {$customSshKeyLocation} -o IdentitiesOnly=yes"; + $deployKeySshCommand = gitSshCommand($customPort, $customSshKeyLocation); $deployKeyGitSshCommand = "GIT_SSH_COMMAND=\"{$deployKeySshCommand}\""; $git_clone_command_base = "{$deployKeyGitSshCommand} {$git_clone_command} {$escapedCustomRepository} {$escapedBaseDir}"; if ($only_checkout) { @@ -2175,6 +2161,7 @@ class Application extends BaseModel 'commands' => $commands, 'branch' => $branch, 'fullRepoUrl' => $fullRepoUrl, + 'git' => "{$deployKeyGitSshCommand} git", ]; } if ($this->deploymentType() === 'other') { @@ -2185,7 +2172,7 @@ class Application extends BaseModel $git_clone_command = str_replace('git clone', "{$gitCommand} clone", $git_clone_command); $git_clone_command = "{$git_clone_command} {$escapedCustomRepository} {$escapedBaseDir}"; $git_clone_command = $this->setGitImportSettings($deployment_uuid, $git_clone_command, public: true, commit: $commit, gitConfigOptions: $gitConfigOption, baseDir: $baseDir, onlyCheckout: $only_checkout); - $otherSshCommand = "ssh -o ConnectTimeout=30 -p {$customPort} -o Port={$customPort} -o LogLevel=ERROR -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i /root/.ssh/id_rsa"; + $otherSshCommand = gitSshCommand($customPort, '/root/.ssh/id_rsa', identitiesOnly: false); if ($pull_request_id !== 0) { if ($git_type === 'gitlab') { @@ -2224,6 +2211,7 @@ class Application extends BaseModel 'commands' => $this->gitCommandsAsShellCommand($commands), 'branch' => $branch, 'fullRepoUrl' => $fullRepoUrl, + 'git' => 'git', ]; } } @@ -2328,43 +2316,59 @@ class Application extends BaseModel */ public function serverCheckoutCommand(string $uuid, string $checkoutDir): string { - ['commands' => $cloneCommand] = $this->generateGitImportCommands(deployment_uuid: $uuid, only_checkout: true, exec_in_docker: false, custom_base_dir: $checkoutDir); - - return str_replace(' clone ', ' clone --quiet ', $this->gitCommandsAsShellCommand($cloneCommand)); + return $this->serverCheckout($uuid, $checkoutDir)['command']; } /** - * Commands that check out only the Compose file on the server and print it. They run on the - * server itself, not in a helper container, so the checkout uses an absolute folder in /tmp. + * The clone command of serverCheckoutCommand(), and the git command with the same credentials for later steps. + * + * @return array{command: string, git: string} + */ + private function serverCheckout(string $uuid, string $checkoutDir): array + { + ['commands' => $cloneCommand, 'git' => $git] = $this->generateGitImportCommands(deployment_uuid: $uuid, only_checkout: true, exec_in_docker: false, custom_base_dir: $checkoutDir); + + return [ + 'command' => str_replace(' clone ', ' clone --quiet ', $this->gitCommandsAsShellCommand($cloneCommand)), + 'git' => $git, + ]; + } + + /** + * Commands that clone the repository on the server and print the Compose file. They run on the + * server itself, not in a helper container, so the clone uses an absolute folder in /tmp. + * + * The file is read from the Git objects, not from a working tree, so no checkout is needed. Only + * regular files are loaded; a symbolic link (mode 120000) gets a clear error message. * * @return Collection */ - private function composeFileReadCommands(string $uuid, string $gitVersion): Collection + private function composeFileReadCommands(string $uuid): Collection { $checkoutDir = "/tmp/{$uuid}/checkout"; - $cloneCommand = $this->serverCheckoutCommand($uuid, $checkoutDir); - $workdir = rtrim($this->base_directory, '/'); - $fileList = collect([".{$workdir}{$this->docker_compose_location}"]); - $composeFilePath = escapeshellarg(".{$workdir}{$this->docker_compose_location}"); - $composeReadLimit = self::MAX_DOCKER_COMPOSE_SIZE_BYTES + 1; - $readComposeFile = "if [ \"$(wc -c < {$composeFilePath})\" -gt ".self::MAX_DOCKER_COMPOSE_SIZE_BYTES." ]; then echo '__COOLIFY_COMPOSE_TOO_LARGE__'; else head -c {$composeReadLimit} {$composeFilePath}; fi"; - - $isConeMode = version_compare($gitVersion, '2.35.1', '>='); - if (! $isConeMode) { - $fileList = $fileList->map(function ($file) { - $parts = explode('/', trim($file, '.')); - $paths = collect(); - $currentPath = ''; - foreach ($parts as $part) { - $currentPath .= ($currentPath ? '/' : '').$part; - if (str($currentPath)->isNotEmpty()) { - $paths->push($currentPath); - } - } - - return $paths; - })->flatten()->unique()->values(); - } + ['command' => $cloneCommand, 'git' => $git] = $this->serverCheckout($uuid, $checkoutDir); + $composeFile = ltrim(rtrim($this->base_directory, '/').$this->docker_compose_location, '/'); + $submoduleCommands = $this->composeFileSubmoduleCommands($git, $composeFile); + // Go into the submodule that contains the Compose file, and read the file from the objects of that submodule. + $enterSubmodule = $submoduleCommands === [] ? '' : implode(' ', [ + 'while sub=$(git ls-tree -r HEAD | while read -r mode type object path; do', + 'if [ "$mode" = 160000 ]; then case "$file" in "$path"/*) echo "$path"; break ;; esac; fi; done) && [ -n "$sub" ];', + 'do cd "$sub" || exit 1; file="${file#"$sub"/}"; done;', + ]); + $readComposeFile = implode(' ', [ + 'file='.escapeshellarg($composeFile).';', + $enterSubmodule, + 'git ls-tree HEAD -- "$file" | {', + 'read -r mode type object path;', + 'case "$mode" in', + '100644|100755) ;;', + "120000) echo 'The Docker Compose file is a symbolic link.' >&2; exit 1 ;;", + "*) echo 'No such file' >&2; exit 1 ;;", + 'esac;', + 'if [ "$(git cat-file -s "$object")" -gt '.self::MAX_DOCKER_COMPOSE_SIZE_BYTES." ]; then echo '__COOLIFY_COMPOSE_TOO_LARGE__';", + 'else git cat-file blob "$object"; fi;', + '}', + ]); return collect([ "rm -rf /tmp/{$uuid}", @@ -2373,13 +2377,34 @@ class Application extends BaseModel // One sh -c line: the non-root sudo parser would turn `&& if ...; then` into invalid `&& sudo if`. 'sh -c '.escapeshellarg($cloneCommand), "cd {$checkoutDir}", - $isConeMode ? 'git sparse-checkout init --cone' : 'git sparse-checkout init', - "git sparse-checkout set {$fileList->implode(' ')}", - 'git read-tree -mu HEAD', - $readComposeFile, + ...$submoduleCommands, + 'sh -c '.escapeshellarg($readComposeFile), ]); } + /** + * The clone has no checkout and no submodules. If the Compose file is inside a submodule, this fills the + * index (without a working tree) and clones only that submodule, with the credentials of the clone. + * + * @return list + */ + private function composeFileSubmoduleCommands(string $git, string $composeFile): array + { + if (! $this->settings?->is_git_submodules_enabled) { + return []; + } + + $depthFlag = $this->settings->is_git_shallow_clone_enabled ? ' --depth=1' : ''; + $escapedComposeFile = escapeshellarg($composeFile); + + // One sh -c line: the non-root sudo parser cannot change commands inside a loop. + return ['git read-tree HEAD', 'sh -c '.escapeshellarg( + 'git ls-files -s | while read -r mode object stage path; do ' + ."if [ \"\$mode\" = 160000 ]; then case {$escapedComposeFile} in \"\$path\"/*) " + ."{$git} submodule update --init --recursive{$depthFlag} -- \"\$path\" >/dev/null || exit 1 ;; esac; fi; done" + )]; + } + /** * Reads the Compose file from the repository and saves it only after validateDockerComposeForInjection() * accepts it, because the parser and the deployment build shell commands from parts of it. @@ -2401,8 +2426,7 @@ class Application extends BaseModel if (! $gitRemoteStatus['is_accessible']) { throw new RuntimeException('Failed to read Git source. Please verify repository access and try again.'.$this->gitFailureReason('Failed to read Git source.', (string) $gitRemoteStatus['error'])); } - $getGitVersion = instant_remote_process(['git --version'], $this->destination->server, false); - $commands = $this->composeFileReadCommands($uuid, (string) str($getGitVersion)->explode(' ')->last()); + $commands = $this->composeFileReadCommands($uuid); try { $composeFileContent = instant_remote_process($commands, $this->destination->server); if ($composeFileContent === '__COOLIFY_COMPOSE_TOO_LARGE__' || strlen($composeFileContent) > self::MAX_DOCKER_COMPOSE_SIZE_BYTES) { @@ -2414,6 +2438,9 @@ class Application extends BaseModel $this->base_directory = $initialBaseDirectory; $this->save(); + if (str($e->getMessage())->contains('The Docker Compose file is a symbolic link.')) { + throw new RuntimeException("Docker Compose file at $workdir$composeFile (branch: {$this->git_branch}) is a symbolic link. Coolify reads only regular files from the repository. Commit the file itself instead of a link to it."); + } if (str($e->getMessage())->contains('No such file')) { throw new RuntimeException("Docker Compose file not found at: $workdir$composeFile (branch: {$this->git_branch})

Check if you used the right extension (.yaml or .yml) in the compose file name."); } diff --git a/app/Models/ApplicationPreview.php b/app/Models/ApplicationPreview.php index 17b791d733..22b1856814 100644 --- a/app/Models/ApplicationPreview.php +++ b/app/Models/ApplicationPreview.php @@ -196,8 +196,23 @@ class ApplicationPreview extends BaseModel return $this->morphMany(LocalPersistentVolume::class, 'resource'); } - public function generate_preview_fqdn(bool $generateWithoutApplicationDomain = false) + /** + * Set the preview domains from the preview URL template. + * + * With $keepExistingDomains, domains already stored on the preview (for example custom + * domains set in the UI) are kept, and only an empty preview gets generated domains. + */ + public function generate_preview_fqdn(bool $generateWithoutApplicationDomain = false, bool $keepExistingDomains = false) { + if ($keepExistingDomains && filled($this->fqdn)) { + $normalized = DomainPortOverrides::normalize($this->fqdn, $this->domain_port_overrides); + $this->fqdn = $normalized['fqdn']; + $this->domain_port_overrides = $normalized['overrides']; + $this->save(); + + return $this; + } + $applicationFqdn = $this->application->fqdn; if (! $applicationFqdn && $generateWithoutApplicationDomain) { $applicationFqdn = generateUrl( @@ -221,7 +236,13 @@ class ApplicationPreview extends BaseModel return $this; } - public function generate_preview_fqdn_compose(bool $generateWithoutApplicationDomain = false) + /** + * Set the preview domain of every compose service from the preview URL template. + * + * With $keepExistingDomains, a service that already has domains on the preview keeps them, + * and only services without a preview domain get generated domains. + */ + public function generate_preview_fqdn_compose(bool $generateWithoutApplicationDomain = false, bool $keepExistingDomains = false) { $applicationDomains = json_decode($this->application->docker_compose_domains ?: '[]', true) ?: []; $previewDomains = json_decode(data_get($this, 'docker_compose_domains') ?: '[]', true) ?: []; @@ -262,6 +283,21 @@ class ApplicationPreview extends BaseModel $docker_compose_domains = []; $previewPortOverrides = []; foreach ($serviceNames as $service_name) { + $existingPreviewDomains = $keepExistingDomains + ? DomainPortOverrides::normalize(getComposeServiceDomainString($previewDomains, $service_name), $this->domain_port_overrides) + : ['fqdn' => null, 'overrides' => null]; + if (filled($existingPreviewDomains['fqdn'])) { + $docker_compose_domains = putComposeServiceDomain( + $docker_compose_domains, + $service_name, + $existingPreviewDomains['fqdn'], + $serviceNames, + ); + $previewPortOverrides = array_merge($previewPortOverrides, $existingPreviewDomains['overrides'] ?? []); + + continue; + } + $domain_string = getComposeServiceDomainString($applicationDomains, $service_name); if (empty($domain_string)) { diff --git a/app/Models/ApplicationSetting.php b/app/Models/ApplicationSetting.php index 13b608d2f0..7a7054e3ee 100644 --- a/app/Models/ApplicationSetting.php +++ b/app/Models/ApplicationSetting.php @@ -53,6 +53,7 @@ use Symfony\Component\Yaml\Yaml; 'include_source_commit_in_build' => ['type' => 'boolean'], 'docker_images_to_keep' => ['type' => 'integer'], 'stop_grace_period' => ['type' => 'integer', 'nullable' => true], + 'rolling_update_drain_seconds' => ['type' => 'integer', 'description' => 'Seconds to wait after the new container is healthy before old containers stop during a rolling update. 0 turns the wait off.'], ] )] class ApplicationSetting extends Model @@ -85,6 +86,7 @@ class ApplicationSetting extends Model 'is_git_shallow_clone_enabled' => 'boolean', 'docker_images_to_keep' => 'integer', 'stop_grace_period' => 'integer', + 'rolling_update_drain_seconds' => 'integer', 'is_log_drain_enabled' => 'boolean', 'is_gpu_enabled' => 'boolean', 'is_include_timestamps' => 'boolean', @@ -143,6 +145,7 @@ class ApplicationSetting extends Model 'include_source_commit_in_build', 'docker_images_to_keep', 'stop_grace_period', + 'rolling_update_drain_seconds', ]; /** @@ -170,6 +173,11 @@ class ApplicationSetting extends Model return DEFAULT_STOP_GRACE_PERIOD_SECONDS; } + public function rollingUpdateDrainSeconds(): int + { + return min(max($this->rolling_update_drain_seconds ?? DEFAULT_ROLLING_UPDATE_DRAIN_SECONDS, 0), MAX_ROLLING_UPDATE_DRAIN_SECONDS); + } + public function deploymentStopGracePeriodSeconds(): int { if (isDev() && $this->stop_grace_period === null) { diff --git a/app/Models/AuditEvent.php b/app/Models/AuditEvent.php index 2742a3f667..3020119781 100644 --- a/app/Models/AuditEvent.php +++ b/app/Models/AuditEvent.php @@ -200,7 +200,7 @@ class AuditEvent extends Model ?? trim(($resourceName ?? Str::headline((string) $resourceType)).' '.Str::headline($action)), 'metadata' => self::redact(Arr::except($context, ['audit_changes'])), 'changes' => $changes, - 'ip_address' => app()->bound('request') ? request()->ip() : null, + 'ip_address' => app()->bound('request') ? (client_ip(request()) ?: null) : null, 'user_agent' => app()->bound('request') ? Str::limit((string) request()->userAgent(), 200, '') : null, ]; } diff --git a/app/Models/Environment.php b/app/Models/Environment.php index 8eeecb0754..7cb659772d 100644 --- a/app/Models/Environment.php +++ b/app/Models/Environment.php @@ -57,6 +57,18 @@ class Environment extends BaseModel }); } + /** + * Sort order of environment names: production, staging, and development first, then the others by name. + * resources/js/environment-order.js has the same order for the canvases. + */ + public static function compareNames(string $first, string $second): int + { + $order = ['production', 'staging', 'development']; + $rank = fn (string $name): int => ($index = array_search(strtolower($name), $order, true)) === false ? count($order) : $index; + + return $rank($first) <=> $rank($second) ?: strnatcasecmp($first, $second); + } + public static function ownedByCurrentTeam() { return Environment::whereRelation('project.team', 'id', currentTeam()->id)->orderBy('name'); diff --git a/app/Models/EnvironmentVariable.php b/app/Models/EnvironmentVariable.php index 54627e92e8..3782c357c7 100644 --- a/app/Models/EnvironmentVariable.php +++ b/app/Models/EnvironmentVariable.php @@ -134,6 +134,7 @@ class EnvironmentVariable extends BaseModel 'is_runtime' => $environment_variable->is_runtime ?? false, 'is_buildtime' => $environment_variable->is_buildtime ?? false, 'is_required' => $environment_variable->is_required ?? false, + 'is_shown_once' => $environment_variable->is_shown_once ?? false, 'comment' => $environment_variable->comment, 'resourceable_type' => Application::class, 'resourceable_id' => $environment_variable->resourceable_id, diff --git a/app/Models/InstanceSettings.php b/app/Models/InstanceSettings.php index 8726a9f0f6..fbc9e7c099 100644 --- a/app/Models/InstanceSettings.php +++ b/app/Models/InstanceSettings.php @@ -105,6 +105,9 @@ class InstanceSettings extends Model 'webhook_allow_localhost' => 'boolean', 'is_dashboard_force_https_enabled' => 'boolean', 'is_passkey_enabled' => 'boolean', + 'server_validation_rate_limit_seconds' => 'integer', + 'server_validation_max_failures_per_ip' => 'integer', + 'server_validation_max_failures_per_team' => 'integer', ]; protected static function booted(): void diff --git a/app/Models/Project.php b/app/Models/Project.php index ea9afa94bb..26309801f8 100644 --- a/app/Models/Project.php +++ b/app/Models/Project.php @@ -33,6 +33,15 @@ class Project extends BaseModel 'uuid', ]; + protected array $auditExclude = ['canvas_position']; + + protected function casts(): array + { + return [ + 'canvas_position' => 'array', + ]; + } + /** * Get query builder for projects owned by current team. * If you need all projects without further query chaining, use ownedByCurrentTeamCached() instead. diff --git a/app/Models/S3Storage.php b/app/Models/S3Storage.php index 3ee7d5f269..9857b4faf2 100644 --- a/app/Models/S3Storage.php +++ b/app/Models/S3Storage.php @@ -273,6 +273,15 @@ class S3Storage extends BaseModel ]); } + /** + * The `docker run` option that makes the MinIO client sign requests for the storage region, as filesystem() does. + * Without it, mc signs for us-east-1, and providers that enforce the region reject the requests. + */ + public function minioClientRegionOption(): string + { + return '-e '.escapeshellarg('MC_REGION='.$this->region); + } + /** * The bundled MinIO container is a trusted internal S3 target, not a user-supplied webhook destination. * diff --git a/app/Models/ScheduledVolumeBackup.php b/app/Models/ScheduledVolumeBackup.php index 0c4443b2e3..2d71318f93 100644 --- a/app/Models/ScheduledVolumeBackup.php +++ b/app/Models/ScheduledVolumeBackup.php @@ -158,6 +158,20 @@ class ScheduledVolumeBackup extends BaseModel return $this->backupable?->resource; } + /** + * Link to the resource in the Coolify UI. Built from base_url(), because queued notifications have no request and route() would use APP_URL. + */ + public function resourceUrl(): ?string + { + $resource = $this->targetResource(); + $linkable = $resource instanceof ServiceApplication || $resource instanceof ServiceDatabase ? $resource->service : $resource; + if (! $linkable || ! method_exists($linkable, 'link') || ! $link = $linkable->link()) { + return null; + } + + return rtrim(base_url(), '/').parse_url($link, PHP_URL_PATH); + } + public function targetType(): string { return $this->backupable instanceof LocalFileVolume ? 'Directory' : 'Volume'; diff --git a/app/Models/Server.php b/app/Models/Server.php index 178b180d22..d5ffa27e76 100644 --- a/app/Models/Server.php +++ b/app/Models/Server.php @@ -34,11 +34,13 @@ use Illuminate\Database\Eloquent\Casts\Attribute; use Illuminate\Database\Eloquent\Factories\HasFactory; use Illuminate\Database\Eloquent\Relations\MorphMany; use Illuminate\Database\Eloquent\SoftDeletes; +use Illuminate\Http\Exceptions\ThrottleRequestsException; use Illuminate\Support\Carbon; use Illuminate\Support\Collection; use Illuminate\Support\Facades\Cache; use Illuminate\Support\Facades\DB; use Illuminate\Support\Facades\Log; +use Illuminate\Support\Facades\RateLimiter; use Illuminate\Support\Facades\Storage; use Illuminate\Support\Str; use Illuminate\Support\Stringable; @@ -131,6 +133,8 @@ class Server extends BaseModel public const PLACEHOLDER_IPS = [self::PLACEHOLDER_IP, '0.0.0.0', '::']; + protected array $auditExclude = ['canvas_position']; + /** * Serves the proxy error pages (503.html) for the Traefik and Caddy catch-all. */ @@ -338,6 +342,7 @@ class Server extends BaseModel 'delete_unused_networks' => 'boolean', 'force_disabled' => 'boolean', 'sentinel_waiting_since' => 'datetime', + 'canvas_position' => 'array', ]; /** @@ -411,6 +416,10 @@ class Server extends BaseModel */ public function backfillPlaceholderIp(?string $ip): bool { + if ($this->isTransferredAway()) { + return false; + } + $ip = blank($ip) ? null : self::normalizeIp($ip); if (self::isPlaceholderIp($ip)) { return false; @@ -455,6 +464,11 @@ class Server extends BaseModel public function refreshHetznerState(): ?string { + // Another instance manages the cloud server of an archive; keep the stored status. + if ($this->isTransferredAway()) { + return $this->hetzner_server_status; + } + if (! $this->hetzner_server_id || ! $this->cloudProviderToken || $this->cloudProviderToken->provider !== 'hetzner') { return $this->hetzner_server_status; } @@ -476,6 +490,11 @@ class Server extends BaseModel public function refreshVultrState(): ?string { + // Another instance manages the cloud server of an archive; keep the stored status. + if ($this->isTransferredAway()) { + return $this->vultr_instance_status; + } + if (! $this->vultr_instance_id || ! $this->cloudProviderToken) { return null; } @@ -512,6 +531,11 @@ class Server extends BaseModel public function refreshDigitalOceanState(): ?string { + // Another instance manages the cloud server of an archive; keep the stored status. + if ($this->isTransferredAway()) { + return $this->digitalocean_droplet_status; + } + if (! $this->digitalocean_droplet_id || ! $this->cloudProviderToken || $this->cloudProviderToken->provider !== 'digitalocean') { return $this->digitalocean_droplet_status; } @@ -553,6 +577,11 @@ class Server extends BaseModel public function refreshHostingerState(): ?string { + // Another instance manages the cloud server of an archive; keep the stored status. + if ($this->isTransferredAway()) { + return $this->hostinger_virtual_machine_status; + } + if (! $this->hostinger_virtual_machine_id || ! $this->cloudProviderToken || $this->cloudProviderToken->provider !== 'hostinger') { return $this->hostinger_virtual_machine_status; } @@ -1331,6 +1360,16 @@ $siteAddress { return $this->proxy->modelScope(); } + /** + * Servers that this instance manages. A transferred server is only an archive and does not use a server slot. + */ + public function scopeNotTransferredAway(Builder $query): Builder + { + return $query->where(fn (Builder $query) => $query + ->whereNull('server_metadata->transfer->status') + ->orWhere('server_metadata->transfer->status', '!=', 'transferred')); + } + public function scopeWhereProxyType(Builder $query, string $proxyType): Builder { return $query->where('proxy->type', $proxyType); @@ -1386,6 +1425,19 @@ $siteAddress { return data_get($this->server_metadata, 'transfer.status') === 'transferred'; } + /** + * Stops an action that changes the host, its cloud server, its Cloudflare tunnel, or its DNS records. + * Another Coolify instance manages a transferred server; this instance keeps only an archive of it. + * + * @throws \RuntimeException + */ + public function assertNotTransferredAway(): void + { + if ($this->isTransferredAway()) { + throw new \RuntimeException('This server is managed by another Coolify instance.'); + } + } + /** * Whether this server may be validated / installed against from this instance. */ @@ -1394,6 +1446,64 @@ $siteAddress { return ! $this->isTransferredAway(); } + /** + * On Coolify Cloud, limit how often SSH validations can run against one IP and from one team. + * The window and the maximums are server_validation_* instance settings. + * Without a limit, users can make Cloud try SSH logins on hosts that they do not own. + * A successful connection gives its attempt back, so only failed logins use up the limit. + * + * @throws ThrottleRequestsException + */ + public static function throttleValidation(int $teamId, string $ip): void + { + if (! isCloud()) { + return; + } + + $limits = self::validationRateLimits($teamId, $ip); + foreach ($limits as $key => $maxAttempts) { + if (RateLimiter::tooManyAttempts($key, $maxAttempts)) { + $seconds = RateLimiter::availableIn($key); + + throw new ThrottleRequestsException("Too many server validations. Please try again in {$seconds} seconds.", headers: ['Retry-After' => $seconds]); + } + } + foreach (array_keys($limits) as $key) { + RateLimiter::hit($key, self::validationRateLimitSeconds()); + } + } + + private static function validationRateLimitSeconds(): int + { + return instanceSettings()->server_validation_rate_limit_seconds; + } + + /** + * @return array rate limiter key => max attempts + */ + private static function validationRateLimits(int $teamId, string $ip): array + { + $settings = instanceSettings(); + + return [ + 'server-validation:ip:'.strtolower(self::normalizeIp($ip)) => $settings->server_validation_max_failures_per_ip, + 'server-validation:team:'.$teamId => $settings->server_validation_max_failures_per_team, + ]; + } + + private function refundValidationAttempt(): void + { + if (! isCloud()) { + return; + } + + foreach (array_keys(self::validationRateLimits($this->team_id, $this->ip)) as $key) { + if (RateLimiter::attempts($key) > 0) { + RateLimiter::decrement($key, self::validationRateLimitSeconds()); + } + } + } + public function forceEnableServer() { if ($this->isTransferredAway()) { @@ -1783,7 +1893,7 @@ $siteAddress { ->concat($dragonflies) ->concat($clickhouses) ->concat($sqlites) - ->filter(fn ($item) => data_get($item, 'name') !== 'coolify-db'); + ->reject(fn ($item) => $item instanceof StandalonePostgresql && $item->isInstanceDatabase()); } public function applications() @@ -2014,6 +2124,30 @@ $siteAddress { return true; } + /** + * Marks a proxy start or restart as running, so status checks do not start a second proxy meanwhile. + * ProxyStatusChangedNotification clears the mark when the start or restart finishes. + */ + public function markProxyTransitionStarted(): void + { + Cache::put($this->proxyTransitionCacheKey(), true, now()->addMinutes(5)); + } + + public function markProxyTransitionFinished(): void + { + Cache::forget($this->proxyTransitionCacheKey()); + } + + public function isProxyTransitionInProgress(): bool + { + return Cache::has($this->proxyTransitionCacheKey()); + } + + private function proxyTransitionCacheKey(): string + { + return "proxy-transition:{$this->uuid}"; + } + public function skipServer() { if ($this->hasPlaceholderIp()) { @@ -2320,7 +2454,9 @@ $siteAddress { return ['uptime' => false, 'error' => 'Server skipped.']; } try { - instant_remote_process(['ls /'], $this, disableMultiplexing: true); + // One attempt only: retries would repeat failed logins on a host that may not belong to the user. + instant_remote_process(['ls /'], $this, disableMultiplexing: true, maxAttempts: 1); + $this->refundValidationAttempt(); if ($this->settings->is_reachable === false) { $this->settings->is_reachable = true; $this->settings->save(); @@ -2455,13 +2591,11 @@ $siteAddress { public function validateDockerEngineVersion() { $dockerVersionRaw = instant_remote_process(['docker version --format json'], $this, false); - $dockerVersionJson = json_decode($dockerVersionRaw, true); - $dockerVersion = data_get($dockerVersionJson, 'Server.Version', '0.0.0'); - $this->rememberDockerVersion(is_string($dockerVersion) ? $dockerVersion : null); + $dockerVersion = dockerEngineVersionFromJson($dockerVersionRaw); + $this->rememberDockerVersion($dockerVersion); $composeVersionRaw = instant_remote_process(['docker compose version --short'], $this, false); $this->rememberComposeVersion(is_string($composeVersionRaw) ? $composeVersionRaw : null); - $dockerVersion = checkMinimumDockerEngineVersion($dockerVersion); - if (is_null($dockerVersion)) { + if (is_null(checkMinimumDockerEngineVersion($dockerVersion))) { $this->settings->is_usable = false; $this->settings->save(); @@ -2475,6 +2609,25 @@ $siteAddress { return true; } + /** + * Explains why validateDockerEngineVersion() failed: the version could not be read, or it is too old. + */ + public function dockerEngineVersionError(): string + { + $requiredVersion = str(config('constants.docker.minimum_required_version'))->before('.'); + $documentation = 'documentation'; + $dockerVersion = $this->dockerVersion(); + + if (is_null($dockerVersion)) { + $command = $this->isNonRoot() ? 'sudo docker version --format json' : 'docker version --format json'; + $user = e($this->user); + + return "Coolify could not read the Docker Engine version. Run \"{$command}\" on the server as the {$user} user. It must exit without an error and print only JSON. Text from shell startup files (for example ~/.bashrc) breaks the output."; + } + + return "Docker Engine {$dockerVersion} is installed, but Coolify needs version {$requiredVersion} or newer. Please upgrade Docker: {$documentation}."; + } + public function validateCoolifyNetwork($isSwarm = false, $isBuildServer = false) { if ($isBuildServer) { diff --git a/app/Models/Service.php b/app/Models/Service.php index 790bd1c978..c16d8974c2 100644 --- a/app/Models/Service.php +++ b/app/Models/Service.php @@ -1427,7 +1427,10 @@ class Service extends BaseModel if (filled($field['value']) && str($field['value'])->startsWith('$SERVICE_')) { $searchValue = str($field['value'])->after('$')->value; $newValue = $this->environment_variables()->where('key', $searchValue)->first(); - if ($newValue) { + if ($newValue?->is_shown_once) { + $field['value'] = null; + $field['isLocked'] = true; + } elseif ($newValue) { $field['value'] = $newValue->value; } } diff --git a/app/Models/ServiceApplication.php b/app/Models/ServiceApplication.php index e4d6282fe2..e28f09f6a2 100644 --- a/app/Models/ServiceApplication.php +++ b/app/Models/ServiceApplication.php @@ -10,6 +10,7 @@ use App\Traits\ReleasesManagedDnsRecords; use Illuminate\Database\Eloquent\Casts\Attribute; use Illuminate\Database\Eloquent\Factories\HasFactory; use Illuminate\Database\Eloquent\SoftDeletes; +use Symfony\Component\Yaml\Exception\ParseException; class ServiceApplication extends BaseModel { @@ -357,6 +358,33 @@ class ServiceApplication extends BaseModel return false; } + /** + * Whether the proxy labels of this container in the Compose file add HTTP basic auth + * (a Traefik `basicauth` middleware or a Caddy `basic_auth` directive). + */ + public function hasProxyBasicAuth(): bool + { + try { + $dockerCompose = parseDockerComposeYaml((string) data_get($this->service, 'docker_compose_raw')); + } catch (ParseException) { + return false; + } + + $labels = $dockerCompose['services'][$this->name]['labels'] ?? []; + if (! is_array($labels)) { + return false; + } + + foreach ($labels as $key => $value) { + $label = is_int($key) ? (string) $value : (string) $key; + if (preg_match('/basic_?auth/i', $label) === 1) { + return true; + } + } + + return false; + } + /** * Get the required port for this service application. * Extracts port from SERVICE_URL_* or SERVICE_FQDN_* environment variables diff --git a/app/Models/StandaloneDragonfly.php b/app/Models/StandaloneDragonfly.php index f9d7b3a29d..ecb844beb8 100644 --- a/app/Models/StandaloneDragonfly.php +++ b/app/Models/StandaloneDragonfly.php @@ -406,6 +406,15 @@ class StandaloneDragonfly extends BaseModel return $environmentVariable; } + /** + * Whether the password variable is locked (shown once) or reads a locked shared variable. + * The form, the connection URLs, and the API must then not show the password. + */ + public function isCredentialLocked(): bool + { + return (bool) $this->serverPasswordEnvironmentVariable()?->isLockedForLogs(); + } + /** * The server password for connection URLs. A remote secret is not fetched here, so the URL * shows its reference instead of a stale or wrong password. diff --git a/app/Models/StandaloneKeydb.php b/app/Models/StandaloneKeydb.php index 8b3aea2e01..d0b0e7a5e1 100644 --- a/app/Models/StandaloneKeydb.php +++ b/app/Models/StandaloneKeydb.php @@ -407,6 +407,15 @@ class StandaloneKeydb extends BaseModel return $environmentVariable; } + /** + * Whether the password variable is locked (shown once) or reads a locked shared variable. + * The form, the connection URLs, and the API must then not show the password. + */ + public function isCredentialLocked(): bool + { + return (bool) $this->serverPasswordEnvironmentVariable()?->isLockedForLogs(); + } + /** * The server password for connection URLs. A remote secret is not fetched here, so the URL * shows its reference instead of a stale or wrong password. diff --git a/app/Models/StandalonePostgresql.php b/app/Models/StandalonePostgresql.php index 6f8dbc3394..3d4c9ef035 100644 --- a/app/Models/StandalonePostgresql.php +++ b/app/Models/StandalonePostgresql.php @@ -321,6 +321,14 @@ class StandalonePostgresql extends BaseModel return 'standalone-postgresql'; } + /** + * Coolify's own database is the instance sentinel row (id 0). Users can give any database the name "coolify-db". + */ + public function isInstanceDatabase(): bool + { + return $this->id === 0; + } + protected function internalDbUrl(): Attribute { return new Attribute( diff --git a/app/Models/StandaloneRedis.php b/app/Models/StandaloneRedis.php index f7efb017b2..c68632132a 100644 --- a/app/Models/StandaloneRedis.php +++ b/app/Models/StandaloneRedis.php @@ -415,6 +415,15 @@ class StandaloneRedis extends BaseModel return $environmentVariable; } + /** + * Whether the password variable is locked (shown once) or reads a locked shared variable. + * The form, the connection URLs, and the API must then not show the password. + */ + public function isCredentialLocked(): bool + { + return (bool) $this->runtimeEnvironmentVariable('REDIS_PASSWORD')?->isLockedForLogs(); + } + /** * The server password for connection URLs. A remote secret is not fetched here, so the URL * shows its reference instead of a stale or wrong password. diff --git a/app/Models/Team.php b/app/Models/Team.php index 678df5a169..0126548963 100644 --- a/app/Models/Team.php +++ b/app/Models/Team.php @@ -60,17 +60,20 @@ class Team extends Model implements SendsDiscord, SendsEmail, SendsPushover, Sen 'custom_server_limit', 'is_mcp_server_enabled', 'is_build_server_fallback_enabled', + 'is_two_factor_required', ]; protected $attributes = [ 'is_mcp_server_enabled' => true, 'is_build_server_fallback_enabled' => true, + 'is_two_factor_required' => false, ]; protected $casts = [ 'personal_team' => 'boolean', 'is_mcp_server_enabled' => 'boolean', 'is_build_server_fallback_enabled' => 'boolean', + 'is_two_factor_required' => 'boolean', ]; protected static function booted() @@ -80,7 +83,7 @@ class Team extends Model implements SendsDiscord, SendsEmail, SendsPushover, Sen }); static::updating(function ($team) { - if (auth()->user()?->isMember()) { + if (auth()->user()?->roleInTeam($team->id) === 'member') { throw new \Exception('You are not allowed to update this team.'); } }); @@ -158,7 +161,7 @@ class Team extends Model implements SendsDiscord, SendsEmail, SendsPushover, Sen return true; } $serverLimit = Team::serverLimit($team); - $servers = $team->servers()->count(); + $servers = $team->managedServers()->count(); return $servers >= $serverLimit; } @@ -192,7 +195,7 @@ class Team extends Model implements SendsDiscord, SendsEmail, SendsPushover, Sen public function serverOverflow() { - if (Team::serverLimit($this) < $this->servers->count()) { + if (Team::serverLimit($this) < $this->managedServers()->count()) { return true; } @@ -392,6 +395,14 @@ class Team extends Model implements SendsDiscord, SendsEmail, SendsPushover, Sen return $this->hasMany(Server::class); } + /** + * Servers that use a server slot. Archives of transferred servers are left out. + */ + public function managedServers() + { + return $this->servers()->notTransferredAway(); + } + public function usesSwarm(): bool { return $this->servers() diff --git a/app/Notifications/ApiTokenExpiringNotification.php b/app/Notifications/ApiTokenExpiringNotification.php index 01f58af7ec..bb89d11637 100644 --- a/app/Notifications/ApiTokenExpiringNotification.php +++ b/app/Notifications/ApiTokenExpiringNotification.php @@ -50,7 +50,7 @@ class ApiTokenExpiringNotification extends CustomEmailNotification color: DiscordMessage::warningColor(), ); - $message->addField('Manage tokens', "[Open Security settings]({$this->manageUrl})"); + $message->addField('Manage tokens', "[Open API Tokens]({$this->manageUrl})"); return $message; } diff --git a/app/Notifications/VolumeBackup/BackupFailed.php b/app/Notifications/VolumeBackup/BackupFailed.php index 781a7b63e3..c1dc753cd7 100644 --- a/app/Notifications/VolumeBackup/BackupFailed.php +++ b/app/Notifications/VolumeBackup/BackupFailed.php @@ -3,8 +3,6 @@ namespace App\Notifications\VolumeBackup; use App\Models\ScheduledVolumeBackup; -use App\Models\ServiceApplication; -use App\Models\ServiceDatabase; use App\Notifications\CustomEmailNotification; use App\Notifications\Dto\DiscordMessage; use App\Notifications\Dto\PushoverMessage; @@ -29,10 +27,7 @@ class BackupFailed extends CustomEmailNotification $this->target = $backup->targetType().' '.$backup->targetName(); $this->frequency = $backup->frequency; - $linkable = $resource instanceof ServiceApplication || $resource instanceof ServiceDatabase ? $resource->service : $resource; - if ($linkable && method_exists($linkable, 'link')) { - $this->url = $linkable->link(); - } + $this->url = $backup->resourceUrl(); } public function via(object $notifiable): array diff --git a/app/Notifications/VolumeBackup/BackupMissing.php b/app/Notifications/VolumeBackup/BackupMissing.php index 63282e3c13..223ee1e0ee 100644 --- a/app/Notifications/VolumeBackup/BackupMissing.php +++ b/app/Notifications/VolumeBackup/BackupMissing.php @@ -3,8 +3,6 @@ namespace App\Notifications\VolumeBackup; use App\Models\ScheduledVolumeBackup; -use App\Models\ServiceApplication; -use App\Models\ServiceDatabase; use App\Notifications\CustomEmailNotification; use App\Notifications\Dto\DiscordMessage; use App\Notifications\Dto\PushoverMessage; @@ -27,10 +25,7 @@ class BackupMissing extends CustomEmailNotification $this->resourceName = $resource?->name ?? 'Unknown resource'; $this->target = $backup->targetType().' '.$backup->targetName(); - $linkable = $resource instanceof ServiceApplication || $resource instanceof ServiceDatabase ? $resource->service : $resource; - if ($linkable && method_exists($linkable, 'link')) { - $this->url = $linkable->link(); - } + $this->url = $backup->resourceUrl(); } public function via(object $notifiable): array diff --git a/app/Notifications/VolumeBackup/BackupSuccess.php b/app/Notifications/VolumeBackup/BackupSuccess.php index 4ff192971a..eaa4c823fb 100644 --- a/app/Notifications/VolumeBackup/BackupSuccess.php +++ b/app/Notifications/VolumeBackup/BackupSuccess.php @@ -3,8 +3,6 @@ namespace App\Notifications\VolumeBackup; use App\Models\ScheduledVolumeBackup; -use App\Models\ServiceApplication; -use App\Models\ServiceDatabase; use App\Notifications\CustomEmailNotification; use App\Notifications\Dto\DiscordMessage; use App\Notifications\Dto\PushoverMessage; @@ -29,10 +27,7 @@ class BackupSuccess extends CustomEmailNotification $this->target = $backup->targetType().' '.$backup->targetName(); $this->frequency = $backup->frequency; - $linkable = $resource instanceof ServiceApplication || $resource instanceof ServiceDatabase ? $resource->service : $resource; - if ($linkable && method_exists($linkable, 'link')) { - $this->url = $linkable->link(); - } + $this->url = $backup->resourceUrl(); } public function via(object $notifiable): array diff --git a/app/Notifications/VolumeBackup/RecoveryFailed.php b/app/Notifications/VolumeBackup/RecoveryFailed.php index ecb2220ec5..94713f76f5 100644 --- a/app/Notifications/VolumeBackup/RecoveryFailed.php +++ b/app/Notifications/VolumeBackup/RecoveryFailed.php @@ -4,8 +4,6 @@ namespace App\Notifications\VolumeBackup; use App\Jobs\VolumeBackupRecoveryJob; use App\Models\ScheduledVolumeBackupExecution; -use App\Models\ServiceApplication; -use App\Models\ServiceDatabase; use App\Notifications\CustomEmailNotification; use App\Notifications\Dto\DiscordMessage; use App\Notifications\Dto\PushoverMessage; @@ -39,10 +37,7 @@ class RecoveryFailed extends CustomEmailNotification }; $this->containersStopped = (bool) $execution->stop_recovery_pending; - $linkable = $resource instanceof ServiceApplication || $resource instanceof ServiceDatabase ? $resource->service : $resource; - if ($linkable && method_exists($linkable, 'link')) { - $this->url = $linkable->link(); - } + $this->url = $backup?->resourceUrl(); } public function via(object $notifiable): array diff --git a/app/Policies/GiteaAppPolicy.php b/app/Policies/GiteaAppPolicy.php index a269ead557..6def71365e 100644 --- a/app/Policies/GiteaAppPolicy.php +++ b/app/Policies/GiteaAppPolicy.php @@ -35,6 +35,14 @@ class GiteaAppPolicy return $user->isAdmin(); } + /** + * Determine whether the user can make a Gitea App available to every team on the instance. + */ + public function makeSystemWide(User $user): bool + { + return $user->canAccessSystemResources(); + } + /** * Determine whether the user can update the model. */ diff --git a/app/Policies/ServerPolicy.php b/app/Policies/ServerPolicy.php index c1369ce67f..2d24355485 100644 --- a/app/Policies/ServerPolicy.php +++ b/app/Policies/ServerPolicy.php @@ -39,6 +39,16 @@ class ServerPolicy return $user->isAdminOfTeam($server->team_id); } + /** + * Determine whether the user can transfer the server to another Coolify instance. + * + * A transfer sends every secret of the server, also locked environment variables, to the target, so only owners may start it. + */ + public function transfer(User $user, Server $server): bool + { + return $user->roleInTeam($server->team_id) === 'owner'; + } + /** * Determine whether the user can delete the model. */ diff --git a/app/Providers/AppServiceProvider.php b/app/Providers/AppServiceProvider.php index 655860a8eb..21db1febfd 100644 --- a/app/Providers/AppServiceProvider.php +++ b/app/Providers/AppServiceProvider.php @@ -13,6 +13,9 @@ use App\Support\AtomicFilesystem; use Illuminate\Database\Eloquent\Model; use Illuminate\Foundation\Console\ConfigCacheCommand; use Illuminate\Foundation\Console\RouteCacheCommand; +use Illuminate\Foundation\Http\Middleware\ConvertEmptyStringsToNull; +use Illuminate\Foundation\Http\Middleware\TrimStrings; +use Illuminate\Http\Request; use Illuminate\Support\Facades\App; use Illuminate\Support\Facades\DB; use Illuminate\Support\Facades\Http; @@ -46,6 +49,18 @@ class AppServiceProvider extends ServiceProvider $this->configureGitLabHttp(); $this->configureOidcSocialite(); $this->configureQueue(); + $this->configureServerTransferImport(); + } + + /** + * A server transfer bundle carries stored values (files, variables, keys, compose files) that must + * arrive unchanged, so the import request skips the global trimming and empty-string conversion. + */ + private function configureServerTransferImport(): void + { + $isImport = fn (Request $request): bool => $request->is('api/v1/servers/import'); + TrimStrings::skipWhen($isImport); + ConvertEmptyStringsToNull::skipWhen($isImport); } /** diff --git a/app/Providers/FortifyServiceProvider.php b/app/Providers/FortifyServiceProvider.php index d6519aac0e..8a2e049d97 100644 --- a/app/Providers/FortifyServiceProvider.php +++ b/app/Providers/FortifyServiceProvider.php @@ -8,6 +8,7 @@ use App\Actions\Fortify\UpdateUserPassword; use App\Models\OauthSetting; use App\Models\User; use App\Support\PasskeyRelyingParty; +use App\Support\RootSetupWindow; use Illuminate\Http\Request; use Illuminate\Support\Facades\Hash; use Illuminate\Support\ServiceProvider; @@ -51,6 +52,7 @@ class FortifyServiceProvider extends ServiceProvider return view('auth.register', [ 'isFirstUser' => $isFirstUser, + 'isRootSetupWindowClosed' => $isFirstUser && ! RootSetupWindow::isOpen(), ]); }); diff --git a/app/Providers/RouteServiceProvider.php b/app/Providers/RouteServiceProvider.php index dedac3a095..a4932d9ce1 100644 --- a/app/Providers/RouteServiceProvider.php +++ b/app/Providers/RouteServiceProvider.php @@ -46,23 +46,23 @@ class RouteServiceProvider extends ServiceProvider { RateLimiter::for('api', function (Request $request) { if ($request->path() === 'api/health') { - return Limit::perMinute(1000)->by($request->user()?->id ?: auth_rate_limit_ip($request)); + return Limit::perMinute(1000)->by($request->user()?->id ?: client_ip($request)); } - return Limit::perMinute((int) config('api.rate_limit'))->by($request->user()?->id ?: auth_rate_limit_ip($request)); + return Limit::perMinute((int) config('api.rate_limit'))->by($request->user()?->id ?: client_ip($request)); }); RateLimiter::for('5', function (Request $request) { - return Limit::perMinute(5)->by($request->user()?->id ?: auth_rate_limit_ip($request)); + return Limit::perMinute(5)->by($request->user()?->id ?: client_ip($request)); }); RateLimiter::for('feedback', function (Request $request) { - return Limit::perMinute(3)->by($request->user()?->id ?: auth_rate_limit_ip($request)); + return Limit::perMinute(3)->by($request->user()?->id ?: client_ip($request)); }); RateLimiter::for('login', function (Request $request) { $email = $request->input('email'); $emailIdentity = normalize_email_identity(is_string($email) ? $email : null); - $limits = [Limit::perMinute(5)->by((is_string($email) ? $email : '').'|'.auth_rate_limit_ip($request))]; + $limits = [Limit::perMinute(5)->by((is_string($email) ? $email : '').'|'.client_ip($request))]; if ($emailIdentity !== null) { $limits[] = Limit::perMinute(5)->by('login:email-identity:'.sha1($emailIdentity)); } @@ -75,12 +75,12 @@ class RouteServiceProvider extends ServiceProvider }); RateLimiter::for('passkeys', function (Request $request) { - return Limit::perMinute(30)->by('passkeys:'.auth_rate_limit_ip($request)); + return Limit::perMinute(30)->by('passkeys:'.client_ip($request)); }); RateLimiter::for('forgot-password', function (Request $request) { $limits = [ - Limit::perMinutes(10, 3)->by('forgot-password:ip:'.sha1(auth_rate_limit_ip($request))), + Limit::perMinutes(10, 3)->by('forgot-password:ip:'.sha1(client_ip($request))), ]; $emailIdentity = normalize_email_identity($request->input('email')); @@ -92,7 +92,7 @@ class RouteServiceProvider extends ServiceProvider }); RateLimiter::for('magic-link', function (Request $request) { - return Limit::perMinute(5)->by(hash('sha256', (string) $request->input('token').'|'.auth_rate_limit_ip($request))); + return Limit::perMinute(5)->by(hash('sha256', (string) $request->input('token').'|'.client_ip($request))); }); RateLimiter::for('force-password-reset', function (Request $request) { diff --git a/app/Rules/ValidGitRepositoryUrl.php b/app/Rules/ValidGitRepositoryUrl.php index 8b2d484e54..e30fb8c22b 100644 --- a/app/Rules/ValidGitRepositoryUrl.php +++ b/app/Rules/ValidGitRepositoryUrl.php @@ -34,6 +34,22 @@ class ValidGitRepositoryUrl implements ValidationRule $value, ); + // Git decodes %XX sequences, so check the decoded URL. A '%' that is left after decoding + // is a malformed or double-encoded sequence and is rejected with the other characters. + $decodedValue = rawurldecode($valueWithoutIpv6Host); + + // Control characters (newline, null byte, tab) can inject git credential-helper lines (CVE-2020-5260) + if (preg_match('/[\x00-\x1F\x7F]/', $decodedValue)) { + Log::warning('Git repository URL validation failed - control character', [ + 'url' => $value, + 'ip' => request()->ip(), + 'user_id' => auth()->id(), + ]); + $fail('The :attribute contains invalid characters.'); + + return; + } + // Check for dangerous shell metacharacters that could be used for command injection $dangerousChars = [ ';', '|', '&', '$', '`', '(', ')', '{', '}', @@ -43,7 +59,7 @@ class ValidGitRepositoryUrl implements ValidationRule ]; foreach ($dangerousChars as $char) { - if (str_contains($valueWithoutIpv6Host, $char)) { + if (str_contains($decodedValue, $char)) { Log::warning('Git repository URL validation failed - dangerous character', [ 'url' => $value, 'character' => $char, @@ -70,7 +86,7 @@ class ValidGitRepositoryUrl implements ValidationRule ]; foreach ($dangerousPatterns as $pattern) { - if (preg_match($pattern, $valueWithoutIpv6Host)) { + if (preg_match($pattern, $decodedValue)) { Log::warning('Git repository URL validation failed - dangerous pattern', [ 'url' => $value, 'pattern' => $pattern, @@ -93,7 +109,7 @@ class ValidGitRepositoryUrl implements ValidationRule // Validate scp-style SSH URL format (user@host:user/repo.git) $scp = parseScpStyleGitUrl($value); - if ($scp === null || preg_match('/^(\[[0-9A-Fa-f:.]+\]|[a-zA-Z0-9.-]+)$/', $scp['host']) !== 1 || preg_match('/^[a-zA-Z0-9\-_\/.~]+$/', $scp['path']) !== 1) { + if ($scp === null || preg_match('/^(\[[0-9A-Fa-f:.]+\]|[a-zA-Z0-9.-]+)$/', $scp['host']) !== 1 || preg_match('/^[a-zA-Z0-9\-_\/.~%]+$/', $scp['path']) !== 1) { $fail('The :attribute is not a valid SSH repository URL.'); return; @@ -106,7 +122,7 @@ class ValidGitRepositoryUrl implements ValidationRule } // Validate ssh:// URL format (ssh://user@host[:port]/user/repo.git) - if (! preg_match('/^ssh:\/\/[a-zA-Z0-9._-]+@(\[[0-9A-Fa-f:.]+\]|[a-zA-Z0-9.-]+)(:[0-9]{1,5})?\/[a-zA-Z0-9\-_\/.~]+$/', $value)) { + if (! preg_match('/^ssh:\/\/[a-zA-Z0-9._-]+@(\[[0-9A-Fa-f:.]+\]|[a-zA-Z0-9.-]+)(:[0-9]{1,5})?\/[a-zA-Z0-9\-_\/.~%]+$/', $value)) { $fail('The :attribute is not a valid SSH repository URL.'); return; @@ -151,6 +167,13 @@ class ValidGitRepositoryUrl implements ValidationRule return; } + // Percent-encoding is only allowed in the path, because hosts and credentials can be decoded differently + if (str_contains(($parsed['host'] ?? '').($parsed['user'] ?? '').($parsed['pass'] ?? ''), '%')) { + $fail('The :attribute contains invalid characters.'); + + return; + } + // Ensure no query parameters or fragments if (! empty($parsed['query']) || ! empty($parsed['fragment'])) { $fail('The :attribute should not contain query parameters or fragments.'); @@ -160,14 +183,14 @@ class ValidGitRepositoryUrl implements ValidationRule // Validate path contains only safe characters $path = $parsed['path'] ?? ''; - if (! empty($path) && ! preg_match('/^[a-zA-Z0-9\-_\/\.@~]+$/', $path)) { + if (! empty($path) && ! preg_match('/^[a-zA-Z0-9\-_\/\.@~%]+$/', $path)) { $fail('The :attribute path contains invalid characters.'); return; } } elseif (str_starts_with($value, 'git://')) { // Validate git:// protocol URL (supports both git://host/path and git://host:port/path with tilde) - if (! preg_match('/^git:\/\/(\[[0-9A-Fa-f:.]+\]|[a-zA-Z0-9\.\-]+)(:[0-9]+)?[:\/][a-zA-Z0-9\-_\/\.~]+$/', $value)) { + if (! preg_match('/^git:\/\/(\[[0-9A-Fa-f:.]+\]|[a-zA-Z0-9\.\-]+)(:[0-9]+)?[:\/][a-zA-Z0-9\-_\/\.~%]+$/', $value)) { $fail('The :attribute is not a valid git:// URL.'); return; diff --git a/app/Services/CloudflareTunnelService.php b/app/Services/CloudflareTunnelService.php index 358b4075d4..09c2ab72bf 100644 --- a/app/Services/CloudflareTunnelService.php +++ b/app/Services/CloudflareTunnelService.php @@ -55,6 +55,7 @@ class CloudflareTunnelService */ public function setUp(Server $server, IntegrationToken $token, string $hostname, ?array $confirmedConflict = null): Activity { + $server->assertNotTransferredAway(); $zone = $this->zoneFor($server, $token, $hostname); $accountId = $zone->account_id; if (blank($accountId)) { @@ -109,6 +110,10 @@ class CloudflareTunnelService if (blank($settings->cloudflare_tunnel_id)) { return []; } + // Another Coolify instance uses the tunnel of a transferred server. + if ($server->isTransferredAway()) { + return ['This server is managed by another Coolify instance.']; + } $warnings = []; $name = self::tunnelName($server); diff --git a/app/Services/ContainerFilesystemService.php b/app/Services/ContainerFilesystemService.php index e7178ed765..5392f7d483 100644 --- a/app/Services/ContainerFilesystemService.php +++ b/app/Services/ContainerFilesystemService.php @@ -3,6 +3,7 @@ namespace App\Services; use App\Data\FileEntry; +use App\Exceptions\ContainerShellUnavailableException; use App\Helpers\SshMultiplexingHelper; use App\Models\LocalFileVolume; use App\Models\Server; @@ -20,9 +21,30 @@ class ContainerFilesystemService /** Largest file, or uncompressed folder, the file browser may download. */ public const MAX_DOWNLOAD_SIZE = 1024 * 1024 * 1024; + /** + * Linux capability names by bit number, as Docker expects them in --cap-add. + */ + private const CAPABILITIES = [ + 'CHOWN', 'DAC_OVERRIDE', 'DAC_READ_SEARCH', 'FOWNER', 'FSETID', 'KILL', 'SETGID', 'SETUID', + 'SETPCAP', 'LINUX_IMMUTABLE', 'NET_BIND_SERVICE', 'NET_BROADCAST', 'NET_ADMIN', 'NET_RAW', + 'IPC_LOCK', 'IPC_OWNER', 'SYS_MODULE', 'SYS_RAWIO', 'SYS_CHROOT', 'SYS_PTRACE', 'SYS_PACCT', + 'SYS_ADMIN', 'SYS_BOOT', 'SYS_NICE', 'SYS_RESOURCE', 'SYS_TIME', 'SYS_TTY_CONFIG', 'MKNOD', + 'LEASE', 'AUDIT_WRITE', 'AUDIT_CONTROL', 'SETFCAP', 'MAC_OVERRIDE', 'MAC_ADMIN', 'SYSLOG', + 'WAKE_ALARM', 'BLOCK_SUSPEND', 'AUDIT_READ', 'PERFMON', 'BPF', 'CHECKPOINT_RESTORE', + ]; + + /** @var array{user: string, groups: array, capabilities: array}|null */ + private ?array $helperIdentity = null; + + /** + * With $useHelper, commands run in a short-lived helper container that + * shares the PID namespace of the target and reaches its files through + * /proc/1/root. This supports images without a shell (scratch, distroless). + */ public function __construct( private Server $server, private string $container, + private bool $useHelper = false, ) {} public function buildListCommand(string $path): string @@ -41,18 +63,34 @@ class ContainerFilesystemService // strips that prefix. Names containing a tab/newline are skipped (v1 scope). $tab = "\t"; $format = '%F'.$tab.'%s'.$tab.'%Y'.$tab.'%a'.$tab.'%U'.$tab.'%G'.$tab.'%n'; - $inner = 'cd '.$escapedPath.' 2>/dev/null || exit 0; ' - .'find . ! -name . -prune -exec stat -c "'.$format.'" -- {} + 2>/dev/null'; + // Exit 127 when find or stat is missing, so list() can switch to the helper. + $inner = 'command -v find >/dev/null && command -v stat >/dev/null || exit 127; ' + .'cd '.$escapedPath.' 2>/dev/null || exit 0; ' + .'find . ! -name . -prune -exec stat -c "'.$format.'" -- {} + 2>/dev/null; exit 0'; return $this->dockerExecShell($inner); } /** + * docker exec exits with 126 or 127 when it cannot start `sh` or the + * listing tools in the container. Other docker exec failures still list + * as an empty folder. In helper mode, every failure is reported. + * * @return array + * + * @throws ContainerShellUnavailableException when the files cannot be listed this way */ public function list(string $path): array { - $raw = instant_remote_process([$this->buildListCommand($path)], $this->server, throwError: false); + try { + $raw = instant_remote_process([$this->buildListCommand($path)], $this->server); + } catch (\RuntimeException $e) { + if ($this->useHelper || in_array($e->getCode(), [126, 127], true)) { + throw new ContainerShellUnavailableException($e->getMessage(), (int) $e->getCode(), $e); + } + + return []; + } return $this->parseListing($raw); } @@ -96,6 +134,10 @@ class ContainerFilesystemService if ($status !== 'OK' || $content === false || strlen($content) > $max) { throw new \RuntimeException('Could not read file.'); } + // BusyBox grep ignores -I, so a binary file can pass the remote probe. + if (str_contains($content, "\0")) { + throw new \RuntimeException('File is not editable (binary or too large).'); + } return $content; } @@ -337,6 +379,9 @@ class ContainerFilesystemService public function buildDownloadCommand(string $path, bool $isDir): string { $escaped = $this->escapePath($path, 'download path'); + if ($this->useHelper) { + return $this->dockerExecShell($isDir ? "tar czf - -C {$escaped} ." : "cat -- {$escaped}"); + } $escapedContainer = escapeshellarg($this->container); return $isDir @@ -414,13 +459,94 @@ class ContainerFilesystemService { validateShellSafePath($path, $label); - return escapeshellarg($path); + return escapeshellarg(($this->useHelper ? '/proc/1/root' : '').$path); } protected function dockerExecShell(string $innerShell): string { $escapedContainer = escapeshellarg($this->container); + if (! $this->useHelper) { + return "docker exec {$escapedContainer} sh -c ".escapeshellarg($innerShell); + } - return "docker exec {$escapedContainer} sh -c ".escapeshellarg($innerShell); + // The helper gets exactly the user, groups and capabilities of the + // target process: the kernel requires them to reach /proc/1/root, + // and nothing more is given. It has no network, a read-only root + // file system, and no Docker socket or host mounts, and it is removed + // when the command ends. Exit 126 means the kernel denied access. + $identity = $this->helperIdentity(); + $options = [ + '--rm', + '--pid container:'.$escapedContainer, + '--user '.escapeshellarg($identity['user']), + ...array_map(fn (string $group) => '--group-add '.escapeshellarg($group), $identity['groups']), + '--cap-drop ALL', + ...array_map(fn (string $capability) => '--cap-add '.$capability, $identity['capabilities']), + '--security-opt no-new-privileges', + '--network none', + '--ipc none', + '--read-only', + '--log-driver none', + '--pids-limit 64', + '--label coolify.file-browser=true', + "--entrypoint ''", + ]; + $image = escapeshellarg(coolifyHelperImage().':'.getHelperVersion()); + + return 'docker run '.implode(' ', $options)." {$image} sh -c " + .escapeshellarg('[ -x /proc/1/root/ ] || exit 126; '.$innerShell); + } + + /** + * Read the effective user, group, supplementary groups and permitted + * capabilities of the main process of the target container. + * + * @return array{user: string, groups: array, capabilities: array} + */ + protected function helperIdentity(): array + { + if ($this->helperIdentity !== null) { + return $this->helperIdentity; + } + + $escapedContainer = escapeshellarg($this->container); + $status = (string) instant_remote_process( + ["cat /proc/$(docker inspect --format '{{.State.Pid}}' {$escapedContainer})/status"], + $this->server, + ); + + return $this->helperIdentity = $this->parseProcessStatus($status); + } + + /** + * @return array{user: string, groups: array, capabilities: array} + */ + public function parseProcessStatus(string $status): array + { + $fields = []; + foreach (explode("\n", $status) as $line) { + [$key, $value] = array_pad(explode(':', $line, 2), 2, ''); + $fields[trim($key)] = preg_split('/\s+/', trim($value), -1, PREG_SPLIT_NO_EMPTY); + } + + $uid = $fields['Uid'][1] ?? ''; + $gid = $fields['Gid'][1] ?? ''; + $capabilityMask = $fields['CapPrm'][0] ?? ''; + if (! ctype_digit($uid) || ! ctype_digit($gid) || ! ctype_xdigit($capabilityMask) || strlen($capabilityMask) > 16) { + throw new \RuntimeException('Could not read the user of the container process.'); + } + + $groups = array_values(array_filter($fields['Groups'] ?? [], fn (string $group) => ctype_digit($group) && $group !== $gid)); + + // Read the mask one hex digit at a time; hexdec() turns 64-bit masks into floats. + $digits = strrev($capabilityMask); + $capabilities = []; + foreach (self::CAPABILITIES as $bit => $name) { + if (hexdec($digits[intdiv($bit, 4)] ?? '0') & (1 << ($bit % 4))) { + $capabilities[] = $name; + } + } + + return ['user' => "{$uid}:{$gid}", 'groups' => $groups, 'capabilities' => $capabilities]; } } diff --git a/app/Services/DeploymentConfiguration/ApplicationConfigurationSnapshot.php b/app/Services/DeploymentConfiguration/ApplicationConfigurationSnapshot.php index b632a1cedc..7ae07245ad 100644 --- a/app/Services/DeploymentConfiguration/ApplicationConfigurationSnapshot.php +++ b/app/Services/DeploymentConfiguration/ApplicationConfigurationSnapshot.php @@ -100,6 +100,36 @@ class ApplicationConfigurationSnapshot ]; } + /** + * Removes the volumes of Docker Compose preview deployments from a snapshot. Older snapshots + * can contain them, because the parser stores preview volumes on the application. + * + * @param array $snapshot + * @return array + */ + public static function withoutPreviewVolumes(array $snapshot, Application $application): array + { + if (! is_array(data_get($snapshot, 'sections.storage.items'))) { + return $snapshot; + } + + $snapshot['sections']['storage']['items'] = collect($snapshot['sections']['storage']['items']) + ->reject(fn (array $item): bool => str_starts_with((string) data_get($item, 'key'), 'volume_') + && self::isPreviewVolumeName((string) data_get($item, 'compare_value.source'), $application)) + ->values() + ->all(); + + return $snapshot; + } + + /** + * The compose parser names preview volumes "{uuid}_{volume}-pr-{id}" ("{uuid}-{volume}-pr-{id}" before parser version 3). + */ + private static function isPreviewVolumeName(string $name, Application $application): bool + { + return preg_match('/^'.preg_quote($application->uuid, '/').'[_-].+-pr-\d+$/', $name) === 1; + } + /** * @return array> */ @@ -237,6 +267,7 @@ class ApplicationConfigurationSnapshot $volumes = $this->application->persistentStorages() ->orderBy('id') ->get(['id', 'name', 'mount_path', 'host_path']) + ->reject(fn (LocalPersistentVolume $volume): bool => blank($volume->host_path) && self::isPreviewVolumeName($volume->name, $this->application)) ->map(function (LocalPersistentVolume $volume): array { $source = $volume->host_path ?: $volume->name; diff --git a/app/Services/DeploymentConfiguration/ConfigurationDiff.php b/app/Services/DeploymentConfiguration/ConfigurationDiff.php index 3f0477ba35..b648331c53 100644 --- a/app/Services/DeploymentConfiguration/ConfigurationDiff.php +++ b/app/Services/DeploymentConfiguration/ConfigurationDiff.php @@ -17,25 +17,13 @@ class ConfigurationDiff return new self; } - public static function legacy(bool $changed): self + /** + * The last deployment stored no configuration snapshot (it ran before v4.1), so the + * deployed configuration is unknown. Report no changes, but build on the next deployment. + */ + public static function legacy(): self { - if (! $changed) { - return self::unchanged(); - } - - return new self([ - [ - 'key' => 'legacy.configuration', - 'section' => 'configuration', - 'section_label' => 'Configuration', - 'label' => 'Configuration', - 'type' => 'changed', - 'impact' => 'build', - 'sensitive' => false, - 'old_display_value' => 'Previously deployed configuration', - 'new_display_value' => 'Current configuration', - ], - ], true); + return new self(legacyFallback: true); } /** @@ -63,7 +51,7 @@ class ConfigurationDiff public function requiresBuild(): bool { - return collect($this->changes)->contains(fn (array $change): bool => $change['impact'] === 'build'); + return $this->legacyFallback || collect($this->changes)->contains(fn (array $change): bool => $change['impact'] === 'build'); } public function requiresRedeploy(): bool diff --git a/app/Services/DeploymentConfiguration/ConfigurationDiffer.php b/app/Services/DeploymentConfiguration/ConfigurationDiffer.php index 688db3f0a4..874d0424cb 100644 --- a/app/Services/DeploymentConfiguration/ConfigurationDiffer.php +++ b/app/Services/DeploymentConfiguration/ConfigurationDiffer.php @@ -50,8 +50,13 @@ class ConfigurationDiffer } $item = $current ?? $previous; - $sensitive = (bool) data_get($item, 'sensitive', false); $type = $previous === null ? 'added' : ($current === null ? 'removed' : 'changed'); + // A value is hidden when either snapshot hides it. A removed environment variable may + // have been locked after the previous deployment, which no snapshot records, so its + // old value is hidden too. + $sensitive = (bool) data_get($previous, 'sensitive', false) + || (bool) data_get($current, 'sensitive', false) + || ($type === 'removed' && data_get($item, 'section') === 'environment'); $displaySummary = $sensitive && $type === 'changed' ? 'Changed' : null; $diffMode = data_get($item, 'diff_mode', 'default'); diff --git a/app/Services/Dns/CloudflareDnsProvider.php b/app/Services/Dns/CloudflareDnsProvider.php index 9cd21a4fbf..9c7748ff8b 100644 --- a/app/Services/Dns/CloudflareDnsProvider.php +++ b/app/Services/Dns/CloudflareDnsProvider.php @@ -4,9 +4,13 @@ namespace App\Services\Dns; use App\Enums\ManagedDnsDeletionResult; use App\Exceptions\DnsRecordConflictException; +use App\Models\Application; +use App\Models\ApplicationPreview; use App\Models\DnsProviderZone; use App\Models\IntegrationToken; use App\Models\ManagedDnsRecord; +use App\Models\Server; +use App\Models\ServiceApplication; use App\Support\DnsRecordHints; use Illuminate\Contracts\Cache\LockTimeoutException; use Illuminate\Database\Eloquent\Model; @@ -141,6 +145,7 @@ class CloudflareDnsProvider */ public function createRecord(DnsProviderZone $zone, string $hostname, string $content, ?Model $resource = null, array $additionalResources = [], ?bool $proxied = null): ManagedDnsRecord { + $this->assertResourcesManagedHere([$resource, ...$additionalResources]); $hostname = strtolower(rtrim($hostname, '.')); return $this->hostnameLock->run( @@ -224,6 +229,7 @@ class CloudflareDnsProvider array $additionalResources = [], bool $proxied = false, ): ManagedDnsRecord { + $this->assertResourcesManagedHere([$resource, ...$additionalResources]); $hostname = strtolower(rtrim($hostname, '.')); return $this->hostnameLock->run( @@ -291,6 +297,27 @@ class CloudflareDnsProvider * Deletes the provider record only when Coolify created it and it still carries Coolify's ownership comment and value. * Removes the local row when the provider record is gone. */ + /** + * Another Coolify instance manages the DNS records of the resources of a transferred server. + * + * @param array $resources + * + * @throws RuntimeException + */ + private function assertResourcesManagedHere(array $resources): void + { + foreach ($resources as $resource) { + $server = match (true) { + $resource instanceof Server => $resource, + $resource instanceof Application => $resource->destination?->server, + $resource instanceof ApplicationPreview => $resource->application?->destination?->server, + $resource instanceof ServiceApplication => $resource->service?->server, + default => null, + }; + $server?->assertNotTransferredAway(); + } + } + public function deleteRecord(ManagedDnsRecord $record, ?Model $resource = null): ManagedDnsDeletionResult { if (! $record->owned) { diff --git a/app/Services/EnvironmentCanvas.php b/app/Services/EnvironmentCanvas.php index 8372af3371..6a7ffcea82 100644 --- a/app/Services/EnvironmentCanvas.php +++ b/app/Services/EnvironmentCanvas.php @@ -137,6 +137,18 @@ class EnvironmentCanvas ]; } + /** + * A saved zone position (projects and servers canvas), or null when the zone is auto placed. + * + * @return array{x: int, y: int}|null + */ + public function zonePosition(mixed $saved): ?array + { + return is_array($saved) && is_numeric($saved['x'] ?? null) && is_numeric($saved['y'] ?? null) + ? ['x' => (int) $saved['x'], 'y' => (int) $saved['y']] + : null; + } + /** * @return array{id: string, name: string, ip: ?string, is_reachable: bool, is_usable: bool, link: string} */ diff --git a/app/Services/FleetMetricsAggregator.php b/app/Services/FleetMetricsAggregator.php index 29e0459d3b..f739abf2e8 100644 --- a/app/Services/FleetMetricsAggregator.php +++ b/app/Services/FleetMetricsAggregator.php @@ -13,25 +13,27 @@ class FleetMetricsAggregator public static function fleetKpis(array $rows): array { $online = array_values(array_filter($rows, fn ($r) => $r['online'] === true)); + // Only servers whose Sentinel returned current metrics give values. + $reporting = array_values(array_filter($online, fn ($r) => $r['reporting'] === true)); - $cpuValues = self::present($online, 'cpu'); + $cpuValues = self::present($reporting, 'cpu'); $cpuAvg = $cpuValues === [] ? 0.0 : round(array_sum($cpuValues) / count($cpuValues), 1); $busiest = null; - foreach ($online as $r) { + foreach ($reporting as $r) { if ($r['cpu'] !== null && ($busiest === null || $r['cpu'] > $busiest['percent'])) { $busiest = ['name' => $r['name'], 'percent' => (float) $r['cpu']]; } } - $memUsed = (int) self::sum($online, 'memUsed'); - $memTotal = (int) self::sum($online, 'memTotal'); - $diskUsed = (int) self::sum($online, 'diskUsed'); - $diskTotal = (int) self::sum($online, 'diskTotal'); - $load1 = self::present($online, 'load1'); + $memUsed = (int) self::sum($reporting, 'memUsed'); + $memTotal = (int) self::sum($reporting, 'memTotal'); + $diskUsed = (int) self::sum($reporting, 'diskUsed'); + $diskTotal = (int) self::sum($reporting, 'diskTotal'); + $load1 = self::present($reporting, 'load1'); $needsAttention = 0; - foreach ($online as $r) { + foreach ($reporting as $r) { $hot = PressureLevel::for($r['cpu'], 'cpu') !== 'ok' || PressureLevel::for(self::percent($r['memUsed'], $r['memTotal']), 'memory') !== 'ok' || PressureLevel::for($r['diskPercent'], 'disk') !== 'ok'; @@ -42,6 +44,7 @@ class FleetMetricsAggregator return [ 'serversOnline' => count($online), + 'serversReporting' => count($reporting), 'serversTotal' => count($rows), 'needsAttention' => $needsAttention, 'cpuAvg' => $cpuAvg, @@ -53,11 +56,11 @@ class FleetMetricsAggregator 'diskUsed' => $diskUsed, 'diskTotal' => $diskTotal, 'diskPercent' => self::percent($diskUsed, $diskTotal), - 'netRx' => (float) self::sum($online, 'netRx'), - 'netTx' => (float) self::sum($online, 'netTx'), + 'netRx' => (float) self::sum($reporting, 'netRx'), + 'netTx' => (float) self::sum($reporting, 'netTx'), 'loadBusiest' => $load1 === [] ? 0.0 : max($load1), 'loadAvg' => $load1 === [] ? 0.0 : round(array_sum($load1) / count($load1), 2), - 'containers' => (int) self::sum($online, 'containers'), + 'containers' => (int) self::sum($reporting, 'containers'), ]; } diff --git a/app/Services/InfisicalService.php b/app/Services/InfisicalService.php index 3be8de7728..86bf64bded 100644 --- a/app/Services/InfisicalService.php +++ b/app/Services/InfisicalService.php @@ -46,6 +46,7 @@ class InfisicalService 'projectId' => $projectId, 'environment' => $environment, 'secretPath' => $secretPath, + 'includeImports' => 'true', ...$valueOptions, ]); @@ -55,6 +56,7 @@ class InfisicalService 'workspaceId' => $projectId, 'environment' => $environment, 'secretPath' => $secretPath, + 'include_imports' => 'true', ...$valueOptions, ]); } @@ -63,7 +65,20 @@ class InfisicalService throw new \RuntimeException('Infisical API error: '.($response->json('message') ?? 'HTTP '.$response->status())); } - return collect($response->json('secrets', [])) + // Like `infisical export`: later imports override earlier ones, and the folder's own secrets override all imports. + return collect($response->json('imports', [])) + ->pluck('secrets') + ->push($response->json('secrets', [])) + ->reduce(fn (array $merged, ?array $secrets) => array_replace($merged, $this->keyValues($secrets ?? [])), []); + } + + /** + * @param array> $secrets + * @return array + */ + private function keyValues(array $secrets): array + { + return collect($secrets) ->mapWithKeys(fn ($secret) => [(string) data_get($secret, 'secretKey') => (string) data_get($secret, 'secretValue', '')]) ->all(); } diff --git a/app/Services/ProjectCanvas.php b/app/Services/ProjectCanvas.php index 57619b2e30..65fd7b2540 100644 --- a/app/Services/ProjectCanvas.php +++ b/app/Services/ProjectCanvas.php @@ -20,11 +20,11 @@ class ProjectCanvas /** * The canvas of every project of a team: one zone per project, cards grouped by environment. * - * @return array{title: string, title_link: string, can_edit: bool, can_move: false, can_move_groups: bool, layout: 'server', zones: array>, nodes: array>, links: array>, positions: array{}} + * @return array{title: string, title_link: string, can_edit: bool, can_move: false, can_move_groups: bool, can_move_zones: bool, layout: 'server', zones: array>, nodes: array>, links: array>, positions: array{}} */ public function teamPayload(int $teamId, bool $canReadSensitive, bool $canEdit = false): array { - $projects = Project::query()->where('team_id', $teamId)->orderBy('name')->get(['id', 'uuid', 'name', 'team_id']); + $projects = Project::query()->where('team_id', $teamId)->orderBy('name')->get(['id', 'uuid', 'name', 'team_id', 'canvas_position']); $environments = Environment::query()->whereIn('project_id', $projects->pluck('id')) ->orderBy('id') ->get(['id', 'uuid', 'name', 'project_id']); @@ -48,6 +48,8 @@ class ProjectCanvas 'can_move' => false, // Admins move groups and cards here; the canvas saves them on their environment canvas. 'can_move_groups' => $canEdit, + // Admins move the project zones; each project saves its place (zone.position). + 'can_move_zones' => $canEdit, // Same grouped layout as the server canvas, with projects as the zones. 'layout' => 'server', 'zones' => $projects->map(fn (Project $project) => [ @@ -55,6 +57,7 @@ class ProjectCanvas 'name' => $project->name, 'icon' => 'projects', 'link' => route('project.show', ['project_uuid' => $project->uuid]), + 'position' => $this->canvas->zonePosition($project->canvas_position), ])->values()->all(), 'nodes' => collect($graph['nodes'])->map(fn (array $node) => [...$node, ...$grouping->get($node['id'])]) ->concat($projects->flatMap(fn (Project $project) => $this->emptyNodes($project, $environments->where('project_id', $project->id), $resources, $project->uuid, $canEdit))) diff --git a/app/Services/ProjectIconStorageService.php b/app/Services/ProjectIconStorageService.php index 0c6d314816..2a0b3dba51 100644 --- a/app/Services/ProjectIconStorageService.php +++ b/app/Services/ProjectIconStorageService.php @@ -22,6 +22,10 @@ class ProjectIconStorageService extends AvatarStorageService $storageType = $settings->avatar_storage_type === 's3' && $settings->avatar_s3_storage_id ? 's3' : 'local'; $s3StorageId = $storageType === 's3' ? $settings->avatar_s3_storage_id : null; $disk = $this->disk($storageType, $s3StorageId); + // The uuid is a path segment, so it must not leave the project-icons directory. + if (preg_match('/\A[a-zA-Z0-9_-]+\z/', (string) $project->uuid) !== 1) { + throw new RuntimeException('Unable to store the project icon: the project UUID is not valid.'); + } $path = "project-icons/{$project->uuid}/icon.jpg"; if (! $disk->put($path, $contents)) { diff --git a/app/Services/SentinelMetricsClient.php b/app/Services/SentinelMetricsClient.php index 7d11edd852..b9ad06a164 100644 --- a/app/Services/SentinelMetricsClient.php +++ b/app/Services/SentinelMetricsClient.php @@ -81,6 +81,10 @@ class SentinelMetricsClient return []; } + // Sentinel keeps the last samples of a removed container until retention deletes them, + // but only a container that Docker still lists has a status. + $running = array_filter($rows, fn ($c) => ($c['status']['state'] ?? null) === 'running'); + return array_values(array_map(function ($c) { $disk = ($c['disk']['writableLayer'] ?? 0) + ($c['disk']['volumesTotal'] ?? 0); $net = ($c['network']['rxBytesPerSec'] ?? 0) + ($c['network']['txBytesPerSec'] ?? 0); @@ -93,7 +97,7 @@ class SentinelMetricsClient 'diskBytes' => isset($c['disk']) ? (int) $disk : null, 'net' => isset($c['network']) ? (float) $net : null, ]; - }, $rows)); + }, $running)); } /** @@ -141,7 +145,8 @@ class SentinelMetricsClient // Disk history interleaves every mount; keep the root filesystem only. if ($rootOnly && array_any($rows, fn ($row) => array_key_exists('mount', $row))) { - $rows = array_filter($rows, fn ($row) => ($row['mount'] ?? null) === '/'); + $mount = self::rootMountName($rows); + $rows = array_filter($rows, fn ($row) => ($row['mount'] ?? null) === $mount); } // Average raw samples into epoch-aligned buckets so every server shares the same @@ -159,14 +164,30 @@ class SentinelMetricsClient return array_map(fn ($bucket) => [$bucket, round($sums[$bucket] / $counts[$bucket], 2)], array_keys($sums)); } + /** + * The mount that stands for the root disk: `/`, else the disk that holds /data/coolify. + * Sentinel runs in a container and reports only the disks of its own mounts: + * 1.1.2 and older as `/app/db`, 1.1.3 and newer by host mount point, which is + * `/data` or `/data/coolify` when that directory is on its own disk. + * + * @param array> $rows + */ + private static function rootMountName(array $rows): ?string + { + $mounts = array_column($rows, 'mount'); + + return collect(['/', '/app/db', '/data/coolify', '/data'])->first(fn ($mount) => in_array($mount, $mounts, true)); + } + /** * @param array> $mounts * @return array{used: ?int, total: ?int, percent: ?float} */ private function rootMount(array $mounts): array { - $root = collect($mounts)->firstWhere('mount', '/'); - if ($root === null && ! array_any($mounts, fn ($mount) => array_key_exists('mount', $mount))) { + if (array_any($mounts, fn ($mount) => array_key_exists('mount', $mount))) { + $root = collect($mounts)->firstWhere('mount', self::rootMountName($mounts)); + } else { $root = $mounts[0] ?? null; } @@ -209,7 +230,9 @@ class SentinelMetricsClient ["docker exec coolify-sentinel sh -c 'curl -s --max-time 10 -H \"Authorization: Bearer {$token}\" \"{$url}\"'"], $this->server, false, - timeout: 15 + timeout: 15, + // Metrics are read again on the next refresh. SSH retries would keep a worker past its timeout. + maxAttempts: 1 ); } diff --git a/app/Services/ServerCanvas.php b/app/Services/ServerCanvas.php index 5539c481c3..afee3cece6 100644 --- a/app/Services/ServerCanvas.php +++ b/app/Services/ServerCanvas.php @@ -25,7 +25,7 @@ class ServerCanvas /** * @param Collection $servers servers of the team to show - * @return array{title: string, title_link: ?string, can_edit: bool, can_move: false, can_move_groups: bool, layout: 'server', zones: array>, nodes: array>, links: array>, positions: array{}} + * @return array{title: string, title_link: ?string, can_edit: bool, can_move: false, can_move_groups: bool, can_move_zones: bool, layout: 'server', zones: array>, nodes: array>, links: array>, positions: array{}} */ public function payload(int $teamId, Collection $servers, ?Server $single, bool $canReadSensitive, bool $canEdit = false): array { @@ -92,9 +92,15 @@ class ServerCanvas 'can_move' => false, // Admins drag the project / environment group boxes; the canvas saves the moved cards on their environment canvas. 'can_move_groups' => $canEdit, + // Admins move the server zones of the every-server canvases; each server saves its place (zone.position). + 'can_move_zones' => $canEdit && ! $single, // Tells the canvas to group cards by project / environment and place linked cards side by side. 'layout' => 'server', - 'zones' => $servers->map(fn (Server $server) => [...$this->canvas->zone($server, $canReadSensitive), 'details' => $this->details($server)]) + 'zones' => $servers->map(fn (Server $server) => [ + ...$this->canvas->zone($server, $canReadSensitive), + 'details' => $this->details($server), + 'position' => $single ? null : $this->canvas->zonePosition($server->canvas_position), + ]) ->when($empty->contains('zone', EnvironmentCanvas::EMPTY_ZONE), fn (Collection $zones) => $zones->push($this->canvas->emptyZone())) ->values()->all(), 'nodes' => $nodes->concat($empty)->concat($bare)->all(), @@ -145,6 +151,8 @@ class ServerCanvas } }; + $server->prefetchMetrics(['cpu', 'memory', 'disk'], 1); + return [ 'cpu' => $latest(fn () => $server->getCpuMetrics(1)), 'memory' => $latest(fn () => $server->getMemoryMetrics(1)), diff --git a/app/Services/ServerTransfer/ServerTransferBundle.php b/app/Services/ServerTransfer/ServerTransferBundle.php index aa5936d14e..dd75f61b2e 100644 --- a/app/Services/ServerTransfer/ServerTransferBundle.php +++ b/app/Services/ServerTransfer/ServerTransferBundle.php @@ -2,12 +2,60 @@ namespace App\Services\ServerTransfer; +use App\Support\ValidationPatterns; +use Closure; +use Illuminate\Support\Facades\Validator; use Illuminate\Validation\ValidationException; +use Throwable; class ServerTransferBundle { public const SCHEMA_VERSION = 1; + /** + * Every `uuid`, `*_uuid`, and `*_uuids` value in a bundle. Coolify ids are random lowercase + * letters and digits (new_public_id) or RFC 4122 UUIDs. The ids go into file paths and local + * shell commands (SSH key file, SSH mux socket, project icon directory). + */ + public const ID_PATTERN = '/\A[a-zA-Z0-9][a-zA-Z0-9_-]{0,63}\z/'; + + /** + * The most items a bundle can have for each list key, summed over the whole bundle. + * + * @var array + */ + public const MAX_ITEMS = [ + 'projects' => 1000, + 'environments' => 5000, + 'applications' => 5000, + 'databases' => 5000, + 'services' => 2000, + 'previews' => 10000, + 'environment_variables' => 50000, + 'environment_variables_preview' => 50000, + 'shared_environment_variables' => 20000, + 'persistent_storages' => 20000, + 'file_storages' => 20000, + 'scheduled_tasks' => 10000, + 'scheduled_backups' => 10000, + 'volume_backups' => 10000, + 'tags' => 20000, + 'resource_links' => 10000, + 'destinations' => 1000, + 'private_keys' => 1000, + 'github_apps' => 1000, + 'gitlab_apps' => 1000, + 'gitea_apps' => 1000, + 's3_storages' => 1000, + 'cloud_provider_tokens' => 1000, + 'integration_tokens' => 1000, + 'dns_zones' => 1000, + 'managed_dns_records' => 10000, + 'ssl_certificates' => 10000, + 'warnings' => 1000, + 'post_transfer_checklist' => 1000, + ]; + /** * @param array $payload */ @@ -76,6 +124,8 @@ class ServerTransferBundle $errors[] = 'Missing projects array.'; } + $errors = array_merge($errors, self::contentErrors($bundle)); + return [ 'valid' => $errors === [], 'errors' => $errors, @@ -95,4 +145,254 @@ class ServerTransferBundle ]); } } + + /** + * Checks the values of a bundle before the import writes anything. A bundle comes from the + * API, so each value gets the validation that the normal API applies to it. + * + * @param array $bundle + * @return list + */ + public static function contentErrors(array $bundle): array + { + $errors = []; + $counts = []; + self::walk($bundle, '', $errors, $counts); + + foreach ($counts as $key => $count) { + if ($count > self::MAX_ITEMS[$key]) { + $errors[] = "The bundle has {$count} {$key}. A bundle can have at most ".self::MAX_ITEMS[$key]." {$key}."; + } + } + + // Stop before the field checks: a bundle with a bad id or too many items is not imported. + if ($errors !== []) { + return $errors; + } + + foreach (self::items(data_get($bundle, 'destinations')) as $i => $destination) { + self::check($errors, "destinations.{$i}.network", $destination['network'] ?? null, ['nullable', 'string', 'max:255', 'regex:'.ValidationPatterns::DOCKER_NETWORK_PATTERN]); + } + + // Only http and https URLs, as for a git source made with the API. The import does not resolve the host: + // the target may not reach the git host yet, and SafeExternalUrl checks the host on every request. + $gitSourceUrl = function (string $attribute, mixed $value, Closure $fail): void { + if (is_string($value) && ! isValidDomainUrl($value)) { + $fail('The :attribute must use the http or https scheme.'); + } + }; + foreach (['github_apps', 'gitlab_apps', 'gitea_apps'] as $key) { + foreach (self::items(data_get($bundle, $key)) as $i => $source) { + foreach (['html_url', 'api_url'] as $field) { + self::check($errors, "{$key}.{$i}.{$field}", $source[$field] ?? null, ['nullable', 'string', 'url', $gitSourceUrl]); + } + } + } + + foreach (self::items(data_get($bundle, 'ssl_certificates')) as $i => $certificate) { + self::checkPath($errors, "ssl_certificates.{$i}.configuration_dir", $certificate['configuration_dir'] ?? null); + self::checkPath($errors, "ssl_certificates.{$i}.mount_path", $certificate['mount_path'] ?? null); + } + + foreach (self::items(data_get($bundle, 'volume_backups')) as $i => $backup) { + self::checkCron($errors, "volume_backups.{$i}.frequency", $backup['frequency'] ?? null); + } + + foreach (self::items(data_get($bundle, 'projects')) as $p => $project) { + foreach (self::items($project['environments'] ?? null) as $e => $environment) { + $at = "projects.{$p}.environments.{$e}"; + + foreach (self::items($environment['applications'] ?? null) as $a => $application) { + self::checkDomains($errors, "{$at}.applications.{$a}.fqdn", data_get($application, 'attributes.fqdn')); + self::checkStorages($errors, "{$at}.applications.{$a}", $application); + self::checkScheduledTasks($errors, "{$at}.applications.{$a}", $application); + foreach (self::items($application['previews'] ?? null) as $r => $preview) { + self::checkDomains($errors, "{$at}.applications.{$a}.previews.{$r}.fqdn", $preview['fqdn'] ?? null); + self::checkStorages($errors, "{$at}.applications.{$a}.previews.{$r}", $preview); + } + } + + foreach (self::items($environment['databases'] ?? null) as $d => $database) { + self::checkStorages($errors, "{$at}.databases.{$d}", $database); + self::checkScheduledBackups($errors, "{$at}.databases.{$d}", $database); + } + + foreach (self::items($environment['services'] ?? null) as $s => $service) { + self::checkScheduledTasks($errors, "{$at}.services.{$s}", $service); + foreach (self::items($service['applications'] ?? null) as $a => $serviceApplication) { + self::checkDomains($errors, "{$at}.services.{$s}.applications.{$a}.fqdn", data_get($serviceApplication, 'attributes.fqdn', $serviceApplication['fqdn'] ?? null)); + self::checkStorages($errors, "{$at}.services.{$s}.applications.{$a}", $serviceApplication); + } + foreach (self::items($service['databases'] ?? null) as $d => $serviceDatabase) { + self::checkStorages($errors, "{$at}.services.{$s}.databases.{$d}", $serviceDatabase); + self::checkScheduledBackups($errors, "{$at}.services.{$s}.databases.{$d}", $serviceDatabase); + } + } + } + } + + return $errors; + } + + /** + * Checks every id in the bundle and counts the items of every list key in MAX_ITEMS. + * + * @param list $errors + * @param array $counts + */ + private static function walk(array $node, string $path, array &$errors, array &$counts): void + { + foreach ($node as $key => $value) { + $at = $path === '' ? (string) $key : "{$path}.{$key}"; + + if (is_string($key)) { + if ($key === 'uuid' || str_ends_with($key, '_uuid')) { + self::checkId($errors, $at, $value); + } elseif (str_ends_with($key, '_uuids')) { + foreach (is_array($value) ? $value : [$value] as $i => $id) { + self::checkId($errors, "{$at}.{$i}", $id); + } + + continue; + } + + if (isset(self::MAX_ITEMS[$key]) && is_array($value)) { + $counts[$key] = ($counts[$key] ?? 0) + (array_is_list($value) + ? count($value) + : array_sum(array_map(fn ($list) => is_array($list) ? count($list) : 0, $value))); + } + } + + if (is_array($value)) { + self::walk($value, $at, $errors, $counts); + } + } + } + + /** + * @param list $errors + */ + private static function checkId(array &$errors, string $at, mixed $value): void + { + if ($value === null || $value === '') { + return; + } + + if (! is_string($value) || preg_match(self::ID_PATTERN, $value) !== 1) { + $errors[] = "{$at} is not a valid id. An id may only contain letters, digits, hyphens, and underscores (at most 64 characters)."; + } + } + + /** + * @return array> + */ + private static function items(mixed $value): array + { + return is_array($value) ? array_filter($value, 'is_array') : []; + } + + /** + * @param list $errors + * @param array $rules + */ + private static function check(array &$errors, string $at, mixed $value, array $rules): void + { + $validator = Validator::make(['value' => $value], ['value' => $rules], [], ['value' => $at]); + foreach ($validator->errors()->all() as $message) { + $errors[] = $message; + } + } + + /** + * @param list $errors + */ + private static function checkDomains(array &$errors, string $at, mixed $value): void + { + foreach (ValidationPatterns::validateApplicationDomains($value) as $message) { + $errors[] = "{$at}: {$message}"; + } + } + + /** + * @param list $errors + */ + private static function checkPath(array &$errors, string $at, mixed $value): void + { + if ($value === null || $value === '') { + return; + } + if (! is_string($value)) { + $errors[] = "{$at} must be a string."; + + return; + } + + try { + validateShellSafePath($value, $at); + } catch (Throwable $e) { + $errors[] = $e->getMessage(); + } + } + + /** + * @param list $errors + */ + private static function checkCron(array &$errors, string $at, mixed $value): void + { + if ($value === null) { + return; + } + + try { + $valid = is_string($value) && validate_cron_expression($value); + } catch (Throwable) { + $valid = false; + } + if (! $valid) { + $errors[] = "{$at} is not a valid cron expression or frequency."; + } + } + + /** + * @param list $errors + * @param array $resource + */ + private static function checkStorages(array &$errors, string $at, array $resource): void + { + foreach (self::items($resource['persistent_storages'] ?? null) as $i => $volume) { + self::checkPath($errors, "{$at}.persistent_storages.{$i}.mount_path", $volume['mount_path'] ?? null); + self::checkPath($errors, "{$at}.persistent_storages.{$i}.host_path", $volume['host_path'] ?? null); + } + foreach (self::items($resource['file_storages'] ?? null) as $i => $storage) { + self::checkPath($errors, "{$at}.file_storages.{$i}.fs_path", $storage['fs_path'] ?? null); + self::checkPath($errors, "{$at}.file_storages.{$i}.mount_path", $storage['mount_path'] ?? null); + } + } + + /** + * The same checks as the scheduled task API. + * + * @param list $errors + * @param array $resource + */ + private static function checkScheduledTasks(array &$errors, string $at, array $resource): void + { + foreach (self::items($resource['scheduled_tasks'] ?? null) as $i => $task) { + self::check($errors, "{$at}.scheduled_tasks.{$i}.command", $task['command'] ?? null, ['required', 'string']); + self::check($errors, "{$at}.scheduled_tasks.{$i}.container", $task['container'] ?? null, ['nullable', ...ValidationPatterns::containerNameRules()]); + self::checkCron($errors, "{$at}.scheduled_tasks.{$i}.frequency", $task['frequency'] ?? ''); + } + } + + /** + * @param list $errors + * @param array $database + */ + private static function checkScheduledBackups(array &$errors, string $at, array $database): void + { + foreach (self::items($database['scheduled_backups'] ?? null) as $i => $backup) { + self::checkCron($errors, "{$at}.scheduled_backups.{$i}.frequency", $backup['frequency'] ?? null); + self::checkPath($errors, "{$at}.scheduled_backups.{$i}.databases_to_backup", $backup['databases_to_backup'] ?? null); + } + } } diff --git a/app/Services/ServerTransfer/ServerTransferClaimer.php b/app/Services/ServerTransfer/ServerTransferClaimer.php index f479b1fb49..9a3862f7df 100644 --- a/app/Services/ServerTransfer/ServerTransferClaimer.php +++ b/app/Services/ServerTransfer/ServerTransferClaimer.php @@ -4,13 +4,16 @@ namespace App\Services\ServerTransfer; use App\Enums\GithubRunnerStatus; use App\Jobs\DeleteResourceJob; +use App\Models\Application; use App\Models\GithubRunnerExecution; use App\Models\ManagedDnsRecord; use App\Models\Server; +use App\Models\Team; use App\Notifications\Internal\GeneralNotification; use App\Services\Dns\CloudflareDnsProvider; use App\Services\Dns\ManagedDnsRecordCleanup; use Illuminate\Database\Eloquent\Model; +use Illuminate\Support\Carbon; use Illuminate\Support\Collection; use Illuminate\Support\Facades\DB; use Illuminate\Support\Facades\Log; @@ -67,13 +70,16 @@ class ServerTransferClaimer $server = Server::query()->with('settings')->lockForUpdate()->findOrFail($server->id); // A disabled server refuses SSH, so the owner marker could not be written. - if ($server->team->serverOverflow()) { + // An archive does not use a server slot, so taking it back needs a free slot. + $overLimit = $server->isTransferredAway() ? Team::serverLimitReached($server->team) : $server->team->serverOverflow(); + if ($overLimit) { throw new RuntimeException('Your team is over its server limit. Upgrade your subscription or remove a server first.'); } if ($server->settings) { $server->settings->sentinel_custom_url = $instanceUrl; - $server->settings->ensureValidSentinelToken(); + // A new token: the instance that managed the server before keeps a copy of the old one. + $server->settings->generateSentinelToken(save: false); $server->settings->is_sentinel_enabled = true; $server->settings->force_disabled = false; $server->settings->save(); @@ -85,6 +91,7 @@ class ServerTransferClaimer 'claimed_at' => $claim['claimed_at'], 'claim' => $claim, ]); + unset($metadata['transfer']['owner_marker_conflict'], $metadata['transfer']['claimed_by_marker']); $server->server_metadata = $metadata; $server->save(); @@ -107,13 +114,13 @@ class ServerTransferClaimer * * @return array{server_uuid: string, message: string} */ - public function markTransferred(Server $server, ?string $exportId = null, ?string $targetInstanceUrl = null): array + public function markTransferred(Server $server, ?string $exportId = null, ?string $targetInstanceUrl = null, bool $fromOwnerMarker = false): array { if ($server->id === 0) { throw new RuntimeException('Cannot transfer the Coolify host itself.'); } - $result = DB::transaction(function () use ($server, $exportId, $targetInstanceUrl) { + $result = DB::transaction(function () use ($server, $exportId, $targetInstanceUrl, $fromOwnerMarker) { $server = Server::query()->with('settings')->lockForUpdate()->findOrFail($server->id); if ($server->settings) { @@ -127,9 +134,11 @@ class ServerTransferClaimer 'status' => 'transferred', 'export_id' => $exportId ?? data_get($metadata, 'transfer.export_id'), 'target_instance_url' => $targetInstanceUrl, + // Anyone who can write files on the host can write the owner marker, so its URL is not trusted. + 'claimed_by_marker' => $fromOwnerMarker, 'transferred_at' => now()->toIso8601String(), ]); - unset($metadata['transfer']['finish'], $metadata['transfer']['taken_back_at']); + unset($metadata['transfer']['finish'], $metadata['transfer']['taken_back_at'], $metadata['transfer']['owner_marker_conflict']); $server->server_metadata = $metadata; $server->save(); @@ -176,18 +185,57 @@ class ServerTransferClaimer } $this->forgetDnsRecords($server); - foreach ($server->definedResources() as $resource) { + $this->detachAdditionalApplications($server); + foreach ($this->archiveResources($server) as $resource) { (new DeleteResourceJob($resource, deleteFromCoolifyOnly: true))->handle(); } $server->forceDelete(); } + /** + * The resources that the archive of a transferred server deletes. An application that has the server only as an + * additional destination runs on another server of this instance, so it is not one of them. + * + * @return Collection + */ + public function archiveResources(Server $server): Collection + { + return $this->primaryApplications($server) + ->concat($server->databases()) + ->concat($server->services()->get()); + } + + /** + * Remove the server from applications that run on it only as an additional destination. + */ + public function detachAdditionalApplications(Server $server): void + { + $primaryIds = $this->primaryApplications($server)->pluck('id'); + DB::table('additional_destinations') + ->where('server_id', $server->id) + ->whereNotIn('application_id', $primaryIds) + ->delete(); + } + + /** + * @return Collection + */ + private function primaryApplications(Server $server): Collection + { + return $server->applications() + ->filter(fn (Application $application) => (int) data_get($application, 'destination.server_id') === (int) $server->id) + ->unique('id') + ->values(); + } + /** * @return Collection */ private function dnsResources(Server $server): Collection { - return $server->applications() + // The server itself owns the DNS record of its Cloudflare tunnel. + return collect([$server]) + ->concat($this->primaryApplications($server)) ->concat($server->services()->with('applications')->get()->flatMap->applications); } @@ -234,10 +282,16 @@ class ServerTransferClaimer } /** - * Stop managing a claimed server when its owner marker names a claim of another instance. + * Stop managing a claimed server when its owner marker names a newer claim of another instance. * A server that was added here without a claim ignores the marker: a marker can be left over on a * host that an old instance deleted, and the next claim overwrites it. * + * Anyone who can write files on the host can write the marker, so it is not proof of a claim. A marker that is + * not newer than the claim of this instance, or that names a URL that this instance would not send credentials + * to, does not release the server: this instance keeps managing it and saves the conflict, which the UI shows. + * A forged marker with a newer time and a valid URL still releases the server; it can be taken back with + * Enable management. + * * @return bool true when this instance released the server */ public function releaseIfClaimedElsewhere(Server $server, ?string $markerJson): bool @@ -253,13 +307,25 @@ class ServerTransferClaimer } if ($markerClaimId === data_get($server->server_metadata, 'transfer.claim.claim_id')) { + if (data_get($server->server_metadata, 'transfer.owner_marker_conflict') !== null) { + $this->recordOwnerMarkerConflict($server, null, null); + } + return false; } + // Any http(s) URL: a default self-hosted target uses http://IP:8000. The URL is not trusted + // (claimed_by_marker), so it is never prefilled for a transfer back. $owner = data_get($marker, 'instance_url'); - $owner = is_string($owner) && $owner !== '' ? $owner : null; + $owner = is_string($owner) && filter_var($owner, FILTER_VALIDATE_URL) && in_array(strtolower((string) parse_url($owner, PHP_URL_SCHEME)), ['http', 'https'], true) + ? rtrim(trim($owner), '/') : null; + if ($owner === null || ! $this->isNewerThanOwnClaim($server, data_get($marker, 'claimed_at'))) { + $this->recordOwnerMarkerConflict($server, $markerClaimId, data_get($marker, 'instance_url')); - $this->markTransferred($server, targetInstanceUrl: $owner); + return false; + } + + $this->markTransferred($server, targetInstanceUrl: $owner, fromOwnerMarker: true); Log::warning('Server is claimed by another Coolify instance; management stopped here', [ 'server_uuid' => $server->uuid, @@ -273,6 +339,50 @@ class ServerTransferClaimer return true; } + private function isNewerThanOwnClaim(Server $server, mixed $markerClaimedAt): bool + { + $ownClaimedAt = data_get($server->server_metadata, 'transfer.claim.claimed_at'); + if (! is_string($markerClaimedAt) || ! is_string($ownClaimedAt)) { + return false; + } + + try { + return Carbon::parse($markerClaimedAt)->isAfter(Carbon::parse($ownClaimedAt)); + } catch (Throwable) { + return false; + } + } + + /** + * Save the untrusted marker of another claim, or remove the saved one when $markerClaimId is null. + */ + private function recordOwnerMarkerConflict(Server $server, ?string $markerClaimId, mixed $instanceUrl): void + { + if ($markerClaimId !== null && data_get($server->server_metadata, 'transfer.owner_marker_conflict.claim_id') === $markerClaimId) { + return; + } + + if ($markerClaimId !== null) { + Log::warning('The owner marker on the server names another claim, but it is not trusted; this instance keeps managing the server', [ + 'server_uuid' => $server->uuid, + 'marker_claim_id' => $markerClaimId, + ]); + } + + DB::transaction(function () use ($server, $markerClaimId, $instanceUrl) { + $locked = Server::query()->lockForUpdate()->findOrFail($server->id); + $metadata = $locked->server_metadata ?? []; + $metadata['transfer']['owner_marker_conflict'] = $markerClaimId === null ? null : [ + 'claim_id' => $markerClaimId, + 'instance_url' => is_string($instanceUrl) ? mb_substr($instanceUrl, 0, 255) : null, + 'seen_at' => now()->toIso8601String(), + ]; + $locked->server_metadata = $metadata; + $locked->save(); + $server->server_metadata = $metadata; + }); + } + /** * @param array $claim */ diff --git a/app/Services/ServerTransfer/ServerTransferCoverage.php b/app/Services/ServerTransfer/ServerTransferCoverage.php index 1c615e304d..47efbb767f 100644 --- a/app/Services/ServerTransfer/ServerTransferCoverage.php +++ b/app/Services/ServerTransfer/ServerTransferCoverage.php @@ -185,6 +185,7 @@ final class ServerTransferCoverage 'hetzner_server_status' => self::RUNTIME, 'vultr_instance_status' => self::RUNTIME, 'digitalocean_droplet_status' => self::RUNTIME, 'hostinger_virtual_machine_status' => self::RUNTIME, 'server_metadata' => 'Facts about the host and the transfer state of the source; the target collects its own.', + 'canvas_position' => 'Place of the server on the servers canvas of the source; the target places it with its own servers.', ], ServerSetting::class => [ 'server_id' => self::LINKED, 'is_reachable' => self::RUNTIME, 'is_usable' => self::RUNTIME, 'force_disabled' => self::RUNTIME, @@ -221,6 +222,7 @@ final class ServerTransferCoverage 'icon_path' => 'The icon file is exported as `icon` (base64 content). The target stores it in its own icon storage and sets the path.', 'icon_storage_type' => 'The target uses its own default icon storage. See icon_path.', 'icon_s3_storage_id' => 'The target uses its own default icon storage. See icon_path.', + 'canvas_position' => 'Place of the project on the projects canvas of the source; the target places it with its own projects.', ], Environment::class => ['project_id' => self::LINKED], ResourceLink::class => [ diff --git a/app/Services/ServerTransfer/ServerTransferExporter.php b/app/Services/ServerTransfer/ServerTransferExporter.php index 55e472020c..0edba09cb5 100644 --- a/app/Services/ServerTransfer/ServerTransferExporter.php +++ b/app/Services/ServerTransfer/ServerTransferExporter.php @@ -449,7 +449,8 @@ class ServerTransferExporter private function collectManagedDnsRecords(Server $server, Collection $applications, Collection $services): array { // Keyed by "morph type:id". Soft-deleted previews are covered, but their references are not exported. - $resourceKeys = []; + // The server itself owns the DNS record of its Cloudflare tunnel. + $resourceKeys = [$server->getMorphClass().':'.$server->id => ['kind' => 'server', 'uuid' => $server->uuid]]; foreach ($applications as $application) { $resourceKeys[$application->getMorphClass().':'.$application->id] = ['kind' => 'application', 'uuid' => $application->uuid]; foreach (ApplicationPreview::withTrashed()->where('application_id', $application->id)->get() as $preview) { @@ -461,10 +462,6 @@ class ServerTransferExporter $resourceKeys[$serviceApplication->getMorphClass().':'.$serviceApplication->id] = ['kind' => 'service_application', 'uuid' => $serviceApplication->uuid]; } } - if ($resourceKeys === []) { - return ['records' => collect(), 'resource_keys' => [], 'warnings' => []]; - } - $records = ManagedDnsRecord::query() ->whereHas('references', function ($query) use ($resourceKeys) { $query->where(function ($query) use ($resourceKeys) { @@ -567,6 +564,8 @@ class ServerTransferExporter $steps[] = 'After the GitHub App sends webhooks to {target_url}, open this server there, check the runner requirements, and enable GitHub runners. ' .'Enabling them stops runners that another Coolify instance started on this server.'; } + $steps[] = 'This instance keeps a copy of the SSH key of the server. On {target_url}, add a new SSH key to the server and select it there, ' + .'then remove the old public key from ~/.ssh/authorized_keys on the server.'; return $steps; } diff --git a/app/Services/ServerTransfer/ServerTransferImporter.php b/app/Services/ServerTransfer/ServerTransferImporter.php index 11bd69663d..9cdb066eee 100644 --- a/app/Services/ServerTransfer/ServerTransferImporter.php +++ b/app/Services/ServerTransfer/ServerTransferImporter.php @@ -181,7 +181,11 @@ class ServerTransferImporter bool $preserveUuids = true, bool $adoptMode = true, ): array { - ServerTransferBundle::assertValid($bundle); + // Checks every id, list size, and imported value before anything is written. + $validation = ServerTransferBundle::validate($bundle); + if (! $validation['valid']) { + throw ValidationException::withMessages(['bundle' => $validation['errors']]); + } $this->assertValidServerPayload((array) data_get($bundle, 'server', [])); if ($adoptMode && ! $preserveUuids) { @@ -190,7 +194,6 @@ class ServerTransferImporter ]); } - $validation = ServerTransferBundle::validate($bundle); $warnings = $validation['warnings']; if (is_array(data_get($bundle, 'warnings'))) { $warnings = array_values(array_unique(array_merge($warnings, $bundle['warnings']))); @@ -206,6 +209,8 @@ class ServerTransferImporter if (! $dryRun) { $this->assertCanImport($serverUuid, $serverIp, $teamId, $preserveUuids); + // The import validates the server over SSH, so it uses the same Cloud limit as a new server. + Server::throttleValidation($teamId, $serverIp); } if ($uuidConflict) { @@ -416,7 +421,7 @@ class ServerTransferImporter $warnings = array_merge($warnings, $this->linkSqliteVolumes()); $this->importDnsZones(data_get($bundle, 'dns_zones', []), $preserveUuids); - $created['managed_dns_records'] = $this->importManagedDnsRecords(data_get($bundle, 'managed_dns_records', []), $teamId, $preserveUuids); + $created['managed_dns_records'] = $this->importManagedDnsRecords(data_get($bundle, 'managed_dns_records', []), $server, $teamId, $preserveUuids); $created['ssl_certificates'] = $this->importSslCertificates(data_get($bundle, 'ssl_certificates', []), $server); $created['volume_backups'] = $this->importVolumeBackups(data_get($bundle, 'volume_backups', []), $teamId); @@ -502,14 +507,14 @@ class ServerTransferImporter { $team = Team::find($teamId); $archive = $preserveUuids ? $this->replaceableArchive($serverUuid, $teamId) : null; - // The archive is replaced, so the import does not add a server. - $limitReached = ! $team || ($archive === null && Team::serverLimitReached($team)); + // An archive does not use a server slot, so replacing it also needs a free slot. + $limitReached = ! $team || Team::serverLimitReached($team); $existingServer = $preserveUuids && $archive === null ? Server::withTrashed()->where('uuid', $serverUuid)->first() : null; $uuidMessage = match (true) { $existingServer === null => null, $existingServer->trashed() => "The server with UUID {$serverUuid} is still being deleted on this instance. Wait a minute, then transfer again.", - $existingServer->isTransferredAway() => "Another team of this instance has the archived copy of server {$existingServer->name}. Transfer the server into that team, or delete the archive there first.", + $existingServer->isTransferredAway() => 'Another team of this instance has the archived copy of this server. Transfer the server into that team, or delete the archive there first.', default => "A server with UUID {$serverUuid} already exists on this instance. Delete/transfer it first, or import with preserve_uuids=false.", }; @@ -590,7 +595,7 @@ class ServerTransferImporter if ($adoptMode) { throw ValidationException::withMessages([ - 'bundle' => ["The {$label} UUID {$uuid} is already used on this instance. Delete that {$label} permanently first, because adopt mode needs the same UUID to keep the running containers."], + 'bundle' => ["The {$label} UUID {$uuid} cannot be used on this instance. Adopt mode needs the same UUID to keep the running containers. If you deleted the {$label} on this instance, delete it permanently first."], ]); } @@ -811,6 +816,9 @@ class ServerTransferImporter $map[$originalUuid !== '' ? $originalUuid : $model->uuid] = $model->fresh(); } + // The created hook of the server made a default destination; the server gets only the destinations of the bundle. + StandaloneDocker::query()->whereKey($existingStandalone->slice($standaloneIndex)->modelKeys())->delete(); + SwarmDocker::query()->whereKey($existingSwarm->slice($swarmIndex)->modelKeys())->delete(); return $map; } @@ -821,17 +829,7 @@ class ServerTransferImporter private function importServerSharedEnvVars(array $variables, Server $server, int $teamId): void { foreach ($variables as $variable) { - SharedEnvironmentVariable::create([ - 'key' => data_get($variable, 'key'), - 'value' => data_get($variable, 'value'), - 'is_multiline' => (bool) data_get($variable, 'is_multiline', false), - 'is_literal' => (bool) data_get($variable, 'is_literal', false), - 'is_shown_once' => (bool) data_get($variable, 'is_shown_once', false), - 'comment' => data_get($variable, 'comment'), - 'type' => 'server', - 'server_id' => $server->id, - 'team_id' => $teamId, - ]); + $this->createSharedVariable($variable, ['type' => 'server', 'server_id' => $server->id, 'team_id' => $teamId]); } } @@ -854,19 +852,37 @@ class ServerTransferImporter continue; } - SharedEnvironmentVariable::create([ - 'key' => $key, - 'value' => data_get($variable, 'value'), - 'is_multiline' => (bool) data_get($variable, 'is_multiline', false), - 'is_literal' => (bool) data_get($variable, 'is_literal', false), - 'is_shown_once' => (bool) data_get($variable, 'is_shown_once', false), - 'comment' => data_get($variable, 'comment'), - 'type' => 'team', - 'team_id' => $teamId, - ]); + $this->createSharedVariable($variable, ['type' => 'team', 'team_id' => $teamId]); } } + /** + * @param array $variable A shared variable of the bundle. + * @param array $scope The type, the team, and the server, project, or environment of the variable. + */ + private function createSharedVariable(array $variable, array $scope): void + { + $shared = new SharedEnvironmentVariable([ + 'value' => data_get($variable, 'value'), + 'is_multiline' => (bool) data_get($variable, 'is_multiline', false), + 'is_literal' => (bool) data_get($variable, 'is_literal', false), + 'is_shown_once' => (bool) data_get($variable, 'is_shown_once', false), + 'comment' => data_get($variable, 'comment'), + ...$scope, + ]); + $this->keepVariableKey($shared, (string) data_get($variable, 'key')); + $shared->save(); + } + + /** + * Sets the key of a variable as it is on the source. The key setter rejects names that only older Coolify + * versions accepted (such as my-var); deployments skip those with a warning, so the transfer keeps them. + */ + private function keepVariableKey(EnvironmentVariable|SharedEnvironmentVariable $variable, string $key): void + { + $variable->setRawAttributes(['key' => $key] + $variable->getAttributes()); + } + /** * @param array $payload * @param array $created @@ -1037,17 +1053,7 @@ class ServerTransferImporter if ($exists) { continue; } - SharedEnvironmentVariable::create([ - 'key' => data_get($variable, 'key'), - 'value' => data_get($variable, 'value'), - 'is_multiline' => (bool) data_get($variable, 'is_multiline', false), - 'is_literal' => (bool) data_get($variable, 'is_literal', false), - 'is_shown_once' => (bool) data_get($variable, 'is_shown_once', false), - 'comment' => data_get($variable, 'comment'), - 'type' => 'project', - 'project_id' => $project->id, - 'team_id' => $teamId, - ]); + $this->createSharedVariable($variable, ['type' => 'project', 'project_id' => $project->id, 'team_id' => $teamId]); } } @@ -1115,17 +1121,7 @@ class ServerTransferImporter if ($exists) { continue; } - SharedEnvironmentVariable::create([ - 'key' => data_get($variable, 'key'), - 'value' => data_get($variable, 'value'), - 'is_multiline' => (bool) data_get($variable, 'is_multiline', false), - 'is_literal' => (bool) data_get($variable, 'is_literal', false), - 'is_shown_once' => (bool) data_get($variable, 'is_shown_once', false), - 'comment' => data_get($variable, 'comment'), - 'type' => 'environment', - 'environment_id' => $environment->id, - 'team_id' => $teamId, - ]); + $this->createSharedVariable($variable, ['type' => 'environment', 'environment_id' => $environment->id, 'team_id' => $teamId]); } } @@ -1304,11 +1300,18 @@ class ServerTransferImporter // Maintenance mode is not fillable. $application->forceFill(collect($attributes)->only(['is_maintenance_enabled', 'maintenance_page'])->all()); $application->save(); + // The created hook gives a new application the newest compose parser and default variables. A transferred + // application keeps its parser, which names its volumes and domains, and gets only the variables of the bundle. + $application->forceFill(collect($attributes)->only('compose_parsing_version')->filter()->all())->saveQuietly(); + EnvironmentVariable::query()->whereMorphedTo('resourceable', $application)->delete(); $settings = (array) data_get($payload, 'settings', []); if ($settings !== [] && $application->settings) { + $portsExposes = $application->ports_exposes; $application->settings->fill(collect($settings)->except(['id', 'application_id'])->all()); $application->settings->save(); + // The is_static setter sets the port of a static application to 80; the user can change it after that. + $application->refresh()->forceFill(['ports_exposes' => $portsExposes])->saveQuietly(); } $this->importEnvVars(data_get($payload, 'environment_variables', []), $application, false); @@ -1415,6 +1418,8 @@ class ServerTransferImporter // Maintenance mode and the Coolify default variables switch are not fillable. $service->forceFill(collect($attributes)->only(['is_maintenance_enabled', 'maintenance_page', 'include_coolify_default_variables'])->all()); $service->save(); + // The created hook gives a new service the newest compose parser; a transferred service keeps its own. + $service->forceFill(collect($attributes)->only('compose_parsing_version')->filter()->all())->saveQuietly(); $this->importEnvVars(data_get($payload, 'environment_variables', []), $service, false); $this->importScheduledTasks(data_get($payload, 'scheduled_tasks', []), $service, $teamId); @@ -1524,7 +1529,6 @@ class ServerTransferImporter $env = new EnvironmentVariable; $env->forceFill([ - 'key' => data_get($variable, 'key'), 'value' => data_get($variable, 'value'), 'is_literal' => (bool) data_get($variable, 'is_literal', false), 'is_multiline' => (bool) data_get($variable, 'is_multiline', false), @@ -1542,6 +1546,7 @@ class ServerTransferImporter 'resourceable_type' => $resource->getMorphClass(), 'resourceable_id' => $resource->id, ]); + $this->keepVariableKey($env, (string) data_get($variable, 'key')); // Older bundles have no version; the column default applies. if (filled(data_get($variable, 'version'))) { $env->version = (string) data_get($variable, 'version'); @@ -2166,7 +2171,7 @@ class ServerTransferImporter * * @param list> $records */ - private function importManagedDnsRecords(array $records, int $teamId, bool $preserveUuids): int + private function importManagedDnsRecords(array $records, Server $server, int $teamId, bool $preserveUuids): int { $count = 0; foreach ($records as $payload) { @@ -2177,6 +2182,7 @@ class ServerTransferImporter 'application' => $this->applicationMap[(string) data_get($reference, 'uuid')] ?? null, 'preview' => $this->previewMap[(string) data_get($reference, 'uuid')] ?? null, 'service_application' => $this->serviceApplicationMap[(string) data_get($reference, 'uuid')] ?? null, + 'server' => $server, default => null, }) ->filter() @@ -2270,7 +2276,7 @@ class ServerTransferImporter $preview = ApplicationPreview::withTrashed()->where('uuid', $uuid)->first(); if ($preview && ! $this->previewBelongsToTeam($preview, $teamId)) { throw ValidationException::withMessages([ - 'previews' => ['An imported preview UUID conflicts with a preview outside the target team.'], + 'previews' => ['An imported preview UUID cannot be used on this instance.'], ]); } if (! $preview) { diff --git a/app/Services/ServerTransfer/ServerTransferMigrator.php b/app/Services/ServerTransfer/ServerTransferMigrator.php index 1c8922c8d5..0f81e423f2 100644 --- a/app/Services/ServerTransfer/ServerTransferMigrator.php +++ b/app/Services/ServerTransfer/ServerTransferMigrator.php @@ -10,6 +10,9 @@ use App\Models\Environment; use App\Models\GithubApp; use App\Models\GitlabApp; use App\Models\Project; +use App\Models\ScheduledDatabaseBackupExecution; +use App\Models\ScheduledTaskExecution; +use App\Models\ScheduledVolumeBackupExecution; use App\Models\Server; use App\Models\Service; use App\Models\User; @@ -25,6 +28,7 @@ use Illuminate\Support\Facades\Http; use Illuminate\Support\Facades\Validator; use Illuminate\Support\Sleep; use Illuminate\Support\Str; +use Illuminate\Validation\ValidationException; use RuntimeException; use Throwable; @@ -48,6 +52,9 @@ class ServerTransferMigrator */ public const MIGRATION_TIMEOUT_SECONDS = 1500; + /** Lifetime of the token that a transfer back gives the other instance: the job timeout plus time in the queue. */ + public const TRANSFER_BACK_TOKEN_SECONDS = 3600; + /** Longer than the job timeout, so a running state older than this one is stale. */ public const STALE_MIGRATION_SECONDS = 1800; @@ -304,6 +311,32 @@ class ServerTransferMigrator 'This server was already transferred to another instance.'), $this->check('no deployment is queued or running on the server', $activeDeployments === 0, "{$activeDeployments} deployment(s) on this server are queued or running. Wait until they finish or cancel them, then transfer the server."), + ...array_map(fn (array $job) => $this->check("no {$job['label']} is running on the server", $job['count'] === 0, + "{$job['count']} {$job['label']}(s) on this server are running. Wait until they finish, then transfer the server."), $this->runningJobs($server)), + ]; + } + + /** + * Backups and scheduled tasks that run on the server. A transfer during a run leaves the run without an owner. + * A run that has been 'running' for a day is left over from a stopped worker, so it does not block. + * + * @return list + */ + private function runningJobs(Server $server): array + { + $onServer = fn ($schedule) => $schedule && (int) $schedule->server()?->id === (int) $server->id; + $running = fn (string $model, string $schedule) => $model::query() + ->where('status', 'running') + ->where('created_at', '>', now()->subDay()) + ->with($schedule) + ->get() + ->filter(fn (Model $execution) => $onServer($execution->{$schedule})) + ->count(); + + return [ + ['label' => 'database backup', 'count' => $running(ScheduledDatabaseBackupExecution::class, 'scheduledDatabaseBackup')], + ['label' => 'volume backup', 'count' => $running(ScheduledVolumeBackupExecution::class, 'scheduledVolumeBackup')], + ['label' => 'scheduled task', 'count' => $running(ScheduledTaskExecution::class, 'scheduledTask')], ]; } @@ -484,7 +517,8 @@ class ServerTransferMigrator 'complete' => $complete, 'warnings' => $warnings, 'checklist' => $checklist, - 'target_server_url' => is_string(data_get($importBody, 'server_url')) ? data_get($importBody, 'server_url') : null, + // The target sends this URL, and the UI renders it as a link. + 'target_server_url' => self::isLinkableUrl(data_get($importBody, 'server_url')) ? data_get($importBody, 'server_url') : null, 'message' => 'Server migrated to '.$targetUrl.'. Automations disabled on this instance.', ]; } @@ -523,7 +557,9 @@ class ServerTransferMigrator throw new RuntimeException('Enter the URL at which the other instance reaches this instance.'); } - $importToken = $user->createToken("transfer back {$archive->uuid}", ['read', 'write'], now()->addHours(3), $archive->team_id); + // The import checks, the import, and the status probes need only write and deploy. The claim after the + // import deletes the token, and the expiry covers a failed import: the transfer job runs at most 25 minutes. + $importToken = $user->createToken(self::transferBackTokenName($archive->uuid), ['write', 'deploy'], now()->addSeconds(self::TRANSFER_BACK_TOKEN_SECONDS), $archive->team_id); $migrateUrl = $managerUrl.'/api/v1/servers/'.rawurlencode($archive->uuid).'/migrate'; try { @@ -554,37 +590,80 @@ class ServerTransferMigrator }); } - private function normalizeTargetUrl(string $targetUrl): string + /** + * The rules for the URL of another Coolify instance that this instance sends credentials to: https and a public + * address, or a local address in development. Every stored or prefilled instance URL must pass them. + * + * @throws RuntimeException|ValidationException + */ + public static function validateInstanceUrl(string $url): string { - $targetUrl = rtrim(trim($targetUrl), '/'); - if ($targetUrl === '' || ! filter_var($targetUrl, FILTER_VALIDATE_URL)) { + $url = rtrim(trim($url), '/'); + if ($url === '' || ! filter_var($url, FILTER_VALIDATE_URL)) { throw new RuntimeException('A valid target instance URL is required (e.g. https://coolify.example.com).'); } - if ($this->isLocalDevelopmentTarget($targetUrl)) { - if (file_exists('/.dockerenv') || is_file('/run/.containerenv')) { - return (string) preg_replace( - '#^(https?://)(localhost|127\.0\.0\.1)(?=[:/]|$)#i', - '$1host.docker.internal', - $targetUrl, - ); - } - - return $targetUrl; + if (self::isLocalDevelopmentTarget($url)) { + return $url; } - if (strtolower((string) parse_url($targetUrl, PHP_URL_SCHEME)) !== 'https') { + if (strtolower((string) parse_url($url, PHP_URL_SCHEME)) !== 'https') { throw new RuntimeException('The target instance URL must use https, because the transfer sends credentials.'); } - Validator::make(['target_url' => $targetUrl], [ + Validator::make(['target_url' => $url], [ 'target_url' => ['required', new SafeWebhookUrl], ])->validate(); + return $url; + } + + public static function isValidInstanceUrl(?string $url): bool + { + try { + self::validateInstanceUrl((string) $url); + + return true; + } catch (RuntimeException|ValidationException) { + return false; + } + } + + /** + * A URL from another instance that the UI may render as a link: https, or http in development. + */ + public static function isLinkableUrl(mixed $url): bool + { + if (! is_string($url) || ! filter_var($url, FILTER_VALIDATE_URL)) { + return false; + } + + $scheme = strtolower((string) parse_url($url, PHP_URL_SCHEME)); + + return $scheme === 'https' || ($scheme === 'http' && isDev()); + } + + public static function transferBackTokenName(string $serverUuid): string + { + return "transfer back {$serverUuid}"; + } + + private function normalizeTargetUrl(string $targetUrl): string + { + $targetUrl = self::validateInstanceUrl($targetUrl); + + if (self::isLocalDevelopmentTarget($targetUrl) && (file_exists('/.dockerenv') || is_file('/run/.containerenv'))) { + return (string) preg_replace( + '#^(https?://)(localhost|127\.0\.0\.1)(?=[:/]|$)#i', + '$1host.docker.internal', + $targetUrl, + ); + } + return $targetUrl; } - private function isLocalDevelopmentTarget(string $url): bool + private static function isLocalDevelopmentTarget(string $url): bool { return isDev() && in_array(strtolower((string) parse_url($url, PHP_URL_SCHEME)), ['http', 'https'], true) @@ -800,7 +879,7 @@ class ServerTransferMigrator private function targetRequest(string $url, string $token, int $timeout): PendingRequest { return Http::timeout($timeout) - ->withOptions($this->isLocalDevelopmentTarget($url) + ->withOptions(self::isLocalDevelopmentTarget($url) ? ['allow_redirects' => false] : SafeWebhookUrl::httpClientOptions($url)) ->acceptJson() diff --git a/app/Services/ServerTransfer/TransferFinishChecklist.php b/app/Services/ServerTransfer/TransferFinishChecklist.php index ee8d93dd79..d894b49cb3 100644 --- a/app/Services/ServerTransfer/TransferFinishChecklist.php +++ b/app/Services/ServerTransfer/TransferFinishChecklist.php @@ -48,14 +48,14 @@ class TransferFinishChecklist ]; private const GROUP_OF_TYPE = [ - 'server_connection' => 'server', 'sentinel' => 'server', 's3_storage' => 'server', + 'server_connection' => 'server', 'sentinel' => 'server', 'previous_ssh_key' => 'server', 's3_storage' => 'server', 'github_app_webhook' => 'git', 'github_app_urls' => 'git', 'gitlab_webhook' => 'git', 'manual_webhook' => 'git', 'git_source' => 'git', 'github_runners' => 'git', 'dns_record' => 'dns', 'domain' => 'dns', ]; /** Types that the user can only mark as done. */ - private const MANUAL_TYPES = ['github_app_urls']; + private const MANUAL_TYPES = ['github_app_urls', 'previous_ssh_key']; /** Types that Coolify can fix with one click. */ private const FIX_LABELS = [ @@ -145,11 +145,16 @@ class TransferFinishChecklist { if (in_array($item['type'], ['github_app_webhook', 'github_app_urls'], true)) { $githubApp = GithubApp::where('uuid', $item['subject'])->first(); + $url = $githubApp?->settingsUrl(); - return $githubApp ? [$githubApp->settingsUrl(), 'Open on GitHub'] : null; + // The html_url of a GitHub App comes from the bundle of another instance, so only web links are shown. + return $url && in_array(strtolower((string) parse_url($url, PHP_URL_SCHEME)), ['http', 'https'], true) ? [$url, 'Open on GitHub'] : null; } - return filled($item['link'] ?? null) ? [$item['link'], $item['link_label'] ?? 'Open'] : null; + $link = $item['link'] ?? null; + + // Saved links are paths of this instance. + return is_string($link) && preg_match('#^/(?![/\\\\])#', $link) ? [$link, $item['link_label'] ?? 'Open'] : null; } public function fixLabel(array $item): ?string @@ -228,6 +233,10 @@ class TransferFinishChecklist $items = [ $this->item('server_connection', 'server', 'Connection to the server'), $this->item('sentinel', 'server', 'Sentinel sends its data to this instance'), + // The SSH key moved with the server, so the previous instance still has a copy of it. + $this->item('previous_ssh_key', 'server', 'Remove the SSH key of the previous instance from the server', + 'The previous instance keeps a copy of the SSH key. Add a new key of this instance to the server and select it here, then remove the old public key from ~/.ssh/authorized_keys on the server.', + [], route('server.private-key', ['server_uuid' => $server->uuid], false), 'Open private key'), ]; $githubApps = $applications->map(fn (Application $application) => $application->source) diff --git a/app/Support/RootSetupWindow.php b/app/Support/RootSetupWindow.php new file mode 100644 index 0000000000..f07180176f --- /dev/null +++ b/app/Support/RootSetupWindow.php @@ -0,0 +1,31 @@ +exists()) { + return; + } + + Cache::put(self::CACHE_KEY, true, now()->addMinutes(self::MINUTES)); + } + + public static function isOpen(): bool + { + return Cache::has(self::CACHE_KEY); + } +} diff --git a/app/Support/RuntimeLogTargets.php b/app/Support/RuntimeLogTargets.php new file mode 100644 index 0000000000..45bdc42329 --- /dev/null +++ b/app/Support/RuntimeLogTargets.php @@ -0,0 +1,446 @@ +>}> $servers + * @return array{ + * deployments: list, + * services: list, + * deployment: string, + * service: string, + * sources: list, + * streamKey: string, + * } + */ + public static function select(array $servers, string $deployment, string $service, bool $groupDeployments): array + { + $containers = self::containers($servers, $groupDeployments); + if ($containers === []) { + return self::emptySelection(); + } + + $deploymentIds = $groupDeployments ? self::sortedDeploymentIds($containers) : []; + $chosenDeployment = $groupDeployments + ? self::pickDeployment($deploymentIds, self::normalizeDeployment($deployment)) + : '0'; + + $visible = array_values(array_filter( + $containers, + fn (array $container): bool => $container['deployment'] === $chosenDeployment, + )); + if ($visible === []) { + return self::emptySelection(); + } + + $groups = self::groupByService($visible); + $chosenKey = self::pickServiceKey($groups, $service); + $sources = self::sourcesFor($groups[$chosenKey]); + + return [ + 'deployments' => $groupDeployments ? self::deploymentOptions($deploymentIds, $containers) : [], + 'services' => self::serviceOptions($groups), + 'deployment' => $chosenDeployment, + 'service' => count($groups) > 1 ? $chosenKey : '', + 'sources' => $sources, + 'streamKey' => self::streamKey($sources), + ]; + } + + /** + * @return array{ + * deployments: list, + * services: list, + * deployment: string, + * service: string, + * sources: list, + * streamKey: string, + * } + */ + private static function emptySelection(): array + { + return [ + 'deployments' => [], + 'services' => [], + 'deployment' => '0', + 'service' => '', + 'sources' => [], + 'streamKey' => '', + ]; + } + + /** + * @param list>}> $servers + * @return list + */ + private static function containers(array $servers, bool $groupDeployments): array + { + $containers = []; + foreach (array_values($servers) as $index => $server) { + foreach ($server['containers'] ?? [] as $container) { + if (! is_array($container)) { + continue; + } + $name = (string) data_get($container, 'Names', ''); + if (! ValidationPatterns::isValidContainerName($name)) { + continue; + } + $labels = self::labels($container); + $state = strtolower(trim((string) data_get($container, 'State', 'running'))); + $containers[] = [ + 'name' => $name, + 'key' => self::serviceKey($name, $labels), + 'state' => $state !== '' ? $state : 'running', + 'deployment' => self::deploymentId($labels, $groupDeployments), + 'server' => (string) ($server['uuid'] ?? ''), + 'serverName' => (string) ($server['name'] ?? ''), + 'serverIndex' => $index, + ]; + } + } + + return $containers; + } + + /** + * @param array $container + */ + private static function labels(array $container): Collection + { + $raw = $container['Labels'] ?? null; + if (is_array($raw)) { + return collect($raw); + } + if (! is_string($raw) || trim($raw) === '') { + return collect(); + } + + $parsed = format_docker_labels_to_json($raw); + + return $parsed instanceof Collection ? $parsed : collect(); + } + + private static function serviceKey(string $containerName, Collection $labels): string + { + $compose = trim((string) $labels->get('com.docker.compose.service', '')); + if ($compose !== '') { + return $compose; + } + + $name = trim((string) $labels->get('coolify.name', '')); + if ($name !== '') { + return $name; + } + + $headline = str($containerName)->beforeLast('-')->headline()->toString(); + + return $headline !== '' ? $headline : $containerName; + } + + private static function deploymentId(Collection $labels, bool $groupDeployments): string + { + if (! $groupDeployments) { + return '0'; + } + + return self::normalizeDeployment((string) $labels->get('coolify.pullRequestId', '')); + } + + private static function normalizeDeployment(string $deployment): string + { + if ($deployment === '' || ! ctype_digit($deployment) || (int) $deployment === 0) { + return '0'; + } + + return (string) (int) $deployment; + } + + /** + * @param list $containers + * @return list + */ + private static function sortedDeploymentIds(array $containers): array + { + $ids = array_values(array_unique(array_column($containers, 'deployment'))); + usort($ids, function (string $left, string $right): int { + if ($left === '0') { + return -1; + } + if ($right === '0') { + return 1; + } + + return (int) $right <=> (int) $left; + }); + + return $ids; + } + + /** + * @param list $deploymentIds + */ + private static function pickDeployment(array $deploymentIds, string $preferred): string + { + if (in_array($preferred, $deploymentIds, true)) { + return $preferred; + } + if (in_array('0', $deploymentIds, true)) { + return '0'; + } + + return $deploymentIds[0] ?? '0'; + } + + /** + * Deployments show the same status dot and caption as services, for all their containers. + * + * @param list $ids + * @param list $containers + * @return list + */ + private static function deploymentOptions(array $ids, array $containers): array + { + return array_map(function (string $id) use ($containers): array { + $states = array_column(array_filter( + $containers, + fn (array $container): bool => $container['deployment'] === $id, + ), 'state'); + + return self::statusOption($id, $id === '0' ? 'Production' : 'PR #'.$id, $states); + }, $ids); + } + + /** + * @param list $states + * @return array{value: string, label: string, dot: string, state?: string} + */ + private static function statusOption(string $value, string $label, array $states): array + { + $option = ['value' => $value, 'label' => $label, 'dot' => self::serviceDot($states)]; + $state = self::serviceStateLabel($states); + if ($state !== null) { + $option['state'] = $state; + } + + return $option; + } + + /** + * @param list $containers + * @return array> + */ + private static function groupByService(array $containers): array + { + $groups = []; + foreach ($containers as $container) { + $groups[$container['key']][] = $container; + } + uksort($groups, fn (string $left, string $right): int => strcasecmp($left, $right)); + + return $groups; + } + + /** + * @param array> $groups + */ + private static function pickServiceKey(array $groups, string $service): string + { + if ($service !== '' && isset($groups[$service])) { + return $service; + } + + foreach ($groups as $key => $containers) { + foreach ($containers as $container) { + if ($container['state'] === 'running') { + return $key; + } + } + } + + return (string) array_key_first($groups); + } + + /** + * @param array> $groups + * @return list + */ + private static function serviceOptions(array $groups): array + { + $options = []; + foreach ($groups as $key => $containers) { + $option = self::statusOption($key, $key, array_column($containers, 'state')); + if (count($containers) > 1) { + $option['description'] = count($containers).' containers'; + } + $options[] = $option; + } + + return $options; + } + + /** + * @param list $states + */ + private static function serviceDot(array $states): string + { + $allRunning = true; + $anyLive = false; + foreach ($states as $state) { + if ($state !== 'running') { + $allRunning = false; + } + if (in_array($state, ['running', 'starting', 'restarting', 'degraded'], true)) { + $anyLive = true; + } + } + if ($allRunning) { + return 'success'; + } + if (! $anyLive) { + return 'error'; + } + + return 'warning'; + } + + /** + * A running service needs no caption. Anything else is named: Stopped when + * nothing is running, the transitional state when that is the only problem, + * and Degraded when the containers disagree. + * + * @param list $states + */ + private static function serviceStateLabel(array $states): ?string + { + $problems = []; + $anyLive = false; + foreach ($states as $state) { + $state = strtolower($state); + if (in_array($state, ['running', 'starting', 'restarting', 'degraded'], true)) { + $anyLive = true; + } + if ($state !== 'running') { + $problems[$state] = true; + } + } + if ($problems === []) { + return null; + } + + $problemStates = array_keys($problems); + if ($problemStates === ['starting']) { + return 'Starting'; + } + if ($problemStates === ['restarting']) { + return 'Restarting'; + } + if ($problemStates === ['degraded']) { + return 'Degraded'; + } + if (! $anyLive) { + return 'Stopped'; + } + + return 'Degraded'; + } + + /** + * @param list $containers + * @return list + */ + private static function sourcesFor(array $containers): array + { + usort($containers, function (array $left, array $right): int { + $byServer = $left['serverIndex'] <=> $right['serverIndex']; + + return $byServer !== 0 ? $byServer : strcmp($left['name'], $right['name']); + }); + + $serverCounts = []; + foreach ($containers as $container) { + $serverCounts[$container['server']] = ($serverCounts[$container['server']] ?? 0) + 1; + } + $multipleServers = count($serverCounts) > 1; + $sources = []; + foreach ($containers as $container) { + $sources[] = [ + 'server' => $container['server'], + 'container' => $container['name'], + 'label' => self::sourceLabel($container, $multipleServers, $serverCounts[$container['server']], count($containers)), + ]; + } + + return self::uniqueLabels($sources); + } + + /** + * @param array{name: string, server: string, serverName: string} $container + */ + private static function sourceLabel(array $container, bool $multipleServers, int $onServer, int $total): string + { + if ($total < 2) { + return $container['name']; + } + + $parts = []; + if ($multipleServers) { + $name = trim($container['serverName']); + $parts[] = $name !== '' ? $name : $container['server']; + } + if ($onServer > 1 || ! $multipleServers) { + $parts[] = $container['name']; + } + + return implode(' ', $parts); + } + + /** + * @param list $sources + * @return list + */ + private static function uniqueLabels(array $sources): array + { + $counts = array_count_values(array_column($sources, 'label')); + foreach ($sources as $index => $source) { + if ($counts[$source['label']] < 2) { + continue; + } + if (! str_contains($source['label'], $source['container'])) { + $sources[$index]['label'] = trim($source['label'].' '.$source['container']); + } + } + + $counts = array_count_values(array_column($sources, 'label')); + foreach ($sources as $index => $source) { + if (($counts[$source['label']] ?? 0) < 2) { + continue; + } + $sources[$index]['label'] = $source['label'].' '.substr($source['server'], 0, 8); + } + + return $sources; + } + + /** + * @param list $sources + */ + private static function streamKey(array $sources): string + { + return implode('|', array_map( + fn (array $source): string => $source['server'].':'.$source['container'], + $sources, + )); + } +} diff --git a/app/Traits/HasDatabaseStatusInfo.php b/app/Traits/HasDatabaseStatusInfo.php index 33a8be0d0c..031e6767a6 100644 --- a/app/Traits/HasDatabaseStatusInfo.php +++ b/app/Traits/HasDatabaseStatusInfo.php @@ -75,7 +75,6 @@ trait HasDatabaseStatusInfo public function mount(): void { - $this->isPasswordHiddenForMember = auth()->user()?->isMember() ?? false; $this->refresh(); } @@ -83,7 +82,8 @@ trait HasDatabaseStatusInfo { $this->database->refresh(); $this->isPasswordHiddenForMember = ! (auth()->user()?->can('update', $this->database) ?? false); - if ($this->isPasswordHiddenForMember) { + $isCredentialLocked = method_exists($this->database, 'isCredentialLocked') && $this->database->isCredentialLocked(); + if ($this->isPasswordHiddenForMember || $isCredentialLocked) { $this->dbUrl = null; $this->dbUrlPublic = null; } else { diff --git a/app/Traits/HasMetrics.php b/app/Traits/HasMetrics.php index cdf5a9c2e9..d39675276d 100644 --- a/app/Traits/HasMetrics.php +++ b/app/Traits/HasMetrics.php @@ -8,6 +8,57 @@ use Illuminate\Support\Facades\Log; trait HasMetrics { + /** + * Raw Sentinel responses from prefetchMetrics(), keyed by "type:mins"; each is read once. + * + * @var array + */ + protected array $prefetchedMetricResponses = []; + + /** + * Fetch several metric types in one SSH round-trip instead of one per type. The + * get*Metrics() getters then read these responses. On a failure nothing is stored, + * so each getter fetches on its own as before. + * + * @param array $types + */ + public function prefetchMetrics(array $types, int $mins): void + { + $server = $this->getMetricsServer(); + if (! $server->isMetricsEnabled()) { + return; + } + + $from = now()->subMinutes($mins)->toIso8601ZuluString(); + $token = $this->metricsToken($server); + // The token reaches curl on stdin (`-H @-`), so it is not a process argument. The token + // only has letters, digits and `._-+/=` (ServerSetting::isValidSentinelToken), so it + // cannot end the heredoc. Responses are separated by a 0x1E record separator. + $script = implode("\nprintf '\\036'\n", array_map( + fn (string $type) => 'curl -s -H @- "'.$this->getMetricsEndpoint($type, $from)."\" <<'COOLIFY_SENTINEL_AUTH'\nAuthorization: Bearer {$token}\nCOOLIFY_SENTINEL_AUTH", + $types + )); + + try { + $output = instant_remote_process( + ["docker exec -i coolify-sentinel sh -s <<'COOLIFY_SENTINEL_SCRIPT'\n{$script}\nCOOLIFY_SENTINEL_SCRIPT"], + $server, + maxAttempts: 1 + ); + } catch (\Throwable) { + return; + } + + $responses = explode("\x1e", (string) $output); + if (count($responses) !== count($types)) { + return; + } + + foreach ($types as $index => $type) { + $this->prefetchedMetricResponses["{$type}:{$mins}"] = trim($responses[$index]); + } + } + public function getCpuMetrics(int $mins = 5): ?array { return $this->getMetrics('cpu', $mins, 'percent'); @@ -90,25 +141,21 @@ trait HasMetrics return null; } - $from = now()->subMinutes($mins)->toIso8601ZuluString(); - $endpoint = $this->getMetricsEndpoint($type, $from); + $key = "{$type}:{$mins}"; + if (array_key_exists($key, $this->prefetchedMetricResponses)) { + $response = $this->prefetchedMetricResponses[$key]; + unset($this->prefetchedMetricResponses[$key]); + } else { + $from = now()->subMinutes($mins)->toIso8601ZuluString(); + $endpoint = $this->getMetricsEndpoint($type, $from); + $token = $this->metricsToken($server); - $previousToken = null; - try { - $previousToken = $server->settings->sentinel_token; - } catch (DecryptException) { - // fall through to ensureValidSentinelToken which will regenerate + $response = instant_remote_process( + ["docker exec coolify-sentinel sh -c 'curl -H \"Authorization: Bearer {$token}\" {$endpoint}'"], + $server, + false + ); } - $token = $server->settings->ensureValidSentinelToken(); - if ($token !== $previousToken) { - Log::warning('Regenerated sentinel token during metrics read; sentinel container restart required', ['server_id' => $server->id]); - } - - $response = instant_remote_process( - ["docker exec coolify-sentinel sh -c 'curl -H \"Authorization: Bearer {$token}\" {$endpoint}'"], - $server, - false - ); if (str($response)->contains('error')) { $error = json_decode($response, true); @@ -125,6 +172,11 @@ trait HasMetrics /** * Map raw rows to [timestampMs, value] pairs, downsampling long ranges. * + * Sentinel keeps raw samples for the last 24 to 48 hours and one-minute averages + * before that. LTTB buckets by point count, so on ranges that reach past the raw + * window the raw part got most of the chart points. Those ranges are averaged per + * minute first, the same as Sentinel does, so the density is even over time. + * * @param array>|null $rows */ private function mapRows(?array $rows, string $valueField, int $mins): ?array @@ -137,6 +189,10 @@ trait HasMetrics return [(int) $metric['time'], (float) ($metric[$valueField] ?? 0.0)]; })->toArray(); + if ($mins > 24 * 60) { + $metrics = $this->averagePerMinute($metrics); + } + if ($mins > 60 && count($metrics) > 1000) { $metrics = downsampleLTTB($metrics, 1000); } @@ -144,6 +200,44 @@ trait HasMetrics return $metrics; } + /** + * Average [timestampMs, value] pairs into one-minute buckets, timestamped at the + * bucket start like Sentinel's own downsampling. Samples that are a minute or more + * apart keep their own value. + * + * @param array $metrics + * @return array + */ + private function averagePerMinute(array $metrics): array + { + $sums = []; + $counts = []; + foreach ($metrics as [$time, $value]) { + $bucket = intdiv($time, 60_000) * 60_000; + $sums[$bucket] = ($sums[$bucket] ?? 0.0) + $value; + $counts[$bucket] = ($counts[$bucket] ?? 0) + 1; + } + ksort($sums); + + return array_map(fn (int $bucket) => [$bucket, $sums[$bucket] / $counts[$bucket]], array_keys($sums)); + } + + private function metricsToken(Server $server): string + { + $previousToken = null; + try { + $previousToken = $server->settings->sentinel_token; + } catch (DecryptException) { + // fall through to ensureValidSentinelToken which will regenerate + } + $token = $server->settings->ensureValidSentinelToken(); + if ($token !== $previousToken) { + Log::warning('Regenerated sentinel token during metrics read; sentinel container restart required', ['server_id' => $server->id]); + } + + return $token; + } + private function isServerMetrics(): bool { return $this instanceof Server; diff --git a/app/Traits/SshRetryable.php b/app/Traits/SshRetryable.php index 537965df12..b275c1e471 100644 --- a/app/Traits/SshRetryable.php +++ b/app/Traits/SshRetryable.php @@ -90,11 +90,12 @@ trait SshRetryable * @param callable $callback The operation to execute * @param array $context Context for logging (server, command, etc.) * @param bool $throwError Whether to throw error on final failure + * @param int|null $maxAttempts Attempts in total, defaults to constants.ssh.max_retries * @return mixed The result from the callback */ - protected function executeWithSshRetry(callable $callback, array $context = [], bool $throwError = true) + protected function executeWithSshRetry(callable $callback, array $context = [], bool $throwError = true, ?int $maxAttempts = null) { - $maxRetries = config('constants.ssh.max_retries'); + $maxRetries = $maxAttempts ?? config('constants.ssh.max_retries'); $lastError = null; $lastErrorMessage = ''; // Randomly fail the command with a key exchange error for testing diff --git a/app/View/Components/Forms/Input.php b/app/View/Components/Forms/Input.php index e94b181375..0c85d5c3e7 100644 --- a/app/View/Components/Forms/Input.php +++ b/app/View/Components/Forms/Input.php @@ -25,7 +25,7 @@ class Input extends Component public bool $readonly = false, public ?string $helper = null, public bool $allowToPeak = true, - public bool $copyable = false, + public ?bool $copyable = null, public bool $isMultiline = false, public string $defaultClass = 'input', public string $autocomplete = 'off', @@ -71,12 +71,18 @@ class Input extends Component if (is_null($this->name)) { $this->name = $this->modelBinding !== 'null' ? $this->modelBinding : (string) $this->id; } + // Password fields get a copy button unless the caller opts out (account password forms). + $this->copyable ??= $this->type === 'password'; + // Durable class (not type-attr based): Alpine may toggle type to "text" when revealing, // and settings-workspace CSS otherwise overrides utility padding-right. $hasPeek = $this->type === 'password' && $this->allowToPeak; if ($hasPeek) { $this->defaultClass = $this->defaultClass.' input-with-password-toggle'; } + if ($this->type === 'number') { + $this->defaultClass = $this->defaultClass.' input-with-stepper'; + } if ($this->copyable) { // Reserve clearance for a single copy button, or for both the peek eye // and the copy button when the field is a maskable password. diff --git a/bootstrap/helpers/api.php b/bootstrap/helpers/api.php index fce0b591a4..d343a5a06e 100644 --- a/bootstrap/helpers/api.php +++ b/bootstrap/helpers/api.php @@ -206,11 +206,13 @@ function sharedDataApplications(?Application $application = null): array 'include_source_commit_in_build' => 'boolean', 'is_env_sorting_enabled' => 'boolean', 'redact_all_env_values_in_logs' => 'boolean', + 'include_coolify_default_variables' => 'boolean', 'is_pr_deployments_public_enabled' => 'boolean', 'is_gzip_enabled' => 'boolean', 'is_stripprefix_enabled' => 'boolean', 'is_permanent_redirect_enabled' => 'boolean', 'traefik_certificate_resolver' => ['nullable', 'string', 'max:255', 'regex:'.ServerSetting::TRAEFIK_CERTIFICATE_RESOLVER_PATTERN], + 'is_container_label_readonly_enabled' => 'boolean', 'is_raw_compose_deployment_enabled' => 'boolean', 'is_log_drain_enabled' => 'boolean', 'is_gpu_enabled' => 'boolean', @@ -228,6 +230,7 @@ function sharedDataApplications(?Application $application = null): array 'preview_url_template' => 'string', 'max_restart_count' => 'integer|min:0', 'stop_grace_period' => 'nullable|integer|min:'.MIN_STOP_GRACE_PERIOD_SECONDS.'|max:'.MAX_STOP_GRACE_PERIOD_SECONDS, + 'rolling_update_drain_seconds' => 'integer|min:0|max:'.MAX_ROLLING_UPDATE_DRAIN_SECONDS, 'docker_images_to_keep' => 'integer|min:0|max:100', 'static_image' => Rule::enum(StaticImageTypes::class), 'domains' => ValidationPatterns::applicationDomainRules(), @@ -487,13 +490,16 @@ function removeUnnecessaryFieldsFromRequest(Request $request) $request->offsetUnset('inject_build_args_to_dockerfile'); $request->offsetUnset('is_env_sorting_enabled'); $request->offsetUnset('redact_all_env_values_in_logs'); + $request->offsetUnset('include_coolify_default_variables'); $request->offsetUnset('is_pr_deployments_public_enabled'); $request->offsetUnset('stop_grace_period'); + $request->offsetUnset('rolling_update_drain_seconds'); $request->offsetUnset('docker_images_to_keep'); $request->offsetUnset('is_gzip_enabled'); $request->offsetUnset('is_stripprefix_enabled'); $request->offsetUnset('is_permanent_redirect_enabled'); $request->offsetUnset('traefik_certificate_resolver'); + $request->offsetUnset('is_container_label_readonly_enabled'); $request->offsetUnset('is_raw_compose_deployment_enabled'); $request->offsetUnset('is_log_drain_enabled'); $request->offsetUnset('is_gpu_enabled'); diff --git a/bootstrap/helpers/applications.php b/bootstrap/helpers/applications.php index 9e1f8867c5..c6d7a20a0c 100644 --- a/bootstrap/helpers/applications.php +++ b/bootstrap/helpers/applications.php @@ -553,3 +553,18 @@ function clone_application(Application $source, $destination, array $overrides = return $newApplication; } + +/** + * The SSH command that git uses to reach a repository. + * The keepalive options stop a stalled connection after about one minute; without them git waits until the deployment timeout. + */ +function gitSshCommand(int|string $port, ?string $identityFile = null, bool $identitiesOnly = true): string +{ + $command = "ssh -o ConnectTimeout=30 -o ServerAliveInterval=15 -o ServerAliveCountMax=4 -p {$port} -o Port={$port} -o LogLevel=ERROR -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null"; + + if ($identityFile !== null) { + $command .= " -i {$identityFile}".($identitiesOnly ? ' -o IdentitiesOnly=yes' : ''); + } + + return $command; +} diff --git a/bootstrap/helpers/audit.php b/bootstrap/helpers/audit.php index aea549a5be..46ec4ba2b4 100644 --- a/bootstrap/helpers/audit.php +++ b/bootstrap/helpers/audit.php @@ -28,7 +28,7 @@ if (! function_exists('auditLog')) { $token = $user?->currentAccessToken(); $payload = AuditEvent::redactContext(array_merge([ 'event' => $event, - 'ip' => $request?->ip(), + 'ip' => $request ? (client_ip($request) ?: null) : null, 'ua' => substr((string) $request?->userAgent(), 0, 200), 'user_id' => $user?->id, 'user_email' => $user?->email, @@ -102,7 +102,7 @@ if (! function_exists('shouldRecordAuditEvent')) { $request = app()->bound('request') ? request() : null; $fingerprint = hash('sha256', json_encode([ $event, - $request?->ip(), + $request ? client_ip($request) : null, data_get($context, 'reason'), data_get($context, 'server_uuid'), Str::lower(trim((string) data_get($context, 'attempted_email'))), diff --git a/bootstrap/helpers/auth.php b/bootstrap/helpers/auth.php index 7580fc1e06..564ae31062 100644 --- a/bootstrap/helpers/auth.php +++ b/bootstrap/helpers/auth.php @@ -2,13 +2,20 @@ use Illuminate\Http\Request; -function auth_rate_limit_ip(Request $request): string +/** + * The client IP for rate limits, allow-lists, and audit logs. Behind Cloudflare, $request->ip() + * is the Cloudflare edge, so a valid CF-Connecting-IP is used: on cloud, and on self-hosted + * instances when the request comes from a Cloudflare IP. A client that connects directly + * cannot set the client IP with the header. + */ +function client_ip(Request $request): string { + $ip = (string) $request->ip(); $cloudflareIp = $request->header('CF-Connecting-IP'); - if (isCloud() && is_string($cloudflareIp) && filter_var($cloudflareIp, FILTER_VALIDATE_IP) !== false) { + if (is_string($cloudflareIp) && filter_var($cloudflareIp, FILTER_VALIDATE_IP) !== false && (isCloud() || isCloudflareIp($ip))) { return $cloudflareIp; } - return (string) $request->ip(); + return $ip; } diff --git a/bootstrap/helpers/constants.php b/bootstrap/helpers/constants.php index 6dd1155a20..a235c9b680 100644 --- a/bootstrap/helpers/constants.php +++ b/bootstrap/helpers/constants.php @@ -63,6 +63,8 @@ const RESTART_MODE = 'unless-stopped'; const DEFAULT_STOP_GRACE_PERIOD_SECONDS = 30; const MIN_STOP_GRACE_PERIOD_SECONDS = 1; const MAX_STOP_GRACE_PERIOD_SECONDS = 3600; +const DEFAULT_ROLLING_UPDATE_DRAIN_SECONDS = 5; +const MAX_ROLLING_UPDATE_DRAIN_SECONDS = 300; const DATABASE_DOCKER_IMAGES = [ 'bitnami/mariadb', diff --git a/bootstrap/helpers/docker.php b/bootstrap/helpers/docker.php index c27eb4270d..4676b1e754 100644 --- a/bootstrap/helpers/docker.php +++ b/bootstrap/helpers/docker.php @@ -787,7 +787,7 @@ function caddySiteAddress(string $scheme, string $host, bool $isForceHttpsEnable return "{$scheme}://{$host}"; } -function fqdnLabelsForCaddy(string $network, string $uuid, Collection $domains, bool $is_force_https_enabled = false, $onlyPort = null, ?Collection $serviceLabels = null, ?bool $is_gzip_enabled = true, ?bool $is_stripprefix_enabled = true, ?string $service_name = null, ?string $image = null, string $redirect_direction = 'both', ?string $predefinedPort = null, bool $is_http_basic_auth_enabled = false, ?string $http_basic_auth_username = null, ?string $http_basic_auth_password = null, ?Collection $noindex_domains = null, bool $is_traffic_analytics_enabled = false, array $domainPortOverrides = [], bool $supports_log_append = false, bool $supports_basic_auth_directive = false, ?int $http_basic_auth_bcrypt_cost = null, array $http_basic_auth_argon2id_options = [], bool $is_permanent_redirect_enabled = false) +function fqdnLabelsForCaddy(string $network, string $uuid, Collection $domains, bool $is_force_https_enabled = false, $onlyPort = null, ?Collection $serviceLabels = null, ?bool $is_gzip_enabled = true, ?bool $is_stripprefix_enabled = true, ?string $service_name = null, ?string $image = null, string $redirect_direction = 'both', ?string $predefinedPort = null, bool $is_http_basic_auth_enabled = false, ?string $http_basic_auth_username = null, ?string $http_basic_auth_password = null, ?Collection $noindex_domains = null, bool $is_traffic_analytics_enabled = false, array $domainPortOverrides = [], bool $supports_log_append = false, bool $supports_basic_auth_directive = false, ?int $http_basic_auth_bcrypt_cost = null, array $http_basic_auth_argon2id_options = [], bool $is_permanent_redirect_enabled = false, bool $is_retry_enabled = false) { $labels = collect([]); if ($serviceLabels) { @@ -840,6 +840,11 @@ function fqdnLabelsForCaddy(string $network, string $uuid, Collection $domains, } else { $labels->push("caddy_{$loop}.{$handle}.{$loop}_reverse_proxy={{upstreams}}"); } + if ($is_retry_enabled) { + // A failed connection, for example to the old container that stops during a rolling + // update, is retried on another upstream. Caddy always retries a connection that failed. + $labels->push("caddy_{$loop}.{$handle}.{$loop}_reverse_proxy.lb_try_duration=5s"); + } $labels->push("caddy_{$loop}.{$handle}={$path}*"); if ($is_gzip_enabled) { $labels->push("caddy_{$loop}.encode=zstd gzip"); @@ -944,7 +949,7 @@ function traefikHostRulesWithoutIpv6Brackets(Collection $labels): Collection ) : $label); } -function fqdnLabelsForTraefik(string $uuid, Collection $domains, bool $is_force_https_enabled = false, $onlyPort = null, ?Collection $serviceLabels = null, ?bool $is_gzip_enabled = true, ?bool $is_stripprefix_enabled = true, ?string $service_name = null, bool $generate_unique_uuid = false, ?string $image = null, string $redirect_direction = 'both', bool $is_http_basic_auth_enabled = false, ?string $http_basic_auth_username = null, ?string $http_basic_auth_password = null, ?Collection $noindex_domains = null, bool $escape_redirect_replacement_for_compose = true, array $domainPortOverrides = [], ?string $certificate_resolver = 'letsencrypt', ?int $http_basic_auth_bcrypt_cost = null, bool $is_permanent_redirect_enabled = false) +function fqdnLabelsForTraefik(string $uuid, Collection $domains, bool $is_force_https_enabled = false, $onlyPort = null, ?Collection $serviceLabels = null, ?bool $is_gzip_enabled = true, ?bool $is_stripprefix_enabled = true, ?string $service_name = null, bool $generate_unique_uuid = false, ?string $image = null, string $redirect_direction = 'both', bool $is_http_basic_auth_enabled = false, ?string $http_basic_auth_username = null, ?string $http_basic_auth_password = null, ?Collection $noindex_domains = null, bool $escape_redirect_replacement_for_compose = true, array $domainPortOverrides = [], ?string $certificate_resolver = 'letsencrypt', ?int $http_basic_auth_bcrypt_cost = null, bool $is_permanent_redirect_enabled = false, bool $is_retry_enabled = false) { $labels = collect([]); $labels->push('traefik.enable=true'); @@ -999,6 +1004,14 @@ function fqdnLabelsForTraefik(string $uuid, Collection $domains, bool $is_force_ ->filter() ->unique(); } + if ($is_retry_enabled) { + // Sends a request to another container when the connection fails, for example to the old + // container that stops during a rolling update. Traefik does not retry after a response. + // A retry can pick the old container again under load; 3 attempts left about 5% of the + // requests failing in a live test, 5 attempts less than 1% (#8627). + $labels->push("traefik.http.middlewares.retry-{$uuid}.retry.attempts=5"); + $middlewares_from_labels->push("retry-{$uuid}"); + } foreach ($domains as $loop => $domain) { try { if ($generate_unique_uuid) { @@ -1264,6 +1277,7 @@ function generateLabelsApplication(Application $application, ?ApplicationPreview http_basic_auth_bcrypt_cost: $application->http_basic_auth_bcrypt_cost, noindex_domains: $noindexDomains, domainPortOverrides: $application->domain_port_overrides ?? [], + is_retry_enabled: true, certificate_resolver: $application->destination->server->traefikCertificateResolver($application->settings?->traefik_certificate_resolver), )); break; @@ -1288,6 +1302,7 @@ function generateLabelsApplication(Application $application, ?ApplicationPreview supports_log_append: $application->destination->server->caddySupportsLogAppend(), supports_basic_auth_directive: $application->destination->server->caddySupportsBasicAuthDirective(), domainPortOverrides: $application->domain_port_overrides ?? [], + is_retry_enabled: true, )); break; } @@ -1308,6 +1323,7 @@ function generateLabelsApplication(Application $application, ?ApplicationPreview noindex_domains: $noindexDomains, escape_redirect_replacement_for_compose: false, domainPortOverrides: $application->domain_port_overrides ?? [], + is_retry_enabled: true, certificate_resolver: $application->destination->server->traefikCertificateResolver($application->settings?->traefik_certificate_resolver), )); $labels = $labels->merge(fqdnLabelsForCaddy( @@ -1330,6 +1346,7 @@ function generateLabelsApplication(Application $application, ?ApplicationPreview supports_log_append: $application->destination->server->caddySupportsLogAppend(), supports_basic_auth_directive: $application->destination->server->caddySupportsBasicAuthDirective(), domainPortOverrides: $application->domain_port_overrides ?? [], + is_retry_enabled: true, )); } } diff --git a/bootstrap/helpers/parsers.php b/bootstrap/helpers/parsers.php index 09c2b69422..5e34a4dc4c 100644 --- a/bootstrap/helpers/parsers.php +++ b/bootstrap/helpers/parsers.php @@ -148,7 +148,8 @@ function composeResourceDirectory(Application|Service $resource): string * Renders a long-syntax bind with its resolved source. It keeps the options that Docker Compose * accepts, such as read_only and bind, and removes other keys, which Compose rejects. Without * `bind.create_host_path`, a missing source is created, as with short syntax: Docker Compose - * before v2.30 does not do that for long syntax. + * before v2.30 does not do that for long syntax. A short-syntax mode in the target + * (`/data/users.yml:ro`) becomes `read_only`, because Docker rejects it in long syntax. * * @param array $volume * @return array @@ -160,6 +161,10 @@ function composeLongBindVolume(array $volume, string $source): array fn ($key) => in_array($key, ['type', 'source', 'target', 'read_only', 'consistency', 'bind'], true) || str_starts_with((string) $key, 'x-'), ARRAY_FILTER_USE_KEY ); + if (preg_match('/^(.+):(ro|rw)$/', (string) ($volume['target'] ?? ''), $matches)) { + $volume['target'] = $matches[1]; + $volume['read_only'] ??= $matches[2] === 'ro'; + } $volume['source'] = $source; $volume['bind'] = is_array($volume['bind'] ?? null) ? $volume['bind'] : []; $volume['bind']['create_host_path'] ??= true; @@ -2018,6 +2023,10 @@ function applicationParser(Application $resource, int $pull_request_id = 0, ?int $serviceEnvKey = str(normalizeComposeServiceName((string) $forServiceName))->upper(); $coolifyEnvironments->put('SERVICE_URL_'.$serviceEnvKey, $coolifyUrl->__toString()); $coolifyEnvironments->put('SERVICE_FQDN_'.$serviceEnvKey, $coolifyFqdn); + // A preview uses its own domains; the stored variables belong to the main application. + if ($isPullRequest) { + continue; + } $resource->environment_variables()->updateOrCreate([ 'resourceable_type' => Application::class, 'resourceable_id' => $resource->id, @@ -2034,7 +2043,7 @@ function applicationParser(Application $resource, int $pull_request_id = 0, ?int 'value' => $coolifyFqdn, 'is_preview' => false, ]); - } else { + } elseif (! $isPullRequest) { $resource->environment_variables()->where('resourceable_type', Application::class) ->where('resourceable_id', $resource->id) ->where('key', 'LIKE', "SERVICE_FQDN_{$serviceNameFormatted}%") @@ -2099,7 +2108,7 @@ function applicationParser(Application $resource, int $pull_request_id = 0, ?int if ($environment->count() > 0) { $environment = $environment->filter(function ($value, $key) { return ! str($key)->startsWith('SERVICE_FQDN_'); - })->map(function ($value, $key) use ($resource) { + })->map(function ($value, $key) use ($resource, $isPullRequest) { // Preserve empty strings and null values with correct Docker Compose semantics: // - Empty string: Variable is set to "" (e.g., HTTP_PROXY="" means "no proxy") // - Null: Variable is unset/removed from container environment (may inherit from host) @@ -2107,8 +2116,9 @@ function applicationParser(Application $resource, int $pull_request_id = 0, ?int // User explicitly wants variable unset - respect that // NEVER override from database - null means "inherit from environment" // Keep as null (will be excluded from container environment) - } elseif ($value === '') { - // Empty string - allow database override for backward compatibility + } elseif ($value === '' && ! ($isPullRequest && str($key)->startsWith('SERVICE_URL_'))) { + // Empty string - allow database override for backward compatibility. + // The stored service URLs belong to the main application, so a preview does not use them. $dbEnv = $resource->environment_variables()->where('key', $key)->first(); // Only use database override if it exists AND has a non-empty value if ($dbEnv && str($dbEnv->value)->isNotEmpty()) { diff --git a/bootstrap/helpers/remoteProcess.php b/bootstrap/helpers/remoteProcess.php index 877d790266..e5dd73de46 100644 --- a/bootstrap/helpers/remoteProcess.php +++ b/bootstrap/helpers/remoteProcess.php @@ -216,7 +216,7 @@ function instant_remote_process_with_timeout(Collection|array $command, Server $ ); } -function instant_remote_process(Collection|array $command, Server $server, bool $throwError = true, bool $no_sudo = false, ?int $timeout = null, bool $disableMultiplexing = false): ?string +function instant_remote_process(Collection|array $command, Server $server, bool $throwError = true, bool $no_sudo = false, ?int $timeout = null, bool $disableMultiplexing = false, ?int $maxAttempts = null): ?string { $command = $command instanceof Collection ? $command->toArray() : $command; @@ -248,7 +248,8 @@ function instant_remote_process(Collection|array $command, Server $server, bool 'command_preview' => substr($command_string, 0, 100), 'function' => 'instant_remote_process', ], - $throwError + $throwError, + $maxAttempts ); } diff --git a/bootstrap/helpers/services.php b/bootstrap/helpers/services.php index 8aff368171..7c664649cb 100644 --- a/bootstrap/helpers/services.php +++ b/bootstrap/helpers/services.php @@ -25,6 +25,9 @@ function service_logo_urls(mixed $logo): array ]; } +use App\Actions\Service\StartService; +use App\Actions\Service\StopService; +use App\Jobs\VolumeCloneJob; use App\Models\Application; use App\Models\GithubApp; use App\Models\GitlabApp; @@ -32,7 +35,11 @@ use App\Models\LocalFileVolume; use App\Models\Service; use App\Models\ServiceApplication; use App\Models\ServiceDatabase; +use App\Models\StandaloneDocker; +use App\Models\SwarmDocker; use Illuminate\Database\Eloquent\Model; +use Illuminate\Support\Facades\Bus; +use Illuminate\Support\Facades\DB; use Illuminate\Support\Facades\Log; use Illuminate\Support\Stringable; use Spatie\Url\Url; @@ -55,7 +62,8 @@ function resource_logo_url(string $name): string * the service template, or how the application is built. Null when there is no specific logo. * * Applications with a domain also get the favicon URL of their first domain. The browser loads it - * and shows it in place of the logo when it loads. Coolify itself never requests it. + * and shows it in place of the logo when it loads. Coolify itself never requests it. Applications + * behind HTTP basic auth get no favicon: the request makes the browser ask for the credentials. * * @return array{src: string, dark: ?string, fallback: ?string, padded: bool, favicon: ?string}|null */ @@ -93,7 +101,9 @@ function resource_logo(Model $resource): ?array default => $resourceLogo('public-repo'), }; - return [...$logo, 'favicon' => resource_favicon_url($resource->fqdn)]; + $favicon = $resource->is_http_basic_auth_enabled ? null : resource_favicon_url($resource->fqdn); + + return [...$logo, 'favicon' => $favicon]; } if (! method_exists($resource, 'type')) { @@ -663,3 +673,128 @@ function applyServiceApplicationPrerequisites(Service $service): void ]); } } + +/** + * Clones a service to a destination. The applications, databases, and volumes of the clone come from parse(), + * so their names are the names that the compose file of the clone uses. + * + * @param array $overrides Attributes for the new service, for example uuid, name, and environment_id. + */ +function clone_service(Service $source, StandaloneDocker|SwarmDocker $destination, array $overrides = [], bool $cloneVolumeData = false): Service +{ + $teamId = $source->team()->id; + $uuid = $overrides['uuid'] ?? new_public_id(); + + $newService = $source->replicate([ + 'id', + 'created_at', + 'updated_at', + ])->fill([ + 'name' => $source->name.'-clone-'.$uuid, + ...$overrides, + 'uuid' => $uuid, + 'destination_id' => $destination->id, + 'destination_type' => $destination->getMorphClass(), + 'server_id' => $destination->server_id, + ]); + $newService->save(); + + foreach ($source->tags as $tag) { + $newService->tags()->attach($tag->id); + } + + foreach ($source->scheduled_tasks()->get() as $task) { + $task->replicate([ + 'id', + 'created_at', + 'updated_at', + ])->fill([ + 'uuid' => new_public_id(), + 'service_id' => $newService->id, + 'team_id' => $teamId, + ])->save(); + } + + foreach ($source->environment_variables()->get() as $environmentVariable) { + $environmentVariable->replicate([ + 'id', + 'created_at', + 'updated_at', + ])->fill([ + 'resourceable_id' => $newService->id, + 'resourceable_type' => $newService->getMorphClass(), + ])->save(); + } + + $source->cloneSecretManagerLinkTo($newService); + + // Service::replicate() does not copy the child rows, so parse() creates them for the clone. + $newService->parse(); + $newService->refresh(); + + $sourceApplicationsByName = $source->applications()->get()->keyBy('name'); + $sourceDatabasesByName = $source->databases()->get()->keyBy('name'); + $pendingVolumeClones = []; + + foreach ([ + [$newService->applications()->get(), $sourceApplicationsByName], + [$newService->databases()->get(), $sourceDatabasesByName], + ] as [$newResources, $sourcesByName]) { + foreach ($newResources as $newResource) { + $newResource->fill(['status' => 'exited'])->save(); + + $sourceResource = $sourcesByName->get($newResource->name); + if (! $sourceResource || ! $cloneVolumeData) { + continue; + } + + $targetVolumesByMount = $newResource->persistentStorages()->get()->keyBy('mount_path'); + foreach ($sourceResource->persistentStorages()->get() as $sourceVolume) { + $targetVolume = $targetVolumesByMount->get($sourceVolume->mount_path); + if ($targetVolume) { + $pendingVolumeClones[] = [$sourceVolume->name, $targetVolume]; + } + } + } + } + + foreach ($newService->databases()->get() as $database) { + $sourceDatabase = $sourceDatabasesByName->get($database->name); + foreach ($sourceDatabase?->scheduledBackups()->get() ?? [] as $backup) { + $newBackup = $backup->replicate([ + 'id', + 'created_at', + 'updated_at', + ])->fill([ + 'uuid' => new_public_id(), + 'database_id' => $database->id, + 'database_type' => $database->getMorphClass(), + 'team_id' => $teamId, + ]); + $newBackup->save(); + $backup->copyS3StoragesTo($newBackup); + } + } + + $sourceServer = $source->destination?->server; + $targetServer = $newService->destination?->server; + if ($pendingVolumeClones !== [] && $sourceServer && $targetServer) { + $chain = [ + function () use ($source) { + StopService::run($source); + }, + ...array_map( + fn (array $clone) => new VolumeCloneJob($clone[0], $clone[1]->name, $sourceServer, $targetServer, $clone[1]), + $pendingVolumeClones, + ), + function () use ($source) { + StartService::run($source); + }, + ]; + + // The jobs load the new volume rows, so they must not run before the transaction of the caller commits. + DB::afterCommit(fn () => Bus::chain($chain)->onQueue('high')->dispatch()); + } + + return $newService; +} diff --git a/bootstrap/helpers/shared.php b/bootstrap/helpers/shared.php index 48aa00a907..2ed69ed773 100644 --- a/bootstrap/helpers/shared.php +++ b/bootstrap/helpers/shared.php @@ -46,7 +46,6 @@ use DanHarrin\LivewireRateLimiting\Exceptions\TooManyRequestsException; use Illuminate\Database\Eloquent\Model; use Illuminate\Database\Eloquent\ModelNotFoundException; use Illuminate\Database\UniqueConstraintViolationException; -use Illuminate\Process\Pool; use Illuminate\Support\Carbon; use Illuminate\Support\Collection; use Illuminate\Support\Facades\Auth; @@ -2449,7 +2448,7 @@ function isCloudflareIp(string $ip): bool $cloudflareIps = [ '173.245.48.0/20', '103.21.244.0/22', '103.22.200.0/22', '103.31.4.0/22', '141.101.64.0/18', '108.162.192.0/18', '190.93.240.0/20', '188.114.96.0/20', - '197.234.240.0/22', '198.41.128.0/17', '162.158.0.0/15', '104.16.0.0/13', + '197.234.240.0/22', '198.41.128.0/17', '162.158.0.0/15', '104.16.0.0/13', '104.24.0.0/14', '172.64.0.0/13', '131.0.72.0/22', '2400:cb00::/32', '2606:4700::/32', '2803:f800::/32', '2405:b500::/32', '2405:8100::/32', '2a06:98c0::/29', '2c0f:f248::/32', @@ -2733,15 +2732,21 @@ function get_public_ips(): void function detect_public_ips(array $columns = ['public_ipv4', 'public_ipv6']): array { $versions = array_intersect_key(['public_ipv4' => 4, 'public_ipv6' => 6], array_flip($columns)); - $results = Process::concurrently(function (Pool $pool) use ($versions) { - foreach ($versions as $column => $version) { - $pool->as($column)->path(__DIR__)->command("curl -{$version}s https://ifconfig.io"); - } - }); + + // Start the lookups together, but wait for each one alone, so a timeout of one family does not discard the other. + $processes = []; + foreach ($versions as $column => $version) { + $processes[$column] = Process::path(__DIR__)->start("curl -{$version}s --max-time 10 https://ifconfig.io"); + } $detected = []; - foreach ($versions as $column => $version) { - $ip = trim($results[$column]->output()); + foreach ($processes as $column => $process) { + try { + $ip = trim($process->wait()->output()); + } catch (Throwable) { + continue; + } + $version = $versions[$column]; if (filter_var($ip, FILTER_VALIDATE_IP, $version === 4 ? FILTER_FLAG_IPV4 : FILTER_FLAG_IPV6) !== false) { $detected[$column] = $ip; } @@ -4623,7 +4628,7 @@ function wireNavigate(): string } /** - * Registers a resource menu that the main sidebar shows in place of its main menu on wide screens + * Registers a resource menu that the main sidebar and the mobile drawer show in place of their main menu * (see components/navbar.blade.php). The menu teleports itself into #main-nav-context-{depth}. * Back on a menu shows the menu one depth lower, and Back at depth 0 shows the main menu. */ @@ -5645,6 +5650,80 @@ function resolveSharedEnvironmentVariables(?string $value, $resource): ?string return str($value)->value(); } +/** + * Replace locked values in the generated compose file of a resource, for display only. + * + * The compose parsers write resolved values into `docker_compose`: environment variables, database + * overrides, and shared variable references. Deployments need those values, so only the displayed + * copy is redacted. Covered: the resource's locked variables (also variables that reference a locked + * shared variable) and locked team, project, and environment variables that the raw compose references. + * Values shorter than 4 characters are not replaced, because they would match unrelated text. + * A parsed compose (array or collection) gets the same replacement in every string value. + * + * @template T of string|array|Collection|null + * + * @param T $compose + * @return T + */ +function redactLockedValuesInGeneratedCompose(string|array|Collection|null $compose, Application|Service $resource): string|array|Collection|null +{ + if (blank($compose)) { + return $compose; + } + + $values = EnvironmentVariable::logRedactionSecretsFor($resource, includePreview: true, resolveRemoteSecrets: false)['locked']; + + preg_match_all('/{{\s*(\w+)\.([^}\s]+)\s*}}/', (string) $resource->docker_compose_raw, $matches, PREG_SET_ORDER); + $references = collect($matches)->filter(fn (array $match): bool => in_array($match[1], ['team', 'project', 'environment'], true)); + $teamId = $resource->team()?->id; + if ($references->isNotEmpty() && $teamId !== null) { + $scopeIds = [ + 'team' => $teamId, + 'project' => $resource->environment?->project_id, + 'environment' => $resource->environment_id, + ]; + $values = SharedEnvironmentVariable::query() + ->where('team_id', $teamId) + ->where('is_shown_once', true) + ->whereIn('key', $references->pluck(2)->unique()->values()) + ->get() + ->filter(fn (SharedEnvironmentVariable $shared): bool => $references->contains(fn (array $match): bool => $match[1] === $shared->type && $match[2] === $shared->key) + && $scopeIds[$shared->type] !== null + && (int) $shared->{"{$shared->type}_id"} === (int) $scopeIds[$shared->type]) + ->pluck('value') + ->merge($values) + ->all(); + } + + $variants = collect($values) + ->filter(fn (mixed $value): bool => is_string($value)) + ->flatMap(fn (string $value): array => [ + $value, + escapeDollarSign($value), + str_replace("'", "''", $value), + substr((string) json_encode($value, JSON_UNESCAPED_SLASHES | JSON_UNESCAPED_UNICODE), 1, -1), + ]) + ->filter(fn (string $value): bool => mb_strlen($value) >= 4) + ->unique() + ->mapWithKeys(fn (string $value): array => [$value => '']); + + if ($variants->isEmpty()) { + return $compose; + } + + // strtr() replaces the longest match first and does not search the replaced text again. + $redact = function (mixed $value) use (&$redact, $variants): mixed { + return match (true) { + is_string($value) => strtr($value, $variants->all()), + is_array($value) => array_map($redact, $value), + $value instanceof Collection => $value->map($redact), + default => $value, + }; + }; + + return $redact($compose); +} + /** * Convert an ISO 3166-1 alpha-2 country code into its regional-indicator flag emoji. * diff --git a/bootstrap/helpers/subscriptions.php b/bootstrap/helpers/subscriptions.php index 95442ca359..f91c165e48 100644 --- a/bootstrap/helpers/subscriptions.php +++ b/bootstrap/helpers/subscriptions.php @@ -91,6 +91,7 @@ function allowedPathsForUnsubscribedAccounts() 'user/passkeys/options', 'passkeys/confirm', 'passkeys/confirm/options', + ...allowedPathsForTwoFactorSetup(), ]; } function allowedPathsForBoardingAccounts() @@ -109,6 +110,27 @@ function allowedPathsForInvalidAccounts() 'force-password-reset', 'two-factor-challenge', 'livewire/update', + ...allowedPathsForTwoFactorSetup(), + ]; +} +/** + * Paths a user needs to configure 2FA when their team requires it. + * + * @return array + */ +function allowedPathsForTwoFactorSetup(): array +{ + return [ + 'two-factor-required', + 'two-factor-challenge', + 'logout', + 'user/confirm-password', + 'user/confirmed-password-status', + 'user/two-factor-authentication', + 'user/confirmed-two-factor-authentication', + 'user/two-factor-qr-code', + 'user/two-factor-secret-key', + 'user/two-factor-recovery-codes', ]; } diff --git a/composer.json b/composer.json index 29d24ed495..68b73d3370 100644 --- a/composer.json +++ b/composer.json @@ -17,7 +17,7 @@ "firebase/php-jwt": "^7.2.1", "guzzlehttp/guzzle": "^7.15.5", "laravel/fortify": "^1.40.0", - "laravel/framework": "^12.69.3", + "laravel/framework": "^12.70.1", "laravel/horizon": "^5.50.0", "laravel/mcp": "^1.0.1", "laravel/pail": "^1.2.7", diff --git a/composer.lock b/composer.lock index 02f3e6112f..1611f77f31 100644 --- a/composer.lock +++ b/composer.lock @@ -4,7 +4,7 @@ "Read more about it at https://getcomposer.org/doc/01-basic-usage.md#installing-dependencies", "This file is @generated automatically" ], - "content-hash": "f97af22f1628cec49170f1d40c582a9d", + "content-hash": "37980c90750e1f51758daaadef1cf06e", "packages": [ { "name": "aws/aws-crt-php", @@ -1903,16 +1903,16 @@ }, { "name": "laravel/framework", - "version": "v12.69.3", + "version": "v12.70.1", "source": { "type": "git", "url": "https://github.com/laravel/framework.git", - "reference": "58ea544a2a80dc03c168e13a5dc9a1d176a88717" + "reference": "e09fc54263d1a56cfe1709958fcbb22a4159b1e3" }, "dist": { "type": "zip", - "url": "https://api.github.com/repos/laravel/framework/zipball/58ea544a2a80dc03c168e13a5dc9a1d176a88717", - "reference": "58ea544a2a80dc03c168e13a5dc9a1d176a88717", + "url": "https://api.github.com/repos/laravel/framework/zipball/e09fc54263d1a56cfe1709958fcbb22a4159b1e3", + "reference": "e09fc54263d1a56cfe1709958fcbb22a4159b1e3", "shasum": "" }, "require": { @@ -2025,10 +2025,10 @@ "league/flysystem-sftp-v3": "^3.25.1", "mockery/mockery": "^1.6.10", "opis/json-schema": "^2.4.1", - "orchestra/testbench-core": "^10.9.0", + "orchestra/testbench-core": "^10.15.0", "pda/pheanstalk": "^5.0.6|^7.0.0", "php-http/discovery": "^1.15", - "phpstan/phpstan": "^2.1.41", + "phpstan/phpstan": "2.2.13", "phpunit/phpunit": "^10.5.35|^11.5.3|^12.0.1", "predis/predis": "^2.3|^3.0", "resend/resend-php": "^0.10.0|^1.0", @@ -2121,7 +2121,7 @@ "issues": "https://github.com/laravel/framework/issues", "source": "https://github.com/laravel/framework" }, - "time": "2026-09-29T12:53:25+00:00" + "time": "2026-10-21T12:55:36+00:00" }, { "name": "laravel/horizon", diff --git a/config/constants.php b/config/constants.php index a55249f041..744eaf1b1f 100644 --- a/config/constants.php +++ b/config/constants.php @@ -5,9 +5,9 @@ $sshCommandTimeout = (int) env('SSH_COMMAND_TIMEOUT', 3600); return [ 'coolify' => [ - 'version' => env('COOLIFY_VERSION') ?: '4.5.0', - 'helper_version' => '1.0.18', - 'railpack_version' => '0.40.1', + 'version' => env('COOLIFY_VERSION') ?: '4.5.1', + 'helper_version' => '1.0.19', + 'railpack_version' => '0.43.0', 'self_hosted' => env('SELF_HOSTED', true), 'autoupdate' => env('AUTOUPDATE'), 'base_config_path' => env('BASE_CONFIG_PATH', '/data/coolify'), diff --git a/config/development-qemu.php b/config/development-qemu.php index 983cd587aa..c9b30d049a 100644 --- a/config/development-qemu.php +++ b/config/development-qemu.php @@ -147,6 +147,8 @@ return [ 'dns' => '1.1.1.1', // MiB per VM. Many dev instances share one host, so keep this small; existing VMs get it on their next start. 'memory' => (int) env('DEVELOPMENT_QEMU_MEMORY', 1024), + // MiB for a VM with a passed-through GPU (dev:qemu --gpu). The NVIDIA driver needs more than the default. + 'gpu_memory' => (int) env('DEVELOPMENT_QEMU_GPU_MEMORY', 4096), 'vcpus' => 2, 'disk_size' => '20G', 'libvirt_network' => $developmentQemuInstance === null ? 'default' : "coolify-dev-{$developmentQemuSlot}", diff --git a/database/migrations/2026_10_23_090005_add_server_validation_limits_to_instance_settings_table.php b/database/migrations/2026_10_23_090005_add_server_validation_limits_to_instance_settings_table.php new file mode 100644 index 0000000000..50a4fbf797 --- /dev/null +++ b/database/migrations/2026_10_23_090005_add_server_validation_limits_to_instance_settings_table.php @@ -0,0 +1,34 @@ +unsignedInteger('server_validation_rate_limit_seconds')->default(600); + $table->unsignedInteger('server_validation_max_failures_per_ip')->default(10); + $table->unsignedInteger('server_validation_max_failures_per_team')->default(30); + }); + } + + /** + * Reverse the migrations. + */ + public function down(): void + { + Schema::table('instance_settings', function (Blueprint $table) { + $table->dropColumn([ + 'server_validation_rate_limit_seconds', + 'server_validation_max_failures_per_ip', + 'server_validation_max_failures_per_team', + ]); + }); + } +}; diff --git a/database/migrations/2026_10_23_182201_add_canvas_position_to_projects_table.php b/database/migrations/2026_10_23_182201_add_canvas_position_to_projects_table.php new file mode 100644 index 0000000000..e7642e0821 --- /dev/null +++ b/database/migrations/2026_10_23_182201_add_canvas_position_to_projects_table.php @@ -0,0 +1,22 @@ +json('canvas_position')->nullable(); + }); + } + + public function down(): void + { + Schema::table('projects', function (Blueprint $table) { + $table->dropColumn('canvas_position'); + }); + } +}; diff --git a/database/migrations/2026_10_23_200902_add_canvas_position_to_servers_table.php b/database/migrations/2026_10_23_200902_add_canvas_position_to_servers_table.php new file mode 100644 index 0000000000..efef6d1cfe --- /dev/null +++ b/database/migrations/2026_10_23_200902_add_canvas_position_to_servers_table.php @@ -0,0 +1,22 @@ +json('canvas_position')->nullable(); + }); + } + + public function down(): void + { + Schema::table('servers', function (Blueprint $table) { + $table->dropColumn('canvas_position'); + }); + } +}; diff --git a/database/migrations/2026_10_23_224202_add_rolling_update_drain_seconds_to_application_settings.php b/database/migrations/2026_10_23_224202_add_rolling_update_drain_seconds_to_application_settings.php new file mode 100644 index 0000000000..88edb7081f --- /dev/null +++ b/database/migrations/2026_10_23_224202_add_rolling_update_drain_seconds_to_application_settings.php @@ -0,0 +1,31 @@ +integer('rolling_update_drain_seconds') + ->default(5) + ->after('stop_grace_period') + ->comment('Seconds to wait after the new container is healthy before old containers stop during a rolling update (0-300). 0 turns the wait off.'); + }); + } + + /** + * Reverse the migrations. + */ + public function down(): void + { + Schema::table('application_settings', function (Blueprint $table) { + $table->dropColumn('rolling_update_drain_seconds'); + }); + } +}; diff --git a/database/migrations/2026_10_25_115811_add_is_two_factor_required_to_teams_table.php b/database/migrations/2026_10_25_115811_add_is_two_factor_required_to_teams_table.php new file mode 100644 index 0000000000..ecc79886c3 --- /dev/null +++ b/database/migrations/2026_10_25_115811_add_is_two_factor_required_to_teams_table.php @@ -0,0 +1,22 @@ +boolean('is_two_factor_required')->default(false); + }); + } + + public function down(): void + { + Schema::table('teams', function (Blueprint $table) { + $table->dropColumn('is_two_factor_required'); + }); + } +}; diff --git a/database/seeders/GitlabAppSeeder.php b/database/seeders/GitlabAppSeeder.php index 5dfb599024..aa31398f43 100644 --- a/database/seeders/GitlabAppSeeder.php +++ b/database/seeders/GitlabAppSeeder.php @@ -4,6 +4,7 @@ namespace Database\Seeders; use App\Models\GitlabApp; use Illuminate\Database\Seeder; +use Illuminate\Support\Facades\DB; class GitlabAppSeeder extends Seeder { @@ -21,5 +22,9 @@ class GitlabAppSeeder extends Seeder 'is_public' => true, 'team_id' => 0, ]); + // The explicit id does not move the Postgres sequence, so the next GitLab App would get id 1 again. + if (DB::getDriverName() === 'pgsql') { + DB::statement("SELECT setval('gitlab_apps_id_seq', (SELECT MAX(id) FROM gitlab_apps))"); + } } } diff --git a/database/seeders/PopulateSshKeysDirectorySeeder.php b/database/seeders/PopulateSshKeysDirectorySeeder.php index 87984769ca..0318508c00 100644 --- a/database/seeders/PopulateSshKeysDirectorySeeder.php +++ b/database/seeders/PopulateSshKeysDirectorySeeder.php @@ -24,8 +24,8 @@ class PopulateSshKeysDirectorySeeder extends Seeder }); if (isDev()) { - Process::run('chown -R 9999:9999 '.storage_path('app/ssh/keys')); - Process::run('chown -R 9999:9999 '.storage_path('app/ssh/mux')); + Process::run('chown -R www-data:www-data '.storage_path('app/ssh/keys')); + Process::run('chown -R www-data:www-data '.storage_path('app/ssh/mux')); } else { Process::run('chown -R 9999:root '.storage_path('app/ssh/keys')); Process::run('chown -R 9999:root '.storage_path('app/ssh/mux')); diff --git a/database/seeders/RootUserSeeder.php b/database/seeders/RootUserSeeder.php index 9bc93a9a9e..a15f04e33f 100644 --- a/database/seeders/RootUserSeeder.php +++ b/database/seeders/RootUserSeeder.php @@ -25,9 +25,12 @@ class RootUserSeeder extends Seeder return; } + // The installer writes an empty ROOT_USERNAME line, so an empty value must also use the default. + $username = env('ROOT_USERNAME') ?: 'Root User'; + $validator = Validator::make([ 'email' => env('ROOT_USER_EMAIL'), - 'username' => env('ROOT_USERNAME', 'Root User'), + 'username' => $username, 'password' => env('ROOT_USER_PASSWORD'), ], [ 'email' => ['required', 'email:rfc,dns', 'max:255'], @@ -48,7 +51,7 @@ class RootUserSeeder extends Seeder try { $user = (new User)->forceFill([ 'id' => 0, - 'name' => env('ROOT_USERNAME', 'Root User'), + 'name' => $username, 'email' => env('ROOT_USER_EMAIL'), 'password' => Hash::make(env('ROOT_USER_PASSWORD')), ]); diff --git a/docker/coolify-helper/Dockerfile b/docker/coolify-helper/Dockerfile index b3c068f437..e3acabdd9a 100644 --- a/docker/coolify-helper/Dockerfile +++ b/docker/coolify-helper/Dockerfile @@ -1,25 +1,28 @@ # Versions # https://hub.docker.com/_/alpine -ARG BASE_IMAGE=alpine:3.21 +ARG BASE_IMAGE=alpine:3.24@sha256:be2656061b536c1883c4034790e21d0a09c5aaaf51968b89368a3c0371ea2c9a +# https://quay.io/repository/minio/aistor/mc?tab=tags +ARG MINIO_CLIENT_IMAGE=quay.io/minio/aistor/mc:RELEASE.2026-09-19T15-24-59Z@sha256:23511e340cbabf07e6a8b53f9c434975708f22f3c047d6beeddc17bf1a5aed88 # https://download.docker.com/linux/static/stable/ -ARG DOCKER_VERSION=29.7.2 +ARG DOCKER_VERSION=29.9.2 # https://github.com/docker/compose/releases -ARG DOCKER_COMPOSE_VERSION=5.5.0 +ARG DOCKER_COMPOSE_VERSION=5.6.1 # https://github.com/docker/buildx/releases -ARG DOCKER_BUILDX_VERSION=0.36.1 +ARG DOCKER_BUILDX_VERSION=0.38.1 # https://github.com/buildpacks/pack/releases -ARG PACK_VERSION=0.38.2 +ARG PACK_VERSION=0.40.10 # https://github.com/railwayapp/nixpacks/releases ARG NIXPACKS_VERSION=1.41.0 # https://github.com/railwayapp/railpack/releases -ARG RAILPACK_VERSION=0.40.1 +ARG RAILPACK_VERSION=0.43.0 # https://github.com/jdx/mise/releases โ€” must match the version pinned at that railpack tag (https://raw.githubusercontent.com/railwayapp/railpack/v${RAILPACK_VERSION}/core/mise/version.txt) -ARG MISE_VERSION=2026.9.15 -FROM quay.io/minio/aistor/mc:RELEASE.2026-09-06T02-44-40Z AS minio-client +ARG MISE_VERSION=2026.10.14 + +FROM ${MINIO_CLIENT_IMAGE} AS minio-client FROM ${BASE_IMAGE} AS base -ARG TARGETPLATFORM +ARG TARGETARCH ARG DOCKER_VERSION ARG DOCKER_COMPOSE_VERSION ARG DOCKER_BUILDX_VERSION @@ -33,41 +36,56 @@ WORKDIR /artifacts ENV RAILPACK_VERSION=${RAILPACK_VERSION} RUN apk upgrade --no-cache && \ apk add --no-cache bash curl git git-lfs openssh-client pigz tar tini -RUN mkdir -p ~/.docker/cli-plugins +SHELL ["/bin/bash", "-euo", "pipefail", "-c"] -# Install mise (musl build) at the path railpack expects (/tmp/railpack/mise/mise-VERSION). -# Railpack skips its own mise download when this file exists. -RUN mkdir -p /tmp/railpack/mise && \ - if [[ ${TARGETPLATFORM} == 'linux/amd64' ]]; then \ - curl -sSL "https://github.com/jdx/mise/releases/download/v${MISE_VERSION}/mise-v${MISE_VERSION}-linux-x64-musl.tar.gz" | tar xz && \ - mv mise/bin/mise "/tmp/railpack/mise/mise-${MISE_VERSION}" && rm -rf mise; \ - elif [[ ${TARGETPLATFORM} == 'linux/arm64' ]]; then \ - curl -sSL "https://github.com/jdx/mise/releases/download/v${MISE_VERSION}/mise-v${MISE_VERSION}-linux-arm64-musl.tar.gz" | tar xz && \ - mv mise/bin/mise "/tmp/railpack/mise/mise-${MISE_VERSION}" && rm -rf mise; \ - fi +# SHA256 of every download, per architecture. After you change a version above, run scripts/helper-checksums to update them. +COPY <<'EOF' /tmp/SHA256SUMS +9ac225999fd5e863f7694c728ef23d480616b1e5639328fa309befd4243d69eb docker-amd64.tgz +873b3e0207857c3b2f28392fb922f3ea6efe37620c16d4d4aaf2fef409c90b73 docker-arm64.tgz +e25d5cd675092bb5d1d03632fdc46cdf3fede7b514dc5f023a105e00d3ce5ad4 docker-buildx-amd64 +3b83b3731916ba433feccaaa689fe5bcdc81c641d92c8e6c2ea865dfa141c97a docker-buildx-arm64 +47737f4236e6d3ccea7291995203f8f018058783b4c3a9469119811a0b5db8c0 docker-compose-amd64 +42cfa75f51ad0ccf290bad37899e0038a8c10a0069e85233ecc1d53e3ff1347d docker-compose-arm64 +b0e388c8014f9ac14a5ddf7a4042b0db28043f7f9ce485ffc2521bff02247f8e mise-amd64 +c71eec4d0015f6ced74ac9d5cd755b56674df04e899d0d1566d533c95511ae6c mise-arm64 +0f55de7874507b9cf7502113120bd96f2ab6979f78d10eaf2eb2ade9207b3af6 nixpacks-amd64.tgz +912bd02dd2bb6f9c3a9ed965fe8a68b4aa318dc7a2546e2eca6f2806a894ba39 nixpacks-arm64.tgz +687c3616e21824a449b56d4f989613ca68c921c6cce670b4de4459b025233806 pack-amd64.tgz +9a294abb0576fd95e19a009b3f9d5f5aa9fecdc2d7da7fa05e7e1e94f5085d77 pack-arm64.tgz +248a8f60f36854aabed6124113631078a802de3461660855fc4de9c26065c1fb railpack-amd64.tgz +732ece14eb4f4676a43ebba19431f954edf7b130acb76159ef0f2ff86ac42ac9 railpack-arm64.tgz +EOF -RUN if [[ ${TARGETPLATFORM} == 'linux/amd64' ]]; then \ - curl -sSL https://github.com/docker/buildx/releases/download/v${DOCKER_BUILDX_VERSION}/buildx-v${DOCKER_BUILDX_VERSION}.linux-amd64 -o ~/.docker/cli-plugins/docker-buildx && \ - curl -sSL https://github.com/docker/compose/releases/download/v${DOCKER_COMPOSE_VERSION}/docker-compose-linux-x86_64 -o ~/.docker/cli-plugins/docker-compose && \ - (curl -sSL https://download.docker.com/linux/static/stable/x86_64/docker-${DOCKER_VERSION}.tgz | tar -C /usr/bin/ --no-same-owner -xzv --strip-components=1 docker/docker) && \ - (curl -sSL https://github.com/buildpacks/pack/releases/download/v${PACK_VERSION}/pack-v${PACK_VERSION}-linux.tgz | tar -C /usr/local/bin/ --no-same-owner -xzv pack) && \ - curl -sSL https://nixpacks.com/install.sh | bash && \ - curl -sSL https://railpack.com/install.sh | RAILPACK_VERSION=${RAILPACK_VERSION} sh && \ - chmod +x ~/.docker/cli-plugins/docker-compose /usr/bin/docker /usr/local/bin/pack /root/.docker/cli-plugins/docker-buildx \ - ;fi - -RUN if [[ ${TARGETPLATFORM} == 'linux/arm64' ]]; then \ - curl -sSL https://github.com/docker/buildx/releases/download/v${DOCKER_BUILDX_VERSION}/buildx-v${DOCKER_BUILDX_VERSION}.linux-arm64 -o ~/.docker/cli-plugins/docker-buildx && \ - curl -sSL https://github.com/docker/compose/releases/download/v${DOCKER_COMPOSE_VERSION}/docker-compose-linux-aarch64 -o ~/.docker/cli-plugins/docker-compose && \ - (curl -sSL https://download.docker.com/linux/static/stable/aarch64/docker-${DOCKER_VERSION}.tgz | tar -C /usr/bin/ --no-same-owner -xzv --strip-components=1 docker/docker) && \ - (curl -sSL https://github.com/buildpacks/pack/releases/download/v${PACK_VERSION}/pack-v${PACK_VERSION}-linux-arm64.tgz | tar -C /usr/local/bin/ --no-same-owner -xzv pack) && \ - curl -sSL https://nixpacks.com/install.sh | bash && \ - curl -sSL https://railpack.com/install.sh | RAILPACK_VERSION=${RAILPACK_VERSION} sh && \ - chmod +x ~/.docker/cli-plugins/docker-compose /usr/bin/docker /usr/local/bin/pack /root/.docker/cli-plugins/docker-buildx \ - ;fi +RUN case "${TARGETARCH}" in \ + amd64) DOCKER_ARCH=x86_64; PACK_SUFFIX=""; NIXPACKS_ARCH=x86_64; RAILPACK_ARCH=x86_64; MISE_ARCH=x64 ;; \ + arm64) DOCKER_ARCH=aarch64; PACK_SUFFIX="-arm64"; NIXPACKS_ARCH=aarch64; RAILPACK_ARCH=arm64; MISE_ARCH=arm64 ;; \ + *) echo "Unsupported architecture: ${TARGETARCH}" >&2; exit 1 ;; \ + esac && \ + mkdir -p /tmp/downloads && cd /tmp/downloads && \ + download() { curl -fsSL --proto '=https' --tlsv1.2 --retry 3 -o "$1" "$2"; } && \ + download "docker-${TARGETARCH}.tgz" "https://download.docker.com/linux/static/stable/${DOCKER_ARCH}/docker-${DOCKER_VERSION}.tgz" && \ + download "docker-compose-${TARGETARCH}" "https://github.com/docker/compose/releases/download/v${DOCKER_COMPOSE_VERSION}/docker-compose-linux-${DOCKER_ARCH}" && \ + download "docker-buildx-${TARGETARCH}" "https://github.com/docker/buildx/releases/download/v${DOCKER_BUILDX_VERSION}/buildx-v${DOCKER_BUILDX_VERSION}.linux-${TARGETARCH}" && \ + download "pack-${TARGETARCH}.tgz" "https://github.com/buildpacks/pack/releases/download/v${PACK_VERSION}/pack-v${PACK_VERSION}-linux${PACK_SUFFIX}.tgz" && \ + download "nixpacks-${TARGETARCH}.tgz" "https://github.com/railwayapp/nixpacks/releases/download/v${NIXPACKS_VERSION}/nixpacks-v${NIXPACKS_VERSION}-${NIXPACKS_ARCH}-unknown-linux-musl.tar.gz" && \ + download "railpack-${TARGETARCH}.tgz" "https://github.com/railwayapp/railpack/releases/download/v${RAILPACK_VERSION}/railpack-v${RAILPACK_VERSION}-${RAILPACK_ARCH}-unknown-linux-musl.tar.gz" && \ + download "mise-${TARGETARCH}" "https://github.com/jdx/mise/releases/download/v${MISE_VERSION}/mise-v${MISE_VERSION}-linux-${MISE_ARCH}-musl" && \ + grep -- "-${TARGETARCH}" /tmp/SHA256SUMS | sha256sum -c - && \ + mkdir -p /root/.docker/cli-plugins /tmp/railpack/mise && \ + tar -xzf "docker-${TARGETARCH}.tgz" -C /usr/bin --no-same-owner --strip-components=1 docker/docker && \ + install -m 0755 "docker-compose-${TARGETARCH}" /root/.docker/cli-plugins/docker-compose && \ + install -m 0755 "docker-buildx-${TARGETARCH}" /root/.docker/cli-plugins/docker-buildx && \ + tar -xzf "pack-${TARGETARCH}.tgz" -C /usr/local/bin --no-same-owner pack && \ + tar -xzf "nixpacks-${TARGETARCH}.tgz" -C /usr/local/bin --no-same-owner nixpacks && \ + tar -xzf "railpack-${TARGETARCH}.tgz" -C /usr/local/bin --no-same-owner railpack && \ + # Railpack skips its own mise download when this file exists. + install -m 0755 "mise-${TARGETARCH}" "/tmp/railpack/mise/mise-${MISE_VERSION}" && \ + cd / && rm -rf /tmp/downloads /tmp/SHA256SUMS && \ + docker --version && docker compose version && docker buildx version && pack --version && \ + nixpacks --version && railpack --version COPY --from=minio-client /usr/bin/mc /usr/bin/mc -RUN chmod +x /usr/bin/mc +RUN chmod +x /usr/bin/mc && mc --version ENTRYPOINT ["/sbin/tini", "--"] CMD ["tail", "-f", "/dev/null"] diff --git a/docker/coolify-terminal/terminal-utils.js b/docker/coolify-terminal/terminal-utils.js index 6acb9645a6..8675e10516 100644 --- a/docker/coolify-terminal/terminal-utils.js +++ b/docker/coolify-terminal/terminal-utils.js @@ -123,10 +123,22 @@ export function createTerminalUpgradeHandler({ wss, authenticate, onRejected = ( const DEFAULT_TERMINAL_PATH = '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin'; +// Variables passed to the SSH process besides PATH. `cloudflared access ssh` reads the +// Access service token from them in the ProxyCommand of Cloudflare Tunnel servers. +const ALLOWED_TERMINAL_ENV = ['TUNNEL_SERVICE_TOKEN_ID', 'TUNNEL_SERVICE_TOKEN_SECRET']; + export function getTerminalProcessEnv(environment = process.env) { - return { + const env = { PATH: environment.PATH || DEFAULT_TERMINAL_PATH, }; + + for (const name of ALLOWED_TERMINAL_ENV) { + if (environment[name]) { + env[name] = environment[name]; + } + } + + return env; } export function getTerminalSessionTimeout() { diff --git a/docker/coolify-terminal/terminal-utils.test.js b/docker/coolify-terminal/terminal-utils.test.js index b00dfa7d9f..d0452e1f72 100644 --- a/docker/coolify-terminal/terminal-utils.test.js +++ b/docker/coolify-terminal/terminal-utils.test.js @@ -54,6 +54,29 @@ test('getTerminalProcessEnv uses the default PATH when PATH is empty', () => { }); }); +test('getTerminalProcessEnv passes the cloudflared Access service token to SSH proxy commands', () => { + assert.deepEqual(getTerminalProcessEnv({ + PATH: '/usr/bin:/bin', + TUNNEL_SERVICE_TOKEN_ID: 'token-id.access', + TUNNEL_SERVICE_TOKEN_SECRET: 'token-secret', + TUNNEL_SERVICE_URL: 'must-not-be-inherited', + APP_KEY: 'must-not-be-inherited', + }), { + PATH: '/usr/bin:/bin', + TUNNEL_SERVICE_TOKEN_ID: 'token-id.access', + TUNNEL_SERVICE_TOKEN_SECRET: 'token-secret', + }); +}); + +test('getTerminalProcessEnv omits empty cloudflared service token variables', () => { + assert.deepEqual(getTerminalProcessEnv({ + PATH: '/usr/bin:/bin', + TUNNEL_SERVICE_TOKEN_ID: '', + }), { + PATH: '/usr/bin:/bin', + }); +}); + test('extractTargetHost normalizes quoted IPv4 hosts from generated ssh commands', () => { const sshArgs = extractSshArgs( "timeout 3600 ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=ERROR -o ServerAliveInterval=20 -o ConnectTimeout=10 'root'@'10.0.0.5' 'bash -se' << \\\\$abc\necho hi\nabc" diff --git a/docker/development/Dockerfile b/docker/development/Dockerfile index a7ce68dffd..ce77ae4ea4 100644 --- a/docker/development/Dockerfile +++ b/docker/development/Dockerfile @@ -1,13 +1,13 @@ # Versions # https://hub.docker.com/r/serversideup/php/tags?name=8.4-fpm-nginx-alpine -ARG SERVERSIDEUP_PHP_VERSION=8.4-fpm-nginx-alpine-v5.0.2 +ARG SERVERSIDEUP_PHP_VERSION=8.4-fpm-nginx-alpine-v5.0.3 # https://github.com/cloudflare/cloudflared/releases ARG CLOUDFLARED_VERSION=2026.10.3 # https://www.postgresql.org/support/versioning/ # Note: We are using version 18 of the postgres client (while still using postgres 15 for the postgres server) as version 15 has been removed from Alpine 3.23+ https://pkgs.alpinelinux.org/packages?name=postgresql*-client&branch=v3.23&repo=&arch=x86_64&origin=&flagged=&maintainer= ARG POSTGRES_VERSION=18 # https://nginx.org/en/linux_packages.html -ARG NGINX_VERSION=1.31.2-r1 +ARG NGINX_VERSION=1.31.8-r1 # https://pecl.php.net/package/uv ARG PHP_UV_VERSION=0.3.0 diff --git a/docker/development/etc/nginx/site-opts.d/http.conf b/docker/development/etc/nginx/site-opts.d/http.conf index d7855ae800..f92cd0625c 100644 --- a/docker/development/etc/nginx/site-opts.d/http.conf +++ b/docker/development/etc/nginx/site-opts.d/http.conf @@ -29,6 +29,24 @@ location /healthcheck { fastcgi_pass 127.0.0.1:9000; } +# Proxy the realtime (Reverb) and terminal WebSockets that run in this container, so a +# reverse proxy that forwards only this port (for example Tailscale Serve) also serves them. +location /app/ { + proxy_pass http://127.0.0.1:6001; + proxy_http_version 1.1; + proxy_set_header Upgrade $http_upgrade; + proxy_set_header Connection "Upgrade"; + proxy_set_header Host $host; +} + +location = /terminal/ws { + proxy_pass http://127.0.0.1:6002; + proxy_http_version 1.1; + proxy_set_header Upgrade $http_upgrade; + proxy_set_header Connection "Upgrade"; + proxy_set_header Host $host; +} + # Have NGINX try searching for PHP files as well location / { try_files $uri $uri/ /index.php?$query_string; diff --git a/docker/development/etc/php/conf.d/zzz-custom-php.ini b/docker/development/etc/php/conf.d/zzz-custom-php.ini index e09eda5386..f008366e37 100644 --- a/docker/development/etc/php/conf.d/zzz-custom-php.ini +++ b/docker/development/etc/php/conf.d/zzz-custom-php.ini @@ -8,3 +8,4 @@ ignore_repeated_source = On upload_max_filesize = 256M post_max_size = 256M memory_limit = ${PHP_MEMORY_LIMIT:-512M} +session.upload_progress.enabled = Off diff --git a/docker/production/Dockerfile b/docker/production/Dockerfile index 108417d79e..476b35b704 100644 --- a/docker/production/Dockerfile +++ b/docker/production/Dockerfile @@ -1,13 +1,13 @@ # Versions # https://hub.docker.com/r/serversideup/php/tags?name=8.4-fpm-nginx-alpine -ARG SERVERSIDEUP_PHP_VERSION=8.4-fpm-nginx-alpine-v5.0.2 +ARG SERVERSIDEUP_PHP_VERSION=8.4-fpm-nginx-alpine-v5.0.3 # https://github.com/cloudflare/cloudflared/releases ARG CLOUDFLARED_VERSION=2026.10.3 # https://www.postgresql.org/support/versioning/ # Note: We are using version 18 of the postgres client (while still using postgres 15 for the postgres server) as version 15 has been removed from Alpine 3.23+ https://pkgs.alpinelinux.org/packages?name=postgresql*-client&branch=v3.23&repo=&arch=x86_64&origin=&flagged=&maintainer= ARG POSTGRES_VERSION=18 # https://nginx.org/en/linux_packages.html -ARG NGINX_VERSION=1.31.2-r1 +ARG NGINX_VERSION=1.31.8-r1 # https://nodejs.org/en/about/previous-releases ARG NODE_VERSION=24.22.1 # https://pecl.php.net/package/uv diff --git a/docker/production/etc/nginx/site-opts.d/http.conf b/docker/production/etc/nginx/site-opts.d/http.conf index d7855ae800..f92cd0625c 100644 --- a/docker/production/etc/nginx/site-opts.d/http.conf +++ b/docker/production/etc/nginx/site-opts.d/http.conf @@ -29,6 +29,24 @@ location /healthcheck { fastcgi_pass 127.0.0.1:9000; } +# Proxy the realtime (Reverb) and terminal WebSockets that run in this container, so a +# reverse proxy that forwards only this port (for example Tailscale Serve) also serves them. +location /app/ { + proxy_pass http://127.0.0.1:6001; + proxy_http_version 1.1; + proxy_set_header Upgrade $http_upgrade; + proxy_set_header Connection "Upgrade"; + proxy_set_header Host $host; +} + +location = /terminal/ws { + proxy_pass http://127.0.0.1:6002; + proxy_http_version 1.1; + proxy_set_header Upgrade $http_upgrade; + proxy_set_header Connection "Upgrade"; + proxy_set_header Host $host; +} + # Have NGINX try searching for PHP files as well location / { try_files $uri $uri/ /index.php?$query_string; diff --git a/docker/production/etc/php/conf.d/zzz-custom-php.ini b/docker/production/etc/php/conf.d/zzz-custom-php.ini index 4002b8f166..83ff824d79 100644 --- a/docker/production/etc/php/conf.d/zzz-custom-php.ini +++ b/docker/production/etc/php/conf.d/zzz-custom-php.ini @@ -8,3 +8,4 @@ ignore_repeated_source = On upload_max_filesize = 256M post_max_size = 256M memory_limit = ${PHP_MEMORY_LIMIT:-512M} +session.upload_progress.enabled = Off diff --git a/docs/server-transfer-full-coverage.md b/docs/server-transfer-full-coverage.md index 1574983fc3..3c39e0e15d 100644 --- a/docs/server-transfer-full-coverage.md +++ b/docs/server-transfer-full-coverage.md @@ -5,7 +5,9 @@ Use values that are NOT the default, so that a lost value is visible after the t Use the kit in `scripts/transfer-test/` (see its README): it starts the source (A) and the target (B) and seeds A with a basic set of deployed resources. -Legend: โœ… `scripts/transfer-test/seed.sh` or `seed-extras.php` already creates it ยท โž• add it ยท โš ๏ธ needs special infrastructure +Legend: โœ… `scripts/transfer-test/seed.sh`, `seed-extras.php`, or `seed-full.php` already creates it ยท โž• add it ยท โš ๏ธ needs special infrastructure + +`scripts/transfer-test/full-e2e.sh` creates the โœ… items, transfers the server, and compares every record and every container automatically. The models come from `ServerTransferCoverage::TRANSFERRED` (`app/Services/ServerTransfer/ServerTransferCoverage.php`). When you add a model or a setting to the transfer, add it to this file too. @@ -15,9 +17,9 @@ When you add a model or a setting to the transfer, add it to this file too. | โœ“ | What | Value to set | |---|---|---| | โœ… | Non-root SSH user | a dev VM with the `ubuntu-non-root` profile | -| โž• | Name, description | any text | +| โœ… | Name, description | any text | | โž• | Port | other than 22, if the VM allows it | -| โž• | Proxy: redirect | redirect enabled + target URL | +| โœ… | Proxy: redirect | redirect enabled + target URL | | โœ… | Proxy: custom error page | custom HTML (`proxy_error_page`) | | โž• | Server role / build server | mark a second server as a build server | | โš ๏ธ | Cloud provider link | Hetzner/Vultr/DigitalOcean/Hostinger server ID + cloud provider token (needs a real cloud server) | @@ -27,21 +29,21 @@ When you add a model or a setting to the transfer, add it to this file too. | โœ“ | What | Value to set | |---|---|---| -| โž• | Wildcard domain | `http://{ip}.sslip.io` style, not empty | -| โž• | Concurrent builds, dynamic timeout, deployment queue limit, connection timeout | 3, 1800, 5, 20 | -| โž• | Server timezone | `Europe/Budapest` | -| โž• | Docker cleanup | force cleanup on, frequency `0 4 * * *`, threshold 70, delete unused volumes on, delete unused networks on | -| โž• | Disk usage notification | threshold 70, check frequency `*/30 * * * *`, interval 12 h | -| โž• | Security > Patches | check frequency `daily`, automatic security updates on at `04:30` (the last check result is not moved; the target shows "never" until its first check) | -| โž• | Terminal | terminal access off | -| โž• | Image retention | "disable application image retention" on | -| โž• | Labels / Traefik | generate exact labels on, Traefik certificate resolver `custom` | -| โž• | Backup compression | CPU percentage 50 | -| โž• | Metrics / Sentinel | metrics on, refresh rate 10 s, history 14 days, push interval 30 s | -| โž• | Traffic analytics | on, top N 20, sample threshold, retention 1h=3 / 1d=60 days, IP mode `anonymized` | -| โž• | GeoIP | on, MaxMind license key, refresh 14 days | -| โž• | Log drains | New Relic (key + base URI), Highlight (project ID), Axiom (dataset + key), custom (config + parser), one of them enabled | -| โœ… | Cloudflare tunnel | tunnel on, tunnel ID, account ID, Cloudflare integration token selected | +| โœ… | Wildcard domain | `http://{ip}.sslip.io` style, not empty | +| โœ… | Concurrent builds, dynamic timeout, deployment queue limit, connection timeout | 3, 1800, 5, 20 | +| โœ… | Server timezone | `Europe/Budapest` | +| โœ… | Docker cleanup | force cleanup on, frequency `0 4 * * *`, threshold 70, delete unused volumes on, delete unused networks on | +| โœ… | Disk usage notification | threshold 70, check frequency `*/30 * * * *`, interval 12 h | +| โœ… | Security > Patches | check frequency `daily`, automatic security updates on at `04:30` (the last check result is not moved; the target shows "never" until its first check) | +| โœ… | Terminal | terminal access off | +| โœ… | Image retention | "disable application image retention" on | +| โœ… | Labels / Traefik | generate exact labels on, Traefik certificate resolver `custom` | +| โœ… | Backup compression | CPU percentage 50 | +| โœ… | Metrics / Sentinel | metrics on, refresh rate 10 s, history 14 days, push interval 30 s | +| โœ… | Traffic analytics | on, top N 20, sample threshold, retention 1h=3 / 1d=60 days, IP mode `anonymized` | +| โœ… | GeoIP | on, MaxMind license key, refresh 14 days | +| โœ… | Log drains | New Relic (key + base URI), Highlight (project ID), Axiom (dataset + key), custom (config + parser), one of them enabled | +| โœ… | Cloudflare tunnel | tunnel on, tunnel ID, account ID, Cloudflare integration token selected, and the DNS record of the SSH hostname. Expected: the record moves with the server; the source keeps it only for a take back (not owned) | | โž• | Jump server flag | on (on a server that is only a jump host) | ## 3. Server-level records @@ -49,9 +51,9 @@ When you add a model or a setting to the transfer, add it to this file too. | โœ“ | What | Notes | |---|---|---| | โœ… | Default destination | created automatically | -| โž• | Second standalone destination | extra Docker network, one resource on it | -| โž• | Server shared variables | 2 variables, one multiline, one literal, one with a comment | -| โž• | SSL certificates | CA certificate (created automatically) + a database with SSL on (creates a server certificate) | +| โœ… | Second standalone destination | extra Docker network, one resource on it | +| โœ… | Server shared variables | 2 variables, one multiline, one literal, one with a comment | +| โœ… | SSL certificates | CA certificate (created automatically) + a database with SSL on (creates a server certificate) | | โœ… | GitHub runner config | labels, max runners 2, docker mode `dind`, image, CPU/memory limits, 3 timeouts, PR jobs off, dedicated on, uses the team GitHub App (see 4). Expected: on the target it is **disabled** and the checklist says to enable it | ## 4. Team credentials the server uses @@ -59,10 +61,10 @@ When you add a model or a setting to the transfer, add it to this file too. | โœ“ | What | Notes | |---|---|---| | โœ… | Server private key | | -| โž• | Deploy key | a second private key used by an application (git over SSH) | +| โœ… | Deploy key | a second private key used by an application (git over SSH) | | โœ… | GitHub App (team, not system-wide) | with runner fields (`organization_self_hosted_runners`, `actions`, `webhook_events`, `runner_group_id`). Expected: checklist step for Webhook/Setup/Callback URL | -| โž• | GitLab App (team) | used by one application. Expected: checklist step for the webhook URL | -| โž• | S3 storage | the dev MinIO service (the VM must reach it), used by a database backup and a volume backup | +| โœ… | GitLab App (team) | used by one application. Expected: checklist step for the webhook URL | +| โœ… | S3 storage | the dev MinIO service (the VM must reach it), used by a database backup and a volume backup | | โœ… | Cloudflare integration token | for DNS zones and the tunnel | | โœ… | Secret manager token | Infisical, Doppler, or Vault | | โš ๏ธ | Cloud provider token | only with a real cloud server | @@ -72,7 +74,7 @@ When you add a model or a setting to the transfer, add it to this file too. | โœ“ | What | Notes | |---|---|---| | โœ… | 2 projects, 2 environments | for example production and staging | -| โœ…/โž• | Project / environment shared variables | the seed tries to create them; check that they exist | +| โœ… | Project / environment shared variables | the seed tries to create them; check that they exist | | โœ… | Team shared variable | `TEAM_KEY`, used as `{{team.TEAM_KEY}}` in an application variable. Expected: created on the target | | โœ… | Project description + icon (JPEG) | Expected: the icon is on the target | | โœ… | Canvas positions | drag the nodes on the environment canvas | @@ -86,22 +88,22 @@ When you add a model or a setting to the transfer, add it to this file too. | โœ… | Docker image app (whoami) | | | โœ… | Dockerfile app | | | โž• | Nixpacks app, public GitHub repo | Expected: checklist step for manual webhooks | -| โž• | Static app (`is_static`, `is_spa`) | | +| โœ… | Static app (`is_static`, `is_spa`) | | | โž• | Docker Compose app | raw compose on, one variable with an empty value (`preserve_compose_null`) | | โž• | App with GitHub App source, app with GitLab App source, app with deploy key | | -| โž• | Domains | 2 domains, noindex domains, domain port overrides, preview URL template | -| โž• | Health check | custom path, port, method, return code, interval, retries, start period | -| โž• | Limits | memory, swap, swappiness, reservation, CPUs, cpuset, CPU shares | +| โœ… | Domains | 2 domains, noindex domains, domain port overrides, preview URL template | +| โœ… | Health check | custom path, port, method, return code, interval, retries, start period | +| โœ… | Limits | memory, swap, swappiness, reservation, CPUs, cpuset, CPU shares | | โž• | Advanced | custom labels, custom docker run options, pre/post deployment command + container, manual webhook secrets | | โœ… | Maintenance | maintenance mode on + custom maintenance page | -| โž• | All application settings that are not the default | git submodules, LFS, shallow clone, auto deploy off, force HTTPS, debug, preview deployments on, PR deployments public, log drain, GPU (driver, count, device IDs, options), include timestamps, swarm-only worker nodes, build server, consistent container name, gzip off, strip prefix off, connect to Docker network, custom internal name, container label escape off, env sorting, read-only labels, preserve repository, disable build cache, use build secrets, stop grace period 30, inject build args off, include source commit, Docker images to keep 5, custom container name prefix, redact env values in logs, Coolify default variables off, Traefik certificate resolver | -| โž• | Environment variables | normal, multiline, literal, shown once, build time only, runtime only, required, with comment, preview-only, and references to `{{team.*}}`, `{{project.*}}`, `{{environment.*}}`, `{{server.*}}` | -| โœ…/โž• | Persistent volume | the seed has one; add a host-path bind mount and preview suffix on | -| โž• | File mount + directory mount | content, chown, chmod | -| โž• | Volume backup | on the persistent volume: S3 on, stop during backup, all retention values, timeout, missing-backup days | -| โœ…/โž• | Scheduled tasks | the seed has one; add a disabled one with container + timeout | +| โœ… | All application settings that are not the default | git submodules, LFS, shallow clone, auto deploy off, force HTTPS, debug, preview deployments on, PR deployments public, log drain, GPU (driver, count, device IDs, options), include timestamps, swarm-only worker nodes, build server, consistent container name, gzip off, strip prefix off, connect to Docker network, custom internal name, container label escape off, env sorting, read-only labels, preserve repository, disable build cache, use build secrets, stop grace period 30, rolling update drain time 12, inject build args off, include source commit, Docker images to keep 5, custom container name prefix, redact env values in logs, Coolify default variables off, Traefik certificate resolver | +| โœ… | Environment variables | normal, multiline, literal, shown once, build time only, runtime only, required, with comment, preview-only, and references to `{{team.*}}`, `{{project.*}}`, `{{environment.*}}`, `{{server.*}}` | +| โœ… | Persistent volume | the seed has one; add a host-path bind mount and preview suffix on | +| โœ… | File mount + directory mount | content, chown, chmod | +| โœ… | Volume backup | on the persistent volume: S3 on, stop during backup, all retention values, timeout, missing-backup days | +| โœ… | Scheduled tasks | the seed has one; add a disabled one with container + timeout | | โœ… | Tag | | -| โž• | Preview deployment | PR preview with domain port overrides + max restart count (needs the GitHub App, or create it in the database) | +| โœ… | Preview deployment | PR preview with domain port overrides + max restart count (needs the GitHub App, or create it in the database) | | โœ… | Secret manager link | the app loads variables from the secret manager token | | โœ… | Managed DNS record | owned record in the Cloudflare zone for the app domain. Expected: the target takes ownership (provider comment changes) | @@ -110,12 +112,12 @@ When you add a model or a setting to the transfer, add it to this file too. | โœ“ | Type | Extra settings | |---|---|---| | โœ… | PostgreSQL | + init script, `postgres_conf`, host auth method, SSL on + mode | -| โž• | MySQL | + `mysql_conf`, SSL | +| โœ… | MySQL | + `mysql_conf`, SSL | | โœ… | MariaDB | + `mariadb_conf`, SSL | -| โž• | MongoDB | + `mongo_conf`, SSL | +| โœ… | MongoDB | + `mongo_conf`, SSL | | โœ… | Redis | + `redis_conf`, SSL | -| โž• | KeyDB, Dragonfly, ClickHouse | defaults are enough, plus the common settings below | -| โž• | SQLite | database files list, volume link | +| โœ… | KeyDB, Dragonfly, ClickHouse | defaults are enough, plus the common settings below | +| โœ… | SQLite | database files list, volume link | Common settings on at least one database: custom image, public port + public on + public port timeout, limits, custom docker run options, health check values, log drain, include timestamps, redact env values, Coolify default variables off, environment variables, persistent volume + volume backup, tags, secret manager link (if the type supports it). @@ -126,15 +128,15 @@ Database backups (โœ… one exists): add S3 on, dump all, databases list, local an | โœ“ | What | Notes | |---|---|---| | โœ… | One-click service (uptime-kuma) | | -| โž• | Custom compose service with an app and a database part | | -| โž• | Service app settings | domain, gzip off, strip prefix off, force HTTPS, exclude from status, redirect, port overrides, log drain, custom image, Traefik resolver | -| โž• | Service database settings | public port + public on, custom type | -| โž• | Service level | connect to network, label escape off, redact env values, Coolify default variables off, maintenance mode + page, environment variables, scheduled task, tags, volume backup on a service volume | +| โœ… | Custom compose service with an app and a database part | | +| โœ… | Service app settings | domain, gzip off, strip prefix off, force HTTPS, exclude from status, redirect, port overrides, log drain, custom image, Traefik resolver | +| โœ… | Service database settings | public port + public on, custom type | +| โœ… | Service level | connect to network, label escape off, redact env values, Coolify default variables off, maintenance mode + page, environment variables, scheduled task, tags, volume backup on a service volume | ## 9. After the transfer, check - Transfer log: every check `OK`, the `NOTE:` line for the shared project. - "After the transfer" checklist: GitHub App URLs, GitLab webhook, manual webhooks, GitHub runners. All URLs use the public URL of the target. - Target: every item above has the same value. Runner config disabled. DNS records owned by the target. -- Source: the server is an archive, the runner config is disabled, and no DNS records are tracked. +- Source: the server is an archive, the runner config is disabled, and no DNS records are tracked. The archive refuses actions on the host, the cloud server (delete, power on), the Cloudflare tunnel, and DNS records. - Delete the archive on the source: the containers, DNS records, and GitHub runners are not changed. diff --git a/openapi.json b/openapi.json index 39d4602289..7b60e74a4b 100644 --- a/openapi.json +++ b/openapi.json @@ -818,6 +818,11 @@ "type": "string", "description": "The Dockerfile location in the repository." }, + "dockerfile_target_build": { + "type": "string", + "nullable": true, + "description": "The Docker build target (stage) in the Dockerfile." + }, "docker_compose_location": { "type": "string", "description": "The Docker Compose location." @@ -903,6 +908,11 @@ "type": "boolean", "description": "Redact all environment variable values in logs. When false, only locked (shown once) values are redacted. Requires team admin permissions." }, + "include_coolify_default_variables": { + "type": "boolean", + "default": true, + "description": "Add the predefined COOLIFY_* variables (COOLIFY_URL, COOLIFY_FQDN, ...) to the environment. Requires team admin permissions. Redeploy to apply it." + }, "is_pr_deployments_public_enabled": { "type": "boolean", "description": "Make pull request deployments public." @@ -914,6 +924,12 @@ "maximum": 3600, "description": "Container stop grace period in seconds." }, + "rolling_update_drain_seconds": { + "type": "integer", + "minimum": 0, + "maximum": 300, + "description": "Seconds to wait after the new container is healthy before old containers stop during a rolling update. 0 turns the wait off. Default: 5." + }, "docker_images_to_keep": { "type": "integer", "minimum": 0, @@ -937,6 +953,11 @@ "nullable": true, "description": "Traefik certificate resolver for this application. Null uses the server setting, \"none\" uses no resolver." }, + "is_container_label_readonly_enabled": { + "type": "boolean", + "default": true, + "description": "Let Coolify manage the container labels. True regenerates custom_labels from domains, redirect, and proxy settings. False keeps custom_labels as you set them: domains, redirect, and basic auth changes then do not change the labels." + }, "is_raw_compose_deployment_enabled": { "type": "boolean", "description": "Deploy the raw Docker Compose definition." @@ -1455,6 +1476,11 @@ "type": "string", "description": "The Dockerfile location in the repository" }, + "dockerfile_target_build": { + "type": "string", + "nullable": true, + "description": "The Docker build target (stage) in the Dockerfile." + }, "docker_compose_location": { "type": "string", "description": "The Docker Compose location." @@ -1540,6 +1566,11 @@ "type": "boolean", "description": "Redact all environment variable values in logs. When false, only locked (shown once) values are redacted. Requires team admin permissions." }, + "include_coolify_default_variables": { + "type": "boolean", + "default": true, + "description": "Add the predefined COOLIFY_* variables (COOLIFY_URL, COOLIFY_FQDN, ...) to the environment. Requires team admin permissions. Redeploy to apply it." + }, "is_pr_deployments_public_enabled": { "type": "boolean", "description": "Make pull request deployments public." @@ -1551,6 +1582,12 @@ "maximum": 3600, "description": "Container stop grace period in seconds." }, + "rolling_update_drain_seconds": { + "type": "integer", + "minimum": 0, + "maximum": 300, + "description": "Seconds to wait after the new container is healthy before old containers stop during a rolling update. 0 turns the wait off. Default: 5." + }, "docker_images_to_keep": { "type": "integer", "minimum": 0, @@ -1574,6 +1611,11 @@ "nullable": true, "description": "Traefik certificate resolver for this application. Null uses the server setting, \"none\" uses no resolver." }, + "is_container_label_readonly_enabled": { + "type": "boolean", + "default": true, + "description": "Let Coolify manage the container labels. True regenerates custom_labels from domains, redirect, and proxy settings. False keeps custom_labels as you set them: domains, redirect, and basic auth changes then do not change the labels." + }, "is_raw_compose_deployment_enabled": { "type": "boolean", "description": "Deploy the raw Docker Compose definition." @@ -2092,6 +2134,11 @@ "type": "string", "description": "The Dockerfile location in the repository." }, + "dockerfile_target_build": { + "type": "string", + "nullable": true, + "description": "The Docker build target (stage) in the Dockerfile." + }, "docker_compose_location": { "type": "string", "description": "The Docker Compose location." @@ -2177,6 +2224,11 @@ "type": "boolean", "description": "Redact all environment variable values in logs. When false, only locked (shown once) values are redacted. Requires team admin permissions." }, + "include_coolify_default_variables": { + "type": "boolean", + "default": true, + "description": "Add the predefined COOLIFY_* variables (COOLIFY_URL, COOLIFY_FQDN, ...) to the environment. Requires team admin permissions. Redeploy to apply it." + }, "is_pr_deployments_public_enabled": { "type": "boolean", "description": "Make pull request deployments public." @@ -2188,6 +2240,12 @@ "maximum": 3600, "description": "Container stop grace period in seconds." }, + "rolling_update_drain_seconds": { + "type": "integer", + "minimum": 0, + "maximum": 300, + "description": "Seconds to wait after the new container is healthy before old containers stop during a rolling update. 0 turns the wait off. Default: 5." + }, "docker_images_to_keep": { "type": "integer", "minimum": 0, @@ -2211,6 +2269,11 @@ "nullable": true, "description": "Traefik certificate resolver for this application. Null uses the server setting, \"none\" uses no resolver." }, + "is_container_label_readonly_enabled": { + "type": "boolean", + "default": true, + "description": "Let Coolify manage the container labels. True regenerates custom_labels from domains, redirect, and proxy settings. False keeps custom_labels as you set them: domains, redirect, and basic auth changes then do not change the labels." + }, "is_raw_compose_deployment_enabled": { "type": "boolean", "description": "Deploy the raw Docker Compose definition." @@ -2709,6 +2772,11 @@ "type": "boolean", "description": "Redact all environment variable values in logs. When false, only locked (shown once) values are redacted. Requires team admin permissions." }, + "include_coolify_default_variables": { + "type": "boolean", + "default": true, + "description": "Add the predefined COOLIFY_* variables (COOLIFY_URL, COOLIFY_FQDN, ...) to the environment. Requires team admin permissions. Redeploy to apply it." + }, "is_pr_deployments_public_enabled": { "type": "boolean", "description": "Make pull request deployments public." @@ -2720,6 +2788,12 @@ "maximum": 3600, "description": "Container stop grace period in seconds." }, + "rolling_update_drain_seconds": { + "type": "integer", + "minimum": 0, + "maximum": 300, + "description": "Seconds to wait after the new container is healthy before old containers stop during a rolling update. 0 turns the wait off. Default: 5." + }, "docker_images_to_keep": { "type": "integer", "minimum": 0, @@ -2743,6 +2817,11 @@ "nullable": true, "description": "Traefik certificate resolver for this application. Null uses the server setting, \"none\" uses no resolver." }, + "is_container_label_readonly_enabled": { + "type": "boolean", + "default": true, + "description": "Let Coolify manage the container labels. True regenerates custom_labels from domains, redirect, and proxy settings. False keeps custom_labels as you set them: domains, redirect, and basic auth changes then do not change the labels." + }, "is_raw_compose_deployment_enabled": { "type": "boolean", "description": "Deploy the raw Docker Compose definition." @@ -3221,6 +3300,11 @@ "type": "boolean", "description": "Redact all environment variable values in logs. When false, only locked (shown once) values are redacted. Requires team admin permissions." }, + "include_coolify_default_variables": { + "type": "boolean", + "default": true, + "description": "Add the predefined COOLIFY_* variables (COOLIFY_URL, COOLIFY_FQDN, ...) to the environment. Requires team admin permissions. Redeploy to apply it." + }, "is_pr_deployments_public_enabled": { "type": "boolean", "description": "Make pull request deployments public." @@ -3232,6 +3316,12 @@ "maximum": 3600, "description": "Container stop grace period in seconds." }, + "rolling_update_drain_seconds": { + "type": "integer", + "minimum": 0, + "maximum": 300, + "description": "Seconds to wait after the new container is healthy before old containers stop during a rolling update. 0 turns the wait off. Default: 5." + }, "docker_images_to_keep": { "type": "integer", "minimum": 0, @@ -3255,6 +3345,11 @@ "nullable": true, "description": "Traefik certificate resolver for this application. Null uses the server setting, \"none\" uses no resolver." }, + "is_container_label_readonly_enabled": { + "type": "boolean", + "default": true, + "description": "Let Coolify manage the container labels. True regenerates custom_labels from domains, redirect, and proxy settings. False keeps custom_labels as you set them: domains, redirect, and basic auth changes then do not change the labels." + }, "is_raw_compose_deployment_enabled": { "type": "boolean", "description": "Deploy the raw Docker Compose definition." @@ -3891,6 +3986,11 @@ "type": "string", "description": "The Dockerfile location in the repository." }, + "dockerfile_target_build": { + "type": "string", + "nullable": true, + "description": "The Docker build target (stage) in the Dockerfile." + }, "docker_compose_location": { "type": "string", "description": "The Docker Compose location." @@ -4001,6 +4101,11 @@ "type": "boolean", "description": "Redact all environment variable values in logs. When false, only locked (shown once) values are redacted. Requires team admin permissions." }, + "include_coolify_default_variables": { + "type": "boolean", + "default": true, + "description": "Add the predefined COOLIFY_* variables (COOLIFY_URL, COOLIFY_FQDN, ...) to the environment. Requires team admin permissions. Redeploy to apply it." + }, "is_pr_deployments_public_enabled": { "type": "boolean", "description": "Make pull request deployments public." @@ -4012,6 +4117,12 @@ "maximum": 3600, "description": "Container stop grace period in seconds." }, + "rolling_update_drain_seconds": { + "type": "integer", + "minimum": 0, + "maximum": 300, + "description": "Seconds to wait after the new container is healthy before old containers stop during a rolling update. 0 turns the wait off. Default: 5." + }, "docker_images_to_keep": { "type": "integer", "minimum": 0, @@ -4035,6 +4146,11 @@ "nullable": true, "description": "Traefik certificate resolver for this application. Null uses the server setting, \"none\" uses no resolver." }, + "is_container_label_readonly_enabled": { + "type": "boolean", + "default": true, + "description": "Let Coolify manage the container labels. True regenerates custom_labels from domains, redirect, and proxy settings. False keeps custom_labels as you set them: domains, redirect, and basic auth changes then do not change the labels." + }, "is_raw_compose_deployment_enabled": { "type": "boolean", "description": "Deploy the raw Docker Compose definition." @@ -7045,6 +7161,11 @@ "description": "Whether to save backups to S3", "default": false }, + "disable_local_backup": { + "type": "boolean", + "description": "Delete the local backup file after the upload to S3. Only allowed when save_s3 is true.", + "default": false + }, "s3_storage_uuid": { "type": "string", "description": "Primary S3 storage UUID. Sent without s3_storage_uuids, it selects this storage as the only destination. Either field is required if save_s3 is true." @@ -7522,6 +7643,14 @@ "redact_all_env_values_in_logs": { "type": "boolean", "description": "Redact all environment variable values in logs. When false, only locked (shown once) values, database passwords, and remote secrets are redacted. Requires team admin permissions." + }, + "include_coolify_default_variables": { + "type": "boolean", + "description": "Add the predefined COOLIFY_* variables (COOLIFY_URL, COOLIFY_FQDN, ...) to the environment. Requires team admin permissions. Restart the database to apply it." + }, + "is_log_drain_enabled": { + "type": "boolean", + "description": "Send the database logs to the log drain of the server. Log drain must be enabled on the server. Restart the database to apply it." } }, "type": "object" @@ -7672,6 +7801,10 @@ "type": "boolean", "description": "Whether data is saved in s3 or not" }, + "disable_local_backup": { + "type": "boolean", + "description": "Delete the local backup file after the upload to S3. Only allowed when save_s3 is true. Turning save_s3 off also turns this off." + }, "s3_storage_uuid": { "type": "string", "description": "Primary S3 storage UUID. Sent without s3_storage_uuids, it selects this storage as the only destination; when it is already a destination, it becomes the primary destination and the other destinations stay." @@ -17219,6 +17352,12 @@ ], "example": "https:\/\/example.com" }, + "proxy_error_page": { + "type": [ + "string", + "null" + ] + }, "generate_exact_labels": { "type": "boolean", "example": false @@ -17294,6 +17433,13 @@ "null" ] }, + "proxy_error_page": { + "description": "Custom HTML error page that the proxy shows for unknown hosts (when redirect_url is empty) and for resources in maintenance mode. Null restores the default page.", + "type": [ + "string", + "null" + ] + }, "generate_exact_labels": { "type": "boolean" }, @@ -17348,6 +17494,12 @@ "null" ] }, + "proxy_error_page": { + "type": [ + "string", + "null" + ] + }, "generate_exact_labels": { "type": "boolean" }, @@ -17462,6 +17614,12 @@ "null" ] }, + "proxy_error_page": { + "type": [ + "string", + "null" + ] + }, "generate_exact_labels": { "type": "boolean" }, @@ -18468,6 +18626,25 @@ "connection_timeout": { "type": "integer", "description": "SSH connection timeout in seconds (1-300). Default: 10." + }, + "wildcard_domain": { + "type": "string", + "nullable": true, + "description": "Wildcard domain (URL) used to generate domains for new resources on this server. Null removes it." + }, + "server_timezone": { + "type": "string", + "description": "Timezone of the server, for example Europe\/Berlin. Scheduled tasks and backups use it." + }, + "backup_compression_cpu_percentage": { + "type": "integer", + "enum": [ + 25, + 50, + 75, + 100 + ], + "description": "Maximum CPU percentage that backup compression can use. Default: 25." } }, "type": "object" @@ -18933,6 +19110,15 @@ "null" ] }, + "redirect": { + "description": "How to handle www and non-www domains. \"www\" redirects non-www to www, \"non-www\" redirects www to non-www. Both need a www and a non-www domain in url.", + "type": "string", + "enum": [ + "both", + "www", + "non-www" + ] + }, "max_restart_count": { "description": "Maximum Docker restart count before Coolify stops the container. Set to 0 to disable the limit.", "type": [ @@ -20356,16 +20542,6 @@ "type": "boolean", "default": true } - }, - { - "name": "delete_from_coolify_only", - "in": "query", - "description": "Remove only Coolify metadata without deleting Docker resources.", - "required": false, - "schema": { - "type": "boolean", - "default": false - } } ], "responses": { @@ -20396,6 +20572,9 @@ }, "403": { "description": "The token needs the write and deploy permissions." + }, + "422": { + "$ref": "#\/components\/responses\/422" } }, "security": [ @@ -20481,6 +20660,11 @@ "type": "boolean", "default": true, "description": "Redact all environment variable values in logs. When false, only locked (shown once) values are redacted. Requires team admin permissions." + }, + "include_coolify_default_variables": { + "type": "boolean", + "default": true, + "description": "Add the predefined COOLIFY_* variables (COOLIFY_URL, COOLIFY_FQDN, ...) to the environment. Requires team admin permissions. Redeploy to apply it." } }, "type": "object" @@ -24981,6 +25165,10 @@ "stop_grace_period": { "type": "integer", "nullable": true + }, + "rolling_update_drain_seconds": { + "type": "integer", + "description": "Seconds to wait after the new container is healthy before old containers stop during a rolling update. 0 turns the wait off." } }, "type": "object" diff --git a/openapi.yaml b/openapi.yaml index 9b18ce4c26..c870fea1e8 100644 --- a/openapi.yaml +++ b/openapi.yaml @@ -537,6 +537,10 @@ paths: dockerfile_location: type: string description: 'The Dockerfile location in the repository.' + dockerfile_target_build: + type: string + nullable: true + description: 'The Docker build target (stage) in the Dockerfile.' docker_compose_location: type: string description: 'The Docker Compose location.' @@ -585,6 +589,10 @@ paths: redact_all_env_values_in_logs: type: boolean description: 'Redact all environment variable values in logs. When false, only locked (shown once) values are redacted. Requires team admin permissions.' + include_coolify_default_variables: + type: boolean + default: true + description: 'Add the predefined COOLIFY_* variables (COOLIFY_URL, COOLIFY_FQDN, ...) to the environment. Requires team admin permissions. Redeploy to apply it.' is_pr_deployments_public_enabled: type: boolean description: 'Make pull request deployments public.' @@ -594,6 +602,11 @@ paths: minimum: 1 maximum: 3600 description: 'Container stop grace period in seconds.' + rolling_update_drain_seconds: + type: integer + minimum: 0 + maximum: 300 + description: 'Seconds to wait after the new container is healthy before old containers stop during a rolling update. 0 turns the wait off. Default: 5.' docker_images_to_keep: type: integer minimum: 0 @@ -612,6 +625,10 @@ paths: type: string nullable: true description: 'Traefik certificate resolver for this application. Null uses the server setting, "none" uses no resolver.' + is_container_label_readonly_enabled: + type: boolean + default: true + description: 'Let Coolify manage the container labels. True regenerates custom_labels from domains, redirect, and proxy settings. False keeps custom_labels as you set them: domains, redirect, and basic auth changes then do not change the labels.' is_raw_compose_deployment_enabled: type: boolean description: 'Deploy the raw Docker Compose definition.' @@ -958,6 +975,10 @@ paths: dockerfile_location: type: string description: 'The Dockerfile location in the repository' + dockerfile_target_build: + type: string + nullable: true + description: 'The Docker build target (stage) in the Dockerfile.' docker_compose_location: type: string description: 'The Docker Compose location.' @@ -1006,6 +1027,10 @@ paths: redact_all_env_values_in_logs: type: boolean description: 'Redact all environment variable values in logs. When false, only locked (shown once) values are redacted. Requires team admin permissions.' + include_coolify_default_variables: + type: boolean + default: true + description: 'Add the predefined COOLIFY_* variables (COOLIFY_URL, COOLIFY_FQDN, ...) to the environment. Requires team admin permissions. Redeploy to apply it.' is_pr_deployments_public_enabled: type: boolean description: 'Make pull request deployments public.' @@ -1015,6 +1040,11 @@ paths: minimum: 1 maximum: 3600 description: 'Container stop grace period in seconds.' + rolling_update_drain_seconds: + type: integer + minimum: 0 + maximum: 300 + description: 'Seconds to wait after the new container is healthy before old containers stop during a rolling update. 0 turns the wait off. Default: 5.' docker_images_to_keep: type: integer minimum: 0 @@ -1033,6 +1063,10 @@ paths: type: string nullable: true description: 'Traefik certificate resolver for this application. Null uses the server setting, "none" uses no resolver.' + is_container_label_readonly_enabled: + type: boolean + default: true + description: 'Let Coolify manage the container labels. True regenerates custom_labels from domains, redirect, and proxy settings. False keeps custom_labels as you set them: domains, redirect, and basic auth changes then do not change the labels.' is_raw_compose_deployment_enabled: type: boolean description: 'Deploy the raw Docker Compose definition.' @@ -1379,6 +1413,10 @@ paths: dockerfile_location: type: string description: 'The Dockerfile location in the repository.' + dockerfile_target_build: + type: string + nullable: true + description: 'The Docker build target (stage) in the Dockerfile.' docker_compose_location: type: string description: 'The Docker Compose location.' @@ -1427,6 +1465,10 @@ paths: redact_all_env_values_in_logs: type: boolean description: 'Redact all environment variable values in logs. When false, only locked (shown once) values are redacted. Requires team admin permissions.' + include_coolify_default_variables: + type: boolean + default: true + description: 'Add the predefined COOLIFY_* variables (COOLIFY_URL, COOLIFY_FQDN, ...) to the environment. Requires team admin permissions. Redeploy to apply it.' is_pr_deployments_public_enabled: type: boolean description: 'Make pull request deployments public.' @@ -1436,6 +1478,11 @@ paths: minimum: 1 maximum: 3600 description: 'Container stop grace period in seconds.' + rolling_update_drain_seconds: + type: integer + minimum: 0 + maximum: 300 + description: 'Seconds to wait after the new container is healthy before old containers stop during a rolling update. 0 turns the wait off. Default: 5.' docker_images_to_keep: type: integer minimum: 0 @@ -1454,6 +1501,10 @@ paths: type: string nullable: true description: 'Traefik certificate resolver for this application. Null uses the server setting, "none" uses no resolver.' + is_container_label_readonly_enabled: + type: boolean + default: true + description: 'Let Coolify manage the container labels. True regenerates custom_labels from domains, redirect, and proxy settings. False keeps custom_labels as you set them: domains, redirect, and basic auth changes then do not change the labels.' is_raw_compose_deployment_enabled: type: boolean description: 'Deploy the raw Docker Compose definition.' @@ -1789,6 +1840,10 @@ paths: redact_all_env_values_in_logs: type: boolean description: 'Redact all environment variable values in logs. When false, only locked (shown once) values are redacted. Requires team admin permissions.' + include_coolify_default_variables: + type: boolean + default: true + description: 'Add the predefined COOLIFY_* variables (COOLIFY_URL, COOLIFY_FQDN, ...) to the environment. Requires team admin permissions. Redeploy to apply it.' is_pr_deployments_public_enabled: type: boolean description: 'Make pull request deployments public.' @@ -1798,6 +1853,11 @@ paths: minimum: 1 maximum: 3600 description: 'Container stop grace period in seconds.' + rolling_update_drain_seconds: + type: integer + minimum: 0 + maximum: 300 + description: 'Seconds to wait after the new container is healthy before old containers stop during a rolling update. 0 turns the wait off. Default: 5.' docker_images_to_keep: type: integer minimum: 0 @@ -1816,6 +1876,10 @@ paths: type: string nullable: true description: 'Traefik certificate resolver for this application. Null uses the server setting, "none" uses no resolver.' + is_container_label_readonly_enabled: + type: boolean + default: true + description: 'Let Coolify manage the container labels. True regenerates custom_labels from domains, redirect, and proxy settings. False keeps custom_labels as you set them: domains, redirect, and basic auth changes then do not change the labels.' is_raw_compose_deployment_enabled: type: boolean description: 'Deploy the raw Docker Compose definition.' @@ -2137,6 +2201,10 @@ paths: redact_all_env_values_in_logs: type: boolean description: 'Redact all environment variable values in logs. When false, only locked (shown once) values are redacted. Requires team admin permissions.' + include_coolify_default_variables: + type: boolean + default: true + description: 'Add the predefined COOLIFY_* variables (COOLIFY_URL, COOLIFY_FQDN, ...) to the environment. Requires team admin permissions. Redeploy to apply it.' is_pr_deployments_public_enabled: type: boolean description: 'Make pull request deployments public.' @@ -2146,6 +2214,11 @@ paths: minimum: 1 maximum: 3600 description: 'Container stop grace period in seconds.' + rolling_update_drain_seconds: + type: integer + minimum: 0 + maximum: 300 + description: 'Seconds to wait after the new container is healthy before old containers stop during a rolling update. 0 turns the wait off. Default: 5.' docker_images_to_keep: type: integer minimum: 0 @@ -2164,6 +2237,10 @@ paths: type: string nullable: true description: 'Traefik certificate resolver for this application. Null uses the server setting, "none" uses no resolver.' + is_container_label_readonly_enabled: + type: boolean + default: true + description: 'Let Coolify manage the container labels. True regenerates custom_labels from domains, redirect, and proxy settings. False keeps custom_labels as you set them: domains, redirect, and basic auth changes then do not change the labels.' is_raw_compose_deployment_enabled: type: boolean description: 'Deploy the raw Docker Compose definition.' @@ -2590,6 +2667,10 @@ paths: dockerfile_location: type: string description: 'The Dockerfile location in the repository.' + dockerfile_target_build: + type: string + nullable: true + description: 'The Docker build target (stage) in the Dockerfile.' docker_compose_location: type: string description: 'The Docker Compose location.' @@ -2656,6 +2737,10 @@ paths: redact_all_env_values_in_logs: type: boolean description: 'Redact all environment variable values in logs. When false, only locked (shown once) values are redacted. Requires team admin permissions.' + include_coolify_default_variables: + type: boolean + default: true + description: 'Add the predefined COOLIFY_* variables (COOLIFY_URL, COOLIFY_FQDN, ...) to the environment. Requires team admin permissions. Redeploy to apply it.' is_pr_deployments_public_enabled: type: boolean description: 'Make pull request deployments public.' @@ -2665,6 +2750,11 @@ paths: minimum: 1 maximum: 3600 description: 'Container stop grace period in seconds.' + rolling_update_drain_seconds: + type: integer + minimum: 0 + maximum: 300 + description: 'Seconds to wait after the new container is healthy before old containers stop during a rolling update. 0 turns the wait off. Default: 5.' docker_images_to_keep: type: integer minimum: 0 @@ -2683,6 +2773,10 @@ paths: type: string nullable: true description: 'Traefik certificate resolver for this application. Null uses the server setting, "none" uses no resolver.' + is_container_label_readonly_enabled: + type: boolean + default: true + description: 'Let Coolify manage the container labels. True regenerates custom_labels from domains, redirect, and proxy settings. False keeps custom_labels as you set them: domains, redirect, and basic auth changes then do not change the labels.' is_raw_compose_deployment_enabled: type: boolean description: 'Deploy the raw Docker Compose definition.' @@ -4535,6 +4629,10 @@ paths: type: boolean description: 'Whether to save backups to S3' default: false + disable_local_backup: + type: boolean + description: 'Delete the local backup file after the upload to S3. Only allowed when save_s3 is true.' + default: false s3_storage_uuid: type: string description: 'Primary S3 storage UUID. Sent without s3_storage_uuids, it selects this storage as the only destination. Either field is required if save_s3 is true.' @@ -4876,6 +4974,12 @@ paths: redact_all_env_values_in_logs: type: boolean description: 'Redact all environment variable values in logs. When false, only locked (shown once) values, database passwords, and remote secrets are redacted. Requires team admin permissions.' + include_coolify_default_variables: + type: boolean + description: 'Add the predefined COOLIFY_* variables (COOLIFY_URL, COOLIFY_FQDN, ...) to the environment. Requires team admin permissions. Restart the database to apply it.' + is_log_drain_enabled: + type: boolean + description: 'Send the database logs to the log drain of the server. Log drain must be enabled on the server. Restart the database to apply it.' type: object responses: '200': @@ -4974,6 +5078,9 @@ paths: save_s3: type: boolean description: 'Whether data is saved in s3 or not' + disable_local_backup: + type: boolean + description: 'Delete the local backup file after the upload to S3. Only allowed when save_s3 is true. Turning save_s3 off also turns this off.' s3_storage_uuid: type: string description: 'Primary S3 storage UUID. Sent without s3_storage_uuids, it selects this storage as the only destination; when it is already a destination, it becomes the primary destination and the other destinations stay.' @@ -10942,6 +11049,7 @@ paths: status: { type: [string, 'null'], example: running } redirect_enabled: { type: boolean, example: true } redirect_url: { type: [string, 'null'], example: 'https://example.com' } + proxy_error_page: { type: [string, 'null'] } generate_exact_labels: { type: boolean, example: false } traefik_certificate_resolver: { description: 'Traefik certificate resolver for generated HTTPS routers. Null when the proxy uses its own certificates.', type: [string, 'null'], example: letsencrypt } configuration: { description: 'Docker Compose proxy configuration when stored in the database. Only present with read:sensitive.', type: [string, 'null'] } @@ -10980,6 +11088,9 @@ paths: redirect_url: description: 'Public http(s) redirect URL, or null to clear.' type: [string, 'null'] + proxy_error_page: + description: 'Custom HTML error page that the proxy shows for unknown hosts (when redirect_url is empty) and for resources in maintenance mode. Null restores the default page.' + type: [string, 'null'] generate_exact_labels: type: boolean traefik_certificate_resolver: @@ -11001,6 +11112,7 @@ paths: status: { type: [string, 'null'] } redirect_enabled: { type: boolean } redirect_url: { type: [string, 'null'] } + proxy_error_page: { type: [string, 'null'] } generate_exact_labels: { type: boolean } traefik_certificate_resolver: { type: [string, 'null'] } configuration: { type: [string, 'null'] } @@ -11057,6 +11169,7 @@ paths: status: { type: [string, 'null'] } redirect_enabled: { type: boolean } redirect_url: { type: [string, 'null'] } + proxy_error_page: { type: [string, 'null'] } generate_exact_labels: { type: boolean } configuration: { type: [string, 'null'] } type: object @@ -11690,6 +11803,17 @@ paths: connection_timeout: type: integer description: 'SSH connection timeout in seconds (1-300). Default: 10.' + wildcard_domain: + type: string + nullable: true + description: 'Wildcard domain (URL) used to generate domains for new resources on this server. Null removes it.' + server_timezone: + type: string + description: 'Timezone of the server, for example Europe/Berlin. Scheduled tasks and backups use it.' + backup_compression_cpu_percentage: + type: integer + enum: [25, 50, 75, 100] + description: 'Maximum CPU percentage that backup compression can use. Default: 25.' type: object responses: '403': @@ -11953,6 +12077,10 @@ paths: is_permanent_redirect_enabled: description: 'Use permanent redirects (301/308) instead of temporary (302/307) for the HTTP to HTTPS and www redirects. Defaults to false.' type: [boolean, 'null'] + redirect: + description: 'How to handle www and non-www domains. "www" redirects non-www to www, "non-www" redirects www to non-www. Both need a www and a non-www domain in url.' + type: string + enum: [both, www, non-www] max_restart_count: description: 'Maximum Docker restart count before Coolify stops the container. Set to 0 to disable the limit.' type: [integer, 'null'] @@ -12864,14 +12992,6 @@ paths: schema: type: boolean default: true - - - name: delete_from_coolify_only - in: query - description: 'Remove only Coolify metadata without deleting Docker resources.' - required: false - schema: - type: boolean - default: false responses: '200': description: 'Delete a service by UUID' @@ -12889,6 +13009,8 @@ paths: $ref: '#/components/responses/404' '403': description: 'The token needs the write and deploy permissions.' + '422': + $ref: '#/components/responses/422' security: - bearerAuth: [] @@ -12945,6 +13067,10 @@ paths: type: boolean default: true description: 'Redact all environment variable values in logs. When false, only locked (shown once) values are redacted. Requires team admin permissions.' + include_coolify_default_variables: + type: boolean + default: true + description: 'Add the predefined COOLIFY_* variables (COOLIFY_URL, COOLIFY_FQDN, ...) to the environment. Requires team admin permissions. Redeploy to apply it.' type: object responses: '200': @@ -15963,6 +16089,9 @@ components: stop_grace_period: type: integer nullable: true + rolling_update_drain_seconds: + type: integer + description: 'Seconds to wait after the new container is healthy before old containers stop during a rolling update. 0 turns the wait off.' type: object Environment: description: 'Environment model' diff --git a/other/nightly/install.sh b/other/nightly/install.sh index ebd309a021..b00f35da86 100755 --- a/other/nightly/install.sh +++ b/other/nightly/install.sh @@ -929,7 +929,8 @@ update_env_var() { # If variable "key=" exists but has no value, update the value of the existing line if grep -q "^${key}=$" "$ENV_FILE"; then - sed -i "s|^${key}=$|${key}=${value}|" "$ENV_FILE" + # awk reads the value from the environment, so characters such as | & \ in it stay literal + KEY="$key" VALUE="$value" awk '$0 == ENVIRON["KEY"] "=" { $0 = ENVIRON["KEY"] "=" ENVIRON["VALUE"] } { print }' "$ENV_FILE" >"$ENV_FILE.tmp" && mv "$ENV_FILE.tmp" "$ENV_FILE" echo " - Updated value of ${key} as the current value was empty" # If variable "key=" doesn't exist, append it to the file with value elif ! grep -q "^${key}=" "$ENV_FILE"; then @@ -938,6 +939,23 @@ update_env_var() { fi } +# Quote a user-supplied value so that Docker Compose and phpdotenv both read it back unchanged +# (spaces, #, $ and quotes otherwise break parsing or change the value). +quote_env_value() { + local value="$1" + case "$value" in + *\'*) + value=${value//\\/\\\\} + value=${value//\"/\\\"} + value=${value//\$/\\\$} + printf '"%s"' "$value" + ;; + *) + printf "'%s'" "$value" + ;; + esac +} + update_env_var "APP_ID" "$(openssl rand -hex 16)" update_env_var "APP_KEY" "base64:$(openssl rand -base64 32)" # update_env_var "DB_USERNAME" "$(openssl rand -hex 16)" # Causes issues: database "random-user" does not exist @@ -948,12 +966,16 @@ update_env_var "PUSHER_APP_KEY" "$(openssl rand -hex 32)" update_env_var "PUSHER_APP_SECRET" "$(openssl rand -hex 32)" update_env_var "PUSHER_BACKEND_PORT" "6001" -# Add default root user credentials from environment variables -if [ -n "$ROOT_USERNAME" ] && [ -n "$ROOT_USER_EMAIL" ] && [ -n "$ROOT_USER_PASSWORD" ]; then +# Add default root user credentials from environment variables (ROOT_USERNAME is optional) +if [ -n "$ROOT_USER_EMAIL" ] && [ -n "$ROOT_USER_PASSWORD" ]; then echo " - Setting predefined root user credentials from environment" - update_env_var "ROOT_USERNAME" "$ROOT_USERNAME" - update_env_var "ROOT_USER_EMAIL" "$ROOT_USER_EMAIL" - update_env_var "ROOT_USER_PASSWORD" "$ROOT_USER_PASSWORD" + if [ -n "$ROOT_USERNAME" ]; then + update_env_var "ROOT_USERNAME" "$(quote_env_value "$ROOT_USERNAME")" + fi + update_env_var "ROOT_USER_EMAIL" "$(quote_env_value "$ROOT_USER_EMAIL")" + update_env_var "ROOT_USER_PASSWORD" "$(quote_env_value "$ROOT_USER_PASSWORD")" +elif [ -n "$ROOT_USERNAME" ] || [ -n "$ROOT_USER_EMAIL" ] || [ -n "$ROOT_USER_PASSWORD" ]; then + warn "Predefined root user skipped: set both ROOT_USER_EMAIL and ROOT_USER_PASSWORD." fi if [ -n "${REGISTRY_URL+x}" ]; then @@ -1006,7 +1028,8 @@ if [ "$IS_COOLIFY_VOLUME_EXISTS" -eq 0 ]; then test -f /data/coolify/ssh/keys/id.$CURRENT_USER@host.docker.internal.pub && rm -f /data/coolify/ssh/keys/id.$CURRENT_USER@host.docker.internal.pub ssh-keygen -t ed25519 -a 100 -f /data/coolify/ssh/keys/id.$CURRENT_USER@host.docker.internal -q -N "" -C coolify chown 9999 /data/coolify/ssh/keys/id.$CURRENT_USER@host.docker.internal - sed -i "/coolify/d" ~/.ssh/authorized_keys + # Remove only the key from a previous install (generated with the comment "coolify"), not other keys that contain "coolify" + sed -i -E "/[[:space:]]coolify[[:space:]]*$/d" ~/.ssh/authorized_keys cat /data/coolify/ssh/keys/id.$CURRENT_USER@host.docker.internal.pub >>~/.ssh/authorized_keys rm -f /data/coolify/ssh/keys/id.$CURRENT_USER@host.docker.internal.pub fi @@ -1144,6 +1167,9 @@ for IP in $PRIVATE_IPS; do fi done +if [ "$IS_COOLIFY_VOLUME_EXISTS" -eq 0 ] && { [ -z "$ROOT_USER_EMAIL" ] || [ -z "$ROOT_USER_PASSWORD" ]; }; then + printf '\n %s! Create the root account within 10 minutes. If the time ends, run %sdocker restart coolify%s%s and try again.%s\n' "$C_YELLOW" "$C_BOLD" "$C_RESET" "$C_YELLOW" "$C_RESET" >&3 +fi printf '\n %s! Back up %s/data/coolify/source/.env%s%s to a safe place outside this server (e.g. a password manager).%s\n' "$C_YELLOW" "$C_BOLD" "$C_RESET" "$C_YELLOW" "$C_RESET" >&3 printf ' %sLog file: %s%s\n\n' "$C_DIM" "$INSTALLATION_LOG_WITH_DATE" "$C_RESET" >&3 diff --git a/other/nightly/versions.json b/other/nightly/versions.json index 39fa524e2b..d0c384b345 100644 --- a/other/nightly/versions.json +++ b/other/nightly/versions.json @@ -1,13 +1,13 @@ { "coolify": { "v4": { - "version": "4.4.7" + "version": "4.5.1" }, "nightly": { - "version": "4.5-rc.1" + "version": "4.6-rc.1" }, "helper": { - "version": "1.0.17" + "version": "1.0.19" }, "sentinel": { "version": "1.1.2" @@ -18,7 +18,7 @@ } }, "traefik": { - "v3.7": "3.7.13", + "v3.7": "3.7.16", "v3.6": "3.6.25", "v3.5": "3.5.6", "v3.4": "3.4.5", @@ -26,6 +26,6 @@ "v3.2": "3.2.5", "v3.1": "3.1.7", "v3.0": "3.0.4", - "v2.11": "2.11.57" + "v2.11": "2.11.60" } } diff --git a/resources/css/app.css b/resources/css/app.css index afdae048ed..3948fa7d6f 100644 --- a/resources/css/app.css +++ b/resources/css/app.css @@ -12,7 +12,9 @@ @source '../../storage/framework/views/*.php'; -@custom-variant dark (&:where(.dark, .dark *)); +/* A custom theme can give the sidebar the other scheme than the page + (data-sidebar-scheme on ). dark: utilities follow the sidebar there. */ +@custom-variant dark (&:where(.dark, .dark *, [data-sidebar-scheme="dark"] :is(.app-sidebar, .app-topbar-brand, .mobile-sidebar-sheet [role="dialog"]), [data-sidebar-scheme="dark"] :is(.app-sidebar, .app-topbar-brand, .mobile-sidebar-sheet [role="dialog"]) *):not(:where([data-sidebar-scheme="light"] :is(.app-sidebar, .app-topbar-brand, .mobile-sidebar-sheet [role="dialog"]), [data-sidebar-scheme="light"] :is(.app-sidebar, .app-topbar-brand, .mobile-sidebar-sheet [role="dialog"]) *))); @theme { --font-sans: 'Geist Sans', Inter, sans-serif; @@ -39,6 +41,13 @@ --color-coollabs-100: oklch(52.34% 0.287 289.16); --color-coollabs-200: oklch(43.76% 0.2369 289.82); --color-coollabs-300: oklch(38.04% 0.2031 290.47); + /* coolLabs brand purple. Primary actions, focus, and active navigation + use these in both themes. Warning yellow is not an accent. */ + --color-brand: #6b16ed; + --color-brand-hover: #7317ff; + --color-brand-press: #5a12c7; + --color-brand-depth: #4a0fa3; + --color-brand-light: #a391ff; --color-coolgray-100: oklch(20.9% 0 0); --color-coolgray-200: oklch(24.35% 0 0); --color-coolgray-300: oklch(26.03% 0 0); @@ -102,7 +111,33 @@ } .dark .button:not(.button-highlighted):not(.button-error):not([isHighlighted]):not(:disabled) { - --button-depth-color: rgb(255 255 255 / 0.08); + --button-depth-color: color-mix(in srgb, var(--color-brand) 35%, transparent); + background-color: color-mix(in srgb, var(--color-brand) 14%, transparent); + border-color: color-mix(in srgb, var(--color-brand-light) 24%, transparent); + color: oklch(from var(--color-brand-light) 93.5% 0.03 h); +} + +.dark .button:not(.button-highlighted):not(.button-error):not([isHighlighted]):not(:disabled):hover { + background-color: color-mix(in srgb, var(--color-brand) 18%, transparent); + border-color: color-mix(in srgb, var(--color-brand-light) 34%, transparent); +} + +.dark .button:not(.button-highlighted):not(.button-error):not([isHighlighted]):not(:disabled) svg { + color: var(--color-brand-light); + opacity: 1; +} + +.dark .button.button-highlighted:not(:disabled), +.dark .button[isHighlighted]:not(:disabled) { + background: var(--color-brand); + border-color: transparent; + color: var(--color-brand-foreground); + --button-depth-color: var(--color-brand-depth); +} + +.dark .button.button-highlighted:not(:disabled):hover, +.dark .button[isHighlighted]:not(:disabled):hover { + background: var(--color-brand-hover); } .button.button-error:not(:disabled) { @@ -161,22 +196,40 @@ transition-duration: 120ms; } -.button { - transition-duration: 120ms, 120ms, 120ms, 80ms, 80ms; -} - .icon-button:not(:disabled):active, -.app-tab:active, -.split-action-main:not(:disabled):active, -.split-action-caret:not(:disabled):active { +.app-tab:active { transform: scale(0.97); } -.button:not(:disabled):active { +.button, +.split-action-main, +.split-action-caret { + transition-duration: 120ms, 120ms, 120ms, 80ms, 80ms; +} + +.button:not(:disabled):active, +.split-action-main:not(:disabled):active, +.split-action-caret:not(:disabled):active { transform: translateY(2px); box-shadow: none; } +/* Appearance โ†’ Style โ†’ Flat (data-nav-items="flat") removes the raised edge + everywhere, not only in the sidebar: the depth color goes transparent (the + !important beats the theme overrides of the variable) and the press uses + the scale feedback of .icon-button instead of the lift and sink. */ +html[data-nav-items="flat"] :is(.button, .split-action-main, .split-action-caret, .toast-icon) { + --button-depth-color: transparent !important; +} + +html[data-nav-items="flat"] :is(.button, .split-action-main, .split-action-caret):not(:disabled):hover { + transform: none; +} + +html[data-nav-items="flat"] :is(.button, .split-action-main, .split-action-caret):not(:disabled):active { + transform: scale(0.97); +} + /* Focus border/ring on any form control should ease, not snap. The accent focus indicator is a border-color and/or box-shadow (ring) change, which the utilities' `transition-colors` did not fully animate. Target the elements @@ -382,11 +435,12 @@ select, position: absolute; inset: 0; border-radius: inherit; + --coolbox-loading-color: var(--color-coollabs); background: - linear-gradient(var(--color-warning), var(--color-warning)), - linear-gradient(var(--color-warning), var(--color-warning)), - linear-gradient(var(--color-warning), var(--color-warning)), - linear-gradient(var(--color-warning), var(--color-warning)); + linear-gradient(var(--coolbox-loading-color), var(--coolbox-loading-color)), + linear-gradient(var(--coolbox-loading-color), var(--coolbox-loading-color)), + linear-gradient(var(--coolbox-loading-color), var(--coolbox-loading-color)), + linear-gradient(var(--coolbox-loading-color), var(--coolbox-loading-color)); background-repeat: no-repeat; animation: coolbox-border-track 2400ms linear infinite; pointer-events: none; @@ -403,6 +457,10 @@ select, z-index: 1; } + .dark .coolbox-loading::before { + --coolbox-loading-color: var(--color-brand-light); + } + .dark .coolbox-loading::after { background: var(--color-coolgray-100); } @@ -490,7 +548,7 @@ a { button:focus-visible, a:focus-visible { - @apply outline-none ring-2 ring-coollabs dark:ring-1 dark:ring-warning ring-offset-2 dark:ring-offset-coolgray-100; + @apply outline-none ring-2 ring-coollabs dark:ring-1 dark:ring-brand-light ring-offset-2 dark:ring-offset-coolgray-100; } label { @@ -545,6 +603,35 @@ tr td:first-child { padding-right: 2.5rem; } +/* Number fields use the brand โˆ’/+ stepper (components/forms/input) instead of the native spinner. */ +.input.input-with-stepper { + padding-right: 4.25rem; + appearance: textfield; + -moz-appearance: textfield; +} + +.input.input-with-stepper::-webkit-outer-spin-button, +.input.input-with-stepper::-webkit-inner-spin-button { + margin: 0; + -webkit-appearance: none; +} + +.number-stepper { + @apply absolute inset-y-px right-px flex; +} + +.number-stepper button { + @apply flex w-8 cursor-pointer items-center justify-center border-l border-neutral-200 text-neutral-500 transition-colors last:rounded-r-[5px] hover:bg-neutral-100 hover:text-black active:bg-neutral-200 disabled:cursor-not-allowed disabled:opacity-40 disabled:hover:bg-transparent dark:border-white/[0.08] dark:text-fg-dim dark:hover:bg-white/[0.06] dark:hover:text-white dark:active:bg-white/[0.1]; +} + +.number-stepper button svg { + @apply transition-transform duration-150 ease-out; +} + +.number-stepper button:active:not(:disabled) svg { + @apply scale-75; +} + /* Room for both the peek eye and the copy button on maskable read-only fields. */ .input.input-with-copy-and-peek { padding-right: 4.25rem; @@ -554,24 +641,24 @@ tr td:first-child { animation: lds-heart 1.2s infinite cubic-bezier(0.215, 0.61, 0.355, 1); } -/* Loading feedback uses the higher-contrast warning accent in dark mode. */ +/* Loading feedback uses the brand-light accent in dark mode. */ .dark .animate-spin { - color: var(--color-warning) !important; + color: var(--color-brand-light) !important; } .dark #nprogress .bar { - background: var(--color-warning) !important; + background: var(--color-brand-light) !important; } .dark #nprogress .peg { box-shadow: - 0 0 10px var(--color-warning), - 0 0 5px var(--color-warning) !important; + 0 0 10px var(--color-brand-light), + 0 0 5px var(--color-brand-light) !important; } .dark #nprogress .spinner-icon { - border-top-color: var(--color-warning) !important; - border-left-color: var(--color-warning) !important; + border-top-color: var(--color-brand-light) !important; + border-left-color: var(--color-brand-light) !important; } html[data-theme="custom"] .loading-indicator, @@ -726,6 +813,14 @@ html.dark .application-console-shell[data-console-theme="system"] { --console-theme-background: var(--coollabs-elevated); } +/* Dark mode: the docked System console uses the dark card colors (darker outer + header, --color-card body, faint white ring), so it matches the cards around it. */ +html.dark .application-console-shell[data-console-theme="system"] { + --coollabs-elevated: color-mix(in srgb, var(--color-card) 66%, black); + --coollabs-base: var(--color-card); + --coollabs-hairline: rgb(255 255 255 / 8%); +} + .console-theme-selector { scrollbar-color: rgb(161 161 170) transparent; scrollbar-width: thin; @@ -988,9 +1083,8 @@ html:not(.dark) .application-console-shell[data-console-theme="system"] .termina .terminal-target-picker { color: rgb(255 255 255 / 0.75); - background: rgb(0 0 0 / 0.18); + background: oklch(18.31% 0.004 285.99); border-color: rgb(255 255 255 / 0.1); - backdrop-filter: blur(16px); } /* Pre-session target list: a normal page card, not the themed console canvas. */ @@ -1254,14 +1348,26 @@ html:not(.dark) .application-console-shell[data-console-theme="system"] .termina /* Card/table edge ring. Crisped from 93.5% (1.17:1 on canvas, invisible) to 85.5% (1.51:1) so surfaces read as bordered cards, not flat fills. */ --coollabs-hairline: oklch(85.5% 0 0); + /* Row and section separators inside cards. Softer than the card edge. + Translucent, so it stays visible on any card or theme surface. */ + --coollabs-divider: oklch(0% 0 0 / 0.04); /* Muted labels/titles. Darkened from 55.6% (failed WCAG AA on elevated 4.46, recessed 4.24, fill 3.76) to 50% so muted text clears 4.5:1 on every light surface above white. Dark stays 70.8% (already passes). */ --coollabs-subtle: oklch(50% 0 0); + /* Border of brand fills and the text on them. A custom theme overrides both. */ + --color-brand-ring: #8a4dff; + --color-brand-foreground: #fff; } .dark { - --color-accent: oklch(88.13% 0.1507 91.7); + /* Purple is the accent in dark mode too. Warning yellow stays on + --color-warning for real warnings only (update available, attention). */ + --color-accent: var(--color-brand); + /* Cards sit one step above the #0a0a0a canvas. The top bar uses the canvas, + not this color. The sidebar uses --color-sidebar. */ + --color-card: #121212; + --color-sidebar: #120e1a; /* Dark surface ladder in hex (sRGB), not oklch: oklch lightness compresses to near-black below ~15%, so oklch(15%) rendered as sRGB 11 and cards could not lift off the content canvas (sRGB 10). These give clear, @@ -1283,6 +1389,7 @@ html:not(.dark) .application-console-shell[data-console-theme="system"] .termina /* Kept about 1.6:1 on the canvas so dark cards and tables show a visible edge instead of blending into the canvas. */ --coollabs-hairline: oklch(36% 0 0); + --coollabs-divider: oklch(100% 0 0 / 0.045); --coollabs-subtle: oklch(70.8% 0 0); /* Neutral (was #a8a8b0 / #7a7a84, both cool-tinted). Sidebar text now shares the same neutral temperature as the content text, so the sidebar no longer @@ -1318,19 +1425,24 @@ html:not(.dark) { --color-error: oklch(52% 0.2 27.33); } -/* Theme surfaces are derived from one brand color. Changing - --theme-base-color is enough to generate a complete dark surface ladder. */ +/* Custom theme. applyCustomTheme() in base.blade.php sets four colors and the + text color that is readable on each. The page and the sidebar each derive + one surface ladder from three variables: + --t-bg the background color + --t-text the text color on it + --t-light 1 for a light background, 0 for a dark one + Light schemes put white cards on a slightly darker canvas. Dark schemes lift + cards toward the text color. */ html[data-theme="custom"] { + --t-bg: var(--theme-bg); + --t-text: var(--theme-text); + --t-light: var(--theme-light); --theme-base-color: oklch(49.65% 0.2709 289.33); --theme-bright-color: color-mix(in srgb, var(--theme-base-color) 85%, oklch(100% 0 0)); --theme-scrollbar-thumb: color-mix(in srgb, var(--theme-bright-color) 70%, var(--theme-accent-foreground)); - --theme-border-color: color-mix(in oklab, var(--theme-base-color) 42%, oklch(44.19% 0.0146 285.79)); /* Accent text/icons sit directly on dark surfaces. Force the lightness into a readable band (and tame chroma) so any picked hue keeps enough contrast. */ --theme-fg-on-surface: oklch(from var(--theme-base-color) clamp(0.72, l, 0.86) min(c, 0.13) h); - /* Placeholders read as a light, near-neutral hint of the hue - never the raw - (possibly dark) base color, which is invisible on the dark input. */ - --theme-placeholder-color: oklch(from var(--theme-base-color) 0.62 min(c, 0.03) h); --color-accent: var(--theme-bright-color); --color-coollabs: var(--theme-bright-color); --color-coollabs-100: color-mix(in oklab, var(--theme-bright-color) 88%, oklch(100% 0 0)); @@ -1338,79 +1450,68 @@ html[data-theme="custom"] { --color-coollabs-300: color-mix(in oklab, var(--theme-bright-color) 72%, oklch(0% 0 0)); /* Legacy dark-theme accent utilities use warning as the brand color. */ --color-warning: var(--theme-bright-color); + /* Brand fills, tints, and accent text follow the picked color. Text on a + brand fill uses the computed foreground, so light colors stay readable. */ + --color-brand: var(--theme-bright-color); + --color-brand-hover: color-mix(in oklab, var(--theme-bright-color) 88%, oklch(100% 0 0)); + --color-brand-press: color-mix(in oklab, var(--theme-bright-color) 85%, oklch(0% 0 0)); + --color-brand-depth: color-mix(in oklab, var(--theme-bright-color) 65%, oklch(0% 0 0)); + --color-brand-ring: color-mix(in oklab, var(--theme-bright-color) 75%, oklch(100% 0 0)); + --color-brand-foreground: var(--theme-accent-foreground); + --color-brand-light: var(--theme-fg-on-surface); --color-accent-foreground: var(--theme-accent-foreground); - --color-app: color-mix(in oklab, var(--theme-base-color) 12%, oklch(14.03% 0.0022 286.11)); - --color-panel: color-mix(in oklab, var(--theme-base-color) 14%, oklch(15.48% 0.0021 286.15)); - --color-surface: color-mix(in oklab, var(--theme-base-color) 18%, oklch(17.35% 0.0020 286.18)); - --color-raised: color-mix(in oklab, var(--theme-base-color) 24%, oklch(19.21% 0.0040 286.02)); - --color-selected: color-mix(in oklab, var(--theme-base-color) 32%, oklch(20.99% 0.0039 286.06)); - --color-coolgray-100: var(--color-surface); - --color-coolgray-200: var(--color-raised); - --color-coolgray-300: var(--color-selected); - --color-coolgray-400: color-mix(in oklab, var(--theme-base-color) 28%, oklch(22.34% 0.0058 285.92)); - --color-coolgray-500: color-mix(in oklab, var(--theme-base-color) 32%, oklch(24.52% 0.0075 285.83)); - --coollabs-canvas: var(--color-app); - --coollabs-elevated: var(--color-surface); - --coollabs-recessed: var(--color-raised); - --coollabs-base: color-mix(in oklab, var(--theme-base-color) 22%, oklch(17.81% 0.0020 286.19)); - --color-content-surface: var(--coollabs-base); - --coollabs-fill: color-mix(in oklab, var(--theme-base-color) 32%, oklch(20.99% 0.0039 286.06)); - --coollabs-line: var(--theme-border-color); - --coollabs-hairline: color-mix(in srgb, var(--theme-border-color) 55%, var(--color-panel)); - --color-nav-text: oklch(91.08% 0.0157 306.4); - --color-nav-muted: oklch(87.24% 0.0188 306.63); - --color-nav-active: oklch(100.0% 0 0); - --color-log: color-mix(in oklab, var(--theme-base-color) 20%, oklch(13.49% 0.0024 286.07)); - --color-log-toolbar: color-mix(in oklab, var(--theme-base-color) 26%, oklch(17.35% 0.0020 286.18)); - --color-fg-dim: oklch(91.08% 0.0157 306.4); - --color-fg-faint: oklch(87.24% 0.0188 306.63); - --coollabs-subtle: oklch(87.24% 0.0188 306.63); + --color-sidebar: var(--theme-sidebar); } -/* Light custom theme: the same single brand color, but surfaces tint toward - white and every foreground token flips to a dark, readable band. This selector - outranks the plain html:not(.dark) light palette, so it wins where they meet. - Accent fills (--theme-bright-color, --color-warning/coollabs) are inherited from - the block above so the JS-computed --theme-accent-foreground stays consistent. */ html[data-theme="custom"]:not(.dark) { --theme-scrollbar-thumb: color-mix(in srgb, var(--theme-base-color) 45%, oklch(60% 0 0)); /* Accent text/icons on light surfaces: force lightness into a dark band. */ --theme-fg-on-surface: oklch(from var(--theme-base-color) clamp(0.30, l, 0.48) min(c, 0.16) h); - /* Placeholder: medium-dark, near-neutral hint of the hue. */ - --theme-placeholder-color: oklch(from var(--theme-base-color) 0.50 min(c, 0.03) h); - /* Surfaces hold a fixed high lightness (stays light, keeps dark text readable) - while injecting the base hue's chroma, so the tint is actually visible. - color-mix into white coupled lightness to the base and washed the tint out - (a 4% mix of any color into near-white is imperceptible). */ - --color-app: oklch(from var(--theme-base-color) 96.5% min(c, 0.035) h); - --color-panel: oklch(from var(--theme-base-color) 98.5% min(c, 0.025) h); - --color-surface: oklch(from var(--theme-base-color) 98% min(c, 0.03) h); - --color-raised: oklch(from var(--theme-base-color) 95.5% min(c, 0.04) h); - --color-selected: oklch(from var(--theme-base-color) 92% min(c, 0.05) h); +} + +html[data-theme="custom"] :is(.app-sidebar, .app-topbar-brand, .mobile-sidebar-sheet [role="dialog"]) { + --t-bg: var(--theme-sidebar); + --t-text: var(--theme-sidebar-text); + --t-light: var(--theme-sidebar-light); +} + +html[data-theme="custom"], +html[data-theme="custom"] :is(.app-sidebar, .app-topbar-brand, .mobile-sidebar-sheet [role="dialog"]) { + --color-app: color-mix(in oklab, var(--t-text) calc(var(--t-light) * 3%), var(--t-bg)); + --color-panel: color-mix(in oklab, var(--t-text) calc((1 - var(--t-light)) * 3%), var(--t-bg)); + --color-surface: color-mix(in oklab, var(--t-text) calc((1 - var(--t-light)) * 4%), var(--t-bg)); + --color-raised: color-mix(in oklab, var(--t-text) calc(4% + (1 - var(--t-light)) * 2%), var(--t-bg)); + --color-selected: color-mix(in oklab, var(--t-text) calc(8% + (1 - var(--t-light)) * 2%), var(--t-bg)); + --color-card: var(--color-surface); --color-coolgray-100: var(--color-surface); --color-coolgray-200: var(--color-raised); --color-coolgray-300: var(--color-selected); - --color-coolgray-400: oklch(from var(--theme-base-color) 90% min(c, 0.05) h); - --color-coolgray-500: oklch(from var(--theme-base-color) 87% min(c, 0.055) h); + --color-coolgray-400: color-mix(in oklab, var(--t-text) calc(9% + (1 - var(--t-light)) * 3%), var(--t-bg)); + --color-coolgray-500: color-mix(in oklab, var(--t-text) calc(12% + (1 - var(--t-light)) * 3%), var(--t-bg)); --coollabs-canvas: var(--color-app); - --coollabs-elevated: oklch(from var(--theme-base-color) 98.5% min(c, 0.025) h); + --coollabs-elevated: var(--color-surface); --coollabs-recessed: var(--color-raised); - --coollabs-base: oklch(from var(--theme-base-color) 99.3% min(c, 0.018) h); + --coollabs-base: color-mix(in oklab, var(--t-text) calc((1 - var(--t-light)) * 5%), var(--t-bg)); --color-content-surface: var(--coollabs-base); - --coollabs-fill: oklch(from var(--theme-base-color) 91% min(c, 0.05) h); - --coollabs-line: oklch(from var(--theme-base-color) 82% min(c, 0.055) h); - --coollabs-hairline: oklch(from var(--theme-base-color) 88% min(c, 0.04) h); - --color-nav-text: oklch(from var(--theme-base-color) 0.40 min(c, 0.04) h); - --color-nav-muted: oklch(from var(--theme-base-color) 0.50 min(c, 0.03) h); - --color-nav-active: oklch(from var(--theme-base-color) 0.22 min(c, 0.05) h); - --color-log: oklch(from var(--theme-base-color) 96.5% min(c, 0.035) h); - --color-log-toolbar: oklch(from var(--theme-base-color) 93% min(c, 0.045) h); - --color-fg: oklch(from var(--theme-base-color) 0.22 min(c, 0.04) h); - --color-fg-dim: oklch(from var(--theme-base-color) 0.40 min(c, 0.04) h); - --color-fg-faint: oklch(from var(--theme-base-color) 0.50 min(c, 0.03) h); - --coollabs-subtle: oklch(from var(--theme-base-color) 0.50 min(c, 0.03) h); + --coollabs-fill: color-mix(in oklab, var(--t-text) calc(8% + (1 - var(--t-light)) * 4%), var(--t-bg)); + --coollabs-line: color-mix(in oklab, var(--t-text) calc(16% + (1 - var(--t-light)) * 6%), var(--t-bg)); + --coollabs-hairline: color-mix(in oklab, var(--t-text) calc(11% + (1 - var(--t-light)) * 3%), var(--t-bg)); + --coollabs-divider: color-mix(in oklab, var(--t-text) calc(5% + (1 - var(--t-light)) * 1%), transparent); + --color-hairline: color-mix(in oklab, var(--t-text) 8%, transparent); + --color-fg: var(--t-text); + --color-fg-dim: color-mix(in oklab, var(--t-text) 74%, var(--t-bg)); + --color-fg-faint: color-mix(in oklab, var(--t-text) 60%, var(--t-bg)); + --coollabs-subtle: var(--color-fg-faint); + --theme-placeholder-color: color-mix(in oklab, var(--t-text) 50%, var(--t-bg)); + --color-nav-text: color-mix(in oklab, var(--t-text) 74%, var(--t-bg)); + --color-nav-muted: color-mix(in oklab, var(--t-text) 60%, var(--t-bg)); + --color-nav-active: var(--t-text); + /* Logs sit a little deeper than the canvas. */ + --color-log: color-mix(in oklab, oklch(0% 0 0) calc((1 - var(--t-light)) * 20%), var(--color-app)); + --color-log-toolbar: var(--color-surface); } + html[data-theme="custom"] .control-selected, html[data-theme="custom"] .logs-viewer-btn-active { color: var(--color-accent-foreground); @@ -1480,14 +1581,20 @@ html[data-theme="custom"] :is( [class~="text-warning"], [class~="text-accent"], [class~="dark:text-coollabs"], - [class~="dark:text-warning"] + [class~="dark:text-warning"], + [class~="dark:text-brand-light"] ) { color: var(--theme-fg-on-surface) !important; } +html.dark[data-theme="custom"] [class~="dark:bg-brand"] { + color: var(--theme-accent-foreground) !important; +} + html[data-theme="custom"] [class~="hover:text-coollabs"]:hover, html[data-theme="custom"] [class~="hover:text-warning"]:hover, html[data-theme="custom"] [class~="dark:hover:text-warning"]:hover, +html[data-theme="custom"] [class~="dark:hover:text-brand-light"]:hover, html[data-theme="custom"] [class~="dark:hover:text-coollabs"]:hover { color: var(--theme-fg-on-surface) !important; } @@ -1497,27 +1604,43 @@ html[data-theme="custom"] [class~="dark:hover:text-coollabs"]:hover { resolve to --color-* vars the block overrides); light mode has no equivalent, so components fall back to hardcoded neutrals. Remap the dominant light surface/border utilities to the tinted custom vars so the color reaches the - whole UI, not just design-system classes. */ -html[data-theme="custom"]:not(.dark) :is([class~="bg-white"], [class~="bg-white/95"], [class~="bg-white/80"]) { + whole UI, not just design-system classes. A light sidebar on a dark page + gets the same remap. */ +html[data-theme="custom"]:not(.dark) :is([class~="bg-white"], [class~="bg-white/95"], [class~="bg-white/80"]), +html[data-sidebar-scheme="light"] :where(.app-sidebar, .app-topbar-brand, .mobile-sidebar-sheet [role="dialog"]) :is([class~="bg-white"], [class~="bg-white/95"], [class~="bg-white/80"]) { background-color: var(--coollabs-elevated) !important; } -html[data-theme="custom"]:not(.dark) [class~="bg-neutral-50"] { +html[data-theme="custom"]:not(.dark) [class~="bg-neutral-50"], +html[data-sidebar-scheme="light"] :where(.app-sidebar, .app-topbar-brand, .mobile-sidebar-sheet [role="dialog"]) [class~="bg-neutral-50"] { background-color: var(--coollabs-canvas) !important; } -html[data-theme="custom"]:not(.dark) :is([class~="bg-neutral-100"], [class~="bg-neutral-200"]) { +html[data-theme="custom"]:not(.dark) :is([class~="bg-neutral-100"], [class~="bg-neutral-200"]), +html[data-sidebar-scheme="light"] :where(.app-sidebar, .app-topbar-brand, .mobile-sidebar-sheet [role="dialog"]) :is([class~="bg-neutral-100"], [class~="bg-neutral-200"]) { background-color: var(--coollabs-fill) !important; } -html[data-theme="custom"]:not(.dark) [class~="border-neutral-200"] { +html[data-theme="custom"]:not(.dark) [class~="border-neutral-200"], +html[data-sidebar-scheme="light"] :where(.app-sidebar, .app-topbar-brand, .mobile-sidebar-sheet [role="dialog"]) [class~="border-neutral-200"] { border-color: var(--coollabs-hairline) !important; } -html[data-theme="custom"]:not(.dark) :is([class~="border-neutral-300"], [class~="border-neutral-100"]) { +html[data-theme="custom"]:not(.dark) :is([class~="border-neutral-300"], [class~="border-neutral-100"]), +html[data-sidebar-scheme="light"] :where(.app-sidebar, .app-topbar-brand, .mobile-sidebar-sheet [role="dialog"]) :is([class~="border-neutral-300"], [class~="border-neutral-100"]) { border-color: var(--coollabs-line) !important; } +/* Row and section separators (one-sided borders and divide-y) use the softer + divider token, so lists read as one card instead of a stack of boxes. Card + edges (full borders) keep the hairline/line tokens. */ +html:not(.dark) :is([class~="border-b"], [class~="border-t"], [class~="border-y"])[class~="border-neutral-200"], +html:not(.dark) [class~="divide-neutral-200"] > :not(:last-child), +.dark :is([class~="border-b"], [class~="border-t"], [class~="border-y"]):is([class~="dark:border-white/[0.06]"], [class~="dark:border-white/[0.07]"], [class~="dark:border-white/[0.08]"]), +.dark :is([class~="dark:divide-white/[0.06]"], [class~="dark:divide-white/[0.07]"], [class~="dark:divide-white/[0.08]"]) > :not(:last-child) { + border-color: var(--coollabs-divider) !important; +} + html[data-theme="custom"] input:read-only, html[data-theme="custom"] textarea:read-only, html[data-theme="custom"] input:disabled, @@ -1683,7 +1806,7 @@ html[data-theme="custom"] textarea:disabled { } .dark .error-code { - color: var(--color-warning); + color: var(--color-brand-light); } .error-code-danger { @@ -1800,6 +1923,11 @@ html[data-theme="custom"] textarea:disabled { --card-fg: oklch(14.5% 0 0); --card-fg-dim: oklch(40% 0 0); --card-fg-faint: var(--coollabs-subtle); +} + +/* Layout only for cards: wells set their own display (often a Tailwind grid). */ +.application-settings-section, +.empty-state { margin: 0; display: flex; width: 100%; @@ -1842,25 +1970,30 @@ html[data-theme="custom"]:not(.dark) :is(.application-settings-section, .applica background: linear-gradient(var(--well-bg), var(--well-bg)), var(--card-underlay); } -/* Dark mode: top-level cards use the topbar color (see DESIGN.md). */ +/* Dark mode: top-level cards are a near-canvas shade (#121212 under 34% black), + and their well is #121212, a step above it. The top bar matches the canvas + (see the brand shell below). */ .dark :is(.application-settings-section, .empty-state):not(:is(.application-settings-section, .application-settings-section-body) *) { - background: var(--color-panel); + background: color-mix(in srgb, var(--color-card) 66%, black); + box-shadow: none; +} + +.dark :is(.application-settings-section, .empty-state):not(:is(.application-settings-section, .application-settings-section-body) *) > :is(.application-settings-section-body, .empty-state-well) { + --well-bg: var(--color-card); + --well-shadow: none; } /* Dark mode: collection tiles, pagination bars, and tables match the cards. - A tile is a card without a well, so it takes the topbar color. A table (a - shell with a header row) takes the topbar color on the header row and the - box content (well) color on the rows. */ + A tile is #121212, like a card well. A table (a shell with a header row) has + #121212 rows and a header row in the darker card outer color. */ .dark [class~="rounded-xl"][class~="dark:bg-white/[0.05]"] { - background: var(--color-panel); -} - -.dark [class~="rounded-xl"][class~="dark:bg-white/[0.05]"]:has(> [class~="border-b"]:first-child) { - background: color-mix(in srgb, var(--color-panel) 66%, black); + background-color: var(--color-card) !important; + border-color: rgb(255 255 255 / 0.08) !important; + box-shadow: none; } .dark [class~="rounded-xl"][class~="dark:bg-white/[0.05]"] > [class~="border-b"]:first-child { - background: var(--color-panel); + background: color-mix(in srgb, var(--color-card) 66%, black) !important; } /* Brief accent ring when a settings nav sub-item scrolls a section into view. @@ -1903,6 +2036,12 @@ html[data-theme="custom"]:not(.dark) :is(.application-settings-section, .applica background: linear-gradient(var(--card-bg), var(--card-bg)), var(--coollabs-canvas); } +/* Dark mode: the modal markup adds an inline hairline ring outside the card + border, which reads as a light double edge. Keep only the card border. */ +.dark .application-settings-section.application-settings-form { + box-shadow: var(--shadow-modal) !important; +} + /* Modals size to content on large screens */ @media (min-width: 1024px) { .application-settings-section.application-settings-form { @@ -1968,12 +2107,17 @@ html[data-theme="custom"]:not(.dark) :is(.application-settings-section, .applica --coollabs-line: rgb(255 255 255 / 8%); } -/* A well without a card around it sits straight on the canvas. In dark mode its - black fill would make it darker than the page, so it takes the card surface. */ +/* A well without a card around it sits straight on the canvas. In dark mode it + matches a top-level card: #121212 content, and a table header row in the + darker card outer color. */ .dark .application-settings-section-body:not(.application-settings-section *) { - --well-bg: var(--card-bg); + --well-bg: var(--color-card); --well-border: var(--card-border); - --well-shadow: 0 1px 2px rgb(0 0 0 / 5%); + --well-shadow: none; +} + +.dark .application-settings-section-body:not(.application-settings-section *) .data-table-header { + background: color-mix(in srgb, var(--color-card) 66%, black); } /* Floating panels in the well must stay opaque: restore the global dark values. */ @@ -2023,6 +2167,16 @@ html[data-theme="custom"]:not(.dark) :is(.application-settings-section, .applica } } +/* A resource menu that the main sidebar (and the mobile drawer) shows instead + (navbar #main-nav-context-{depth}). */ +.application-settings-navigation.in-main-nav { + display: none; +} + +.application-settings-navigation.in-main-nav ~ * { + grid-column: 1 / -1; +} + @media (min-width: 1280px) { /* Fixed like the main sidebar (3rem โ†’ bottom); left tracks its width (--sidebar-w). Fixed leaves the grid, so the content sibling is pinned to column 2 below. */ @@ -2049,15 +2203,6 @@ html[data-theme="custom"]:not(.dark) :is(.application-settings-section, .applica grid-column: 2; } - /* A resource menu that the main sidebar shows instead (navbar #main-nav-context-{depth}). */ - .application-settings-navigation.in-main-nav { - display: none; - } - - .application-settings-navigation.in-main-nav ~ * { - grid-column: 1 / -1; - } - html:not(.dark) .application-settings-navigation { --rail-fill: var(--coollabs-base); } @@ -2177,6 +2322,16 @@ html[data-theme="custom"]:not(.dark) :is(.application-settings-section, .applica padding-right: 4.25rem; } +.application-settings-workspace .input.input-with-stepper, +.application-settings-form .input.input-with-stepper { + padding-right: 4.25rem; +} + +.application-settings-workspace .number-stepper button:last-child, +.application-settings-form .number-stepper button:last-child { + border-radius: 0 7px 7px 0; +} + .application-settings-workspace .input:focus-visible, .application-settings-workspace .select:focus-visible, .application-settings-form .input:focus-visible, @@ -2391,7 +2546,7 @@ html[data-theme="custom"]:not(.dark) :is(.application-settings-section, .applica /* * Active primary tab styles. * The base rules above set background/color/box-shadow with higher specificity - * than Tailwind utilities (bg-warning/15, text-warning, ring-*), so active + * than Tailwind utilities (bg-brand/15, text-brand-light, ring-*), so active * tabs need an explicit override or they look identical to inactive ones. */ .application-heading-actions .app-tab[aria-current='page'], @@ -2409,15 +2564,15 @@ html[data-theme="custom"]:not(.dark) :is(.application-settings-section, .applica .dark .application-heading-actions .app-tab[aria-current='page'], .dark .application-heading-actions .app-tab.app-tab-active { - background: color-mix(in srgb, var(--color-warning) 15%, transparent); - color: var(--color-warning); - box-shadow: inset 0 0 0 1px color-mix(in srgb, var(--color-warning) 25%, transparent); + background: color-mix(in srgb, var(--color-brand) 15%, transparent); + color: var(--color-brand-light); + box-shadow: inset 0 0 0 1px color-mix(in srgb, var(--color-brand-light) 25%, transparent); } .dark .application-heading-actions .app-tab[aria-current='page']:hover, .dark .application-heading-actions .app-tab.app-tab-active:hover { - background: color-mix(in srgb, var(--color-warning) 20%, transparent); - color: var(--color-warning); + background: color-mix(in srgb, var(--color-brand) 20%, transparent); + color: var(--color-brand-light); } .application-heading-actions .relative > button[x-ref='trigger'] { @@ -2493,12 +2648,30 @@ html[data-theme="custom"]:not(.dark) :is(.application-settings-section, .applica opacity: 0.45; } +/* Same physical edge as highlighted .button: hover lifts, press sinks. */ +.split-action-main:not(:disabled), +.split-action-caret:not(:disabled) { + --button-depth-color: var(--color-coollabs-300); + box-shadow: 0 2px 0 var(--button-depth-color); +} + +.dark .split-action-main:not(:disabled), +.dark .split-action-caret:not(:disabled) { + --button-depth-color: var(--color-brand-depth); +} + +.split-action-main:not(:disabled):hover, +.split-action-caret:not(:disabled):hover { + transform: translateY(-1px); + box-shadow: 0 3px 0 var(--button-depth-color); +} + .split-action-main:focus-visible, .split-action-caret:focus-visible { - outline: none; + outline: 1px solid var(--color-accent); + outline-offset: 2px; position: relative; z-index: 1; - box-shadow: 0 0 0 1px var(--color-accent); } /* Custom listbox (replaces native - -
diff --git a/resources/views/components/application/configuration-sidebar.blade.php b/resources/views/components/application/configuration-sidebar.blade.php index 29f56af40d..592c81e48f 100644 --- a/resources/views/components/application/configuration-sidebar.blade.php +++ b/resources/views/components/application/configuration-sidebar.blade.php @@ -193,25 +193,14 @@ 'project.application.configuration' => array_values(array_filter([ ['id' => 'application-details-section', 'label' => 'Application details'], ['id' => 'access-section', 'label' => 'Access'], - ['id' => 'build-pipeline-section', 'label' => 'Build pipeline'], - $isComposeApp ? null : ['id' => 'container-image-section', 'label' => 'Container image'], - $isComposeApp ? null : ['id' => 'networking-section', 'label' => 'Networking'], - $isComposeApp ? null : ['id' => 'runtime-section', 'label' => 'Runtime'], - $isComposeApp ? null : ['id' => 'security-section', 'label' => 'Security'], - ['id' => 'deployment-lifecycle-section', 'label' => 'Deployment lifecycle'], - $isComposeApp ? null : ['id' => 'container-labels-section', 'label' => 'Container labels'], + ['id' => 'build-pipeline-section', 'label' => 'Build & deploy'], + $isComposeApp ? null : ['id' => 'container-section', 'label' => 'Container'], ])), - 'project.application.advanced' => array_values(array_filter([ - ['id' => 'advanced-build-section', 'label' => 'Build'], + 'project.application.advanced' => [ + ['id' => 'advanced-build-section', 'label' => 'Build & deploy'], ['id' => 'advanced-container-section', 'label' => 'Container'], - $application->git_based() ? ['id' => 'advanced-deployment-section', 'label' => 'Deployment'] : null, - $application->git_based() ? ['id' => 'advanced-git-section', 'label' => 'Git'] : null, - $isComposeApp ? ['id' => 'advanced-compose-section', 'label' => 'Docker compose'] : null, ['id' => 'advanced-proxy-section', 'label' => 'Proxy'], - ['id' => 'advanced-operations-section', 'label' => 'Operations'], - ['id' => 'advanced-logs-section', 'label' => 'Logs'], - $isComposeApp ? null : ['id' => 'advanced-gpu-section', 'label' => 'GPU'], - ])), + ], 'project.application.webhooks' => [ ['id' => 'deploy-webhook-section', 'label' => 'Deploy webhook'], ['id' => 'manual-git-webhooks-section', 'label' => 'Manual Git webhooks'], @@ -225,25 +214,10 @@ ])), 'project.application.healthcheck' => [ ['id' => 'healthcheck-configuration-section', 'label' => 'Configuration'], - [ - 'id' => $application->health_check_type === 'cmd' - ? 'healthcheck-command-section' - : 'healthcheck-request-section', - 'label' => $application->health_check_type === 'cmd' ? 'Command' : 'HTTP request', - ], ['id' => 'healthcheck-timing-section', 'label' => 'Timing and retries'], ], - 'project.application.rollback' => [ - ['id' => 'rollback-retention-section', 'label' => 'Image retention'], - ['id' => 'rollback-images-section', 'label' => 'Available images'], - ], - 'project.application.resource-limits' => [ - ['id' => 'cpu-limits-section', 'label' => 'CPU'], - ['id' => 'memory-limits-section', 'label' => 'Memory'], - ], 'project.application.resource-operations' => [ - ['id' => 'clone-destination-section', 'label' => 'Clone destination'], - ['id' => 'clone-environment-section', 'label' => 'Clone environment'], + ['id' => 'clone-resource-section', 'label' => 'Clone resource'], ['id' => 'move-resource-section', 'label' => 'Move resource'], ], ]; @@ -253,11 +227,11 @@ @endphp @php - // The main sidebar shows this menu on wide screens (see components/navbar.blade.php). + // The main sidebar and the mobile drawer show this menu (see components/navbar.blade.php). registerMainNavContext(0, 'Dashboard', 'Back to the main menu'); @endphp -{{-- Rendered twice: in the page (below xl), and teleported into the main sidebar (xl and up). +{{-- Rendered twice: in the page (CSS hides it; it keeps the command palette items), and teleported into the main sidebar. navOnly renders only the sidebar copy, as the parent menu of a deeper page (e.g. a backup). --}} @foreach ($navOnly ? ['main-nav'] : ['page', 'main-nav'] as $placement) @if ($placement === 'main-nav') @@ -265,19 +239,15 @@
@else