fix(deployment): remove the helper container before restarting it with the same name

Docker frees the name of a stopped --rm container in the background, so
the second helper docker run for build secrets could fail with a name
conflict. Remove the old helper with docker rm -f before the run.

Closes #9363

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

CHANGELOG.md

@@ -20,6 +20,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 
 ### Fixed
 
+- Fix deployments that failed with "The container name ... is already in use" when **Use build secrets** is on. To set the real `SOURCE_COMMIT`, Coolify starts the helper container a second time with the same name. Before, Coolify stopped the old helper and started the new one immediately, but Docker removes a stopped `--rm` container in the background, so the name was sometimes still in use. Coolify now removes the old helper and waits until Docker frees the name ([#9363](https://github.com/coollabsio/coolify/issues/9363))
 - Fix Docker builds that failed with "BuildKit is enabled but the buildx component is missing or broken" when the application has a build-time variable `HOME` or `DOCKER_CONFIG`. Coolify loads the build-time variables before it runs `docker build` in the helper container, so the variable moved the Docker CLI away from its plugins in `/root/.docker/cli-plugins`. The Docker CLI in the helper now always uses `/root/.docker`. A `HOME` build argument keeps its value ([#9726](https://github.com/coollabsio/coolify/issues/9726))
 - Keep the trailing newline of multiline environment variables, for example an SSH private key that a Dockerfile mounts as a build secret. Before, Coolify removed the edge whitespace of every value, so OpenSSH rejected the key with "error in libcrypto". A value with more than one line is now saved exactly as you enter it, in the UI and in the API. Single-line values still lose their edge whitespace. Save a variable again to add the newline that Coolify removed before ([#9438](https://github.com/coollabsio/coolify/issues/9438))
 - Use the same `COOLIFY_BRANCH` value in all parts of a preview deployment: `pull/<pull request number>/head`. Before, a variable that refers to `${COOLIFY_BRANCH}` got the branch of the application (for example `main`), but the container got `pull/<number>/head`. Other build packs got `pull/<number>/head:pr-<number>-coolify`, and applications from a public repository got the branch of the application ([#9572](https://github.com/coollabsio/coolify/issues/9572))

app/Jobs/ApplicationDeploymentJob.php

@@ -3250,7 +3250,8 @@ class ApplicationDeploymentJob implements ShouldBeEncrypted, ShouldQueue
             $this->application_deployment_queue->addLogEntry('Preparing container with helper image with updated envs.');
         }
 
-        $this->graceful_shutdown_container($this->deployment_uuid, skipRemove: true);
+        // "docker stop" returns before the --rm auto-removal frees the name, so the "docker run" below can conflict with it.
+        $this->removeContainerWithTimeout($this->deployment_uuid);
         $this->execute_remote_command(
             [
                 $runCommand,
@@ -3266,13 +3267,10 @@ class ApplicationDeploymentJob implements ShouldBeEncrypted, ShouldQueue
 
     private function restart_builder_container_with_actual_commit()
     {
-        // Stop the current helper container (no need for rm -f as it was started with --rm)
-        $this->graceful_shutdown_container($this->deployment_uuid, skipRemove: true);
-
         // Clear cached env_args to force regeneration with actual SOURCE_COMMIT value
         $this->env_args = null;
 
-        // Restart the helper container with updated environment variables (including actual SOURCE_COMMIT)
+        // prepare_builder_image() removes the current helper before it starts the new one
         $this->prepare_builder_image(firstTry: false);
     }
 

tests/Feature/DeploymentSensitiveCommandFailureTest.php

@@ -658,3 +658,26 @@ it('mounts the seccomp profile from custom docker options into the helper contai
         ->and($helperCommands[0])->toContain('type=bind,source=/etc/docker/seccomp-profile.json,target=/etc/docker/seccomp-profile.json,readonly')
         ->and($exception->getMessage())->toContain('bind source path does not exist: /etc/docker/seccomp-profile.json');
 });
+
+it('removes the old helper synchronously before it starts the helper again with the same name', function () {
+    [$job, $reflection] = makeSensitiveCommandFailureJob();
+    $server = readSensitiveFailureJobProperty($job, $reflection, 'server');
+    setSensitiveFailureJobProperties($job, $reflection, ['destination' => $server->standaloneDockers()->firstOrFail()]);
+
+    $commands = [];
+    Process::fake(function (PendingProcess $process) use (&$commands) {
+        $commands[] = remoteProcessScript($process);
+
+        return Process::result(output: str_contains(remoteProcessScript($process), 'echo $HOME') ? '/root' : 'NOK');
+    });
+
+    invokeSensitiveFailureJobMethod($job, $reflection, 'restart_builder_container_with_actual_commit');
+
+    $runIndex = collect($commands)->search(fn (string $command) => str_contains($command, 'docker run -d'));
+    $beforeRun = collect($commands)->take($runIndex);
+
+    // "docker stop" lets the --rm auto-removal hold the name after it returns; "docker rm -f" frees the name before it returns.
+    expect($runIndex)->toBeInt()
+        ->and($beforeRun->contains(fn (string $command) => str_contains($command, 'docker rm -f') && str_contains($command, 'sensitive-failure-deployment')))->toBeTrue()
+        ->and($beforeRun->contains(fn (string $command) => str_contains($command, 'docker stop')))->toBeFalse();
+});