- Part of #1404.
- Evolves API to use NetworkResource that conforms to ManagedResource.
- BREAKING CHANGE - compile time impact due to changes to Swift client
API signatures. No change to persistent data or API server XPC
protocols.
## Type of Change
- [x] Bug fix
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation update
## Motivation and Context
`CLITest.run(arguments:stdin:currentDirectory:env:)` reads stdout to EOF
before stderr begins draining
(`Tests/CLITests/Utilities/CLITest.swift:182-183`). If a child process
writes more than a pipe-buffer's worth of data to stderr (~64 KB on
macOS), the child blocks in `write()` on stderr while we block in
`readDataToEndOfFile()` on stdout, and neither side makes progress until
`process.waitUntilExit()` returns. This is a latent deadlock that any
future test or CLI verbosity bump can trigger.
This PR drains both streams concurrently using `readabilityHandler`
closures backed by `Mutex<Data>` buffers. After
`process.waitUntilExit()` returns the handlers are cleared and
`readDataToEndOfFile()` flushes any bytes the kernel buffered between
the last handler invocation and exit. The error path also clears
handlers so a failed `process.run()` does not leak callbacks. `Mutex`
matches the locking primitive the file already uses for `commandSeq`.
Fixes#1456
## Testing
- [x] Tested locally
- [ ] Added/updated tests
- [ ] Added/updated docs
`swift build --target CLITests` passes against the change. `make
swift-fmt-check` is clean. Behavior is unchanged for any test command
that previously fit within a single pipe buffer, so the existing
CLITests still exercise the helper end-to-end.
I did not add a new test that emits >64 KB to stderr because every
existing CLITest invocation goes through `executablePath` (the
`container` binary), and reproducing the deadlock requires a child that
emits a controllable amount on stderr. Happy to follow up with a small
refactor that extracts the drain into a static helper plus a regression
test that drives it via `/bin/sh` if that would be useful.
---------
Signed-off-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
- Don't prefix match localhost, use exact
match only.
- Only check default local domain if one is
defined.
- Use IPv4Address for private IP CIDR tests.
Changes ssh socket path in container to
`/var/host-services/ssh-auth.sock` as the path `/run` (where the ssh
socket is mounted initially) is often mounted again by init systems, and
the ssh socket is hidden.
## Type of Change
- [X] Bug fix
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation update
## Motivation and Context
Makes ssh socket forwarding reliable.
## Testing
- [X] Tested locally
- [ ] Added/updated tests
- [ ] Added/updated docs
- Fixes single file mounts. Closes#1251.
- Adds journal modes to `EXT4.Formatter`.
- vminitd fix for client timeouts resulting in EBADF and other minor
improvements.
- Reenable block size for EXT4 formatter, and other fixes.
- Update vminit client and server to grpc-swift-2.
This PR resolves#1343.
This PR depends on apple/container-builder-shim#72.
Do not create staging directory under build context, but pass
dockerignore file bytes to the container-builder-shim.
## Type of Change
- [x] Bug fix
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation update
## Motivation and Context
[Why is this change needed?]
## Testing
- [x] Tested locally
- [ ] Added/updated tests
- [ ] Added/updated docs
- Closes#874.
- Image save now fails fast with a clear error when the requested
platform is not available locally.
- The message lists the platforms that ARE available so the user
knows what they can save.
This PR fixes#357, passing `SSH_AUTH_SOCK` env variable from current
terminal to the `SandboxService` so that the container can mount the
correct ssh auth socket. For that, it introduces `env` parameters to
`bootstrap` RPC of both `ContainersService` and `SandboxService`. This
parameter is used only for passing `SSH_AUTH_SOCK` now, but can be
extended to pass more runtime env variables.
This PR is a follow up PR of #1214.
## Type of Change
- [X] Bug fix
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation update
## Motivation and Context
Container run `--ssh` was inheriting `SSH_AUTH_SOCK` env variable from
launchd, not from current terminal.
## Testing
- [X] Tested locally
- [ ] Added/updated tests
- [ ] Added/updated docs
This change migrates plugin config format away from using JSON to use
TOML.
- Addresses the plugin portion of discussion #1336
## Type of Change
- [ ] Bug fix
- [X] New feature
- [ ] Breaking change
- [ ] Documentation update
## Motivation and Context
Context within https://github.com/apple/container/discussions/1336
> We're looking to improve the user experience and overall functionality
of setting defaults for container to better enable future use cases.
>
> Today container uses macOS's
[UserDefaults](https://developer.apple.com/documentation/foundation/userdefaults)
to configure settings needed at runtime. As mentioned in
https://github.com/apple/container/issues/608, UserDefaults may be
idiomatic for macOS, but they apply globally to all sessions and do not
handle representing complex, hierarchical data well. Additionally, we
currently have two ways of setting these defaults, either directly with
macOS's defaults command or through container system property.
>
> https://github.com/apple/container/issues/608 proposes moving to use
environment variables in place of UserDefaults. However, we do not
believe this is sufficient. Environment variables are not in a
consistent location, are not sourced from data, and also do not handle
representing complex, hierarchical data well.
As part of this migration to TOML for user settings we want to also move
plugin settings to use TOML so that we can have a singular file format
for `container` configurations.
## Testing
- [X] Tested locally
- [X] Added/updated tests
- [ ] Added/updated docs
- Adds a color progress output mode (`--progress color`)
that renders ANSI-colored progress output with visual
differentiation between progress states.
- Like `ansi` mode, this feature requires a TTY.
- Closes#1366
Closes#1352
Containerization has had support for a bit, it was just never brought
over here. It's exposed on the CLI via the classic `--cap-add` and
`--cap-drop` UX.
- Closes#1359.
- Create a ResourceLabels type and extract the label validation from
NetworkConfiguration into the new type.
- Create a base AppError type that is compatible with structured logging
and delegates message presentation to the error receiver.
- Define LabelError over AppError for label validation.
- Slightly reworks NetworkConfiguration entity migration code in
NetworksService.
## Motivation and Context
This PR consolidates duplicated list-output formatting across the CLI
into shared rendering infrastructure.
Currently, each list command has its own copy of the same
json/quiet/table branching. This PR pulls that into shared rendering
infrastructure in `ContainerCommands`:
- `ListDisplayable` protocol for table + quiet output
- `renderJSON`, `renderTable`, `renderList` as pure functions that
return strings
- `emit()` as the single stdout boundary (no-ops on empty strings to
avoid blank-line regressions)
- `JSONOptions` so all JSON encoding goes through one path, including
volume inspect's pretty + ISO 8601 case
JSON encoding remains separate from display formatting: each command
still chooses its own JSON model, while `ListDisplayable` is used only
for table and quiet output. `ImageList` remains the intentional
exception for quiet mode so it can avoid unnecessary async work.
This change also replaces inline `JSONEncoder` usage with `renderJSON`,
removes the old `Codable+JSON.swift` helper, and moves `TableOutput` and
`ListFormat` into `ContainerCommands`. It also adds unit tests for the
shared rendering helpers and expands integration coverage for image,
network, and registry list formatting.
## Testing
- [x] Tested locally
- [x] Added/updated tests
- [ ] Added/updated docs
- Closes#387.
- Override the swift-argument-parser `help` command
with a version that prints the reformatted help that
includes plugin information.
---------
Signed-off-by: ChengHao Yang <17496418+tico88612@users.noreply.github.com>
- Clarified the `.plain` progress contract. This also
documents how `clearOnFinish` interacts with plain
output. Added test coverage for the current behavior.
- Adds a `plain` progress output mode (`--progress plain`)
that outputs newline-separated progress lines without
ANSI escape sequences. This is useful for CI environments,
piped output, and non-TTY contexts where ANSI cursor
control is not supported. Unlike `--progress ansi`, plain
mode works even when stderr is not a TTY.
- Closes#1365
- Related: #641, #808
- TestCLIStatus isn't in the integration test.
- Update the output to align with the current output.
- Follow up #1193
---------
Signed-off-by: ChengHao Yang <17496418+tico88612@users.noreply.github.com>
Same as we'd done with kill.
```
➜ container git:(stop-kill-throw-error) ✗ ./bin/container stop foo bar baz
Warning! Running debug build. Performance may be degraded.
Error: internalError: "failed to stop container" (cause: "notFound: "container with ID baz not found"")
internalError: "failed to stop container" (cause: "notFound: "container with ID bar not found"")
internalError: "failed to stop container" (cause: "notFound: "container with ID foo not found"")
➜ container git:(stop-kill-throw-error) ✗ ./bin/container kill foo bar baz
Warning! Running debug build. Performance may be degraded.
Error: internalError: "failed to kill container" (cause: "notFound: "container with ID foo not found"")
internalError: "failed to kill container" (cause: "notFound: "container with ID bar not found"")
internalError: "failed to kill container" (cause: "notFound: "container with ID baz not found"")
➜ container git:(stop-kill-throw-error) ✗ ./bin/container delete foo bar baz
Warning! Running debug build. Performance may be degraded.
Error: internalError: "failed to delete container" (cause: "notFound: "container with ID foo not found"")
internalError: "failed to delete container" (cause: "notFound: "container with ID baz not found"")
internalError: "failed to delete container" (cause: "notFound: "container with ID bar not found"")
```
## Type of Change
- [x] Bug fix
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation update
## Motivation and Context
`TestCLIVersion` isn't in the integration test; we should include this.
And update the output to align with the real output.
Follow up #911
## Testing
- [x] Tested locally
- [x] Added/updated tests
- [ ] Added/updated docs
---------
Signed-off-by: ChengHao Yang <17496418+tico88612@users.noreply.github.com>
- Closes#1268.
- The types we were using weren't very usable with Swift 6 structured
concurrency.
- Implements just the subset of records that we use.
- Use notImplemented instead of formatError for unknown record types.
- Use pure actor for LocalhostDNSHandler now that we have sendable
types.
- Use DNSName as key for table lookups in LocalhostDNSHandler and
HostTableResolver.
- Utilize dot-suffixed domain names everywhere in the lookup chain.
- Huge thanks to @manojmahapatra and @katiewasnothere for their
diligent and patient review :D
This PR fixes `container build` to use docker specific ignore (#1169).
This PR relies on apple/container-builder-shim#68.
When docker specific ignore file is detected, it creates a hidden docker
directory (i.e., `.hidden-docker-dir`) under the build context, and
places Dockerfile and ignore file under there. This hidden directory is
passed to the builder shim and used by buildkit daemon to correctly
figure out the docker specific ignore file.
Docker specific ignore file is detected regardless of whether i) they
are under nested directory, ii) outside build context, iii) using custom
names. If docker specific ignore is not provided, hidden directory is
also not created, and buildkit daemon will use the docker ignore file at
build context root as usual.
## Type of Change
- [X] Bug fix
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation update
## Motivation and Context
Docker specific ignore works.
## Testing
- [X] Tested locally
- [ ] Added/updated tests
- [ ] Added/updated docs
- Closes#1252 (part of #913).
- Commands that accept `--platform` currently require it
to be passed explicitly every time. Users working
consistently with a non-native platform (e.g. `linux/amd64`
on Apple Silicon) have no way to set a default, unlike
Docker's `DOCKER_DEFAULT_PLATFORM`.
- This adds support for the `CONTAINER_DEFAULT_PLATFORM`
environment variable as a fallback when `--platform` is not
explicitly provided.
- Previously we weren't tolerating a throw from
the check.
- We have #1250 open for root cause analysis of
this particular throw that trips the test up.
This PR fixes#1277.
`SandboxService.waiters` had a consistency issue (not exactly race).
`SandboxService.wait` XPC can be executed on arbitrary `id`, and it will
hang forever if no other handler resumes it. Without knowing this
internal, the high level entity can run into this issue, and deadlock.
This PR simplifies the mental model: **`SandboxService.waiters[id]:
ExitWaiter(continuations, exitCode)` can only be in three states: i)
non-existing, ii) existing with nil `exitCode`, and iii) existing with
concrete `exitCode`.**
**If it is non-existing, no handler has been registered to resume it
later. If existing with nil `exitCode`, It is guaranteed the registered
`continuations` will be resumed later with a concrete `exitCode`.
Finally, if already a concrete `exitCode`, a handler has been
registered, and already resumed (with that `exitCode`).**
Thus, `SandboxService.wait` should return immediately if `waiters[id]`
is non-existing or existing with a concrete `exitCode` (as no handler
will resume it later). It should only block when `waiters[id]` is
existing with nil `exitCode` as it is guaranteed to be resumed later. By
doing so, we can guarantee there is no deadlock at all.
For that this PR does followings:
1. Introduce `ExitMonitor` class to updates `continuations` and
`exitCode` all together atomically. Initially, `state` variable saved
the `exitCode`, but it cannot be tied with `continuations` as they are
protected by different primitives (i.e., lock and actor).
2. Gather `waiters` related operations into a single actor method,
guaranteeing those are performed atomically under actor
protection---i.e., we actually don't need Mutex here.
3. Ensure initialized `waiters` are released (i.e., resumed) later
(under any possible circumstances).
4. Move `process.wait` after `process.start` in `io.handleProcess` to
run `SandboxService.wait` only after the `waiters[id]` is initialized.
By doing fourth step, we can guarantee `SandboxService.wait` can meet
only one of two following `ExitMonitor` state: i) existing with nil
`exitCode`, or ii) existing with concrete `exitCode` (in case the
process exited too early). In both cases, `exitCode` is preserved and
returned.
## Type of Change
- [X] Bug fix
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation update
## Motivation and Context
[Why is this change needed?]
## Testing
- [X] Tested locally
- [ ] Added/updated tests
- [ ] Added/updated docs
Fix container hang when starting an invalid executable (#1277).
Ensure the `SandboxService.waiters` updated atomically, preventing a slow `wait` from registering continuation after the `SandboxService` has been stopped.
Current `ProcessIO.handleProcess` has races between `process.wait()` and
`process.start()`. In `process.wait`, `SandboxService.wait` registers
and block on a continuation so that later exit of that `LinuxContainer`
can resume it.
Even if `SandboxService.wait` checks the container state, there can be a
race so that `SandboxService.startInitProcess` fails earlier with no
continuation to resume, then `SandboxService.wait` registers the
continuation, which forever hang the thread.
- Closes#1270.
- The current test is a bit baroque. It tries to run `nc`, either to
`8.8.8.8:53` or a proxy server port if `HTTP_PROXY` is in the
environment. It seems that for an alpine image, a proxy-aware
`Dockerfile` and a `RUN apk install curl` directive is a more natural
test of "network access".
- Current DirectoryWatcher fails if `/etc/resolver` does not exist. This
PR fixes DirectoryWatcher to handle non-existing `/etc/resolver`
directory. If that directory does not exist, it first watches `/etc`
directory to check if `/etc/resolver` directory is created later. Once
it detects new `/etc/resolver` directory, it starts watching new DNS
resolver files there.
- This PR also fixes to log the exception thrown by API server's tasks.
- Closes#1207
Closes#1225
Add a flag to signify that we'd like to run a minimal init process that
can reap zombie processes. The actual support for this is in the
Containerization library so the plumbing here is very simple.
J LoganandGitHub
ChengHao YangandGitHub
Novikov KirillandGitHub
Noah ThorntonandGitHub
Nikunj ThakurandGitHub
Aditya RamaniandGitHub
Danny CanterandGitHub
RajandGitHub
Manoj MahapatraandGitHub
Dmitry KovbaandGitHub
Ryan GouldenandGitHub
Willjianger9andGitHub