Each container runs in its own guest VM sized to
`--memory` with no swap, so the guest kernel's
stock `vm` sysctl defaults are hit far too easily:
- `vm.overcommit_memory=0` (heuristic overcommit)
rejects an oversized `mmap()` upfront whenever the
reservation exceeds the small, swap-less VM's free
RAM — even if the memory is never touched —
returning `ENOMEM`.
- `vm.max_map_count=65530` caps per-process
mapping count, which mapping-heavy applications
(e.g. Elasticsearch, many JVMs) can exceed.
- When container is not running, the runtime helper
traverses the container's root fs and writes it to the
specified tar archive or stdout.
- When the container is running, the helper performs
the same operation but wraps it in freeze/thaw
to ensure data integrity for the resulting archive.
- Closes#2001.
- Handle "container exists" error gracefully instead
of failing, when trying to start the buildkit container.
- Move build tests to parallel suites, while the builder
lifecycle tests remain serial. Parallel builds don't
use the fixture lock that deletes and restarts the
builder and runs a build block in isolation.
Closes https://github.com/apple/container/issues/1687
The default kernel archive is downloaded from a remote release URL
during first-run setup and via `container system kernel set
--recommended`. Previously, the archive contents were not verified after
download, so integrity depended on HTTPS and the release artifact
remaining unchanged.
This change adds digest verification for kernel archives. The
recommended/default kernel now has pinned digest metadata using an
algorithm-prefixed value such as `sha256:<hex>`. `container system
kernel set --tar` accepts `--digest`; remote tar URLs require it, and
local tar archives can also be verified before unpacking and
installation.
The system config also supports `kernel.digest`, and a custom
`kernel.url` must provide a digest for that archive.
- Closes#1812.
- The network plugin is the source of truth for the variant, if any,
that applies to the network. Resolving a missing variant configuration
option in the API server can create a situation where the variant the
runtime uses for interface selection is incorrect.
- Adds serial suites trait to tests to see whether it helps current CI
issues.
## Type of Change
- [x] Bug fix
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation update
## Motivation and Context
Fixes a flaw in our interface strategy logic.
## Testing
- [x] Tested locally
- [ ] Added/updated tests
- [ ] Added/updated docs
- Closes#1756.
- `RuntimeService.gracefulStopContainer(_:signal:timeout:)`
wraps the graceful-stop attempt in `do { … } catch {}`. The
empty catch silently discards any thrown error before falling
through to the unconditional `lc.stop()`. It is the only catch
in this file that does not log; every other one uses
`self.log.error(…, metadata: ["error": "\(error)"])`.
- This adds a single log line matching that convention, so
a failed graceful stop (and the resulting fall-through to a
forced VM shutdown) is more diagnosable. The intentional
fall-through to `lc.stop()` is unchanged.
- Instead of using force-unwrap to append to a list-valued
dictionary entry that should always exist, assign the value
with a default fallback and append to the (non-optional)
result.
- In `ProcessIO.swift`, the readability handlers for stdout and stderr
used `try!` when writing data to the output file handles. This would
cause crashes If the pipe is broken such that the force-try executes.
- Changed to handle a failed write similarly to an EOF.
- Closes#1750.
- Applies permission code used for the `--ssh` mount to all
host-to-container socket mounts.
- Adds a user option to the `doExec` test support function.
- Updates the `testRunCommandUnixSocketMount` to install `nc` in the
test container, and check the socket permission, and check the mounted
socket using `nc` as the guest user.
This also includes custom kernels for container machine. Its required
with nested virt as CONFIG_KVM needs to be enabled.
---------
Signed-off-by: michael_crosby <michael_crosby@apple.com>
Remove manually specified default value from help string since
ArgumentParser already appends it automatically from the property's
default value.
Signed-off-by: Charlie Le <charlie_le@apple.com>
## Type of Change
- [ ] Bug fix
- [x] New feature
- [ ] Breaking change
- [ ] Documentation update
## Motivation and Context
`container` runs each workload in an ephemeral VM, so there's no
built-in way to keep a persistent Linux environment you can log into and
work in. `container machine` adds one.
A container machine is a lightweight, persistent, and integrated Linux
environments that feel like an extension of your Mac, created from
standard OCI images with a familiar UX. The login user matches your host
account with passwordless `sudo`, your home directory is mounted inside
the VM, and each machine keeps its filesystem and runs the image's own
init system (such as`systemd` or `openrc`).
```bash
container machine create alpine:3.22 --name my-machine
container machine run -n my-machine # interactive shell
container machine set -n my-machine cpus=4 memory=8G
```
Subcommands: `create`, `run`, `list` (`ls`), `inspect`, `set`,
`set-default`, `logs`, `stop`, `delete` (`rm`); `m` aliases `machine`.
Docs added to `docs/command-reference.md` (Machine Management) and
`docs/how-to.md` ("Use container machines").
## Testing
- [x] Tested locally
- [x] Added/updated tests
- [x] Added/updated docs
Signed-off-by: Raj Aryan Singh <rajaryan_singh@apple.com>
Co-authored-by: Jaewon Hur <jaewon_hur@apple.com>
Co-authored-by: John Logan <john_logan@apple.com>
Co-authored-by: Michael Crosby <michael_crosby@apple.com>
Co-authored-by: Eric Ernst <eric_ernst@apple.com>
Co-authored-by: Danny Canter <danny_canter@apple.com>
- This PR changes the shape of the JSON
output for image resources to align closer with
`VolumeResource` and `NetworkResource`.
This includes adding `creationDate` in the
`configuration` section of the image output.
- This PR additionally cleans up some of the logic
around the `ImageResource` struct construction.
Signed-off-by: Kathryn Baldauf <k_baldauf@apple.com>
- Closes#1647.
- `id` will become a system assigned (Docker-like) identifier
for the managed resource, and `configuration.name` is the
user-assigned name.
Today when we send a signal we don't wait for the container to exit, as
we don't know what signals the program will actually do anything with.
However, sigkill does not fit this mold, and we should wait for the
container to exit (or be removed for --rm containers).
Normalize JSON output for network and volume resources. (#1624)
- Reworks both JSON output presets to use sorted keys, ISO timestamps.
`compact` is used for `ls` output, and `pretty` is used for `inspect`.
- Extracts non-DRY option configuration into presets.
- Closes#1623.
- Closes#1625.
- For now, the ImageResource compliance happens
at the CLI level. We will work on pushing that into or
closer to the APIServer at a later time.
## Testing
- [x] Tested locally
---------
Signed-off-by: Kathryn Baldauf <k_baldauf@apple.com>
- Closes#1526 and #1527.
## Type of Change
- [x] Bug fix
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation update
## Motivation and Context
This PR fixes `system df` to report actual on-disk allocated bytes
(content blobs + snapshots) instead of summing per-image snapshot sizes.
Orphaned blobs are now included as reclaimable, and storage shared
across tags is no longer double counted. Also consolidates three
identical `calculateDirectorySize` implementations into a shared
`FileManager.allocatedSize(of:)` extension.
## Testing
- [x] Tested locally
- [x] Added/updated tests
- [ ] Added/updated docs
- Refactor network model types: replace `NetworkState` enum and
phase-based NetworkStatus with a flat `NetworkStatus` struct.
- Simplify API server ↔ plugin protocol: plugin `status()` returns
runtime status only, API server owns configuration.
- `NetworksService` `list()`/`create()` now return `NetworkResource`
directly.
- Remove lifecycle phase checks and state machine guards throughout CLI
and API server.
- `variant` is plugin-specific, it's not a required property. This PR
replaces `NetworkPluginInfo` with a `plugin` name property on
`NetworkConfiguration` and an `options` list similar to that for
volumes.
- Moved `variant` to the option list.
- Part of #1404.
- Updates containerization to 0.33.2.
- Reorganizes network plugin targets into:
- `ContainerNetworkClient` - network plugin client and default types
- `ContainerNetworkServer` - separate protocols for `Network` which
manages the underlying virtual network, `NetworkService`, which takes a
network and implements the API, and an actor `NetworkHarness` that
marshals between the API and the XPC protocol. The service-harness
separation will help us ensure XPC protocol compatibility in both
directions as we evolve the plugin APIs.
- Removes `disableAllocator()` which is no longer used since #1545
switched over to using XPC connections between runtime and network
plugin instances to track whether a network has attached containers.
- Closes#1610.
- Discovered, and originally filed as a security advisory, by:
PresidentL <131139636+liyander@users.noreply.github.com>.
- `PublishPort` currently can store invalid combinations of starting
port and range that can overflow UInt16 values when summed, crashing the
process.
- Updates `PublishPort` to validate inputs on initialization.
## Type of Change
- [x] New feature
## Motivation and Context
Related to https://github.com/apple/container/issues/1404.
This PR adds the initial work to have volume resources conform to
ManagedResource, in alignment with other resources such as networks (see
[here](https://github.com/apple/container/pull/1421)). Further work is
necessary to move the use of `VolumeResource` down to the APIServer (in
the VolumesService) and in the volume client. Volumes do not currently
have any plugin or runtime state, so that information is not included in
the `VolumeResource`, but could be added later if needed.
## Testing
- [x] Tested locally
Signed-off-by: Kathryn Baldauf <k_baldauf@apple.com>
## Type of Change
- [x] Bug fix
## Motivation and Context
Ensure all fields are initialized in the management flags' init
function. Without this change, if someone calls init() on this set of
flags, they will get an error like
```
Can't read a value from a parsable argument definition.
This error indicates that a property declared with an `@Argument`,
`@Option`, `@Flag`, or `@OptionGroup` property wrapper was neither
initialized to a value nor decoded from command-line arguments.
To get a valid value, either call one of the static parsing methods
(`parse`, `parseAsRoot`, or `main`) or define an initializer that
initializes _every_ property of your parsable type.
```
## Testing
- [x] Tested locally
Signed-off-by: Kathryn Baldauf <k_baldauf@apple.com>
- Closes#1581.
- This adds stop signal support to the cli. The priority is:
1. If an explicit stop signal is passed on the cli use this.
2. If not, check if there is a stop signal in the image config.
3. Finally, use the default (TERM).
- Replace `URL` with `FilePath` in `container copy` (#1557).
In addition, make `copyIn`/`copyOut` API to use `String` for path as we
need to preserve the trailing slash to the `LinuxContainer.copy`---i.e.,
this trailing slash is used in `LinuxContainer.copy` to determine copy
behavior.
## Type of Change
- [ ] Bug fix
- [x] New feature
- [ ] Breaking change
- [x] Documentation update
## Motivation and Context
Adds the `container copy` (aliased as `cp`) command to copy files
between a running container and the local filesystem.
I saw [#1023](https://github.com/apple/container/pull/1023) and the
feedback from @dcantah — the previous attempt relied on tar being
installed inside the container.
This implementation takes the recommended approach:
file transfers go through the guest agent via the existing
`copyIn`/`copyOut` methods on the core `Containerization`, with no
dependency on container tooling.
## Testing
- [x] Tested locally
- [x] Added/updated tests
- [x] Added/updated docs
---------
Co-authored-by: jwhur <57657645+JaewonHur@users.noreply.github.com>
- This change adds two types to support opaque runtime
data passing through the APIServer:
- RuntimeConfiguration `data`
- RuntimeLinuxData type
- LinuxRuntimeData defines runtime specific information.
With this change, it will support `variant` only. The
optional RuntimeConfiguration `data` field encodes
runtime specific data to pass through the APIServer.
It is decoded as needed by the runtime.
- The idea is to eventually move all runtime specific data
into the `data` field so that the APIServer is only aware
of generic container information.
- Closes#1318.
- Closes#1378.
- Reduces the complexity and coupling for IP allocation.
- Runtimes connect to networks for the life of the running container.
The runtime shuts down on connection loss.
- Networks automatically deallocate a runtime's IP address and hostname
record on connection loss.
- Removes AllocatedAttachment as this is no longer necessary. The
`bootstrap()` XPC now takes a `NetworkBootstrapInfo` array which
parallels the attachments in the bundle config and provides the network
plugin attributes needed to create VM network interface configurations.
## Type of Change
- [x] Bug fix
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation update
## Motivation and Context
Simplify IP allocation and make deallocation more reliable.
## Testing
- [x] Tested locally
- [ ] Added/updated tests
- [ ] Added/updated docs
- Part of #1318.
- Part of #1378.
- Removes network plugin `deallocate()`, and allocate takes an
`XPCServerSession` that registers an `onDisconnect` handler that
performs deallocation.
- ContainerService now tracks `networkSessions` for allocations.
- Closes#1517.
- Using URL for filesystem paths is bad practice. FilePath is safer and
more ergonomic.
- Sibling of #1480 (HostDNSResolver), same pattern.
Kathryn BaldaufandGitHub
Aditya RamaniandGitHub
adityabagchi24andGitHub
Saehej KangandGitHub
divyanshandGitHub
0x4C33andGitHub
Hugo, CY LAUandGitHub
Bhavesh VarmaandGitHub
SEPURI-SAI-KRISHNAandGitHub
Michael CrosbyandGitHub
Charlie LeandGitHub
Chris GeorgeandGitHub
jwhurandGitHub
AJ EmoryandGitHub
Shaurya SinghandGitHub