## Type of Change
- [ ] Bug fix
- [x] New feature
- [ ] Breaking change
- [ ] Documentation update
## Motivation and Context
`container` runs each workload in an ephemeral VM, so there's no
built-in way to keep a persistent Linux environment you can log into and
work in. `container machine` adds one.
A container machine is a lightweight, persistent, and integrated Linux
environments that feel like an extension of your Mac, created from
standard OCI images with a familiar UX. The login user matches your host
account with passwordless `sudo`, your home directory is mounted inside
the VM, and each machine keeps its filesystem and runs the image's own
init system (such as`systemd` or `openrc`).
```bash
container machine create alpine:3.22 --name my-machine
container machine run -n my-machine # interactive shell
container machine set -n my-machine cpus=4 memory=8G
```
Subcommands: `create`, `run`, `list` (`ls`), `inspect`, `set`,
`set-default`, `logs`, `stop`, `delete` (`rm`); `m` aliases `machine`.
Docs added to `docs/command-reference.md` (Machine Management) and
`docs/how-to.md` ("Use container machines").
## Testing
- [x] Tested locally
- [x] Added/updated tests
- [x] Added/updated docs
Signed-off-by: Raj Aryan Singh <rajaryan_singh@apple.com>
Co-authored-by: Jaewon Hur <jaewon_hur@apple.com>
Co-authored-by: John Logan <john_logan@apple.com>
Co-authored-by: Michael Crosby <michael_crosby@apple.com>
Co-authored-by: Eric Ernst <eric_ernst@apple.com>
Co-authored-by: Danny Canter <danny_canter@apple.com>
- Fixes#1642.
- The `ensure-hawkeye-exists.sh` now actually
ensures that hawkeye is installed. If it is not
installed, the script informs that the installation
uses `curl | sh` and asks the user to confirm
before proceeding.
- Automated workflows can bypass the prompt
by invoking the script with the `-y`/`--auto-install`
option, or by setting the environment variable
`HAWKEYE_AUTO_INSTALL=1`.
- Export HAWKEYE_AUTO_INSTALL=1 in every
Git workflow job that runs make check, to
ensure license/format checks don't stall.
- Closes#1641.
- This PR replaces the existing hook pathname computation
for `make pre-commit` with `git rev-parse --git-path hooks/...`,
which resolves to the shared hooks directory in both the main
checkout and any worktree.
- Closes#1526 and #1527.
## Type of Change
- [x] Bug fix
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation update
## Motivation and Context
This PR fixes `system df` to report actual on-disk allocated bytes
(content blobs + snapshots) instead of summing per-image snapshot sizes.
Orphaned blobs are now included as reclaimable, and storage shared
across tags is no longer double counted. Also consolidates three
identical `calculateDirectorySize` implementations into a shared
`FileManager.allocatedSize(of:)` extension.
## Testing
- [x] Tested locally
- [x] Added/updated tests
- [ ] Added/updated docs
## Type of Change
- [x] New feature
## Motivation and Context
When developing, there are times when I want to run a specific set of
CLI tests. This PR allows users to set what integration tests they want
to run by setting the makefile variable `INTEGRATION_TEST_SUITES`.
Example usage:
```
% INTEGRATION_TEST_SUITES="TestCLIVolumes TestCLIAnonymousVolumes" make all integration
```
## Testing
- [x] Tested locally
Signed-off-by: Kathryn Baldauf <k_baldauf@apple.com>
This utilizes swift test `--skip-build` in combination with a single big
swift test compilation step. By doing this we reduce duplicated swift
compilation between unit and integration.
## Type of Change
- [ ] Bug fix
- [X] New feature
- [ ] Breaking change
- [ ] Documentation update
## Motivation and Context
Coverage GitHub action is taking too long, so hopefully reducing
duplicated work decreases time.
## Testing
- [X] Tested locally
- [ ] Added/updated tests
- [ ] Added/updated docs
This extends the github actions to by default run the coverage
generation targets. The targets will generate the coverage artifacts,
which will then be used by a seprate github action to comment on the PR
the coverage numbers.
This is a sanity check for testing information for new features and
refactors alike. It will allow reviewers to quickly identify if there
are major issues with new tests, or large changes to coverage percentage
that indicate a problem.
This improves the integration test suite speed by running all
integration tests in a single `swift test` invocation. We maintain the
one `@Suite` at a time behavior by introducing a new test trait that
uses a single mutex to limit inter-suite parallelism.
This adds make targets that generate json, html, and plain text coverage
reports for unit tests, integration tests, and an aggregation of both
suites. These new targets can be run via `make coverage` (aggregate),
`make coverage-unit` (only unit tests), and `make coverage-integration`
(only integration tests).
## Type of Change
- [ ] Bug fix
- [X] New feature
- [ ] Breaking change
- [ ] Documentation update
## Motivation and Context
This provides developers a means to measure test coverage for new and
old features alike, to ensure that testing for aforementinoed features
is adequate.
## Testing
- [X] Tested locally
- [ ] Added/updated tests
- [ ] Added/updated docs
This change migrates plugin config format away from using JSON to use
TOML.
- Addresses the plugin portion of discussion #1336
## Type of Change
- [ ] Bug fix
- [X] New feature
- [ ] Breaking change
- [ ] Documentation update
## Motivation and Context
Context within https://github.com/apple/container/discussions/1336
> We're looking to improve the user experience and overall functionality
of setting defaults for container to better enable future use cases.
>
> Today container uses macOS's
[UserDefaults](https://developer.apple.com/documentation/foundation/userdefaults)
to configure settings needed at runtime. As mentioned in
https://github.com/apple/container/issues/608, UserDefaults may be
idiomatic for macOS, but they apply globally to all sessions and do not
handle representing complex, hierarchical data well. Additionally, we
currently have two ways of setting these defaults, either directly with
macOS's defaults command or through container system property.
>
> https://github.com/apple/container/issues/608 proposes moving to use
environment variables in place of UserDefaults. However, we do not
believe this is sufficient. Environment variables are not in a
consistent location, are not sourced from data, and also do not handle
representing complex, hierarchical data well.
As part of this migration to TOML for user settings we want to also move
plugin settings to use TOML so that we can have a singular file format
for `container` configurations.
## Testing
- [X] Tested locally
- [X] Added/updated tests
- [ ] Added/updated docs
Closes#1352
Containerization has had support for a bit, it was just never brought
over here. It's exposed on the CLI via the classic `--cap-add` and
`--cap-drop` UX.
- Closes#387.
- Override the swift-argument-parser `help` command
with a version that prints the reformatted help that
includes plugin information.
---------
Signed-off-by: ChengHao Yang <17496418+tico88612@users.noreply.github.com>
- TestCLIStatus isn't in the integration test.
- Update the output to align with the current output.
- Follow up #1193
---------
Signed-off-by: ChengHao Yang <17496418+tico88612@users.noreply.github.com>
- Part of #1295.
- Needed for #1342.
- If APP_ROOT is defined, remove everything but kernel from data
directory before starting integration test.
- Adds `--enable-kernel-install` for system start on integration test.
Same as we'd done with kill.
```
➜ container git:(stop-kill-throw-error) ✗ ./bin/container stop foo bar baz
Warning! Running debug build. Performance may be degraded.
Error: internalError: "failed to stop container" (cause: "notFound: "container with ID baz not found"")
internalError: "failed to stop container" (cause: "notFound: "container with ID bar not found"")
internalError: "failed to stop container" (cause: "notFound: "container with ID foo not found"")
➜ container git:(stop-kill-throw-error) ✗ ./bin/container kill foo bar baz
Warning! Running debug build. Performance may be degraded.
Error: internalError: "failed to kill container" (cause: "notFound: "container with ID foo not found"")
internalError: "failed to kill container" (cause: "notFound: "container with ID bar not found"")
internalError: "failed to kill container" (cause: "notFound: "container with ID baz not found"")
➜ container git:(stop-kill-throw-error) ✗ ./bin/container delete foo bar baz
Warning! Running debug build. Performance may be degraded.
Error: internalError: "failed to delete container" (cause: "notFound: "container with ID foo not found"")
internalError: "failed to delete container" (cause: "notFound: "container with ID baz not found"")
internalError: "failed to delete container" (cause: "notFound: "container with ID bar not found"")
```
## Type of Change
- [x] Bug fix
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation update
## Motivation and Context
`TestCLIVersion` isn't in the integration test; we should include this.
And update the output to align with the real output.
Follow up #911
## Testing
- [x] Tested locally
- [x] Added/updated tests
- [ ] Added/updated docs
---------
Signed-off-by: ChengHao Yang <17496418+tico88612@users.noreply.github.com>
- Adds a a `--log-root` option to `swift system start`, propagating the
value as `CONTAINER_LOG_ROOT` to services for logging to files instead
of the OS log facility. This is not a "production" capability as it
neither merges nor rotates logs.
- Currently we don't collect logs on CI builds, and we don't have
permission to run the `log` command there. The PR adds `--log-root` to
the CI test phase, archives the results, and uploads the archive as an
artifact.
- Use FilePath from swift-system for the log root. Foundation URL is a
bit of a footgun for filesystem paths, so unless we identify a
showstopper, we should incrementally transition to this type everywhere
except where we really need network URLs.
- Output the hostname of the CI runner at the start of the test phase so
we can identify runner-specific issues where they exist.
- Fix formatting for log messages with multiple metadata items, and fix
unstructured messages on instances that weren't found using `grep -r
'log\.' Sources`.
- Adds command reference documentation for `--log-root`.
- Closes#1206.
- Closes#1185.
- Closes#507.
- Addresses existing log messages for #642.
- Nondeterministic CI errors are resulting from very slow launch times
for the first runtime helper, which causes ContainersService to be
locked for longer than our 20 sec timeout. Bumping the timeout to 60
seconds addresses this case for now.
- Since many log messages needed to be changed to troubleshoot the
issue, updated all log messages to use structured logging, and
implemented consistent entry/exit logging for all service operations.
- Added logging for ContainerService lock acquisition to help with
finding root cause for the slow service startup.
- Plumbed the `--debug` flag on both `container system start` and
`container system logs` so that the flag is actually useful.
- Updated the `install-init.sh` script so that can install in a custom
app root directory.
## Type of Change
- [x] Bug fix
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation update
## Motivation and Context
- CI build is failing because runners don't have an /etc/resolver
directory, causing the directory monitor to fail. This occurs while the
install-kernel make target is running, so it appears that kernel
download is failing when it's just that the API server is dying. Created
#1207 for the issue.
- Fixing the initial scan for the directory just moves the problem to
the filesystem watch loop, likely because we're not testing the result
of `open()` for errors.
- Right now the priority is getting CI running and PRs merged, so just
commenting out the realhost DNS server setup.
- Also seeing that under some conditions it can take quite a while for
launchd to start the helper for the default network (8 seconds or more).
With the 10 second health check timeout after API server registration,
this means that some CI runs can exhibit this failure mode. Added a
`--timeout` option to SystemStart and set a 60 second timeout for
install-kernel and integration Makefile targets.
- Fixed a bug where `--debug` was being placed in the wrong location in
the api server startup args.
- Disabled all network CLI tests due to container bootstrap errors when
trying to run the container immediately after creating the network. The
slow network helper launch could be the reason behind the failures that
drove us to serialize these tests. Filed #1206 for this issue.
## Testing
- [x] Tested locally
- [ ] Added/updated tests
- [ ] Added/updated docs
- Fixed#892.
- By contrast with `rm`, `prune` should display
the amount of reclaimed storage, so added code
to retrieve it.
Signed-off-by: ChengHao Yang <17496418+tico88612@users.noreply.github.com>
- Closes#639.
- Adds swift format configuration that removes lint checks so we can use
`swift lint` to perform format-only tests.
- Adds `check` target that invokes format and header checks.
- Adds pre-commit script that runs `make check`.
- Adds `pre-commit` target that installs the check script as a
pre-commit hook.
## Type of Change
- [ ] Bug fix
- [x] New feature
- [ ] Breaking change
- [x] Documentation update
## Motivation and Context
Avoids wasting time and commit rewrites.
## Testing
- [x] Tested locally
- [ ] Added/updated tests
- [x] Added/updated docs
- TestCLIRunCommand now run so many tests concurrently that the API
server gets swamped and tests randomly time out.
- The parallelism options on `swift test` only work for XCTest, not
swift-testing.
- Work around this while retaining some parallelism (good for stress
testing) by breaking the tests into two suites.
- It is common to have `gnu-tar` alongside other GNU tools
installed and aliased for compatibility reasons. However, this
breaks the current make build.
- Use BSD-only binaries (no GNU equivalents that are
commonly aliased), making the Makefile more portable.
Often times I'll be making a change that only touches the cli and I
don't feel like sitting through the potential song and dance of the
other components building/installing.
- Updates to `image prune` for consistency with how
other `prune` commands are done. Added missing
test cases as well for the command
- Relates to the discussion from #914
- Fixes#943.
- Use images other than alpine:3.20 for image concurrency test so as not
to interfere with tests using that image.
- Rename test files to match suite names.
Closes#824
This implements statistics gathering across the various components, but
ultimately this is for implementing a new CLI command: `container
stats`. This shows memory usage, cpu usage, network and block i/o and
the number of processes in the container. The new command can inspect
stats for 1-N containers and by default continuously updates in a `top`
like stream.
## Type of Change
- [ ] Bug fix
- [x] New feature
- [ ] Breaking change
- [ ] Documentation update
## Testing
- [x] Tested locally
- [x] Added/updated tests
- [x] Added/updated docs
To facilitate ability to gather code coverage metrics without slowing
down our integration target, let's add a coverage target.
From this we can run `xcrun llvm-profdata merge` to get profdata and
gather overall code coverage metrics
## Type of Change
- [ ] Bug fix
- [x] New feature
- [ ] Breaking change
- [ ] Documentation update
## Motivation and Context
[Why is this change needed?]
## Testing
- [x] Tested locally
- [ ] Added/updated tests
- [ ] Added/updated docs
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
## Type of Change
- [x] New feature
## Description
Add option to force kernel setting and tests for CLI `kernel set`.
Related to https://github.com/apple/container/pull/575.
## Motivation and Context
This PR adds additional tests to ensure that we can set kernels from
local files, remote tar files, and local tar files. A new option `force`
is added to the `kernel set` subcommand which will overwrite an existing
kernel with the same name if one exists to make testing easier. The
tests ensure that a container can be started with the newly set kernel
and resets to the default recommended kernel when complete.
## Testing
- [x] Tested locally
- [x] Added/updated tests
Signed-off-by: Kathryn Baldauf <k_baldauf@apple.com>
- Disabled tests that failed with release config.
- Changed so merge builds build with release config so this doesn't burn
us next release.
- Added `swift --version` to Makefile so we can see what we're running
up in CI.
Closes#339.
This change adds named volume support to container, providing volume
management CLI commands - `create, delete, list and inspect`. The
implementation uses EXT4 block-based persistent storage with a new
`VolumesService` actor for thread-safe operations, integrates seamlessly
with the existing container mount system through a new `.volume`
filesystem type, and provides atomic volume operations with XPC-based
API communication. Volumes are stored in isolated directories with
configurable sizes (default 512GB) and include proper cleanup and
container usage tracking for safe deletion.
Example Usage:
```
# Create a volume
container volume create mydata
# Use volume in container
container run -v mydata:/data alpine
# List volumes
container volume list
# Inspect volume details
container volume inspect mydata
# Clean up
container volume rm mydata
```
This matches other container commands that rely on user arguments at the
end, such as `container run`. Closes#395.
Signed-off-by: Kathryn Baldauf <k_baldauf@apple.com>
See discussion below for example. For multiple network interfaces in a
single container we'll want to integrate against a containerization that
includes apple/containerization#156.
The change bumps the containerization dependency to 0.2.0 and addresses
the breaking API changes.
```console
% container network
OVERVIEW: Manage container networks
USAGE: container network <subcommand>
OPTIONS:
--version Show the version.
-h, --help Show help information.
SUBCOMMANDS:
create Create a new network
delete, rm Delete one or more networks
list, ls List networks
inspect Display information about one or more networks
See 'container help network <subcommand>' for detailed help.
```
Harshit Singh BhandariandGitHub
Harshit Singh BhandariandGitHub
Di Warachet S.andGitHub
Kathryn BaldaufandGitHub
Noah ThorntonandGitHub
Noah ThorntonandGitHub
ChengHao YangandGitHub
Saehej KangandGitHub
jwhurandGitHub
Manu SchillerandGitHub
Dmitry KovbaandGitHub
RajandGitHub
J LoganandGitHub
Aditya RamaniandGitHub