This adds make targets that generate json, html, and plain text coverage
reports for unit tests, integration tests, and an aggregation of both
suites. These new targets can be run via `make coverage` (aggregate),
`make coverage-unit` (only unit tests), and `make coverage-integration`
(only integration tests).
## Type of Change
- [ ] Bug fix
- [X] New feature
- [ ] Breaking change
- [ ] Documentation update
## Motivation and Context
This provides developers a means to measure test coverage for new and
old features alike, to ensure that testing for aforementinoed features
is adequate.
## Testing
- [X] Tested locally
- [ ] Added/updated tests
- [ ] Added/updated docs
This change migrates plugin config format away from using JSON to use
TOML.
- Addresses the plugin portion of discussion #1336
## Type of Change
- [ ] Bug fix
- [X] New feature
- [ ] Breaking change
- [ ] Documentation update
## Motivation and Context
Context within https://github.com/apple/container/discussions/1336
> We're looking to improve the user experience and overall functionality
of setting defaults for container to better enable future use cases.
>
> Today container uses macOS's
[UserDefaults](https://developer.apple.com/documentation/foundation/userdefaults)
to configure settings needed at runtime. As mentioned in
https://github.com/apple/container/issues/608, UserDefaults may be
idiomatic for macOS, but they apply globally to all sessions and do not
handle representing complex, hierarchical data well. Additionally, we
currently have two ways of setting these defaults, either directly with
macOS's defaults command or through container system property.
>
> https://github.com/apple/container/issues/608 proposes moving to use
environment variables in place of UserDefaults. However, we do not
believe this is sufficient. Environment variables are not in a
consistent location, are not sourced from data, and also do not handle
representing complex, hierarchical data well.
As part of this migration to TOML for user settings we want to also move
plugin settings to use TOML so that we can have a singular file format
for `container` configurations.
## Testing
- [X] Tested locally
- [X] Added/updated tests
- [ ] Added/updated docs
Closes#1352
Containerization has had support for a bit, it was just never brought
over here. It's exposed on the CLI via the classic `--cap-add` and
`--cap-drop` UX.
- Closes#387.
- Override the swift-argument-parser `help` command
with a version that prints the reformatted help that
includes plugin information.
---------
Signed-off-by: ChengHao Yang <17496418+tico88612@users.noreply.github.com>
- TestCLIStatus isn't in the integration test.
- Update the output to align with the current output.
- Follow up #1193
---------
Signed-off-by: ChengHao Yang <17496418+tico88612@users.noreply.github.com>
- Part of #1295.
- Needed for #1342.
- If APP_ROOT is defined, remove everything but kernel from data
directory before starting integration test.
- Adds `--enable-kernel-install` for system start on integration test.
Same as we'd done with kill.
```
➜ container git:(stop-kill-throw-error) ✗ ./bin/container stop foo bar baz
Warning! Running debug build. Performance may be degraded.
Error: internalError: "failed to stop container" (cause: "notFound: "container with ID baz not found"")
internalError: "failed to stop container" (cause: "notFound: "container with ID bar not found"")
internalError: "failed to stop container" (cause: "notFound: "container with ID foo not found"")
➜ container git:(stop-kill-throw-error) ✗ ./bin/container kill foo bar baz
Warning! Running debug build. Performance may be degraded.
Error: internalError: "failed to kill container" (cause: "notFound: "container with ID foo not found"")
internalError: "failed to kill container" (cause: "notFound: "container with ID bar not found"")
internalError: "failed to kill container" (cause: "notFound: "container with ID baz not found"")
➜ container git:(stop-kill-throw-error) ✗ ./bin/container delete foo bar baz
Warning! Running debug build. Performance may be degraded.
Error: internalError: "failed to delete container" (cause: "notFound: "container with ID foo not found"")
internalError: "failed to delete container" (cause: "notFound: "container with ID baz not found"")
internalError: "failed to delete container" (cause: "notFound: "container with ID bar not found"")
```
## Type of Change
- [x] Bug fix
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation update
## Motivation and Context
`TestCLIVersion` isn't in the integration test; we should include this.
And update the output to align with the real output.
Follow up #911
## Testing
- [x] Tested locally
- [x] Added/updated tests
- [ ] Added/updated docs
---------
Signed-off-by: ChengHao Yang <17496418+tico88612@users.noreply.github.com>
- Adds a a `--log-root` option to `swift system start`, propagating the
value as `CONTAINER_LOG_ROOT` to services for logging to files instead
of the OS log facility. This is not a "production" capability as it
neither merges nor rotates logs.
- Currently we don't collect logs on CI builds, and we don't have
permission to run the `log` command there. The PR adds `--log-root` to
the CI test phase, archives the results, and uploads the archive as an
artifact.
- Use FilePath from swift-system for the log root. Foundation URL is a
bit of a footgun for filesystem paths, so unless we identify a
showstopper, we should incrementally transition to this type everywhere
except where we really need network URLs.
- Output the hostname of the CI runner at the start of the test phase so
we can identify runner-specific issues where they exist.
- Fix formatting for log messages with multiple metadata items, and fix
unstructured messages on instances that weren't found using `grep -r
'log\.' Sources`.
- Adds command reference documentation for `--log-root`.
- Closes#1206.
- Closes#1185.
- Closes#507.
- Addresses existing log messages for #642.
- Nondeterministic CI errors are resulting from very slow launch times
for the first runtime helper, which causes ContainersService to be
locked for longer than our 20 sec timeout. Bumping the timeout to 60
seconds addresses this case for now.
- Since many log messages needed to be changed to troubleshoot the
issue, updated all log messages to use structured logging, and
implemented consistent entry/exit logging for all service operations.
- Added logging for ContainerService lock acquisition to help with
finding root cause for the slow service startup.
- Plumbed the `--debug` flag on both `container system start` and
`container system logs` so that the flag is actually useful.
- Updated the `install-init.sh` script so that can install in a custom
app root directory.
## Type of Change
- [x] Bug fix
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation update
## Motivation and Context
- CI build is failing because runners don't have an /etc/resolver
directory, causing the directory monitor to fail. This occurs while the
install-kernel make target is running, so it appears that kernel
download is failing when it's just that the API server is dying. Created
#1207 for the issue.
- Fixing the initial scan for the directory just moves the problem to
the filesystem watch loop, likely because we're not testing the result
of `open()` for errors.
- Right now the priority is getting CI running and PRs merged, so just
commenting out the realhost DNS server setup.
- Also seeing that under some conditions it can take quite a while for
launchd to start the helper for the default network (8 seconds or more).
With the 10 second health check timeout after API server registration,
this means that some CI runs can exhibit this failure mode. Added a
`--timeout` option to SystemStart and set a 60 second timeout for
install-kernel and integration Makefile targets.
- Fixed a bug where `--debug` was being placed in the wrong location in
the api server startup args.
- Disabled all network CLI tests due to container bootstrap errors when
trying to run the container immediately after creating the network. The
slow network helper launch could be the reason behind the failures that
drove us to serialize these tests. Filed #1206 for this issue.
## Testing
- [x] Tested locally
- [ ] Added/updated tests
- [ ] Added/updated docs
- Fixed#892.
- By contrast with `rm`, `prune` should display
the amount of reclaimed storage, so added code
to retrieve it.
Signed-off-by: ChengHao Yang <17496418+tico88612@users.noreply.github.com>
- Closes#639.
- Adds swift format configuration that removes lint checks so we can use
`swift lint` to perform format-only tests.
- Adds `check` target that invokes format and header checks.
- Adds pre-commit script that runs `make check`.
- Adds `pre-commit` target that installs the check script as a
pre-commit hook.
## Type of Change
- [ ] Bug fix
- [x] New feature
- [ ] Breaking change
- [x] Documentation update
## Motivation and Context
Avoids wasting time and commit rewrites.
## Testing
- [x] Tested locally
- [ ] Added/updated tests
- [x] Added/updated docs
- TestCLIRunCommand now run so many tests concurrently that the API
server gets swamped and tests randomly time out.
- The parallelism options on `swift test` only work for XCTest, not
swift-testing.
- Work around this while retaining some parallelism (good for stress
testing) by breaking the tests into two suites.
- It is common to have `gnu-tar` alongside other GNU tools
installed and aliased for compatibility reasons. However, this
breaks the current make build.
- Use BSD-only binaries (no GNU equivalents that are
commonly aliased), making the Makefile more portable.
Often times I'll be making a change that only touches the cli and I
don't feel like sitting through the potential song and dance of the
other components building/installing.
- Updates to `image prune` for consistency with how
other `prune` commands are done. Added missing
test cases as well for the command
- Relates to the discussion from #914
- Fixes#943.
- Use images other than alpine:3.20 for image concurrency test so as not
to interfere with tests using that image.
- Rename test files to match suite names.
Closes#824
This implements statistics gathering across the various components, but
ultimately this is for implementing a new CLI command: `container
stats`. This shows memory usage, cpu usage, network and block i/o and
the number of processes in the container. The new command can inspect
stats for 1-N containers and by default continuously updates in a `top`
like stream.
## Type of Change
- [ ] Bug fix
- [x] New feature
- [ ] Breaking change
- [ ] Documentation update
## Testing
- [x] Tested locally
- [x] Added/updated tests
- [x] Added/updated docs
To facilitate ability to gather code coverage metrics without slowing
down our integration target, let's add a coverage target.
From this we can run `xcrun llvm-profdata merge` to get profdata and
gather overall code coverage metrics
## Type of Change
- [ ] Bug fix
- [x] New feature
- [ ] Breaking change
- [ ] Documentation update
## Motivation and Context
[Why is this change needed?]
## Testing
- [x] Tested locally
- [ ] Added/updated tests
- [ ] Added/updated docs
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
## Type of Change
- [x] New feature
## Description
Add option to force kernel setting and tests for CLI `kernel set`.
Related to https://github.com/apple/container/pull/575.
## Motivation and Context
This PR adds additional tests to ensure that we can set kernels from
local files, remote tar files, and local tar files. A new option `force`
is added to the `kernel set` subcommand which will overwrite an existing
kernel with the same name if one exists to make testing easier. The
tests ensure that a container can be started with the newly set kernel
and resets to the default recommended kernel when complete.
## Testing
- [x] Tested locally
- [x] Added/updated tests
Signed-off-by: Kathryn Baldauf <k_baldauf@apple.com>
- Disabled tests that failed with release config.
- Changed so merge builds build with release config so this doesn't burn
us next release.
- Added `swift --version` to Makefile so we can see what we're running
up in CI.
Closes#339.
This change adds named volume support to container, providing volume
management CLI commands - `create, delete, list and inspect`. The
implementation uses EXT4 block-based persistent storage with a new
`VolumesService` actor for thread-safe operations, integrates seamlessly
with the existing container mount system through a new `.volume`
filesystem type, and provides atomic volume operations with XPC-based
API communication. Volumes are stored in isolated directories with
configurable sizes (default 512GB) and include proper cleanup and
container usage tracking for safe deletion.
Example Usage:
```
# Create a volume
container volume create mydata
# Use volume in container
container run -v mydata:/data alpine
# List volumes
container volume list
# Inspect volume details
container volume inspect mydata
# Clean up
container volume rm mydata
```
This matches other container commands that rely on user arguments at the
end, such as `container run`. Closes#395.
Signed-off-by: Kathryn Baldauf <k_baldauf@apple.com>
See discussion below for example. For multiple network interfaces in a
single container we'll want to integrate against a containerization that
includes apple/containerization#156.
The change bumps the containerization dependency to 0.2.0 and addresses
the breaking API changes.
```console
% container network
OVERVIEW: Manage container networks
USAGE: container network <subcommand>
OPTIONS:
--version Show the version.
-h, --help Show help information.
SUBCOMMANDS:
create Create a new network
delete, rm Delete one or more networks
list, ls List networks
inspect Display information about one or more networks
See 'container help network <subcommand>' for detailed help.
```
Right now swift testing does not have finer grain test parallelization
controls. As a result our easy options are either to have ALL tests in
ALL test suites run sequentially or have tests within a given test suite
run sequentially while other test suites are run at the same time.
This has been problematic for our CI since we opted for the second
option above, where, for examples, the tests in the builder test suite
run sequentially, but the builder test suite runs at the same time as
the container run test suite. When this happens, a lot of different
tests try to pull the necessary images for testing at the same time,
causing some tests to timeout.
Signed-off-by: Kathryn Baldauf <k_baldauf@apple.com>
Replaces the `--install-dependencies` flag with the
`(enable/disable)-kernel-install` flag. If the flag is not passed - the
default behavior is prompting the user for a response.
Also adds a make target `install-kernel`
```
➜ bin/container system start
Verifying apiserver is running...
Installing base container filesystem...
No default kernel configured.
Install the recommended default kernel from [https://github.com/kata-containers/kata-containers/releases/download/3.17.0/kata-static-3.17.0-arm64.tar.xz]? [Y/n]: n
Please use the `container system kernel set --recommended` command to configure the default kernel
➜ bin/container system start --disable-kernel-install
Verifying apiserver is running...
➜ bin/container system start --enable-kernel-install
Verifying apiserver is running...
Installing kernel...
➜ bin/container system start --help
USAGE: container system start ... [--enable-kernel-install] [--disable-kernel-install]
...
--enable-kernel-install/--disable-kernel-install
Specify if the default kernel should be installed or not.
```
---------
Signed-off-by: Aditya Ramani <a_ramani@apple.com>
This change updates the UX when we install required dependencies at
first launch
### Fresh install - respond `y` to prompt
```
➜ bin/container system start
Verifying apiserver is running...
Installing base container filesystem...
No default kernel configured.
Install the recommended default kernel from [https://github.com/kata-containers/kata-containers/releases/download/3.17.0/kata-static-3.17.0-arm64.tar.xz]? [Y/n]: y
Installing kernel...
```
### Fresh install - respond `n` to prompt and then run the provided
command
```
➜ bin/container system start
Verifying apiserver is running...
Installing base container filesystem...
No default kernel configured.
Install the recommended default kernel from [https://github.com/kata-containers/kata-containers/releases/download/3.17.0/kata-static-3.17.0-arm64.tar.xz]? [Y/n]: n
Please use the `container system kernel set --recommended` command to configure the default kernel
➜ bin/container system kernel set --recommended
Installing the recommended kernel from https://github.com/kata-containers/kata-containers/releases/download/3.17.0/kata-static-3.17.0-arm64.tar.xz...
```
### Fresh install - respond `n` to prompt and then run a container
```
➜ bin/container system start
Verifying apiserver is running...
Installing base container filesystem...
No default kernel configured.
Install the recommended default kernel from [https://github.com/kata-containers/kata-containers/releases/download/3.17.0/kata-static-3.17.0-arm64.tar.xz]? [Y/n]: n
Please use the `container system kernel set --recommended` command to configure the default kernel
➜ bin/container run alpine uname
Error: notFound: "Default kernel not configured for architecture arm64. Please use the `container system kernel set` command to configure it
```
---------
Signed-off-by: Aditya Ramani <a_ramani@apple.com>
Noah ThorntonandGitHub
Noah ThorntonandGitHub
Danny CanterandGitHub
ChengHao YangandGitHub
J LoganandGitHub
Saehej KangandGitHub
jwhurandGitHub
Manu SchillerandGitHub
Kathryn BaldaufandGitHub
Dmitry KovbaandGitHub
RajandGitHub
Eric ErnstandGitHub
J LoganandGitHub
Aditya RamaniandGitHub
Josh SorefandGitHub