Commit Graph
35396 Commits
Author SHA1 Message Date
Kir Kolyshkin f00e9a9036 mount.Unmount(): don't look into /proc/self/mountinfo
Now, every Unmount() call takes a burden to parse the whole nine yards
of /proc/self/mountinfo to figure out whether the given mount point is
mounted or not (and returns an error in case parsing fails somehow).

Instead, let's just call umount() and ignore EINVAL, which results
in the same behavior, but much better performance.

Note that EINVAL is returned from umount(2) not only in the case when
`target` is not mounted, but also for invalid flags. As the flags are
hardcoded here, it can't be the case.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: a1d095199ddb9b4811e1417b6adcdfadad7d73f4
Component: engine
2018-04-19 14:49:50 -07:00
Kir Kolyshkin 96e23c4152 pkg/mount/mountinfo_linux: parser speed up
The mountinfo parser implemented via `fmt.Sscanf()` is slower than the one
using `strings.Split()` and `strconv.Atoi()`. This rewrite helps to speed it
up to a factor of 8x, here is a result from go bench:

> BenchmarkParsingScanf-4      	     300	  22294112 ns/op
> BenchmarkParsingSplit-4      	    3000	   2780703 ns/op

I tried other approaches, such as using `fmt.Sscanf()` for the first
three (integer) fields and `strings.Split()` for the rest, but it slows
things down considerably:

> BenchmarkParsingMixed-4      	    1000	   8827058 ns/op

Note the old code uses `fmt.Sscanf`, when a linear search for '-' field,
when a split for the last 3 fields. The new code relies on a single
split.

I have also added more comments to aid in future development.

Finally, the test data is fixed to now have white space before the first field.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: c611f18a7f16d8aa878a5a5c7537d23a0937c40a
Component: engine
2018-04-19 14:49:42 -07:00
Kir Kolyshkin cd146e5f0d getSourceMount(): simplify
The flow of getSourceMount was:
 1 get all entries from /proc/self/mountinfo
 2 do a linear search for the `source` directory
 3 if found, return its data
 4 get the parent directory of `source`, goto 2

The repeated linear search through the whole mountinfo (which can have
thousands of records) is inefficient. Instead, let's just

 1 collect all the relevant records (only those mount points
   that can be a parent of `source`)
 2 find the record with the longest mountpath, return its data

This was tested manually with something like

```go
func TestGetSourceMount(t *testing.T) {
	mnt, flags, err := getSourceMount("/sys/devices/msr/")
	assert.NoError(t, err)
	t.Logf("mnt: %v, flags: %v", mnt, flags)
}
```

...but it relies on having a specific mount points on the system
being used for testing.

[v2: add unit tests for ParentsFilter]

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 871c957242df9f8c74faf751a2f14eb5178d4140
Component: engine
2018-04-19 14:49:17 -07:00
Kir Kolyshkin 7e50bdbeb9 daemon.cleanupMounts(): use mount.SingleEntryFilter
Use mount.SingleEntryFilter as we're only interested in a single entry.

Test case data of TestShouldUnmountRoot is modified accordingly, as
from now on:

1. `info` can't be nil;

2. the mountpoint check is not performed (as SingleEntryFilter
   guarantees it to be equal to daemon.root).

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: d3ebcde82aae79be8cbecab365367b17adac4b3e
Component: engine
2018-04-19 14:48:25 -07:00
Kir Kolyshkin a97e7a41c8 pkg/mount: implement/use filter for mountinfo parsing
Functions `GetMounts()` and `parseMountTable()` return all the entries
as read and parsed from /proc/self/mountinfo. In many cases the caller
is only interested only one or a few entries, not all of them.

One good example is `Mounted()` function, which looks for a specific
entry only. Another example is `RecursiveUnmount()` which is only
interested in mount under a specific path.

This commit adds `filter` argument to `GetMounts()` to implement
two things:
 1. filter out entries a caller is not interested in
 2. stop processing if a caller is found what it wanted

`nil` can be passed to get a backward-compatible behavior, i.e. return
all the entries.

A few filters are implemented:
 - `PrefixFilter`: filters out all entries not under `prefix`
 - `SingleEntryFilter`: looks for a specific entry

Finally, `Mounted()` is modified to use `SingleEntryFilter()`, and
`RecursiveUnmount()` is using `PrefixFilter()`.

Unit tests are added to check filters are working.

[v2: ditch NoFilter, use nil]
[v3: ditch GetMountsFiltered()]
[v4: add unit test for filters]
[v5: switch to gotestyourself]

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: bb934c6aca3e77541dd4fd51b9ab2706294dadda
Component: engine
2018-04-19 14:48:09 -07:00
Brian Goff 6099e61dfc Merge pull request #36886 from vdemeester/experimental-plugins-migration
Migrate test-integration-cli experimental plugin tests to integration
Upstream-commit: e396b27b7f207f4f0da9ed66d70a69b219bbafaf
Component: engine
2018-04-19 16:13:05 -04:00
Kir Kolyshkin 1376a6d8fc TestContainerStopTimeout: fix
The unit test is checking that setting of non-default StopTimeout
works, but it checked the value of StopSignal instead.

Amazingly, the test was working since the default StopSignal is SIGTERM,
which has the numeric value of 15.

Fixes: commit e66d21089 ("Add config parameter to change ...")
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 743f5afcd13659e2b6b76791415876797e2ede52
Component: engine
2018-04-19 11:09:46 -07:00
Brian Goff 3f0393a442 Move mount parsing to separate package.
This moves the platform specific stuff in a separate package and keeps
the `volume` package and the defined interfaces light to import.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 6a70fd222b95643a8a6b88e2634d5f085ae4122a
Component: engine
2018-04-19 06:35:54 -04:00
Vincent Demeester 8348a70a7d Migrate test-integration-cli experimental plugin tests to integration
All `plugins` tests that require an `ExperimentalDaemon` are migrated
to `integration/plugin/*` and start an experimental daemon to test on
it.

The end goal being to remove the `experimental` build.

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 69bab3832ce1d72dd1d3ed2f2b53143ec9ff102c
Component: engine
2018-04-19 11:57:28 +02:00
Vincent Demeester f766069f3c Merge pull request #36684 from cpuguy83/bump_containerd_client
Bump containerd client
Upstream-commit: 8bb5a28eed5eba5651c6e48eb401c03be938b4c1
Component: engine
2018-04-19 10:38:43 +02:00
Sebastiaan van Stijn 1a74ffcae2 Merge pull request #36865 from vdemeester/integration-move-swarm-plugin-test
Migrate TestAPISwarmServicesPlugin to integration
Upstream-commit: f1783da7361ab3dc985df12446357e6c4c9ef388
Component: engine
2018-04-18 21:15:59 -07:00
Sebastiaan van Stijn 21ab0afd1a Merge pull request #36894 from cpuguy83/bump_x_net
Bump golang.org/x/net to go1.10 release commit
Upstream-commit: f0b2f9362fbd6be58404b9d6bc3de53c6bc884e9
Component: engine
2018-04-18 18:04:51 -07:00
Brian Goff 068e200711 Extra check before unmounting on shutdown
This makes sure that if the daemon root was already a self-binded mount
(thus meaning the daemonc only performed a remount) that the daemon does
not try to unmount.

Example:

```
$ sudo mount --bind /var/lib/docker /var/lib/docker
$ sudo dockerd &
```

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: c403f0036b9945bd58a84e2c93f794ed9861fe99
Component: engine
2018-04-18 20:43:42 -04:00
Sebastiaan van Stijn 5eb096d4f8 Merge pull request #36848 from cpuguy83/libcontainerd_client_locking
Relax some libcontainerd client locking
Upstream-commit: 69a5611dde5b46785c8f071a744f5b4b7547d5b9
Component: engine
2018-04-18 16:47:23 -07:00
Brian Goff 320288f128 Merge pull request #36637 from cpuguy83/no_global_driver_store
No global volume driver store
Upstream-commit: 6cd806aa53de41e6f04346150d08e19ec63311e0
Component: engine
2018-04-18 16:32:23 -04:00
Brian Goff c5fc4e9f5a Bump golang.org/x/net to go1.10 release commit
The primary need for this is x/net/context now is just a type alias to
the stdlib context package.
This makes issues with conflicts between "golang.org/x/net/context" and
the stdib "context" go away (primarily a concern in interface
definitions/implementations).

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 848ef1da8b0d1c01a84fbe215f24f00fa741690d
Component: engine
2018-04-18 15:57:24 -04:00
Vincent Demeester f40401e83b Remove daemon.BuildImageWithOut and use cli helpers function
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 5d2afe4f51edc4c8393c4cbee7b478f7dc1e88ab
Component: engine
2018-04-18 16:45:55 +02:00
Vincent Demeester a6beebd548 Migrate TestAPISwarmServicesPlugin to integration
Also starts to create more "poll/check" function to `internal/test/daemon`.

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: aa0db6f9e1d0fd0420271f419d0d225089a95bbd
Component: engine
2018-04-18 16:18:53 +02:00
Vincent Demeester 917c353f43 Add a Cleanup function that cleans exec root and swarm files
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 3b01e92c9a34036b891853402888dcf461836ef8
Component: engine
2018-04-18 16:14:50 +02:00
Vincent Demeester a90ad7bf98 Merge pull request #36832 from vdemeester/integration-daemon-api-helpers
More integration-cli/integration refactoring + request package
Upstream-commit: 25ff68a80f315a5e0a4526cc15a1ece83843ae2b
Component: engine
2018-04-18 16:12:14 +02:00
Alexandre Jomin f6b5cf688c pkg/term add build tag for netbsd
Signed-off-by: Alexandre Jomin <alexandrejomin@gmail.com>
Upstream-commit: 0d82bc142c2448c09b567773867344aa8fc172fb
Component: engine
2018-04-18 15:16:06 +02:00
Brian Goff a1bdeeb8dc Merge pull request #36858 from j-haj/36414-standardize_build_paths
Standardize artefact path for all build stages
Upstream-commit: 4923f8cc639e1ab01a251429495920e516ef779f
Component: engine
2018-04-18 06:58:38 -04:00
Akihiro Suda 6f9cc3826f Merge pull request #36880 from thaJeztah/bump-swarmkit
Bump SwarmKit to 9c2aa152c3054371b833483a7ddad8d15052ec4f
Upstream-commit: 5dae4828f3248d7f442c53165aa0628020c6b74e
Component: engine
2018-04-18 19:10:55 +09:00
Sebastiaan van Stijn c48e2f3808 Bump SwarmKit to 9c2aa152c3054371b833483a7ddad8d15052ec4f
Relevant changes:

- docker/swarmkit#2551 RoleManager will remove deleted nodes from the cluster membership
- docker/swarmkit#2574 Scheduler/TaskReaper: handle unassigned tasks marked for shutdown
- docker/swarmkit#2561 Avoid predefined error log
- docker/swarmkit#2557 Task reaper should delete tasks with removed slots that were not yet assigned
- docker/swarmkit#2587 [fips] Agent reports FIPS status
- docker/swarmkit#2603 Fix manager/state/store.timedMutex

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 333b2f28fef4ba857905e7263e7b9bbbf7c522fc
Component: engine
2018-04-17 13:44:49 -07:00
Brian Goff 856b3acadf Remove use of global volume driver store
Instead of using a global store for volume drivers, scope the driver
store to the caller (e.g. the volume store). This makes testing much
simpler.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 977109d808ae94eb3931ae920338b1aa669f627e
Component: engine
2018-04-17 14:07:08 -04:00
Brian Goff e2550466e7 Rename volumedrivers package to drivers
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: b047f675ea958fa730f4bf56f909faceb9636506
Component: engine
2018-04-17 14:06:53 -04:00
Brian Goff 8f56510165 Remove old/uneeded volume migration from vers 1.7
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 0023abbad34282762d5bd17302776d2a8521fffc
Component: engine
2018-04-17 14:06:53 -04:00
Brian Goff 27a6dfb147 Move direct volume driver interaction to store
Since the volume store already provides this functionality, we should
just use it rather than duplicating it.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 63826e291ba3b88443b64802084bbb3931857b56
Component: engine
2018-04-17 14:06:53 -04:00
Brian Goff c9ce394894 Relax global client lock on containerd restore.
This unblocks the client to take other restore requests and makes sure
that a long/stuck request can't block the client forever.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 806700e410a1b30aa1033d71d825e57880c75298
Component: engine
2018-04-17 12:07:06 -04:00
Vincent Demeester a05c5e914a Move integration-cli/request to internal/test/request…
… and change a bit the method signature

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 42f6fdf059807ecbe66cce90676021363a541154
Component: engine
2018-04-17 16:25:59 +02:00
Vincent Demeester 3d6d872b1c Add api helpers to internal/test/daemon.Daemon
Porting helpers from `integration-cli/daemon.Daemon` to this struct
and use the API instead of the cli.

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 9722214c8add2a83f6c4974848139cadc79d9f0e
Component: engine
2018-04-17 16:24:31 +02:00
Vincent Demeester e20647387e Merge pull request #36868 from vdemeester/integration-move-fake-internal
Move fakecontext, fakegit and fakestorage to internal/test
Upstream-commit: 648ae4979e5a44fd046c292b48a02725a16f9304
Component: engine
2018-04-17 16:23:22 +02:00
Jeff Hajewski f4e6667402 Standardize artefact path for all build stages
Update all build stages to output artefacts to /build

Signed-off-by: Jeff Hajewski <jeff.hajewski@gmail.com>
Upstream-commit: 00555f7b261111084b966920da607a504b068953
Component: engine
2018-04-17 09:08:25 -05:00
Vincent Demeester a425bbca33 Move fakecontext, fakegit and fakestorage to internal/test
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 062564084a22f71cf5807ae5dfad7d29afb12e04
Component: engine
2018-04-17 09:53:09 +02:00
Sebastiaan van Stijn b77fed9a5f Merge pull request #36866 from fntlnz/poule-assign-pr-fntlnz
Add myself to poule to get 2w/old PRs
Upstream-commit: 13771fede318028d67e5d521bb97bcb54a4aaec5
Component: engine
2018-04-16 18:32:32 -07:00
Anusha Ragunathan f4b8a312a4 Merge pull request #36841 from yongtang/02222018-tag-test
Migrate image tag tests from integration-cli to api tests
Upstream-commit: abb1661907fa5e831872a7f99f1af2dcc88724bf
Component: engine
2018-04-16 17:53:48 -07:00
Anusha Ragunathan 84f41cfc7e Merge pull request #36838 from vdemeester/integration-fixtures-refactoring
Clean some integration-cli/fixtures package/files
Upstream-commit: f9c4e630269be438cddfe843c727c21c3343f1c2
Component: engine
2018-04-16 17:49:29 -07:00
Anusha Ragunathan 9c85effe88 Merge pull request #36854 from vdemeester/integration-swarm-more-in-internal
[test/integration] Small daemon refactoring and add swarm init/join helpers
Upstream-commit: c65d365eb3f98df7b10ac07b919ebaa1e4b75212
Component: engine
2018-04-16 12:49:49 -07:00
Lorenzo Fontana e26e074b4e Add myself to poule to get 2w/old PRs
Signed-off-by: Lorenzo Fontana <lo@linux.com>
Upstream-commit: 691e0c06cef55a455d72a4d1ec9dc4d072cf5e3b
Component: engine
2018-04-16 18:33:21 +02:00
Vincent Demeester c2c6cea1b0 Clean some integration-cli/fixtures package/files
- Move go package used by both `integration-cli` and `integration` to
  `internal/test/fixtures`.
- Remove fixtures that are not used anymore (moved to `docker/cli` a
  while ago) : deploy, notary, secrets.

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 5f56503f583f21d655394f755f71849381bd58c7
Component: engine
2018-04-16 10:48:58 +02:00
Vincent Demeester bd6e299583 Small daemon refactoring and add swarm init/join helpers
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 239a8a518904dfb51fe62087d8702519c20ce808
Component: engine
2018-04-16 10:20:10 +02:00
Yong Tang 1b1d5ffe90 Merge pull request #36862 from yongtang/04152018-container.Create
Some enhancement in integration tests
Upstream-commit: ea579a6194525ce3b98eba04f74de19a106bf918
Component: engine
2018-04-15 09:12:52 -07:00
Yong Tang fe0deb4aa6 Merge pull request #36738 from Microsoft/jjh/getlayerpath
Windows: Add GetLayerPath implementation in graphdriver
Upstream-commit: 5c2e6064a1c5cb816081958e5a7079bc6b474940
Component: engine
2018-04-14 09:59:38 -07:00
Yong Tang 691aeed45e Some enhancement in integration tests
This fix converts some `client.ContainerCreate` to `container.Create`,
and removes some unneeded `name` fields when test containers are created.

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: ab9bb47b05b1dde445a5e4ba78ae97303208dc8b
Component: engine
2018-04-14 16:52:02 +00:00
Tibor Vass 566b2857cf Merge pull request #35739 from thaJeztah/bump-go-1.10
Bump Golang to 1.10.1
Upstream-commit: 61138fb5fc0f5fc1799cabfa86d28fb88a0f472c
Component: engine
2018-04-13 13:56:38 -10:00
Sebastiaan van Stijn fce8015aac Merge pull request #36845 from medakk/refactor-daemon
Refactor code in cmd/dockerd/daemon.go
Upstream-commit: 3881f1a5219ba5d0a1ed54121ba20cd1700fec52
Component: engine
2018-04-13 15:26:48 -07:00
Sebastiaan van Stijn 6f1e5f20ef Merge pull request #36839 from vdemeester/integration-registry-refactoring
Move and refactor integration-cli/registry to internal/test
Upstream-commit: a9f502d04b5a493904a24c173fa3d6d6b17ad740
Component: engine
2018-04-13 15:02:53 -07:00
Yong Tang 10abb9454f Migrate image tag tests from integration-cli to api tests
This fix migrates image tag tests from integration-cli to api tests.

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 9bcb960508a6066811cffcca1e35ca44d7f1cf94
Component: engine
2018-04-13 14:46:40 +00:00
Vincent Demeester af04bf6300 Move and refactor integration-cli/registry to internal/test
- Move the code from `integration-cli` to `internal/test`.
- Use `testingT` and `assert` when creating the registry.

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 66de2e6e3b6d927a3396743cd7c363aa9f7b776e
Component: engine
2018-04-13 10:45:34 +02:00
Akihiro Suda 1fe7cf068a Merge pull request #36840 from cpuguy83/bump_libnetwork
Bump libnetwork commit
Upstream-commit: 544ec0994f3979314b0c3a892a4e352145d0cc0b
Component: engine
2018-04-13 13:54:52 +09:00