Merge pull request #454 from seemethere/cherry_pick_packaging_88

[18.03] Fixes binary installation
Fixes binary installation
2026-08-24 10:05:37 -05:00 · 2018-03-06 14:11:59 -08:00 · 2018-03-06 21:50:00 +00:00 · 2018-03-06 13:05:52 -08:00 · 2018-03-06 13:05:24 -08:00 · 2018-03-06 11:30:36 -08:00
60 changed files with 904 additions and 854 deletions
@@ -1,25 +1,20 @@
 # Changelog 
-Items starting with `DEPRECATE` are important deprecation notices. For more
-information on the list of deprecated flags and APIs please have a look at
-https://docs.docker.com/engine/deprecated/ where target removal dates can also
-be found.
+For more information on the list of deprecated flags and APIs please have a look at
+https://docs.docker.com/engine/deprecated/ where you can find the target removal dates 

 ## 18.03.0-ce (2018-03-DD)


 ### Builder

-* switch to -buildmode=pie [moby/moby#34369](https://github.com/moby/moby/pull/34369)
-* Allow Dockerfile from outside build-context [docker/cli#886](https://github.com/docker/cli/pull/886)
+* Switch to -buildmode=pie [moby/moby#34369](https://github.com/moby/moby/pull/34369)
+* Allow Dockerfile to be outside of build-context [docker/cli#886](https://github.com/docker/cli/pull/886) 
 * Builder: fix wrong cache hits building from tars [moby/moby#36329](https://github.com/moby/moby/pull/36329)
-* Update containerd/continuity to fix ARM 32-bit builds [moby/moby#36339](https://github.com/moby/moby/pull/36339)
-* Updated docker-on-docker build-notes. [moby/moby#35892](https://github.com/moby/moby/pull/35892)
- Fix string type for buildargs API definition [moby/moby#36246](https://github.com/moby/moby/pull/36246)
+- Fixes files leaking to other images in a multi-stage build [moby/moby#36338](https://github.com/moby/moby/pull/36338)

 ### Client

 * Simplify the marshaling of compose types.Config [docker/cli#895](https://github.com/docker/cli/pull/895)
-* Update reference docs for docker stack deploy [docker/cli#871](https://github.com/docker/cli/pull/871)
 + Add support for multiple composefile when deploying [docker/cli#569](https://github.com/docker/cli/pull/569)
 - Fix broken Kubernetes stack flags [docker/cli#831](https://github.com/docker/cli/pull/831)
 - Fix stack marshaling for Kubernetes [docker/cli#890](https://github.com/docker/cli/pull/890)
@@ -30,36 +25,34 @@ be found.
 * Improve and fix bash completion for images [docker/cli#717](https://github.com/docker/cli/pull/717)
 + Added check for empty source in bind mount [docker/cli#824](https://github.com/docker/cli/pull/824)
 - Fix TLS from environment variables in client [moby/moby#36270](https://github.com/moby/moby/pull/36270)
-* Explain the columns shown in docker stats [docker/cli#860](https://github.com/docker/cli/pull/860)
-* GetAll -> Get to retrieve credentials from credential helpers [docker/cli#840](https://github.com/docker/cli/pull/840)
-+ Add Engine version to docker node ls [docker/cli#885](https://github.com/docker/cli/pull/885)
+* docker build now runs faster when registry-specific credential helper(s) are configured [docker/cli#840](https://github.com/docker/cli/pull/840)
 * Update event filter zsh completion with `disable`, `enable`, `install` and `remove` [docker/cli#372](https://github.com/docker/cli/pull/372)
-* Produce errors when empty ids are passed into inspect calls. [moby/moby#36144](https://github.com/moby/moby/pull/36144)
-* Marshall version [docker/cli#891](https://github.com/docker/cli/pull/891)
-* Replace go-bindata with esc [docker/cli#874](https://github.com/docker/cli/pull/874)
-* Set a non-zero timeout for HTTP client communication with plugin backend. [docker/cli#883](https://github.com/docker/cli/pull/883)
+* Produce errors when empty ids are passed into inspect calls [moby/moby#36144](https://github.com/moby/moby/pull/36144)
+* Marshall version for the k8s controller [docker/cli#891](https://github.com/docker/cli/pull/891)
+* Set a non-zero timeout for HTTP client communication with plugin backend [docker/cli#883](https://github.com/docker/cli/pull/883)
 + Add DOCKER_TLS environment variable for --tls option [docker/cli#863](https://github.com/docker/cli/pull/863)
+ Add --template-driver option for secrets/configs [docker/cli#896](https://github.com/docker/cli/pull/896)

 ### Logging

-* Awslogs - don't add new lines to maximum sized events [moby/moby#36078](https://github.com/moby/moby/pull/36078)
+* AWS logs - don't add new lines to maximum sized events [moby/moby#36078](https://github.com/moby/moby/pull/36078)
 * Move log validator logic after plugins are loaded [moby/moby#36306](https://github.com/moby/moby/pull/36306)
-* Support a proxy in splunk log driver [moby/moby#36220](https://github.com/moby/moby/pull/36220)
+* Support a proxy in Splunk log driver [moby/moby#36220](https://github.com/moby/moby/pull/36220)
 - Fix log tail with empty logs [moby/moby#36305](https://github.com/moby/moby/pull/36305)

 ### Networking

-* Bump libnetwork to 5ab4ab830062fe8a30a44b75b0bda6b1f4f166a4 [moby/moby#36099](https://github.com/moby/moby/pull/36099)
-* Document long form of --network and --network-add [docker/cli#843](https://github.com/docker/cli/pull/843)
 * Libnetwork revendoring [moby/moby#36137](https://github.com/moby/moby/pull/36137)
-* Migrates TestContainersAPINetworkMountsNoChown to api tests [moby/moby#36198](https://github.com/moby/moby/pull/36198)
+- Fix for deadlock on exit with Memberlist revendor [docker/libnetwork#2040](https://github.com/docker/libnetwork/pull/2040)
+* Fix user specified ndots option [docker/libnetwork#2065](https://github.com/docker/libnetwork/pull/2065) 
+- Fix to use ContainerID for Windows instead of SandboxID [docker/libnetwork#2010](https://github.com/docker/libnetwork/pull/2010)
 * Verify NetworkingConfig to make sure EndpointSettings is not nil [moby/moby#36077](https://github.com/moby/moby/pull/36077)
-+ Add description to TestContainerNetworkMountsNoChown [moby/moby#36226](https://github.com/moby/moby/pull/36226)
 - Fix `DockerNetworkInternalMode` issue [moby/moby#36298](https://github.com/moby/moby/pull/36298)
 - Fix race in attachable network attachment [moby/moby#36191](https://github.com/moby/moby/pull/36191)
- Fix the network option table [docker/cli#848](https://github.com/docker/cli/pull/848)
 - Fix timeout issue of `InspectNetwork` on AArch64 [moby/moby#36257](https://github.com/moby/moby/pull/36257)
 * Verbose info is missing for partial overlay ID [moby/moby#35989](https://github.com/moby/moby/pull/35989)
+* Update `FindNetwork` to address network name duplications [moby/moby#30897](https://github.com/moby/moby/pull/30897)
+* Disallow attaching ingress network [docker/swarmkit#2523](https://github.com/docker/swarmkit/pull/2523)

 ### Runtime

@@ -68,28 +61,24 @@ be found.
 * LCOW: Regular mount if only one layer [moby/moby#36052](https://github.com/moby/moby/pull/36052)
 * Remove interim env var LCOW_API_PLATFORM_IF_OMITTED [moby/moby#36269](https://github.com/moby/moby/pull/36269)
 * Revendor Microsoft/opengcs @ v0.3.6 [moby/moby#36108](https://github.com/moby/moby/pull/36108)
-* Windows: Bump to final RS3 build number [moby/moby#36268](https://github.com/moby/moby/pull/36268)
 - Fix issue of ExitCode and PID not show up in Task.Status.ContainerStatus [moby/moby#36150](https://github.com/moby/moby/pull/36150)
- Fix issue with plugin scanner going to deep [moby/moby#36119](https://github.com/moby/moby/pull/36119)
-* Do not make graphdriver homes private mounts. [moby/moby#36047](https://github.com/moby/moby/pull/36047)
+- Fix issue with plugin scanner going too deep [moby/moby#36119](https://github.com/moby/moby/pull/36119)
+* Do not make graphdriver homes private mounts [moby/moby#36047](https://github.com/moby/moby/pull/36047)
 * Do not recursive unmount on cleanup of zfs/btrfs [moby/moby#36237](https://github.com/moby/moby/pull/36237)
 * Don't restore image if layer does not exist [moby/moby#36304](https://github.com/moby/moby/pull/36304)
-** Adjust minimum API version for templated configs/secrets [moby/moby#36366](https://github.com/moby/moby/pull/36366)
+* Adjust minimum API version for templated configs/secrets [moby/moby#36366](https://github.com/moby/moby/pull/36366)
 * Bump containerd to 1.0.2 (cfd04396dc68220d1cecbe686a6cc3aa5ce3667c) [moby/moby#36308](https://github.com/moby/moby/pull/36308)
 * Bump Golang to 1.9.4 [moby/moby#36243](https://github.com/moby/moby/pull/36243)
 * Ensure daemon root is unmounted on shutdown [moby/moby#36107](https://github.com/moby/moby/pull/36107)
- Fix import path [moby/moby#36322](https://github.com/moby/moby/pull/36322)
 * Update runc to 6c55f98695e902427906eed2c799e566e3d3dfb5 [moby/moby#36222](https://github.com/moby/moby/pull/36222)
 - Fix container cleanup on daemon restart [moby/moby#36249](https://github.com/moby/moby/pull/36249)
-* Bump golang to 1.9.4 [docker/cli#868](https://github.com/docker/cli/pull/868)
 * Support SCTP port mapping (bump up API to v1.37) [moby/moby#33922](https://github.com/moby/moby/pull/33922)
 * Support SCTP port mapping [docker/cli#278](https://github.com/docker/cli/pull/278)
 - Fix Volumes property definition in ContainerConfig [moby/moby#35946](https://github.com/moby/moby/pull/35946)
 * Bump moby and dependencies [docker/cli#829](https://github.com/docker/cli/pull/829)
-* Bump moby to 0ede01237c9ab871f1b8db0364427407f3e46541 [docker/cli#894](https://github.com/docker/cli/pull/894)
-* Bump moby vendor and dependencies [docker/cli#892](https://github.com/docker/cli/pull/892)
 * C.RWLayer: check for nil before use [moby/moby#36242](https://github.com/moby/moby/pull/36242)
 + Add `REMOVE` and `ORPHANED` to TaskState [moby/moby#36146](https://github.com/moby/moby/pull/36146)
+- Fixed error detection using `IsErrNotFound` and `IsErrNotImplemented` for `ContainerStatPath`, `CopyFromContainer`, and `CopyToContainer` methods [moby/moby#35979](https://github.com/moby/moby/pull/35979)
 + Add an integration/internal/container helper package [moby/moby#36266](https://github.com/moby/moby/pull/36266)
 + Add canonical import path [moby/moby#36194](https://github.com/moby/moby/pull/36194)
 + Add/use container.Exec() to integration [moby/moby#36326](https://github.com/moby/moby/pull/36326)
@@ -104,17 +93,24 @@ be found.
 * Graphdriver: Fix RefCounter memory leak [moby/moby#36256](https://github.com/moby/moby/pull/36256)
 * Use continuity fs package for volume copy [moby/moby#36290](https://github.com/moby/moby/pull/36290)
 * Use proc/exe for reexec [moby/moby#36124](https://github.com/moby/moby/pull/36124)
-* Templated secrets and configs [moby/moby#33702](https://github.com/moby/moby/pull/33702)
+ Add API support for templated secrets and configs [moby/moby#33702](https://github.com/moby/moby/pull/33702) and [moby/moby#36366](https://github.com/moby/moby/pull/36366)
 * Use rslave propagation for mounts from daemon root [moby/moby#36055](https://github.com/moby/moby/pull/36055)
 + Add /proc/keys to masked paths [moby/moby#36368](https://github.com/moby/moby/pull/36368)
+* Bump Runc to 1.0.0-rc5 [moby/moby#36449](https://github.com/moby/moby/pull/36449)
+- Fixes `runc exec` on big-endian architectures [moby/moby#36449](https://github.com/moby/moby/pull/36449)
+* Use chroot when mount namespaces aren't provided [moby/moby#36449](https://github.com/moby/moby/pull/36449)
+- Fix systemd slice expansion so that it could be consumed by cAdvisor [moby/moby#36449](https://github.com/moby/moby/pull/36449)
+- Fix devices mounted with wrong uid/gid [moby/moby#36449](https://github.com/moby/moby/pull/36449)

 ### Swarm Mode

-* Bump SwarmKit to f74983e7c015a38a81c8642803a78b8322cf7eac [moby/moby#36274](https://github.com/moby/moby/pull/36274)
-* Clarify network plugins and swarm mode [docker/cli#869](https://github.com/docker/cli/pull/869)
-* Migrates several swarm configs tests from integration-cli to api tests [moby/moby#36291](https://github.com/moby/moby/pull/36291)
-* Migrates several swarm secrets from integration-cli to api tests [moby/moby#36283](https://github.com/moby/moby/pull/36283)
-* Update swarmkit to 68a376dc30d8c4001767c39456b990dbd821371b [moby/moby#36131](https://github.com/moby/moby/pull/36131)
-* [compose] Share the compose loading code between swarm and k8s stack deploy [docker/cli#845](https://github.com/docker/cli/pull/845)
+* Replace EC Private Key with PKCS#8 PEMs [docker/swarmkit#2246](https://github.com/docker/swarmkit/pull/2246)
+* Fix IP overlap with empty EndpointSpec [docker/swarmkit #2505](https://github.com/docker/swarmkit/pull/2505)
+* Add support for Support SCTP port mapping [docker/swarmkit#2298](https://github.com/docker/swarmkit/pull/2298)
+* Do not reschedule tasks if only placement constraints change and are satisfied by the assigned node [docker/swarmkit#2496](https://github.com/docker/swarmkit/pull/2496)
+* Ensure task reaper stopChan is closed no more than once [docker/swarmkit #2491](https://github.com/docker/swarmkit/pull/2491)
+* Synchronization fixes [docker/swarmkit#2495](https://github.com/docker/swarmkit/pull/2495)
+* Add log message to indicate message send retry if streaming unimplemented [docker/swarmkit#2483](https://github.com/docker/swarmkit/pull/2483)
+* Debug logs for session, node events on dispatcher, heartbeats [docker/swarmkit#2486](https://github.com/docker/swarmkit/pull/2486)
 + Add swarm types to bash completion event type filter [docker/cli#888](https://github.com/docker/cli/pull/888)
 - Fix issue where network inspect does not show Created time for networks in swarm scope [moby/moby#36095](https://github.com/moby/moby/pull/36095)
@@ -1 +1 @@
-18.03.0-ce-rc1
+18.03.0-ce-rc2
@@ -1 +1 @@
-18.03.0-ce-rc1
+18.03.0-ce-rc2
@@ -16,9 +16,10 @@ import (
 )

 type createOptions struct {
-	name   string
-	file   string
-	labels opts.ListOpts
+	name           string
+	templateDriver string
+	file           string
+	labels         opts.ListOpts
 }

 func newConfigCreateCommand(dockerCli command.Cli) *cobra.Command {
@@ -38,6 +39,8 @@ func newConfigCreateCommand(dockerCli command.Cli) *cobra.Command {
 	}
 	flags := cmd.Flags()
 	flags.VarP(&createOpts.labels, "label", "l", "Config labels")
+	flags.StringVar(&createOpts.templateDriver, "template-driver", "", "Template driver")
+	flags.SetAnnotation("driver", "version", []string{"1.37"})

 	return cmd
 }
@@ -68,7 +71,11 @@ func runConfigCreate(dockerCli command.Cli, options createOptions) error {
 		},
 		Data: configData,
 	}
-
+	if options.templateDriver != "" {
+		spec.Templating = &swarm.Driver{
+			Name: options.templateDriver,
+		}
+	}
 	r, err := client.ConfigCreate(ctx, spec)
 	if err != nil {
 		return err
@@ -82,14 +82,21 @@ func TestConfigCreateWithLabels(t *testing.T) {
 	}
 	name := "foo"

+	data, err := ioutil.ReadFile(filepath.Join("testdata", configDataFile))
+	assert.NoError(t, err)
+
+	expected := swarm.ConfigSpec{
+		Annotations: swarm.Annotations{
+			Name:   name,
+			Labels: expectedLabels,
+		},
+		Data: data,
+	}
+
 	cli := test.NewFakeCli(&fakeClient{
 		configCreateFunc: func(spec swarm.ConfigSpec) (types.ConfigCreateResponse, error) {
-			if spec.Name != name {
-				return types.ConfigCreateResponse{}, errors.Errorf("expected name %q, got %q", name, spec.Name)
-			}
-
-			if !reflect.DeepEqual(spec.Labels, expectedLabels) {
-				return types.ConfigCreateResponse{}, errors.Errorf("expected labels %v, got %v", expectedLabels, spec.Labels)
+			if !reflect.DeepEqual(spec, expected) {
+				return types.ConfigCreateResponse{}, errors.Errorf("expected %+v, got %+v", expected, spec)
 			}

 			return types.ConfigCreateResponse{
@@ -105,3 +112,32 @@ func TestConfigCreateWithLabels(t *testing.T) {
 	assert.NoError(t, cmd.Execute())
 	assert.Equal(t, "ID-"+name, strings.TrimSpace(cli.OutBuffer().String()))
 }
+
+func TestConfigCreateWithTemplatingDriver(t *testing.T) {
+	expectedDriver := &swarm.Driver{
+		Name: "template-driver",
+	}
+	name := "foo"
+
+	cli := test.NewFakeCli(&fakeClient{
+		configCreateFunc: func(spec swarm.ConfigSpec) (types.ConfigCreateResponse, error) {
+			if spec.Name != name {
+				return types.ConfigCreateResponse{}, errors.Errorf("expected name %q, got %q", name, spec.Name)
+			}
+
+			if spec.Templating.Name != expectedDriver.Name {
+				return types.ConfigCreateResponse{}, errors.Errorf("expected driver %v, got %v", expectedDriver, spec.Labels)
+			}
+
+			return types.ConfigCreateResponse{
+				ID: "ID-" + spec.Name,
+			}, nil
+		},
+	})
+
+	cmd := newConfigCreateCommand(cli)
+	cmd.SetArgs([]string{name, filepath.Join("testdata", configDataFile)})
+	cmd.Flags().Set("template-driver", expectedDriver.Name)
+	assert.NoError(t, cmd.Execute())
+	assert.Equal(t, "ID-"+name, strings.TrimSpace(cli.OutBuffer().String()))
+}
@@ -16,6 +16,7 @@ type fakeClient struct {
 	execInspectFunc       func(execID string) (types.ContainerExecInspect, error)
 	execCreateFunc        func(container string, config types.ExecConfig) (types.IDResponse, error)
 	createContainerFunc   func(config *container.Config, hostConfig *container.HostConfig, networkingConfig *network.NetworkingConfig, containerName string) (container.ContainerCreateCreatedBody, error)
+	containerStartFunc    func(container string, options types.ContainerStartOptions) error
 	imageCreateFunc       func(parentReference string, options types.ImageCreateOptions) (io.ReadCloser, error)
 	infoFunc              func() (types.Info, error)
 	containerStatPathFunc func(container, path string) (types.ContainerPathStat, error)
@@ -116,3 +117,10 @@ func (f *fakeClient) ContainerWait(_ context.Context, container string, _ contai
 	}
 	return nil, nil
 }
+
+func (f *fakeClient) ContainerStart(_ context.Context, container string, options types.ContainerStartOptions) error {
+	if f.containerStartFunc != nil {
+		return f.containerStartFunc(container, options)
+	}
+	return nil
+}
@@ -3,6 +3,7 @@ package container
 import (
 	"io"
 	"io/ioutil"
+	"os"
 	"runtime"
 	"strings"
 	"testing"
@@ -111,6 +112,33 @@ func TestRunCopyFromContainerToFilesystemMissingDestinationDirectory(t *testing.
 	testutil.ErrorContains(t, err, destDir.Join("missing"))
 }

+func TestRunCopyToContainerFromFileWithTrailingSlash(t *testing.T) {
+	srcFile := fs.NewFile(t, t.Name())
+	defer srcFile.Remove()
+
+	options := copyOptions{
+		source:      srcFile.Path() + string(os.PathSeparator),
+		destination: "container:/path",
+	}
+	cli := test.NewFakeCli(&fakeClient{})
+	err := runCopy(cli, options)
+	testutil.ErrorContains(t, err, "not a directory")
+}
+
+func TestRunCopyToContainerSourceDoesNotExist(t *testing.T) {
+	options := copyOptions{
+		source:      "/does/not/exist",
+		destination: "container:/path",
+	}
+	cli := test.NewFakeCli(&fakeClient{})
+	err := runCopy(cli, options)
+	expected := "no such file or directory"
+	if runtime.GOOS == "windows" {
+		expected = "cannot find the file specified"
+	}
+	testutil.ErrorContains(t, err, expected)
+}
+
 func TestSplitCpArg(t *testing.T) {
 	var testcases = []struct {
 		doc               string
@@ -145,7 +145,7 @@ func addFlags(flags *pflag.FlagSet) *containerOptions {
 		expose:            opts.NewListOpts(nil),
 		extraHosts:        opts.NewListOpts(opts.ValidateExtraHost),
 		groupAdd:          opts.NewListOpts(nil),
-		labels:            opts.NewListOpts(opts.ValidateLabel),
+		labels:            opts.NewListOpts(nil),
 		labelsFile:        opts.NewListOpts(nil),
 		linkLocalIPs:      opts.NewListOpts(nil),
 		links:             opts.NewListOpts(opts.ValidateLink),
@@ -0,0 +1,25 @@
+package container
+
+import (
+	"testing"
+
+	"github.com/docker/cli/internal/test"
+	"github.com/docker/docker/api/types/container"
+	"github.com/docker/docker/api/types/network"
+	"github.com/stretchr/testify/assert"
+)
+
+func TestRunLabel(t *testing.T) {
+	cli := test.NewFakeCli(&fakeClient{
+		createContainerFunc: func(_ *container.Config, _ *container.HostConfig, _ *network.NetworkingConfig, _ string) (container.ContainerCreateCreatedBody, error) {
+			return container.ContainerCreateCreatedBody{
+				ID: "id",
+			}, nil
+		},
+		Version: "1.36",
+	})
+	cmd := NewRunCommand(cli)
+	cmd.Flags().Set("detach", "true")
+	cmd.SetArgs([]string{"--label", "foo", "busybox"})
+	assert.NoError(t, cmd.Execute())
+}
@@ -16,10 +16,11 @@ import (
 )

 type createOptions struct {
-	name   string
-	driver string
-	file   string
-	labels opts.ListOpts
+	name           string
+	driver         string
+	templateDriver string
+	file           string
+	labels         opts.ListOpts
 }

 func newSecretCreateCommand(dockerCli command.Cli) *cobra.Command {
@@ -43,6 +44,8 @@ func newSecretCreateCommand(dockerCli command.Cli) *cobra.Command {
 	flags.VarP(&options.labels, "label", "l", "Secret labels")
 	flags.StringVarP(&options.driver, "driver", "d", "", "Secret driver")
 	flags.SetAnnotation("driver", "version", []string{"1.31"})
+	flags.StringVar(&options.templateDriver, "template-driver", "", "Template driver")
+	flags.SetAnnotation("driver", "version", []string{"1.37"})

 	return cmd
 }
@@ -71,7 +74,11 @@ func runSecretCreate(dockerCli command.Cli, options createOptions) error {
 			Name: options.driver,
 		}
 	}
-
+	if options.templateDriver != "" {
+		spec.Templating = &swarm.Driver{
+			Name: options.templateDriver,
+		}
+	}
 	r, err := client.SecretCreate(ctx, spec)
 	if err != nil {
 		return err
@@ -11,7 +11,6 @@ import (
 	"github.com/docker/cli/internal/test/testutil"
 	"github.com/docker/docker/api/types"
 	"github.com/docker/docker/api/types/swarm"
-	"github.com/gotestyourself/gotestyourself/golden"
 	"github.com/pkg/errors"
 	"github.com/stretchr/testify/assert"
 )
@@ -52,15 +51,22 @@ func TestSecretCreateErrors(t *testing.T) {

 func TestSecretCreateWithName(t *testing.T) {
 	name := "foo"
-	var actual []byte
+	data, err := ioutil.ReadFile(filepath.Join("testdata", secretDataFile))
+	assert.NoError(t, err)
+
+	expected := swarm.SecretSpec{
+		Annotations: swarm.Annotations{
+			Name:   name,
+			Labels: make(map[string]string),
+		},
+		Data: data,
+	}
+
 	cli := test.NewFakeCli(&fakeClient{
 		secretCreateFunc: func(spec swarm.SecretSpec) (types.SecretCreateResponse, error) {
-			if spec.Name != name {
-				return types.SecretCreateResponse{}, errors.Errorf("expected name %q, got %q", name, spec.Name)
+			if !reflect.DeepEqual(spec, expected) {
+				return types.SecretCreateResponse{}, errors.Errorf("expected %+v, got %+v", expected, spec)
 			}
-
-			actual = spec.Data
-
 			return types.SecretCreateResponse{
 				ID: "ID-" + spec.Name,
 			}, nil
@@ -70,7 +76,6 @@ func TestSecretCreateWithName(t *testing.T) {
 	cmd := newSecretCreateCommand(cli)
 	cmd.SetArgs([]string{name, filepath.Join("testdata", secretDataFile)})
 	assert.NoError(t, cmd.Execute())
-	golden.Assert(t, string(actual), secretDataFile)
 	assert.Equal(t, "ID-"+name, strings.TrimSpace(cli.OutBuffer().String()))
 }

@@ -86,7 +91,7 @@ func TestSecretCreateWithDriver(t *testing.T) {
 				return types.SecretCreateResponse{}, errors.Errorf("expected name %q, got %q", name, spec.Name)
 			}

-			if !reflect.DeepEqual(spec.Driver.Name, expectedDriver.Name) {
+			if spec.Driver.Name != expectedDriver.Name {
 				return types.SecretCreateResponse{}, errors.Errorf("expected driver %v, got %v", expectedDriver, spec.Labels)
 			}

@@ -103,6 +108,35 @@ func TestSecretCreateWithDriver(t *testing.T) {
 	assert.Equal(t, "ID-"+name, strings.TrimSpace(cli.OutBuffer().String()))
 }

+func TestSecretCreateWithTemplatingDriver(t *testing.T) {
+	expectedDriver := &swarm.Driver{
+		Name: "template-driver",
+	}
+	name := "foo"
+
+	cli := test.NewFakeCli(&fakeClient{
+		secretCreateFunc: func(spec swarm.SecretSpec) (types.SecretCreateResponse, error) {
+			if spec.Name != name {
+				return types.SecretCreateResponse{}, errors.Errorf("expected name %q, got %q", name, spec.Name)
+			}
+
+			if spec.Templating.Name != expectedDriver.Name {
+				return types.SecretCreateResponse{}, errors.Errorf("expected driver %v, got %v", expectedDriver, spec.Labels)
+			}
+
+			return types.SecretCreateResponse{
+				ID: "ID-" + spec.Name,
+			}, nil
+		},
+	})
+
+	cmd := newSecretCreateCommand(cli)
+	cmd.SetArgs([]string{name})
+	cmd.Flags().Set("template-driver", expectedDriver.Name)
+	assert.NoError(t, cmd.Execute())
+	assert.Equal(t, "ID-"+name, strings.TrimSpace(cli.OutBuffer().String()))
+}
+
 func TestSecretCreateWithLabels(t *testing.T) {
 	expectedLabels := map[string]string{
 		"lbl1": "Label-foo",
@@ -343,12 +343,12 @@ $ docker run -t -i --mount type=bind,src=/data,dst=/data busybox sh
 ### Publish or expose port (-p, --expose)

 ```bash
-$ docker run -p 127.0.0.1:80:8080 ubuntu bash
+$ docker run -p 127.0.0.1:80:8080/tcp ubuntu bash
 ```

-This binds port `8080` of the container to port `80` on `127.0.0.1` of the host
-machine. The [Docker User
-Guide](https://docs.docker.com/engine/userguide/networking/default_network/dockerlinks/)
+This binds port `8080` of the container to TCP port `80` on `127.0.0.1` of the host
+machine. You can also specify `udp` and `sctp` ports.
+The [Docker User Guide](https://docs.docker.com/engine/userguide/networking/default_network/dockerlinks/)
 explains in detail how to manipulate ports in Docker.

 ```bash
@@ -592,6 +592,7 @@ Docker supports the following restart policies:
 |:---------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
 | `no`                       | Do not automatically restart the container when it exits. This is the default.                                                                                                                                                                                   |
 | `on-failure[:max-retries]` | Restart only if the container exits with a non-zero exit status. Optionally, limit the number of restart retries the Docker daemon attempts.                                                                                                                     |
+| `unless-stopped`           | Restart the container unless it is explicitly stopped or Docker itself is stopped or restarted. |
 | `always`                   | Always restart the container regardless of the exit status. When you specify always, the Docker daemon will try to restart the container indefinitely. The container will also always start on daemon startup, regardless of the current state of the container. |

 ```bash
@@ -16,13 +16,13 @@ keywords: ["secret, create"]
 # secret create

 ```Markdown
-Usage:	docker secret create [OPTIONS] SECRET file|-
+Usage:	docker secret create [OPTIONS] SECRET [file|-]

 Create a secret from a file or STDIN as content

 Options:
-      --help          Print usage
-  -l, --label list    Secret labels (default [])
+  -l, --label list               Secret labels
+      --template-driver string   Template driver
 ```

 ## Description
@@ -813,7 +813,7 @@ The options you can specify are:
  <td><tt>--publish 8080:80/tcp</tt></td>
  <td><tt>--publish published=8080,target=80,protocol=tcp</tt></td>
  <td><p>
-    The protocol to use, either <tt>tcp</tt> or <tt>udp</tt> Defaults to
+    The protocol to use, <tt>tcp</tt> , <tt>udp</tt>, or <tt>sctp</tt>. Defaults to
    <tt>tcp</tt>. To bind a port for both protocols, specify the <tt>-p</tt> or
    <tt>--publish</tt> flag twice.
  </p></td>
@@ -13,7 +13,7 @@ github.com/docker/go d30aec9fd63c35133f8f79c3412ad91a3b08be06
 github.com/docker/go-connections 7beb39f0b969b075d1325fecb092faf27fd357b6
 github.com/docker/go-events 9461782956ad83b30282bf90e31fa6a70c255ba9
 github.com/docker/go-units 9e638d38cf6977a37a8ea0078f3ee75a7cdb2dd1
-github.com/docker/swarmkit f74983e7c015a38a81c8642803a78b8322cf7eac
+github.com/docker/swarmkit 11d7b06f48bc1d73fc6d8776c3552a4b11c94301
 github.com/emicklei/go-restful ff4f55a206334ef123e4f79bbf348980da81ca46
 github.com/emicklei/go-restful-swagger12 dcef7f55730566d41eae5db10e7d6981829720f6
 github.com/flynn-archive/go-shlex 3f9db97f856818214da2e1057f8ad84803971cff
@@ -187,11 +187,12 @@ RUN ./contrib/download-frozen-image-v2.sh /docker-frozen-images \
 	hello-world:latest@sha256:be0cd392e45be79ffeffa6b05338b98ebb16c87b255f48e297ec7f98e123905c
 # See also ensureFrozenImagesLinux() in "integration-cli/fixtures_linux_daemon_test.go" (which needs to be updated when adding images to this list)

-# Install tomlv, vndr, runc, containerd, tini, docker-proxy dockercli
-# Please edit hack/dockerfile/install-binaries.sh to update them.
-COPY hack/dockerfile/binaries-commits /tmp/binaries-commits
-COPY hack/dockerfile/install-binaries.sh /tmp/install-binaries.sh
-RUN /tmp/install-binaries.sh tomlv vndr runc containerd tini proxy dockercli gometalinter
+# Install tomlv, vndr, runc, containerd, tini, proxy dockercli
+# Please edit hack/dockerfile/install/<name>.installer to update them.
+COPY hack/dockerfile/install hack/dockerfile/install
+RUN for i in tomlv vndr tini gometalinter proxy dockercli runc containerd; \
+		do hack/dockerfile/install/install.sh $i; \
+	done
 ENV PATH=/usr/local/cli:$PATH

 # Activate bash completion and include Docker's completion if mounted with DOCKER_BASH_COMPLETION_PATH
@@ -151,14 +151,17 @@ RUN ./contrib/download-frozen-image-v2.sh /docker-frozen-images \
 	debian:jessie@sha256:287a20c5f73087ab406e6b364833e3fb7b3ae63ca0eb3486555dc27ed32c6e60 \
 	hello-world:latest@sha256:be0cd392e45be79ffeffa6b05338b98ebb16c87b255f48e297ec7f98e123905c
 # See also ensureFrozenImagesLinux() in "integration-cli/fixtures_linux_daemon_test.go" (which needs to be updated when adding images to this list)
+#

-# Install tomlv, vndr, runc, containerd, tini, docker-proxy
-# Please edit hack/dockerfile/install-binaries.sh to update them.
-COPY hack/dockerfile/binaries-commits /tmp/binaries-commits
-COPY hack/dockerfile/install-binaries.sh /tmp/install-binaries.sh
-RUN /tmp/install-binaries.sh tomlv vndr runc containerd tini proxy dockercli gometalinter
+# Install tomlv, vndr, runc, containerd, tini, proxy dockercli
+# Please edit hack/dockerfile/install/<name>.installer to update them.
+COPY hack/dockerfile/install hack/dockerfile/install
+RUN for i in tomlv vndr tini gometalinter proxy dockercli runc containerd; \
+		do hack/dockerfile/install/install.sh $i; \
+	done
 ENV PATH=/usr/local/cli:$PATH

+
 # Wrap all commands in the "docker-in-docker" script to allow nested containers
 ENTRYPOINT ["hack/dind"]

@@ -138,11 +138,12 @@ RUN ./contrib/download-frozen-image-v2.sh /docker-frozen-images \
 	hello-world:latest@sha256:be0cd392e45be79ffeffa6b05338b98ebb16c87b255f48e297ec7f98e123905c
 # See also ensureFrozenImagesLinux() in "integration-cli/fixtures_linux_daemon_test.go" (which needs to be updated when adding images to this list)

-# Install tomlv, vndr, runc, containerd, tini, docker-proxy
-# Please edit hack/dockerfile/install-binaries.sh to update them.
-COPY hack/dockerfile/binaries-commits /tmp/binaries-commits
-COPY hack/dockerfile/install-binaries.sh /tmp/install-binaries.sh
-RUN /tmp/install-binaries.sh tomlv vndr runc containerd tini proxy dockercli gometalinter
+# Install tomlv, vndr, runc, containerd, tini, proxy dockercli
+# Please edit hack/dockerfile/install/<name>.installer to update them.
+COPY hack/dockerfile/install hack/dockerfile/install
+RUN for i in tomlv vndr tini gometalinter proxy dockercli runc containerd; \
+		do hack/dockerfile/install/install.sh $i; \
+	done
 ENV PATH=/usr/local/cli:$PATH

 ENTRYPOINT ["hack/dind"]
@@ -23,7 +23,7 @@ RUN contrib/download-frozen-image-v2.sh /output/docker-frozen-images \

 # Download Docker CLI binary
 COPY hack/dockerfile hack/dockerfile
-RUN hack/dockerfile/install-binaries.sh dockercli
+RUN hack/dockerfile/install.sh dockercli

 # Set tag and add sources
 ARG DOCKER_GITCOMMIT
@@ -136,11 +136,12 @@ RUN ./contrib/download-frozen-image-v2.sh /docker-frozen-images \
 	hello-world:latest@sha256:be0cd392e45be79ffeffa6b05338b98ebb16c87b255f48e297ec7f98e123905c
 # See also ensureFrozenImagesLinux() in "integration-cli/fixtures_linux_daemon_test.go" (which needs to be updated when adding images to this list)

-# Install tomlv, vndr, runc, containerd, tini, docker-proxy
-# Please edit hack/dockerfile/install-binaries.sh to update them.
-COPY hack/dockerfile/binaries-commits /tmp/binaries-commits
-COPY hack/dockerfile/install-binaries.sh /tmp/install-binaries.sh
-RUN /tmp/install-binaries.sh tomlv vndr runc containerd tini proxy dockercli gometalinter
+# Install tomlv, vndr, runc, containerd, tini, proxy dockercli
+# Please edit hack/dockerfile/install/<name>.installer to update them.
+COPY hack/dockerfile/install hack/dockerfile/install
+RUN for i in tomlv vndr tini gometalinter proxy dockercli runc containerd; \
+		do hack/dockerfile/install/install.sh $i; \
+	done
 ENV PATH=/usr/local/cli:$PATH

 # Wrap all commands in the "docker-in-docker" script to allow nested containers
@@ -130,11 +130,12 @@ RUN ./contrib/download-frozen-image-v2.sh /docker-frozen-images \
 	hello-world:latest@sha256:be0cd392e45be79ffeffa6b05338b98ebb16c87b255f48e297ec7f98e123905c
 # See also ensureFrozenImagesLinux() in "integration-cli/fixtures_linux_daemon_test.go" (which needs to be updated when adding images to this list)

-# Install tomlv, vndr, runc, containerd, tini, docker-proxy
-# Please edit hack/dockerfile/install-binaries.sh to update them.
-COPY hack/dockerfile/binaries-commits /tmp/binaries-commits
-COPY hack/dockerfile/install-binaries.sh /tmp/install-binaries.sh
-RUN /tmp/install-binaries.sh tomlv vndr runc containerd tini proxy dockercli gometalinter
+# Install tomlv, vndr, runc, containerd, tini, proxy dockercli
+# Please edit hack/dockerfile/install/<name>.installer to update them.
+COPY hack/dockerfile/install hack/dockerfile/install
+RUN for i in tomlv vndr tini gometalinter proxy dockercli runc containerd; \
+		do hack/dockerfile/install/install.sh $i; \
+	done
 ENV PATH=/usr/local/cli:$PATH

 # Wrap all commands in the "docker-in-docker" script to allow nested containers
@@ -50,10 +50,11 @@ ENV GOPATH /go
 ENV CGO_LDFLAGS -L/lib

 # Install runc, containerd, tini and docker-proxy
-# Please edit hack/dockerfile/install-binaries.sh to update them.
-COPY hack/dockerfile/binaries-commits /tmp/binaries-commits
-COPY hack/dockerfile/install-binaries.sh /tmp/install-binaries.sh
-RUN /tmp/install-binaries.sh runc containerd tini proxy dockercli
+# Please edit hack/dockerfile/install/<name>.installer to update them.
+COPY hack/dockerfile/install hack/dockerfile/install
+RUN for i in runc containerd tini proxy dockercli; \
+		do hack/dockerfile/install/install.sh $i; \
+	done
 ENV PATH=/usr/local/cli:$PATH

 ENV AUTO_GOPATH 1
@@ -53,7 +53,7 @@ type Backend interface {

 // ImageBackend are the interface methods required from an image component
 type ImageBackend interface {
-	GetImageAndReleasableLayer(ctx context.Context, refOrID string, opts backend.GetImageAndLayerOptions) (Image, ReleaseableLayer, error)
+	GetImageAndReleasableLayer(ctx context.Context, refOrID string, opts backend.GetImageAndLayerOptions) (Image, ROLayer, error)
 }

 // ExecBackend contains the interface methods required for executing containers
@@ -100,10 +100,16 @@ type Image interface {
 	OperatingSystem() string
 }

-// ReleaseableLayer is an image layer that can be mounted and released
-type ReleaseableLayer interface {
+// ROLayer is a reference to image rootfs layer
+type ROLayer interface {
 	Release() error
-	Mount() (containerfs.ContainerFS, error)
-	Commit() (ReleaseableLayer, error)
+	NewRWLayer() (RWLayer, error)
 	DiffID() layer.DiffID
 }
+
+// RWLayer is active layer that can be read/modified
+type RWLayer interface {
+	Release() error
+	Root() containerfs.ContainerFS
+	Commit() (ROLayer, error)
+}
@@ -72,8 +72,12 @@ type copier struct {
 	source      builder.Source
 	pathCache   pathCache
 	download    sourceDownloader
-	tmpPaths    []string
 	platform    string
+	// for cleanup. TODO: having copier.cleanup() is error prone and hard to
+	// follow. Code calling performCopy should manage the lifecycle of its params.
+	// Copier should take override source as input, not imageMount.
+	activeLayer builder.RWLayer
+	tmpPaths    []string
 }

 func copierFromDispatchRequest(req dispatchRequest, download sourceDownloader, imageSource *imageMount) copier {
@@ -155,6 +159,10 @@ func (o *copier) Cleanup() {
 		os.RemoveAll(path)
 	}
 	o.tmpPaths = []string{}
+	if o.activeLayer != nil {
+		o.activeLayer.Release()
+		o.activeLayer = nil
+	}
 }

 // TODO: allowWildcards can probably be removed by refactoring this function further.
@@ -166,9 +174,15 @@ func (o *copier) calcCopyInfo(origPath string, allowWildcards bool) ([]copyInfo,
 	// done on image Source?
 	if imageSource != nil {
 		var err error
-		o.source, err = imageSource.Source()
+		rwLayer, err := imageSource.NewRWLayer()
 		if err != nil {
-			return nil, errors.Wrapf(err, "failed to copy from %s", imageSource.ImageID())
+			return nil, err
+		}
+		o.activeLayer = rwLayer
+
+		o.source, err = remotecontext.NewLazySource(rwLayer.Root())
+		if err != nil {
+			return nil, errors.Wrapf(err, "failed to create context for copy from %s", rwLayer.Root().Path())
 		}
 	}

@@ -127,7 +127,7 @@ func TestFromScratch(t *testing.T) {
 func TestFromWithArg(t *testing.T) {
 	tag, expected := ":sometag", "expectedthisid"

-	getImage := func(name string) (builder.Image, builder.ReleaseableLayer, error) {
+	getImage := func(name string) (builder.Image, builder.ROLayer, error) {
 		assert.Equal(t, "alpine"+tag, name)
 		return &mockImage{id: "expectedthisid"}, nil, nil
 	}
@@ -159,7 +159,7 @@ func TestFromWithArg(t *testing.T) {
 func TestFromWithUndefinedArg(t *testing.T) {
 	tag, expected := "sometag", "expectedthisid"

-	getImage := func(name string) (builder.Image, builder.ReleaseableLayer, error) {
+	getImage := func(name string) (builder.Image, builder.ROLayer, error) {
 		assert.Equal(t, "alpine", name)
 		return &mockImage{id: "expectedthisid"}, nil, nil
 	}
@@ -433,7 +433,7 @@ func TestRunWithBuildArgs(t *testing.T) {
 		return imageCache
 	}
 	b.imageProber = newImageProber(mockBackend, nil, false)
-	mockBackend.getImageFunc = func(_ string) (builder.Image, builder.ReleaseableLayer, error) {
+	mockBackend.getImageFunc = func(_ string) (builder.Image, builder.ROLayer, error) {
 		return &mockImage{
 			id:     "abcdef",
 			config: &container.Config{Cmd: origCmd},
@@ -5,7 +5,6 @@ import (

 	"github.com/docker/docker/api/types/backend"
 	"github.com/docker/docker/builder"
-	"github.com/docker/docker/builder/remotecontext"
 	dockerimage "github.com/docker/docker/image"
 	"github.com/docker/docker/pkg/system"
 	"github.com/pkg/errors"
@@ -13,7 +12,7 @@ import (
 	"golang.org/x/net/context"
 )

-type getAndMountFunc func(string, bool) (builder.Image, builder.ReleaseableLayer, error)
+type getAndMountFunc func(string, bool) (builder.Image, builder.ROLayer, error)

 // imageSources mounts images and provides a cache for mounted images. It tracks
 // all images so they can be unmounted at the end of the build.
@@ -24,7 +23,7 @@ type imageSources struct {
 }

 func newImageSources(ctx context.Context, options builderOptions) *imageSources {
-	getAndMount := func(idOrRef string, localOnly bool) (builder.Image, builder.ReleaseableLayer, error) {
+	getAndMount := func(idOrRef string, localOnly bool) (builder.Image, builder.ROLayer, error) {
 		pullOption := backend.PullOptionNoPull
 		if !localOnly {
 			if options.Options.PullParent {
@@ -92,32 +91,14 @@ func (m *imageSources) Add(im *imageMount) {
 type imageMount struct {
 	image  builder.Image
 	source builder.Source
-	layer  builder.ReleaseableLayer
+	layer  builder.ROLayer
 }

-func newImageMount(image builder.Image, layer builder.ReleaseableLayer) *imageMount {
+func newImageMount(image builder.Image, layer builder.ROLayer) *imageMount {
 	im := &imageMount{image: image, layer: layer}
 	return im
 }

-func (im *imageMount) Source() (builder.Source, error) {
-	if im.source == nil {
-		if im.layer == nil {
-			return nil, errors.Errorf("empty context")
-		}
-		mountPath, err := im.layer.Mount()
-		if err != nil {
-			return nil, errors.Wrapf(err, "failed to mount %s", im.image.ImageID())
-		}
-		source, err := remotecontext.NewLazySource(mountPath)
-		if err != nil {
-			return nil, errors.Wrapf(err, "failed to create lazycontext for %s", mountPath)
-		}
-		im.source = source
-	}
-	return im.source, nil
-}
-
 func (im *imageMount) unmount() error {
 	if im.layer == nil {
 		return nil
@@ -133,8 +114,8 @@ func (im *imageMount) Image() builder.Image {
 	return im.image
 }

-func (im *imageMount) Layer() builder.ReleaseableLayer {
-	return im.layer
+func (im *imageMount) NewRWLayer() (builder.RWLayer, error) {
+	return im.layer.NewRWLayer()
 }

 func (im *imageMount) ImageID() string {
@@ -17,6 +17,7 @@ import (
 	"github.com/docker/docker/api/types"
 	"github.com/docker/docker/api/types/backend"
 	"github.com/docker/docker/api/types/container"
+	"github.com/docker/docker/builder"
 	"github.com/docker/docker/image"
 	"github.com/docker/docker/pkg/archive"
 	"github.com/docker/docker/pkg/chrootarchive"
@@ -114,8 +115,8 @@ func (b *Builder) commitContainer(dispatchState *dispatchState, id string, conta
 	return err
 }

-func (b *Builder) exportImage(state *dispatchState, imageMount *imageMount, runConfig *container.Config) error {
-	newLayer, err := imageMount.Layer().Commit()
+func (b *Builder) exportImage(state *dispatchState, layer builder.RWLayer, parent builder.Image, runConfig *container.Config) error {
+	newLayer, err := layer.Commit()
 	if err != nil {
 		return err
 	}
@@ -124,7 +125,7 @@ func (b *Builder) exportImage(state *dispatchState, imageMount *imageMount, runC
 	// if there is an error before we can add the full mount with image
 	b.imageSources.Add(newImageMount(nil, newLayer))

-	parentImage, ok := imageMount.Image().(*image.Image)
+	parentImage, ok := parent.(*image.Image)
 	if !ok {
 		return errors.Errorf("unexpected image type")
 	}
@@ -177,7 +178,13 @@ func (b *Builder) performCopy(state *dispatchState, inst copyInstruction) error
 		return errors.Wrapf(err, "failed to get destination image %q", state.imageID)
 	}

-	destInfo, err := createDestInfo(state.runConfig.WorkingDir, inst, imageMount, b.options.Platform)
+	rwLayer, err := imageMount.NewRWLayer()
+	if err != nil {
+		return err
+	}
+	defer rwLayer.Release()
+
+	destInfo, err := createDestInfo(state.runConfig.WorkingDir, inst, rwLayer, b.options.Platform)
 	if err != nil {
 		return err
 	}
@@ -203,10 +210,10 @@ func (b *Builder) performCopy(state *dispatchState, inst copyInstruction) error
 			return errors.Wrapf(err, "failed to copy files")
 		}
 	}
-	return b.exportImage(state, imageMount, runConfigWithCommentCmd)
+	return b.exportImage(state, rwLayer, imageMount.Image(), runConfigWithCommentCmd)
 }

-func createDestInfo(workingDir string, inst copyInstruction, imageMount *imageMount, platform string) (copyInfo, error) {
+func createDestInfo(workingDir string, inst copyInstruction, rwLayer builder.RWLayer, platform string) (copyInfo, error) {
 	// Twiddle the destination when it's a relative path - meaning, make it
 	// relative to the WORKINGDIR
 	dest, err := normalizeDest(workingDir, inst.dest, platform)
@@ -214,12 +221,7 @@ func createDestInfo(workingDir string, inst copyInstruction, imageMount *imageMo
 		return copyInfo{}, errors.Wrapf(err, "invalid %s", inst.cmdName)
 	}

-	destMount, err := imageMount.Source()
-	if err != nil {
-		return copyInfo{}, errors.Wrapf(err, "failed to mount copy source")
-	}
-
-	return newCopyInfoFromSource(destMount, dest, ""), nil
+	return copyInfo{root: rwLayer.Root(), path: dest}, nil
 }

 // normalizeDest normalises the destination of a COPY/ADD command in a
@@ -20,7 +20,7 @@ import (
 type MockBackend struct {
 	containerCreateFunc func(config types.ContainerCreateConfig) (container.ContainerCreateCreatedBody, error)
 	commitFunc          func(backend.CommitConfig) (image.ID, error)
-	getImageFunc        func(string) (builder.Image, builder.ReleaseableLayer, error)
+	getImageFunc        func(string) (builder.Image, builder.ROLayer, error)
 	makeImageCacheFunc  func(cacheFrom []string) builder.ImageCache
 }

@@ -66,7 +66,7 @@ func (m *MockBackend) CopyOnBuild(containerID string, destPath string, srcRoot s
 	return nil
 }

-func (m *MockBackend) GetImageAndReleasableLayer(ctx context.Context, refOrID string, opts backend.GetImageAndLayerOptions) (builder.Image, builder.ReleaseableLayer, error) {
+func (m *MockBackend) GetImageAndReleasableLayer(ctx context.Context, refOrID string, opts backend.GetImageAndLayerOptions) (builder.Image, builder.ROLayer, error) {
 	if m.getImageFunc != nil {
 		return m.getImageFunc(refOrID)
 	}
@@ -124,14 +124,25 @@ func (l *mockLayer) Release() error {
 	return nil
 }

-func (l *mockLayer) Mount() (containerfs.ContainerFS, error) {
-	return containerfs.NewLocalContainerFS("mountPath"), nil
-}
-
-func (l *mockLayer) Commit() (builder.ReleaseableLayer, error) {
-	return nil, nil
+func (l *mockLayer) NewRWLayer() (builder.RWLayer, error) {
+	return &mockRWLayer{}, nil
 }

 func (l *mockLayer) DiffID() layer.DiffID {
 	return layer.DiffID("abcdef")
 }
+
+type mockRWLayer struct {
+}
+
+func (l *mockRWLayer) Release() error {
+	return nil
+}
+
+func (l *mockRWLayer) Commit() (builder.ROLayer, error) {
+	return nil, nil
+}
+
+func (l *mockRWLayer) Root() containerfs.ContainerFS {
+	return nil
+}
@@ -15,130 +15,126 @@ import (
 	"github.com/docker/docker/pkg/system"
 	"github.com/docker/docker/registry"
 	"github.com/pkg/errors"
-	"github.com/sirupsen/logrus"
 	"golang.org/x/net/context"
 )

-type releaseableLayer struct {
+type roLayer struct {
 	released   bool
 	layerStore layer.Store
 	roLayer    layer.Layer
-	rwLayer    layer.RWLayer
 }

-func (rl *releaseableLayer) Mount() (containerfs.ContainerFS, error) {
-	var err error
-	var mountPath containerfs.ContainerFS
+func (l *roLayer) DiffID() layer.DiffID {
+	if l.roLayer == nil {
+		return layer.DigestSHA256EmptyTar
+	}
+	return l.roLayer.DiffID()
+}
+
+func (l *roLayer) Release() error {
+	if l.released {
+		return nil
+	}
+	if l.roLayer != nil {
+		metadata, err := l.layerStore.Release(l.roLayer)
+		layer.LogReleaseMetadata(metadata)
+		if err != nil {
+			return errors.Wrap(err, "failed to release ROLayer")
+		}
+	}
+	l.roLayer = nil
+	l.released = true
+	return nil
+}
+
+func (l *roLayer) NewRWLayer() (builder.RWLayer, error) {
 	var chainID layer.ChainID
-	if rl.roLayer != nil {
-		chainID = rl.roLayer.ChainID()
+	if l.roLayer != nil {
+		chainID = l.roLayer.ChainID()
 	}

 	mountID := stringid.GenerateRandomID()
-	rl.rwLayer, err = rl.layerStore.CreateRWLayer(mountID, chainID, nil)
+	newLayer, err := l.layerStore.CreateRWLayer(mountID, chainID, nil)
 	if err != nil {
 		return nil, errors.Wrap(err, "failed to create rwlayer")
 	}

-	mountPath, err = rl.rwLayer.Mount("")
+	rwLayer := &rwLayer{layerStore: l.layerStore, rwLayer: newLayer}
+
+	fs, err := newLayer.Mount("")
 	if err != nil {
-		// Clean up the layer if we fail to mount it here.
-		metadata, err := rl.layerStore.ReleaseRWLayer(rl.rwLayer)
-		layer.LogReleaseMetadata(metadata)
-		if err != nil {
-			logrus.Errorf("Failed to release RWLayer: %s", err)
-		}
-		rl.rwLayer = nil
+		rwLayer.Release()
 		return nil, err
 	}

-	return mountPath, nil
+	rwLayer.fs = fs
+
+	return rwLayer, nil
 }

-func (rl *releaseableLayer) Commit() (builder.ReleaseableLayer, error) {
-	var chainID layer.ChainID
-	if rl.roLayer != nil {
-		chainID = rl.roLayer.ChainID()
-	}
+type rwLayer struct {
+	released   bool
+	layerStore layer.Store
+	rwLayer    layer.RWLayer
+	fs         containerfs.ContainerFS
+}

-	stream, err := rl.rwLayer.TarStream()
+func (l *rwLayer) Root() containerfs.ContainerFS {
+	return l.fs
+}
+
+func (l *rwLayer) Commit() (builder.ROLayer, error) {
+	stream, err := l.rwLayer.TarStream()
 	if err != nil {
 		return nil, err
 	}
 	defer stream.Close()

-	newLayer, err := rl.layerStore.Register(stream, chainID)
+	var chainID layer.ChainID
+	if parent := l.rwLayer.Parent(); parent != nil {
+		chainID = parent.ChainID()
+	}
+
+	newLayer, err := l.layerStore.Register(stream, chainID)
 	if err != nil {
 		return nil, err
 	}
 	// TODO: An optimization would be to handle empty layers before returning
-	return &releaseableLayer{layerStore: rl.layerStore, roLayer: newLayer}, nil
+	return &roLayer{layerStore: l.layerStore, roLayer: newLayer}, nil
 }

-func (rl *releaseableLayer) DiffID() layer.DiffID {
-	if rl.roLayer == nil {
-		return layer.DigestSHA256EmptyTar
-	}
-	return rl.roLayer.DiffID()
-}
-
-func (rl *releaseableLayer) Release() error {
-	if rl.released {
+func (l *rwLayer) Release() error {
+	if l.released {
 		return nil
 	}
-	if err := rl.releaseRWLayer(); err != nil {
-		// Best effort attempt at releasing read-only layer before returning original error.
-		rl.releaseROLayer()
-		return err
+
+	if l.fs != nil {
+		if err := l.rwLayer.Unmount(); err != nil {
+			return errors.Wrap(err, "failed to unmount RWLayer")
+		}
+		l.fs = nil
 	}
-	if err := rl.releaseROLayer(); err != nil {
-		return err
+
+	metadata, err := l.layerStore.ReleaseRWLayer(l.rwLayer)
+	layer.LogReleaseMetadata(metadata)
+	if err != nil {
+		return errors.Wrap(err, "failed to release RWLayer")
 	}
-	rl.released = true
+	l.released = true
 	return nil
 }

-func (rl *releaseableLayer) releaseRWLayer() error {
-	if rl.rwLayer == nil {
-		return nil
-	}
-	if err := rl.rwLayer.Unmount(); err != nil {
-		logrus.Errorf("Failed to unmount RWLayer: %s", err)
-		return err
-	}
-	metadata, err := rl.layerStore.ReleaseRWLayer(rl.rwLayer)
-	layer.LogReleaseMetadata(metadata)
-	if err != nil {
-		logrus.Errorf("Failed to release RWLayer: %s", err)
-	}
-	rl.rwLayer = nil
-	return err
-}
-
-func (rl *releaseableLayer) releaseROLayer() error {
-	if rl.roLayer == nil {
-		return nil
-	}
-	metadata, err := rl.layerStore.Release(rl.roLayer)
-	layer.LogReleaseMetadata(metadata)
-	if err != nil {
-		logrus.Errorf("Failed to release ROLayer: %s", err)
-	}
-	rl.roLayer = nil
-	return err
-}
-
-func newReleasableLayerForImage(img *image.Image, layerStore layer.Store) (builder.ReleaseableLayer, error) {
+func newROLayerForImage(img *image.Image, layerStore layer.Store) (builder.ROLayer, error) {
 	if img == nil || img.RootFS.ChainID() == "" {
-		return &releaseableLayer{layerStore: layerStore}, nil
+		return &roLayer{layerStore: layerStore}, nil
 	}
 	// Hold a reference to the image layer so that it can't be removed before
 	// it is released
-	roLayer, err := layerStore.Get(img.RootFS.ChainID())
+	layer, err := layerStore.Get(img.RootFS.ChainID())
 	if err != nil {
 		return nil, errors.Wrapf(err, "failed to get layer for image %s", img.ImageID())
 	}
-	return &releaseableLayer{layerStore: layerStore, roLayer: roLayer}, nil
+	return &roLayer{layerStore: layerStore, roLayer: layer}, nil
 }

 // TODO: could this use the regular daemon PullImage ?
@@ -170,12 +166,12 @@ func (daemon *Daemon) pullForBuilder(ctx context.Context, name string, authConfi
 // GetImageAndReleasableLayer returns an image and releaseable layer for a reference or ID.
 // Every call to GetImageAndReleasableLayer MUST call releasableLayer.Release() to prevent
 // leaking of layers.
-func (daemon *Daemon) GetImageAndReleasableLayer(ctx context.Context, refOrID string, opts backend.GetImageAndLayerOptions) (builder.Image, builder.ReleaseableLayer, error) {
+func (daemon *Daemon) GetImageAndReleasableLayer(ctx context.Context, refOrID string, opts backend.GetImageAndLayerOptions) (builder.Image, builder.ROLayer, error) {
 	if refOrID == "" {
 		if !system.IsOSSupported(opts.OS) {
 			return nil, nil, system.ErrNotSupportedOperatingSystem
 		}
-		layer, err := newReleasableLayerForImage(nil, daemon.layerStores[opts.OS])
+		layer, err := newROLayerForImage(nil, daemon.layerStores[opts.OS])
 		return nil, layer, err
 	}

@@ -189,7 +185,7 @@ func (daemon *Daemon) GetImageAndReleasableLayer(ctx context.Context, refOrID st
 			if !system.IsOSSupported(image.OperatingSystem()) {
 				return nil, nil, system.ErrNotSupportedOperatingSystem
 			}
-			layer, err := newReleasableLayerForImage(image, daemon.layerStores[image.OperatingSystem()])
+			layer, err := newROLayerForImage(image, daemon.layerStores[image.OperatingSystem()])
 			return image, layer, err
 		}
 	}
@@ -201,7 +197,7 @@ func (daemon *Daemon) GetImageAndReleasableLayer(ctx context.Context, refOrID st
 	if !system.IsOSSupported(image.OperatingSystem()) {
 		return nil, nil, system.ErrNotSupportedOperatingSystem
 	}
-	layer, err := newReleasableLayerForImage(image, daemon.layerStores[image.OperatingSystem()])
+	layer, err := newROLayerForImage(image, daemon.layerStores[image.OperatingSystem()])
 	return image, layer, err
 }

@@ -18,6 +18,7 @@ import (
 	containertypes "github.com/docker/docker/api/types/container"
 	"github.com/docker/docker/api/types/events"
 	containerpkg "github.com/docker/docker/container"
+	"github.com/docker/docker/daemon"
 	"github.com/docker/docker/daemon/cluster/convert"
 	executorpkg "github.com/docker/docker/daemon/cluster/executor"
 	"github.com/docker/libnetwork"
@@ -153,7 +154,11 @@ func (c *containerAdapter) createNetworks(ctx context.Context) error {
 			if _, ok := err.(libnetwork.NetworkNameError); ok {
 				continue
 			}
-
+			// We will continue if CreateManagedNetwork returns PredefinedNetworkError error.
+			// Other callers still can treat it as Error.
+			if _, ok := err.(daemon.PredefinedNetworkError); ok {
+				continue
+			}
 			return err
 		}
 	}
@@ -24,6 +24,16 @@ import (
 	"golang.org/x/net/context"
 )

+// PredefinedNetworkError is returned when user tries to create predefined network that already exists.
+type PredefinedNetworkError string
+
+func (pnr PredefinedNetworkError) Error() string {
+	return fmt.Sprintf("operation is not permitted on predefined %s network ", string(pnr))
+}
+
+// Forbidden denotes the type of this error
+func (pnr PredefinedNetworkError) Forbidden() {}
+
 // NetworkControllerEnabled checks if the networking stack is enabled.
 // This feature depends on OS primitives and it's disabled in systems like Windows.
 func (daemon *Daemon) NetworkControllerEnabled() bool {
@@ -267,9 +277,8 @@ func (daemon *Daemon) CreateNetwork(create types.NetworkCreateRequest) (*types.N
 }

 func (daemon *Daemon) createNetwork(create types.NetworkCreateRequest, id string, agent bool) (*types.NetworkCreateResponse, error) {
-	if runconfig.IsPreDefinedNetwork(create.Name) && !agent {
-		err := fmt.Errorf("%s is a pre-defined network and cannot be created", create.Name)
-		return nil, errdefs.Forbidden(err)
+	if runconfig.IsPreDefinedNetwork(create.Name) {
+		return nil, PredefinedNetworkError(create.Name)
 	}

 	var warning string
@@ -1,23 +0,0 @@
-#!/bin/sh
-
-# When updating TOMLV_COMMIT, consider updating github.com/BurntSushi/toml
-# in vendor.conf accordingly
-TOMLV_COMMIT=a368813c5e648fee92e5f6c30e3944ff9d5e8895
-
-# When updating RUNC_COMMIT, also update runc in vendor.conf accordingly
-RUNC_COMMIT=6c55f98695e902427906eed2c799e566e3d3dfb5
-
-# containerd is also pinned in vendor.conf. When updating the binary
-# version you may also need to update the vendor version to pick up bug
-# fixes or new APIs.
-CONTAINERD_COMMIT=cfd04396dc68220d1cecbe686a6cc3aa5ce3667c # v1.0.2
-TINI_COMMIT=949e6facb77383876aeff8a6944dde66b3089574
-
-# LIBNETWORK_COMMIT is used to build the docker-userland-proxy binary. When
-# updating the binary version, consider updating github.com/docker/libnetwork
-# in vendor.conf accordingly
-LIBNETWORK_COMMIT=ed2130d117c11c542327b4d5216a5db36770bc65
-VNDR_COMMIT=a6e196d8b4b0cbbdc29aebdb20c59ac6926bb384
-
-# Linting
-GOMETALINTER_COMMIT=bfcc1d6942136fd86eb6f1a6fb328de8398fbd80
@@ -1,176 +0,0 @@
-#!/usr/bin/env bash
-set -e
-set -x
-
-. $(dirname "$0")/binaries-commits
-
-RM_GOPATH=0
-
-TMP_GOPATH=${TMP_GOPATH:-""}
-
-if [ -z "$TMP_GOPATH" ]; then
-	export GOPATH="$(mktemp -d)"
-	RM_GOPATH=1
-else
-	export GOPATH="$TMP_GOPATH"
-fi
-
-# Do not build with ambient capabilities support
-RUNC_BUILDTAGS="${RUNC_BUILDTAGS:-"seccomp apparmor selinux"}"
-
-install_runc() {
-	echo "Install runc version $RUNC_COMMIT"
-	git clone https://github.com/opencontainers/runc.git "$GOPATH/src/github.com/opencontainers/runc"
-	cd "$GOPATH/src/github.com/opencontainers/runc"
-	git checkout -q "$RUNC_COMMIT"
-	make BUILDTAGS="$RUNC_BUILDTAGS" $1
-	cp runc /usr/local/bin/docker-runc
-}
-
-install_containerd() {
-	echo "Install containerd version $CONTAINERD_COMMIT"
-	git clone https://github.com/containerd/containerd.git "$GOPATH/src/github.com/containerd/containerd"
-	cd "$GOPATH/src/github.com/containerd/containerd"
-	git checkout -q "$CONTAINERD_COMMIT"
-	(
-		export GOPATH
-		make
-	)
-	cp bin/containerd /usr/local/bin/docker-containerd
-	cp bin/containerd-shim /usr/local/bin/docker-containerd-shim
-	cp bin/ctr /usr/local/bin/docker-containerd-ctr
-}
-
-install_containerd_static() {
-	echo "Install containerd version $CONTAINERD_COMMIT"
-	git clone https://github.com/containerd/containerd.git "$GOPATH/src/github.com/containerd/containerd"
-	cd "$GOPATH/src/github.com/containerd/containerd"
-	git checkout -q "$CONTAINERD_COMMIT"
-	(
-		export GOPATH
-		make BUILDTAGS='static_build netgo' EXTRA_FLAGS="-buildmode pie" EXTRA_LDFLAGS='-extldflags "-fno-PIC -static"'
-	)
-	cp bin/containerd /usr/local/bin/docker-containerd
-	cp bin/containerd-shim /usr/local/bin/docker-containerd-shim
-	cp bin/ctr /usr/local/bin/docker-containerd-ctr
-}
-
-install_proxy() {
-	echo "Install docker-proxy version $LIBNETWORK_COMMIT"
-	git clone https://github.com/docker/libnetwork.git "$GOPATH/src/github.com/docker/libnetwork"
-	cd "$GOPATH/src/github.com/docker/libnetwork"
-	git checkout -q "$LIBNETWORK_COMMIT"
-	go build -buildmode=pie -ldflags="$PROXY_LDFLAGS" -o /usr/local/bin/docker-proxy github.com/docker/libnetwork/cmd/proxy
-}
-
-install_dockercli() {
-	DOCKERCLI_CHANNEL=${DOCKERCLI_CHANNEL:-edge}
-	DOCKERCLI_VERSION=${DOCKERCLI_VERSION:-17.06.0-ce}
-	echo "Install docker/cli version $DOCKERCLI_VERSION from $DOCKERCLI_CHANNEL"
-
-	arch=$(uname -m)
-	# No official release of these platforms
-	if [[ "$arch" != "x86_64" ]] && [[ "$arch" != "s390x" ]]; then
-		build_dockercli
-		return
-	fi
-
-	url=https://download.docker.com/linux/static
-	curl -Ls $url/$DOCKERCLI_CHANNEL/$arch/docker-$DOCKERCLI_VERSION.tgz | \
-	tar -xz docker/docker
-	mv docker/docker /usr/local/bin/
-	rmdir docker
-}
-
-build_dockercli() {
-	DOCKERCLI_VERSION=${DOCKERCLI_VERSION:-17.06.0-ce}
-	git clone https://github.com/docker/docker-ce "$GOPATH/tmp/docker-ce"
-	cd "$GOPATH/tmp/docker-ce"
-	git checkout -q "v$DOCKERCLI_VERSION"
-	mkdir -p "$GOPATH/src/github.com/docker"
-	mv components/cli "$GOPATH/src/github.com/docker/cli"
-	go build -buildmode=pie -o /usr/local/bin/docker github.com/docker/cli/cmd/docker
-}
-
-install_gometalinter() {
-	echo "Installing gometalinter version $GOMETALINTER_COMMIT"
-	go get -d github.com/alecthomas/gometalinter
-	cd "$GOPATH/src/github.com/alecthomas/gometalinter"
-	git checkout -q "$GOMETALINTER_COMMIT"
-	go build -buildmode=pie -o /usr/local/bin/gometalinter github.com/alecthomas/gometalinter
-	GOBIN=/usr/local/bin gometalinter --install
-}
-
-for prog in "$@"
-do
-	case $prog in
-		tomlv)
-			echo "Install tomlv version $TOMLV_COMMIT"
-			git clone https://github.com/BurntSushi/toml.git "$GOPATH/src/github.com/BurntSushi/toml"
-			cd "$GOPATH/src/github.com/BurntSushi/toml" && git checkout -q "$TOMLV_COMMIT"
-			go build -buildmode=pie -v -o /usr/local/bin/tomlv github.com/BurntSushi/toml/cmd/tomlv
-			;;
-
-		runc)
-			install_runc static
-			;;
-
-		runc-dynamic)
-			install_runc
-			;;
-
-		containerd)
-			install_containerd_static
-			;;
-
-		containerd-dynamic)
-			install_containerd
-			;;
-
-		gometalinter)
-			install_gometalinter
-			;;
-
-		tini)
-			echo "Install tini version $TINI_COMMIT"
-			git clone https://github.com/krallin/tini.git "$GOPATH/tini"
-			cd "$GOPATH/tini"
-			git checkout -q "$TINI_COMMIT"
-			cmake .
-			make tini-static
-			cp tini-static /usr/local/bin/docker-init
-			;;
-
-		proxy)
-			(
-				export CGO_ENABLED=0
-				install_proxy
-			)
-			;;
-
-		proxy-dynamic)
-			PROXY_LDFLAGS="-linkmode=external" install_proxy
-			;;
-
-		vndr)
-			echo "Install vndr version $VNDR_COMMIT"
-			git clone https://github.com/LK4D4/vndr.git "$GOPATH/src/github.com/LK4D4/vndr"
-			cd "$GOPATH/src/github.com/LK4D4/vndr"
-			git checkout -q "$VNDR_COMMIT"
-			go build -buildmode=pie -v -o /usr/local/bin/vndr .
-			;;
-
-		dockercli)
-			install_dockercli
-			;;
-
-		*)
-			echo echo "Usage: $0 [tomlv|runc|runc-dynamic|containerd|containerd-dynamic|tini|proxy|proxy-dynamic|vndr|dockercli|gometalinter]"
-			exit 1
-
-	esac
-done
-
-if [ $RM_GOPATH -eq 1 ]; then
-	rm -rf "$GOPATH"
-fi
@@ -0,0 +1,31 @@
+#!/bin/sh
+
+
+# containerd is also pinned in vendor.conf. When updating the binary
+# version you may also need to update the vendor version to pick up bug
+# fixes or new APIs.
+CONTAINERD_COMMIT=cfd04396dc68220d1cecbe686a6cc3aa5ce3667c # v1.0.2
+
+install_containerd() {
+	echo "Install containerd version $CONTAINERD_COMMIT"
+	git clone https://github.com/containerd/containerd.git "$GOPATH/src/github.com/containerd/containerd"
+	cd "$GOPATH/src/github.com/containerd/containerd"
+	git checkout -q "$CONTAINERD_COMMIT"
+
+	(
+
+		if [ "$1" == "static" ]; then
+			export BUILDTAGS='static_build netgo'
+			export EXTRA_FLAGS='-buildmod pie'
+			export EXTRA_LDFLAGS='-extldflags "-fno-PIC -static"'
+		fi
+
+		make
+	)
+
+	mkdir -p ${PREFIX}
+
+	cp bin/containerd ${PREFIX}/docker-containerd
+	cp bin/containerd-shim ${PREFIX}/docker-containerd-shim
+	cp bin/ctr ${PREFIX}/docker-containerd-ctr
+}
@@ -0,0 +1,31 @@
+#!/bin/sh
+
+DOCKERCLI_CHANNEL=${DOCKERCLI_CHANNEL:-edge}
+DOCKERCLI_VERSION=${DOCKERCLI_VERSION:-17.06.0-ce}
+
+install_dockercli() {
+	echo "Install docker/cli version $DOCKERCLI_VERSION from $DOCKERCLI_CHANNEL"
+
+	arch=$(uname -m)
+	# No official release of these platforms
+	if [[ "$arch" != "x86_64" ]] && [[ "$arch" != "s390x" ]]; then
+		build_dockercli
+		return
+	fi
+
+	url=https://download.docker.com/linux/static
+	curl -Ls $url/$DOCKERCLI_CHANNEL/$arch/docker-$DOCKERCLI_VERSION.tgz | \
+	tar -xz docker/docker
+	mkdir -p ${PREFIX}
+	mv docker/docker ${PREFIX}/
+	rmdir docker
+}
+
+build_dockercli() {
+	git clone https://github.com/docker/docker-ce "$GOPATH/tmp/docker-ce"
+	cd "$GOPATH/tmp/docker-ce"
+	git checkout -q "v$DOCKERCLI_VERSION"
+	mkdir -p "$GOPATH/src/github.com/docker"
+	mv components/cli "$GOPATH/src/github.com/docker/cli"
+	go build -buildmode=pie -o ${PREFIX}/docker github.com/docker/cli/cmd/docker
+}
@@ -0,0 +1,12 @@
+#!/bin/sh
+
+GOMETALINTER_COMMIT=bfcc1d6942136fd86eb6f1a6fb328de8398fbd80
+
+install_gometalinter() {
+	echo "Installing gometalinter version $GOMETALINTER_COMMIT"
+	go get -d github.com/alecthomas/gometalinter
+	cd "$GOPATH/src/github.com/alecthomas/gometalinter"
+	git checkout -q "$GOMETALINTER_COMMIT"
+	go build -buildmode=pie -o ${PREFIX}/gometalinter github.com/alecthomas/gometalinter
+	GOBIN=${PREFIX} ${PREFIX}/gometalinter --install
+}
@@ -0,0 +1,30 @@
+#!/bin/bash
+
+set -e
+set -x
+
+RM_GOPATH=0
+
+TMP_GOPATH=${TMP_GOPATH:-""}
+
+: ${PREFIX:="/usr/local/bin"}
+
+if [ -z "$TMP_GOPATH" ]; then
+	export GOPATH="$(mktemp -d)"
+	RM_GOPATH=1
+else
+	export GOPATH="$TMP_GOPATH"
+fi
+
+dir="$(dirname $0)"
+
+bin=$1
+shift
+
+if [ ! -f "${dir}/${bin}.installer" ]; then
+	echo "Could not find installer for \"$bin\""
+	exit 1
+fi
+
+. $dir/$bin.installer
+install_$bin "$@"
@@ -0,0 +1,37 @@
+#!/bin/sh
+
+# LIBNETWORK_COMMIT is used to build the docker-userland-proxy binary. When
+# updating the binary version, consider updating github.com/docker/libnetwork
+# in vendor.conf accordingly
+LIBNETWORK_COMMIT=ed2130d117c11c542327b4d5216a5db36770bc65
+
+install_proxy() {
+	case "$1" in
+		"dynamic")
+			install_proxy_dynamic
+			return
+			;;
+		"")
+			export CGO_ENABLED=0
+			_install_proxy
+			;;
+		*)
+			echo 'Usage: $0 [dynamic]'
+			;;
+	esac
+}
+
+install_proxy_dynamic() {
+	export PROXY_LDFLAGS="-linkmode=external" install_proxy
+	_install_proxy
+}
+
+_install_proxy() {
+	echo "Install docker-proxy version $LIBNETWORK_COMMIT"
+	git clone https://github.com/docker/libnetwork.git "$GOPATH/src/github.com/docker/libnetwork"
+	cd "$GOPATH/src/github.com/docker/libnetwork"
+	git checkout -q "$LIBNETWORK_COMMIT"
+	go build -buildmode=pie -ldflags="$PROXY_LDFLAGS" -o ${PREFIX}/docker-proxy github.com/docker/libnetwork/cmd/proxy
+}
+
+
@@ -0,0 +1,17 @@
+#!/bin/sh
+
+# When updating RUNC_COMMIT, also update runc in vendor.conf accordingly
+RUNC_COMMIT=4fc53a81fb7c994640722ac585fa9ca548971871
+
+install_runc() {
+	# Do not build with ambient capabilities support
+	RUNC_BUILDTAGS="${RUNC_BUILDTAGS:-"seccomp apparmor selinux"}"
+
+	echo "Install runc version $RUNC_COMMIT"
+	git clone https://github.com/opencontainers/runc.git "$GOPATH/src/github.com/opencontainers/runc"
+	cd "$GOPATH/src/github.com/opencontainers/runc"
+	git checkout -q "$RUNC_COMMIT"
+	make BUILDTAGS="$RUNC_BUILDTAGS" $1
+	mkdir -p ${PREFIX}
+	cp runc ${PREFIX}/docker-runc
+}
@@ -0,0 +1,14 @@
+#!/bin/sh
+
+TINI_COMMIT=949e6facb77383876aeff8a6944dde66b3089574
+
+install_tini() {
+	echo "Install tini version $TINI_COMMIT"
+	git clone https://github.com/krallin/tini.git "$GOPATH/tini"
+	cd "$GOPATH/tini"
+	git checkout -q "$TINI_COMMIT"
+	cmake .
+	make tini-static
+	mkdir -p ${PREFIX}
+	cp tini-static ${PREFIX}/docker-init
+}
@@ -0,0 +1,12 @@
+#!/bin/sh
+
+# When updating TOMLV_COMMIT, consider updating github.com/BurntSushi/toml
+# in vendor.conf accordingly
+TOMLV_COMMIT=a368813c5e648fee92e5f6c30e3944ff9d5e8895
+
+install_tomlv() {
+	echo "Install tomlv version $TOMLV_COMMIT"
+	git clone https://github.com/BurntSushi/toml.git "$GOPATH/src/github.com/BurntSushi/toml"
+	cd "$GOPATH/src/github.com/BurntSushi/toml" && git checkout -q "$TOMLV_COMMIT"
+	go build -v -buildmode=pie -o ${PREFIX}/tomlv github.com/BurntSushi/toml/cmd/tomlv
+}
@@ -0,0 +1,11 @@
+#!/bin/sh
+
+VNDR_COMMIT=a6e196d8b4b0cbbdc29aebdb20c59ac6926bb384
+
+install_vndr() {
+	echo "Install vndr version $VNDR_COMMIT"
+	git clone https://github.com/LK4D4/vndr.git "$GOPATH/src/github.com/LK4D4/vndr"
+	cd "$GOPATH/src/github.com/LK4D4/vndr"
+	git checkout -q "$VNDR_COMMIT"
+	go build -buildmode=pie -v -o ${PREFIX}/vndr .
+}
@@ -2,7 +2,9 @@

 rm -rf autogen

-source hack/dockerfile/binaries-commits
+source hack/dockerfile/install/runc.installer
+source hack/dockerfile/install/tini.installer
+source hack/dockerfile/install/containerd.installer

 cat > dockerversion/version_autogen.go <<DVEOF
 // +build autogen
@@ -1439,6 +1439,7 @@ func (s *DockerSuite) TestBuildRelativeCopy(c *check.C) {
 	))
 }

+// FIXME(vdemeester) should be unit test
 func (s *DockerSuite) TestBuildBlankName(c *check.C) {
 	name := "testbuildblankname"
 	testCases := []struct {
@@ -2066,6 +2067,7 @@ func (s *DockerSuite) TestBuildNoContext(c *check.C) {
 	}
 }

+// FIXME(vdemeester) migrate to docker/cli e2e
 func (s *DockerSuite) TestBuildDockerfileStdin(c *check.C) {
 	name := "stdindockerfile"
 	tmpDir, err := ioutil.TempDir("", "fake-context")
@@ -2085,6 +2087,7 @@ CMD ["cat", "/foo"]`),
 	c.Assert(strings.TrimSpace(string(res)), checker.Equals, `[cat /foo]`)
 }

+// FIXME(vdemeester) migrate to docker/cli tests (unit or e2e)
 func (s *DockerSuite) TestBuildDockerfileStdinConflict(c *check.C) {
 	name := "stdindockerfiletarcontext"
 	icmd.RunCmd(icmd.Cmd{
@@ -2401,6 +2404,7 @@ func (s *DockerSuite) TestBuildDockerignoringDockerfile(c *check.C) {
 		build.WithFile("Dockerfile", dockerfile),
 		build.WithFile(".dockerignore", "Dockerfile\n"),
 	))
+	// FIXME(vdemeester) why twice ?
 	buildImageSuccessfully(c, name, build.WithBuildContext(c,
 		build.WithFile("Dockerfile", dockerfile),
 		build.WithFile(".dockerignore", "./Dockerfile\n"),
@@ -2420,6 +2424,7 @@ func (s *DockerSuite) TestBuildDockerignoringRenamedDockerfile(c *check.C) {
 		build.WithFile("MyDockerfile", dockerfile),
 		build.WithFile(".dockerignore", "MyDockerfile\n"),
 	))
+	// FIXME(vdemeester) why twice ?
 	buildImageSuccessfully(c, name, cli.WithFlags("-f", "MyDockerfile"), build.WithBuildContext(c,
 		build.WithFile("Dockerfile", "Should not use me"),
 		build.WithFile("MyDockerfile", dockerfile),
@@ -3045,6 +3050,7 @@ func (s *DockerSuite) TestBuildAddTarXzGz(c *check.C) {
 	buildImageSuccessfully(c, name, build.WithExternalBuildContext(ctx))
 }

+// FIXME(vdemeester) most of the from git tests could be moved to `docker/cli` e2e tests
 func (s *DockerSuite) TestBuildFromGit(c *check.C) {
 	name := "testbuildfromgit"
 	git := fakegit.New(c, "repo", map[string]string{
@@ -3422,6 +3428,7 @@ func (s *DockerSuite) TestBuildLabelsCache(c *check.C) {

 }

+// FIXME(vdemeester) port to docker/cli e2e tests (api tests should test suppressOutput option though)
 func (s *DockerSuite) TestBuildNotVerboseSuccess(c *check.C) {
 	// This test makes sure that -q works correctly when build is successful:
 	// stdout has only the image ID (long image ID) and stderr is empty.
@@ -3472,6 +3479,7 @@ func (s *DockerSuite) TestBuildNotVerboseSuccess(c *check.C) {

 }

+// FIXME(vdemeester) migrate to docker/cli tests
 func (s *DockerSuite) TestBuildNotVerboseFailureWithNonExistImage(c *check.C) {
 	// This test makes sure that -q works correctly when build fails by
 	// comparing between the stderr output in quiet mode and in stdout
@@ -3492,6 +3500,7 @@ func (s *DockerSuite) TestBuildNotVerboseFailureWithNonExistImage(c *check.C) {
 	}
 }

+// FIXME(vdemeester) migrate to docker/cli tests
 func (s *DockerSuite) TestBuildNotVerboseFailure(c *check.C) {
 	// This test makes sure that -q works correctly when build fails by
 	// comparing between the stderr output in quiet mode and in stdout
@@ -3519,6 +3528,7 @@ func (s *DockerSuite) TestBuildNotVerboseFailure(c *check.C) {
 	}
 }

+// FIXME(vdemeester) migrate to docker/cli tests
 func (s *DockerSuite) TestBuildNotVerboseFailureRemote(c *check.C) {
 	// This test ensures that when given a wrong URL, stderr in quiet mode and
 	// stderr in verbose mode are identical.
@@ -3548,6 +3558,7 @@ func (s *DockerSuite) TestBuildNotVerboseFailureRemote(c *check.C) {
 	}
 }

+// FIXME(vdemeester) migrate to docker/cli tests
 func (s *DockerSuite) TestBuildStderr(c *check.C) {
 	// This test just makes sure that no non-error output goes
 	// to stderr
@@ -3688,67 +3699,6 @@ CMD cat /foo/file`),

 }

-// FIXME(vdemeester) part of this should be unit test, other part should be clearer
-func (s *DockerSuite) TestBuildRenamedDockerfile(c *check.C) {
-	ctx := fakecontext.New(c, "", fakecontext.WithFiles(map[string]string{
-		"Dockerfile":       "FROM busybox\nRUN echo from Dockerfile",
-		"files/Dockerfile": "FROM busybox\nRUN echo from files/Dockerfile",
-		"files/dFile":      "FROM busybox\nRUN echo from files/dFile",
-		"dFile":            "FROM busybox\nRUN echo from dFile",
-		"files/dFile2":     "FROM busybox\nRUN echo from files/dFile2",
-	}))
-	defer ctx.Close()
-
-	cli.Docker(cli.Args("build", "-t", "test1", "."), cli.InDir(ctx.Dir)).Assert(c, icmd.Expected{
-		Out: "from Dockerfile",
-	})
-
-	cli.Docker(cli.Args("build", "-f", filepath.Join("files", "Dockerfile"), "-t", "test2", "."), cli.InDir(ctx.Dir)).Assert(c, icmd.Expected{
-		Out: "from files/Dockerfile",
-	})
-
-	cli.Docker(cli.Args("build", fmt.Sprintf("--file=%s", filepath.Join("files", "dFile")), "-t", "test3", "."), cli.InDir(ctx.Dir)).Assert(c, icmd.Expected{
-		Out: "from files/dFile",
-	})
-
-	cli.Docker(cli.Args("build", "--file=dFile", "-t", "test4", "."), cli.InDir(ctx.Dir)).Assert(c, icmd.Expected{
-		Out: "from dFile",
-	})
-
-	dirWithNoDockerfile, err := ioutil.TempDir(os.TempDir(), "test5")
-	c.Assert(err, check.IsNil)
-	nonDockerfileFile := filepath.Join(dirWithNoDockerfile, "notDockerfile")
-	if _, err = os.Create(nonDockerfileFile); err != nil {
-		c.Fatal(err)
-	}
-	cli.Docker(cli.Args("build", fmt.Sprintf("--file=%s", nonDockerfileFile), "-t", "test5", "."), cli.InDir(ctx.Dir)).Assert(c, icmd.Expected{
-		ExitCode: 1,
-		Err:      fmt.Sprintf("unable to prepare context: the Dockerfile (%s) must be within the build context", nonDockerfileFile),
-	})
-
-	cli.Docker(cli.Args("build", "-f", filepath.Join("..", "Dockerfile"), "-t", "test6", ".."), cli.InDir(filepath.Join(ctx.Dir, "files"))).Assert(c, icmd.Expected{
-		Out: "from Dockerfile",
-	})
-
-	cli.Docker(cli.Args("build", "-f", filepath.Join(ctx.Dir, "files", "Dockerfile"), "-t", "test7", ".."), cli.InDir(filepath.Join(ctx.Dir, "files"))).Assert(c, icmd.Expected{
-		Out: "from files/Dockerfile",
-	})
-
-	cli.Docker(cli.Args("build", "-f", filepath.Join("..", "Dockerfile"), "-t", "test8", "."), cli.InDir(filepath.Join(ctx.Dir, "files"))).Assert(c, icmd.Expected{
-		ExitCode: 1,
-		Err:      "must be within the build context",
-	})
-
-	tmpDir := os.TempDir()
-	cli.Docker(cli.Args("build", "-t", "test9", ctx.Dir), cli.InDir(tmpDir)).Assert(c, icmd.Expected{
-		Out: "from Dockerfile",
-	})
-
-	cli.Docker(cli.Args("build", "-f", "dFile2", "-t", "test10", "."), cli.InDir(filepath.Join(ctx.Dir, "files"))).Assert(c, icmd.Expected{
-		Out: "from files/dFile2",
-	})
-}
-
 func (s *DockerSuite) TestBuildFromMixedcaseDockerfile(c *check.C) {
 	testRequires(c, UnixCli) // Dockerfile overwrites dockerfile on windows
 	testRequires(c, DaemonIsLinux)
@@ -3772,6 +3722,7 @@ func (s *DockerSuite) TestBuildFromMixedcaseDockerfile(c *check.C) {
 	})
 }

+// FIXME(vdemeester) should migrate to docker/cli tests
 func (s *DockerSuite) TestBuildFromURLWithF(c *check.C) {
 	server := fakestorage.New(c, "", fakecontext.WithFiles(map[string]string{"baz": `FROM busybox
 RUN echo from baz
@@ -3798,6 +3749,7 @@ RUN find /tmp/`}))

 }

+// FIXME(vdemeester) should migrate to docker/cli tests
 func (s *DockerSuite) TestBuildFromStdinWithF(c *check.C) {
 	testRequires(c, DaemonIsLinux) // TODO Windows: This test is flaky; no idea why
 	ctx := fakecontext.New(c, "", fakecontext.WithDockerfile(`FROM busybox
@@ -3840,61 +3792,6 @@ func (s *DockerSuite) TestBuildFromOfficialNames(c *check.C) {
 	}
 }

-func (s *DockerSuite) TestBuildDockerfileOutsideContext(c *check.C) {
-	testRequires(c, UnixCli, DaemonIsLinux) // uses os.Symlink: not implemented in windows at the time of writing (go-1.4.2)
-
-	name := "testbuilddockerfileoutsidecontext"
-	tmpdir, err := ioutil.TempDir("", name)
-	c.Assert(err, check.IsNil)
-	defer os.RemoveAll(tmpdir)
-	ctx := filepath.Join(tmpdir, "context")
-	if err := os.MkdirAll(ctx, 0755); err != nil {
-		c.Fatal(err)
-	}
-	if err := ioutil.WriteFile(filepath.Join(ctx, "Dockerfile"), []byte("FROM scratch\nENV X Y"), 0644); err != nil {
-		c.Fatal(err)
-	}
-	wd, err := os.Getwd()
-	if err != nil {
-		c.Fatal(err)
-	}
-	defer os.Chdir(wd)
-	if err := os.Chdir(ctx); err != nil {
-		c.Fatal(err)
-	}
-	if err := ioutil.WriteFile(filepath.Join(tmpdir, "outsideDockerfile"), []byte("FROM scratch\nENV x y"), 0644); err != nil {
-		c.Fatal(err)
-	}
-	if err := os.Symlink(filepath.Join("..", "outsideDockerfile"), filepath.Join(ctx, "dockerfile1")); err != nil {
-		c.Fatal(err)
-	}
-	if err := os.Symlink(filepath.Join(tmpdir, "outsideDockerfile"), filepath.Join(ctx, "dockerfile2")); err != nil {
-		c.Fatal(err)
-	}
-
-	for _, dockerfilePath := range []string{
-		filepath.Join("..", "outsideDockerfile"),
-		filepath.Join(ctx, "dockerfile1"),
-		filepath.Join(ctx, "dockerfile2"),
-	} {
-		result := dockerCmdWithResult("build", "-t", name, "--no-cache", "-f", dockerfilePath, ".")
-		result.Assert(c, icmd.Expected{
-			Err:      "must be within the build context",
-			ExitCode: 1,
-		})
-		deleteImages(name)
-	}
-
-	os.Chdir(tmpdir)
-
-	// Path to Dockerfile should be resolved relative to working directory, not relative to context.
-	// There is a Dockerfile in the context, but since there is no Dockerfile in the current directory, the following should fail
-	out, _, err := dockerCmdWithError("build", "-t", name, "--no-cache", "-f", "Dockerfile", ctx)
-	if err == nil {
-		c.Fatalf("Expected error. Out: %s", out)
-	}
-}
-
 // FIXME(vdemeester) should be a unit test
 func (s *DockerSuite) TestBuildSpaces(c *check.C) {
 	// Test to make sure that leading/trailing spaces on a command
@@ -4186,6 +4083,7 @@ func (s *DockerTrustSuite) TestTrustedBuildUntrustedTag(c *check.C) {
 	})
 }

+// FIXME(vdemeester) should migrate to docker/cli e2e tests
 func (s *DockerTrustSuite) TestBuildContextDirIsSymlink(c *check.C) {
 	testRequires(c, DaemonIsLinux)
 	tempDir, err := ioutil.TempDir("", "test-build-dir-is-symlink-")
@@ -5132,6 +5030,7 @@ func (s *DockerSuite) TestBuildCacheRootSource(c *check.C) {
 }

 // #19375
+// FIXME(vdemeester) should migrate to docker/cli tests
 func (s *DockerSuite) TestBuildFailsGitNotCallable(c *check.C) {
 	buildImage("gitnotcallable", cli.WithEnvironmentVariables("PATH="),
 		build.WithContextPath("github.com/docker/v1.10-migrator.git")).Assert(c, icmd.Expected{
@@ -6449,6 +6348,7 @@ CMD echo foo
 	c.Assert(strings.TrimSpace(out), checker.Equals, `["/bin/sh","-c","echo foo"]`)
 }

+// FIXME(vdemeester) should migrate to docker/cli tests
 func (s *DockerSuite) TestBuildIidFile(c *check.C) {
 	tmpDir, err := ioutil.TempDir("", "TestBuildIidFile")
 	if err != nil {
@@ -6473,6 +6373,7 @@ ENV BAR BAZ`),
 	c.Assert(d.String(), checker.Equals, getIDByName(c, name))
 }

+// FIXME(vdemeester) should migrate to docker/cli tests
 func (s *DockerSuite) TestBuildIidFileCleanupOnFail(c *check.C) {
 	tmpDir, err := ioutil.TempDir("", "TestBuildIidFileCleanupOnFail")
 	if err != nil {
@@ -6495,6 +6396,7 @@ func (s *DockerSuite) TestBuildIidFileCleanupOnFail(c *check.C) {
 	c.Assert(os.IsNotExist(err), check.Equals, true)
 }

+// FIXME(vdemeester) should migrate to docker/cli tests
 func (s *DockerSuite) TestBuildIidFileSquash(c *check.C) {
 	testRequires(c, ExperimentalDaemon)
 	tmpDir, err := ioutil.TempDir("", "TestBuildIidFileSquash")
@@ -8,8 +8,6 @@ import (
 	"github.com/go-check/check"
 )

-// docker cp CONTAINER:PATH LOCALPATH
-
 // Try all of the test cases from the archive package which implements the
 // internals of `docker cp` and ensure that the behavior matches when actually
 // copying to and from containers.
@@ -20,67 +18,9 @@ import (
 // 3. DST parent directory must exist.
 // 4. If DST exists as a file, it must not end with a trailing separator.

-// First get these easy error cases out of the way.
-
-// Test for error when SRC does not exist.
-func (s *DockerSuite) TestCpFromErrSrcNotExists(c *check.C) {
-	containerID := makeTestContainer(c, testContainerOptions{})
-
-	tmpDir := getTestDir(c, "test-cp-from-err-src-not-exists")
-	defer os.RemoveAll(tmpDir)
-
-	err := runDockerCp(c, containerCpPath(containerID, "file1"), tmpDir, nil)
-	c.Assert(err, checker.NotNil)
-
-	c.Assert(isCpNotExist(err), checker.True, check.Commentf("expected IsNotExist error, but got %T: %s", err, err))
-}
-
-// Test for error when SRC ends in a trailing
-// path separator but it exists as a file.
-func (s *DockerSuite) TestCpFromErrSrcNotDir(c *check.C) {
-	testRequires(c, DaemonIsLinux)
-	containerID := makeTestContainer(c, testContainerOptions{addContent: true})
-
-	tmpDir := getTestDir(c, "test-cp-from-err-src-not-dir")
-	defer os.RemoveAll(tmpDir)
-
-	err := runDockerCp(c, containerCpPathTrailingSep(containerID, "file1"), tmpDir, nil)
-	c.Assert(err, checker.NotNil)
-
-	c.Assert(isCpNotDir(err), checker.True, check.Commentf("expected IsNotDir error, but got %T: %s", err, err))
-}
-
-// Test for error when DST ends in a trailing
-// path separator but exists as a file.
-func (s *DockerSuite) TestCpFromErrDstNotDir(c *check.C) {
-	testRequires(c, DaemonIsLinux)
-	containerID := makeTestContainer(c, testContainerOptions{addContent: true})
-
-	tmpDir := getTestDir(c, "test-cp-from-err-dst-not-dir")
-	defer os.RemoveAll(tmpDir)
-
-	makeTestContentInDir(c, tmpDir)
-
-	// Try with a file source.
-	srcPath := containerCpPath(containerID, "/file1")
-	dstPath := cpPathTrailingSep(tmpDir, "file1")
-
-	err := runDockerCp(c, srcPath, dstPath, nil)
-	c.Assert(err, checker.NotNil)
-
-	c.Assert(isCpNotDir(err), checker.True, check.Commentf("expected IsNotDir error, but got %T: %s", err, err))
-
-	// Try with a directory source.
-	srcPath = containerCpPath(containerID, "/dir1")
-
-	err = runDockerCp(c, srcPath, dstPath, nil)
-	c.Assert(err, checker.NotNil)
-
-	c.Assert(isCpNotDir(err), checker.True, check.Commentf("expected IsNotDir error, but got %T: %s", err, err))
-}
-
 // Check that copying from a container to a local symlink copies to the symlink
 // target and does not overwrite the local symlink itself.
+// TODO: move to docker/cli and/or integration/container/copy_test.go
 func (s *DockerSuite) TestCpFromSymlinkDestination(c *check.C) {
 	testRequires(c, DaemonIsLinux)
 	containerID := makeTestContainer(c, testContainerOptions{addContent: true})
@@ -2,15 +2,11 @@ package main

 import (
 	"os"
-	"runtime"
-	"strings"

 	"github.com/docker/docker/integration-cli/checker"
 	"github.com/go-check/check"
 )

-// docker cp LOCALPATH CONTAINER:PATH
-
 // Try all of the test cases from the archive package which implements the
 // internals of `docker cp` and ensure that the behavior matches when actually
 // copying to and from containers.
@@ -21,124 +17,6 @@ import (
 // 3. DST parent directory must exist.
 // 4. If DST exists as a file, it must not end with a trailing separator.

-// First get these easy error cases out of the way.
-
-// Test for error when SRC does not exist.
-func (s *DockerSuite) TestCpToErrSrcNotExists(c *check.C) {
-	containerID := makeTestContainer(c, testContainerOptions{})
-
-	tmpDir := getTestDir(c, "test-cp-to-err-src-not-exists")
-	defer os.RemoveAll(tmpDir)
-
-	srcPath := cpPath(tmpDir, "file1")
-	dstPath := containerCpPath(containerID, "file1")
-	_, srcStatErr := os.Stat(srcPath)
-	c.Assert(os.IsNotExist(srcStatErr), checker.True)
-
-	err := runDockerCp(c, srcPath, dstPath, nil)
-	if runtime.GOOS == "windows" {
-		// Go 1.9+ on Windows returns a different error for `os.Stat()`, see
-		// https://github.com/golang/go/commit/6144c7270e5812d9de8fb97456ee4e5ae657fcbb#diff-f63e1a4b4377b2fe0b05011db3df9599
-		//
-		// Go 1.8: CreateFile C:\not-exist: The system cannot find the file specified.
-		// Go 1.9: GetFileAttributesEx C:\not-exist: The system cannot find the file specified.
-		//
-		// Due to the CLI using a different version than the daemon, comparing the
-		// error message won't work, so just hard-code the common part here.
-		//
-		// TODO this should probably be a test in the CLI repository instead
-		c.Assert(strings.ToLower(err.Error()), checker.Contains, "cannot find the file specified")
-		c.Assert(strings.ToLower(err.Error()), checker.Contains, strings.ToLower(tmpDir))
-	} else {
-		c.Assert(strings.ToLower(err.Error()), checker.Contains, strings.ToLower(srcStatErr.Error()))
-	}
-}
-
-// Test for error when SRC ends in a trailing
-// path separator but it exists as a file.
-func (s *DockerSuite) TestCpToErrSrcNotDir(c *check.C) {
-	containerID := makeTestContainer(c, testContainerOptions{})
-
-	tmpDir := getTestDir(c, "test-cp-to-err-src-not-dir")
-	defer os.RemoveAll(tmpDir)
-
-	makeTestContentInDir(c, tmpDir)
-
-	srcPath := cpPathTrailingSep(tmpDir, "file1")
-	dstPath := containerCpPath(containerID, "testDir")
-
-	err := runDockerCp(c, srcPath, dstPath, nil)
-	c.Assert(err, checker.NotNil)
-
-	c.Assert(isCpNotDir(err), checker.True, check.Commentf("expected IsNotDir error, but got %T: %s", err, err))
-}
-
-// Test for error when SRC is a valid file or directory,
-// but the DST parent directory does not exist.
-func (s *DockerSuite) TestCpToErrDstParentNotExists(c *check.C) {
-	testRequires(c, DaemonIsLinux)
-	containerID := makeTestContainer(c, testContainerOptions{addContent: true})
-
-	tmpDir := getTestDir(c, "test-cp-to-err-dst-parent-not-exists")
-	defer os.RemoveAll(tmpDir)
-
-	makeTestContentInDir(c, tmpDir)
-
-	// Try with a file source.
-	srcPath := cpPath(tmpDir, "file1")
-	dstPath := containerCpPath(containerID, "/notExists", "file1")
-
-	err := runDockerCp(c, srcPath, dstPath, nil)
-	c.Assert(err, checker.NotNil)
-
-	c.Assert(isCpNotExist(err), checker.True, check.Commentf("expected IsNotExist error, but got %T: %s", err, err))
-
-	// Try with a directory source.
-	srcPath = cpPath(tmpDir, "dir1")
-
-	err = runDockerCp(c, srcPath, dstPath, nil)
-	c.Assert(err, checker.NotNil)
-
-	c.Assert(isCpNotExist(err), checker.True, check.Commentf("expected IsNotExist error, but got %T: %s", err, err))
-}
-
-// Test for error when DST ends in a trailing path separator but exists as a
-// file. Also test that we cannot overwrite an existing directory with a
-// non-directory and cannot overwrite an existing
-func (s *DockerSuite) TestCpToErrDstNotDir(c *check.C) {
-	testRequires(c, DaemonIsLinux)
-	containerID := makeTestContainer(c, testContainerOptions{addContent: true})
-
-	tmpDir := getTestDir(c, "test-cp-to-err-dst-not-dir")
-	defer os.RemoveAll(tmpDir)
-
-	makeTestContentInDir(c, tmpDir)
-
-	// Try with a file source.
-	srcPath := cpPath(tmpDir, "dir1/file1-1")
-	dstPath := containerCpPathTrailingSep(containerID, "file1")
-
-	// The client should encounter an error trying to stat the destination
-	// and then be unable to copy since the destination is asserted to be a
-	// directory but does not exist.
-	err := runDockerCp(c, srcPath, dstPath, nil)
-	c.Assert(err, checker.NotNil)
-
-	c.Assert(isCpDirNotExist(err), checker.True, check.Commentf("expected DirNotExist error, but got %T: %s", err, err))
-
-	// Try with a directory source.
-	srcPath = cpPath(tmpDir, "dir1")
-
-	// The client should encounter an error trying to stat the destination and
-	// then decide to extract to the parent directory instead with a rebased
-	// name in the source archive, but this directory would overwrite the
-	// existing file with the same name.
-	err = runDockerCp(c, srcPath, dstPath, nil)
-	c.Assert(err, checker.NotNil)
-
-	c.Assert(isCannotOverwriteNonDirWithDir(err), checker.True, check.Commentf("expected CannotOverwriteNonDirWithDir error, but got %T: %s", err, err))
-}
-
 // Check that copying from a local path to a symlink in a container copies to
 // the symlink target and does not overwrite the container symlink itself.
 func (s *DockerSuite) TestCpToSymlinkDestination(c *check.C) {
@@ -228,18 +228,10 @@ func getTestDir(c *check.C, label string) (tmpDir string) {
 	return
 }

-func isCpNotExist(err error) bool {
-	return strings.Contains(strings.ToLower(err.Error()), "could not find the file")
-}
-
 func isCpDirNotExist(err error) bool {
 	return strings.Contains(err.Error(), archive.ErrDirNotExists.Error())
 }

-func isCpNotDir(err error) bool {
-	return strings.Contains(err.Error(), archive.ErrNotDirectory.Error()) || strings.Contains(err.Error(), "filename, directory name, or volume label syntax is incorrect")
-}
-
 func isCpCannotCopyDir(err error) bool {
 	return strings.Contains(err.Error(), archive.ErrCannotCopyDir.Error())
 }
@@ -248,10 +240,6 @@ func isCpCannotCopyReadOnly(err error) bool {
 	return strings.Contains(err.Error(), "marked read-only")
 }

-func isCannotOverwriteNonDirWithDir(err error) bool {
-	return strings.Contains(err.Error(), "cannot overwrite non-directory")
-}
-
 func fileContentEquals(c *check.C, filename, contents string) (err error) {
 	c.Logf("checking that file %q contains %q\n", filename, contents)

@@ -301,6 +301,46 @@ COPY bar /`
 	require.NotContains(t, out.String(), "Using cache")
 }

+// docker/for-linux#135
+// #35641
+func TestBuildMultiStageLayerLeak(t *testing.T) {
+	ctx := context.TODO()
+	defer setupTest(t)()
+
+	// all commands need to match until COPY
+	dockerfile := `FROM busybox
+WORKDIR /foo
+COPY foo .
+FROM busybox
+WORKDIR /foo
+COPY bar .
+RUN [ -f bar ]
+RUN [ ! -f foo ]
+`
+
+	source := fakecontext.New(t, "",
+		fakecontext.WithFile("foo", "0"),
+		fakecontext.WithFile("bar", "1"),
+		fakecontext.WithDockerfile(dockerfile))
+	defer source.Close()
+
+	apiclient := testEnv.APIClient()
+	resp, err := apiclient.ImageBuild(ctx,
+		source.AsTarReader(t),
+		types.ImageBuildOptions{
+			Remove:      true,
+			ForceRemove: true,
+		})
+
+	out := bytes.NewBuffer(nil)
+	require.NoError(t, err)
+	_, err = io.Copy(out, resp.Body)
+	resp.Body.Close()
+	require.NoError(t, err)
+
+	assert.Contains(t, out.String(), "Successfully built")
+}
+
 func writeTarRecord(t *testing.T, w *tar.Writer, fn, contents string) {
 	err := w.WriteHeader(&tar.Header{
 		Name:     fn,
@@ -0,0 +1,65 @@
+package container // import "github.com/docker/docker/integration/container"
+
+import (
+	"context"
+	"fmt"
+	"testing"
+
+	"github.com/docker/docker/api/types"
+	"github.com/docker/docker/client"
+	"github.com/docker/docker/integration/internal/container"
+	"github.com/docker/docker/internal/testutil"
+	"github.com/gotestyourself/gotestyourself/skip"
+	"github.com/stretchr/testify/require"
+)
+
+func TestCopyFromContainerPathDoesNotExist(t *testing.T) {
+	defer setupTest(t)()
+
+	ctx := context.Background()
+	apiclient := testEnv.APIClient()
+	cid := container.Create(t, ctx, apiclient)
+
+	_, _, err := apiclient.CopyFromContainer(ctx, cid, "/dne")
+	require.True(t, client.IsErrNotFound(err))
+	expected := fmt.Sprintf("No such container:path: %s:%s", cid, "/dne")
+	testutil.ErrorContains(t, err, expected)
+}
+
+func TestCopyFromContainerPathIsNotDir(t *testing.T) {
+	defer setupTest(t)()
+	skip.If(t, testEnv.OSType == "windows")
+
+	ctx := context.Background()
+	apiclient := testEnv.APIClient()
+	cid := container.Create(t, ctx, apiclient)
+
+	_, _, err := apiclient.CopyFromContainer(ctx, cid, "/etc/passwd/")
+	require.Contains(t, err.Error(), "not a directory")
+}
+
+func TestCopyToContainerPathDoesNotExist(t *testing.T) {
+	defer setupTest(t)()
+	skip.If(t, testEnv.OSType == "windows")
+
+	ctx := context.Background()
+	apiclient := testEnv.APIClient()
+	cid := container.Create(t, ctx, apiclient)
+
+	err := apiclient.CopyToContainer(ctx, cid, "/dne", nil, types.CopyToContainerOptions{})
+	require.True(t, client.IsErrNotFound(err))
+	expected := fmt.Sprintf("No such container:path: %s:%s", cid, "/dne")
+	testutil.ErrorContains(t, err, expected)
+}
+
+func TestCopyToContainerPathIsNotDir(t *testing.T) {
+	defer setupTest(t)()
+	skip.If(t, testEnv.OSType == "windows")
+
+	ctx := context.Background()
+	apiclient := testEnv.APIClient()
+	cid := container.Create(t, ctx, apiclient)
+
+	err := apiclient.CopyToContainer(ctx, cid, "/etc/passwd/", nil, types.CopyToContainerOptions{})
+	require.Contains(t, err.Error(), "not a directory")
+}
@@ -0,0 +1,70 @@
+package network // import "github.com/docker/docker/integration/network"
+
+import (
+	"runtime"
+	"testing"
+	"time"
+
+	"github.com/docker/docker/api/types"
+	"github.com/docker/docker/api/types/filters"
+	"github.com/docker/docker/api/types/swarm"
+	"github.com/docker/docker/client"
+	"github.com/gotestyourself/gotestyourself/poll"
+	"github.com/stretchr/testify/require"
+	"golang.org/x/net/context"
+)
+
+func TestServiceWithPredefinedNetwork(t *testing.T) {
+	defer setupTest(t)()
+	d := newSwarm(t)
+	defer d.Stop(t)
+	client, err := client.NewClientWithOpts(client.WithHost((d.Sock())))
+	require.NoError(t, err)
+
+	hostName := "host"
+	var instances uint64 = 1
+	serviceName := "TestService"
+	serviceSpec := swarmServiceSpec(serviceName, instances)
+	serviceSpec.TaskTemplate.Networks = append(serviceSpec.TaskTemplate.Networks, swarm.NetworkAttachmentConfig{Target: hostName})
+
+	serviceResp, err := client.ServiceCreate(context.Background(), serviceSpec, types.ServiceCreateOptions{
+		QueryRegistry: false,
+	})
+	require.NoError(t, err)
+
+	pollSettings := func(config *poll.Settings) {
+		if runtime.GOARCH == "arm64" || runtime.GOARCH == "arm" {
+			config.Timeout = 50 * time.Second
+			config.Delay = 100 * time.Millisecond
+		}
+	}
+
+	serviceID := serviceResp.ID
+	poll.WaitOn(t, serviceRunningCount(client, serviceID, instances), pollSettings)
+
+	_, _, err = client.ServiceInspectWithRaw(context.Background(), serviceID, types.ServiceInspectOptions{})
+	require.NoError(t, err)
+
+	err = client.ServiceRemove(context.Background(), serviceID)
+	require.NoError(t, err)
+
+	poll.WaitOn(t, serviceIsRemoved(client, serviceID), pollSettings)
+	poll.WaitOn(t, noTasks(client), pollSettings)
+
+}
+
+func serviceRunningCount(client client.ServiceAPIClient, serviceID string, instances uint64) func(log poll.LogT) poll.Result {
+	return func(log poll.LogT) poll.Result {
+		filter := filters.NewArgs()
+		filter.Add("service", serviceID)
+		services, err := client.ServiceList(context.Background(), types.ServiceListOptions{})
+		if err != nil {
+			return poll.Error(err)
+		}
+
+		if len(services) != int(instances) {
+			return poll.Continue("Service count at %d waiting for %d", len(services), instances)
+		}
+		return poll.Success()
+	}
+}
@@ -32,7 +32,7 @@ github.com/tonistiigi/fsutil dea3a0da73aee887fc02142d995be764106ac5e2

 #get libnetwork packages

-# When updating, also update LIBNETWORK_COMMIT in hack/dockerfile/binaries-commits accordingly
+# When updating, also update LIBNETWORK_COMMIT in hack/dockerfile/install/proxy accordingly
 github.com/docker/libnetwork ed2130d117c11c542327b4d5216a5db36770bc65
 github.com/docker/go-events 9461782956ad83b30282bf90e31fa6a70c255ba9
 github.com/armon/go-radix e39d623f12e8e41c7b5529e9a9dd67a1e2261f80
@@ -47,7 +47,7 @@ github.com/docker/libkv 1d8431073ae03cdaedb198a89722f3aab6d418ef
 github.com/vishvananda/netns 604eaf189ee867d8c147fafc28def2394e878d25
 github.com/vishvananda/netlink b2de5d10e38ecce8607e6b438b6d174f389a004e

-# When updating, consider updating TOMLV_COMMIT in hack/dockerfile/binaries-commits accordingly
+# When updating, consider updating TOMLV_COMMIT in hack/dockerfile/install/tomlv accordingly
 github.com/BurntSushi/toml a368813c5e648fee92e5f6c30e3944ff9d5e8895
 github.com/samuel/go-zookeeper d0e0d8e11f318e000a8cc434616d69e329edc374
 github.com/deckarep/golang-set ef32fa3046d9f249d399f98ebaf9be944430fd1d
@@ -70,8 +70,8 @@ github.com/pborman/uuid v1.0

 google.golang.org/grpc v1.3.0

-# When updating, also update RUNC_COMMIT in hack/dockerfile/binaries-commits accordingly
-github.com/opencontainers/runc 6c55f98695e902427906eed2c799e566e3d3dfb5
+# When updating, also update RUNC_COMMIT in hack/dockerfile/install/runc accordingly
+github.com/opencontainers/runc 4fc53a81fb7c994640722ac585fa9ca548971871
 github.com/opencontainers/runtime-spec v1.0.1
 github.com/opencontainers/image-spec v1.0.1
 github.com/seccomp/libseccomp-golang 32f571b70023028bd57d9288c20efbcb237f3ce0
@@ -113,14 +113,14 @@ github.com/containerd/containerd 3fa104f843ec92328912e042b767d26825f202aa
 github.com/containerd/fifo fbfb6a11ec671efbe94ad1c12c2e98773f19e1e6
 github.com/containerd/continuity d8fb8589b0e8e85b8c8bbaa8840226d0dfeb7371
 github.com/containerd/cgroups c0710c92e8b3a44681d1321dcfd1360fc5c6c089
-github.com/containerd/console 84eeaae905fa414d03e07bcd6c8d3f19e7cf180e
+github.com/containerd/console 2748ece16665b45a47f884001d5831ec79703880
 github.com/containerd/go-runc 4f6e87ae043f859a38255247b49c9abc262d002f
 github.com/containerd/typeurl f6943554a7e7e88b3c14aad190bf05932da84788
 github.com/dmcgowan/go-tar go1.10
 github.com/stevvooe/ttrpc d4528379866b0ce7e9d71f3eb96f0582fc374577

 # cluster
-github.com/docker/swarmkit f74983e7c015a38a81c8642803a78b8322cf7eac
+github.com/docker/swarmkit 11d7b06f48bc1d73fc6d8776c3552a4b11c94301
 github.com/gogo/protobuf v0.4
 github.com/cloudflare/cfssl 7fb22c8cba7ecaf98e4082d22d65800cf45e042a
 github.com/google/certificate-transparency d90e65c3a07988180c5b1ece71791c0b6506826e
@@ -13,25 +13,21 @@ const (
 	cmdTcSet = unix.TCSETS
 )

-func ioctl(fd, flag, data uintptr) error {
-	if _, _, err := unix.Syscall(unix.SYS_IOCTL, fd, flag, data); err != 0 {
+// unlockpt unlocks the slave pseudoterminal device corresponding to the master pseudoterminal referred to by f.
+// unlockpt should be called before opening the slave side of a pty.
+func unlockpt(f *os.File) error {
+	var u int32
+	if _, _, err := unix.Syscall(unix.SYS_IOCTL, f.Fd(), unix.TIOCSPTLCK, uintptr(unsafe.Pointer(&u))); err != 0 {
 		return err
 	}
 	return nil
 }

-// unlockpt unlocks the slave pseudoterminal device corresponding to the master pseudoterminal referred to by f.
-// unlockpt should be called before opening the slave side of a pty.
-func unlockpt(f *os.File) error {
-	var u int32
-	return ioctl(f.Fd(), unix.TIOCSPTLCK, uintptr(unsafe.Pointer(&u)))
-}
-
 // ptsname retrieves the name of the first available pts for the given master.
 func ptsname(f *os.File) (string, error) {
-	n, err := unix.IoctlGetInt(int(f.Fd()), unix.TIOCGPTN)
-	if err != nil {
+	var u uint32
+	if _, _, err := unix.Syscall(unix.SYS_IOCTL, f.Fd(), unix.TIOCGPTN, uintptr(unsafe.Pointer(&u))); err != 0 {
 		return "", err
 	}
-	return fmt.Sprintf("/dev/pts/%d", n), nil
+	return fmt.Sprintf("/dev/pts/%d", u), nil
 }
@@ -644,7 +644,7 @@ func (s *Server) CreateService(ctx context.Context, request *api.CreateServiceRe
 		return nil, err
 	}

-	if err := s.validateNetworks(request.Spec.Networks); err != nil {
+	if err := s.validateNetworks(request.Spec.Task.Networks); err != nil {
 		return nil, err
 	}

@@ -727,6 +727,10 @@ func (s *Server) UpdateService(ctx context.Context, request *api.UpdateServiceRe
 		return nil, err
 	}

+	if err := s.validateNetworks(request.Spec.Task.Networks); err != nil {
+		return nil, err
+	}
+
 	var service *api.Service
 	s.store.View(func(tx store.ReadTx) {
 		service = store.GetService(tx, request.ServiceID)
@@ -125,12 +125,17 @@ type clusterUpdate struct {

 // Dispatcher is responsible for dispatching tasks and tracking agent health.
 type Dispatcher struct {
-	// mu is a lock to provide  mutually exclusive access to dispatcher fields
-	// e.g. lastSeenManagers, networkBootstrapKeys, lastSeenRootCert etc.
+	// Mutex to synchronize access to dispatcher shared state e.g. nodes,
+	// lastSeenManagers, networkBootstrapKeys etc.
+	// TODO(anshul): This can potentially be removed and rpcRW used in its place.
 	mu sync.Mutex
-	// shutdownWait is used by stop() to wait for existing operations to finish.
-	shutdownWait sync.WaitGroup
-
+	// WaitGroup to handle the case when Stop() gets called before Run()
+	// has finished initializing the dispatcher.
+	wg sync.WaitGroup
+	// This RWMutex synchronizes RPC handlers and the dispatcher stop().
+	// The RPC handlers use the read lock while stop() uses the write lock
+	// and acts as a barrier to shutdown.
+	rpcRW                sync.RWMutex
 	nodes                *nodeStore
 	store                *store.MemoryStore
 	lastSeenManagers     []*api.WeightedPeer
@@ -253,11 +258,8 @@ func (d *Dispatcher) Run(ctx context.Context) error {
 	defer cancel()
 	d.ctx, d.cancel = context.WithCancel(ctx)
 	ctx = d.ctx
-
-	// If Stop() is called, it should wait
-	// for Run() to complete.
-	d.shutdownWait.Add(1)
-	defer d.shutdownWait.Done()
+	d.wg.Add(1)
+	defer d.wg.Done()
 	d.mu.Unlock()

 	publishManagers := func(peers []*api.Peer) {
@@ -320,15 +322,18 @@ func (d *Dispatcher) Stop() error {
 		return errors.New("dispatcher is already stopped")
 	}

-	// Cancel dispatcher context.
-	// This should also close the the streams in Tasks(), Assignments().
+	log := log.G(d.ctx).WithField("method", "(*Dispatcher).Stop")
+	log.Info("dispatcher stopping")
 	d.cancel()
 	d.mu.Unlock()

-	// Wait for the RPCs that are in-progress to finish.
-	d.shutdownWait.Wait()
-
+	// The active nodes list can be cleaned out only when all
+	// existing RPCs have finished.
+	// RPCs that start after rpcRW.Unlock() should find the context
+	// cancelled and should fail organically.
+	d.rpcRW.Lock()
 	d.nodes.Clean()
+	d.rpcRW.Unlock()

 	d.processUpdatesLock.Lock()
 	// In case there are any waiters. There is no chance of any starting
@@ -338,6 +343,14 @@ func (d *Dispatcher) Stop() error {
 	d.processUpdatesLock.Unlock()

 	d.clusterUpdateQueue.Close()
+
+	// TODO(anshul): This use of Wait() could be unsafe.
+	// According to go's documentation on WaitGroup,
+	// Add() with a positive delta that occur when the counter is zero
+	// must happen before a Wait().
+	// As is, dispatcher Stop() can race with Run().
+	d.wg.Wait()
+
 	return nil
 }

@@ -485,13 +498,13 @@ func nodeIPFromContext(ctx context.Context) (string, error) {

 // register is used for registration of node with particular dispatcher.
 func (d *Dispatcher) register(ctx context.Context, nodeID string, description *api.NodeDescription) (string, error) {
+	logLocal := log.G(ctx).WithField("method", "(*Dispatcher).register")
+	// prevent register until we're ready to accept it
 	dctx, err := d.isRunningLocked()
 	if err != nil {
 		return "", err
 	}

-	logLocal := log.G(ctx).WithField("method", "(*Dispatcher).register")
-
 	if err := d.nodes.CheckRateLimit(nodeID); err != nil {
 		return "", err
 	}
@@ -539,15 +552,8 @@ func (d *Dispatcher) register(ctx context.Context, nodeID string, description *a
 // UpdateTaskStatus updates status of task. Node should send such updates
 // on every status change of its tasks.
 func (d *Dispatcher) UpdateTaskStatus(ctx context.Context, r *api.UpdateTaskStatusRequest) (*api.UpdateTaskStatusResponse, error) {
-	// shutdownWait.Add() followed by isRunning() to ensures that
-	// if this rpc sees the dispatcher running,
-	// it will already have called Add() on the shutdownWait wait,
-	// which ensures that Stop() will wait for this rpc to complete.
-	// Note that Stop() first does Dispatcher.ctx.cancel() followed by
-	// shutdownWait.Wait() to make sure new rpc's don't start before waiting
-	// for existing ones to finish.
-	d.shutdownWait.Add(1)
-	defer d.shutdownWait.Done()
+	d.rpcRW.RLock()
+	defer d.rpcRW.RUnlock()

 	dctx, err := d.isRunningLocked()
 	if err != nil {
@@ -740,15 +746,8 @@ func (d *Dispatcher) processUpdates(ctx context.Context) {
 // of tasks which should be run on node, if task is not present in that list,
 // it should be terminated.
 func (d *Dispatcher) Tasks(r *api.TasksRequest, stream api.Dispatcher_TasksServer) error {
-	// shutdownWait.Add() followed by isRunning() to ensures that
-	// if this rpc sees the dispatcher running,
-	// it will already have called Add() on the shutdownWait wait,
-	// which ensures that Stop() will wait for this rpc to complete.
-	// Note that Stop() first does Dispatcher.ctx.cancel() followed by
-	// shutdownWait.Wait() to make sure new rpc's don't start before waiting
-	// for existing ones to finish.
-	d.shutdownWait.Add(1)
-	defer d.shutdownWait.Done()
+	d.rpcRW.RLock()
+	defer d.rpcRW.RUnlock()

 	dctx, err := d.isRunningLocked()
 	if err != nil {
@@ -873,15 +872,8 @@ func (d *Dispatcher) Tasks(r *api.TasksRequest, stream api.Dispatcher_TasksServe
 // Assignments is a stream of assignments for a node. Each message contains
 // either full list of tasks and secrets for the node, or an incremental update.
 func (d *Dispatcher) Assignments(r *api.AssignmentsRequest, stream api.Dispatcher_AssignmentsServer) error {
-	// shutdownWait.Add() followed by isRunning() to ensures that
-	// if this rpc sees the dispatcher running,
-	// it will already have called Add() on the shutdownWait wait,
-	// which ensures that Stop() will wait for this rpc to complete.
-	// Note that Stop() first does Dispatcher.ctx.cancel() followed by
-	// shutdownWait.Wait() to make sure new rpc's don't start before waiting
-	// for existing ones to finish.
-	d.shutdownWait.Add(1)
-	defer d.shutdownWait.Done()
+	d.rpcRW.RLock()
+	defer d.rpcRW.RUnlock()

 	dctx, err := d.isRunningLocked()
 	if err != nil {
@@ -1140,20 +1132,13 @@ func (d *Dispatcher) markNodeNotReady(id string, state api.NodeStatus_State, mes
 // Node should send new heartbeat earlier than now + TTL, otherwise it will
 // be deregistered from dispatcher and its status will be updated to NodeStatus_DOWN
 func (d *Dispatcher) Heartbeat(ctx context.Context, r *api.HeartbeatRequest) (*api.HeartbeatResponse, error) {
-	// shutdownWait.Add() followed by isRunning() to ensures that
-	// if this rpc sees the dispatcher running,
-	// it will already have called Add() on the shutdownWait wait,
-	// which ensures that Stop() will wait for this rpc to complete.
-	// Note that Stop() first does Dispatcher.ctx.cancel() followed by
-	// shutdownWait.Wait() to make sure new rpc's don't start before waiting
-	// for existing ones to finish.
-	d.shutdownWait.Add(1)
-	defer d.shutdownWait.Done()
+	d.rpcRW.RLock()
+	defer d.rpcRW.RUnlock()

-	// isRunningLocked() is not needed since its OK if
-	// the dispatcher context is cancelled while this call is in progress
-	// since Stop() which cancels the dispatcher context will wait for
-	// Heartbeat() to complete.
+	// Its OK to call isRunning() here instead of isRunningLocked()
+	// because of the rpcRW readlock above.
+	// TODO(anshul) other uses of isRunningLocked() can probably
+	// also be removed.
 	if !d.isRunning() {
 		return nil, status.Errorf(codes.Aborted, "dispatcher is stopped")
 	}
@@ -1192,15 +1177,8 @@ func (d *Dispatcher) getRootCACert() []byte {
 // a special boolean field Disconnect which if true indicates that node should
 // reconnect to another Manager immediately.
 func (d *Dispatcher) Session(r *api.SessionRequest, stream api.Dispatcher_SessionServer) error {
-	// shutdownWait.Add() followed by isRunning() to ensures that
-	// if this rpc sees the dispatcher running,
-	// it will already have called Add() on the shutdownWait wait,
-	// which ensures that Stop() will wait for this rpc to complete.
-	// Note that Stop() first does Dispatcher.ctx.cancel() followed by
-	// shutdownWait.Wait() to make sure new rpc's don't start before waiting
-	// for existing ones to finish.
-	d.shutdownWait.Add(1)
-	defer d.shutdownWait.Done()
+	d.rpcRW.RLock()
+	defer d.rpcRW.RUnlock()

 	dctx, err := d.isRunningLocked()
 	if err != nil {
@@ -1208,6 +1186,7 @@ func (d *Dispatcher) Session(r *api.SessionRequest, stream api.Dispatcher_Sessio
 	}

 	ctx := stream.Context()
+
 	nodeInfo, err := ca.RemoteNode(ctx)
 	if err != nil {
 		return err
@@ -41,8 +41,18 @@ make
 sudo make install
 ```

+You can also use `go get` to install to your `GOPATH`, assuming that you have a `github.com` parent folder already created under `src`:
+
+```bash
+go get github.com/opencontainers/runc
+cd $GOPATH/src/github.com/opencontainers/runc
+make
+sudo make install
+```
+
 `runc` will be installed to `/usr/local/sbin/runc` on your system.

+
 #### Build Tags

 `runc` supports optional build tags for compiling support of various features.
@@ -21,5 +21,5 @@ github.com/urfave/cli d53eb991652b1d438abdd34ce4bfa3ef1539108e
 golang.org/x/sys 7ddbeae9ae08c6a06a59597f0c9edbc5ff2444ce https://github.com/golang/sys

 # console dependencies
-github.com/containerd/console 84eeaae905fa414d03e07bcd6c8d3f19e7cf180e
+github.com/containerd/console 2748ece16665b45a47f884001d5831ec79703880
 github.com/pkg/errors v0.8.0
@@ -7,15 +7,15 @@ if [[ -z "$DEB_VERSION" ]]; then
    exit 1
 fi

-# I want to rip this install-binaries script out so badly
-cd engine
-TMP_GOPATH="/go" bash hack/dockerfile/install-binaries.sh runc-dynamic containerd-dynamic proxy-dynamic tini
-if [[ $? -ne 0 ]]; then
-    echo "Binaries required for package building not installed correctly."
-    echo "Exiting..."
-    exit 1
-fi
-cd -
+(
+    set -e
+    cd engine
+    # I want to rip this install-binaries script out so badly
+    for component in tini proxy runc containerd;do
+        TMP_GOPATH="/go" hack/dockerfile/install/install.sh $component
+    done
+)
+
 echo VERSION AAA $VERSION

 VERSION=${VERSION:-$( cat engine/VERSION )}
@@ -67,7 +67,9 @@ pushd /go/src/github.com/docker/cli
 make VERSION=%{_origversion} GITCOMMIT=%{_gitcommit} dynbinary manpages # cli
 popd
 pushd engine
-TMP_GOPATH="/go" hack/dockerfile/install-binaries.sh runc-dynamic containerd-dynamic proxy-dynamic tini
+for component in tini proxy runc containerd;do
+    TMP_GOPATH="/go" hack/dockerfile/install/install.sh $component
+done
 VERSION=%{_origversion} hack/make.sh dynbinary
 popd

@@ -65,7 +65,9 @@ pushd /go/src/github.com/docker/cli
 make VERSION=%{_origversion} GITCOMMIT=%{_gitcommit} dynbinary manpages # cli
 popd
 pushd engine
-TMP_GOPATH="/go" hack/dockerfile/install-binaries.sh runc-dynamic containerd-dynamic proxy-dynamic tini
+for component in tini proxy runc containerd;do
+    TMP_GOPATH="/go" hack/dockerfile/install/install.sh $component
+done
 VERSION=%{_origversion} hack/make.sh dynbinary
 popd
 mkdir -p plugin
@@ -66,7 +66,9 @@ pushd /go/src/github.com/docker/cli
 make VERSION=%{_origversion} GITCOMMIT=%{_gitcommit} dynbinary manpages # cli
 popd
 pushd engine
-TMP_GOPATH="/go" hack/dockerfile/install-binaries.sh runc-dynamic containerd-dynamic proxy-dynamic tini
+for component in tini proxy runc containerd;do
+    TMP_GOPATH="/go" hack/dockerfile/install/install.sh $component
+done
 VERSION=%{_origversion} hack/make.sh dynbinary
 popd
 mkdir -p plugin
Author	SHA1	Message	Date
Andrew HsuandGitHub	cbc5bef54f	Merge pull request #454 from seemethere/cherry_pick_packaging_88 [18.03] Fixes binary installation	2018-03-06 14:11:59 -08:00
Eli Uriegas	88176d01f4	Fixes binary installation Binary installation was broken after the hack/dockerfile/install-binaries script was removed. This remedies that. Signed-off-by: Eli Uriegas <eli.uriegas@docker.com> (cherry picked from commit 59164bedeab571029805a107e8e5a32fc9cd56b3) Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>	2018-03-06 21:50:00 +00:00
Andrew HsuandGitHub	3e53917a28	Merge pull request #443 from jose-bigio/18.03_versionBump [18.03] Bump version to 18.03.0-ce-rc2	2018-03-06 13:05:52 -08:00
Andrew HsuandGitHub	5613f516dd	Merge pull request #440 from jose-bigio/18.03_changelog [18.03] Updating Changelog for 18.03	2018-03-06 13:05:24 -08:00
jose-bigio	7bc0502750	Updating Changelog for 18.03 Signed-off-by: jose-bigio <jose.bigio@docker.com>	2018-03-06 11:30:36 -08:00
Andrew HsuandGitHub	91bb2aeb67	Merge pull request #453 from thaJeztah/18.03-swarmkit-ingress-attach [18.03] disallow attaching ingress network	2018-03-06 11:27:15 -08:00
Andrew HsuandGitHub	5ba2b1a74d	Merge pull request #452 from thaJeztah/18.03-backport-selansen-36247 [18.03] Fix to address regression caused by PR 30897	2018-03-06 10:04:37 -08:00
Sebastiaan van Stijn	dbe2a19e83	CLI bump swarmkit to 11d7b06f48bc1d73fc6d8776c3552a4b11c94301 Contains no changes for the CLI, but keeps the version in sync with engine Signed-off-by: Sebastiaan van Stijn <github@gone.nl>	2018-03-06 17:32:08 +01:00
Sebastiaan van Stijn	2d690d4e87	Engine: bump swarmkit to 11d7b06f48bc1d73fc6d8776c3552a4b11c94301 Ingress network should not be attachable Ingress network is a special network used only to expose ports. For this reason the network cannot be explicitly attached during service create or service update Signed-off-by: Sebastiaan van Stijn <github@gone.nl>	2018-03-06 17:31:43 +01:00
Andrew HsuandGitHub	3ab4d93c66	Merge pull request #451 from andrewhsu/md [18.03] vndr swarmkit to 49a9d7f	2018-03-06 07:39:11 -08:00
Andrew HsuandGitHub	cb1018ea72	Merge pull request #449 from tonistiigi/layer-leak-fix [18.03] builder: fix layer lifecycle leak	2018-03-06 07:37:26 -08:00
selansenandSebastiaan van Stijn	3310baba0f	Fix to address regression caused by PR 30897 With the inclusion of PR 30897, creating service for host network fails in 18.02. Modified IsPreDefinedNetwork check and return NetworkNameError instead of errdefs.Forbidden to address this issue Signed-off-by: selansen <elango.siva@docker.com> (cherry picked from commit 7cf8b20762cc9491f52ff3f3d94c880378183696) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>	2018-03-06 13:42:37 +01:00
Andrew HsuandGitHub	9dbc108a14	Merge pull request #441 from thaJeztah/18.03-backport-templated-configs-secrets [18.03] Add --template-driver option for secrets/configs	2018-03-05 22:25:55 -08:00
Andrew HsuandGitHub	977f2704b3	Merge pull request #450 from thaJeztah/18.03-backport-bump-runc-1.0-rc5 [18.03] Bump Runc to 1.0.0-rc5	2018-03-05 22:22:01 -08:00
Andrew Hsu	1b39f8bd26	cli: vndr swarmkit 49a9d7f Signed-off-by: Andrew Hsu <andrewhsu@docker.com>	2018-03-06 01:45:21 +00:00
Andrew Hsu	652953a81f	engine: vndr swarmkit 49a9d7f Signed-off-by: Andrew Hsu <andrewhsu@docker.com>	2018-03-06 01:43:57 +00:00
Andrew HsuandGitHub	7000ca4203	Merge pull request #447 from vdemeester/tests-fixes [18.03] Fixes tests to have the ci green	2018-03-05 17:19:37 -08:00
Sebastiaan van Stijn	5bc239fe16	bump containerd/console to 2748ece16665b45a47f884001d5831ec79703880 Fix runc exec on big-endian, causing: container_linux.go:265: starting container process caused "open /dev/pts/4294967296: no such file or directory" Signed-off-by: Sebastiaan van Stijn <github@gone.nl> (cherry picked from commit aab5eaddccb8cb196fdb1e285890dfa94a071b14) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>	2018-03-05 23:56:22 +01:00
Sebastiaan van Stijn	aca674de82	Bump Runc to 1.0.0-rc5 / 4fc53a81fb7c994640722ac585fa9ca548971871 Release notes: https://github.com/opencontainers/runc/releases/tag/v1.0.0-rc5 Possibly relevant changes included: - chroot when no mount namespaces is provided - fix systemd slice expansion so that it could be consumed by cAdvisor - libcontainer/capabilities_linux: Drop os.Getpid() call - Update console dependency to fix runc exec on BE (causing: `container_linux.go:265: starting container process caused "open /dev/pts/4294967296: no such file or directory"`) - libcontainer: setupUserNamespace is always called (fixes: Devices are mounted with wrong uid/gid) Signed-off-by: Sebastiaan van Stijn <github@gone.nl> (cherry picked from commit a2f5a1a5b2d77d694c5bd47798be15b3c0bcdf70) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>	2018-03-05 23:56:13 +01:00
Brian GoffandSebastiaan van Stijn	c709b18bfd	Split binary installers/commit scripts Originally I worked on this for the multi-stage build Dockerfile changes. Decided to split this out as we are still waiting for multi-stage to be available on CI and rebasing these is pretty annoying. Signed-off-by: Brian Goff <cpuguy83@gmail.com> (cherry picked from commit b529d1b0936b90ae14d584c73f7332919f8d76b7) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>	2018-03-05 23:56:02 +01:00
Tonis Tiigi	6f6e5c5f2c	builder: fix layer lifecycle leak Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com> (cherry picked from commit 7ad41d53df94c4277574d14809211b42dca2becc) Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>	2018-03-05 14:11:22 -08:00
Vincent Demeester	50c9a31b4c	Fix `--label` behavior on run Commit `2b17f4c8a8` fixed the way empty labels are taken into account (i.e. not interpolated from environment variable), but it created a regression. `ValidateLabel` functions doesn't allow empty label value, but it has always been possible to pass an empty label via the cli (`docker run --label foo`). This fixes that by not validating the label flag. Signed-off-by: Vincent Demeester <vincent@sbr.pm> (cherry picked from commit `31dc5c0a9a`) Signed-off-by: Vincent Demeester <vincent@sbr.pm>	2018-03-02 10:21:53 +01:00
Andrew HsuandGitHub	f8e0c47b29	Merge pull request #442 from thaJeztah/18.03-docs-cherry-picks [18.03] docs cherry-picks	2018-02-28 13:32:40 -08:00
Lydell MangantiandSebastiaan van Stijn	767a8f6227	Update run.md --restart to include unless-stopped Update --restart option to include unless-stopped to be consistent with https://docs.docker.com/config/containers/start-containers-automatically/#use-a-restart-policy Signed-off-by: Lydell Manganti <lydell.manganti@gmail.com> (cherry picked from commit `d281b72a98`) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>	2018-02-28 18:10:39 +01:00
Daniel NephinandSebastiaan van Stijn	36343864e2	Migrate some copy tests to integration Signed-off-by: Daniel Nephin <dnephin@docker.com> (cherry picked from commit 00d409f03ed825f623b6ef8ec5a3a91cd26194c2) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>	2018-02-28 12:29:10 +01:00
Daniel NephinandSebastiaan van Stijn	7d395933ee	Add more container cp tests Signed-off-by: Daniel Nephin <dnephin@docker.com> (cherry picked from commit `07cb69e9bc`) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>	2018-02-28 11:16:23 +01:00
Vincent Demeester	c70b6c9f35	Clean some docker_cli_build_tests that are cli-only Remove TestBuildRenamedDockerfile and TestBuildDockerfileOutsideContext that are cli-only tests (and already tested in the docker/cli repository). Also adds some comments on few tests that could be migrate to docker/cli. Signed-off-by: Vincent Demeester <vincent@sbr.pm> (cherry picked from commit 894c213b3bd6f4d8f344837b5b5084360a013680) Signed-off-by: Vincent Demeester <vincent@sbr.pm>	2018-02-28 10:13:03 +01:00
jose-bigio	dd1b760bad	Bump version to 18.03.0-ce-rc2 Signed-off-by: jose-bigio <jose.bigio@docker.com>	2018-02-27 09:34:44 -08:00
Akihiro SudaandSebastiaan van Stijn	de4362e128	docs: mention sctp Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp> (cherry picked from commit `b85d87b8ab`) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>	2018-02-26 11:40:06 +01:00
Sebastiaan van Stijn	eda1e25f5c	Add --template-driver option for secrets/configs Signed-off-by: Sebastiaan van Stijn <github@gone.nl> (cherry picked from commit `d11b5ccdfa`) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>	2018-02-26 11:34:23 +01:00