Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1554531

Summary:	Docker daemon panics with "out of memory" when large docker cp executed
Product:	OpenShift Container Platform	Reporter:	Clayton Coleman <ccoleman>
Component:	Containers	Assignee:	Antonio Murdaca <amurdaca>
Status:	CLOSED DUPLICATE	QA Contact:	DeShuai Ma <dma>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	3.9.0	CC:	aos-bugs, jhonce, jlebon, jokerman, mmccomas
Target Milestone:	---
Target Release:	3.9.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2018-03-13 17:31:51 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Clayton Coleman 2018-03-12 21:50:53 UTC

https://github.com/openshift/origin/issues/18952

On a CI job we got an OOM panic from the docker daemon while we were invoking docker cp to retrieve contents from the container. This happened when I added a new large binary to the cp, which implies that the docker daemon is loading the entire request/response for the docker cp in memory, which is not good.

Logs are here, the size of the CP is probably 3-5gb and this is an 8gb box

https://storage.googleapis.com/origin-ci-test/pr-logs/pull/18686/test_pull_request_origin_extended_conformance_gce/17539/artifacts/journals/docker.service

Mar 12 20:56:26 ip-172-18-9-32.ec2.internal dockerd-current[1576]: time="2018-03-12T20:56:26.985884917Z" level=debug msg="copying \"local\" from \"/var/lib/docker/devicemapper/mnt/feca6766a495350c416fe42e14603675cfc4d81e0acb5244c5c7645032eb3571/rootfs/go/src/github.com/openshift/origin/_output\""
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: fatal error: runtime: out of memory
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: runtime stack:
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: runtime.throw(0x1a2d90c, 0x16)
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /usr/lib/golang/src/runtime/panic.go:596 +0x95 fp=0x7f2fb97f9a68 sp=0x7f2fb97f9a48
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: runtime.sysMap(0xc6bc010000, 0x104ea0000, 0x2637000, 0x2651958)
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /usr/lib/golang/src/runtime/mem_linux.go:216 +0x1d0 fp=0x7f2fb97f9aa8 sp=0x7f2fb97f9a68
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: runtime.(*mheap).sysAlloc(0x2637060, 0x104ea0000, 0x438322)
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /usr/lib/golang/src/runtime/malloc.go:440 +0x374 fp=0x7f2fb97f9b30 sp=0x7f2fb97f9aa8
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: runtime.(*mheap).grow(0x2637060, 0x8274c, 0x0)
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /usr/lib/golang/src/runtime/mheap.go:774 +0x62 fp=0x7f2fb97f9b88 sp=0x7f2fb97f9b30
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: runtime.(*mheap).allocSpanLocked(0x2637060, 0x8274c, 0x262f5f0)
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /usr/lib/golang/src/runtime/mheap.go:678 +0x44f fp=0x7f2fb97f9bc8 sp=0x7f2fb97f9b88
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: runtime.(*mheap).alloc_m(0x2637060, 0x8274c, 0x100000000, 0xffffffffffffffff)
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /usr/lib/golang/src/runtime/mheap.go:562 +0xe2 fp=0x7f2fb97f9c00 sp=0x7f2fb97f9bc8
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: runtime.(*mheap).alloc.func1()
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /usr/lib/golang/src/runtime/mheap.go:627 +0x4b fp=0x7f2fb97f9c38 sp=0x7f2fb97f9c00
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: runtime.systemstack(0x7f2fb97f9c58)
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /usr/lib/golang/src/runtime/asm_amd64.s:343 +0xab fp=0x7f2fb97f9c40 sp=0x7f2fb97f9c38
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: runtime.(*mheap).alloc(0x2637060, 0x8274c, 0x100000000, 0xc420020600)
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /usr/lib/golang/src/runtime/mheap.go:628 +0xa0 fp=0x7f2fb97f9c90 sp=0x7f2fb97f9c40
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: runtime.largeAlloc(0x104e98000, 0xc421594100, 0x4)
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /usr/lib/golang/src/runtime/malloc.go:807 +0x93 fp=0x7f2fb97f9cd0 sp=0x7f2fb97f9c90
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: runtime.mallocgc.func1()
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /usr/lib/golang/src/runtime/malloc.go:702 +0x3e fp=0x7f2fb97f9d00 sp=0x7f2fb97f9cd0
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: runtime.systemstack(0xc420021900)
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /usr/lib/golang/src/runtime/asm_amd64.s:327 +0x79 fp=0x7f2fb97f9d08 sp=0x7f2fb97f9d00
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: runtime.mstart()
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /usr/lib/golang/src/runtime/proc.go:1132 fp=0x7f2fb97f9d10 sp=0x7f2fb97f9d08
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: goroutine 2017 [running]:
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: runtime.systemstack_switch()
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /usr/lib/golang/src/runtime/asm_amd64.s:281 fp=0xc4202c7260 sp=0xc4202c7258
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: runtime.mallocgc(0x104e98000, 0x0, 0xc4202c7300, 0x480b7d)
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /usr/lib/golang/src/runtime/malloc.go:703 +0x930 fp=0xc4202c7300 sp=0xc4202c7260
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: runtime.growslice(0x1716140, 0xc5e724a000, 0xd0ba8000, 0xd0bac000, 0xd0bb0000, 0xc422248000, 0x8000, 0x8000)
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /usr/lib/golang/src/runtime/slice.go:140 +0x23e fp=0xc4202c7388 sp=0xc4202c7300
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: github.com/docker/docker/pkg/authorization.(*responseModifier).Write(0xc4229c9000, 0xc422248000, 0x8000, 0x8000, 0x8000, 0x0, 0x0)
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /builddir/build/BUILD/docker-774336db27bb8931c1705e47781b3842d290c968/_build/src/github.com/docker/docker/pkg/authorization/response.go:124 +0x125 fp=0xc4202c73f0 sp=0xc4202c7388
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: io.copyBuffer(0x7f2fe0fb1510, 0xc4229c9000, 0x7f2fe0fb2700, 0xc420b7dfa0, 0xc422248000, 0x8000, 0x8000, 0x17cc160, 0x1a21b00, 0x7f2fe0fb2700)
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /usr/lib/golang/src/io/io.go:392 +0x20b fp=0xc4202c7458 sp=0xc4202c73f0
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: io.Copy(0x7f2fe0fb1510, 0xc4229c9000, 0x7f2fe0fb2700, 0xc420b7dfa0, 0xc420b7dfa0, 0x25e5020, 0xc420b7dfa0)
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /usr/lib/golang/src/io/io.go:360 +0x68 fp=0xc4202c74b8 sp=0xc4202c7458
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: github.com/docker/docker/api/server/router/container.(*containerRouter).getContainersArchive(0xc420322c00, 0x25f4ca0, 0xc421e029c0, 0x7f2fe0f5d880, 0xc4229c9000, 0xc42219ee00, 0xc421e02480, 0x0, 0x0)
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /builddir/build/BUILD/docker-774336db27bb8931c1705e47781b3842d290c968/_build/src/github.com/docker/docker/api/server/router/container/copy.go:106 +0x247 fp=0xc4202c7540 sp=0xc4202c74b8
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: github.com/docker/docker/api/server/router/container.(*containerRouter).(github.com/docker/docker/api/server/router/container.getContainersArchive)-fm(0x25f4ca0, 0xc421e029c0, 0x7f2fe0f5d880, 0xc4229c9000, 0xc42219ee00, 0xc421e02480, 0x41fb78, 0x30)
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /builddir/build/BUILD/docker-774336db27bb8931c1705e47781b3842d290c968/_build/src/github.com/docker/docker/api/server/router/container/container.go:53 +0x69 fp=0xc4202c7598 sp=0xc4202c7540
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: github.com/docker/docker/api/server/middleware.ExperimentalMiddleware.WrapHandler.func1(0x25f4ca0, 0xc421e029c0, 0x7f2fe0f5d880, 0xc4229c9000, 0xc42219ee00, 0xc421e02480, 0x15, 0x196abc0)
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /builddir/build/BUILD/docker-774336db27bb8931c1705e47781b3842d290c968/_build/src/github.com/docker/docker/api/server/middleware/experimental.go:27 +0xd8 fp=0xc4202c7600 sp=0xc4202c7598
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: github.com/docker/docker/api/server/middleware.VersionMiddleware.WrapHandler.func1(0x25f4ca0, 0xc421e02540, 0x7f2fe0f5d880, 0xc4229c9000, 0xc42219ee00, 0xc421e02480, 0xc421e00880, 0x80)
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /builddir/build/BUILD/docker-774336db27bb8931c1705e47781b3842d290c968/_build/src/github.com/docker/docker/api/server/middleware/version.go:47 +0x407 fp=0xc4202c7748 sp=0xc4202c7600
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: github.com/docker/docker/pkg/authorization.(*Middleware).WrapHandler.func1(0x25f4ca0, 0xc421e02540, 0x25f1320, 0xc421e4a620, 0xc42219ee00, 0xc421e02480, 0xc421b0da28, 0x2)
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /builddir/build/BUILD/docker-774336db27bb8931c1705e47781b3842d290c968/_build/src/github.com/docker/docker/pkg/authorization/middleware.go:69 +0x405 fp=0xc4202c78f8 sp=0xc4202c7748
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: github.com/docker/docker/api/server/middleware.AuditMiddleware.WrapHandler.func1(0x25f4ca0, 0xc421e02540, 0x25f1320, 0xc421e4a620, 0xc42219ee00, 0xc421e02480, 0xc421b0da98, 0x756ea15b2610e2)
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /builddir/build/BUILD/docker-774336db27bb8931c1705e47781b3842d290c968/_build/src/github.com/docker/docker/api/server/middleware/audit_linux.go:27 +0x9d fp=0xc4202c7950 sp=0xc4202c78f8
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: github.com/docker/docker/api/server/middleware.DebugRequestMiddleware.func1(0x25f4ca0, 0xc421e02540, 0x25f1320, 0xc421e4a620, 0xc42219ee00, 0xc421e02480, 0xc4212f8cf8, 0xc421f367b0)
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /builddir/build/BUILD/docker-774336db27bb8931c1705e47781b3842d290c968/_build/src/github.com/docker/docker/api/server/middleware/debug.go:22 +0x180 fp=0xc4202c7ab0 sp=0xc4202c7950
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: github.com/docker/docker/api/server.(*Server).makeHTTPHandler.func1(0x25f1320, 0xc421e4a620, 0xc42219ee00)
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /builddir/build/BUILD/docker-774336db27bb8931c1705e47781b3842d290c968/_build/src/github.com/docker/docker/api/server/server.go:146 +0x230 fp=0xc4202c7bc0 sp=0xc4202c7ab0
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: net/http.HandlerFunc.ServeHTTP(0xc420aadbe0, 0x25f1320, 0xc421e4a620, 0xc42219ee00)
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /usr/lib/golang/src/net/http/server.go:1942 +0x44 fp=0xc4202c7be8 sp=0xc4202c7bc0
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: github.com/docker/docker/vendor/github.com/gorilla/mux.(*Router).ServeHTTP(0xc4201d8550, 0x25f1320, 0xc421e4a620, 0xc42219ee00)
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /builddir/build/BUILD/docker-774336db27bb8931c1705e47781b3842d290c968/_build/src/github.com/docker/docker/vendor/github.com/gorilla/mux/mux.go:103 +0x255 fp=0xc4202c7cd8 sp=0xc4202c7be8
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: github.com/docker/docker/api/server.(*routerSwapper).ServeHTTP(0xc420d055b0, 0x25f1320, 0xc421e4a620, 0xc42219ee00)
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /builddir/build/BUILD/docker-774336db27bb8931c1705e47781b3842d290c968/_build/src/github.com/docker/docker/api/server/router_swapper.go:29 +0x70 fp=0xc4202c7d10 sp=0xc4202c7cd8
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: net/http.serverHandler.ServeHTTP(0xc4203af600, 0x25f1320, 0xc421e4a620, 0xc42219ee00)
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /usr/lib/golang/src/net/http/server.go:2568 +0x92 fp=0xc4202c7d58 sp=0xc4202c7d10
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: net/http.(*conn).serve(0xc4214e3d60, 0x25f2fe0, 0xc4229c8a40)
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /usr/lib/golang/src/net/http/server.go:1825 +0x612 fp=0xc4202c7fc8 sp=0xc4202c7d58
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: runtime.goexit()
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /usr/lib/golang/src/runtime/asm_amd64.s:2197 +0x1 fp=0xc4202c7fd0 sp=0xc4202c7fc8
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: created by net/http.(*Server).Serve
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /usr/lib/golang/src/net/http/server.go:2668 +0x2ce
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: goroutine 1 [chan receive, 35 minutes]:
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: runtime.gopark(0x1a96430, 0xc420d12838, 0x1a1789f, 0xc, 0x1a96517, 0x3)
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /usr/lib/golang/src/runtime/proc.go:271 +0x13a fp=0xc4202cb678 sp=0xc4202cb648
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: runtime.goparkunlock(0xc420d12838, 0x1a1789f, 0xc, 0x17, 0x3)
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /usr/lib/golang/src/runtime/proc.go:277 +0x5e fp=0xc4202cb6b8 sp=0xc4202cb678
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: runtime.chanrecv(0x1703340, 0xc420d127e0, 0xc4209dbaa8, 0xc4209db701, 0x154d7d9)
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /usr/lib/golang/src/runtime/chan.go:513 +0x371 fp=0xc4202cb758 sp=0xc4202cb6b8
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: runtime.chanrecv1(0x1703340, 0xc420d127e0, 0xc4209dbaa8)
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /usr/lib/golang/src/runtime/chan.go:395 +0x35 fp=0xc4202cb790 sp=0xc4202cb758
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: main.(*DaemonCli).start(0xc42042a8d0, 0x0, 0x1a2e162, 0x17, 0xc420202000, 0xc420056f00, 0xc42042cab0, 0x0, 0x0)
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /builddir/build/BUILD/docker-774336db27bb8931c1705e47781b3842d290c968/_build/src/github.com/docker/docker/cmd/dockerd/daemon.go:348 +0x1cf0 fp=0xc4202cbc10 sp=0xc4202cb790
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: main.runDaemon(0x0, 0x1a2e162, 0x17, 0xc420202000, 0xc420056f00, 0xc42042cab0, 0x1e, 0x0)
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /builddir/build/BUILD/docker-774336db27bb8931c1705e47781b3842d290c968/_build/src/github.com/docker/docker/cmd/dockerd/docker.go:86 +0x8c fp=0xc4202cbc68 sp=0xc4202cbc10
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: main.newDaemonCommand.func1(0xc420088480, 0xc4201021e0, 0x0, 0x1e, 0x0, 0x0)
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /builddir/build/BUILD/docker-774336db27bb8931c1705e47781b3842d290c968/_build/src/github.com/docker/docker/cmd/dockerd/docker.go:42 +0x6f fp=0xc4202cbcc0 sp=0xc4202cbc68
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: github.com/docker/docker/vendor/github.com/spf13/cobra.(*Command).execute(0xc420088480, 0xc420010210, 0x1e, 0x1f, 0xc420088480, 0xc420010210)
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /builddir/build/BUILD/docker-774336db27bb8931c1705e47781b3842d290c968/_build/src/github.com/docker/docker/vendor/github.com/spf13/cobra/command.go:646 +0x44e fp=0xc4202cbd78 sp=0xc4202cbcc0
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: github.com/docker/docker/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xc420088480, 0x17edc20, 0x1, 0xc42041bc60)
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /builddir/build/BUILD/docker-774336db27bb8931c1705e47781b3842d290c968/_build/src/github.com/docker/docker/vendor/github.com/spf13/cobra/command.go:742 +0x349 fp=0xc4202cbec8 sp=0xc4202cbd78
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: github.com/docker/docker/vendor/github.com/spf13/cobra.(*Command).Execute(0xc420088480, 0xc42041bc60, 0x0)
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /builddir/build/BUILD/docker-774336db27bb8931c1705e47781b3842d290c968/_build/src/github.com/docker/docker/vendor/github.com/spf13/cobra/command.go:695 +0x2b fp=0xc4202cbef8 sp=0xc4202cbec8
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: main.main()
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /builddir/build/BUILD/docker-774336db27bb8931c1705e47781b3842d290c968/_build/src/github.com/docker/docker/cmd/dockerd/docker.go:106 +0xe6 fp=0xc4202cbf88 sp=0xc4202cbef8
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: runtime.main()
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /usr/lib/golang/src/runtime/proc.go:185 +0x20a fp=0xc4202cbfe0 sp=0xc4202cbf88
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: runtime.goexit()
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /usr/lib/golang/src/runtime/asm_amd64.s:2197 +0x1 fp=0xc4202cbfe8 sp=0xc4202cbfe0
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: goroutine 17 [syscall, 35 minutes, locked to thread]:
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: runtime.goexit()
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /usr/lib/golang/src/runtime/asm_amd64.s:2197 +0x1 fp=0xc42003afe8 sp=0xc42003afe0
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: goroutine 2 [force gc (idle), 3 minutes]:
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: runtime.gopark(0x1a96430, 0x262df60, 0x1a1d176, 0xf, 0x1ff302fd914, 0x1)
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /usr/lib/golang/src/runtime/proc.go:271 +0x13a fp=0xc420028768 sp=0xc420028738
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: runtime.goparkunlock(0x262df60, 0x1a1d176, 0xf, 0xc420000114, 0x1)
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /usr/lib/golang/src/runtime/proc.go:277 +0x5e fp=0xc4200287a8 sp=0xc420028768
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: runtime.forcegchelper()
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /usr/lib/golang/src/runtime/proc.go:226 +0x9e fp=0xc4200287e0 sp=0xc4200287a8
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: runtime.goexit()
Mar 12 20:56:40 ip-172-18-9-32.ec2.internal dockerd-current[1576]: /usr/li

Comment 1 Clayton Coleman 2018-03-12 21:51:29 UTC

This does not seem to be happening in 1.12, or at least I have not seen it before.

Comment 2 Clayton Coleman 2018-03-12 22:51:13 UTC

Holding the entire contents of request or response in memory is an attack vector for anything shared like the docker daemon.  If we have anything reading everything it needs to only be done for requests below a certain size, otherwise this could be used to DoS the shared daemon.

Comment 3 Jonathan Lebon 2018-03-13 14:32:49 UTC

This is probably a dupe of https://bugzilla.redhat.com/show_bug.cgi?id=1489517. See also the Fedora version, https://bugzilla.redhat.com/show_bug.cgi?id=1489505, which has some more updates.

Comment 4 Antonio Murdaca 2018-03-13 17:31:51 UTC


*** This bug has been marked as a duplicate of bug 1489517 ***