Bug 1530222

Summary:	/usr/bin/docker-current: Error response from daemon: oci runtime error: container_linux.go:247: starting container process caused "process_linux.go:339: running prestart hook 1 caused \"error running hook: exit status 1, stdout: , stderr: \"".
Product:	[Fedora] Fedora	Reporter:	Avi Kivity <avi.kivity>
Component:	docker	Assignee:	Daniel Walsh <dwalsh>
Status:	CLOSED EOL	QA Contact:	Fedora Extras Quality Assurance <extras-qa>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	27	CC:	adimania, admiller, amurdaca, dwalsh, fkluknav, ichavero, jcajka, kevin, lester.dev, lsm5, marianne, nalin, santiago, vbatts
Target Milestone:	---
Target Release:	---
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2018-11-30 19:45:45 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Avi Kivity 2018-01-02 09:59:43 UTC

Description of problem:

Running docker containers results in errors, about 10% of the time.


Version-Release number of selected component (if applicable):
docker-1.13.1-44.git584d391.fc27.x86_64

How reproducible:
10%



Steps to Reproduce:

I have a disccd server that runs gcc via docker (gcc is containerized, distcc is not). A significant fraction of the time it terminates with an error.

Actual results:

Stuff like:

Jan 02 11:54:48 loki.cloudius-systems.com oci-systemd-hook[3037]: systemdhook <debug>: Skipping as container command is g++, not init or systemd
Jan 02 11:54:48 loki.cloudius-systems.com oci-umount[3039]: umounthook <debug>: a1464b57e647: poststop ignored
Jan 02 11:54:48 loki.cloudius-systems.com docker-containerd-current[910]: time="2018-01-02T11:54:48.549270059+02:00" level=error msg="containerd: deleting container" error="exit status 1: \"container a1464b57e6471b7113fdc61c05f4ff6e09cff8e745515146c3b5202e88617532 does not exist\\none or more of the container deletions failed\\n\""





Expected results:

No errors

Additional info:

Using overlay2 storage. The container just runs gcc from a Fedora container (used to support multiple versions of gcc).

Comment 1 Daniel Walsh 2018-01-02 12:58:52 UTC

Can you remove oci-umount and see if the problem goes away?

Comment 2 Avi Kivity 2018-01-02 13:20:16 UTC

Do you mean the oci-umount rpm? docker depends on it. I can --nodeps, of course.

Comment 3 Daniel Walsh 2018-01-02 13:31:33 UTC

For the time being just mv it out of the /usr/share/oci/hooks.d directory to see if this is causing the issue.  You might want to remove all of the hooks out of that directory to see if they are causing this problem.

Comment 4 Avi Kivity 2018-01-02 13:50:15 UTC

I still see


 /usr/bin/docker-current: Error response from daemon: oci runtime error: container_linux.go:247: starting container process caused "process_linux.go:339: running prestart hook 1 caused \"error running hook: exit status 1, stdout: , stderr: \"".

so I'll try with more hooks removed.

Comment 5 Avi Kivity 2018-01-04 08:41:10 UTC

With oci-unregister-machine and oci-umount removed, things appear to work well.

Comment 6 Avi Kivity 2018-01-04 11:57:23 UTC

With oci-unregister-machine and oci-umount removed, things appear to work well.

Comment 7 Avi Kivity 2018-01-04 11:57:59 UTC

Work well, except for duplicate comments.

Comment 8 Daniel Walsh 2018-01-04 15:21:02 UTC

Now put back oci-umount and see if it still works well?
We have a known issue with oci-register-machine and a cgroupfs back end?

Comment 9 Avi Kivity 2018-01-04 18:43:51 UTC

Did so, will update.

Comment 10 Avi Kivity 2018-01-09 08:20:42 UTC

So far, no problems with oci-umount reinstalled.

Comment 11 Lester 2018-01-11 11:51:51 UTC

Hello Avi, I have the same problem with 'error running hook' trying to run docker containers. Do you remove the package oci-umount by hand or with dnf?

Comment 12 Lester 2018-01-11 12:11:27 UTC

Didn't see the comment with --nodeps. Actually after removing oci-register-machine  the problem disappeared. Once I install it the problem returns.

Comment 13 Ben Cotton 2018-11-27 17:06:58 UTC

This message is a reminder that Fedora 27 is nearing its end of life.
On 2018-Nov-30  Fedora will stop maintaining and issuing updates for
Fedora 27. It is Fedora's policy to close all bug reports from releases
that are no longer maintained. At that time this bug will be closed as
EOL if it remains open with a Fedora  'version' of '27'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 27 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 14 Ben Cotton 2018-11-30 19:45:45 UTC

Fedora 27 changed to end-of-life (EOL) status on 2018-11-30. Fedora 27 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.