Bug 1373780 - Running systemd in fedora:rawhide container does not show status
Summary: Running systemd in fedora:rawhide container does not show status
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Fedora
Classification: Fedora
Component: docker
Version: 30
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Daniel Walsh
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-09-07 06:45 UTC by Jan Pazdziora
Modified: 2019-12-23 13:45 UTC (History)
30 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-12-23 13:45:12 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Jan Pazdziora 2016-09-07 06:45:03 UTC
Description of problem:

When running /usr/sbin/init in docker run -ti in fedora:rawhide container, the status is not shown.

Version-Release number of selected component (if applicable):

On the host:

kernel-4.7.2-201.fc24.x86_64
systemd-229-13.fc24.x86_64
selinux-policy-3.13.1-191.14.fc24.noarch
docker-1.10.3-26.git1ecb834.fc24.x86_64

In the container:

docker.io/fedora rawhide 3bcdeb6ee43b 3 weeks ago 174 MB
systemd-231-3.fc26.x86_64

How reproducible:

Deterministic.

Steps to Reproduce:
1. docker run --rm -ti -e container=docker -v /sys/fs/cgroup:/sys/fs/cgroup:ro --tmpfs /run --tmpfs /tmp fedora:rawhide /usr/sbin/init

Actual results:

No output.

Expected results:

Status of systemd booting, similar to output from fedora:24 container:

systemd 229 running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN)
Detected virtualization docker.
Detected architecture x86-64.
Running with unpopulated /etc.

Welcome to Fedora 24 (Twenty Four)!

Set hostname to <791ed8c1887f>.
Initializing machine ID from random generator.
Failed to populate /etc with preset unit settings, ignoring: No such file or directory
[  OK  ] Listening on Journal Socket (/dev/log).
[  OK  ] Reached target Local File Systems.
[  OK  ] Reached target Encrypted Volumes.
[  OK  ] Listening on /dev/initctl Compatibility Named Pipe.
[  OK  ] Reached target Swap.
[  OK  ] Listening on Journal Socket.
[  OK  ] Started Dispatch Password Requests to Console Directory Watch.
[  OK  ] Listening on Process Core Dump Socket.
[  OK  ] Reached target Remote File Systems.
[  OK  ] Created slice System Slice.
         Starting First Boot Wizard...
         Starting Rebuild Dynamic Linker Cache...
[  OK  ] Reached target Slices.
         Starting Load/Save Random Seed...
         Starting Rebuild Journal Catalog...
[  OK  ] Started Forward Password Requests to Wall Directory Watch.
[  OK  ] Reached target Paths.
         Starting Journal Service...
[  OK  ] Started Load/Save Random Seed.
[  OK  ] Started First Boot Wizard.
         Starting Create System Users...
[  OK  ] Started Rebuild Journal Catalog.
[  OK  ] Started Create System Users.
[  OK  ] Started Journal Service.
         Starting Flush Journal to Persistent Storage...
[  OK  ] Started Flush Journal to Persistent Storage.
         Starting Create Volatile Files and Directories...
[  OK  ] Started Create Volatile Files and Directories.
         Starting Update UTMP about System Boot/Shutdown...
[  OK  ] Started Update UTMP about System Boot/Shutdown.
[  OK  ] Started Rebuild Dynamic Linker Cache.
         Starting Update is Completed...
[  OK  ] Started Update is Completed.
[  OK  ] Reached target System Initialization.
[  OK  ] Listening on D-Bus System Message Bus Socket.
[  OK  ] Reached target Sockets.
[  OK  ] Started Daily Cleanup of Temporary Directories.
[  OK  ] Started dnf makecache timer.
[  OK  ] Reached target Timers.
[  OK  ] Reached target Basic System.
[  OK  ] Started D-Bus System Message Bus.
         Starting Permit User Sessions...
[  OK  ] Started Permit User Sessions.
[  OK  ] Reached target Multi-User System.
         Starting Update UTMP about System Runlevel Changes...
[  OK  ] Started Update UTMP about System Runlevel Changes.

Additional info:

Running in permissive to make sure bug 1373746 or bug 1373772 aren't the reason.

Adding --show-status does not help.

Comment 1 Jan Pazdziora 2016-09-07 06:46:42 UTC
This is a regression against fedora:24 image.

Comment 2 Jan Pazdziora 2016-09-07 06:47:05 UTC
Running

docker exec -ti adoring_kare journalctl -l

produces


-- Logs begin at Wed 2016-09-07 06:45:43 UTC, end at Wed 2016-09-07 06:45:44 UTC. --
Sep 07 06:45:43 70a94aa6978c systemd-journald[21]: Runtime journal (/run/log/journal/) is 820.0K, max 6.4M, 5.6M free.
Sep 07 06:45:44 70a94aa6978c systemd-journald[21]: System journal (/var/log/journal/) is 8.0M, max 1022.2M, 1014.2M free.
Sep 07 06:45:44 70a94aa6978c systemd-journald[21]: Time spent on flushing to /var is 472us for 2 entries.
Sep 07 06:45:44 70a94aa6978c systemd-journald[21]: Journal started
Sep 07 06:45:44 70a94aa6978c systemd-sysusers[19]: Creating group systemd-coredump with gid 994.
Sep 07 06:45:44 70a94aa6978c systemd-sysusers[19]: Creating user systemd-coredump (systemd Core Dumper) with uid 994 and gid 994.
Sep 07 06:45:44 70a94aa6978c systemd-sysusers[19]: Failed to write files: Operation not permitted
Sep 07 06:45:44 70a94aa6978c systemd[1]: Starting Flush Journal to Persistent Storage...
Sep 07 06:45:44 70a94aa6978c systemd[1]: systemd-journald.service: Couldn't add fd to fd store: Operation not permitted
Sep 07 06:45:44 70a94aa6978c systemd[1]: systemd-journald.service: Couldn't add fd to fd store: Operation not permitted
Sep 07 06:45:44 70a94aa6978c systemd[1]: Started Flush Journal to Persistent Storage.
Sep 07 06:45:44 70a94aa6978c systemd[1]: Starting Create Volatile Files and Directories...
Sep 07 06:45:44 70a94aa6978c systemd-tmpfiles[25]: Cannot set file attribute for '/var/log/journal', value=0x00800000, mask=0x
00800000: Operation not supported
Sep 07 06:45:44 70a94aa6978c systemd-tmpfiles[25]: Cannot set file attribute for '/var/log/journal/f3578012acfd4ac9869c7194edc
ed12a', value=0x00800000, mask=0x00800000: Operation not supported
Sep 07 06:45:44 70a94aa6978c systemd[1]: Started Create Volatile Files and Directories.
Sep 07 06:45:44 70a94aa6978c systemd[1]: Starting Update UTMP about System Boot/Shutdown...
Sep 07 06:45:44 70a94aa6978c systemd[1]: Started Update UTMP about System Boot/Shutdown.
Sep 07 06:45:44 70a94aa6978c systemd[1]: Started Rebuild Dynamic Linker Cache.
Sep 07 06:45:44 70a94aa6978c systemd[1]: Starting Update is Completed...
Sep 07 06:45:44 70a94aa6978c systemd[1]: Started Update is Completed.
Sep 07 06:45:44 70a94aa6978c systemd[1]: Reached target System Initialization.
Sep 07 06:45:44 70a94aa6978c systemd[1]: Listening on D-Bus System Message Bus Socket.
Sep 07 06:45:44 70a94aa6978c systemd[1]: Reached target Sockets.
Sep 07 06:45:44 70a94aa6978c systemd[1]: Started Daily Cleanup of Temporary Directories.
Sep 07 06:45:44 70a94aa6978c systemd[1]: Started dnf makecache timer.
Sep 07 06:45:44 70a94aa6978c systemd[1]: Reached target Timers.
Sep 07 06:45:44 70a94aa6978c systemd[1]: Reached target Basic System.
Sep 07 06:45:44 70a94aa6978c systemd[1]: Starting Login Service...
Sep 07 06:45:44 70a94aa6978c systemd[1]: Starting Permit User Sessions...
Sep 07 06:45:44 70a94aa6978c systemd[1]: Started D-Bus System Message Bus.
Sep 07 06:45:44 70a94aa6978c systemd[1]: systemd-journald.service: Couldn't add fd to fd store: Operation not permitted
Sep 07 06:45:44 70a94aa6978c systemd[1]: systemd-journald.service: Couldn't add fd to fd store: Operation not permitted
Sep 07 06:45:44 70a94aa6978c systemd[1]: Started Permit User Sessions.
Sep 07 06:45:44 70a94aa6978c systemd-logind[28]: New seat seat0.
Sep 07 06:45:44 70a94aa6978c systemd[1]: Started Console Getty.
Sep 07 06:45:44 70a94aa6978c systemd[1]: Reached target Login Prompts.
Sep 07 06:45:44 70a94aa6978c systemd[1]: Started Login Service.
Sep 07 06:45:44 70a94aa6978c systemd[1]: Reached target Multi-User System.
Sep 07 06:45:44 70a94aa6978c systemd[1]: Starting Update UTMP about System Runlevel Changes...
Sep 07 06:45:44 70a94aa6978c systemd[1]: systemd-journald.service: Couldn't add fd to fd store: Operation not permitted
Sep 07 06:45:44 70a94aa6978c systemd[1]: Started Update UTMP about System Runlevel Changes.
Sep 07 06:45:44 70a94aa6978c systemd[1]: Startup finished in 1.629s.
Sep 07 06:45:44 70a94aa6978c systemd[1]: systemd-journald.service: Couldn't add fd to fd store: Operation not permitted

Comment 3 Jan Pazdziora 2016-11-29 08:32:29 UTC
I see this problem both with fedora:25 and fedora:rawhide.

Could we get any opinion about the issue from the maintainers?

Comment 4 Zbigniew Jędrzejewski-Szmek 2016-11-29 14:35:01 UTC
docker closes the pty that is container's /dev/console when it sees a POLLHUP. systemd (by mistake in the code), was keeping /dev/console open, but was recently fixed to not do that, and thus exposed the issue in docker.

See https://github.com/systemd/systemd/pull/4262, https://github.com/docker/docker/issues/27202.

Comment 7 Zbigniew Jędrzejewski-Szmek 2017-01-13 17:10:27 UTC
*** Bug 1413099 has been marked as a duplicate of this bug. ***

Comment 8 Fedora End Of Life 2017-02-28 10:13:01 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 26 development cycle.
Changing version to '26'.

Comment 9 Jan Pazdziora 2017-10-27 09:30:33 UTC
I've retested the issue now with docker-1.12.6-61.git85d7426.el7.x86_64 on RHEL 7 and with fedora:25, fedora:26, and fedora:rawhide images, and the behaviour is nondeterministic now -- sometimes the startup status is shown, sometimes it's not.

With fedora:24, it is always shown.

Comment 10 Pavel Raiskup 2018-02-08 15:19:44 UTC
Workaround:
$ cat systemd
#! /bin/bash
export CONSOLE_FD
exec {CONSOLE_FD}<> /dev/console
exec /sbin/init

$ cat Dockerfile
...
ADD systemd /systemd
...
CMD ["/systemd"]

I'm curious whether oci-systemd-hooks would be able to workaround this
directly.

Comment 11 Jan Pazdziora 2018-03-22 12:19:29 UTC
The approach from comment 10 works for plain docker but it does not seem to work in OpenShift where (the equivalent of -ti is not done).

But running

if ! [ -e /dev/console ] ; then
    socat -u pty,link=/dev/console stdout &
fi
exec /usr/sbin/init "$@"

in that systemd wrapper seems to work fine and produce the status output in the pod logs.

I guess the question still holds -- what should be the expected behaviour. Frankly, I'd expect /usr/sbin/init --show-status=true to always provide the status output, console or no-console. Especially in the container. And yes, I can imagine that in the container the default of the --show-status could be false.

Comment 12 Fedora End Of Life 2018-05-03 08:02:18 UTC
This message is a reminder that Fedora 26 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 26. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '26'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version'
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not
able to fix it before Fedora 26 is end of life. If you would still like
to see this bug fixed and are able to reproduce it against a later version
of Fedora, you are encouraged  change the 'version' to a later Fedora
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.

Comment 13 Jan Pazdziora 2018-05-09 09:30:15 UTC
The bug is still present on fedora:27 image

REPOSITORY                          TAG                 IMAGE ID            CREATED             SIZE
registry.fedoraproject.org/fedora   27                  6e2c293f3942        2 weeks ago         235 MB


with systemd-234-10.git5f8984e.fc27.x86_64, when run under docker-1.13.1-51.git4032bd5.fc28.x86_64.

Comment 14 Jan Pazdziora 2018-08-13 13:37:51 UTC
I'd like to point out that lately, running docker run -t fails to start systemd in the container completely: bug 1615082.

Could we take some look at the interaction of docker and systemd and get these issues fixed?

Comment 15 Tom Sweeney 2018-08-13 13:41:30 UTC
The last I heard dwalsh/rhatdan was running down a selinux bug with the kernel folks.  I'm not sure where that's at.  Dan any updates?

Comment 16 Jan Pazdziora 2018-08-13 14:02:53 UTC
For this bugzilla, for bug 1615082, or for something else? I'm aware of the need of having to have the container_manage_cgroup SELinux boolean, which I have set. Dan, are you aware of the status / plan?

Comment 17 Daniel Walsh 2018-08-13 18:27:38 UTC
I have not heard anything new from the kernel guys.  I will send out a whine to see if anything has moved forward.

Comment 18 Ben Cotton 2018-11-27 13:29:32 UTC
This message is a reminder that Fedora 27 is nearing its end of life.
On 2018-Nov-30  Fedora will stop maintaining and issuing updates for
Fedora 27. It is Fedora's policy to close all bug reports from releases
that are no longer maintained. At that time this bug will be closed as
EOL if it remains open with a Fedora  'version' of '27'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 27 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 19 Alexander Bokovoy 2018-11-28 10:29:20 UTC
This still happens in F29 container

Nov 28 08:45:15 ipa.example.test systemd[1]: Started Create Volatile Files and Directories.
Nov 28 08:45:15 ipa.example.test systemd[1]: Reached target System Initialization.
Nov 28 08:45:15 ipa.example.test systemd[1]: Starting Configure IPA server upon the first start...
Nov 28 08:45:15 ipa.example.test systemd[1]: Listening on D-Bus System Message Bus Socket.
Nov 28 08:45:15 ipa.example.test systemd[1]: Started D-Bus System Message Bus.
Nov 28 08:45:15 ipa.example.test systemd[1]: Reached target Minimal target for containerized FreeIPA server.
Nov 28 08:45:15 ipa.example.test systemd[1]: systemd-journald.service: Failed to add fd to store: Operation not permitted

Comment 20 Ben Cotton 2019-10-31 19:14:58 UTC
This message is a reminder that Fedora 29 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 29 on 2019-11-26.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '29'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 29 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 21 Jan Pazdziora 2019-11-01 10:13:34 UTC
The issues is still present on Fedora 30 with registry.fedoraproject.org/fedora:rawhide container.

Comment 22 Zbigniew Jędrzejewski-Szmek 2019-12-21 16:07:30 UTC
Please don't reassign this back to systemd. As described in comment #4 above, we do not
consider that there's anything to fix in systemd, so this bug will only go even more stale.
The linked mobi bug 27202 claims the issue was fixed in containerd, so maybe the issue is fixed.

Comment 23 Daniel Walsh 2019-12-23 13:45:12 UTC
Fedora no longer supports Docker, so closing.


Note You need to log in before you can comment on or make changes to this bug.