Bug 1309719 - systemd based container remove the subscription manager entitlements
systemd based container remove the subscription manager entitlements
Status: ON_QA
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: rhel-server-container (Show other bugs)
7.2
x86_64 Linux
high Severity high
: rc
: ---
Assigned To: Frantisek Kluknavsky
atomic-bugs@redhat.com
: Extras
Depends On:
Blocks: 1305420
  Show dependency treegraph
 
Reported: 2016-02-18 09:50 EST by Mohamed Ashiq
Modified: 2018-04-27 17:35 EDT (History)
5 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Mohamed Ashiq 2016-02-18 09:50:55 EST
Description of problem:


When starting a systemd container in privileged mode, the subscription manager entitlements are not found in the container. 




Version-Release number of selected component (if applicable):

RHEL 7.2 Atomic Host

How reproducible:

-bash-4.2# docker run -d --net=host --privileged --volumes-from glusterdata -v /mnt/brick1/:/b1 -v /mnt/brick2/:/b2 -v /mnt/brick3/:/b3 -v /mnt/brick4/:/b4 brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhgs3/rhgs-server-rhel7  
Unable to find image 'brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhgs3/rhgs-server-rhel7:latest' locally
a6c50dd10511: Download complete 
Status: Image is up to date for brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhgs3/rhgs-server-rhel7:latest

c047f9924ebde3cb17b68fd0e0848c83d2531f06f9fc4a3943f9d56040727440
-bash-4.2# docker exec -ti c047f9924ebde3cb17b68fd0e0848c83d2531f06f9fc4a3943f9d56040727440 /bin/bash
[root@dhcp42-23 /]# 
[root@dhcp42-23 /]# 
[root@dhcp42-23 /]# yum repolist
Loaded plugins: ovl, product-id, search-disabled-repos, subscription-manager
This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
repolist: 0


Steps to Reproduce:

As shown above

Actual results:

The subscription manager identity is not shown inside the container.

Expected results:

It should show the subscription manager identity inside the container.

Additional info:

This works perfectly when we start the container without 'privileged' mode.
Comment 2 Mohamed Ashiq 2016-02-24 02:05:24 EST
Running init script in Rhel7.2 container without privileged:

-bash-4.2# docker run -d -v /sys/fs/cgroup:/sys/fs/cgroup:ro rhel7.2 /usr/sbin/init
bbb791ac74455c09a0dbfb5b3a64332f81e9cb2ae6efcc9cb80e0e0efdc80892
-bash-4.2# docker exec -it bbb791ac74455c09a0dbfb5b3a64332f81e9cb2ae6efcc9cb80e0e0efdc80892 /bin/bash
[root@bbb791ac7445 /]# ls /etc/pki/entitlement
[root@bbb791ac7445 /]# ls /etc/pki/entitlement-host/
2437659666097364716-key.pem  2437659666097364716.pem
[root@bbb791ac7445 /]#


Running init script in Rhel7.2 container with privileged:

-bash-4.2# docker run -d --privileged -v /sys/fs/cgroup:/sys/fs/cgroup:ro rhel7.2 /usr/sbin/init
4b778ae1bcab0d25312d280113b3d409ce613f1b198f8ae99ad2e79a7c036514
-bash-4.2# docker exec -it 4b778ae1bcab0d25312d280113b3d409ce613f1b198f8ae99ad2e79a7c036514 /bin/bash
[root@4b778ae1bcab /]# ls /etc/pki/entitlement
[root@4b778ae1bcab /]# ls /etc/pki/entitlement-host
/etc/pki/entitlement-host
[root@4b778ae1bcab /]#


Atomic host:

-bash-4.2# ls /etc/pki/entitlement/
2437659666097364716-key.pem  2437659666097364716.pem
Comment 4 Frantisek Kluknavsky 2016-02-25 08:13:30 EST
Hi,

could you please try to run `journalctl -r` and `ps afxu` in the container? I suspect that systemd does not run normally.

I am not prepared to explain how to run systemd in docker but as a hint, `docker run -v /run -v /sys/fs/cgroup:/sys/fs/cgroup -d rhel7 /usr/sbin/init` might help you.
Comment 5 Alex Jia 2016-02-25 22:25:10 EST
(In reply to Frantisek Kluknavsky from comment #4)
> `docker run -v /run -v /sys/fs/cgroup:/sys/fs/cgroup -d rhel7
> /usr/sbin/init` might help you.

I gave a try on docker-1.9.1-16.el7.x86_64 w/ and w/o --privileged option per above cmdline, the rhel7 image id is 6c3a84d798dc, it looks fine for me, I both can find /run/secrets/ directory. 

But w/ --privileged option, I found extra some error information in log, it should be irrelevant w/ the bug, I just did a record in here, the details as follows.


[root@dell-per630-02 ~]# docker run -v /run -v /sys/fs/cgroup:/sys/fs/cgroup -d --privileged rhel7 /usr/sbin/init
e7d1cd8dafcfdb53a8163fbb46111e62223e46fc0b4c3b18184ab372b46cec5f
Usage of loopback devices is strongly discouraged for production use. Either use `--storage-opt dm.thinpooldev` or use `--storage-opt dm.no_warn_on_loop_devices=true` to suppress this warning.

[root@dell-per630-02 ~]# docker exec -it e7d1cd8dafcf /bin/bash
[root@e7d1cd8dafcf /]# ps auxf
USER        PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root        111  0.7  0.0  11768  1768 ?        Ss   21:54   0:00 /bin/bash
root        123  0.0  0.0  35884  1444 ?        R+   21:54   0:00  \_ ps auxf
root          1  0.5  0.0  40744  3100 ?        Ss   21:54   0:00 /usr/sbin/init
root         17  0.3  0.0  36816  6428 ?        Ss   21:54   0:00 /usr/lib/systemd/systemd-journald
root         29  0.9  0.0  41776  2120 ?        Ss   21:54   0:00 /usr/lib/systemd/systemd-udevd
dbus         37  0.0  0.0  26460  1484 ?        Ss   21:54   0:00 /bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation
root         38  0.0  0.0   9756   620 ?        Ss   21:54   0:00 /usr/bin/rhsmcertd
root         52  0.0  0.0  24272  1516 ?        Ss   21:54   0:00 /usr/lib/systemd/systemd-logind
root         54  0.0  0.0   6452   832 tty1     Ss+  21:54   0:00 /sbin/agetty --noclear tty1 linux

[root@e7d1cd8dafcf /]# ls /run/secrets
etc-pki-entitlement  rhel7.repo  rhsm
[root@e7d1cd8dafcf /]# ls /etc/pki/entitlement-host/
7681441574793056919-key.pem  7681441574793056919.pem
[root@e7d1cd8dafcf /]# journalctl -r|grep error     
Feb 25 22:14:53 e7d1cd8dafcf systemd-udevd[29]: error: /dev/dm-5: No such file or directory
Feb 25 22:14:53 e7d1cd8dafcf systemd-udevd[29]: error: /dev/dm-5: No such file or directory
Feb 25 22:14:53 e7d1cd8dafcf systemd-udevd[29]: error: /dev/dm-5: No such file or directory
Feb 25 22:01:22 e7d1cd8dafcf systemd-udevd[29]: error: /dev/dm-5: No such file or directory
Feb 25 22:01:22 e7d1cd8dafcf systemd-udevd[29]: error: /dev/dm-5: No such file or directory
Feb 25 22:01:22 e7d1cd8dafcf systemd-udevd[29]: error: /dev/dm-5: No such file or directory
Feb 25 21:59:59 e7d1cd8dafcf systemd-udevd[29]: error: /dev/dm-5: No such file or directory
Feb 25 21:59:59 e7d1cd8dafcf systemd-udevd[29]: error: /dev/dm-5: No such file or directory
Feb 25 21:59:59 e7d1cd8dafcf systemd-udevd[29]: error: /dev/dm-5: No such file or directory
Feb 25 21:59:35 e7d1cd8dafcf systemd-udevd[29]: error: /dev/dm-5: No such file or directory
Feb 25 21:59:35 e7d1cd8dafcf systemd-udevd[29]: error: /dev/dm-5: No such file or directory
Feb 25 21:59:34 e7d1cd8dafcf systemd-udevd[29]: error: /dev/dm-5: No such file or directory
Feb 25 21:59:12 e7d1cd8dafcf systemd-udevd[29]: error: /dev/dm-5: No such file or directory
Feb 25 21:59:11 e7d1cd8dafcf systemd-udevd[29]: error: /dev/dm-5: No such file or directory
Feb 25 21:59:11 e7d1cd8dafcf systemd-udevd[29]: error: /dev/dm-5: No such file or directory
Feb 25 21:58:59 e7d1cd8dafcf systemd-udevd[29]: error: /dev/dm-5: No such file or directory
Feb 25 21:58:59 e7d1cd8dafcf systemd-udevd[29]: error: /dev/dm-5: No such file or directory
Feb 25 21:58:59 e7d1cd8dafcf systemd-udevd[29]: error: /dev/dm-5: No such file or directory
Feb 25 21:58:06 e7d1cd8dafcf systemd-udevd[29]: error: /dev/dm-5: No such file or directory
Feb 25 21:58:05 e7d1cd8dafcf systemd-udevd[29]: error: /dev/dm-5: No such file or directory
Feb 25 21:58:05 e7d1cd8dafcf systemd-udevd[29]: error: /dev/dm-5: No such file or directory
Feb 25 21:57:32 e7d1cd8dafcf systemd-udevd[29]: error: /dev/dm-5: No such file or directory
Feb 25 21:57:32 e7d1cd8dafcf systemd-udevd[29]: error: /dev/dm-5: No such file or directory
Feb 25 21:57:32 e7d1cd8dafcf systemd-udevd[29]: error: /dev/dm-5: No such file or directory
Feb 25 21:54:24 e7d1cd8dafcf systemd-vconsole-setup[105]: /usr/bin/loadkeys failed with error code 1.
Feb 25 21:54:24 e7d1cd8dafcf systemd-vconsole-setup[104]: /usr/bin/loadkeys failed with error code 1.
Feb 25 21:54:24 e7d1cd8dafcf systemd-vconsole-setup[105]: /usr/bin/setfont failed with error code 1.
Feb 25 21:54:24 e7d1cd8dafcf systemd-vconsole-setup[104]: /usr/bin/setfont failed with error code 1.
Feb 25 21:54:24 e7d1cd8dafcf systemd-vconsole-setup[19]: /usr/bin/loadkeys failed with error code 1.
Feb 25 21:54:24 e7d1cd8dafcf systemd-vconsole-setup[19]: /usr/bin/setfont failed with error code 1.
Feb 25 21:54:24 e7d1cd8dafcf systemd[1]:                                          Set default log error output for services
Feb 25 21:54:24 e7d1cd8dafcf systemd[1]: systemd.default_standard_error=null|tty|syslog|syslog+console|kmsg|kmsg+console|journal|journal+console
Feb 25 21:54:24 e7d1cd8dafcf kernel: ioapic: probe of 0000:80:05.4 failed with error -22
Feb 25 21:54:24 e7d1cd8dafcf kernel: ioapic: probe of 0000:00:05.4 failed with error -22
Comment 6 Humble Chirammal 2016-02-26 04:48:16 EST
(In reply to Frantisek Kluknavsky from comment #4)
> Hi,
> 
> could you please try to run `journalctl -r` and `ps afxu` in the container?
> I suspect that systemd does not run normally.
> 
> I am not prepared to explain how to run systemd in docker but as a hint,
> `docker run -v /run -v /sys/fs/cgroup:/sys/fs/cgroup -d rhel7
> /usr/sbin/init` might help you.

We had tried it as an isolation step where /run was exported as -v /run:/run:ro and got some errors. If we bind mount as 'rw', the docker deamon is misbehaving.
Comment 7 Frantisek Kluknavsky 2016-02-26 10:36:10 EST
(In reply to Humble Chirammal from comment #6)
> (In reply to Frantisek Kluknavsky from comment #4)
> > Hi,
> > 
> > could you please try to run `journalctl -r` and `ps afxu` in the container?
> > I suspect that systemd does not run normally.
> > 
> > I am not prepared to explain how to run systemd in docker but as a hint,
> > `docker run -v /run -v /sys/fs/cgroup:/sys/fs/cgroup -d rhel7
> > /usr/sbin/init` might help you.
> 
> We had tried it as an isolation step where /run was exported as -v
> /run:/run:ro and got some errors. If we bind mount as 'rw', the docker
> deamon is misbehaving.

/run certainly should not be read-only, its purpose is to store volatile data. Also, you probably do not want to bind mount host's /run into container, mixing data from the two together in unpredictable ways. Try `-v /run`, but not `-v /run:/run`
Comment 8 Frantisek Kluknavsky 2016-02-26 10:47:54 EST
(In reply to Alex Jia from comment #5)
> (In reply to Frantisek Kluknavsky from comment #4)
> > `docker run -v /run -v /sys/fs/cgroup:/sys/fs/cgroup -d rhel7
> > /usr/sbin/init` might help you.
> 
> I gave a try on docker-1.9.1-16.el7.x86_64 w/ and w/o --privileged option
> per above cmdline, the rhel7 image id is 6c3a84d798dc, it looks fine for me,
> I both can find /run/secrets/ directory. 
> 
> But w/ --privileged option, I found extra some error information in log, it
> should be irrelevant w/ the bug, I just did a record in here, the details as
> follows.
> 
> 
> [root@dell-per630-02 ~]# docker run -v /run -v /sys/fs/cgroup:/sys/fs/cgroup
> -d --privileged rhel7 /usr/sbin/init
> e7d1cd8dafcfdb53a8163fbb46111e62223e46fc0b4c3b18184ab372b46cec5f
> Usage of loopback devices is strongly discouraged for production use. Either
> use `--storage-opt dm.thinpooldev` or use `--storage-opt
> dm.no_warn_on_loop_devices=true` to suppress this warning.
> 
> [root@dell-per630-02 ~]# docker exec -it e7d1cd8dafcf /bin/bash
> [root@e7d1cd8dafcf /]# ps auxf
> USER        PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> root        111  0.7  0.0  11768  1768 ?        Ss   21:54   0:00 /bin/bash
> root        123  0.0  0.0  35884  1444 ?        R+   21:54   0:00  \_ ps auxf
> root          1  0.5  0.0  40744  3100 ?        Ss   21:54   0:00
> /usr/sbin/init
> root         17  0.3  0.0  36816  6428 ?        Ss   21:54   0:00
> /usr/lib/systemd/systemd-journald
> root         29  0.9  0.0  41776  2120 ?        Ss   21:54   0:00
> /usr/lib/systemd/systemd-udevd
> dbus         37  0.0  0.0  26460  1484 ?        Ss   21:54   0:00
> /bin/dbus-daemon --system --address=systemd: --nofork --nopidfile
> --systemd-activation
> root         38  0.0  0.0   9756   620 ?        Ss   21:54   0:00
> /usr/bin/rhsmcertd
> root         52  0.0  0.0  24272  1516 ?        Ss   21:54   0:00
> /usr/lib/systemd/systemd-logind
> root         54  0.0  0.0   6452   832 tty1     Ss+  21:54   0:00
> /sbin/agetty --noclear tty1 linux
> 
> [root@e7d1cd8dafcf /]# ls /run/secrets
> etc-pki-entitlement  rhel7.repo  rhsm
> [root@e7d1cd8dafcf /]# ls /etc/pki/entitlement-host/
> 7681441574793056919-key.pem  7681441574793056919.pem
> [root@e7d1cd8dafcf /]# journalctl -r|grep error     
> Feb 25 22:14:53 e7d1cd8dafcf systemd-udevd[29]: error: /dev/dm-5: No such
> file or directory
> ...

I probably can not help it as a maintainer of the image, the right thing to do might be to file a bug against systemd (or docker?).
Comment 9 Mohamed Ashiq 2016-05-03 05:35:35 EDT
(In reply to Humble Chirammal from comment #6)
> (In reply to Frantisek Kluknavsky from comment #4)
> > Hi,
> > 
> > could you please try to run `journalctl -r` and `ps afxu` in the container?
> > I suspect that systemd does not run normally.
> > 
> > I am not prepared to explain how to run systemd in docker but as a hint,
> > `docker run -v /run -v /sys/fs/cgroup:/sys/fs/cgroup -d rhel7
> > /usr/sbin/init` might help you.
> 
> We had tried it as an isolation step where /run was exported as -v
> /run:/run:ro and got some errors. If we bind mount as 'rw', the docker
> deamon is misbehaving.

This seems to be working for our Image. Thanks.
Comment 10 Daniel Walsh 2016-08-19 18:03:25 EDT
With docker-latest-1.12

You should be able to run systemd in a locked down container.

Note You need to log in before you can comment on or make changes to this bug.