Bug 1384472 - /var/lock/iscsi some cases has wrong selinux label
Summary: /var/lock/iscsi some cases has wrong selinux label
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-selinux
Version: 10.0 (Newton)
Hardware: Unspecified
OS: Unspecified
urgent
unspecified
Target Milestone: rc
: 10.0 (Newton)
Assignee: Lon Hohberger
QA Contact: lkuchlan
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-10-13 10:58 UTC by Attila Fazekas
Modified: 2016-12-14 16:18 UTC (History)
6 users (show)

Fixed In Version: openstack-selinux-0.7.12-1.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-12-14 16:18:41 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
Strace output during command (Gzipped (1.12 MB, application/octet-stream)
2016-11-01 20:59 UTC, Lon Hohberger
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2016:2948 0 normal SHIPPED_LIVE Red Hat OpenStack Platform 10 enhancement update 2016-12-14 19:55:27 UTC

Description Attila Fazekas 2016-10-13 10:58:49 UTC
Description of problem:
/var/lock/iscsi is missing, for unknown reason same cases I get that directory with var_lock_t instead of iscsi_lock_t at openstack runs.

The package installation itself should ensure the directory exists with the correct selinux label.


Version-Release number of selected component (if applicable):
iscsi-initiator-utils-iscsiuio.x86_64 0:6.2.0.873-35.el7 

How reproducible:
With above yum command, always.
In system it is more difficult to see when it gets wrong.


Actual results:
Package does not install the  /var/lock/iscsi , even when it is installed via the
sudo "yum install /var/lock/iscsi ".

Expected results:
/var/lock/iscsi  is directory.
sudo ls -lZd /var/lock/iscsi/
drw-------. root root system_u:object_r:iscsi_lock_t:s0 /var/lock/iscsi/

Additional info:
[root@6tdig5aio-1 cloud-user]# sudo yum install /var/lock/iscsi
Loaded plugins: search-disabled-repos
rhelosp-10.0-ceph-2.0-mon/filelists_db                                                                                                                                                      | 120 kB  00:00:00     
rhelosp-10.0-ceph-2.0-osd/filelists_db                                                                                                                                                      |  13 kB  00:00:00     
rhelosp-10.0-ceph-2.0-tools/filelists_db                                                                                                                                                    |  15 kB  00:00:00     
rhelosp-10.0-devtools-puddle/x86_64/filelists                                                                                                                                               | 2.4 kB  00:00:00     
rhelosp-10.0-fast-datapth/filelists_db                                                                                                                                                      |  15 kB  00:00:00     
rhelosp-10.0-puddle/x86_64/filelists                                                                                                                                                        | 728 kB  00:00:00     
rhelosp-rhel-7-common/x86_64/filelists_db                                                                                                                                                   |  89 kB  00:00:00     
rhelosp-rhel-7.3-extras/x86_64/filelists_db                                                                                                                                                 | 193 kB  00:00:00     
rhelosp-rhel-7.3-ha/x86_64/filelists_db                                                                                                                                                     |  49 kB  00:00:00     
rhelosp-rhel-7.3-server/x86_64/filelists_db                                                                                                                                                 | 3.3 MB  00:00:00     
rhos-release/filelists_db                                                                                                                                                                   |  21 kB  00:00:00     
rhos-release-extras/7Server/filelists_db                                                                                                                                                    | 2.6 kB  00:00:00     
Resolving Dependencies
--> Running transaction check
---> Package iscsi-initiator-utils.x86_64 0:6.2.0.873-35.el7 will be installed
--> Processing Dependency: iscsi-initiator-utils-iscsiuio >= 6.2.0.873-35.el7 for package: iscsi-initiator-utils-6.2.0.873-35.el7.x86_64
--> Running transaction check
---> Package iscsi-initiator-utils-iscsiuio.x86_64 0:6.2.0.873-35.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

===================================================================================================================================================================================================================
 Package                                                       Arch                                  Version                                          Repository                                              Size
===================================================================================================================================================================================================================
Installing:
 iscsi-initiator-utils                                         x86_64                                6.2.0.873-35.el7                                 rhelosp-rhel-7.3-server                                417 k
Installing for dependencies:
 iscsi-initiator-utils-iscsiuio                                x86_64                                6.2.0.873-35.el7                                 rhelosp-rhel-7.3-server                                 85 k

Transaction Summary
===================================================================================================================================================================================================================
Install  1 Package (+1 Dependent package)

Total download size: 502 k
Installed size: 2.5 M
Is this ok [y/d/N]: y
Downloading packages:
(1/2): iscsi-initiator-utils-6.2.0.873-35.el7.x86_64.rpm                                                                                                                                    | 417 kB  00:00:00     
(2/2): iscsi-initiator-utils-iscsiuio-6.2.0.873-35.el7.x86_64.rpm                                                                                                                           |  85 kB  00:00:00     
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                                                              3.0 MB/s | 502 kB  00:00:00     
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : iscsi-initiator-utils-6.2.0.873-35.el7.x86_64                                                                                                                                                   1/2 
  Installing : iscsi-initiator-utils-iscsiuio-6.2.0.873-35.el7.x86_64                                                                                                                                          2/2 
  Verifying  : iscsi-initiator-utils-iscsiuio-6.2.0.873-35.el7.x86_64                                                                                                                                          1/2 
  Verifying  : iscsi-initiator-utils-6.2.0.873-35.el7.x86_64                                                                                                                                                   2/2 

Installed:
  iscsi-initiator-utils.x86_64 0:6.2.0.873-35.el7                                                                                                                                                                  

Dependency Installed:
  iscsi-initiator-utils-iscsiuio.x86_64 0:6.2.0.873-35.el7                                                                                                                                                         

Complete!
[root@6tdig5aio-1 cloud-user]# ls -l /var/lock/iscsi
ls: cannot access /var/lock/iscsi: No such file or directory




type=AVC msg=audit(1476290296.164:25487): avc:  denied  { read write } for  pid=11595 comm="iscsiadm" name="lock" dev="tmpfs" ino=182951 scontext=system_u:system_r:iscsid_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lock_t:s0 tclass=file
type=SYSCALL msg=audit(1476290296.164:25487): arch=c000003e syscall=2 success=no exit=-13 a0=7faca8eb0397 a1=42 a2=1b6 a3=45 items=0 ppid=11714 pid=11595 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="iscsiadm" exe="/usr/sbin/iscsiadm" subj=system_u:system_r:iscsid_t:s0-s0:c0.c1023 key=(null)
type=AVC msg=audit(1476290296.164:25488): avc:  denied  { link } for  pid=11595 comm="iscsiadm" name="lock" dev="tmpfs" ino=182951 scontext=system_u:system_r:iscsid_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_lock_t:s0 tclass=file

Comment 2 Attila Fazekas 2016-10-18 12:52:43 UTC
If the 'Expected results'  is not doable (install the /var/lock/iscsi), please reassign this to openstack-selinux.

Comment 4 Attila Fazekas 2016-10-18 14:44:18 UTC
/var/lock/iscsi/ is on a tmpfs the package cannot fix this, moving to openstack-selinux.

Comment 5 Alan Pevec 2016-10-18 15:03:32 UTC
/var/lock/iscsi is a %ghost file, owned by the package but not installed, because /var/lock is tmpfs, introduced in i-i-utils in bug 656605.
File should be still created with the correct selinux context, let's move to openstack-selinux to investigate further.

Comment 6 Attila Fazekas 2016-10-18 15:11:11 UTC
The issue does not happens in tripleo deployed system, (at least not seen so far), it happens with packstack deployed one. The two config may differ for ex. in rootwrap settings.

Comment 7 Ryan Hallisey 2016-10-19 13:13:41 UTC
can a restorecon be run after that package is installed with packstack?

Comment 8 Attila Fazekas 2016-10-20 13:04:32 UTC
The file does not exists just after installation, restorecon has nothing to change.

[cloud-user@n471jnaio-1 ~]$ sudo restorecon -R -v /var/lock
restorecon:  Warning no default label for /run/lock/subsys
restorecon:  Warning no default label for /run/lock/subsys/rhnsd
restorecon:  Warning no default label for /run/lock/subsys/network
[cloud-user@n471jnaio-1 ~]$ ls -lZ /var/lock
lrwxrwxrwx. root root system_u:object_r:var_lock_t:s0  /var/lock -> ../run/lock
[cloud-user@n471jnaio-1 ~]$ ls -lZ /var/lock/
-rw-r--r--. root root system_u:object_r:kdump_lock_t:s0 kdump
drwxrwxr-x. root lock system_u:object_r:lockdev_lock_t:s0 lockdev
drwx------. root root system_u:object_r:lvm_lock_t:s0  lvm
drwxr-xr-x. root root system_u:object_r:pppd_lock_t:s0 ppp
drwxr-xr-x. root root system_u:object_r:var_lock_t:s0  subsys

Comment 9 Attila Fazekas 2016-10-20 16:34:41 UTC
I have two CI job, both expected to do the same setup, one runs less tempest test -> no selinux issue.
The other runs more, we have selinux issue.

So far failed to connect the issue to particular test case.

sudo systemd-tmpfiles --create # also works as expected.

What else can touch these files ?
What else can be in the background ?

I'll try to use some tracer to find out what happening ..

Comment 10 Attila Fazekas 2016-10-21 06:07:27 UTC
$ sudo rm -Rf /var/lock/iscsi/
$ ostestr -r tempest.api.volume.admin.test_volumes_backup.VolumesBackupsAdminV1Test # passes
$ sudo ls -lZ /var/lock/iscsi/
-rw-------. root root system_u:object_r:var_lock_t:s0  lock
# WRONG

after the above test the  `ostestr -r minimum` # fails,  lock access rejected by selinux.

If the ostestr -r minimum would run first, the label would be correct, and even the tempest.api.volume.admin.test_volumes_backup.VolumesBackupsAdminV1Test does not makes it wrong.


IMHO even if the lock file is ghost and on tmpfs , the package could have create it at install time (post script), instead of waiting for the next reboot/systemd-tmpfiles to do so.


In the tripleo setups we have the nova-compute and the cinder-backup on different nodes so they cannot cause harm to each other, also the  iscsi-initiator-utils-iscsiuio is part of the base image, so the lock file will be there at boot time.

It is very likely the cinder-bakup service initiated the wrong lock file creation.


You can have a wrongly labeled lock file by:
$ sudo rm -Rf /var/lock/iscsi/ ; cinder backup-create `cinder create 1 |awk '/ id /{print $4}'` ; sleep 15; sudo ls -lZ  /var/lock/iscsi/
+-----------+--------------------------------------+
| Property  | Value                                |
+-----------+--------------------------------------+
| id        | c067f661-cb4c-44a0-9b5d-9f69768ecfe7 |
| name      | None                                 |
| volume_id | 4cf6980d-75bd-4a1d-831f-e8312e70d784 |
+-----------+--------------------------------------+
-rw-------. root root system_u:object_r:var_lock_t:s0  lock

Comment 11 Attila Fazekas 2016-10-21 07:41:39 UTC
"/var/lock/iscsi" is created by a command like 
"sudo /sbin/iscsiadm iscsiadm -m node -T iqn.2010-10.org.openstack:volume-40837a86-4521-4399-b28d-359e2329f7a9 -p 192.168.1.13:3260"
execute by the "/usr/bin/python /bin/privsep-helper --config-file /usr/share/cinder/cinder-dist.conf --config-file /etc/cinder/cinder.conf --privsep_context os_brick.privileged.default --privsep_sock_path /tmp/tmp9D1mmy/privsep.sock" which has system_u:system_r:cinder_backup_t:s0 conext .

(The privsep-helper and cinder-backup using a socket for communication.)


If I execute the same command myself (cloud-user logged in via ssh), it creates the right lock file.

$ sudo  rm -Rf /var/lock/iscsi/
$ sudo /sbin/iscsiadm iscsiadm -m node -T iqn.2010-10.org.openstack:volume-40837a86-4521-4399-b28d-359e2329f7a9 -p 192.168.1.13:3260
iscsiadm: No records found
$ sudo  ls -lZ /var/lock/iscsi/
-rw-------. root root unconfined_u:object_r:iscsi_lock_t:s0 lock
$ 


Looks like the lock file is only wrong when it is created by an iscsiadm command which is executed by the privsep-helper which is under the cinder_backup_t.

Comment 13 Lon Hohberger 2016-11-01 20:59:10 UTC
Created attachment 1216245 [details]
Strace output during command (Gzipped

It's possible a writing a domain transition from cinder_backup_t to iscsi_exec_t will resolve this.

Comment 14 Lon Hohberger 2016-11-02 12:48:17 UTC
Writing a domain transition resolves this.

Comment 15 Lon Hohberger 2016-11-02 13:01:20 UTC
[cloud-user@9sdolxaio-1 ~(keystone_admin)]$ sudo rm -Rf /var/lock/iscsi/ ; cinder backup-create `cinder create 1 |awk '/ id /{print $4}'` ; sleep 15; sudo ls -lZ  /var/lock/iscsi/
+-----------+--------------------------------------+
| Property  | Value                                |
+-----------+--------------------------------------+
| id        | 980ac436-6bc1-496e-859d-eca70021d44b |
| name      | None                                 |
| volume_id | fc787e59-023d-489b-b74f-2fb55b56ebbc |
+-----------+--------------------------------------+
-rw-------. root root system_u:object_r:var_lock_t:s0  lock
[cloud-user@9sdolxaio-1 ~(keystone_admin)]$ sudo su
[root@9sdolxaio-1 cloud-user(keystone_admin)]# rpm -Uvh openstack-selinux-0.7.11-3.el7ost.noarch.rpm
Preparing...                          ################################# [100%]
Updating / installing...
   1:openstack-selinux-0.7.11-3.el7ost################################# [ 50%]
Full path required for exclude: net:[4026532245].
Full path required for exclude: net:[4026532245].
Full path required for exclude: net:[4026532339].
Full path required for exclude: net:[4026532339].
Full path required for exclude: net:[4026532245].
Full path required for exclude: net:[4026532245].
Full path required for exclude: net:[4026532339].
Full path required for exclude: net:[4026532339].
Full path required for exclude: net:[4026532245].
Full path required for exclude: net:[4026532245].
Full path required for exclude: net:[4026532339].
Full path required for exclude: net:[4026532339].
Full path required for exclude: net:[4026532245].
Full path required for exclude: net:[4026532245].
Full path required for exclude: net:[4026532339].
Full path required for exclude: net:[4026532339].
Full path required for exclude: net:[4026532245].
Full path required for exclude: net:[4026532245].
Full path required for exclude: net:[4026532339].
Full path required for exclude: net:[4026532339].
Cleaning up / removing...
   2:openstack-selinux-0.7.11-1.el7ost################################# [100%]
[root@9sdolxaio-1 cloud-user(keystone_admin)]# exit
[cloud-user@9sdolxaio-1 ~(keystone_admin)]$ sudo rm -Rf /var/lock/iscsi/ ; cinder backup-create `cinder create 1 |awk '/ id /{print $4}'` ; sleep 15; sudo ls -lZ  /var/lock/iscsi/
+-----------+--------------------------------------+
| Property  | Value                                |
+-----------+--------------------------------------+
| id        | 75701d1d-1f5b-49e1-9c33-5317c36ebffa |
| name      | None                                 |
| volume_id | 1e88c1e0-a357-4766-b1e3-999e1fef2334 |
+-----------+--------------------------------------+
-rw-------. root root system_u:object_r:iscsi_lock_t:s0 lock

Comment 17 lkuchlan 2016-11-08 09:11:40 UTC
Tested using:
openstack-selinux-0.7.12-1.el7ost.noarch

Verification flow:
[stack@undercloud-0 ~]$ sudo ls -ldZ  /var/lock/iscsi/
drw-------. root root system_u:object_r:var_lock_t:s0  /var/lock/iscsi/

Comment 19 errata-xmlrpc 2016-12-14 16:18:41 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-2948.html


Note You need to log in before you can comment on or make changes to this bug.