Some other info for your reference.
1. This can not be reproduce while disable the selinux.
# getenforce 
Permissive

2. the selinux label will be changed after start the 2nd guest.
2.1 original selinux label of the shareable sg device. 
# ll -Z /dev/sg0
crw-rw----. root root system_u:object_r:scsi_generic_device_t:s0 /dev/sg0

2.2 start 1st guest, check the shareable sg device selinux label.
# virsh start rhel7-2
Domain rhel7-2 started

# ll -Z /dev/sg0
crw-rw----. qemu qemu system_u:object_r:svirt_image_t:s0:c727,c793 /dev/sg0

2.3 start 2nd guest, the selinux label is changed.
# virsh start rhel7-xuzhang
Domain rhel7-xuzhang started

# ll -Z /dev/sg0
crw-rw----. qemu qemu system_u:object_r:svirt_image_t:s0:c219,c388 /dev/sg0

(In reply to Zhang Xuesong from comment #1)
> Some other info for your reference.
> 1. This can not be reproduce while disable the selinux.

Correct one typo, not "disable the selinux", it should be "permissive mode"

> # getenforce 
> Permissive
> 
> 2. the selinux label will be changed after start the 2nd guest.
> 2.1 original selinux label of the shareable sg device. 
> # ll -Z /dev/sg0
> crw-rw----. root root system_u:object_r:scsi_generic_device_t:s0 /dev/sg0
> 
> 2.2 start 1st guest, check the shareable sg device selinux label.
> # virsh start rhel7-2
> Domain rhel7-2 started
> 
> # ll -Z /dev/sg0
> crw-rw----. qemu qemu system_u:object_r:svirt_image_t:s0:c727,c793 /dev/sg0
> 
> 2.3 start 2nd guest, the selinux label is changed.
> # virsh start rhel7-xuzhang
> Domain rhel7-xuzhang started
> 
> # ll -Z /dev/sg0
> crw-rw----. qemu qemu system_u:object_r:svirt_image_t:s0:c219,c388 /dev/sg0

In order to help more quickly reproduce, set things up, etc. - providing a few more details on the configuration would be helpful.  The following:

Steps to Reproduce:
1. make sure the sg module is probed.
# lsmod|grep sg
sg                     36533  0 

2. Add the shareable sg device in 2 shutoff guests.
......
    <hostdev mode='subsystem' type='scsi' managed='no'>
      <source>
        <adapter name='scsi_host0'/>
        <address bus='0' target='0' unit='0'/>
      </source>
      <shareable>
    </hostdev>
......


Is very vague.  Missing is what you did to make sure 'sg' is loaded for lsmod to find it, how you went about creating the shareable device that 'scsi_host0' is using. Is it an iSCSI device? Is it a /dev/sdxn (eg, /dev/sda3)?

Also I assume by the 'hostdev' XML that this only fails for host devices?  Do you know if it works when the 'shared' disk is defined within a <disk...> element?   Being 7 months after first reported - I have to wonder if this works now as there have been many changes. Getting a similar configuration knowing details of how you set things up just makes it easier for me to recreate.

(In reply to John Ferlan from comment #3)
> In order to help more quickly reproduce, set things up, etc. - providing a
> few more details on the configuration would be helpful.  The following:
> 
> Steps to Reproduce:
> 1. make sure the sg module is probed.
> # lsmod|grep sg
> sg                     36533  0 
> 
> 2. Add the shareable sg device in 2 shutoff guests.
> ......
>     <hostdev mode='subsystem' type='scsi' managed='no'>
>       <source>
>         <adapter name='scsi_host0'/>
>         <address bus='0' target='0' unit='0'/>
>       </source>
>       <shareable>
>     </hostdev>
> ......
> 
> 
> Is very vague.  Missing is what you did to make sure 'sg' is loaded for
> lsmod to find it, how you went about creating the shareable device that
> 'scsi_host0' is using. Is it an iSCSI device? Is it a /dev/sdxn (eg,
> /dev/sda3)?

Here is the details prepared steps for your reference:
1. probe the sg module
# modprobe sg
# lsmod |grep sg
sg                     40721  0 

2. checking the sg devices on my host, '/dev/sda' is my boot disk, the '/dev/sdb' is my data disk. So, I'd like to use the data disk "/dev/sg2" for the shared sg device in the following steps. 
# lsscsi -g
[0:0:0:0]    disk    ATA      WDC WD5000AAKS-7 05.0  /dev/sda   /dev/sg0 
[1:0:0:0]    cd/dvd  PLDS     DVD+-RW DH-16AAS JD12  /dev/sr0   /dev/sg1 
[2:0:0:0]    disk    ATA      WDC WD3200AAKS-7 01.0  /dev/sdb   /dev/sg2 
[7:0:0:0]    storage IET      Controller       0001  -          /dev/sg3 
[7:0:0:1]    disk    IET      VIRTUAL-DISK     0001  /dev/sdd   /dev/sg4 

3. checking the adapter name of sg device /dev/sg2, it's "scsi_host2".
# virsh nodedev-list --tree
......
  |   +- scsi_host2
  |   |   |
  |   |   +- scsi_target2_0_0
  |   |       |
  |   |       +- scsi_2_0_0_0
  |   |           |
  |   |           +- block_sdb_WDC_WD3200AAKS_75L9A0_WD_WMAV26572317
  |   |           +- scsi_generic_sg2
......

4. add the following shareable sg disk to 2 shutoff guests.
     <hostdev mode='subsystem' type='scsi' managed='no'>
       <source>
         <adapter name='scsi_host0'/>
         <address bus='0' target='0' unit='0'/>
       </source>
       <shareable/>
     </hostdev>


So, the following steps, you can reference the bug description.

> 
> Also I assume by the 'hostdev' XML that this only fails for host devices? 
> Do you know if it works when the 'shared' disk is defined within a <disk...>

The shared disk is working well with the following disk element:
<disk type='block' device='disk'>
      <driver name='qemu' type='raw' cache='none' io='native'/>
      <source dev='/dev/sdb'/>
      <backingStore/>
      <target dev='hdb' bus='ide'/>
      <shareable/>
      <alias name='ide0-0-1'/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>


> element?   Being 7 months after first reported - I have to wonder if this
> works now as there have been many changes. Getting a similar configuration

I tried just now, this issue can be produced with the latest libvirt + qemu-kvm-rhev + kernel.

> knowing details of how you set things up just makes it easier for me to
> recreate.

(In reply to Zhang Xuesong from comment #4)
> (In reply to John Ferlan from comment #3)
> > In order to help more quickly reproduce, set things up, etc. - providing a
> > few more details on the configuration would be helpful.  The following:
> > 
> > Steps to Reproduce:
> > 1. make sure the sg module is probed.
> > # lsmod|grep sg
> > sg                     36533  0 
> > 
> > 2. Add the shareable sg device in 2 shutoff guests.
> > ......
> >     <hostdev mode='subsystem' type='scsi' managed='no'>
> >       <source>
> >         <adapter name='scsi_host0'/>

Sorry, the adapter name should be the following one.
<adapter name='scsi_host2'/>

> >         <address bus='0' target='0' unit='0'/>
> >       </source>
> >       <shareable>
> >     </hostdev>
> > ......
> > 
> > 
> > Is very vague.  Missing is what you did to make sure 'sg' is loaded for
> > lsmod to find it, how you went about creating the shareable device that
> > 'scsi_host0' is using. Is it an iSCSI device? Is it a /dev/sdxn (eg,
> > /dev/sda3)?
> 
> Here is the details prepared steps for your reference:
> 1. probe the sg module
> # modprobe sg
> # lsmod |grep sg
> sg                     40721  0 
> 
> 2. checking the sg devices on my host, '/dev/sda' is my boot disk, the
> '/dev/sdb' is my data disk. So, I'd like to use the data disk "/dev/sg2" for
> the shared sg device in the following steps. 
> # lsscsi -g
> [0:0:0:0]    disk    ATA      WDC WD5000AAKS-7 05.0  /dev/sda   /dev/sg0 
> [1:0:0:0]    cd/dvd  PLDS     DVD+-RW DH-16AAS JD12  /dev/sr0   /dev/sg1 
> [2:0:0:0]    disk    ATA      WDC WD3200AAKS-7 01.0  /dev/sdb   /dev/sg2 
> [7:0:0:0]    storage IET      Controller       0001  -          /dev/sg3 
> [7:0:0:1]    disk    IET      VIRTUAL-DISK     0001  /dev/sdd   /dev/sg4 
> 
> 3. checking the adapter name of sg device /dev/sg2, it's "scsi_host2".
> # virsh nodedev-list --tree
> ......
>   |   +- scsi_host2
>   |   |   |
>   |   |   +- scsi_target2_0_0
>   |   |       |
>   |   |       +- scsi_2_0_0_0
>   |   |           |
>   |   |           +- block_sdb_WDC_WD3200AAKS_75L9A0_WD_WMAV26572317
>   |   |           +- scsi_generic_sg2
> ......
> 
> 4. add the following shareable sg disk to 2 shutoff guests.
>      <hostdev mode='subsystem' type='scsi' managed='no'>
>        <source>
>          <adapter name='scsi_host0'/>
>          <address bus='0' target='0' unit='0'/>
>        </source>
>        <shareable/>
>      </hostdev>
> 
> 
> So, the following steps, you can reference the bug description.
> 
> > 
> > Also I assume by the 'hostdev' XML that this only fails for host devices? 
> > Do you know if it works when the 'shared' disk is defined within a <disk...>
> 
> The shared disk is working well with the following disk element:
> <disk type='block' device='disk'>
>       <driver name='qemu' type='raw' cache='none' io='native'/>
>       <source dev='/dev/sdb'/>
>       <backingStore/>
>       <target dev='hdb' bus='ide'/>
>       <shareable/>
>       <alias name='ide0-0-1'/>
>       <address type='drive' controller='0' bus='0' target='0' unit='1'/>
>     </disk>
> 
> 
> > element?   Being 7 months after first reported - I have to wonder if this
> > works now as there have been many changes. Getting a similar configuration
> 
> I tried just now, this issue can be produced with the latest libvirt +
> qemu-kvm-rhev + kernel.
> 
> > knowing details of how you set things up just makes it easier for me to
> > recreate.

OK - great thanks for the details and ensuring that the <disk> works just fine.  That narrows the frame of reference for the issue to just when a disk/lun is being used for hostdev passthru. 

Of course I don't have a second physical disk, but that shouldn't matter since this is more of a hostdev for sg type issue...

(In reply to John Ferlan from comment #6)
> OK - great thanks for the details and ensuring that the <disk> works just
> fine.  That narrows the frame of reference for the issue to just when a
> disk/lun is being used for hostdev passthru. 
> 
> Of course I don't have a second physical disk, but that shouldn't matter
> since this is more of a hostdev for sg type issue...

I think you can use the USB for the second physical disk, it's also ok. Such as the the "/dev/sg3" in following example, it is one USB device on my host.

# lsscsi -g
[0:0:0:0]    disk    ATA      ST3250310AS      3.AD  /dev/sda   /dev/sg0 
[1:0:0:0]    cd/dvd  PLDS     DVD+-RW DH-16AAS JD12  /dev/sr0   /dev/sg1 
[2:0:0:0]    disk    ATA      WDC WD5000AAKS-7 05.0  /dev/sdb   /dev/sg2 
[6:0:0:0]    disk    SanDisk  Cruzer Force     1.26  /dev/sdc   /dev/sg3

I have the following:

# lsscsi -g
[0:0:0:0]    disk    ATA      HGST HTS725032A7 B550  /dev/sda   /dev/sg0 
[1:0:0:0]    cd/dvd  PLDS     DVD-RW DS8A8SH   KU51  /dev/sr0   /dev/sg1 
[7:0:0:0]    storage IET      Controller       0001  -          /dev/sg2 
[7:0:0:1]    disk    IET      VIRTUAL-DISK     0001  /dev/sdb   /dev/sg5 
[7:0:0:2]    disk    IET      VIRTUAL-DISK     0001  /dev/sde   /dev/sg8 
[10:0:0:0]   storage IET      Controller       0001  -          /dev/sg3 
[10:0:0:1]   disk    IET      VIRTUAL-DISK     0001  /dev/sdc   /dev/sg6 
[10:0:0:2]   disk    IET      VIRTUAL-DISK     0001  /dev/sdg   /dev/sg10
[11:0:0:0]   storage IET      Controller       0001  -          /dev/sg4 
[11:0:0:1]   disk    IET      VIRTUAL-DISK     0001  /dev/sdd   /dev/sg7 
[11:0:0:2]   disk    IET      VIRTUAL-DISK     0001  /dev/sdf   /dev/sg9 


What I did to test was "use" one of the iSCSI virtual disks that I created, but created a scsi hostdev to reference it

    <hostdev mode='subsystem' type='scsi' managed='no'>
      <source>
        <adapter name='scsi_host7'/>
        <address bus='0' target='0' unit='1'/>
      </source>
      <shareable/>
    </hostdev>


I didn't see the same issue as described in this bug with "Permissive" mode, but if I go with "Enforcing", then I saw the issue.

I've created a potential fix which keeps the volume visible for both guests, but I'm troubled with another aspect over the contents of the volume. Creating files has some odd interactions, but I think that just could be because the backend is an iSCSI rather than what it "should" be. I didn't restart iSCSI server in between changing selinux enforcement...  One thing at a time I guess.

I'll be putting together and posting a patch today...

(In reply to John Ferlan from comment #8)
> I have the following:
> 
> # lsscsi -g
> [0:0:0:0]    disk    ATA      HGST HTS725032A7 B550  /dev/sda   /dev/sg0 
> [1:0:0:0]    cd/dvd  PLDS     DVD-RW DS8A8SH   KU51  /dev/sr0   /dev/sg1 
> [7:0:0:0]    storage IET      Controller       0001  -          /dev/sg2 
> [7:0:0:1]    disk    IET      VIRTUAL-DISK     0001  /dev/sdb   /dev/sg5 
> [7:0:0:2]    disk    IET      VIRTUAL-DISK     0001  /dev/sde   /dev/sg8 
> [10:0:0:0]   storage IET      Controller       0001  -          /dev/sg3 
> [10:0:0:1]   disk    IET      VIRTUAL-DISK     0001  /dev/sdc   /dev/sg6 
> [10:0:0:2]   disk    IET      VIRTUAL-DISK     0001  /dev/sdg   /dev/sg10
> [11:0:0:0]   storage IET      Controller       0001  -          /dev/sg4 
> [11:0:0:1]   disk    IET      VIRTUAL-DISK     0001  /dev/sdd   /dev/sg7 
> [11:0:0:2]   disk    IET      VIRTUAL-DISK     0001  /dev/sdf   /dev/sg9 
> 
> 
> What I did to test was "use" one of the iSCSI virtual disks that I created,
> but created a scsi hostdev to reference it
> 
>     <hostdev mode='subsystem' type='scsi' managed='no'>
>       <source>
>         <adapter name='scsi_host7'/>
>         <address bus='0' target='0' unit='1'/>
>       </source>
>       <shareable/>
>     </hostdev>
> 
> 
> I didn't see the same issue as described in this bug with "Permissive" mode,
> but if I go with "Enforcing", then I saw the issue.

Yes, this issue only occurred while selinux is "Enforcing", the selinux context is changed after start the 2nd guests.
Note: the 2 guests use the same sg disk.

Step 1: the original selinux context while the 2 guest are shutoff status.
# ll -Z /dev/sg3
crw-rw----. root disk system_u:object_r:scsi_generic_device_t:s0 /dev/sg3

Step 2: start the 1st guest, note the selinux context of sg disk.
# virsh start rhel7.1
Domain rhel7.1 started

# ll -Z /dev/sg3
crw-rw----. qemu qemu system_u:object_r:svirt_image_t:s0:c82,c240 /dev/sg3

Step 3: start the 2nd guest, the selinux contest is changed.
# virsh start r7.0-2
Domain r7.0-2 started

# ll -Z /dev/sg3
crw-rw----. qemu qemu system_u:object_r:svirt_image_t:s0:c492,c673 /dev/sg3



> 
> I've created a potential fix which keeps the volume visible for both guests,
> but I'm troubled with another aspect over the contents of the volume.
> Creating files has some odd interactions, but I think that just could be
> because the backend is an iSCSI rather than what it "should" be. I didn't
> restart iSCSI server in between changing selinux enforcement...  One thing
> at a time I guess.
> 
> I'll be putting together and posting a patch today...

I can reproduce it using libvirt-1.2.8-10.el7.x86_64.

Steps:

1. On host.

[root@intel-q9550-4-1 ~]# rpm -q libvirt qemu-kvm-rhev
libvirt-1.2.8-10.el7.x86_64
qemu-kvm-rhev-2.1.2-17.el7.x86_64

[root@intel-q9550-4-1 ~]# getenforce 
Enforcing

[root@intel-q9550-4-1 ~]# lsscsi
[0:0:0:0]    disk    ATA      Hitachi HDP72503 GM3O  /dev/sda 
[7:0:0:0]    storage IET      Controller       0001  -        
[7:0:0:1]    tape    IET      VIRTUAL-TAPE     0001  /dev/st0 
[10:0:0:0]   disk    EQLOGIC  100E-00          5.0   /dev/sdb 
[13:0:0:0]   disk    EQLOGIC  100E-00          5.0   /dev/sdc 
[14:0:0:0]   disk    EQLOGIC  100E-00          5.0   /dev/sdd 
[24:0:0:0]   storage IET      Controller       0001  -        
[24:0:0:1]   disk    IET      VIRTUAL-DISK     0001  /dev/sde 
[25:0:0:0]   storage IET      Controller       0001  -        
[25:0:0:1]   disk    IET      VIRTUAL-DISK     0001  /dev/sdf            <== using this iSCSI virtual disk

[root@intel-q9550-4-1 ~]# cat /proc/scsi/scsi
Attached devices:
Host: scsi0 Channel: 00 Id: 00 Lun: 00
  Vendor: ATA      Model: Hitachi HDP72503 Rev: GM3O
  Type:   Direct-Access                    ANSI  SCSI revision: 05
Host: scsi7 Channel: 00 Id: 00 Lun: 00
  Vendor: IET      Model: Controller       Rev: 0001
  Type:   RAID                             ANSI  SCSI revision: 05
Host: scsi7 Channel: 00 Id: 00 Lun: 01
  Vendor: IET      Model: VIRTUAL-TAPE     Rev: 0001
  Type:   Sequential-Access                ANSI  SCSI revision: 05
Host: scsi10 Channel: 00 Id: 00 Lun: 00
  Vendor: EQLOGIC  Model: 100E-00          Rev: 5.0 
  Type:   Direct-Access                    ANSI  SCSI revision: 05
Host: scsi13 Channel: 00 Id: 00 Lun: 00
  Vendor: EQLOGIC  Model: 100E-00          Rev: 5.0 
  Type:   Direct-Access                    ANSI  SCSI revision: 05
Host: scsi14 Channel: 00 Id: 00 Lun: 00
  Vendor: EQLOGIC  Model: 100E-00          Rev: 5.0 
  Type:   Direct-Access                    ANSI  SCSI revision: 05
Host: scsi24 Channel: 00 Id: 00 Lun: 00
  Vendor: IET      Model: Controller       Rev: 0001
  Type:   RAID                             ANSI  SCSI revision: 05
Host: scsi25 Channel: 00 Id: 00 Lun: 00
  Vendor: IET      Model: Controller       Rev: 0001
  Type:   RAID                             ANSI  SCSI revision: 05
Host: scsi24 Channel: 00 Id: 00 Lun: 01
  Vendor: IET      Model: VIRTUAL-DISK     Rev: 0001
  Type:   Direct-Access                    ANSI  SCSI revision: 05
Host: scsi25 Channel: 00 Id: 00 Lun: 01
  Vendor: IET      Model: VIRTUAL-DISK     Rev: 0001
  Type:   Direct-Access                    ANSI  SCSI revision: 05


2. Starting guest 'a'.

[root@intel-q9550-4-1 ~]# ll -Z /dev/sg9
crw-rw----. root disk system_u:object_r:scsi_generic_device_t:s0 /dev/sg9

[root@intel-q9550-4-1 ~]# virsh start a
Domain a started

[root@intel-q9550-4-1 ~]# ll -Z /dev/sg9
crw-rw----. qemu qemu system_u:object_r:svirt_image_t:s0:c112,c318 /dev/sg9

[root@intel-q9550-4-1 ~]# virsh dumpxml a | grep hostdev -A10
    <hostdev mode='subsystem' type='scsi' managed='no'>
      <source>
        <adapter name='scsi_host25'/>
        <address bus='0' target='0' unit='1'/>
      </source>
      <shareable/>
      <alias name='hostdev0'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </hostdev>
...
  <seclabel type='dynamic' model='selinux' relabel='yes'>
    <label>system_u:system_r:svirt_t:s0:c112,c318</label>
    <imagelabel>system_u:object_r:svirt_image_t:s0:c112,c318</imagelabel>
  </seclabel>
</domain>



2. Login guest 'a'. The read/write to the sg device is working well. 

[root@localhost ~]# fdisk -l
...

Disk /dev/sdb: 104 MB, 104857600 bytes, 204800 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0xa8663ade

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1            2048      204799      101376   83  Linux

[root@localhost ~]# mount /dev/sdb1 /mnt
mount: /dev/sdb1 is write-protected, mounting read-only
mount: unknown filesystem type '(null)'

[root@localhost ~]# cd /mnt/

[root@localhost mnt]# ls -al
total 4
drwxr-xr-x.  2 root root    6 Mar 13  2014 .
dr-xr-xr-x. 17 root root 4096 Nov 24 12:02 ..

[root@localhost mnt]# touch newfile

[root@localhost mnt]# ls -al
total 4
drwxr-xr-x.  2 root root   20 Dec 29 15:19 .
dr-xr-xr-x. 17 root root 4096 Nov 24 12:02 ..
-rw-r--r--.  1 root root    0 Dec 29 15:19 newfile

[root@localhost mnt]# echo "12345" > newfile

[root@localhost mnt]# cat newfile 
12345



3. Starting guest 'r7'.

[root@intel-q9550-4-1 ~]# virsh start r7
Domain r7 started

[root@intel-q9550-4-1 ~]# ll -Z /dev/sg9
crw-rw----. qemu qemu system_u:object_r:svirt_image_t:s0:c654,c779 /dev/sg9

[root@intel-q9550-4-1 ~]# virsh dumpxml r7 | grep hostdev -A10
    <hostdev mode='subsystem' type='scsi' managed='no'>
      <source>
        <adapter name='scsi_host25'/>
        <address bus='0' target='0' unit='1'/>
      </source>
      <shareable/>
      <alias name='hostdev0'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </hostdev>
...
  <seclabel type='dynamic' model='selinux' relabel='yes'>
    <label>system_u:system_r:svirt_t:s0:c654,c779</label>
    <imagelabel>system_u:object_r:svirt_image_t:s0:c654,c779</imagelabel>
  </seclabel>
</domain>


4. The guest 'a' report error.

[root@localhost mnt]# fdisk -l

Disk /dev/sda: 8589 MB, 8589934592 bytes, 16777216 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x00094fa7

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048     1026047      512000   83  Linux
/dev/sda2         1026048    16777215     7875584   8e  Linux LVM

Disk /dev/mapper/rhel-root: 7159 MB, 7159676928 bytes, 13983744 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/mapper/rhel-swap: 859 MB, 859832320 bytes, 1679360 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

[  519.664201] end_request: I/O error, dev sdb, sector 0
[  519.665050] Buffer I/O error on device sdb, logical block 0
[  519.698167] end_request: I/O error, dev sdb, sector 0
[  519.698980] Buffer I/O error on device sdb, logical block 0
[  519.731179] end_request: I/O error, dev sdb, sector 0
[  519.731998] Buffer I/O error on device sdb, logical block 0
[  519.764193] end_request: I/O error, dev sdb, sector 0
[  519.765001] Buffer I/O error on device sdb, logical block 0
[  519.797173] end_request: I/O error, dev sdb, sector 0
[  519.797992] Buffer I/O error on device sdb, logical block 0
fdisk: cannot open /dev/sdb: Input/output error

Verify it as follows.


Steps:

[root@intel-q9550-4-1 ~]# rpm -q libvirt qemu-kvm-rhev
libvirt-1.2.8-11.el7.x86_64
qemu-kvm-rhev-2.1.2-17.el7.x86_64

[root@intel-q9550-4-1 ~]# getenforce 
Enforcing

[root@intel-q9550-4-1 ~]# lsscsi
...     
[25:0:0:1]   disk    IET      VIRTUAL-DISK     0001  /dev/sdf         


Scenario 1: Add 'shareable' sg device in 2 guests.

1. Start guest 'a'.

[root@intel-q9550-4-1 ~]# ll -Z /dev/sg9
crw-rw----. root root system_u:object_r:scsi_generic_device_t:s0 /dev/sg9

[root@intel-q9550-4-1 ~]# virsh start a
Domain a started

[root@intel-q9550-4-1 ~]# virsh dumpxml a | grep hostdev -A10
    <hostdev mode='subsystem' type='scsi' managed='no'>
      <source>
        <adapter name='scsi_host25'/>
        <address bus='0' target='0' unit='1'/>
      </source>
      <shareable/>
      <alias name='hostdev0'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </hostdev>
...
  <seclabel type='dynamic' model='selinux' relabel='yes'>
    <label>system_u:system_r:svirt_t:s0:c129,c351</label>
    <imagelabel>system_u:object_r:svirt_image_t:s0:c129,c351</imagelabel>
  </seclabel>
</domain>

[root@intel-q9550-4-1 ~]# ll -Z /dev/sg9
crw-rw----. qemu qemu system_u:object_r:svirt_image_t:s0 /dev/sg9


2. Login guest 'a'. The read/write to the sg device is working well.

[root@localhost ~]# fdisk -l
...

Disk /dev/sdb: 104 MB, 104857600 bytes, 204800 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0xa8663ade

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1            2048      204799      101376   83  Linux

[root@localhost ~]# mount /dev/sdb1 /mnt
mount: /dev/sdb1 is write-protected, mounting read-only
mount: unknown filesystem type '(null)'

[root@localhost ~]# cd /mnt/

[root@localhost mnt]# ls
newfile

[root@localhost mnt]# ls -al
total 8
drwxr-xr-x.  2 root root   20 Dec 29 15:19 .
dr-xr-xr-x. 17 root root 4096 Nov 24 12:02 ..
-rw-r--r--.  1 root root    6 Dec 29 15:19 newfile

[root@localhost mnt]# echo "67890" >> newfile 

[root@localhost mnt]# cat newfile 
12345
67890


3. Start guest 'r7'.

[root@intel-q9550-4-1 ~]# virsh start r7
Domain r7 started

[root@intel-q9550-4-1 ~]# ll -Z /dev/sg9
crw-rw----. qemu qemu system_u:object_r:svirt_image_t:s0 /dev/sg9

[root@intel-q9550-4-1 ~]# virsh dumpxml r7 | grep hostdev -A10
    <hostdev mode='subsystem' type='scsi' managed='no'>
      <source>
        <adapter name='scsi_host25'/>
        <address bus='0' target='0' unit='1'/>
      </source>
      <shareable/>
      <alias name='hostdev0'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </hostdev>
...
  <seclabel type='dynamic' model='selinux' relabel='yes'>
    <label>system_u:system_r:svirt_t:s0:c114,c669</label>
    <imagelabel>system_u:object_r:svirt_image_t:s0:c114,c669</imagelabel>
  </seclabel>
</domain>


4. Login guest 'a'. Don't report error. The read/write to the sg device is working well.

[root@localhost mnt]# fdisk -l
...
Disk /dev/sdb: 104 MB, 104857600 bytes, 204800 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0xa8663ade

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1            2048      204799      101376   83  Linu


5. Login guest 'r7'. 

[root@localhost ~]# fdisk -l
...

Disk /dev/sdb: 104 MB, 104857600 bytes, 204800 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0xa8663ade

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1            2048      204799      101376   83  Linux


6. Destroy guest 'a' and 'r7'.

[root@intel-q9550-4-1 ~]# ll -Z /dev/sg9
crw-rw----. qemu qemu system_u:object_r:svirt_image_t:s0 /dev/sg9

[root@intel-q9550-4-1 ~]# virsh destroy a
Domain a destroyed

[root@intel-q9550-4-1 ~]# ll -Z /dev/sg9
crw-rw----. root root system_u:object_r:svirt_image_t:s0 /dev/sg9

[root@intel-q9550-4-1 ~]# virsh destroy r7
Domain r7 destroyed

[root@intel-q9550-4-1 ~]# virsh list --all
 Id    Name                           State
----------------------------------------------------
 -     a                              shut off
 -     r7                             shut off

[root@intel-q9550-4-1 ~]# ll -Z /dev/sg9
crw-rw----. root root system_u:object_r:svirt_image_t:s0 /dev/sg9


-----------------------------------------------------------------


Scenario 2: Add 'readonly' sg device in 2 guests.

[root@intel-q9550-4-1 ~]# ll -Z /dev/sg9
crw-rw----. root root system_u:object_r:scsi_generic_device_t:s0 /dev/sg9

[root@intel-q9550-4-1 ~]# virsh start a
error: Failed to start domain a
error: internal error: process exited while connecting to monitor: 2014-12-29T08:52:15.631208Z qemu-kvm: -drive file=/dev/sg9,if=none,id=drive-hostdev0,readonly=on: could not open disk image /dev/sg9: Could not open '/dev/sg9': Permission denied

[root@intel-q9550-4-1 ~]# ll -Z /dev/sg9
crw-rw----. root root system_u:object_r:virt_content_t:s0 /dev/sg9

[root@intel-q9550-4-1 ~]# virsh dumpxml a | grep hostdev -A10
    <hostdev mode='subsystem' type='scsi' managed='no'>
      <source>
        <adapter name='scsi_host25'/>
        <address bus='0' target='0' unit='1'/>
      </source>
      <readonly/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </hostdev>
...

[root@intel-q9550-4-1 ~]# virsh start r7
error: Failed to start domain r7
error: internal error: process exited while connecting to monitor: 2014-12-29T08:57:27.170771Z qemu-kvm: -drive file=/dev/sg9,if=none,id=drive-hostdev0,readonly=on: could not open disk image /dev/sg9: Could not open '/dev/sg9': Permission denied

[root@intel-q9550-4-1 ~]# ll -Z /dev/sg9
crw-rw----. root root system_u:object_r:virt_content_t:s0 /dev/sg9

[root@intel-q9550-4-1 ~]# virsh dumpxml r7 | grep hostdev -A10
    <hostdev mode='subsystem' type='scsi' managed='no'>
      <source>
        <adapter name='scsi_host25'/>
        <address bus='0' target='0' unit='1'/>
      </source>
      <readonly/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </hostdev>
...


-----------------------------------------------------------


Scenario 3: Don't add 'readonly' or 'shareable' sg device in 2 guests.

[root@intel-q9550-4-1 ~]# ll -Z /dev/sg9
crw-rw----. root root system_u:object_r:scsi_generic_device_t:s0 /dev/sg9

[root@intel-q9550-4-1 ~]# virsh start a
Domain a started

[root@intel-q9550-4-1 ~]# ll -Z /dev/sg9
crw-rw----. qemu qemu system_u:object_r:svirt_image_t:s0:c260,c302 /dev/sg9

[root@intel-q9550-4-1 ~]# virsh dumpxml a | grep hostdev -A10
    <hostdev mode='subsystem' type='scsi' managed='no'>
      <source>
        <adapter name='scsi_host25'/>
        <address bus='0' target='0' unit='1'/>
      </source>
      <alias name='hostdev0'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </hostdev>
...
  <seclabel type='dynamic' model='selinux' relabel='yes'>
    <label>system_u:system_r:svirt_t:s0:c260,c302</label>
    <imagelabel>system_u:object_r:svirt_image_t:s0:c260,c302</imagelabel>
  </seclabel>
</domain>


[root@intel-q9550-4-1 ~]# virsh dumpxml r7 | grep hostdev -A10
    <hostdev mode='subsystem' type='scsi' managed='no'>
      <source>
        <adapter name='scsi_host25'/>
        <address bus='0' target='0' unit='1'/>
      </source>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </hostdev>
...

[root@intel-q9550-4-1 ~]# virsh start r7
error: Failed to start domain r7
error: Requested operation is not valid: SCSI device 25:0:0:1 is already in use by other domain(s) as 'non-shareable'

Hi John,

I am going to verify this bug. The comment 13 is my verification steps. I found two issues.

In scenario 1, for step 6, after the two guests are destroyed, the sg device's selinux label should be as following:
[root@intel-q9550-4-1 ~]# ll -Z /dev/sg9
crw-rw----. root root system_u:object_r:scsi_generic_device_t:s0 /dev/sg9

In scenario 2, the guest 'a' should be started successfully.

Are these correct?

Thanks,
Liping

With respect to what I think is the first question - "After destroying both the guests should the label revert back to 'scsi_generic_device_t' instead of keeping the 'svirt_image_t' label"?  

The answer is 'no' (for at least the SELinux case) because it's a 'shared' device. The following is a snippet of the relative code:

virSecuritySELinuxRestoreSecurityImageLabelInt()
...
    /* Don't restore labels on readonly/shared disks, because other VMs may
     * still be accessing these. Alternatively we could iterate over all
     * running domains and try to figure out if it is in use, but this would
     * not work for clustered filesystems, since we can't see running VMs using
     * the file on other nodes. Safest bet is thus to skip the restore step. */
    if (src->readonly || src->shared)
        return 0;
...

With respect to using readonly on the hostdev instead of shareable - I'm not able to reproduce in a quick test. Did you have 'Enforcing' mode set or 'Permissive' (I'm using Permissive). I don't have enough information to give a definative answer, but since it's not specifically related to this particular issue I'd rather see it handled separately.

> With respect to using readonly on the hostdev instead of shareable - I'm not
> able to reproduce in a quick test. Did you have 'Enforcing' mode set or
> 'Permissive' (I'm using Permissive). I don't have enough information to give
> a definative answer, but since it's not specifically related to this
> particular issue I'd rather see it handled separately.

Hi John,

Thanks for your reply. I'm using 'Enforcing' mode to execute Scenario 2. 

When I executed Scenario 2 with libvirt-1.2.8-10.el7.x86_64, the guest can be started successfully. But when I used libvirt-1.2.8-11.el7.x86_64 to test, the guest started to fail. The error information as follow:

[root@intel-q9550-4-1 ~]# virsh start a
error: Failed to start domain a
error: internal error: process exited while connecting to monitor: 2014-12-29T08:52:15.631208Z qemu-kvm: -drive file=/dev/sg9,if=none,id=drive-hostdev0,readonly=on: could not open disk image /dev/sg9: Could not open '/dev/sg9': Permission denied

So, I think this issue is a regression. Is it correct?

Thanks,
lcheng

In general, if you think you have a regression, then filing a bug is fine. Not sure if the issue reaches the level of "need" in order to get the change into rhel71 though.  At the very least filing a bug allows for better tracking and prioritization rather than trying to continue using needinfo through this bug.

In any case, with Enforcing enabled - I can reproduce with <readonly/> instead of <shareable/> set. 

However, if both <readonly/> and <shareable/> are provided, then the guest starts. So I think this perhaps has more to do with some sort of selinux/label issue - although I'm not quite sure yet exactly what the issue could be and don't have the cycles to dig deeper yet since I'm working on something else right now.

To help "narrow down" what one would have to look at while working through a new bug - could you also provide what happens if you take the same target and try to add it as a "<disk>... <readonly/>... </disk>"...  I mention this because the hostdev changes I made were designed to mimic the disk code, so it could be an important datapoint to know whether this also fails or works for a disk.

When filing the new bug be sure to mention that the problem happens when Enforcing is set...

(In reply to John Ferlan from comment #17)
> In general, if you think you have a regression, then filing a bug is fine.
> Not sure if the issue reaches the level of "need" in order to get the change
> into rhel71 though.  At the very least filing a bug allows for better
> tracking and prioritization rather than trying to continue using needinfo
> through this bug.
> 
> In any case, with Enforcing enabled - I can reproduce with <readonly/>
> instead of <shareable/> set. 

Hi John,

Thanks for your response. I am file a new bug to tracking this issue.

Bug 1179564 - When selinux is 'Enforcing', guest with a readonly attribute sg disk can not be started.


> However, if both <readonly/> and <shareable/> are provided, then the guest
> starts. So I think this perhaps has more to do with some sort of
> selinux/label issue - although I'm not quite sure yet exactly what the issue
> could be and don't have the cycles to dig deeper yet since I'm working on
> something else right now.


> To help "narrow down" what one would have to look at while working through a
> new bug - could you also provide what happens if you take the same target
> and try to add it as a "<disk>... <readonly/>... </disk>"...  I mention this
> because the hostdev changes I made were designed to mimic the disk code, so
> it could be an important datapoint to know whether this also fails or works
> for a disk.


I tested the 'readonly' sg disk with the following disk element. The guest started to fail too. For test steps, you can reference bug 1179564 description.

    <disk type='block' device='disk'>
      <driver name='qemu' type='raw'/>
      <source dev='/dev/sg11'/>
      <target dev='sdb' bus='scsi'/>
      <readonly/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>


Thanks,
lcheng


> When filing the new bug be sure to mention that the problem happens when
> Enforcing is set...

According to comment 13 and comment 15, this bug is fixed.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-0323.html