Bug 1361592

Description of problem:
When trying to attach an image as a device to a running vm the attachment fails if the image file has a backing file (e.g. from an external snapshot). Afaics the apparmor profile of the VM is extended only for the image file itself but not it's backing file. Libvirt fails to attach when trying to access the images backing file afterwards. The libvirt log shows:

------libvirt.log excerpt-------
2016-07-27 10:31:23.135+0000: 16633: debug : qemuDomainPCIAddressReserveAddr:2114 : Reserving PCI slot 0000:00:05.0 (multifunction='off')
2016-07-27 10:31:23.193+0000: 16633: debug : qemuSetupDiskPathAllow:60 : Process path /mnt/quobyte-volume/abfa1002557ab2b21ec218a86487dd92/volume-605eb107-baf5-4a10-a2b9-60a3a77f2fc0.1a65bc67-f300-4927-9231-d480
5690e81b for disk
2016-07-27 10:31:23.194+0000: 16633: debug : qemuSetupDiskPathAllow:60 : Process path /mnt/quobyte-volume/abfa1002557ab2b21ec218a86487dd92/volume-605eb107-baf5-4a10-a2b9-60a3a77f2fc0.2f3c2d0e-5315-48f3-9fc7-7e39
2c7958e1 for disk
2016-07-27 10:31:23.194+0000: 16633: debug : qemuSetupDiskPathAllow:60 : Process path /mnt/quobyte-volume/abfa1002557ab2b21ec218a86487dd92/volume-605eb107-baf5-4a10-a2b9-60a3a77f2fc0 for disk
2016-07-27 10:31:23.197+0000: 16633: debug : virCommandRunAsync:2282 : About to run /usr/lib/libvirt/virt-aa-helper -p 0 -r -u libvirt-4df0bca7-5e5c-47b8-9c27-d40277351fb0 -f /mnt/quobyte-volume/abfa1002557ab2b21ec218a86487dd92/volume-605eb107-baf5-4a10-a2b9-60a3a77f2fc0.1a65bc67-f300-4927-9231-d4805690e81b
2016-07-27 10:31:23.197+0000: 16633: debug : virCommandRunAsync:2285 : Command result 0, with PID 10895
2016-07-27 10:31:23.521+0000: 16633: debug : virCommandRun:2142 : Result status 0, stdout: '' stderr: ''
2016-07-27 10:31:23.521+0000: 16633: debug : qemuDomainPCIAddressGetNextSlot:2270 : PCI slot 0000:00:01 already in use
2016-07-27 10:31:23.521+0000: 16633: debug : qemuDomainPCIAddressGetNextSlot:2270 : PCI slot 0000:00:02 already in use
2016-07-27 10:31:23.521+0000: 16633: debug : qemuDomainPCIAddressGetNextSlot:2270 : PCI slot 0000:00:03 already in use
2016-07-27 10:31:23.521+0000: 16633: debug : qemuDomainPCIAddressGetNextSlot:2270 : PCI slot 0000:00:04 already in use
2016-07-27 10:31:23.521+0000: 16633: debug : qemuDomainPCIAddressGetNextSlot:2313 : Found free PCI slot 0000:00:05
2016-07-27 10:31:23.521+0000: 16633: debug : qemuDomainPCIAddressReserveAddr:2114 : Reserving PCI slot 0000:00:05.0 (multifunction='off')
2016-07-27 10:31:23.521+0000: 16633: debug : qemuDomainObjEnterMonitorInternal:1278 : Entering monitor (mon=0x7fc904181380 vm=0x7fc8f80117a0 name=instance-00000015)
2016-07-27 10:31:23.521+0000: 16633: debug : qemuMonitorAddDrive:3011 : mon=0x7fc904181380 drive=file=/mnt/quobyte-volume/abfa1002557ab2b21ec218a86487dd92/volume-605eb107-baf5-4a10-a2b9-60a3a77f2fc0.1a65bc67-f30
0-4927-9231-d4805690e81b,if=none,id=drive-virtio-disk1,format=qcow2,serial=605eb107-baf5-4a10-a2b9-60a3a77f2fc0,cache=none,aio=native
2016-07-27 10:31:23.521+0000: 16633: debug : qemuMonitorJSONAddDrive:3126 : drive_add command not found, trying HMP
2016-07-27 10:31:23.521+0000: 16633: debug : qemuMonitorJSONCommandWithFd:264 : Send command '{"execute":"human-monitor-command","arguments":{"command-line":"drive_add dummy file=/mnt/quobyte-volume/abfa1002557a
b2b21ec218a86487dd92/volume-605eb107-baf5-4a10-a2b9-60a3a77f2fc0.1a65bc67-f300-4927-9231-d4805690e81b,if=none,id=drive-virtio-disk1,format=qcow2,serial=605eb107-baf5-4a10-a2b9-60a3a77f2fc0,cache=none,aio=native"
},"id":"libvirt-16"}' for write with FD -1
2016-07-27 10:31:23.521+0000: 16633: debug : qemuMonitorSend:959 : QEMU_MONITOR_SEND_MSG: mon=0x7fc904181380 msg={"execute":"human-monitor-command","arguments":{"command-line":"drive_add dummy file=/mnt/quobyte-
volume/abfa1002557ab2b21ec218a86487dd92/volume-605eb107-baf5-4a10-a2b9-60a3a77f2fc0.1a65bc67-f300-4927-9231-d4805690e81b,if=none,id=drive-virtio-disk1,format=qcow2,serial=605eb107-baf5-4a10-a2b9-60a3a77f2fc0,cache=none,aio=native"},"id":"libvirt-16"}
 fd=-1
2016-07-27 10:31:23.521+0000: 16628: debug : qemuMonitorIOWrite:504 : QEMU_MONITOR_IO_WRITE: mon=0x7fc904181380 buf={"execute":"human-monitor-command","arguments":{"command-line":"drive_add dummy file=/mnt/quobyte-volume/abfa1002557ab2b21ec218a86487dd92/volume-605eb107-baf5-4a10-a2b9-60a3a77f2fc0.1a65bc67-f300-4927-9231-d4805690e81b,if=none,id=drive-virtio-disk1,format=qcow2,serial=605eb107-baf5-4a10-a2b9-60a3a77f2fc0,cache=none,aio=native"},"id":"libvirt-16"}
 len=350 ret=350 errno=11
2016-07-27 10:31:23.632+0000: 16628: debug : qemuMonitorIOProcess:396 : QEMU_MONITOR_IO_PROCESS: mon=0x7fc904181380 buf={"return": "could not open disk image /mnt/quobyte-volume/abfa1002557ab2b21ec218a86487dd92/volume-605eb107-baf5-4a10-a2b9-60a3a77f2fc0.1a65bc67-f300-4927-9231-d4805690e81b: Could not open backing file: Could not open '/mnt/quobyte-volume/abfa1002557ab2b21ec218a86487dd92/volume-605eb107-baf5-4a10-a2b9-60a3a77f2fc0.2f3c2d0e-5315-48f3-9fc7-7e392c7958e1': Permission denied\r\n", "id": "libvirt-16"}
 len=399
2016-07-27 10:31:23.632+0000: 16628: debug : qemuMonitorJSONIOProcessLine:157 : Line [{"return": "could not open disk image /mnt/quobyte-volume/abfa1002557ab2b21ec218a86487dd92/volume-605eb107-baf5-4a10-a2b9-60a3a77f2fc0.1a65bc67-f300-4927-9231-d4805690e81b: Could not open backing file: Could not open '/mnt/quobyte-volume/abfa1002557ab2b21ec218a86487dd92/volume-605eb107-baf5-4a10-a2b9-60a3a77f2fc0.2f3c2d0e-5315-48f3-9fc7-7e392c7958e1': Permission denied\r\n", "id": "libvirt-16"}]
------end of excerpt----------

(image file: /mnt/quobyte-volume/abfa1002557ab2b21ec218a86487dd92/volume-605eb107-baf5-4a10-a2b9-60a3a77f2fc0.1a65bc67-f300-4927-9231-d4805690e81b)
(backing file: /mnt/quobyte-volume/abfa1002557ab2b21ec218a86487dd92/volume-605eb107-baf5-4a10-a2b9-60a3a77f2fc0)

I do not know if there should be two calls to the virt-aa-helper (one for the image itself and one for it's backing file) or if virt-aa-helper should analyse the image and deduct the requirement of adding the images backing file by itself.

Attaching the image works if the vm is shut down and started afterwards. In this latter case the apparmor profile of the VM contains the files of the image as well as it's backing file and everything is fine.

Things also work if the image has no backing file, the image file device is added to the profile ok and the device is added in the vm (this is from a different test run therefore different filename & timestamps):

-------------libvirt.log excerpt---------------
2016-07-29 12:53:51.645+0000: 6958: debug : qemuDomainObjEnterMonitorInternal:1278 : Entering monitor (mon=0x7fa6c800c1f0 vm=0x7fa6dc003a30 name=instance-0000000a)  
2016-07-29 12:53:51.645+0000: 6958: debug : qemuMonitorAddDrive:3011 : mon=0x7fa6c800c1f0 drive=file=/mnt/quobyte-volume/abfa1002557ab2b21ec218a86487dd92/volume-48535ec1-31be-422a-8285-da24fbf5cc44,if=none,id=drive-virtio-disk1,format=qcow2,serial=48535ec1-31be-422a-8285-da24fbf5cc44,cache=none,aio=native
2016-07-29 12:53:51.645+0000: 6958: debug : qemuMonitorJSONAddDrive:3126 : drive_add command not found, trying HMP  
2016-07-29 12:53:51.645+0000: 6958: debug : qemuMonitorJSONCommandWithFd:264 : Send command '{"execute":"human-monitor-command","arguments":{"command-line":"drive_add dummy file=/mnt/quobyte-volume/abfa1002557ab2b21ec218a86487dd92/volume-48535ec1-31be-422a-8285-da24fbf5cc44,if=none,id=drive-virtio-disk1,format=qcow2,serial=48535ec1-31be-422a-8285-da24fbf5cc44,cache=none,aio=native"},"id":"libvirt-16"}' for write with FD -1  
2016-07-29 12:53:51.645+0000: 6958: debug : qemuMonitorSend:959 : QEMU_MONITOR_SEND_MSG: mon=0x7fa6c800c1f0 msg={"execute":"human-monitor-command","arguments":{"command-line":"drive_add dummy file=/mnt/quobyte-volume/abfa1002557ab2b21ec218a86487dd92/volume-48535ec1-31be-422a-8285-da24fbf5cc44,if=none,id=drive-virtio-disk1,format=qcow2,serial=48535ec1-31be-422a-8285-da24fbf5cc44,cache=none,aio=native"},"id":"libvirt-16"}
 fd=-1  
2016-07-29 12:53:51.648+0000: 6955: debug : qemuMonitorIOWrite:504 : QEMU_MONITOR_IO_WRITE: mon=0x7fa6c800c1f0 buf={"execute":"human-monitor-command","arguments":{"command-line":"drive_add dummy file=/mnt/quobyte-volume/abfa1002557ab2b21ec218a86487dd92/volume-48535ec1-31be-422a-8285-da24fbf5cc44,if=none,id=drive-virtio-disk1,format=qcow2,serial=48535ec1-31be-422a-8285-da24fbf5cc44,cache=none,aio=native"},"id":"libvirt-16"}
 len=313 ret=313 errno=11
2016-07-29 12:53:51.671+0000: 6955: debug : qemuMonitorIOProcess:396 : QEMU_MONITOR_IO_PROCESS: mon=0x7fa6c800c1f0 buf={"return": "OK\r\n", "id": "libvirt-16"}
 len=42
2016-07-29 12:53:51.671+0000: 6955: debug : qemuMonitorJSONIOProcessLine:157 : Line [{"return": "OK\r\n", "id": "libvirt-16"}]

-------------end of excerpt--------------------

This issue came up in the context of the OpenStack project but can be reproduced in a plain libvirt setup.


Version-Release number of selected component (if applicable):
These issues came up with an ubuntu 16.04 (apparmor=2.10.95-0ubuntu2, libvirt-bin=1.3.1-1ubuntu10.1) as well as an ubuntu 14.04 setup (apparmor 2.8.95~2430-0ubuntu5.3, libvirt 1.2.2-0ubuntu13.1.17).


How reproducible:
always


Steps to Reproduce:
1) create a volume -> works
2) attach volume to guest -> works
3) detach volume from guest -> works
4) create snapshot from volume -> works
5) attach volume(!) to guest -> FAIL
6) delete snapshot -> works
7) attach volume to guest -> works(!)
As described earlier, when running this procedure with a stopped guest things work fine, e.g.:
[..]
4) create snapshot from volume -> works
5) attach volume to (shutdown) guest -> works (checked by powering up and looking for the volumes device)

Actual results:
image cannot be attached as a device.

Expected results:
image should be attached as requested

Additional info:
- Most of the testing was done in an Openstack dev setup (DevStack) but i could verify the same results by running the underlying libvirt commands manually via virsh. Results stayed the same.
- The /etc/libvirt/qemu.conf file has some specific settings:
    user=<non root nova user>
    group=<non root cinder group>
    dynamic_ownership = 0
- security_driver is set to default in this setup