Bug 1564885

Summary: guestmount ignores AppArmor and the setting "security_require_confined = 1"
Product: [Community] Virtualization Tools Reporter: Ralph <rm>
Component: libguestfsAssignee: Richard W.M. Jones <rjones>
Status: NEW --- QA Contact:
Severity: low Docs Contact:
Priority: unspecified    
Version: unspecifiedCC: ptoscano, rm
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Ralph 2018-04-08 16:05:35 UTC
guestfs-tools as guestmount seem to ignore AppArmor settings (if SE-Linux is deactivated). Maybe this is not a bug - but it is at least a deficit. And it is inconsistent with the handling of regular QEMU/KVM virtual guests/machines. 
 
I use guestmount frequently on an Opensuse System - which has AppArmor activated. 

Security relevant settings in "/etc/libvirt/qemu.conf" are: 

 security_driver = "apparmor"
 security_default_confined = 1
 security_require_confined = 1

These settings work well for standard virtual (KVM) QEMU-machines (despite an initial  delay of the startup of the guest system; but no significant performance impact once the KVM guest has started). Whenever I start a standard virtual KVM/qemu-machine as an unprivileged user (e.g. "myself") from virt-manager the security model is correctly set; example:   

myself@mytux ~:  ps aux | grep qemu
qemu     25256  7.6  0.1 9335916 76824 ?       Dl   17:12   0:00 /usr/bin/qemu-system-x86_64 -machine accel=kvm -name guest=debian9

Of course in this situation the process owner is qemu.   

mytux:/var/lib/libvirt/qemu # virsh dominfo debian9
Id:             2
Name:           debian
UUID:           745678...
OS Type:        hvm
State:          running
CPU(s):         3
CPU time:       6.5s
Max memory:     8388608 KiB
Used memory:    8388608 KiB
Persistent:     yes
Autostart:      disable
Managed save:   no
Security model: apparmor
Security DOI:   0
Security label: libvirt-745678... (enforcing)  

However, this does not work with guestmount and its qemu-machine for an unprivileged user - even if I delegate the start of the supermin qemu-machine to libvirt by setting an environment variable:

myself@mytux ~: export LIBGUESTFS_ATTACH_METHOD=libvirt 

After e.g. 
myself@mytux ~: guestmount -a W7_x64_ssdx.vmdk  -m /dev/sda2  --ro /mnt/vmdk

The virtual machine is started by libvirt for the user "myself": 

myself@mytux ~: ps aux | grep qemu

myself      23491  0.2  0.4 2188212 267952 ?      Sl   16:41   0:01 /usr/bin/qemu-system-x86_64 -machine accel=kvm -name guest=guestfs-kedv5jvf5nz9zv06,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/home/myself/.config/libvirt/qemu/lib/domain-2-guestfs-kedv5jvf5nz9/master-key.aes -machine pc-i440fx-2.9,accel=kvm,usb=off,dump-guest-core=off -cpu host -m 500 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid b5d4f596-2355-44a2-a040-791b666af7b4 -display none -no-user-config -nodefaults -device sga -chardev socket,id=charmonitor,path=/home/myself/.config/libvirt/qemu/lib/domain-2-guestfs-kedv5jvf5nz9/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=delay -no-hpet -no-reboot -no-acpi -boot strict=on -kernel /var/tmp/.guestfs-1004/appliance.d/kernel -initrd /var/tmp/.guestfs-1004/appliance.d/initrd -append panic=1 console=ttyS0 udevtimeout=6000 udev.event-timeout=6000 no_timer_check acpi=off printk.time=1 cgroup_disable=memory root=/dev/sdb selinux=0 TERM=xterm-256color -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x2 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x3 -drive file=/tmp/libguestfsBXEKjN/overlay1,format=qcow2,if=none,id=drive-scsi0-0-0-0,cache=unsafe -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1 -drive file=/tmp/libguestfsBXEKjN/overlay2,format=qcow2,if=none,id=drive-scsi0-0-1-0,cache=unsafe -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=1,lun=0,drive=drive-scsi0-0-1-0,id=scsi0-0-1-0 -chardev socket,id=charserial0,path=/tmp/libguestfsBXEKjN/console.sock -device isa-serial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,path=/tmp/libguestfsBXEKjN/guestfsd.sock -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.libguestfs.channel.0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4 -msg timestamp=on   

myself@mytux ~: virsh list
 Id    Name                           Status
----------------------------------------------------
 2     guestfs-kedv5jvf5nz9zv06       laufend

BUT: 

myself@mytux ~: virsh dominfo guestfs-kedv5jvf5nz9zv06
Id:             2
Name:           guestfs-kedv5jvf5nz9zv06
UUID:           b5d4f596-2355-44a2-a040-791b666af7b4
OS Typ:         hvm
Status:         laufend
CPU(s):         1
CPU-Zeit:       1,5s
Max Speicher:   512000 KiB
Verwendeter Speicher: 512000 KiB
Bleibend:       nein
Automatischer Start: deaktiviert
Verwaltete Sicherung: nein
Sicherheits-Modell: none
Sicherheits-DOI: 0

The security model is "non". 
(Its a German machine => Sicherheits-Modell = Security model.  )

=> Obviously Apparmor is ignored by the temporary minimal qemu machine used by libguestfs/guestmount. 

As the unprivileged user "myself" is not allowed to write to /etc/apparmor.d/libvirt no AppArmor profile files can be created there for the libguestfs qemu-machines. It would not help much either as the UUID of these  temporary machines would change all the time - and eventually the directory would be cluttered. (This can be seen when one tries to use guestmount as root with "export LIBGUESTFS_ATTACH_METHOD=libvirt" - however this leads to different kinds of trouble and guestmount aborts). 

The fact that AppArmor is ignored by libguestfs worries me : 

1) It indicates an inconsistent behavior of qemu with sVirt and AppArmor for different types of virtual qemu-machines (and resulting process owners of the qemu process) 

2) Due to the settings   
security_require_confined = 1
the start of the minimum qemu-machine for guestmount should have been prevented.  

Is this a bug? Do I do something wrong? Is there a remedy/workaround?

Regards 
Ralph 

----
 
Version-Release number of selected component (if applicable):
1.32

How reproducible:
Use guestmount (with libvirt for starting the virtual machine)

Steps to Reproduce: See above

Actual results: No security model chosen

Expected results: AppArmor should have been chosen

Comment 1 Richard W.M. Jones 2018-04-08 16:46:35 UTC
You may be right, but I've literally no idea how to fix this or even
what component should be responsible.  For SELinux/sVirt, we only support
that when using the 'libvirt' backend, and we do make a few small changes
to the XML (mainly to work around problems).  It may be for AppArmor
we also need to change the libvirt XML -- it's my understanding that
libvirt can use AppArmor when available.

Comment 2 Pino Toscano 2018-04-09 07:32:09 UTC
Which distribution is this?
Can you please list the exact versions of libguestfs and libvirt packages?

Comment 3 Ralph 2018-04-09 07:55:17 UTC
(In reply to Pino Toscano from comment #2)
> Which distribution is this?
> Can you please list the exact versions of libguestfs and libvirt packages?

Distribution is Opensuse Leap 42.3. Packages with version numbers there: 

guestfs-data  	1.32.4-10.10   (standard Leap 42.3 OSS repository)
guestfs-tools	1.32.4.10.10   (standard Leap 42.3 OSS repository)
libguestfs0     1.32.4.10.10   (standard Leap 42.3 OSS repository)

libvirt 	3.3.0-12.1     (standard Leap 42.3 Update Repository) 
The same for all other libvirt packages 

libapparnmor1 	2.10.3-16.1    (standard Leap 42.3 Update Repository) 
The same for all other apparmor packages 

Sorry, that I have no Red Hat distribution available at the site where the problem came up.  Hope the information helps nevertheless 

Regarding the standard KVM/QEMU-guest-machines (Kali 2018, Debian Jessie, Opensuse Leap 42.3) for testing the use of Apparmor security profiles  => created and started with virt-manager by standard unprivileged user.

Apparmor profile-files for the virtual machines are created in /etc/apparmor.d/libvirt/ the first time the guests are started. These profiles are used later on.   
This does not happen if an unprivileged user starts guestmount and the supermin QEMU-machine.

Comment 4 Ralph 2018-04-09 08:14:16 UTC
(In reply to Richard W.M. Jones from comment #1)
> You may be right, but I've literally no idea how to fix this or even
> what component should be responsible.  For SELinux/sVirt, we only support
> that when using the 'libvirt' backend, and we do make a few small changes
> to the XML (mainly to work around problems).  It may be for AppArmor
> we also need to change the libvirt XML -- it's my understanding that
> libvirt can use AppArmor when available.

It seems to be complicated to my very limited understanding. All parts (libguestfs, libvirt and apparmor) are involved somehow.   

The first remarkable thing is the user/UID who/wich is assigned to the the process for the QEMU-machine: 

When virt-manager/libvirt is used by an unprivileged user (who, however, is member of the libvirt group) to start a normal KVM/Qemu-guest-machine the user for the qemu-process is "qemu".  

In case of guestmount the started QEMU-machine is assigned to the UID of the unprivileged user. 

This shows already that the qemu-machines are handled differently. This may have an impact on rights and the required creation of an Apparmor profile for the qemu-machine.  

Another question is whether, where and how to create/delete an Apparmor profile for a qemu-machine which is only temporarily (!) used. And which has no permanent UUID. Should this be a kind of standard profile which is kept permanently ? Or should it be deleted again at the point when a libguestfs-tool is stopped ?

Comment 5 Richard W.M. Jones 2018-04-09 10:22:32 UTC
(In reply to Ralph from comment #4)
> The first remarkable thing is the user/UID who/wich is assigned to the the
> process for the QEMU-machine: 
> 
> When virt-manager/libvirt is used by an unprivileged user (who, however, is
> member of the libvirt group) to start a normal KVM/Qemu-guest-machine the
> user for the qemu-process is "qemu".  

This is because when you run virt-manager as non-root, it uses policykit
(or some similar mechanism) to elevate its libvirt privileges to root.

> In case of guestmount the started QEMU-machine is assigned to the UID of the
> unprivileged user. 

This is correct, and is how it should be.

The bug is that when we run libguestfs as root, libvirt runs qemu as a non-root
user (qemu.qemu) (https://bugzilla.redhat.com/show_bug.cgi?id=890291) instead of root.

> This shows already that the qemu-machines are handled differently. This may
> have an impact on rights and the required creation of an Apparmor profile
> for the qemu-machine.  

I don't think this conclusion is correct, as you are misunderstanding how
virt-manager / policykit works - see above.

Comment 6 Ralph 2018-04-09 12:11:47 UTC
(In reply to Richard W.M. Jones from comment #5)
> (In reply to Ralph from comment #4)

> 
> The bug is that when we run libguestfs as root, libvirt runs qemu as a
> non-root
> user (qemu.qemu) (https://bugzilla.redhat.com/show_bug.cgi?id=890291)
> instead of root.
> 
> > This shows already that the qemu-machines are handled differently. This may
> > have an impact on rights and the required creation of an Apparmor profile
> > for the qemu-machine.  
> 
> I don't think this conclusion is correct, as you are misunderstanding how
> virt-manager / policykit works - see above.

As I am no expert, yes, I probably misunderstand the privilege handling between a non-root user and libvirt. 

However, I agree that libvirt gets root privileges by whatever policy rules when I start a normal virtual machine (not a libguestfs-qemu-machine) via "virt-manager". It then assigns the QEMU process to user "qemu". 

Now back to guestmount. To avoid misunderstandings, I want to separate different aspects of the problem for different types of users more clearly: 

Assumptions for a test: 
The img-file W7_x64_ssdx.vmdk has access rights for everybody (777). It is not located in the /root-directory, but in /vmtest/sddx - which I made also fully accessible.  


Situation 1 - for user rmx: 
------------
rmx starts guestmount with (!) delegation of the start of the libguestfs qemu-machine to libvirt. From the options chosen for the qemu-machine one can see that really libvirt started the machine; the options are different in case  rmx does not delegate to libvirt. 
Then the resulting user for the qemu process is 'rmx' and not 'qemu'. This may be OK. However:

1) Apparmor is ignored by libvirt - despite security settings  in /etc/libvirt/qemu.conf. We end up with "security-model:none" for the libguestfs-qemu-machine. This is the real bug I am addressing. 
 
2) Consistently, no apparmor profile-file is written to /etc/apparmor.d/libvirt. (Due to whatever reasons - lack of privileges included)
      

Situation 2 - for user root: 
------------
root starts guestmount for the same img-file accessible to everybody and NOT even located in the /root-directory.
root delegates the start of the libguestfs-qemu-machine to libvirt (# export LIBGUESTFS_ATTACH_METHOD=libvirt). 

Then: 

1) An apparmor profile-file for the libguestfs-qemu-machine IS written into /etc/apparmor.d/libvirt. So, obviously, libvirt now follows the sec-settings in qemu.conf.  

2) However, the process for the start of the qemu-machine fails with the following error message: 
  
 # guestmount -a /vmtest/ssdx/W7_x64_ssdx.vmdk  -m /dev/sda2  --ro /mnt3    
libguestfs: error: could not create appliance through libvirt.                                                                                    
                                                                                                                                                  
Try running qemu directly without libvirt using this environment variable:                                                                        
export LIBGUESTFS_BACKEND=direct                                                                                                                  
                                                                                                                                                  
Original error from libvirt: internal error: process exited while connecting to monitor: 2018-04-09T11:49:06.055520Z qemu-system-x86_64: -drive file=/tmp/libguestfsCvfwxs/overlay1,format=qcow2,if=none,id=drive-scsi0-0-0-0,cache=unsafe: Could not open backing file: Could not open '/vmtest/ssdx/W7_x64_ssdx.vmdk': Permission denied [code=1 int1=-1] 
 
Which seems to be strange taking into account the file access rights - but probably other policies get hurt, too. Actually, I cannot see which user is chosen to start the qemu-process - it may be qemu.qemu in this case.   

I think what happens in situation 2 has indeed a lot to do with  https://bugzilla.redhat.com/show_bug.cgi?id=890291. 

But isn't the problem occurring in situation 1 not a bit different?