Bug 1984938 - qemu-nbd has no way to set the SELinux process label of the socket
Summary: qemu-nbd has no way to set the SELinux process label of the socket
Keywords:
Status: VERIFIED
Alias: None
Product: Red Hat Enterprise Linux 9
Classification: Red Hat
Component: qemu-kvm
Version: 9.0
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: beta
: ---
Assignee: Richard W.M. Jones
QA Contact: Tingting Mao
URL:
Whiteboard:
Depends On:
Blocks: TRACKER-bugs-affecting-libguestfs 2027636
TreeView+ depends on / blocked
 
Reported: 2021-07-22 14:01 UTC by Richard W.M. Jones
Modified: 2022-01-07 02:27 UTC (History)
13 users (show)

Fixed In Version: qemu-kvm-6.2.0-1.el9
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
Type: Bug
Target Upstream Version:


Attachments (Terms of Use)

Description Richard W.M. Jones 2021-07-22 14:01:20 UTC
Description of problem:

First thing to get out of the way is that Unix domain sockets
have two SELinux labels.  The one on the disk (fine - we can set
that).  And the different one in memory.  We cannot set the latter
when starting qemu-nbd, which prevents us from starting an NBD
server and attaching to it with SVirt enabled.

Version-Release number of selected component (if applicable):

qemu-kvm-6.0.0-1.el9.x86_64

How reproducible:

100%

Steps to Reproduce:

$ qemu-img create -f qcow2 disk.qcow2 1M
Formatting 'disk.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1048576 lazy_refcounts=off refcount_bits=16
$ rm -f sock
$ qemu-nbd -t -k $PWD/sock disk.qcow2 &
[1] 1692523
$ ls -l sock
srwxr-xr-x. 1 rjones rjones 0 Jul 22 14:56 sock

$ guestfish --format=raw -a 'nbd:///?socket=sock' run

This fails with an AVC:

type=AVC msg=audit(1626962282.736:26250): avc:  denied  { write } for  pid=1692658 comm="qemu-kvm" name="sock" dev="dm-0" ino=134296282 scontext=unconfined_u:unconfined_r:svirt_t:s0:c409,c747 tcontext=unconfined_u:object_r:user_tmp_t:s0 tclass=sock_file permissive=0

Now if we label the socket on disk (the first label):

$ chcon system_u:object_r:svirt_image_t:s0 sock
$ guestfish --format=raw -a 'nbd:///?socket=sock' run

This fails again with a different AVC:

type=AVC msg=audit(1626962339.750:26261): avc:  denied  { connectto } for  pid=1692711 comm="qemu-kvm" path="/var/tmp/sock" scontext=unconfined_u:unconfined_r:svirt_t:s0:c798,c879 tcontext=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 tclass=unix_stream_socket permissive=0

It's the second AVC which is the problem here.  There is
no way (except by modifying qemu-nbd) to set the second label
to avoid this AVC.

nbdkit has an --selinux-label option which lets you set the second
label.

Comment 1 Richard W.M. Jones 2021-07-22 15:03:34 UTC
Compare this to nbdkit which has a way to label the socket:

$ rm -f /tmp/sock
$ nbdkit -f -U /tmp/sock --selinux-label=system_u:object_r:svirt_socket_t:s0 memory 1M &
$ chcon system_u:object_r:svirt_image_t:s0 /tmp/sock
$ guestfish --format=raw -a 'nbd:///?socket=/tmp/sock' run
   <-- no error this time

Comment 2 Richard W.M. Jones 2021-07-22 16:46:52 UTC
Patch posted:
https://lists.nongnu.org/archive/html/qemu-block/2021-07/threads.html#00713

Comment 3 Richard W.M. Jones 2021-07-23 10:49:53 UTC
v2 patch:
https://lists.nongnu.org/archive/html/qemu-block/2021-07/threads.html#00719

Comment 4 Eric Blake 2021-09-30 19:24:24 UTC
v3 patch will be needed, after a failure to build in the pull request containing the v2 patch:
https://lists.gnu.org/archive/html/qemu-devel/2021-09/msg07547.html

Comment 5 Klaus Heinrich Kiwi 2021-10-05 21:35:20 UTC
(In reply to Eric Blake from comment #4)
> v3 patch will be needed, after a failure to build in the pull request
> containing the v2 patch:
> https://lists.gnu.org/archive/html/qemu-devel/2021-09/msg07547.html

Can that patch still make qemu 6.2? Otherwise we'll probably need to backport it for RHEL 9.0 GA

Comment 6 Eric Blake 2021-11-15 19:48:07 UTC
(In reply to Klaus Heinrich Kiwi from comment #5)
> (In reply to Eric Blake from comment #4)
> > v3 patch will be needed, after a failure to build in the pull request
> > containing the v2 patch:
> > https://lists.gnu.org/archive/html/qemu-devel/2021-09/msg07547.html
> 
> Can that patch still make qemu 6.2? Otherwise we'll probably need to
> backport it for RHEL 9.0 GA

Trying now to include it in the next rc build of 6.2

Comment 7 Eric Blake 2021-11-15 20:36:56 UTC
Latest upstream attempt:
https://lists.gnu.org/archive/html/qemu-devel/2021-11/msg03218.html

Comment 8 Eric Blake 2021-11-16 22:06:07 UTC
Made it into upstream qemu 6.2:
https://lists.gnu.org/archive/html/qemu-devel/2021-11/msg03468.html

Comment 10 Richard W.M. Jones 2021-11-22 16:39:33 UTC
This is the backport I did for Fedora:
https://src.fedoraproject.org/rpms/qemu/c/1609e9b0cde5035ea162567e5bf0146d7cb56e7e?branch=rawhide

Comment 12 Richard W.M. Jones 2021-12-17 10:06:39 UTC
Just doing the chcon command isn't enough, you have to also add
the qemu-nbd flag:

$ qemu-nbd --selinux-label system_u:object_r:svirt_socket_t:s0 -t -k $PWD/sock disk.qcow2 &
$ chcon system_u:object_r:svirt_image_t:s0 $PWD/sock 

Note you have to do both, because there are two labels on sockets.

See:

https://github.com/libguestfs/virt-v2v/blob/1673fc4b640f754fbdc942a6636fd0e95b996144/lib/nbdkit.ml#L143
https://github.com/libguestfs/virt-v2v/blob/1673fc4b640f754fbdc942a6636fd0e95b996144/lib/nbdkit.ml#L195

Comment 15 Tingting Mao 2021-12-20 10:07:53 UTC
Hi Richard again :)

Tried to reproduce this issues in  qemu-kvm-6.1.0-7.el9, but guestfish does not failed with AVC as comment0.

While in latest qemu-kvm-6.2.0-1.el9, user can indeed add '--selinux-label' to the qemu-nbd now.

Could you please help to check whether the steps are okay? Whether are the test results okay to set this bug as verified?

Thanks.



Tried to reproduce this bug in qemu-kvm-6.1.0-7.el9:
[root@dell-per740xd-01 bug]# su - tingting
[tingting@dell-per740xd-01 ~]$ qemu-img create -f qcow2 disk.qcow2 1M
Formatting 'disk.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1048576 lazy_refcounts=off refcount_bits=16
[tingting@dell-per740xd-01 ~]$ qemu-nbd -t -k $PWD/sock disk.qcow2 &
[1] 8616
[tingting@dell-per740xd-01 ~]$ ls -lZ sock 
srwxr-xr-x 1 tingting tingting ? 0 Dec 20 04:49 sock
[tingting@dell-per740xd-01 ~]$ guestfish --format=raw -a 'nbd:///?socket=sock' run ---------------------------------------> No errors
[tingting@dell-per740xd-01 ~]$ echo $?   
0
[tingting@dell-per740xd-01 ~]$ chcon system_u:object_r:svirt_image_t:s0 sock
[tingting@dell-per740xd-01 ~]$ guestfish --format=raw -a 'nbd:///?socket=sock' run -------------------------------------> No errors
[tingting@dell-per740xd-01 ~]$ echo $?
0



Tried in latest qemu-kvm-6.2.0-1.el9

[root@dell-per740xd-01 bug]# su - tingting
[tingting@dell-per740xd-01 ~]$ qemu-img create -f qcow2 disk.qcow2 1M
Formatting 'disk.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1048576 lazy_refcounts=off refcount_bits=16
[tingting@dell-per740xd-01 ~]$ qemu-nbd --selinux-label system_u:object_r:svirt_socket_t:s0 -t -k $PWD/sock disk.qcow2 &
[1] 9273
[tingting@dell-per740xd-01 ~]$ ls -lZ sock 
srwxr-xr-x 1 tingting tingting ? 0 Dec 20 04:55 sock
[tingting@dell-per740xd-01 ~]$ guestfish --format=raw -a 'nbd:///?socket=sock' run

[tingting@dell-per740xd-01 ~]$ 
[tingting@dell-per740xd-01 ~]$ echo $?
0
[tingting@dell-per740xd-01 ~]$ chcon system_u:object_r:svirt_image_t:s0 $PWD/sock
[tingting@dell-per740xd-01 ~]$ guestfish --format=raw -a 'nbd:///?socket=sock' run
[tingting@dell-per740xd-01 ~]$ echo $?
0

Comment 16 Richard W.M. Jones 2021-12-20 10:52:11 UTC
I feel this is probably not enough to verify this bug unfortunately, unless
you just want to do a sanity check verification that qemu-nbd has the new flag.
Really we should be reproducing the original virt-v2v failure when SELinux
is enabled.  I'm on holiday at the moment but can look at this in January.

Comment 17 Yanan Fu 2021-12-20 12:44:41 UTC
QE bot(pre verify): Set 'Verified:Tested,SanityOnly' as gating/tier1 test pass.

Comment 21 Richard W.M. Jones 2022-01-05 10:34:34 UTC
Here is how to reproduce the bug and verify the fix.

With:
  libguestfs-1.46.1-2.el9.x86_64
  libnbd-1.10.2-1.el9.x86_64
  nbdkit-1.28.4-1.el9.x86_64
  qemu-img-6.1.0-8.el9.x86_64
  virt-v2v-1.45.95-3.el9.x86_64

Reproduce the bug like this:

  $ virt-v2v -i disk fedora-35.qcow2 -o null
  $ virt-v2v -i disk fedora-35.qcow2 -o null
  [   1.0] Opening the source
  virt-v2v: error: libguestfs error: could not create appliance through 
  libvirt.

  Try running qemu directly without libvirt using this environment variable:
  export LIBGUESTFS_BACKEND=direct

  Original error from libvirt: internal error: process exited while 
  connecting to monitor: 2022-01-05T10:28:45.617384Z qemu-kvm: -blockdev 
  {"driver":"nbd","server":{"type":"unix","path":"/tmp/v2v.c7XAl6/in0"},"node-name":"libvirt-2-storage","cache":{"direct":false,"no-flush":true},"auto-read-only":true,"discard":"unmap"}: Failed to connect to '/tmp/v2v.c7XAl6/in0': Permission denied [code=1 int1=-1]

Note that we get permission denied connecting to the input socket ("in0"),
and in the SELinux logs there is an SELinux alert connecting to the
same socket:

  $ sudo ausearch -m avc -ts recent
  ----
  time->Wed Jan  5 10:28:45 2022
  type=AVC msg=audit(1641378525.615:17938): avc:  denied  { connectto } for  pid=2495557 comm="nbd-connect" path="/tmp/v2v.c7XAl6/in0" scontext=unconfined_u:unconfined_r:svirt_t:s0:c878,c886 tcontext=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 tclass=unix_stream_socket permissive=0

Now upgrading just qemu-kvm to:
  qemu-img-6.2.0-1.el9.x86_64

and the same virt-v2v command will work:

  $ virt-v2v -i disk fedora-35.qcow2 -o null
  [   1.0] Opening the source
  [   6.6] Inspecting the source
  [   8.2] Checking for sufficient free disk space in the guest
  etc...

Also while virt-v2v is running you can look at qemu-nbd and see that the --selinux-label
flag is passed to it:

  $ ps ax | grep qemu-nbd
  2496224 pts/0    Sl+    0:00 qemu-nbd -t --pid-file /run/user/1000/v2vqemunbd.URaIKo/qemunbd1.pid --socket /tmp/v2v.fgJ9Tf/in0 -s --selinux-label system_u:object_r:svirt_socket_t:s0 --format qcow2 fedora-35.qcow2

Also there should be no SELinux alerts.

Comment 22 Richard W.M. Jones 2022-01-06 10:39:14 UTC
(In reply to Richard W.M. Jones from comment #21)
>   $ virt-v2v -i disk fedora-35.qcow2 -o null

Sorry, I cut and pasted the wrong command.  The first command should be:

  $ virt-builder fedora-35 --format=qcow2

The virt-v2v command below is correct:

>   $ virt-v2v -i disk fedora-35.qcow2 -o null
>   [   1.0] Opening the source
>   virt-v2v: error: libguestfs error: could not create appliance through 
>   libvirt.

Note the format must be qcow2 because that causes virt-v2v to use
qemu-nbd (instead of nbdkit), which causes the bug because qemu-nbd
didn't set the right label before.

Comment 23 mxie@redhat.com 2022-01-06 15:05:35 UTC
Reproduce the bug with below builds:
qemu-img-6.1.0-8.el9.x86_64
virt-v2v-1.45.95-3.el9.x86_64
libguestfs-1.46.1-2.el9.x86_64
nbdkit-server-1.28.3-3.el9.x86_64
libvirt-libs-7.10.0-1.el9.x86_64

Steps to reproduce:
1. Switch user from root to regular user and enable selinux
# su - mxie

$ getenforce
Enforcing

2.Convert a guest from disk by v2v
$ virt-v2v -i disk /home/RHEL-8.6-x86_64-latest.qcow2 -o null
[   1.1] Opening the source
virt-v2v: error: libguestfs error: could not create appliance through 
libvirt.

Try running qemu directly without libvirt using this environment variable:
export LIBGUESTFS_BACKEND=direct

Original error from libvirt: internal error: process exited while 
connecting to monitor: 2022-01-06T14:53:13.614462Z qemu-kvm: -blockdev 
{"driver":"nbd","server":{"type":"unix","path":"/tmp/v2v.87N5Pr/in0"},"node-name":"libvirt-2-storage","cache":{"direct":false,"no-flush":true},"auto-read-only":true,"discard":"unmap"}: 
Failed to connect to '/tmp/v2v.87N5Pr/in0': Permission denied [code=1 
int1=-1]

If reporting bugs, run virt-v2v with debugging enabled and include the 
complete output:

  virt-v2v -v -x [...]

3.Check the avc error, which is same with comment21
$  sudo ausearch -m avc -ts recent
......
time->Thu Jan  6 09:53:13 2022
type=PROCTITLE msg=audit(1641480793.612:5871): proctitle=2F7573722F6C6962657865632F71656D752D6B766D002D6E616D650067756573743D677565737466732D6161623235637839697872306C6831342C64656275672D746872656164733D6F6E002D53002D6F626A656374007B22716F6D2D74797065223A22736563726574222C226964223A226D61737465724B657930222C2266
type=SYSCALL msg=audit(1641480793.612:5871): arch=c000003e syscall=42 success=no exit=-13 a0=d a1=7fbf5bffff68 a2=6e a3=5f items=0 ppid=1 pid=175889 auid=0 uid=1001 gid=1001 euid=1001 suid=1001 fsuid=1001 egid=1001 sgid=1001 fsgid=1001 tty=(none) ses=53 comm="nbd-connect" exe="/usr/libexec/qemu-kvm" subj=unconfined_u:unconfined_r:svirt_t:s0:c533,c990 key=(null)
type=AVC msg=audit(1641480793.612:5871): avc:  denied  { connectto } for  pid=175889 comm="nbd-connect" path="/tmp/v2v.87N5Pr/in0" scontext=unconfined_u:unconfined_r:svirt_t:s0:c533,c990 tcontext=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 tclass=unix_stream_socket permissive=0




Verify the bug with below builds:
qemu-img-6.2.0-1.el9.x86_64
virt-v2v-1.45.95-3.el9.x86_64
libguestfs-1.46.1-2.el9.x86_64
nbdkit-server-1.28.3-3.el9.x86_64
libvirt-libs-7.10.0-1.el9.x86_64

Steps:
1. Switch user from root to regular user and enable selinux
# su - mxie

$ getenforce
Enforcing

2.Convert a guest from disk by v2v, conversion can finish without error
$ virt-v2v -i disk /home/RHEL-8.6-x86_64-latest.qcow2 -o null
[   1.1] Opening the source
[   7.8] Inspecting the source
[  17.3] Checking for sufficient free disk space in the guest
[  17.3] Converting Red Hat Enterprise Linux 8.6 Beta (Ootpa) to run on KVM
virt-v2v: warning: /files/boot/grub2/device.map/hd0 references unknown 
device "vda".  You may have to fix this entry manually after conversion.
virt-v2v: This guest has virtio drivers installed.
[  58.9] Mapping filesystem data to avoid copying unused and blank areas
[  61.5] Closing the overlay
[  61.7] Assigning disks to buses
[  61.7] Checking if the guest needs BIOS or UEFI to boot
[  62.8] Copying disk 1/1
█ 100% [****************************************]
[  68.8] Creating output metadata
[  68.8] Finishing off

3.Check the qemu-nbd process during v2v conversion
$ ps ax | grep qemu-nbd
 176440 pts/0    Sl+    0:13 qemu-nbd -t --pid-file /tmp/v2vqemunbd.i3jF1a/qemunbd1.pid --socket /tmp/v2v.ATRkEo/in0 -s --selinux-label system_u:object_r:svirt_socket_t:s0 --format qcow2 /home/RHEL-8.6-x86_64-latest.qcow2

Comment 24 Tingting Mao 2022-01-07 02:22:03 UTC
Thanks mxie, and set this bug as verified according to comment 23.


Note You need to log in before you can comment on or make changes to this bug.