Description of problem: [cgroup2] When detach a device from vm, the device entry not removed from vm's bpf map Version-Release number of selected component (if applicable): libvirt-6.0.0-7.module+el8.2.0+5869+c23fe68b.x86_64 systemd-239-23.el8.x86_64 How reproducible: 100% Steps to Reproduce: 0. Enable cgorup v2 1. [root@hp-dl320eg8-05 bz1717396]# virsh start vm1 Domain vm1 started [root@hp-dl320eg8-05 bz1717396]# bpftool cgroup list /sys/fs/cgroup/machine.slice/machine-qemu\\x2d2\\x2dvm1.scope/ ID AttachType AttachFlags Name 18 device [root@hp-dl320eg8-05 bz1717396]# bpftool prog show id 18 18: cgroup_device tag b95e404d31962705 gpl loaded_at 2020-03-03T23:33:56-0500 uid 0 xlated 616B jited 350B memlock 4096B map_ids 18 [root@hp-dl320eg8-05 bz1717396]# bpftool map dump id 18 key: 07 00 00 00 01 00 00 00 value: 02 00 06 00 key: ff ff ff ff 88 00 00 00 value: 02 00 06 00 key: 03 00 00 00 01 00 00 00 value: 02 00 06 00 key: 02 00 00 00 05 00 00 00 value: 02 00 06 00 key: 09 00 00 00 01 00 00 00 value: 02 00 06 00 key: 00 00 00 00 fb 00 00 00 value: 02 00 06 00 key: 08 00 00 00 01 00 00 00 value: 02 00 06 00 key: 90 00 00 00 41 00 00 00 value: 01 00 06 00 key: 05 00 00 00 01 00 00 00 value: 02 00 06 00 key: e8 00 00 00 0a 00 00 00 value: 02 00 06 00 Found 10 elements [root@hp-dl320eg8-05 bz1717396]# cat disk.xml <disk type='block' device='disk'> <driver name='qemu' type='raw'/> <source dev='/dev/sdbb'/> <target dev='vdbd' bus='virtio'/> </disk> [root@hp-dl320eg8-05 bz1717396]# virsh attach-device vm1 disk.xml Device attached successfully [root@hp-dl320eg8-05 bz1717396]# bpftool map dump id 18 key: 07 00 00 00 01 00 00 00 value: 02 00 06 00 key: ff ff ff ff 88 00 00 00 value: 02 00 06 00 key: 03 00 00 00 01 00 00 00 value: 02 00 06 00 key: 02 00 00 00 05 00 00 00 value: 02 00 06 00 key: 09 00 00 00 01 00 00 00 value: 02 00 06 00 key: 00 00 00 00 fb 00 00 00 value: 02 00 06 00 key: 08 00 00 00 01 00 00 00 value: 02 00 06 00 key: 90 00 00 00 41 00 00 00 value: 01 00 06 00 key: 50 00 00 00 43 00 00 00 value: 01 00 06 00 key: 05 00 00 00 01 00 00 00 value: 02 00 06 00 key: e8 00 00 00 0a 00 00 00 value: 02 00 06 00 Found 11 elements <====== NEW ENTRY IS - key: 50 00 00 00 43 00 00 00 value: 01 00 06 00 [root@hp-dl320eg8-05 bz1717396]# python get_bpf_map.py /sys/fs/cgroup/machine.slice/machine-qemu\\x2d2\\x2dvm1.scope/ c 1:7 rw full c 136:* rw c 1:3 rw null c 5:2 rw ptmx c 1:9 rw urandom c 251:0 rw rtc0 c 1:8 rw random b 65:144 rw sdz b 67:80 rw sdbb c 1:5 rw zero c 10:232 rw kvm [root@hp-dl320eg8-05 bz1717396]# virsh detach-device vm1 disk.xml Device detached successfully [root@hp-dl320eg8-05 bz1717396]# bpftool map dump id 18 key: 07 00 00 00 01 00 00 00 value: 02 00 06 00 key: ff ff ff ff 88 00 00 00 value: 02 00 06 00 key: 03 00 00 00 01 00 00 00 value: 02 00 06 00 key: 02 00 00 00 05 00 00 00 value: 02 00 06 00 key: 09 00 00 00 01 00 00 00 value: 02 00 06 00 key: 00 00 00 00 fb 00 00 00 value: 02 00 06 00 key: 08 00 00 00 01 00 00 00 value: 02 00 06 00 key: 90 00 00 00 41 00 00 00 value: 01 00 06 00 key: 50 00 00 00 43 00 00 00 value: 00 00 00 00 key: 05 00 00 00 01 00 00 00 value: 02 00 06 00 key: e8 00 00 00 0a 00 00 00 value: 02 00 06 00 Found 11 elements <===== Not back to 10 elements [root@hp-dl320eg8-05 bz1717396]# python get_bpf_map.py /sys/fs/cgroup/machine.slice/machine-qemu\\x2d2\\x2dvm1.scope/ c 1:7 rw full c 136:* rw c 1:3 rw null c 5:2 rw ptmx c 1:9 rw urandom c 251:0 rw rtc0 c 1:8 rw random b 65:144 rw sdz None 67:80 c 1:5 rw zero c 10:232 rw kvm <==== Now the major:minor==67:80 device still exists in map but value has been set to all zeros That means, when dump map id=18 following entry still exists, but the value is erase to all zeros FROM: # bpftool map dump id 18 key: 50 00 00 00 43 00 00 00 value: 01 00 06 00 ... TO: # bpftool map dump id 18 key: 50 00 00 00 43 00 00 00 value: 00 00 00 00 ... Actual results: Device still exists in vm's bpf map after detachment. Expected results: Device entry removed when detach it from vm Additional info: This is a minor issue when testing bz1717396, the python file used here can be found in that bug's attachment.
Upstream commit: commit 0cbcd21b1f981dcca7879130a6653725db87d33a Author: Pavel Hrdina <phrdina> Date: Wed Nov 25 17:21:49 2020 +0100 vircgroupv2: fix virCgroupV2DenyDevice
Tested with libvirt-7.0.0-1.module+el8.4.0+9464+3e71831a.x86_64 result is : PASS 1. Set selinux to permissive to make sure bpftool can get ebpf info [root@dell-per740-39 ~]# setenforce 0 2. Get the bpf map info for running vm (vm1) [root@dell-per740-39 ~]# bpftool cgroup list /sys/fs/cgroup/machine.slice/machine-qemu\\x2d1\\x2dvm1.scope/ ID AttachType AttachFlags Name 149 device <== vm's eBPF program id is 149 [root@dell-per740-39 ~]# bpftool prog show id 149 149: cgroup_device tag b95e404d31962705 gpl loaded_at 2021-01-17T03:13:31-0500 uid 0 xlated 616B jited 347B memlock 4096B map_ids 149 <== vm's eBPF map id is 149 [root@dell-per740-39 ~]# bpftool map dump id 149 key: 03 00 00 00 01 00 00 00 value: 02 00 06 00 key: 05 00 00 00 01 00 00 00 value: 02 00 06 00 key: 07 00 00 00 01 00 00 00 value: 02 00 06 00 key: 09 00 00 00 01 00 00 00 value: 02 00 06 00 key: ff ff ff ff 88 00 00 00 value: 02 00 06 00 key: 02 00 00 00 05 00 00 00 value: 02 00 06 00 key: e8 00 00 00 0a 00 00 00 value: 02 00 06 00 key: 08 00 00 00 01 00 00 00 value: 02 00 06 00 Found 8 elements <== Here is 8 elements now, means vm has access right to 8 devices 3. prepare a block device on host, here we use targetcli to prepare a iscsi device (/dev/sdb) [root@dell-per740-39 ~]# lsscsi [0:2:0:0] disk DELL PERC H730P Adp 4.30 /dev/sda [15:0:0:0] disk LIO-ORG device.logical- 4.0 /dev/sdb 4. Attach the deivce to vm [root@dell-per740-39 ~]# cat disk.xml <disk type='block' device='disk'> <driver name='qemu' type='raw'/> <source dev='/dev/sdb'/> <target dev='vdbd' bus='virtio'/> </disk> [root@dell-per740-39 ~]# virsh attach-device vm1 disk.xml Device attached successfully 5. Check the bpf map [root@dell-per740-39 ~]# bpftool map dump id 149 key: 03 00 00 00 01 00 00 00 value: 02 00 06 00 key: 10 00 00 00 08 00 00 00 value: 01 00 06 00 <==== newly attached device key: 05 00 00 00 01 00 00 00 value: 02 00 06 00 key: 07 00 00 00 01 00 00 00 value: 02 00 06 00 key: 09 00 00 00 01 00 00 00 value: 02 00 06 00 key: ff ff ff ff 88 00 00 00 value: 02 00 06 00 key: 02 00 00 00 05 00 00 00 value: 02 00 06 00 key: e8 00 00 00 0a 00 00 00 value: 02 00 06 00 key: 08 00 00 00 01 00 00 00 value: 02 00 06 00 Found 9 elements <== now we found 9 elements, and the newly added one is 'key: 10 00 00 00 08 00 00 00 value: 01 00 06 00' 6. Detach the device from vm [root@dell-per740-39 ~]# virsh detach-device vm1 disk.xml Device detached successfully 7. Check the bpf map [root@dell-per740-39 ~]# bpftool map dump id 149 key: 03 00 00 00 01 00 00 00 value: 02 00 06 00 key: 05 00 00 00 01 00 00 00 value: 02 00 06 00 key: 07 00 00 00 01 00 00 00 value: 02 00 06 00 key: 09 00 00 00 01 00 00 00 value: 02 00 06 00 key: ff ff ff ff 88 00 00 00 value: 02 00 06 00 key: 02 00 00 00 05 00 00 00 value: 02 00 06 00 key: e8 00 00 00 0a 00 00 00 value: 02 00 06 00 key: 08 00 00 00 01 00 00 00 value: 02 00 06 00 Found 8 elements <==== As expected, the entry 'key: 10 00 00 00 08 00 00 00 value: 01 00 06 00' removed from the bpf map
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (virt:av bug fix and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2021:2098