Description of problem: Failed to start guest with tpm passthroughed when remember_owner = 1 Version-Release number of selected component (if applicable): libvirt-daemon-5.6.0-6.module+el8.1.0+4244+9aa4e6bb.x86_64 qemu-kvm-4.1.0-12.module+el8.1.0+4292+1293ae0c.x86_64 tpm2-tools-3.1.4-5.el8.x86_64 tpm2-tss-2.0.0-4.el8.x86_64 How reproducible: 100% Steps to Reproduce: 1. Prepare a host with a tpm2.0 chip, and enable it in bios. 2. Try to start a guest with tpm passthrough: <tpm model='tpm-crb'> <backend type='passthrough'> <device path='/dev/tpm0'/> </backend> <alias name='tpm0'/> </tpm> # virsh start avocado-vt-vm1 error: Failed to start domain avocado-vt-vm1 error: internal error: child reported (status=125): unable to open /dev/tpm0: Device or resource busy 2019-09-26 08:41:30.842+0000: 4291: error : virProcessRunInFork:1170 : internal error: child reported (status=125): unable to open /dev/tpm0: Device or resource busy 2019-09-26 08:41:30.847+0000: 4291: error : virProcessRunInFork:1170 : internal error: child reported (status=125): unable to open /dev/tpm0: Device or resource busy Actual results: Expected results: Additional info: Guest can be started successfully when set remember_owner = 0.
Supplementary info for comment0: The parameter is set in /etc/libvirt/qemu.conf: # Whether libvirt should remember and restore the original # ownership over files it is relabeling. Defaults to 1, set # to 0 to disable the feature. #remember_owner = 1
Patches posted upstream: https://www.redhat.com/archives/libvir-list/2019-October/msg00047.html
Patches pushed upstream: 4e95cdcbb3 security: Don't remember labels for TPM 2b44cf8c32 security_dac: Allow selective remember/recall for chardevs 1a84a1ced1 security: Try to lock only paths with remember == true v5.8.0-80-g4e95cdcbb3
Verify on: libvirt-daemon-6.0.0-1.module+el8.2.0+5453+31b2b136.x86_64 qemu-kvm-4.2.0-6.module+el8.2.0+5453+31b2b136.x86_64 Steps: Scenaro1: Keep default value for remember_owner 1. # grep remember_owner -i /etc/libvirt/qemu.conf #remember_owner = 1 2. prepare tpm2.0 chip on host: # ll -Z /dev/tpm0 crw-------. 1 root root system_u:object_r:tpm_device_t:s0 10, 224 Jan 19 06:07 /dev/tpm0 # getfattr -m trusted.libvirt.security -d /dev/tpm0 # tpm2_getrandom 5 0x5C 0xAB 0xD9 0x39 0x30 3. Start guest with tpm passthroughed: # virsh start rhel8.1-ovmf Domain rhel8.1-ovmf started # tpm2_getrandom 5 ERROR:tcti:src/tss2-tcti/tcti-device.c:281:Tss2_Tcti_Device_Init() Failed to open device file /dev/tpm0: Device or resource busy ERROR: tcti init allocation routine failed for library: "device" options: "(null)" ERROR: Could not load tcti, got: "device" # virsh console rhel8.1-ovmf Connected to domain rhel8.1-ovmf Escape character is ^] Red Hat Enterprise Linux 8.2 Beta (Ootpa) Kernel 4.18.0-167.el8.x86_64 on an x86_64 [root@localhost ~]# tpm2_getrandom 5 0x99 0x31 0x66 0x53 0x5C 4. Shutdown the guest: # virsh shutdown rhel8.1-ovmf Domain rhel8.1-ovmf is being shutdown # tpm2_getrandom 5 0x72 0x27 0x8A 0x84 0x99 During this period, the output of 'll -Z' and 'getfattr' never changed. Scenario2: set remember_owner = 1 # grep remember_owner -i /etc/libvirt/qemu.conf remember_owner = 1 # systemctl restart libvirtd Got same result. Scenario2: set remember_owner = 0 # grep remember_owner -i /etc/libvirt/qemu.conf remember_owner = 0 # systemctl restart libvirtd Got same result.
Chardev regression test: Pkgs: libvirt-daemon-6.0.0-3.module+el8.2.0+5633+b0e06c1a.x86_64 qemu-kvm-4.2.0-8.module+el8.2.0+5607+dc756904.x86_64 Steps: S1. Serial: pty pci-serial <serial type='pty'> <source path='/dev/pts/3'/> <target type='pci-serial' port='0'> <model name='pci-serial'/> </target> <alias name='ua-d830c2c4-93ac-4eb7-b714-593483f10000'/> </serial> 1. start guest # virsh start avocado-vt-vm1 Domain avocado-vt-vm1 started # virsh dumpxml avocado-vt-vm1 |grep 'serial t' -A6 <serial type='pty'> <source path='/dev/pts/1'/> <target type='pci-serial' port='0'> <model name='pci-serial'/> </target> <alias name='ua-d830c2c4-93ac-4eb7-b714-593483f10000'/> <address type='pci' domain='0x0000' bus='0x09' slot='0x01' function='0x0'/> -- # ll -Z /dev/pts/1; getfattr -m trusted.libvirt.security -d /dev/pts/1 crw--w----. 1 qemu tty system_u:object_r:svirt_devpts_t:s0 136, 1 Feb 5 03:09 /dev/pts/1 Communication works well: [host]# cat /dev/pts/1 [guest]# echo abc123 > /dev/ttyS1 [host]# cat /dev/pts/1 abc123 2. hotplug and unplug # virsh attach-device avocado-vt-vm1 serial-pty.xml Device attached successfully # virsh dumpxml avocado-vt-vm1 |grep 'serial t' -A6 <serial type='pty'> <source path='/dev/pts/1'/> <target type='pci-serial' port='0'> <model name='pci-serial'/> </target> <alias name='ua-d830c2c4-93ac-4eb7-b714-593483f10000'/> <address type='pci' domain='0x0000' bus='0x09' slot='0x01' function='0x0'/> -- … <serial type='pty'> <source path='/dev/pts/4'/> <target type='pci-serial' port='0'> <model name='pci-serial'/> </target> <alias name='ua-d830c2c4-93ac-4eb7-b714-593483f10001'/> <address type='pci' domain='0x0000' bus='0x09' slot='0x02' function='0x0'/> # ll -Z /dev/pts/4; getfattr -m trusted.libvirt.security -d /dev/pts/4 crw--w----. 1 qemu tty system_u:object_r:svirt_devpts_t:s0 136, 4 Feb 5 03:21 /dev/pts/4 Communication also works well: # cat /dev/pts/4 [guest]# echo hello! > /dev/ttyS2 [host]# cat /dev/pts/4 hello! # virsh detach-device avocado-vt-vm1 serial-pty.xml Device detached successfully # ll -Z /dev/pts/4; getfattr -m trusted.libvirt.security -d /dev/pts/4 ls: cannot access '/dev/pts/4': No such file or directory getfattr: /dev/pts/4: No such file or directory 3. restart libvirtd # systemctl restart libvirtd # ll -Z /dev/pts/1; getfattr -m trusted.libvirt.security -d /dev/pts/1 crw--w----. 1 qemu tty system_u:object_r:svirt_devpts_t:s0 136, 1 Feb 5 03:13 /dev/pts/1 4.shutdown guest # virsh shutdown avocado-vt-vm1 Domain avocado-vt-vm1 is being shutdown # ll -Z /dev/pts/1; getfattr -m trusted.libvirt.security -d /dev/pts/1 ls: cannot access '/dev/pts/1': No such file or directory getfattr: /dev/pts/1: No such file or directory Special need be mentioned: file can only be auto generated when guest start, edit assign will dispear in xml; no xattr for the file; file will be auto cleaned when vm not running. S2. Console: unix bind <console type='unix'> <source mode='bind' path='/var/lib/libvirt/qemu/foo'/> <target type='virtio' port='6'/> <alias name='ua-c769f8ac-922a-4518-9946-2a44172d7407'/> </console> 1. start guest: # virsh start avocado-vt-vm1 Domain avocado-vt-vm1 started # ll -Z /var/lib/libvirt/qemu/foo; getfattr -m trusted.libvirt.security -d /var/lib/libvirt/qemu/foo srwxrwxr-x. 1 qemu qemu system_u:object_r:qemu_var_run_t:s0 0 Feb 5 04:42 /var/lib/libvirt/qemu/foo getfattr: Removing leading '/' from absolute path names # file: var/lib/libvirt/qemu/foo trusted.libvirt.security.dac="+0:+0" trusted.libvirt.security.ref_dac="1" trusted.libvirt.security.timestamp_dac="1580718107" # virsh dumpxml avocado-vt-vm1 |grep 'console t' -A6 ... <console type='unix'> <source mode='bind' path='/var/lib/libvirt/qemu/foo'/> <target type='virtio' port='1'/> <alias name='ua-c769f8ac-922a-4518-9946-2a44172d7407'/> </console> [guest]# echo hello > /dev/hvc0 [host]# nc -U /var/lib/libvirt/qemu/foo hello 2. hotplug unplug: # virsh attach-device avocado-vt-vm1 console-unix.xml Device attached successfully # virsh dumpxml avocado-vt-vm1 |grep 'console t' -A6 ... <console type='unix'> <source mode='bind' path='/var/lib/libvirt/qemu/foo'/> <target type='virtio' port='1'/> <alias name='ua-c769f8ac-922a-4518-9946-2a44172d7407'/> </console> <console type='unix'> <source mode='bind' path='/var/lib/libvirt/qemu/foo2'/> <target type='virtio' port='6'/> <alias name='ua-c769f8ac-922a-4518-9946-2a44172d7408'/> </console> # ll -Z /var/lib/libvirt/qemu/foo2; getfattr -m trusted.libvirt.security -d /var/lib/libvirt/qemu/foo2 srwxrwxr-x. 1 qemu qemu system_u:object_r:qemu_var_run_t:s0 0 Feb 5 05:05 /var/lib/libvirt/qemu/foo2 # virsh detach-device avocado-vt-vm1 console-unix.xml Device detached successfully [guest]# echo abc123 > /dev/hvc1 [host]# nc -U /var/lib/libvirt/qemu/foo2 abc123 # ll -Z /var/lib/libvirt/qemu/foo2; getfattr -m trusted.libvirt.security -d /var/lib/libvirt/qemu/foo2 ls: cannot access '/var/lib/libvirt/qemu/foo2': No such file or directory getfattr: /var/lib/libvirt/qemu/foo2: No such file or directory 3. restart libvirtd # systemctl restart libvirtd # ll -Z /var/lib/libvirt/qemu/foo; getfattr -m trusted.libvirt.security -d /var/lib/libvirt/qemu/foo srwxrwxr-x. 1 qemu qemu system_u:object_r:qemu_var_run_t:s0 0 Feb 5 05:14 /var/lib/libvirt/qemu/foo getfattr: Removing leading '/' from absolute path names # file: var/lib/libvirt/qemu/foo trusted.libvirt.security.dac="+0:+0" trusted.libvirt.security.ref_dac="1" trusted.libvirt.security.timestamp_dac="1580718107" 4.shutdown # virsh shutdown avocado-vt-vm1 Domain avocado-vt-vm1 is being shutdown # ll -Z /var/lib/libvirt/qemu/foo; getfattr -m trusted.libvirt.security -d /var/lib/libvirt/qemu/foo ls: cannot access '/var/lib/libvirt/qemu/foo': No such file or directory getfattr: /var/lib/libvirt/qemu/foo: No such file or directory Special need mention: file will disappear when vm not running. no xattr for hotplugged one. S3: Channel: unix bind <channel type='unix'> <source mode='bind' path='/var/lib/libvirt/qemu/test.agent'/> <target type='virtio' name='unixvirtio' state='disconnected'/> <alias name='ua-d830c2c4-93ac-4eb7-b714-593483f10007'/> </channel> 1. start guest # virsh start avocado-vt-vm1 Domain avocado-vt-vm1 started # ll -Z /var/lib/libvirt/qemu/test.agent; getfattr -m trusted.libvirt.security -d /var/lib/libvirt/qemu/test.agent srwxrwxr-x. 1 qemu qemu system_u:object_r:qemu_var_run_t:s0 0 Feb 5 05:32 /var/lib/libvirt/qemu/test.agent getfattr: Removing leading '/' from absolute path names # file: var/lib/libvirt/qemu/test.agent trusted.libvirt.security.dac="+0:+0" trusted.libvirt.security.ref_dac="1" trusted.libvirt.security.timestamp_dac="1580718107" # virsh dumpxml avocado-vt-vm1 |grep 'channel t' -A6 ... <channel type='unix'> <source mode='bind' path='/var/lib/libvirt/qemu/test.agent'/> <target type='virtio' name='unixvirtio' state='disconnected'/> <alias name='ua-d830c2c4-93ac-4eb7-b714-593483f10007'/> <address type='virtio-serial' controller='0' bus='0' port='2'/> </channel> [guest]# echo qwer > /dev/vport1p2 [host]# nc -U /var/lib/libvirt/qemu/test.agent qwer 2. hotplug unplug: # virsh attach-device avocado-vt-vm1 channel-unix.xml Device attached successfully # ll -Z /var/lib/libvirt/qemu/test2.agent; getfattr -m trusted.libvirt.security -d /var/lib/libvirt/qemu/test2.agent srwxrwxr-x. 1 qemu qemu system_u:object_r:qemu_var_run_t:s0 0 Feb 5 06:08 /var/lib/libvirt/qemu/test2.agent [guest]# echo qwer2 > /dev/vport1p3 [host]# nc -U /var/lib/libvirt/qemu/test2.agent qwer2 # virsh detach-device avocado-vt-vm1 channel-unix.xml Device detached successfully # ll -Z /var/lib/libvirt/qemu/test2.agent; getfattr -m trusted.libvirt.security -d /var/lib/libvirt/qemu/test2.agent ls: cannot access '/var/lib/libvirt/qemu/test2.agent': No such file or directory getfattr: /var/lib/libvirt/qemu/test2.agent: No such file or directory 3. restart libvirtd # systemctl restart libvirtd # ll -Z /var/lib/libvirt/qemu/test.agent; getfattr -m trusted.libvirt.security -d /var/lib/libvirt/qemu/test.agent srwxrwxr-x. 1 qemu qemu system_u:object_r:qemu_var_run_t:s0 0 Feb 5 05:32 /var/lib/libvirt/qemu/test.agent getfattr: Removing leading '/' from absolute path names # file: var/lib/libvirt/qemu/test.agent trusted.libvirt.security.dac="+0:+0" trusted.libvirt.security.ref_dac="1" trusted.libvirt.security.timestamp_dac="1580718107" 4. shutdown # virsh shutdown avocado-vt-vm1 Domain avocado-vt-vm1 is being shutdown # ll -Z /var/lib/libvirt/qemu/test.agent; getfattr -m trusted.libvirt.security -d /var/lib/libvirt/qemu/test.agent ls: cannot access '/var/lib/libvirt/qemu/test.agent': No such file or directory getfattr: /var/lib/libvirt/qemu/test.agent: No such file or directory Special need mention: file will disappear when vm not running. no xattr for hotplugged one. S4. disable remember_owner # grep remember_owner /etc/libvirt/qemu.conf remember_owner = 0 # systemctl restart libvirtd 1. Serial: pty pci-serial start and hotplug functions start guest: # ll -Z /dev/pts/2; getfattr -m trusted.libvirt.security -d /dev/pts/2 crw--w----. 1 qemu tty system_u:object_r:svirt_devpts_t:s0 136, 2 Feb 5 06:18 /dev/pts/2 # cat /dev/pts/2 ABC123 # virsh attach-device avocado-vt-vm1 serial-pty.xml Device attached successfully # cat /dev/pts/4 ABC123 ^C # ll -Z /dev/pts/4; getfattr -m trusted.libvirt.security -d /dev/pts/4 crw--w----. 1 qemu tty system_u:object_r:svirt_devpts_t:s0 136, 4 Feb 5 06:29 /dev/pts/4 2. Console: unix bind start and hotplug functions # virsh start avocado-vt-vm1 Domain avocado-vt-vm1 started # ll -Z /var/lib/libvirt/qemu/foo; getfattr -m trusted.libvirt.security -d /var/lib/libvirt/qemu/foo srwxrwxr-x. 1 qemu qemu system_u:object_r:qemu_var_run_t:s0 0 Feb 5 06:32 /var/lib/libvirt/qemu/foo # nc -U /var/lib/libvirt/qemu/foo hello # virsh attach-device avocado-vt-vm1 console-unix.xml Device attached successfully # nc -U /var/lib/libvirt/qemu/foo2 hello # ll -Z /var/lib/libvirt/qemu/foo2; getfattr -m trusted.libvirt.security -d /var/lib/libvirt/qemu/foo2 srwxrwxr-x. 1 qemu qemu system_u:object_r:qemu_var_run_t:s0 0 Feb 5 06:35 /var/lib/libvirt/qemu/foo2 3. Channel: unix bind start and hotplug functions # virsh start avocado-vt-vm1 Domain avocado-vt-vm1 started # ll -Z /var/lib/libvirt/qemu/test.agent; getfattr -m trusted.libvirt.security -d /var/lib/libvirt/qemu/test.agent srwxrwxr-x. 1 qemu qemu system_u:object_r:qemu_var_run_t:s0 0 Feb 5 06:38 /var/lib/libvirt/qemu/test.agent # nc -U /var/lib/libvirt/qemu/test.agent qwer # virsh attach-device avocado-vt-vm1 channel-unix.xml Device attached successfully # ll -Z /var/lib/libvirt/qemu/test2.agent; getfattr -m trusted.libvirt.security -d /var/lib/libvirt/qemu/test2.agent srwxrwxr-x. 1 qemu qemu system_u:object_r:qemu_var_run_t:s0 0 Feb 5 06:40 /var/lib/libvirt/qemu/test2.agent # nc -U /var/lib/libvirt/qemu/test2.agent qwer Special need mention: no xattr for files; function works well.
Hi Michal, I have two questions about chardev testing per comment6. Could you help check pls? When remember_owner is enabled in S1-S3: 1. For pty device in S1, the /dev/pts/* has no xattr result, is it as expected for it? 2. In S2-S3, the 'hotplug'ed chardev files get no xattr result, not align with the 'start with' situation. Is it an issue here or as expected? (same behaviour as libvirt-5.6.0, not regression by this bug) Thanks.
(In reply to yanqzhan from comment #7) > Hi Michal, > > I have two questions about chardev testing per comment6. Could you help > check pls? > When remember_owner is enabled in S1-S3: > 1. For pty device in S1, the /dev/pts/* has no xattr result, is it as > expected for it? Yes, this is expected; /dev/pts/* files are not created by libvirt nor their seclabels are changed by libvirt. > 2. In S2-S3, the 'hotplug'ed chardev files get no xattr result, not align > with the 'start with' situation. Is it an issue here or as expected? (same > behaviour as libvirt-5.6.0, not regression by this bug) Yes, this is expected.
(In reply to Michal Privoznik from comment #8) > > 2. In S2-S3, the 'hotplug'ed chardev files get no xattr result, *not* align > > with the 'start with' situation. Is it an issue here or as expected? (same > > behaviour as libvirt-5.6.0, not regression by this bug) > > Yes, this is expected. Hi Michal, Could you help explain why? Thank you.
(In reply to yanqzhan from comment #9) > Could you help explain why? So I might have misunderstood what you are really asking. I mean, in case of UNIX sockets, if they don't exist at the time libvirt tries to label them, the labelling is skipped and thus no label is remembered. The file doesn't exist so there is nothing to remember anyway. Is this what you are asking?
(In reply to Michal Privoznik from comment #10) > (In reply to yanqzhan from comment #9) > > Could you help explain why? > > So I might have misunderstood what you are really asking. I mean, in case of > UNIX sockets, if they don't exist at the time libvirt tries to label them, > the labelling is skipped and thus no label is remembered. The file doesn't > exist so there is nothing to remember anyway. Is this what you are asking? Thanks Michal. Then there's still a question: why libvirt can label them when start guest? The file also doesn't exist before guest starting. What's difference here?
(In reply to yanqzhan from comment #11) > (In reply to Michal Privoznik from comment #10) > > (In reply to yanqzhan from comment #9) > > > Could you help explain why? > > > > So I might have misunderstood what you are really asking. I mean, in case of > > UNIX sockets, if they don't exist at the time libvirt tries to label them, > > the labelling is skipped and thus no label is remembered. The file doesn't > > exist so there is nothing to remember anyway. Is this what you are asking? > > Thanks Michal. Then there's still a question: why libvirt can label them > when start guest? The file also doesn't exist before guest starting. What's > difference here? Because when starting a domain, libvirt will create the UNIX socket and pass FD to it on the command line. The security label setting is done after that, so the file exists. When hotplugging it, libvirt tries to do seclabel setting (if the file exists upront) and after that it instructs qemu to create the socket. That is why you see two different behaviours.
On latest build, the console or channel chardev file also has no xattr when remember_owner = 1. FYI. libvirt-daemon-6.0.0-17.module+el8.2.0+6257+0d066c28.x86_64 qemu-kvm-4.2.0-19.module+el8.2.0+6296+6b821950.x86_64
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:2017