Bug 2152188

Summary:	VMs requiring vTPM fails to create
Product:	Red Hat Enterprise Linux 8	Reporter:	Jaroslav Suchanek <jsuchane>
Component:	libvirt	Assignee:	Michal Privoznik <mprivozn>
Status:	CLOSED ERRATA	QA Contact:	Yanqiu Zhang <yanqzhan>
Severity:	high	Docs Contact:
Priority:	high
Version:	8.7	CC:	acardace, danken, fdeutsch, jdenemar, kbidarka, lmen, lpivarc, mprivozn, sgott, virt-maint, yalzhang, yanqzhan, ycui, ymankad
Target Milestone:	rc	Keywords:	Triaged, ZStream
Target Release:	---	Flags:	pm-rhel: mirror+
Hardware:	All
OS:	All
Whiteboard:
Fixed In Version:	libvirt-8.0.0-12.module+el8.8.0+17545+95582d4e	Doc Type:	Bug Fix
Doc Text:	Cause: when starting or stopping swtpm emulator or vhost-user-gpu helper libvirt would read their pidfile and then check against /proc/[pid]/exe whether the PID is still valid. This works, but also requires CAP_SYS_PTRACE capability which might not be available in restricted containers. Fortunately, there is a way for libvirt to check the PID without accessing /proc and validating the pidfile. Consequence: The CAP_SYS_PTRACE capability was needed, which did not play nicely with rootless containers. Fix: The code was changed so that the pidfile is locked when swtpm or vhost-user-gpu is started. Now, whenever libvirt wants to read the PID it also tries to lock the pidfile. If the pidfile can be locked successfully, it means that the swtpm process which held the file locked is gone and the PID is invalid. And vice verca - if the file can't be locked then the process is still running and the PID is valid. Result: The swtpm PID and/or vhost-user-gpu PID can be read without the CAP_SYS_PTRACE capability.	Story Points:	---
Clone Of:	2150004
Clones:	2154412 2154413 (view as bug list)		Environment:
Last Closed:	2023-05-16 08:18:36 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	2150004
Bug Blocks:	2149227, 2150730, 2150731, 2154412, 2154413

Description Jaroslav Suchanek 2022-12-09 16:25:05 UTC

+++ This bug was initially created as a clone of Bug #2150004 +++

+++ This bug was initially created as a clone of Bug #2149227 +++

Description of problem:
Recent upstream changes in Kubevirt removed ptrace capability. This means downstream builds need to remove the capability too, causing the Libvirt to fail when vTPM is requested.

This is impacting Windows VMs (as they require vTPM) but not exclusively.

Andrea Bolognani made a preliminary list of commits,

The libvirt change is

  commit a9c500d2b50c5c041a1bb6ae9724402cf1cec8fe
  Author: Vasiliy Ulyanov <vulyanov>
  Date:   Wed Feb 2 17:28:16 2022 +0100

    qemu: tpm: Get swtpm pid without binary validation

plus I guess

  commit 4d7bb0177a33c4e90fd001edfe27bc030354d875
  Author: Michal Prívozník <mprivozn>
  Date:   Mon Mar 21 13:33:06 2022 +0100

    qemu_tpm: Do async IO when starting swtpm emulator

Comment 2 John Ferlan 2022-12-12 14:26:36 UTC

Can we get the qa_ack+ so that release+ can occur?  tks!

Strange when this bug was copied that the release bot didn't request the various flags

Comment 3 Yanqiu Zhang 2022-12-13 09:47:07 UTC

Test on rhel8.8 with:
libvirt-8.0.0-12.el8_rc.1fec897c86.x86_64
swtpm-0.7.0-4.20211109gitb79fd91.module+el8.8.0+16781+9f4724c2.x86_64
qemu-kvm-6.2.0-27.module+el8.8.0+17473+3286524b.x86_64
libtpms-0.9.1-1.20211126git1ff6fe1f43.module+el8.8.0+16781+9f4724c2.x86_64

# virsh dumpxml vm-ovmf --inactive|grep /tpm -B7
    <tpm model='tpm-crb'>
      <backend type='emulator' version='2.0'>
        <encryption secret='a2a3e901-72ad-469b-b6fa-8fc8cc3b1d02'/>
        <active_pcr_banks>
          <sha256/>
        </active_pcr_banks>
      </backend>
    </tpm>

# strace -o output.txt -T -tt -e trace=all -f -p `pidof libvirtd`

# virsh start vm-ovmf 
Domain 'vm-ovmf' started

# cat output.txt|grep -nE 'swtpm.pid|"/usr/bin/swtpm", "socket"|capset|execve'
...
16238:185337 04:40:26.483589 execve("/usr/bin/swtpm", ["/usr/bin/swtpm", "socket", "--ctrl", "type=unixio,path=/run/libvirt/qe"..., "--tpmstate", "dir=/var/lib/libvirt/swtpm/0a74f"..., "--log", "file=/var/log/swtpm/libvirt/qemu"..., "--terminate", "--tpm2", "--key", "pwdfd=40,mode=aes-256-cbc", "--migration-key", "pwdfd=43,mode=aes-256-cbc"], 0x7ffcb468a9d8 /* 7 vars */ <unfinished ...>
16244:185163 04:40:26.483910 openat(AT_FDCWD, "/run/libvirt/qemu/swtpm/1-vm-ovmf-swtpm.pid", O_RDONLY) = 40 <0.000012>
16253:185337 04:40:26.484985 <... execve resumed>) = 0 <0.001362>
...

Comment 6 Yanqiu Zhang 2022-12-14 09:13:47 UTC

Verify on rhel8.8 with downstream pkg:
libvirt-8.0.0-12.module+el8.8.0+17545+95582d4e.x86_64

On env of comment3, upgrade libvirt and restart libvirtd. Then do same test steps for vtpm, get log as below:

11426:212927 04:08:29.845142 execve("/usr/bin/swtpm", ["/usr/bin/swtpm", "socket", "--ctrl", "type=unixio,path=/run/libvirt/qe"..., "--tpmstate", "dir=/var/lib/libvirt/swtpm/0a74f"..., "--log", "file=/var/log/swtpm/libvirt/qemu"..., "--terminate", "--tpm2", "--key", "pwdfd=40,mode=aes-256-cbc", "--migration-key", "pwdfd=43,mode=aes-256-cbc"], 0x7fff0549c958 /* 7 vars */ <unfinished ...>
11432:212712 04:08:29.845463 openat(AT_FDCWD, "/run/libvirt/qemu/swtpm/1-vm-ovmf-swtpm.pid", O_RDONLY) = 40 <0.000012>
11441:212927 04:08:29.846585 <... execve resumed>) = 0 <0.001419>

Comment 15 errata-xmlrpc 2023-05-16 08:18:36 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: virt:rhel and virt-devel:rhel security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2023:2757