Bug 2141965
Summary: | [TPM][vhost-vdpa][rhel9.2]Boot a guest with "vhost-vdpa + TPM emulator", qemu output: qemu-kvm: vhost_vdpa_listener_region_add received unaligned region | ||
---|---|---|---|
Product: | Red Hat Enterprise Linux 9 | Reporter: | Lei Yang <leiyang> |
Component: | qemu-kvm | Assignee: | Laurent Vivier <lvivier> |
qemu-kvm sub component: | Networking | QA Contact: | Lei Yang <leiyang> |
Status: | CLOSED ERRATA | Docs Contact: | |
Severity: | medium | ||
Priority: | medium | CC: | chayang, coli, eperezma, eric.auger, jinzhao, juzhang, lulu, lvivier, marcandre.lureau, virt-maint, wquan, yalzhang, yama, yanqzhan |
Version: | 9.2 | Keywords: | Triaged |
Target Milestone: | rc | ||
Target Release: | --- | ||
Hardware: | Unspecified | ||
OS: | Unspecified | ||
Whiteboard: | |||
Fixed In Version: | qemu-kvm-8.0.0-10.el9 | Doc Type: | If docs needed, set a value |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | 2023-11-07 08:26:38 UTC | Type: | --- |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: |
Description
Lei Yang
2022-11-11 09:08:37 UTC
Lei, are you able to reproduce the problem with the vDPA simulator? Cindy, this BZ seems related to BZ 2111317, could you have a look? Thanks (In reply to Laurent Vivier from comment #1) > Lei, > > are you able to reproduce the problem with the vDPA simulator? Yep. vDPA simulator also can reproduced this problem. (In reply to Laurent Vivier from comment #2) > Cindy, > > this BZ seems related to BZ 2111317, could you have a look? > > Thanks sure, I will work on this Thanks Cindy (In reply to Lei Yang from comment #3) > (In reply to Laurent Vivier from comment #1) > > Lei, > > > > are you able to reproduce the problem with the vDPA simulator? > > Yep. vDPA simulator also can reproduced this problem. Hi lei, do you know who is taking the response for tpm device? I'm not clear about this device, maybe we need help from them. Why is the memory unaligned here Thanks Cindy (In reply to lulu from comment #5) > (In reply to Lei Yang from comment #3) > > (In reply to Laurent Vivier from comment #1) > > > Lei, > > > > > > are you able to reproduce the problem with the vDPA simulator? > > > > Yep. vDPA simulator also can reproduced this problem. > > Hi lei, > do you know who is taking the response for tpm device? > I'm not clear about this device, maybe we need help from them. > Why is the memory unaligned here > > Thanks > Cindy Hello Marc-André Could you please help me review this issue? Thanks in advance. Thanks Lei This issue was being worked on by @eric.auger . Apparently, no fixes were merged, and the last series was https://patchew.org/QEMU/20220208133842.112017-1-eric.auger@redhat.com/. Eric, are you still working on it? Hi, what was pulled is: [PATCH v5 0/2] vfio/common: remove spurious tpm-crb-cmd misalignment warning https://lore.kernel.org/all/20220506132510.1847942-1-eric.auger@redhat.com/ plus [PATCH] vfio/common: remove spurious warning on vfio_listener_region_del https://lore.kernel.org/all/20220524091405.416256-1-eric.auger@redhat.com/ So I guess something similar should be done for vhost listeners ... (In reply to Eric Auger from comment #8) > Hi, what was pulled is: > > [PATCH v5 0/2] vfio/common: remove spurious tpm-crb-cmd misalignment warning > https://lore.kernel.org/all/20220506132510.1847942-1-eric.auger@redhat.com/ > > plus > > [PATCH] vfio/common: remove spurious warning on vfio_listener_region_del > https://lore.kernel.org/all/20220524091405.416256-1-eric.auger@redhat.com/ > > So I guess something similar should be done for vhost listeners ... Thanks Eric, I will check this setting in vDPA device Patch posted upstream: https://lists.nongnu.org/archive/html/qemu-devel/2022-11/msg03670.html Hopefully it can be merged in time for qemu-7.2.0, but if not we'll need to backport. I'll reassign to Marc-Andre for now since he posted the upstream patch. Dropping the ITR and sending this back to the vhost/vdpa team resolve as it seems the proposed solution from Marc-Andre was not accepted by the maintainer. Having to change the vhost/vdpa to not do the DMA on devices w/ unaligned memory only or somehow skip would be something the vhost/vdpa team would need to handle. Hit same issue on qemu-kvm-8.0.0-1.el9.x86_64 Test Version: kernel-5.14.0-303.el9.x86_64 qemu-kvm-8.0.0-1.el9.x86_64 swtpm-0.8.0-1.el9.x86_64 (In reply to Lei Yang from comment #17) > (In reply to Laurent Vivier from comment #16) ... > > QE tested the current bug with above build. QEMU no longer prints error > messages "qemu-kvm: vhost_vdpa_listener_region_add received unaligned > region". > > Test version: > kernel-5.14.0-324.el9.x86_64 > qemu-kvm-8.0.0-5.el9.BZ2141965.x86_64 > iproute-6.2.0-5.el9.x86_64 > edk2-ovmf-20230301gitf80f052277c8-5.el9.noarch > > Test Steps: > 1. 1.create TPM emulator > /usr/bin/swtpm socket --ctrl > type=unixio,path=/root/avocado/data/avocado-vt/swtpm/avocado-vt- > vm1_tpm0_swtpm.sock,mode=0600 --tpmstate > dir=/root/avocado/data/avocado-vt/swtpm/avocado-vt-vm1_tpm0_state,mode=0600 > --terminate --tpm2 --log > file=/root/avocado/job-results/job-2023-06-19T21.29-af3003b/test-results/1- > Host_RHEL.m9.u3.vdpa.ovmf.qcow2.virtio_scsi.up.virtio_net.Guest.RHEL.9.3.0. > x86_64.io-github-autotest-qemu.boot.q35/vtpm_avocado-vt-vm1_tpm0_swtpm.log > > 2.Create vhost-vdpa base on mellanox card > vdpa dev add name vdpa${i} mgmtdev pci/$pci_addr mac > 00:11:22:33:44:${macaddr} > > 3.Boot a guest with "vhost-vdpa + TPM emulator" > -device '{"driver": "virtio-net-pci", "mac": "00:11:22:33:44:00", "id": > "net0", "netdev": "hostnet0", "mq": true, "vectors": 18, "bus": > "pcie-root-port-3", "addr": "0x0"}' \ > -netdev vhost-vdpa,id=hostnet0,vhostdev=/dev/vhost-vdpa-0,queues=8 \ > -chardev > socket,id=char_vtpm_avocado-vt-vm1_tpm0,path=/root/avocado/data/avocado-vt/ > swtpm/avocado-vt-vm1_tpm0_swtpm.sock \ > -tpmdev > emulator,chardev=char_vtpm_avocado-vt-vm1_tpm0,id=emulator_vtpm_avocado-vt- > vm1_tpm0 \ > -device '{"id": "tpm-crb_vtpm_avocado-vt-vm1_tpm0", "tpmdev": > "emulator_vtpm_avocado-vt-vm1_tpm0", "driver": "tpm-crb"}' \ > > 4.Guest boot up succeed and qemu no longer prints error messages "qemu-kvm: > vhost_vdpa_listener_region_add received unaligned region". Could you check vDPA device acts as expected and allows the system to communicate with other systems? (In reply to Laurent Vivier from comment #18) > (In reply to Lei Yang from comment #17) > > (In reply to Laurent Vivier from comment #16) > ... > > > > QE tested the current bug with above build. QEMU no longer prints error > > messages "qemu-kvm: vhost_vdpa_listener_region_add received unaligned > > region". > > > > Test version: > > kernel-5.14.0-324.el9.x86_64 > > qemu-kvm-8.0.0-5.el9.BZ2141965.x86_64 > > iproute-6.2.0-5.el9.x86_64 > > edk2-ovmf-20230301gitf80f052277c8-5.el9.noarch > > > > Test Steps: > > 1. 1.create TPM emulator > > /usr/bin/swtpm socket --ctrl > > type=unixio,path=/root/avocado/data/avocado-vt/swtpm/avocado-vt- > > vm1_tpm0_swtpm.sock,mode=0600 --tpmstate > > dir=/root/avocado/data/avocado-vt/swtpm/avocado-vt-vm1_tpm0_state,mode=0600 > > --terminate --tpm2 --log > > file=/root/avocado/job-results/job-2023-06-19T21.29-af3003b/test-results/1- > > Host_RHEL.m9.u3.vdpa.ovmf.qcow2.virtio_scsi.up.virtio_net.Guest.RHEL.9.3.0. > > x86_64.io-github-autotest-qemu.boot.q35/vtpm_avocado-vt-vm1_tpm0_swtpm.log > > > > 2.Create vhost-vdpa base on mellanox card > > vdpa dev add name vdpa${i} mgmtdev pci/$pci_addr mac > > 00:11:22:33:44:${macaddr} > > > > 3.Boot a guest with "vhost-vdpa + TPM emulator" > > -device '{"driver": "virtio-net-pci", "mac": "00:11:22:33:44:00", "id": > > "net0", "netdev": "hostnet0", "mq": true, "vectors": 18, "bus": > > "pcie-root-port-3", "addr": "0x0"}' \ > > -netdev vhost-vdpa,id=hostnet0,vhostdev=/dev/vhost-vdpa-0,queues=8 \ > > -chardev > > socket,id=char_vtpm_avocado-vt-vm1_tpm0,path=/root/avocado/data/avocado-vt/ > > swtpm/avocado-vt-vm1_tpm0_swtpm.sock \ > > -tpmdev > > emulator,chardev=char_vtpm_avocado-vt-vm1_tpm0,id=emulator_vtpm_avocado-vt- > > vm1_tpm0 \ > > -device '{"id": "tpm-crb_vtpm_avocado-vt-vm1_tpm0", "tpmdev": > > "emulator_vtpm_avocado-vt-vm1_tpm0", "driver": "tpm-crb"}' \ > > > > 4.Guest boot up succeed and qemu no longer prints error messages "qemu-kvm: > > vhost_vdpa_listener_region_add received unaligned region". > > Could you check vDPA device acts as expected and allows the system to > communicate with other systems? 1. Boot a guest with vdpa device and TPM emulator device, there are no error messages in the host and guest dmesg. 2. ping from guest to host pass, without packet loss. Cindy, in fact, we don't need to a add a check to bypass (like in comment 10) the vhost_vdpa_listener_region_add() as it is already done when the memory is not aligned. Could we simply ignore the error message (and close the BZ as NOTABUG) or remove the error_report()? 306 static void vhost_vdpa_listener_region_add(MemoryListener *listener, 307 MemoryRegionSection *section) 308 { ... 325 if (unlikely((section->offset_within_address_space & ~TARGET_PAGE_MASK) != 326 (section->offset_within_region & ~TARGET_PAGE_MASK))) { 327 error_report("%s received unaligned region %s", __func__, section->mr->name); 328 return; 329 } ... What is the purpose of the error_report()? (In reply to Laurent Vivier from comment #20) > Cindy, > > in fact, we don't need to a add a check to bypass (like in comment 10) the > vhost_vdpa_listener_region_add() as it is already done when the memory is > not aligned. > > Could we simply ignore the error message (and close the BZ as NOTABUG) or > remove the error_report()? > > 306 static void vhost_vdpa_listener_region_add(MemoryListener *listener, > 307 MemoryRegionSection *section) > 308 { > ... > 325 if (unlikely((section->offset_within_address_space & > ~TARGET_PAGE_MASK) != > 326 (section->offset_within_region & ~TARGET_PAGE_MASK))) { > 327 error_report("%s received unaligned region %s", __func__, > section->mr->name); > 328 return; > 329 } > ... > > What is the purpose of the error_report()? Sure, I think maybe we can change this to WARN? Thanks Cindy Lei, from QE point of view, do you think it's acceptable to have a warning? like: qemu-kvm: warning: vhost_vdpa_listener_region_add received unaligned region tpm-crb-cmd qemu-kvm: warning: vhost_vdpa_listener_region_del received unaligned region tpm-crb-cmd (In reply to Laurent Vivier from comment #22) > Lei, > > from QE point of view, do you think it's acceptable to have a warning? > > like: > > qemu-kvm: warning: vhost_vdpa_listener_region_add received unaligned region > tpm-crb-cmd > qemu-kvm: warning: vhost_vdpa_listener_region_del received unaligned region > tpm-crb-cmd Hi Laurent I would like to confirm a problem,we had to accept as a warning to fix this issue because the patch(https://lists.nongnu.org/archive/html/qemu-devel/2022-11/msg03670.html) for this fix was rejected upstream? If this is the reason, it seems that this is the only way to deal with it. If not, without any output is more ideal, because both qemu output and qemu-kvm warning are annoying prompts for users. Thanks Lei (In reply to Lei Yang from comment #23) > (In reply to Laurent Vivier from comment #22) > > Lei, > > > > from QE point of view, do you think it's acceptable to have a warning? > > > > like: > > > > qemu-kvm: warning: vhost_vdpa_listener_region_add received unaligned region > > tpm-crb-cmd > > qemu-kvm: warning: vhost_vdpa_listener_region_del received unaligned region > > tpm-crb-cmd > > Hi Laurent > > I would like to confirm a problem,we had to accept as a warning to fix this > issue because the > patch(https://lists.nongnu.org/archive/html/qemu-devel/2022-11/msg03670. > html) for this fix was rejected upstream? Yes, and I proposed another solution that was rejected too: https://patchew.org/QEMU/20230620195054.23929-1-lvivier@redhat.com/ > If this is the reason, it seems that this is the only way to deal with it. > If not, without any output is more ideal, because both qemu output and > qemu-kvm warning are annoying prompts for users. I understand your point of view. Cindy, I think this information is only useful for debugging purpose, perhaps we can replace it by a trace? (In reply to Laurent Vivier from comment #24) > (In reply to Lei Yang from comment #23) > > (In reply to Laurent Vivier from comment #22) > > > Lei, > > > > > > from QE point of view, do you think it's acceptable to have a warning? > > > > > > like: > > > > > > qemu-kvm: warning: vhost_vdpa_listener_region_add received unaligned region > > > tpm-crb-cmd > > > qemu-kvm: warning: vhost_vdpa_listener_region_del received unaligned region > > > tpm-crb-cmd > > > > Hi Laurent > > > > I would like to confirm a problem,we had to accept as a warning to fix this > > issue because the > > patch(https://lists.nongnu.org/archive/html/qemu-devel/2022-11/msg03670. > > html) for this fix was rejected upstream? > > Yes, and I proposed another solution that was rejected too: > https://patchew.org/QEMU/20230620195054.23929-1-lvivier@redhat.com/ > > > If this is the reason, it seems that this is the only way to deal with it. > > If not, without any output is more ideal, because both qemu output and > > qemu-kvm warning are annoying prompts for users. > > I understand your point of view. > > Cindy, I think this information is only useful for debugging purpose, > perhaps we can replace it by a trace? Sure, I agree with this Thanks cindy Fix is merged upstream: commit 77812aa7b1fdf8f547c35a7f9a4eb1cbf3a073db Author: Laurent Vivier <lvivier> Date: Tue Jul 4 09:19:31 2023 +0200 vhost-vdpa: mute unaligned memory error report With TPM CRM device, vhost-vdpa reports an error when it tries to register a listener for a non aligned memory region: qemu-system-x86_64: vhost_vdpa_listener_region_add received unaligned region qemu-system-x86_64: vhost_vdpa_listener_region_del received unaligned region This error can be confusing for the user whereas we only need to skip the region (as it's already done after the error_report()) Rather than introducing a special case for TPM CRB memory section to not display the message in this case, simply replace the error_report() by a trace function (with more information, like the memory region name). Signed-off-by: Laurent Vivier <lvivier> Message-Id: <20230704071931.575888-2-lvivier> Reviewed-by: David Hildenbrand <david> Acked-by: Jason Wang <jasowang> Reviewed-by: Michael S. Tsirkin <mst> Signed-off-by: Michael S. Tsirkin <mst> QE bot(pre verify): Set 'Verified:Tested,SanityOnly' as gating/tier1 test pass. ==>Reproduced this bug on the qemu-kvm-8.0.0-9.el9.x86_64 =>Test Version: qemu-kvm-8.0.0-9.el9.x86_64 kernel-5.14.0-348.el9.x86_64 iproute-6.2.0-5.el9.x86_64 # flint -d 0000:17:00.0 q Image type: FS4 FW Version: 22.37.0154 FW Release Date: 17.3.2023 Product Version: 22.37.0154 Description: UID GuidsNumber Base GUID: b8cef603000a110c 4 Base MAC: b8cef60a110c 4 Image VSD: N/A Device VSD: N/A PSID: MT_0000000359 Security Attributes: N/A =>Test steps 1. Create a vdpa device vdpa dev add name vdpa0 mgmtdev pci/0000:17:00.2 mac 00:11:22:33:44:00 2. Create TPM emulator /usr/bin/swtpm socket --ctrl type=unixio,path=/root/avocado/data/avocado-vt/swtpm/avocado-vt-vm1_tpm0_swtpm.sock,mode=0600 --tpmstate dir=/root/avocado/data/avocado-vt/swtpm/avocado-vt-vm1_tpm0_state,mode=0600 --terminate --tpm2 --log file=/root/avocado/job-results/job-2023-08-01T22.35-26a53f2/test-results/1-Host_RHEL.m9.u3.vdpa.ovmf.qcow2.virtio_scsi.up.virtio_net.Guest.RHEL.9.3.0.x86_64.io-github-autotest-qemu.boot.q35/vtpm_avocado-vt-vm1_tpm0_swtpm.log 3.Boot a guest with "vhost-vdpa + TPM emulator" -device '{"id": "pcie-root-port-3", "port": 3, "driver": "pcie-root-port", "addr": "0x1.0x3", "bus": "pcie.0", "chassis": 4}' \ -device '{"driver": "virtio-net-pci", "mac": "00:11:22:33:44:00", "id": "idj5i2hq", "mq": true, "vectors": 18, "netdev": "idZXtYxU", "bus": "pcie-root-port-3", "addr": "0x0"}' \ -netdev vhost-vdpa,id=idZXtYxU,vhostdev=/dev/vhost-vdpa-0,queues=8 \ -chardev socket,id=char_vtpm_avocado-vt-vm1_tpm0,path=/root/avocado/data/avocado-vt/swtpm/avocado-vt-vm1_tpm0_swtpm.sock \ -tpmdev emulator,chardev=char_vtpm_avocado-vt-vm1_tpm0,id=emulator_vtpm_avocado-vt-vm1_tpm0 \ -device '{"id": "tpm-crb_vtpm_avocado-vt-vm1_tpm0", "tpmdev": "emulator_vtpm_avocado-vt-vm1_tpm0", "driver": "tpm-crb"}' \ 4. qemu output error info: (qemu) qemu-kvm: vhost_vdpa_listener_region_add received unaligned region qemu-kvm: vhost_vdpa_listener_region_del received unaligned region qemu-kvm: vhost_vdpa_listener_region_add received unaligned region ==>Based on the above test result, reproduced this problem on the qemu-kvm-8.0.0-9.el9.x86_64 ==>Verified this problem on the qemu-kvm-8.0.0-10.el9.x86_64 =>Test Version: qemu-kvm-8.0.0-10.el9.x86_64 kernel-5.14.0-348.el9.x86_64 iproute-6.2.0-5.el9.x86_64 # flint -d 0000:17:00.0 q Image type: FS4 FW Version: 22.37.0154 FW Release Date: 17.3.2023 Product Version: 22.37.0154 Description: UID GuidsNumber Base GUID: b8cef603000a110c 4 Base MAC: b8cef60a110c 4 Image VSD: N/A Device VSD: N/A PSID: MT_0000000359 Security Attributes: N/A =>Test steps Repeat the above test steps, qemu no longer prints error messages "qemu-kvm: vhost_vdpa_listener_region_add received unaligned region". ==> So based on the above test result, this problem has been fixed very well on qemu-kvm-8.0.0-10.el9.x86_64. Based on the Comment 29, move to "VERIFIED". Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: qemu-kvm security, bug fix, and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2023:6368 |