Bug 1728958

Summary: Hot unplug vfio-pci NIC devices from sev guest will cause qemu-kvm: sev_ram_block_removed: failed to unregister region
Product: Red Hat Enterprise Linux 8 Reporter: Pei Zhang <pezhang>
Component: qemu-kvmAssignee: Alex Williamson <alex.williamson>
Status: CLOSED ERRATA QA Contact: Pei Zhang <pezhang>
Severity: high Docs Contact:
Priority: high    
Version: 8.1CC: alex.williamson, chayang, ehabkost, jinzhao, juzhang, knoel, leiyang, rbalakri, virt-maint, zhguo
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: qemu-kvm-2.12.0-82.module+el8.1.0+3738+0d8c0249 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1728947 Environment:
Last Closed: 2019-11-05 20:51:02 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1728947    
Bug Blocks:    

Description Pei Zhang 2019-07-11 06:26:10 UTC
+++ This bug was initially created as a clone of Bug #1728947 +++

Description of problem:
Boot guest with sev and device assignment, qemu print failed info.

Version-Release number of selected component (if applicable):
kernel-4.18.0-114.el8.x86_64
qemu-kvm-4.0.0-5.module+el8.1.0+3622+5812d9bf.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Boot guest with sev and device assignment

/usr/libexec/qemu-kvm \
-enable-kvm \
-cpu EPYC \
-smp 4 \
-m 4G \
-object sev-guest,id=sev0,cbitpos=47,reduced-phys-bits=1 \
-machine q35,memory-encryption=sev0 \
-drive if=pflash,format=raw,unit=0,file=/usr/share/edk2/ovmf/sev/OVMF_CODE.secboot.fd,readonly \
-drive if=pflash,format=raw,unit=1,file=/usr/share/edk2/ovmf/sev/OVMF_VARS.fd \
-device pcie-root-port,id=root.1,chassis=1 \
-device pcie-root-port,id=root.2,chassis=2 \
-device pcie-root-port,id=root.3,chassis=3 \
-device pcie-root-port,id=root.4,chassis=4 \
-device pcie-root-port,id=root.5,chassis=5 \
-device virtio-scsi-pci,iommu_platform=on,id=scsi0,bus=root.1,addr=0x0 \
-drive file=/home/sev_guest.qcow2,format=qcow2,if=none,id=drive-scsi0-0-0-0 \
-device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scssi0-0-0-0,bootindex=1 \
-netdev tap,id=hostnet0,vhost=off \
-device virtio-net-pci,netdev=hostnet0,id=net0,mac=18:66:da:57:dd:03,bus=root.2,iommu_platform=true \
-vnc :0 \
-monitor stdio \
-serial unix:/tmp/console,server,nowait \
-device vfio-pci,host=0000:e3:00.0,bus=root.3,id=pf1 \
-device vfio-pci,host=0000:e3:00.1,bus=root.4,id=pf2 \


2. Hot unplug vfio-pci devices, qemu prints error info:

(qemu) device_del pf1 
(qemu) qemu-kvm: sev_ram_block_removed: failed to unregister region (0x7f5537000000+0x1000000)
qemu-kvm: sev_ram_block_removed: failed to unregister region (0x7f555ee1a000+0x8000)

(qemu) 
(qemu) device_del pf2
(qemu) qemu-kvm: sev_ram_block_removed: failed to unregister region (0x7f555ec44000+0x8000)
qemu-kvm: sev_ram_block_removed: failed to unregister region (0x7f5536000000+0x1000000)


Actual results:
qemu prints failed info.


Expected results:
qemu should not print any failed info.

Additional info:
1. Both host and guest work well, only qemu terminal shows failed info.

Comment 1 Pei Zhang 2019-07-11 06:29:04 UTC
Versions:
4.18.0-114.el8.x86_64
qemu-kvm-2.12.0-81.module+el8.1.0+3619+dfe1ae01.x86_64

Comment 3 Alex Williamson 2019-07-12 21:23:13 UTC
The issue seems to be harmless, code was added to skip assigned device mmio regions from sev mapping when added, but the reciprocal code on removal was missed.  Please test if this brew build resolves the warning: https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=22630995

Comment 4 Pei Zhang 2019-07-15 01:07:21 UTC
(In reply to Alex Williamson from comment #3)
> The issue seems to be harmless, code was added to skip assigned device mmio
> regions from sev mapping when added, but the reciprocal code on removal was
> missed.  Please test if this brew build resolves the warning:
> https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=22630995

This build works well, the issue is gone. Thanks.

Best regards,

Pei

Comment 5 Alex Williamson 2019-07-15 14:33:50 UTC
Thanks for testing!

Posted upstream: https://lists.gnu.org/archive/html/qemu-devel/2019-07/msg03451.html

Comment 6 Alex Williamson 2019-07-16 15:21:52 UTC
Fixed in upstream commit which will be included in 4.1-rc1

commit 56e2ec9488b3b281130e064929f8ae5595d6ac39
Author: Alex Williamson <alex.williamson>
Date:   Mon Jul 15 08:28:39 2019 -0600

    target/i386: sev: Do not unpin ram device memory region
    
    The commit referenced below skipped pinning ram device memory when
    ram blocks are added, we need to do the same when they're removed.
    
    Cc: Brijesh Singh <brijesh.singh>
    Cc: Paolo Bonzini <pbonzini>
    Fixes: cedc0ad539af ("target/i386: sev: Do not pin the ram device memory region")
    Signed-off-by: Alex Williamson <alex.williamson>
    Message-Id: <156320087103.2556.10983987500488190423.stgit>
    Reviewed-by: Singh, Brijesh <brijesh.singh>
    Signed-off-by: Paolo Bonzini <pbonzini>

Comment 7 Eduardo Habkost 2019-07-22 16:49:15 UTC
(In reply to Alex Williamson from comment #6)
> Fixed in upstream commit which will be included in 4.1-rc1

This is a RHEL BZ, isn't a backport necessary?

Comment 8 Alex Williamson 2019-07-22 16:59:20 UTC
(In reply to Eduardo Habkost from comment #7)
> (In reply to Alex Williamson from comment #6)
> > Fixed in upstream commit which will be included in 4.1-rc1
> 
> This is a RHEL BZ, isn't a backport necessary?

Yes, I didn't immediately recognize that this was a RHEL bz rather than RHAV since the initial report was based on QEMU 4.0.  Moving back to assigned.

Comment 12 Pei Zhang 2019-07-24 03:09:46 UTC
Verified by qemu-kvm-2.12.0-82.module+el8.1.0+3738+0d8c0249.x86_64:

Steps:

1. Boot guest with sev and device assignment

2. Hot unplug vfio-pci devices, qemu works well.

(qemu) device_del pf1 
(qemu) device_del pf2
(qemu) 

3. Reboot/shutdown guest, works well. No any error.

So this bug has been fixed very well. 

Move to 'VERIFIED'.

Comment 14 errata-xmlrpc 2019-11-05 20:51:02 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2019:3345