Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1728958

Summary: Hot unplug vfio-pci NIC devices from sev guest will cause qemu-kvm: sev_ram_block_removed: failed to unregister region
Product: Red Hat Enterprise Linux 8 Reporter: Pei Zhang <pezhang>
Component: qemu-kvmAssignee: Alex Williamson <alex.williamson>
Status: CLOSED ERRATA QA Contact: Pei Zhang <pezhang>
Severity: high Docs Contact:
Priority: high    
Version: 8.1CC: alex.williamson, chayang, ehabkost, jinzhao, juzhang, knoel, leiyang, rbalakri, virt-maint, zhguo
Target Milestone: rcFlags: pm-rhel: mirror+
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: qemu-kvm-2.12.0-82.module+el8.1.0+3738+0d8c0249 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1728947 Environment:
Last Closed: 2019-11-05 20:51:02 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1728947    
Bug Blocks:    

Description Pei Zhang 2019-07-11 06:26:10 UTC 
+++ This bug was initially created as a clone of Bug #1728947 +++

Description of problem:
Boot guest with sev and device assignment, qemu print failed info.

Version-Release number of selected component (if applicable):
kernel-4.18.0-114.el8.x86_64
qemu-kvm-4.0.0-5.module+el8.1.0+3622+5812d9bf.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Boot guest with sev and device assignment

/usr/libexec/qemu-kvm \
-enable-kvm \
-cpu EPYC \
-smp 4 \
-m 4G \
-object sev-guest,id=sev0,cbitpos=47,reduced-phys-bits=1 \
-machine q35,memory-encryption=sev0 \
-drive if=pflash,format=raw,unit=0,file=/usr/share/edk2/ovmf/sev/OVMF_CODE.secboot.fd,readonly \
-drive if=pflash,format=raw,unit=1,file=/usr/share/edk2/ovmf/sev/OVMF_VARS.fd \
-device pcie-root-port,id=root.1,chassis=1 \
-device pcie-root-port,id=root.2,chassis=2 \
-device pcie-root-port,id=root.3,chassis=3 \
-device pcie-root-port,id=root.4,chassis=4 \
-device pcie-root-port,id=root.5,chassis=5 \
-device virtio-scsi-pci,iommu_platform=on,id=scsi0,bus=root.1,addr=0x0 \
-drive file=/home/sev_guest.qcow2,format=qcow2,if=none,id=drive-scsi0-0-0-0 \
-device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scssi0-0-0-0,bootindex=1 \
-netdev tap,id=hostnet0,vhost=off \
-device virtio-net-pci,netdev=hostnet0,id=net0,mac=18:66:da:57:dd:03,bus=root.2,iommu_platform=true \
-vnc :0 \
-monitor stdio \
-serial unix:/tmp/console,server,nowait \
-device vfio-pci,host=0000:e3:00.0,bus=root.3,id=pf1 \
-device vfio-pci,host=0000:e3:00.1,bus=root.4,id=pf2 \


2. Hot unplug vfio-pci devices, qemu prints error info:

(qemu) device_del pf1 
(qemu) qemu-kvm: sev_ram_block_removed: failed to unregister region (0x7f5537000000+0x1000000)
qemu-kvm: sev_ram_block_removed: failed to unregister region (0x7f555ee1a000+0x8000)

(qemu) 
(qemu) device_del pf2
(qemu) qemu-kvm: sev_ram_block_removed: failed to unregister region (0x7f555ec44000+0x8000)
qemu-kvm: sev_ram_block_removed: failed to unregister region (0x7f5536000000+0x1000000)


Actual results:
qemu prints failed info.


Expected results:
qemu should not print any failed info.

Additional info:
1. Both host and guest work well, only qemu terminal shows failed info.
Comment 1 Pei Zhang 2019-07-11 06:29:04 UTC 
Versions:
4.18.0-114.el8.x86_64
qemu-kvm-2.12.0-81.module+el8.1.0+3619+dfe1ae01.x86_64
Comment 3 Alex Williamson 2019-07-12 21:23:13 UTC 
The issue seems to be harmless, code was added to skip assigned device mmio regions from sev mapping when added, but the reciprocal code on removal was missed.  Please test if this brew build resolves the warning: https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=22630995
Comment 4 Pei Zhang 2019-07-15 01:07:21 UTC 
(In reply to Alex Williamson from comment #3)
> The issue seems to be harmless, code was added to skip assigned device mmio
> regions from sev mapping when added, but the reciprocal code on removal was
> missed.  Please test if this brew build resolves the warning:
> https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=22630995

This build works well, the issue is gone. Thanks.

Best regards,

Pei
Comment 5 Alex Williamson 2019-07-15 14:33:50 UTC 
Thanks for testing!

Posted upstream: https://lists.gnu.org/archive/html/qemu-devel/2019-07/msg03451.html
Comment 6 Alex Williamson 2019-07-16 15:21:52 UTC 
Fixed in upstream commit which will be included in 4.1-rc1

commit 56e2ec9488b3b281130e064929f8ae5595d6ac39
Author: Alex Williamson <alex.williamson>
Date:   Mon Jul 15 08:28:39 2019 -0600

    target/i386: sev: Do not unpin ram device memory region
    
    The commit referenced below skipped pinning ram device memory when
    ram blocks are added, we need to do the same when they're removed.
    
    Cc: Brijesh Singh <brijesh.singh>
    Cc: Paolo Bonzini <pbonzini>
    Fixes: cedc0ad539af ("target/i386: sev: Do not pin the ram device memory region")
    Signed-off-by: Alex Williamson <alex.williamson>
    Message-Id: <156320087103.2556.10983987500488190423.stgit>
    Reviewed-by: Singh, Brijesh <brijesh.singh>
    Signed-off-by: Paolo Bonzini <pbonzini>
Comment 7 Eduardo Habkost 2019-07-22 16:49:15 UTC 
(In reply to Alex Williamson from comment #6)
> Fixed in upstream commit which will be included in 4.1-rc1

This is a RHEL BZ, isn't a backport necessary?
Comment 8 Alex Williamson 2019-07-22 16:59:20 UTC 
(In reply to Eduardo Habkost from comment #7)
> (In reply to Alex Williamson from comment #6)
> > Fixed in upstream commit which will be included in 4.1-rc1
> 
> This is a RHEL BZ, isn't a backport necessary?

Yes, I didn't immediately recognize that this was a RHEL bz rather than RHAV since the initial report was based on QEMU 4.0.  Moving back to assigned.
Comment 12 Pei Zhang 2019-07-24 03:09:46 UTC 
Verified by qemu-kvm-2.12.0-82.module+el8.1.0+3738+0d8c0249.x86_64:

Steps:

1. Boot guest with sev and device assignment

2. Hot unplug vfio-pci devices, qemu works well.

(qemu) device_del pf1 
(qemu) device_del pf2
(qemu) 

3. Reboot/shutdown guest, works well. No any error.

So this bug has been fixed very well. 

Move to 'VERIFIED'.
Comment 14 errata-xmlrpc 2019-11-05 20:51:02 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2019:3345