Bug 1590472

Summary:	RFE: QEMU VFIO based block driver for NVMe devices (OSP)
Product:	Red Hat OpenStack	Reporter:	Ademar Reis <areis>
Component:	openstack-nova	Assignee:	OSP DFG:Compute <osp-dfg-compute>
Status:	CLOSED WONTFIX	QA Contact:	OSP DFG:Compute <osp-dfg-compute>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	unspecified	CC:	alifshit, dasmith, dyuan, egallen, eglynn, eskultet, jhakimra, kchamart, lmen, lyarwood, markmc, sbauza, sgordon, srevivo, stephenfin, virt-bugs, virt-maint, vromanso, xuzhang, yisun
Target Milestone:	---	Keywords:	FutureFeature, Triaged
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Enhancement
Doc Text:		Story Points:	---
Clone Of:	1416182	Environment:
Last Closed:	2022-09-23 13:37:13 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1416180, 1416182, 1519004, 1519005
Bug Blocks:

Description Ademar Reis 2018-06-12 16:12:14 UTC

+++ This bug was initially created as a clone of Bug #1416182 +++

+++ This bug was initially created as a clone of Bug #1416180 +++

This BZ tracks the upstream work currently being done by Fam to introduce a VFIO based NVMe driver to QEMU:

https://lists.gnu.org/archive/html/qemu-devel/2016-12/msg02812.html

Date: Wed, 21 Dec 2016 00:31:35 +0800
From: Fam Zheng <famz>
Subject: [Qemu-devel] [PATCH 0/4] RFC: A VFIO based block driver for NVMe device

This series adds a new protocol driver that is intended to achieve about 20%
better performance for latency bound workloads (i.e. synchronous I/O) than
linux-aio when guest is exclusively accessing a NVMe device, by talking to the
device directly instead of through kernel file system layers and its NVMe
driver.

This applies on top of Stefan's block-next tree which has the busy polling
patches - the new driver also supports it.

A git branch is also available as:

    https://github.com/famz/qemu nvme

See patch 4 for benchmark numbers.

Tests are done on QEMU's NVMe emulation and a real Intel P3700 SSD NVMe card.
Most of dd/fio/mkfs/kernel build and OS installation testings work well, but an
weird write fault looking similar to [1] is consistently seen when installing
RHEL 7.3 guest, which is still under investigation.

[1]: http://lists.infradead.org/pipermail/linux-nvme/2015-May/001840.html

Also, the ram notifier is not enough for hot plugged block device because in
that case the notifier is installed _after_ ram blocks are added so it won't
get the events.

Comment 1 Kashyap Chamarthy 2018-07-20 10:09:05 UTC

I'm yet to try out the NVMe driver in QEMU, so we still need to investigate how best to expose this in Nova.

That said, I read the dependent bugs from QEMU and libvirt:

  - The QEMU patches seem to be merged 
    <https://bugzilla.redhat.com/show_bug.cgi?id=1416180> and are
    available via: qemu-kvm-rhev-2.12.0-4.el7

  - The libvirt work remains to be done
    <https://bugzilla.redhat.com/show_bug.cgi?id=1416182>

Comment 2 Kashyap Chamarthy 2018-08-22 13:29:04 UTC

This needs to be deferred, as the libvirt upstream work remains to be done, based on the below RHEL-8 bug: https://bugzilla.redhat.com/show_bug.cgi?id=1519005#c2

And as near as I know, we don't have open customer requests for this.

Comment 3 Kashyap Chamarthy 2019-04-08 13:10:48 UTC

(In reply to Kashyap Chamarthy from comment #1)
> I'm yet to try out the NVMe driver in QEMU, so we still need to investigate
> how best to expose this in Nova.
> 
> That said, I read the dependent bugs from QEMU and libvirt:
> 
>   - The QEMU patches seem to be merged 
>     <https://bugzilla.redhat.com/show_bug.cgi?id=1416180> and are
>     available via: qemu-kvm-rhev-2.12.0-4.el7
> 
>   - The libvirt work remains to be done
>     <https://bugzilla.redhat.com/show_bug.cgi?id=1416182>

NOTE: The last URL is the libvirt work for RHEL-7, and it was closed "WONTFIX" for RHEL-7.  

The libvirt work is being tracked as part of RHEL-8 (which is linked in comment#2).

Comment 4 Artom Lifshitz 2022-09-23 13:37:13 UTC

Whlie this would be nice to have, there is no customer case or clear indication of the importance of having this in Nova. Closing as WONTFIX for now. If the situation changes, we can open a new tracker.