RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1237250 - aarch64: libguestfs should now prefer virtio-pci
Summary: aarch64: libguestfs should now prefer virtio-pci
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: libguestfs
Version: 7.2
Hardware: aarch64
OS: Unspecified
high
medium
Target Milestone: rc
: ---
Assignee: Richard W.M. Jones
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On: 1359086
Blocks: 1212027 1221569 1288337 1301891
TreeView+ depends on / blocked
 
Reported: 2015-06-30 15:52 UTC by Andrew Jones
Modified: 2017-08-01 22:08 UTC (History)
8 users (show)

Fixed In Version: libguestfs-1.36.1-1.el7
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-08-01 22:08:55 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)
log.txt (89.39 KB, text/plain)
2015-06-30 16:30 UTC, Richard W.M. Jones 		no flags Details
0001-arm-Prefer-virtio-pci-instead-of-virtio-mmio-RHBZ-12.patch (4.19 KB, patch)
2015-06-30 18:06 UTC, Richard W.M. Jones 		no flags Details | Diff
the output with "LIBGUESTFS_BACKEND=direct libguestfs-test-tool" (211.93 KB, text/plain)
2017-03-31 12:45 UTC, YongkuiGuo 		no flags Details
the output with "libguestfs-test-tool" (217.20 KB, text/plain)
2017-03-31 12:47 UTC, YongkuiGuo 		no flags Details
../libvirt/qemu/log/guestfs-xxx.log for libvirt case (2.95 KB, text/plain)
2017-03-31 12:55 UTC, YongkuiGuo 		no flags Details
View All


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2017:2023 0 normal SHIPPED_LIVE libguestfs bug fix and enhancement update 2017-08-01 19:32:01 UTC

Description Andrew Jones 2015-06-30 15:52:49 UTC 
RHELSA 7.2 qemu-kvm-rhev will very soon support ACPI generation (patches getting merged now). The RHELSA 7.2 guest kernel has support for the PCIe host bridge (which is also in the virt-rhelsa7.2 machine type) when it is boot with ACPI. We can now prefer using virtio-pci instead of virtio-mmio and should.
Comment 1 Richard W.M. Jones 2015-06-30 16:29:53 UTC 
I couldn't get this to work upstream, unfortunately.  Any tips?
I'm using latest qemu from git and kernel 4.1.0-0.rc7.git0.1.fc23.aarch64

The qemu command line is:

/home/rjones/d/qemu/aarch64-softmmu/qemu-system-aarch64 \
    -global virtio-blk-pci.scsi=off \
    -nodefconfig \
    -enable-fips \
    -nodefaults \
    -display none \
    -M virt \
    -cpu host \
    -machine accel=kvm:tcg \
    -m 768 \
    -no-reboot \
    -rtc driftfix=slew \
    -global kvm-pit.lost_tick_policy=discard \
    -drive if=pflash,format=raw,file=/usr/share/edk2.git/aarch64/QEMU_EFI-pflash.raw,readonly \
    -drive if=pflash,format=raw,file=/home/rjones/d/libguestfs/tmp/libguestfsUATPeC/vars.fd.2 \
    -kernel /home/rjones/d/libguestfs/tmp/.guestfs-1000/appliance.d/kernel \
    -initrd /home/rjones/d/libguestfs/tmp/.guestfs-1000/appliance.d/initrd \
    -device virtio-scsi-pci,id=scsi \
    -drive file=/home/rjones/d/libguestfs/tmp/libguestfsUATPeC/scratch.1,cache=unsafe,format=raw,id=hd0,if=none \
    -device scsi-hd,drive=hd0 \
    -drive file=/home/rjones/d/libguestfs/tmp/.guestfs-1000/appliance.d/root,snapshot=on,id=appliance,cache=unsafe,if=none \
    -device scsi-hd,drive=appliance \
    -device virtio-serial-pci \
    -serial stdio \
    -chardev socket,path=/home/rjones/d/libguestfs/tmp/libguestfsUATPeC/guestfsd.sock,id=channel0 \
    -device virtserialport,chardev=channel0,name=org.libguestfs.channel.0 \
    -append 'panic=1 console=ttyAMA0 earlyprintk=pl011,0x9000000 ignore_loglevel efi-rtc=noprobe udevtimeout=6000 udev.event-timeout=6000 no_timer_check acpi=off printk.time=1 cgroup_disable=memory root=/dev/sdb selinux=0 guestfs_verbose=1 TERM=xterm-256color'

UEFI loads and runs, apparently normally.

The kernel loads and starts running, apparently normally.

However the kernel cannot see the SCSI drives.

There are virtio_scsi.ko and virtio_pci.ko modules and they are loaded by
the guest kernel.

I will attach the full log shortly.
Comment 2 Richard W.M. Jones 2015-06-30 16:30:41 UTC 
Created attachment 1044732 [details]
log.txt
Comment 3 Richard W.M. Jones 2015-06-30 16:35:44 UTC 
No difference after removing acpi=off from the command line.
Comment 4 Richard W.M. Jones 2015-06-30 18:06:30 UTC 
Created attachment 1044770 [details]
0001-arm-Prefer-virtio-pci-instead-of-virtio-mmio-RHBZ-12.patch

The patch would be something like this one, but as mentioned
above I cannot get it to work on Fedora.
Comment 5 Richard W.M. Jones 2015-06-30 18:16:54 UTC 
There's an interesting thing: The patch in comment 3 *works* on
armv7hl.  Now it's not an exact comparison because I'm using quite
a different (actually, somewhat older) kernel on 32 bit ARM:
4.0.6-300.fc22.armv7hl+lpae

So I don't exactly know what to make of that.
Comment 6 Andrew Jones 2015-07-01 10:11:51 UTC 
(In reply to Richard W.M. Jones from comment #1)
> I couldn't get this to work upstream, unfortunately.  Any tips?
> I'm using latest qemu from git and kernel 4.1.0-0.rc7.git0.1.fc23.aarch64
> 

I'm not sure the Fedora kernel has everything it needs for ACPI+PCI. I was able to boot a RHELSA guest with a nearly identical commandline as is in comment 1. I only changed the kernel and initrd to 4.1.0-0.rc7.10.el7.aarch64 (the initrd also needed virtio-pci added), changed the root disk image path to my guest image, and removed acpi=off from the command line. I also used latest qemu from git.

I'm not sure what to make of comment 5 either (I haven't looked at the status of the arm kernel and pci support), but I guess it's possible that the PCIe host bridge support actually works with that guest kernel and devicetree.
Comment 8 Richard W.M. Jones 2016-06-27 15:37:18 UTC 
In RHEL 7.3 virtio-mmio will be preferred.  In 7.4 we can look
at defaulting to virtio-pci.
Comment 9 Jon Masters 2016-08-30 07:17:22 UTC 
Thanks - looking forward to 7.4 :)
Comment 10 Richard W.M. Jones 2017-02-16 14:20:33 UTC 
Upstream commit is 4a9af91e3632047579c3a7e011c1484c97bd959a, so this
is expected to be fixed in the rebase.
Comment 12 YongkuiGuo 2017-03-31 11:27:57 UTC 
packages info:
libguestfs-1.36.3-1.el7.aarch64
qemu-kvm-rhev-2.6.0-28.el7_3.9.aarch64
kernel-4.5.0-15.el7.aarch64

I didn't understand this bug clearly. I just executed the command of libguestfs-test-tool on a rhel7.3 machine with aarch64 cpu. I don't kown how to verify whether virtio-pci has been used instead of virtio-mmio in new libguestfs version. Any other tips that you can give me about the reproduce steps? thanks.
Comment 13 Richard W.M. Jones 2017-03-31 11:37:18 UTC 
(In reply to YongkuiGuo from comment #12)
> packages info:
> libguestfs-1.36.3-1.el7.aarch64
> qemu-kvm-rhev-2.6.0-28.el7_3.9.aarch64
> kernel-4.5.0-15.el7.aarch64
> 
> I didn't understand this bug clearly. I just executed the command of
> libguestfs-test-tool on a rhel7.3 machine with aarch64 cpu. I don't kown how
> to verify whether virtio-pci has been used instead of virtio-mmio in new
> libguestfs version. Any other tips that you can give me about the reproduce
> steps? thanks.

libguestfs-test-tool is the correct approach.  Can you attach the
full log from that to this bug and I will show you what to look
for.
Comment 14 Richard W.M. Jones 2017-03-31 11:38:48 UTC 
(In reply to Richard W.M. Jones from comment #13)
> (In reply to YongkuiGuo from comment #12)
> > packages info:
> > libguestfs-1.36.3-1.el7.aarch64
> > qemu-kvm-rhev-2.6.0-28.el7_3.9.aarch64
> > kernel-4.5.0-15.el7.aarch64
> > 
> > I didn't understand this bug clearly. I just executed the command of
> > libguestfs-test-tool on a rhel7.3 machine with aarch64 cpu. I don't kown how
> > to verify whether virtio-pci has been used instead of virtio-mmio in new
> > libguestfs version. Any other tips that you can give me about the reproduce
> > steps? thanks.
> 
> libguestfs-test-tool is the correct approach.  Can you attach the
> full log from that to this bug and I will show you what to look
> for.

And I should say that you actually need to run libguestfs-test-tool
both ways:

LIBGUESTFS_BACKEND=direct libguestfs-test-tool > test.direct 2>&1
libguestfs-test-tool > test.libvirt 2>&1

and for the libvirt case you'll also need to find the right file
~/.cache/libvirt/qemu/log/guestfs-XXXX.log and attach that.
Comment 15 YongkuiGuo 2017-03-31 12:45:56 UTC 
Created attachment 1267845 [details]
the output with "LIBGUESTFS_BACKEND=direct libguestfs-test-tool"
Comment 16 YongkuiGuo 2017-03-31 12:47:07 UTC 
Created attachment 1267846 [details]
the output with "libguestfs-test-tool"
Comment 17 Richard W.M. Jones 2017-03-31 12:54:39 UTC 
(In reply to YongkuiGuo from comment #15)
> Created attachment 1267845 [details]
> the output with "LIBGUESTFS_BACKEND=direct libguestfs-test-tool"

In this one (running qemu directly), notice the use of virtio-scsi-pci:

    -device virtio-scsi-pci,id=scsi \
    -drive file=/tmp/libguestfs9QSxvc/scratch.1,cache=unsafe,format=raw,id=hd0,if=none \
    -device scsi-hd,drive=hd0 \
    -drive file=/var/tmp/.guestfs-0/appliance.d/root,snapshot=on,id=appliance,cache=unsafe,if=none,format=raw \
    -device scsi-hd,drive=appliance \

Previously this would have been using 'virtio-scsi-device' which
is qemu's name for virtio-mmio.

(In reply to YongkuiGuo from comment #16)
> Created attachment 1267846 [details]
> the output with "libguestfs-test-tool"

You'll need to find /var/log/libvirt/qemu/guestfs-mn4kkegshfkaihjr.log
to see the differences, since there is no visible difference in
the libvirt XML that libguestfs generates, but there should be
a difference in the qemu command line which libvirt generates.
Comment 18 YongkuiGuo 2017-03-31 12:55:49 UTC 
Created attachment 1267852 [details]
../libvirt/qemu/log/guestfs-xxx.log for libvirt case
Comment 19 Richard W.M. Jones 2017-03-31 13:08:45 UTC 
(In reply to YongkuiGuo from comment #18)
> Created attachment 1267852 [details]
> ../libvirt/qemu/log/guestfs-xxx.log for libvirt case

In this one it's the presence again of:

-device virtio-scsi-pci,id=scsi0,bus=pci.1,addr=0x0

which shows that this is using virtio-pci.  If it said
"virtio-scsi-device", it would still be using virtio-mmio.

This bug can be marked as VERIFIED based on the evidence provided.
Comment 20 YongkuiGuo 2017-03-31 13:26:22 UTC 
In the file of guestfs-mn4kkegshfkaihjr.log, I find out  virtio-scsi-pci, virtio-serial-pci and virtio-rng-pci, which were modified in 0001-arm-Prefer-virtio-pci-instead-of-virtio-mmio-RHBZ-12.patch. Now I understand. Thanks again.
Comment 21 YongkuiGuo 2017-03-31 13:40:54 UTC 
Verified with packages info:
libguestfs-1.36.3-1.el7.aarch64


Step:
1. Prepare a rhel7.3 machine with aarch64 cpu
2. LIBGUESTFS_BACKEND=direct libguestfs-test-tool
---------------------------------------
... ...
device virtio-scsi-pci,id=scsi \
    -drive file=/tmp/libguestfs9QSxvc/scratch.1,cache=unsafe,format=raw,id=hd0,if=none \
----------------------------------------
3. libguestfs-test-tool
4. cat ./libvirt/qemu/log/guestfs-xxx.log
----------------------------------------
... ...
-device virtio-scsi-pci,id=scsi0,bus=pci.1,addr=0x0
-device virtio-serial-pci,id=virtio-serial0,bus=pci.2,addr=0x0
-device virtio-rng-pci,rng=objrng0,id=rng0,bus=pci.3,addr=0x0
----------------------------------------

It is using virtio-pci clearly,rather than virtio-mmio. 
So verified.
Comment 22 errata-xmlrpc 2017-08-01 22:08:55 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2023
Note You need to log in before you can comment on or make changes to this bug.