Bug 2003706

Summary: virt-install armv7 guest on aarch64 host doesn't work, no LPAE support and no PCI devices
Product: [Fedora] Fedora Reporter: Richard W.M. Jones <rjones>
Component: virt-managerAssignee: Cole Robinson <crobinso>
Status: CLOSED WONTFIX QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 34CC: berrange, crobinso, philip.wyett, pwhalen, rjones
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-02-24 21:07:41 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 245418, 1517014    
Attachments:
Description Flags
dmesg output none

Description Richard W.M. Jones 2021-09-13 13:34:15 UTC 
Description of problem:

$ virt-install \
  --transient \
  --name=tmp-jffrn1m6 \
  --ram=4096 \
  --arch=armv7l \
  --features pae=on \
  --vcpus=1 \
  --os-variant=fedora34 \
  --initrd-inject=fedora-34-armv7l.ks \
  --extra-args="inst.ks=file:/fedora-34-armv7l.ks  console=tty0 console=ttyAMA0,115200 rd_NO_PLYMOUTH" \
  --disk=/home/rjones/d/guestfs-tools/builder/templates/tmp-jffrn1m6.img,size=6,format=raw \
  --location=https://mirror.bytemark.co.uk/fedora/linux/releases/34/Server/armhfp/os/ \
  --serial=pty \
  --nographics

The kickstart is fairly uninteresting but I'll attach it.

This fails at the dracut stage.  Inside the rescue shell we can see
that there are no virtio devices, because no PCI devices.

Version-Release number of selected component (if applicable):

virt-install-3.2.0-3.fc34.noarch
qemu-6.1.0-5.fc36.aarch64

How reproducible:

100%

Steps to Reproduce:
1. See above
Comment 1 Richard W.M. Jones 2021-09-13 13:35:55 UTC 
Created attachment 1822773 [details]
dmesg output

I should note the --features pae=on flag seems to make no difference.

There is a weird error in the attached dmesg output:

[   11.737291] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 rang
es:
[   11.741242] pci-host-generic 4010000000.pcie:       IO 0x003eff0000..0x003eff
ffff -> 0x0000000000
[   11.745955] pci-host-generic 4010000000.pcie:      MEM 0x0010000000..0x003efe
ffff -> 0x0010000000
[   11.750621] pci-host-generic 4010000000.pcie:      MEM 0x8000000000..0xffffff
ffff -> 0x8000000000
[   11.755398] pci-host-generic 4010000000.pcie: can't claim ECAM area [mem 0x10000000-0x1fffffff]: address conflict with pcie@10000000 [mem 0x10000000-0x3efeffff]
[   11.762959] pci-host-generic: probe of 4010000000.pcie failed with error -16

I previously tried with less RAM (2G or 3.5G) and it made no difference.
Comment 2 Richard W.M. Jones 2021-09-13 13:37:31 UTC 
Created attachment 1822774 [details]
fedora-34-armv7l.ks
Comment 3 Richard W.M. Jones 2021-09-13 13:46:52 UTC 
Seems very similar to bug 1633328.

"virt-install --machine virt-2.11" workaround still works.
Comment 4 Richard W.M. Jones 2021-09-13 13:47:15 UTC 
*** Bug 1633328 has been marked as a duplicate of this bug. ***
Comment 5 Richard W.M. Jones 2021-09-14 07:50:42 UTC 
Another more direct workaround would be to use qemu's
-machine highmem=off flag.  However virt-install doesn't
support passing this flag, and I'm not clear that libvirt
does either.
Comment 6 Cole Robinson 2022-02-24 20:07:11 UTC 
Rich did the VM use uefi? I think nowadays fedora arm32 -M virt UEFI is basically required. virt-install will prefer it if it's available. Need to install edk2-arm

I just added a Requires: edk2-arm to qemu-system-arm in f36+, that was missing.

And yeah, highmem= flag was never added to libvirt
Comment 7 Richard W.M. Jones 2022-02-24 21:07:41 UTC 
I don't know the answer, but ...

I'm going to close this because I was trying to build a 32 bit armv7
Fedora image for virt-builder.  32 bit armv7 is now dead in Fedora >= 37.