Bug 1817773
| Summary: | ppc64le: guest kernel fails to boot if qemu option -virtfs with error "pseries: h_cppr must only be called for emulated XICS" | ||||||
|---|---|---|---|---|---|---|---|
| Product: | [Fedora] Fedora | Reporter: | Gustavo Luiz Duarte <gustavold> | ||||
| Component: | SLOF | Assignee: | Fedora Virtualization Maintainers <virt-maint> | ||||
| Status: | CLOSED ERRATA | QA Contact: | Fedora Extras Quality Assurance <extras-qa> | ||||
| Severity: | medium | Docs Contact: | |||||
| Priority: | unspecified | ||||||
| Version: | 32 | CC: | airlied, bskeggs, bugproxy, clg, crobinso, dan, gkurz, gustavold, hannsj_uhl, hdegoede, ichavero, itamar, jarodwilson, jcajka, jeremy, jglisse, john.j5live, jonathan, josef, kernel-maint, linville, masami256, mchehab, mjg59, pbonzini, rjones, steved, virt-maint | ||||
| Target Milestone: | --- | ||||||
| Target Release: | --- | ||||||
| Hardware: | ppc64le | ||||||
| OS: | Linux | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | SLOF-0.1.git20200327-1.fc32 | Doc Type: | If docs needed, set a value | ||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2020-04-25 02:16:17 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Bug Depends On: | |||||||
| Bug Blocks: | 1071880 | ||||||
| Attachments: |
|
||||||
The guest is using the interrupt mode XICS but XIVE seems to have been negotiated : ... [ 0.000000] NR_IRQS: 512, nr_irqs: 512, preallocated irqs: 16 qemu-system-ppc64: pseries: h_cppr must only be called for emulated XICS [ 0.000000] icp_hv_set_cppr: bad return code cppr cppr=0xff returned -1 [ 0.000000] ------------[ cut here ]------------ [ 0.000000] WARNING: CPU: 0 PID: 0 at arch/powerpc/sysdev/xics/icp-hv.c:44 icp_hv_set_cpu_priority+0x9c/0xb0 ... But something goes wrong in the CAS negotiation process : ... Calling ibm,client-architecture-support...Node not supported Node not supported not implemented ... ------- Comment From KURZGREG.com 2020-03-27 13:53 EDT------- (In reply to comment #7) > Thanks Greg! > So I could build out custom QEMU and SLOF from the src RPMs: qemu-4.2.0-7.fc32.src.rpm SLOF-0.1.git20191022-2.fc32.src.rpm and indeed, the issue is easily reproduced. > Please note that the Fedora QEMU is carrying the upstream patch ("spapr: > Don't trigger a CAS reboot for XICS/XIVE mode changeover"). See: > https://src.fedoraproject.org/rpms/qemu/c/ > 023288b71ad15f5abc09597f209748628e6a7ab2?branch=f32 Ouch... this patch opened a can of worms in SLOF which didn't handle the full rendered FDT at CAS very well. I could have qemu-4.2.0-7.fc32 to work when used with upstream SLOF commit: commit dea21476b10262ef7e01e6e1cedd87fd8d0cfd26 Author: Alexey Kardashevskiy <aik> Date: Wed Feb 5 19:21:37 2020 +0100 fdt: Fix creating new nodes at H_CAS which was officially released with commit: commit 7c98b3b613338a385722a953517349664aa4f532 (tag: qemu-slof-20200221) Author: Alexey Kardashevskiy <aik> Date: Mon Feb 17 12:02:49 2020 +1100 version: update to 20200221 Signed-off-by: Alexey Kardashevskiy <aik> So if you want to keep the cool behavior of not rebooting at CAS on POWER9, I suggest you bump SLOF to this version. Thanks Greg! Moving component to SLOF... can we bump the SLOF version to 20200221 ? FEDORA-2020-15c2b8fb0c has been submitted as an update to Fedora 32. https://bodhi.fedoraproject.org/updates/FEDORA-2020-15c2b8fb0c FEDORA-2020-15c2b8fb0c has been pushed to the Fedora 32 testing repository. In short time you'll be able to install the update with the following command: `sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-15c2b8fb0c` You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-15c2b8fb0c See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates. FEDORA-2020-15c2b8fb0c has been pushed to the Fedora 32 stable repository. If problem still persists, please make note of it in this bug report. |
Created attachment 1673904 [details] full boot log Description of problem: # qemu-system-ppc64 -accel kvm -smp cores=4,threads=4 -nodefaults -nographic -m 2048 -kernel /usr/lib/modules/5.6.0-0.rc7.git0.2.fc32.ppc64le/vmlinuz -initrd /boot/initramfs-5.6.0-0.rc7.git0.2.fc32.ppc64le.img -virtfs local,id=workdir,path=/srv,security_model=none,mount_tag=workdir -serial stdio -append 'rd.break' [ 0.000000] NR_IRQS: 512, nr_irqs: 512, preallocated irqs: 16 qemu-system-ppc64: pseries: h_cppr must only be called for emulated XICS [ 0.000000] icp_hv_set_cppr: bad return code cppr cppr=0xff returned -1 [ 0.000000] ------------[ cut here ]------------ [ 0.000000] WARNING: CPU: 0 PID: 0 at arch/powerpc/sysdev/xics/icp-hv.c:44 icp_hv_set_cpu_priority+0x9c/0xb0 [ 0.000000] Modules linked in: [ 0.000000] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.6.0-0.rc7.git0.2.fc32.ppc64le #1 [ 0.000000] NIP: c0000000000c870c LR: c0000000000c8708 CTR: c000000002c40000 [ 0.000000] REGS: c0000000029f7aa0 TRAP: 0700 Not tainted (5.6.0-0.rc7.git0.2.fc32.ppc64le) [ 0.000000] MSR: 8000000002021033 <SF,VEC,ME,IR,DR,RI,LE> CR: 28000442 XER: 20040000 [ 0.000000] CFAR: c000000000215844 IRQMASK: 1 [ 0.000000] GPR00: c0000000000c8708 c0000000029f7d30 c0000000029fbd00 000000000000003b [ 0.000000] GPR04: 0000000000000001 0000000000000000 0a57af02000000c0 5837af02000000c0 [ 0.000000] GPR08: 000000007d2e0000 c0000000025d5240 0000000000000000 0000000000000004 [ 0.000000] GPR12: 0000000000000000 c000000002c40000 00000000016a6c10 00000000016a6f18 [ 0.000000] GPR16: 00000000016a6d40 00000000016a72f8 00000000016dc0a0 00000000017a10d8 [ 0.000000] GPR20: 00000000016a7320 0000000000000002 fffffffffffffffd 0000000002e7bd00 [ 0.000000] GPR24: c000000002073808 c00000007fff2400 c000000002073808 0000000000000000 [ 0.000000] GPR28: c00000007fff2f80 0000000000000000 c000000002ab0080 00000000000000ff [ 0.000000] NIP [c0000000000c870c] icp_hv_set_cpu_priority+0x9c/0xb0 [ 0.000000] LR [c0000000000c8708] icp_hv_set_cpu_priority+0x98/0xb0 [ 0.000000] Call Trace: [ 0.000000] [c0000000029f7d30] [c0000000000c8708] icp_hv_set_cpu_priority+0x98/0xb0 (unreliable) [ 0.000000] [c0000000029f7da0] [c0000000000c7554] xics_setup_cpu+0x44/0xe0 [ 0.000000] [c0000000029f7e10] [c000000002021a10] pseries_init_irq+0x4c/0x250 [ 0.000000] [c0000000029f7eb0] [c000000002007c24] init_IRQ+0x3c/0x50 [ 0.000000] [c0000000029f7ed0] [c0000000020046d0] start_kernel+0x784/0xa2c [ 0.000000] [c0000000029f7f90] [c00000000000accc] start_here_common+0x1c/0x3e8 [ 0.000000] Instruction dump: [ 0.000000] e8010010 ebe1fff8 7c0803a6 4e800020 7c661b78 3c82fe63 3c62fe8c 7fe5fb78 [ 0.000000] 38849510 3863b398 4814d0f9 60000000 <0fe00000> 4bffffc4 60000000 60000000 [ 0.000000] ---[ end trace fb262ad5a6fb6046 ]--- The kernel boots fine if the -virtfs option is removed from the qemu command line. Host system is a POWER9 ZZ ppc64le Opal server. Version-Release number of selected component (if applicable): qemu-system-ppc-core-4.2.0-7.fc32.ppc64le host: kernel-4.18.0-187.el8.ppc64le (I'm running the guest from a Fedora 32 container on a RHEL8 kernel) guest: kernel-5.6.0-0.rc7.git0.2.fc32.ppc64le How reproducible: Always Steps to Reproduce: 1.qemu-system-ppc64 -accel kvm -smp cores=4,threads=4 -nodefaults -nographic -m 2048 -kernel /usr/lib/modules/5.6.0-0.rc7.git0.2.fc32.ppc64le/vmlinuz -initrd /boot/initramfs-5.6.0-0.rc7.git0.2.fc32.ppc64le.img -virtfs local,id=workdir,path=/srv,security_model=none,mount_tag=workdir -serial stdio -append 'rd.break' 2. 3. Actual results: Expected results: Kernel boots into dracut shell Additional info: