Bug 1668751 - Kernel 4.20.3-200.fc29.ppc64le not booting with multipath storage
Summary: Kernel 4.20.3-200.fc29.ppc64le not booting with multipath storage
Status: CLOSED DUPLICATE of bug 1669235
Alias: None
Product: Fedora
Classification: Fedora
Component: kernel
Version: 29
Hardware: ppc64le
OS: Linux
unspecified
high
Target Milestone: ---
Assignee: Kernel Maintainer List
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Keywords:
Depends On:
Blocks: PPCTracker
TreeView+ depends on / blocked
 
Reported: 2019-01-23 14:08 UTC by Christoph Buchmann
Modified: 2019-02-22 09:55 UTC (History)
27 users (show)

(edit)
Clone Of:
(edit)
Last Closed: 2019-02-22 09:55:20 UTC


Attachments (Terms of Use)
journalctl --no-hostname -b | grep -v sshd > dmesg.txt (127.58 KB, text/plain)
2019-01-23 14:08 UTC, Christoph Buchmann
no flags Details
qemu log from FAH 20190205 (12.68 KB, text/plain)
2019-02-08 11:13 UTC, Sinny Kumari
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
IBM Linux Technology Center 175460 None None None 2019-05-07 08:40 UTC

Description Christoph Buchmann 2019-01-23 14:08:33 UTC
Created attachment 1522677 [details]
journalctl --no-hostname -b | grep -v sshd > dmesg.txt

1. Please describe the problem:
after updating fedora 29 from 4.18.16-300.fc29.ppc64le to 4.20.3-200.fc29.ppc64le Fedora did not boot anymore. Rebooting with Kernel 4.18 worked
Stopped boot process after crng:
[    9.658698] random: crng done (trusting CPU's manufacturer)
[  143.141563] dracut-initqueue[449]: Warning: dracut-initqueue timeout - starting timeout scripts

2. What is the Version-Release number of the kernel:
4.20.3-200

3. Did it work previously in Fedora? If so, what kernel version did the issue
   *first* appear?  Old kernels are available for download at
   https://koji.fedoraproject.org/koji/packageinfo?packageID=8 :


4. Can you reproduce this issue? If so, please provide the steps to reproduce
   the issue below:
yes, reboot using Kernel 4.20.3-200

5. Does this problem occur with the latest Rawhide kernel? To install the
   Rawhide kernel, run ``sudo dnf install fedora-repos-rawhide`` followed by
   ``sudo dnf update --enablerepo=rawhide kernel``:
this actually fails: 
warning: /var/cache/dnf/rawhide-1c578a7639193129/packages/kernel-5.0.0-0.rc3.git0.1.fc30.ppc64le.rpm: Header V3 RSA/SHA256 Signature, key ID cfc659b9: NOKEY
Fedora - Rawhide - Developmental packages for the next Fedora release                                                          1.6 kB/s | 1.6 kB     00:01
Importing GPG key 0x429476B4:
 Userid     : "Fedora 29 (29) <fedora-29@fedoraproject.org>"
 Fingerprint: 5A03 B4DD 8254 ECA0 2FDA 1637 A20A A56B 4294 76B4
 From       : /etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-29-ppc64le
Is this ok [y/N]: y
Key imported successfully
Import of key(s) didn't help, wrong key(s)?
Public key for kernel-5.0.0-0.rc3.git0.1.fc30.ppc64le.rpm is not installed. Failing package is: kernel-5.0.0-0.rc3.git0.1.fc30.ppc64le
 GPG Keys are configured as: file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-29-ppc64le
Public key for kernel-core-5.0.0-0.rc3.git0.1.fc30.ppc64le.rpm is not installed. Failing package is: kernel-core-5.0.0-0.rc3.git0.1.fc30.ppc64le
 GPG Keys are configured as: file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-29-ppc64le
Public key for kernel-modules-5.0.0-0.rc3.git0.1.fc30.ppc64le.rpm is not installed. Failing package is: kernel-modules-5.0.0-0.rc3.git0.1.fc30.ppc64le
 GPG Keys are configured as: file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-29-ppc64le
The downloaded packages were saved in cache until the next successful transaction.
You can remove cached packages by executing 'dnf clean packages'.
Error: GPG check FAILED

6. Are you running any modules that not shipped with directly Fedora's kernel?:
no

7. Please attach the kernel logs. You can get the complete kernel log
   for a boot with ``journalctl --no-hostname -k > dmesg.txt``. If the
   issue occurred on a previous boot, use the journalctl ``-b`` flag.

this actually does not work either and shows the log of kernel 4.18 only, but I've nevertheless attached the output

Comment 1 Sinny Kumari 2019-02-05 15:11:59 UTC
Even I am seeing similar issue on ppc64le with kernel-4.20.3-200.fc29.ppc64le .

I am trying to install a fresh vm using Atomic Host iso from https://kojipkgs.fedoraproject.org/compose/updates/Fedora-29-updates-20190122.0/compose/AtomicHost/ppc64le/iso/Fedora-AtomicHost-ostree-ppc64le-29-20190122.0.iso . Vm install hangs at following steps during boot:

# virt-install --name fah-20190122 --ram 2048 --vcpus 1 --os-type=linux --os-variant=fedora26 --disk path=/var/lib/libvirt/images/f29_20190122.qcow2,size=10,bus=virtio,format=qcow2 --network bridge=virbr0 --nographic --check path_in_use=off -c Fedora-AtomicHost-ostree-ppc64le-29-20190122.0.iso

...
OF stdout device is: /vdevice/vty@30000000
Preparing to boot Linux version 4.20.3-200.fc29.ppc64le (mockbuild@buildvm-ppc64le-04.ppc.fedoraproject.org) (gcc version 8.2.1 20181215 (Red Hat 8.2.1-6) (GCC)) #1 SMP Thu Jan 17 15:04:41 UTC 2019
Detected machine type: 0000000000000101
command line: BOOT_IMAGE=/ppc/ppc64/vmlinuz ro
Max number of cores passed to firmware: 1024 (NR_CPUS = 1024)
Calling ibm,client-architecture-support... done
memory layout at init:
  memory_limit : 0000000000000000 (16 MB aligned)
  alloc_bottom : 00000000070b0000
  alloc_top    : 0000000030000000
  alloc_top_hi : 0000000080000000
  rmo_top      : 0000000030000000
  ram_top      : 0000000080000000
instantiating rtas at 0x000000002fff0000... done
prom_hold_cpus: skipped
copying OF device tree...
Building dt strings...
Building dt structure...
Device tree strings 0x00000000070c0000 -> 0x00000000070c0ad5
Device tree struct  0x00000000070d0000 -> 0x00000000070e0000
Quiescing Open Firmware ...
Booting Linux via __start() @ 0x0000000002000000 ...


kernel 4.20.3-200.fc29 was introduced in Atomic Host compose Fedora-29-updates-20190122.0 . Atomic Host iso from compose Fedora-29-updates-20190121.0(https://kojipkgs.fedoraproject.org/compose/updates/Fedora-29-updates-20190121.0/compose/AtomicHost/ppc64le/iso/Fedora-AtomicHost-ostree-ppc64le-29-20190121.0.iso) contains kernel 4.19.15-300.fc29 which boots perfectly fine.

By looking at rpm-ostree diff for commit from Fedora-29-updates-20190121.0 and Fedora-29-updates-20190122.0 , it also shows that only kernel was updated:
$ rpm-ostree db diff cdcbea2ccac7804770be806befd30895457de080d1525ee6050a5bebdfeefeb7 dfa61b4b9c11713dca18df38df705e9f742e78e3f561d10faf1c9265fea331e3
ostree diff commit old: cdcbea2ccac7804770be806befd30895457de080d1525ee6050a5bebdfeefeb7
ostree diff commit new: dfa61b4b9c11713dca18df38df705e9f742e78e3f561d10faf1c9265fea331e3
Upgraded:
  kernel 4.19.15-300.fc29 -> 4.20.3-200.fc29
  kernel-core 4.19.15-300.fc29 -> 4.20.3-200.fc29
  kernel-modules 4.19.15-300.fc29 -> 4.20.3-200.fc29

Comment 2 Dan Horák 2019-02-05 16:33:00 UTC
kernel-4.20.6-200.fc29.ppc64le boots fine in my VM, Power9 HW, qemu 3.1.0 on host

Comment 3 Sinny Kumari 2019-02-05 16:59:45 UTC
(In reply to Dan Horák from comment #2)
> kernel-4.20.6-200.fc29.ppc64le boots fine in my VM, Power9 HW, qemu 3.1.0 on
> host

Tried booting image with kernel-4.20.6-200.fc29.ppc64le (https://kojipkgs.fedoraproject.org/compose/updates/Fedora-29-updates-20190205.0/compose/AtomicHost/ppc64le/images/Fedora-AtomicHost-29-20190205.0.ppc64le.qcow2) , same hang result for me.
I am using Power8 guest.

Dan, can you try on Power8 and let us know the result?

Comment 4 Sinny Kumari 2019-02-05 17:36:58 UTC
Christoph, Where are you seeing the issue, on Power 8 or Power9?

Comment 5 Christoph Buchmann 2019-02-05 19:08:44 UTC
My machine is a Power 8 (S822)

Comment 6 Michel Normand 2019-02-07 16:51:59 UTC
FYI: I have an openQA test of f29, running on a f29 P8 machine, and do not have net install problem of last delivered kernel, the guest properly rebooted with kernel kernel-4.20.6-200.fc29.ppc64le

Comment 7 Sinny Kumari 2019-02-07 17:52:37 UTC
(In reply to Michel Normand from comment #6)
> FYI: I have an openQA test of f29, running on a f29 P8 machine, and do not
> have net install problem of last delivered kernel, the guest properly
> rebooted with kernel kernel-4.20.6-200.fc29.ppc64le

Thanks Michel for the information.
I also observed that I have a successfully installed and booted F28/F29 ppc64le vm (P8 with kernel 4.20.6-100.fc{28,29}.ppc64le ) with netboot.

But on those vm, when I using virt-install to boot image for example qcow2 image it hangs. Observed same with latest FAH qcow2 ( https://kojipkgs.fedoraproject.org/compose/updates/Fedora-29-updates-20190205.0/compose/AtomicHost/ppc64le/images/Fedora-AtomicHost-29-20190205.0.ppc64le.qcow2) and F29-cloud-base (https://kojipkgs.fedoraproject.org/compose/cloud/Fedora-Cloud-29-20190205.0/compose/Cloud/ppc64le/images/Fedora-Cloud-Base-29-20190205.0.ppc64le.qcow2)


command I used was something like:
$ virt-install --name VM_NAME --ram 1024 --vcpus 1 --disk /var/lib/libvirt/images/IMAGE.ppc64le.qcow2 --os-type linux --os-variant fedora26 --network bridge=virbr0 --nographics  --cdrom /var/lib/libvirt/images/init.iso

This works fine on same host with older qcow2 image which has kernel 4.19.*

Also, on same host I booted a working Atomic Host qcow2 image (https://kojipkgs.fedoraproject.org/compose/updates/Fedora-29-updates-20190119.0/compose/AtomicHost/ppc64le/images/Fedora-AtomicHost-29-20190119.0.ppc64le.qcow2) which boots perfectly fine. Later on updated just kernel packages to 4.20.6-100.fc29.ppc64le and after reboot vm hangs at same place.

Can you try using any of these scenarios mentioned above using virt-install and let us know?

Comment 8 Sinny Kumari 2019-02-07 17:55:13 UTC
Just a note, on Atomic Host system you can override kernel (base package) using:

$ sudo rpm-ostree override replace pkg1 pkg2 ...

Comment 9 Jakub Čajka 2019-02-08 08:49:38 UTC
For the record I have hit this(or the issue Sinny is describing) with nested virtualization on P8 machine. It hangs after kexecing from the fw with one CPU core used by the qemu at 100%.
Phy  Host  RHEL 7    kernel-3.10.0-957.1.3.el7.ppc64le qemu-2.12.0-18.el7_6.1.ppc64le
Virt Host  Fedora 29 kernel-4.20.6-200.fc29.ppc64le qemu-2.12.0-3.fc39 (run in privileged container)
Virt Guest Fedora 29 kernel-4.20.6-200.fc29.ppc64le

...
Booting from memory...
OF stdout device is: /vdevice/vty@71000000
Preparing to boot Linux version 4.20.6-200.fc29.ppc64le (mockbuild@buildvm-ppc64le-06.ppc.fedoraproject.org) (gcc version 8.2.1 20181215 (Red Hat 8.2.1-6) (GCC)) #1 SMP Thu Jan 31 15:31:01 UTC 2019
Detected machine type: 0000000000000101
command line: panic=1 console=hvc0 console=ttyS0 edd=off udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=/dev/sdb selinux=0 guestfs_verbose=1 TERM=xterm
Max number of cores passed to firmware: 1024 (NR_CPUS = 1024)
Calling ibm,client-architecture-support... done
memory layout at init:
  memory_limit : 0000000000000000 (16 MB aligned)
  alloc_bottom : 0000000001cc0000
  alloc_top    : 0000000030000000
  alloc_top_hi : 0000000040000000
  rmo_top      : 0000000030000000
  ram_top      : 0000000040000000
instantiating rtas at 0x000000002fff0000... done
prom_hold_cpus: skipped
copying OF device tree...
Building dt strings...
Building dt structure...
Device tree strings 0x0000000001ed0000 -> 0x0000000001ed0aa6
Device tree struct  0x0000000001ee0000 -> 0x0000000001ef0000
Quiescing Open Firmware ...
Booting Linux via __start() @ 0x0000000000400000 ...

Comment 10 Christoph Buchmann 2019-02-08 09:20:25 UTC
My environment is PowerVM (so no nested virtualization) on a IBM 8247-22L. The LPAR is running with 2 cores and 8GB RAM.

Comment 11 Christoph Buchmann 2019-02-08 10:16:16 UTC
It's possible that the problem is related to disk or multipathd:
Here's what I see, when I'm booting with kernel 4.20:
[...]
[    2.046685] sr 0:0:1:0: Attached scsi generic sg0 type 5
[    2.046981] sd 0:0:2:0: Attached scsi generic sg1 type 0
[    2.047598] sd 0:0:2:0: [sda] 62914560 512-byte logical blocks: (32.2 GB/30.0 GiB)
[    2.047669] sd 0:0:2:0: [sda] Write Protect is off
[    2.047753] sd 0:0:2:0: [sda] Cache data unavailable
[    2.047760] sd 0:0:2:0: [sda] Assuming drive cache: write through
[    2.049168]  sda: sda1 sda2 sda3
[    2.050343] sd 0:0:2:0: [sda] Attached SCSI disk
[    2.072695] sd 1:0:1:0: Attached scsi generic sg2 type 0
[    2.072945] sd 1:0:1:0: [sdb] 62914560 512-byte logical blocks: (32.2 GB/30.0 GiB)
[    2.073017] sd 1:0:1:0: [sdb] Write Protect is off
[    2.073087] sd 1:0:1:0: [sdb] Cache data unavailable
[    2.073094] sd 1:0:1:0: [sdb] Assuming drive cache: write through
[    2.074453]  sdb: sdb1 sdb2 sdb3
[    2.075586] sd 1:0:1:0: [sdb] Attached SCSI disk
[  OK  ] Started udev Wait for Complete Device Initialization.
         Starting Device-Mapper Multipath Device Controller...
[  OK  ] Started Device-Mapper Multipath Device Controller.
[  OK  ] Reached target Local File Systems (Pre).
[  OK  ] Reached target Local File Systems.
         Starting Create Volatile Files and Directories...
[  OK  ] Started Create Volatile Files and Directories.
[  OK  ] Reached target System Initialization.
[  OK  ] Reached target Basic System.
[    2.194757] device-mapper: multipath service-time: version 0.3.0 loaded
[    2.195095] device-mapper: table: table load rejected: not all devices are blk-mq request-stackable
[    2.195108] device-mapper: table: unable to determine table type
[    2.198454] device-mapper: table: table load rejected: not all devices are blk-mq request-stackable
[    2.198471] device-mapper: table: unable to determine table type
[    8.988724] random: crng init done
[    8.988741] random: 7 urandom warning(s) missed due to ratelimiting
[    8.988762] random: crng done (trusting CPU's manufacturer)

---> stops here, later I'm falling to an emergency shell: 
dracut:/# multipathd -k
multipathd> list paths
hcil    dev dev_t pri dm_st chk_st dev_st  next_check
0:0:2:0 sda 8:0   1   undef undef  unknown orphan
1:0:1:0 sdb 8:16  1   undef undef  unknown orphan

(no fdisk or lsblk in this shell)

*****
This shows up when (successfully) booting with kernel 4.18:
[...]
[    2.295306] sr 0:0:1:0: Attached scsi generic sg0 type 5
[    2.295555] sd 0:0:2:0: Attached scsi generic sg1 type 0
[    2.296082] sd 0:0:2:0: [sda] 62914560 512-byte logical blocks: (32.2 GB/30.0 GiB)
[    2.296160] sd 0:0:2:0: [sda] Write Protect is off
[    2.296230] sd 0:0:2:0: [sda] Cache data unavailable
[    2.296236] sd 0:0:2:0: [sda] Assuming drive cache: write through
[    2.297716]  sda: sda1 sda2 sda3
[    2.298969] sd 0:0:2:0: [sda] Attached SCSI disk
[    2.303265] ibmveth 30000003 env3: renamed from eth1
[    2.339388] sd 1:0:1:0: Attached scsi generic sg2 type 0
[    2.339917] sd 1:0:1:0: [sdb] 62914560 512-byte logical blocks: (32.2 GB/30.0 GiB)
[    2.339987] sd 1:0:1:0: [sdb] Write Protect is off
[    2.340054] sd 1:0:1:0: [sdb] Cache data unavailable
[    2.340060] sd 1:0:1:0: [sdb] Assuming drive cache: write through
[    2.341450]  sdb: sdb1 sdb2 sdb3
[    2.342595] sd 1:0:1:0: [sdb] Attached SCSI disk
[  OK  ] Started udev Wait for Complete Device Initialization.
         Starting Device-Mapper Multipath Device Controller...
[  OK  ] Started Device-Mapper Multipath Device Controller.
[  OK  ] Reached target Local File Systems (Pre).
[  OK  ] Reached target Local File Systems.
         Starting Create Volatile Files and Directories...
[  OK  ] Started Create Volatile Files and Directories.
[  OK  ] Reached target System Initialization.
[  OK  ] Reached target Basic System.
[    2.452935] device-mapper: multipath service-time: version 0.3.0 loaded
[  OK  ] Found device /dev/mapper/fedora-root.
[  OK  ] Reached target Initrd Root Device.
[  OK  ] Started dracut initqueue hook.
         Starting File System Check on /dev/mapper/fedora-root...
[  OK  ] Reached target Remote File Systems (Pre).
[  OK  ] Reached target Remote File Systems.
[  OK  ] Started File System Check on /dev/mapper/fedora-root.
         Mounting /sysroot...
[    3.524325] SGI XFS with ACLs, security attributes, scrub, no debug enabled
[    3.533152] random: crng init done
[    3.533176] random: 7 urandom warning(s) missed due to ratelimiting
[    3.533210] random: crng done (trusting CPU's manufacturer)
[    3.534153] XFS (dm-4): Mounting V5 Filesystem
[    3.573470] XFS (dm-4): Ending clean mount
[  OK  ] Mounted /sysroot.
[  OK  ] Reached target Initrd Root File System.
         Starting Reload Configuration from the Real Root...
[  OK  ] Started Reload Configuration from the Real Root.
[  OK  ] Reached target Initrd File Systems.
[  OK  ] Reached target Initrd Default Target.
         Starting dracut pre-pivot and cleanup hook...
[  OK  ] Started dracut pre-pivot and cleanup hook.
         Starting Cleaning Up and Shutting Down Daemons...
[  OK  ] Started Cleaning Up and Shutting Down Daemons.
         Starting Setup Virtual Console...
         Starting Plymouth switch root service...
[  OK  ] Stopped dracut pre-pivot and cleanup hook.
[  OK  ] Stopped target Initrd Default Target.
[  OK  ] Stopped target Initrd Root Device.
[  OK  ] Stopped target Basic System.
[  OK  ] Stopped target Paths.
[  OK  ] Stopped target System Initialization.
[  OK  ] Stopped Create Volatile Files and Directories.
         Stopping udev Kernel Device Manager...
[  OK  ] Stopped target Swap.
[  OK  ] Stopped target Remote File Systems.
[  OK  ] Stopped target Remote File Systems (Pre).
[  OK  ] Stopped dracut initqueue hook.
[  OK  ] Stopped target Sockets.
[  OK  ] Stopped target Local File Systems.
[  OK  ] Stopped target Local File Systems (Pre).
         Stopping Device-Mapper Multipath Device Controller...
[  OK  ] Stopped Apply Kernel Variables.
[  OK  ] Stopped target Timers.
[  OK  ] Stopped target Slices.
[  OK  ] Stopped Device-Mapper Multipath Device Controller.
[  OK  ] Started Plymouth switch root service.
[  OK  ] Stopped udev Wait for Complete Device Initialization.
[  OK  ] Stopped udev Coldplug all Devices.
[  OK  ] Started Setup Virtual Console.
[  OK  ] Stopped udev Kernel Device Manager.
[  OK  ] Stopped Create Static Device Nodes in /dev.
[  OK  ] Stopped Create list of required staâ¦vice nodes for the current kernel.
[  OK  ] Stopped dracut pre-udev hook.
[  OK  ] Stopped dracut cmdline hook.
[  OK  ] Stopped dracut ask for additional cmdline parameters.
[  OK  ] Closed udev Kernel Socket.
[  OK  ] Closed udev Control Socket.
         Starting Cleanup udevd DB...
[  OK  ] Started Cleanup udevd DB.
[  OK  ] Reached target Switch Root.
         Starting Switch Root...
[    4.330963] systemd-journald[252]: Received SIGTERM from PID 1 (systemd).
[    4.361458] systemd: 13 output lines suppressed due to ratelimiting
[...]

Comment 12 Christoph Buchmann 2019-02-08 10:18:07 UTC
multipath output on working system: 

[root@fed29 ~]# multipathd -k
multipathd> list paths
hcil    dev dev_t pri dm_st  chk_st dev_st  next_check
0:0:2:0 sda 8:0   1   active ready  running XXXXXXX... 15/20
1:0:1:0 sdb 8:16  1   active ready  running XXXXXXX... 15/20
multipathd>

Comment 13 Christoph Buchmann 2019-02-08 10:27:25 UTC
For info: 
the PowerVM setup can be done with one or more virtual IO servers (VIOS). If using multiple VIOS these servers provide the same LUN/disk via multiple paths, in my case two VIOS/paths.

Comment 14 Dan Horák 2019-02-08 10:31:51 UTC
Isn't anything useful in /var/log/libvirt/qemu/<vm name>?

Comment 15 Sinny Kumari 2019-02-08 11:13 UTC
Created attachment 1528049 [details]
qemu log from FAH 20190205

/var/log/libvirt/qemu/fah log fron one of the virt-insatll attempt which hangs

Comment 16 Jakub Čajka 2019-02-11 10:06:42 UTC
From what has been posted, this seems like two separate issues. One that Christoph describes on the LPAR(later in the boot) and other that me and Sinny are observing in VMs(kexecing kernel, possibly also qemu/kvm bug). IMO only link is that something has changed within recent kernel.

For the record nothing is in the /var/log/libvirt/qemu/ directory for me and when investigating the qemu process with gdb it seems that it is stuck busy polling/waiting for something.
#0  0x00007fffb5194364 in __GI_ppoll (fds=0x202, nfds=6, timeout=<optimized out>, sigmask=0x0) at ../sysdeps/unix/sysv/linux/ppoll.c:39
#1  0x00000001235d982c in ppoll (__ss=0x0, __timeout=0x0, __nfds=<optimized out>, __fds=<optimized out>) at /usr/include/bits/poll2.h:77
#2  qemu_poll_ns (fds=<optimized out>, nfds=<optimized out>, timeout=<optimized out>) at /usr/src/debug/qemu-2.12.0-3.fc29.ppc64le/util/qemu-timer.c:322
#3  0x00000001235daf94 in os_host_main_loop_wait (timeout=<optimized out>) at /usr/src/debug/qemu-2.12.0-3.fc29.ppc64le/util/main-loop.c:258
#4  main_loop_wait (nonblocking=<optimized out>) at /usr/src/debug/qemu-2.12.0-3.fc29.ppc64le/util/main-loop.c:522
#5  0x0000000123019a40 in main_loop () at /usr/src/debug/qemu-2.12.0-3.fc29.ppc64le/vl.c:1943
#6  main (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>) at /usr/src/debug/qemu-2.12.0-3.fc29.ppc64le/vl.c:4734

Comment 17 Menanteau Guy 2019-02-11 10:35:43 UTC
I have the same problem as Christoph on a baremetal P8 machine with multipath (8247-22L). I have a f29 I updated and got kernel-4.20.6-200.fc29.ppc64le. 

on the console:
[  OK  ] Started Create Volatile Files and Directories.
[  OK  ] Reached target System Initialization.
[  OK  ] Reached target Basic System.
[   72.504522] device-mapper: multipath service-time: version 0.3.0 loaded
[   72.504764] device-mapper: table: table load rejected: not all devices are blk-mq request-stackable
[   72.504785] device-mapper: table: unable to determine table type
[   72.507232] device-mapper: table: table load rejected: not all devices are blk-mq request-stackable
[   72.507277] device-mapper: table: unable to determine table type
[   72.511441] device-mapper: table: table load rejected: not all devices are blk-mq request-stackable
[   72.511507] device-mapper: table: unable to determine table type
[   72.515748] device-mapper: table: table load rejected: not all devices are blk-mq request-stackable
[   72.515801] device-mapper: table: unable to determine table type
[  149.761624] dracut-initqueue[1348]: Warning: dracut-initqueue timeout - starting timeout scripts
[  150.341936] dracut-initqueue[1348]: Warning: dracut-initqueue timeout - starting timeout scripts
[  150.921886] dracut-initqueue[1348]: Warning: dracut-initqueue timeout - starting timeout scripts
...
[  218.221867] dracut-initqueue[1348]: Warning: Could not boot.

dracut:/tmp# multipathd -k
multipathd> list paths
hcil    dev dev_t pri dm_st chk_st dev_st  next_check
0:2:0:0 sda 8:0   50  undef undef  unknown orphan    
0:2:1:0 sdb 8:16  50  undef undef  unknown orphan    
1:2:0:0 sdc 8:32  10  undef undef  unknown orphan    
1:2:1:0 sdd 8:48  10  undef undef  unknown orphan    


This machine f29 boot correctly with 4.19.6-300.fc29.ppc64le kernel.

Comment 18 Dan Horák 2019-02-11 11:12:51 UTC
Right, I'm changing the title of this bug to mention the multipath storage. And we will open another bug to track the nested virt issues.

Comment 19 Jakub Čajka 2019-02-12 12:08:03 UTC
For the record I have opened bug for the LPAR nested virt. BZ#1676475

Comment 20 Sinny Kumari 2019-02-12 12:15:26 UTC
(In reply to Jakub Čajka from comment #19)
> For the record I have opened bug for the LPAR nested virt. BZ#1676475

Thanks Jakub!

Comment 21 Michel Normand 2019-02-12 13:07:33 UTC
removing the need-info as as previous comment#7 tracked by new bug#1676475 I just commented in.

Comment 22 Matt K Light 2019-02-13 20:37:41 UTC
I am seeing a similar issue after upgrading from Fedora27 that was using 4.18.19-100.fc27.ppc64 to Fedora28 using 4.20.7-100.fc28.ppc64
I found this post
https://lore.kernel.org/lkml/20181105135157.GA11485@redhat.com/
and tried adding scsi_mod.use_blk_mq=Y to the kernel commandline.
Doing so enabled my ppc64 system to boot and find the multipath devices with the 4.20 kernel.

Comment 23 Christoph Buchmann 2019-02-13 21:55:03 UTC
Matt,

this actually also worked with my Fedora 29 installation. 

Thanks for the hint!

Comment 24 Dan Horák 2019-02-14 12:56:35 UTC
Is there any reason why this would be ppc64le specific? Sounds like a generic problem in the block layer to me.

Comment 25 Pierguido Lambri 2019-02-22 09:48:30 UTC
In fact this seems to be the same issue as in bz#1669235

Comment 26 Dan Horák 2019-02-22 09:55:20 UTC

*** This bug has been marked as a duplicate of bug 1669235 ***


Note You need to log in before you can comment on or make changes to this bug.