Created attachment 1091678 [details]
ls -l /dev

*** Bug 1270251 has been marked as a duplicate of this bug. ***

Created attachment 1122669 [details]
anaconda log file

Created attachment 1122671 [details]
storage log file

Created attachment 1122672 [details]
Operating system messages when IPLing a system with LUKS root partition

Created attachment 1122673 [details]
strace plymouthd

------- Comment From ROESCH3.com 2016-08-09 13:23 EDT-------
(In reply to comment #23)
> Description of problem:
> The content of /sys/class/tty/console/active refers to a nonexistent device
> when running on an s390x LPAR:
>
> [root@rtt9 ~]# cat /sys/class/tty/console/active
> ttyS0
> [root@rtt9 ~]# ls /dev/ttyS0
> ls: cannot access /dev/ttyS0: No such file or directory
> [root@rtt9 ~]# ls -l /sys/class/tty/
> total 0
> lrwxrwxrwx. 1 root root 0 Nov  9 03:08 console ->
> ../../devices/virtual/tty/console
> lrwxrwxrwx. 1 root root 0 Nov  9 03:08 ptmx -> ../../devices/virtual/tty/ptmx
> lrwxrwxrwx. 1 root root 0 Nov  9 03:08 sclp_line0 ->
> ../../devices/virtual/tty/sclp_line0
> lrwxrwxrwx. 1 root root 0 Nov  9 03:08 tty -> ../../devices/virtual/tty/tty
> lrwxrwxrwx. 1 root root 0 Nov  9 03:08 ttysclp0 ->
> ../../devices/virtual/tty/ttysclp0
> [root@rtt9 ~]#
>
> Version-Release number of selected component (if applicable):
> kernel-3.10.0-229.el7 (RHEL-7.1 GA)
> kernel-3.10.0-327.el7
>
> How reproducible:
> always (on LPAR)
>
> Steps to Reproduce:
> 1. Install RHEL-7 on an LPAR and run:
> cat /sys/class/tty/console/active
>
> Actual results:
> ttyS0 (this device doesn't exist)
>
> Expected results:
> an existing device
>
> Additional info:
> sclp_line0 is the "Operating System Messages" terminal, this is where the
> kernel messages are displayed
> ttysclp0 is the "Integrated ASCII Console"
>
> *** Bug 1270251 has been marked as a duplicate of this bug. ***

I have just tested with RHEL 7.3 A1 and the issue seems to still be there. Is the workaround to disable plymouth still necessary?

Here's the console output:

zIPL v1.23.0-27.el7 interactive boot menu

0. default (linux)

1. linux
2. linux-0-rescue-c3a892a4546e4c47ba0bfbdba643d287
Please choose (default will boot in 5 seconds):
Booting default (linux)
Uncompressing Linux...
Ok, booting the kernel.

?    0.568608! Initializing cgroup subsys cpuset
?    0.568612! Initializing cgroup subsys cpu
?    0.568614! Initializing cgroup subsys cpuacct
?    0.568618! Linux version 3.10.0-481.el7.s390x (mockbuild.bos.
redhat.com) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-4) (GCC) ) #1 SMP Wed Jul
27 18:25:22 EDT 2016
?    0.568624! setup: Linux is running natively in 64-bit mode
?    0.568663! setup: Reserving 161MB of memory at 3935MB for crashkernel (Syste
m RAM: 4096MB)
?    0.571362! cpu: The CPU configuration topology of the machine is: 0 0 0 8 3
8 / 3
?    0.582306! Zone ranges:
?    0.582309!   DMA      ?mem 0x00000000-0x7fffffff!
?    0.582311!   Normal   ?mem 0x80000000-0xffffffff!
?    0.582312! Movable zone start for each node
?    0.582313! Early memory node ranges
?    0.582314!   node   0: ?mem 0x00000000-0xf5efffff!
?    0.589898! PERCPU: Embedded 15 pages/cpu @0000000084300000 s24064 r8192 d291
84 u61440
?    0.590080! Built 1 zonelists in Zone order, mobility grouping on.  Total pag
es: 991620
?    0.590081! Kernel command line: root=/dev/mapper/luks-08f946fa-f766-418f-903
e-512456374fd3 crashkernel=auto rd.dasd=0.0.e874 rd.luks.uuid=luks-08f946fa-f766
-418f-903e-512456374fd3 cio_ignore=all,|condev,|0.0.e874,|0.0.f500-0.0.f502 rd.z
net=qeth,0.0.f500,0.0.f501,0.0.f502,layer2=1 LANG=en_US.UTF-8 BOOT_IMAGE=0
?    0.591078! PID hash table entries: 4096 (order: 3, 32768 bytes)
?    0.591538! Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes
)
?    0.592147! Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes)
?    0.599753! Memory: 3910560k/4194304k available (6770k kernel code, 0k reserv
ed, 3658k data, 488k init)
?    0.599755! Write protected kernel read-only data: 0x100000 - 0x985fff
?    0.600010! SLUB: HWalign=256, Order=0-3, MinObjects=0, CPUs=256, Nodes=1
?    0.600191! Hierarchical RCU implementation.
?    0.709931! console ?ttyS0! enabled
?    0.754825! allocated 16777216 bytes of page_cgroup
?    0.754826! please try 'cgroup_disable=memory' option if you don't want memor
y cgroups
?    0.754907! pid_max: default: 262144 minimum: 2048
?    0.754970! Security Framework initialized
?    0.754980! SELinux:  Initializing.
?    0.755012! Mount-cache hash table entries: 4096
?    0.755359! Initializing cgroup subsys memory
?    0.755382! Initializing cgroup subsys devices
?    0.755383! Initializing cgroup subsys freezer
?    0.755385! Initializing cgroup subsys net_cls
?    0.755386! Initializing cgroup subsys blkio
?    0.755388! Initializing cgroup subsys perf_event
?    0.755393! Initializing cgroup subsys pids
?    0.755394! Initializing cgroup subsys net_prio
?    0.755426! ftrace: allocating 17438 entries in 69 pages
?    0.762898! cpu: 8 configured CPUs, 0 standby CPUs
?    0.764869! Brought up 8 CPUs
?    0.765863! devtmpfs: initialized
?    0.766709! atomic64 test passed
?    0.766865! NET: Registered protocol family 16
?    0.810827! SCSI subsystem initialized
?    0.842727! NetLabel: Initializing
?    0.842729! NetLabel:  domain hash size = 128
?    0.842730! NetLabel:  protocols = UNLABELED CIPSOv4
?    0.842740! NetLabel:  unlabeled traffic allowed by default
?    1.760616! NET: Registered protocol family 2
?    1.761087! TCP established hash table entries: 32768 (order: 6, 262144 bytes
)
?    1.761180! TCP bind hash table entries: 32768 (order: 7, 524288 bytes)
?    1.761331! TCP: Hash tables configured (established 32768 bind 32768)
?    1.761352! TCP: reno registered
?    1.761371! UDP hash table entries: 2048 (order: 4, 65536 bytes)
?    1.761393! UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes)
?    1.761719! NET: Registered protocol family 1
?    1.761749! Unpacking initramfs...
?    2.017723! Freeing initrd memory: 19832K (12a3000 - 2601000)
?    2.017927! Runtime instrumentation facility initialized
?    2.020364! futex hash table entries: 4096 (order: 8, 1048576 bytes)
?    2.020501! Initialise system trusted keyring
?    2.020518! audit: initializing netlink socket (disabled)
?    2.020524! type=2000 audit(1470763322.305:1): initialized
?    2.048005! HugeTLB registered 1 MB page size, pre-allocated 0 pages
?    2.048831! VFS: Disk quotas dquot_6.5.2
?    2.048888! Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
?    2.049099! msgmni has been set to 7676
?    2.049157! Key type big_key registered
?    2.049848! NET: Registered protocol family 38
?    2.049854! Key type asymmetric registered
?    2.049857! Asymmetric key parser 'x509' registered
?    2.049889! Block layer SCSI generic (bsg) driver version 0.4 loaded (major 2
53)
?    2.049964! io scheduler noop registered
?    2.049967! io scheduler deadline registered (default)
?    2.049986! io scheduler cfq registered
?    2.050013! pci_hotplug: PCI Hot Plug PCI Core version: 0.5
?    2.050044! hvc_iucv: The z/VM IUCV HVC device driver cannot be used without
z/VM
?    2.050133! rdac: device handler registered
?    2.050164! hp_sw: device handler registered
?    2.050166! emc: device handler registered
?    2.050168! alua: device handler registered
?    2.050209! cio: Channel measurement facility initialized using format extend
ed (mode autodetected)
?    2.355454! drop_monitor: Initializing network drop monitor service
?    2.355612! TCP: cubic registered
?    2.355616! Initializing XFRM netlink socket
?    2.355708! NET: Registered protocol family 10
?    2.356119! NET: Registered protocol family 17
?    2.356206! Loading compiled-in X.509 certificates
?    2.356775! Loaded X.509 cert 'Red Hat Enterprise Linux Driver Update Program
(key 3): bf57f3e87362bc7229d9f465321773dfd1f77a80'
?    2.357330! Loaded X.509 cert 'Red Hat Enterprise Linux kpatch signing key: 4
d38fd864ebe18c5f0b72e3852e2014c3a676fc8'
?    2.357885! Loaded X.509 cert 'Red Hat Enterprise Linux kernel signing key: b
5a643ffd0912a8f9cf253c5d635e8179c2720a6'
?    2.357894! registered taskstats version 1
?    2.358755! Freeing unused kernel memory: 488K (a30000 - aaa000)
?    2.362445! systemd?1!: systemd 219 running in system mode. (+PAM +AUDIT +SEL
INUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +
XZ -LZ4 -SECCOMP +BLKID +ELFUTILS +KMOD +IDN)
?    2.362610! systemd?1!: Detected architecture s390x.
?    2.362612! systemd?1!: Running in initial RAM disk.
?    2.362645! systemd?1!: Set hostname to <localhost.localdomain>.
?    2.382393! systemd?1!: Reached target Timers.
?    2.382399! systemd?1!: Starting Timers.
?    2.382414! systemd?1!: Reached target Swap.
?    2.382418! systemd?1!: Starting Swap.
?    2.382514! systemd?1!: Created slice Root Slice.
?    2.382519! systemd?1!: Starting Root Slice.
?    2.382545! systemd?1!: Listening on udev Control Socket.
?    2.382550! systemd?1!: Starting udev Control Socket.
?    2.382572! systemd?1!: Listening on udev Kernel Socket.
?    2.382576! systemd?1!: Starting udev Kernel Socket.
?    2.382588! systemd?1!: Reached target Local File Systems.
?    2.382592! systemd?1!: Starting Local File Systems.
?    2.382639! systemd?1!: Created slice System Slice.
?    2.382644! systemd?1!: Starting System Slice.
?    2.382693! systemd?1!: Created slice system-systemd\x2dcryptsetup.slice.
?    2.382698! systemd?1!: Starting system-systemd\x2dcryptsetup.slice.
?    2.382730! systemd?1!: Listening on Journal Socket.
?    2.382735! systemd?1!: Starting Journal Socket.
?    2.382995! systemd?1!: Starting Create list of required static device nodes
for the current kernel...
?    2.383348! systemd?1!: Starting Apply Kernel Variables...
?    2.383396! systemd?1!: Reached target Sockets.
?    2.383401! systemd?1!: Starting Sockets.
?    2.383724! systemd?1!: Starting dracut cmdline hook...
?    2.384027! systemd?1!: Starting Journal Service...
?    2.384080! systemd?1!: Reached target Slices.
?    2.384089! systemd?1!: Starting Slices.
?    2.384405! systemd?1!: Started Create list of required static device nodes f
or the current kernel.
?    2.384638! systemd?1!: Started Apply Kernel Variables.
?    2.385142! systemd?1!: Starting Create Static Device Nodes in /dev...
?    2.386750! systemd?1!: Started Create Static Device Nodes in /dev.
?    2.388247! systemd?1!: Started Journal Service.

Welcome to ?0;34mRed Hat Enterprise Linux Server 7.3 Beta (Maipo) dracut-033-450
.el7 (Initramfs)?0m|

??32m  OK  ?0m! Reached target Timers.
??32m  OK  ?0m! Reached target Swap.
??32m  OK  ?0m! Created slice Root Slice.
??32m  OK  ?0m! Listening on udev Control Socket.
??32m  OK  ?0m! Listening on udev Kernel Socket.
??32m  OK  ?0m! Reached target Local File Systems.
??32m  OK  ?0m! Created slice System Slice.
??32m  OK  ?0m! Created slice system-systemd\x2dcryptsetup.slice.
??32m  OK  ?0m! Listening on Journal Socket.
Starting Create list of required st... nodes for the current kernel...
Starting Apply Kernel Variables...
??32m  OK  ?0m! Reached target Sockets.
Starting dracut cmdline hook...
Starting Journal Service...
??32m  OK  ?0m! Reached target Slices.
??32m  OK  ?0m! Started Create list of required sta...ce nodes for the current k
ernel.
??32m  OK  ?0m! Started Apply Kernel Variables.
Starting Create Static Device Nodes in /dev...
??32m  OK  ?0m! Started Create Static Device Nodes in /dev.
??32m  OK  ?0m! Started Journal Service.
??32m  OK  ?0m! Started dracut cmdline hook.
Starting dracut pre-udev hook...
??32m  OK  ?0m! Started dracut pre-udev hook.
Starting udev Kernel Device Manager...
??32m  OK  ?0m! Started udev Kernel Device Manager.
Starting udev Coldplug all Devices...
Mounting Configuration File System...
??32m  OK  ?0m! Mounted Configuration File System.
??32m  OK  ?0m! Started udev Coldplug all Devices.
Starting dracut initqueue hook...
Starting Show Plymouth Boot Screen...
?    3.691551! device-mapper: uevent: version 1.0.3
?    3.691594! device-mapper: ioctl: 4.34.0-ioctl (2015-10-28) initialised: dm-d
evel
?    3.720646! qeth: loading core functions
?    3.723879! qeth 0.0.f500: Priority Queueing not supported
?    3.725768! qeth: register layer 2 discipline
?    3.731309! qdio: 0.0.f502 OSA on SC 3f24 using AI:1 QEBSM:0 PRI:1 TDD:1 SIGA
: W AP
?    3.762739! netif_napi_add() called with weight 128 on device eth%d
?    3.763447! qeth 0.0.f500: MAC address 02:00:00:9c:69:35 successfully registe
red on device enccw0.0.f500
?    3.763450! qeth 0.0.f500: Device is a OSD Express card (level: 076a)
?    3.763450! with link type OSD_1000 (portname: )
?    3.764309! qeth 0.0.f500: QDIO data connection isolation is deactivated
?    4.273480! dasd-eckd 0.0.e874: A channel path to the device has become opera
tional
?    4.275772! dasd-eckd 0.0.e874: New DASD 3390/0C (CU 3990/01) with 30051 cyli
nders, 15 heads, 224 sectors
?    4.276316! dasd-eckd 0.0.e874: DASD with 4 KB/block, 21636720 KB total size,
48 KB/track, compatible disk layout
?    4.278064!  dasda:VOL1/  0XE874: dasda1 dasda2 dasda3

Yes, the workaround (appending "plymouth.enable=0" on the kernel command like) is still necessary to get the prompt for entering the passphrase.

------- Comment From ROESCH3.com 2016-08-10 11:09 EDT-------
(In reply to comment #28)
> Yes, the workaround (appending "plymouth.enable=0" on the kernel command
> like) is still necessary to get the prompt for entering the passphrase.

OK, with that parameter set, the passphrase prompt is displayed.

The problem with the nonexistent device seems to be still present however.

[root@s83lp36 ~]# cat /sys/class/tty/console/active

[root@s83lp36 ~]# ls /dev/ttyS0

oops ... the previous comment has to read:
"
OK, with that parameter set, the passphrase prompt is displayed. 

The problem with the nonexistent device seems to be still present however. 

[root@s83lp36 ~]# cat /sys/class/tty/console/active
ttyS0

[root@s83lp36 ~]# ls /dev/ttyS0
ls: cannot access /dev/ttyS0: No such file or directory
"
...

------- Comment From GHILLER.com 2016-08-31 11:11 EDT-------
Bug still exist in RHEL 7.3 B1.
Test on z13 LPAR.
IPLing the system with LUKS root partition.
Workaround (appending "plymouth.enable=0" to zipl.conf) provides passphrase prompt.

------- Comment From PSCHMIDT.com 2016-10-20 07:39 EDT-------
Bug still exist in RHEL 7.3 RC2
Test on z13 LPAR.

Hello Red Hat,
... for this bugzilla we would like to propsoe a kbase article
with the following text:
-------------------------
1. Choose another Linux system that has access to the DASD or SCSI devices where the RHEL7.3 system with the LUKS partition was installed. 
Login there as root.

2. Set the required DASD/SCSI devices online by 'chccwdev -e <target_dev_bus_id>'. Do not forget to remove them from cio_ignore as well.

3. Mount the file systems (where <root> - RHEL7.3 root (/) file system partition, <boot> - RHEL7.3 boot system partition):
	* mount /dev/<root> /mnt
	* mount /dev/ /mnt/dev --bind
   Chroot the /mnt:
    * chroot /mnt
	* mount -t proc none /proc
	* mount -t sysfs none /sys
   Mount Boot partition to have the opportunity to change the Kernel Boot Parameters:
    * mount /dev/<boot> /boot

4. Change the Kernel Boot Parameters using /etc/zipl.conf file:
	* vi /etc/zipl.conf

5. Add the following parameter into /etc/zipl.conf kernel parameters line:
	"plymouth.enable=0"

6. Save the file and exit.

7. To accept the changes in the Kernel Boot Parameters run:
	* zipl -V

8. Umount all the directories mentioned above in the following order:
	* umount /boot
	* umount /sys
	* umount /proc
	* exit (leaving chroot)
	* umount /mnt/dev
	* umount /mnt

9. Set DASD/SCSI devices offline by 'chccwdev -s <target_dev_bus_id>'

10. Boot the RHEL7.3 system with the LUKS partition again using the HMC by selecting Recovery - Load and entering the required DASD/SCSI device (where your RHEL7.3 root partition is located).

Please note that additionally, if you have encrypted the root partition, it is required to decrypt it before you can mount it. 

So maybe you could add something like this under the above listed step 3.:

3.1 If you have encrypted the root partition, you have to decrypt it first to make it accessible. To find out the device name of the encrypted partitions, enter: 

blkid | grep crypto

Sample output:

/dev/dasda2: UUID="UUID IS DISPLAYED HERE" TYPE="crypto_LUKS"

After you have found the correct partition, decrypt it with cryptsetup. You will be prompted for your chosen passphrase

cryptsetup luksOpen /dev/dasda2/ cryptroot

The device will then be mapped to the path /dev/mapper/cryptroot and can be mounted from there using
mount /dev/mapper/cryptroot /mnt
-------------------------
.
... please review this proposal as soon as possible and 
please keep us informed in case of any questions.
Thanks in advance for your support.

Hello,
The requested kbase has been published:
https://access.redhat.com/solutions/2756891
Please let me know if you would like any changes made to it.

Thank You
Joe Kachuck

------- Comment From ROESCH3.com 2016-11-08 10:18 EDT-------
(In reply to comment #40)
> Hello,
> The requested kbase has been published:
> https://access.redhat.com/solutions/2756891
> Please let me know if you would like any changes made to it.
>
> Thank You
> Joe Kachuck

Correct me if I'm wrong, but the initial problem that led to this bug was "Console doesn't prompt the LUKS passphrase after installation (anaconda/plymouth)".
In my opinion, this kbase article is missing the relation to this symptom (i.e. the missing passphrase prompt when using LUKS), so that if people having a problem with LUKS, are able to find this.

I would suggest that the title of this kbase should be something like "Console does not prompt for passphrase after installation when using LUKS due to nonexistent device...."

And the "issue"-section should also describe in a few words what is happening.

The rest of the article seems good to me

(In reply to IBM Bug Proxy from comment #16)
> ------- Comment From ROESCH3.com 2016-11-08 10:18 EDT-------
> (In reply to comment #40)
> > Hello,
> > The requested kbase has been published:
> > https://access.redhat.com/solutions/2756891
> > Please let me know if you would like any changes made to it.
> >
> > Thank You
> > Joe Kachuck
> 
> Correct me if I'm wrong, but the initial problem that led to this bug was
> "Console doesn't prompt the LUKS passphrase after installation
> (anaconda/plymouth)".
> In my opinion, this kbase article is missing the relation to this symptom
> (i.e. the missing passphrase prompt when using LUKS), so that if people
> having a problem with LUKS, are able to find this.
> 
> I would suggest that the title of this kbase should be something like
> "Console does not prompt for passphrase after installation when using LUKS
> due to nonexistent device...."
> 
> And the "issue"-section should also describe in a few words what is
> happening.
> 
> The rest of the article seems good to me
.
for the records ... the 'initial problem' was the issue in
Red Hat bug 1270251 - Console doesn't prompt the LUKS passphrase after installation ...

Hello,
I have updated the change to the title.

Thank You
Joe Kachuck

------- Comment From ROESCH3.com 2016-11-14 06:35 EDT-------
(In reply to comment #44)
> Hello,
> I have updated the change to the title.
> Thank You
> Joe Kachuck

Hi,

I would say that it's good now.

------- Comment From ROESCH3.com 2016-11-14 07:35 EDT-------

------- Comment From GHILLER.com 2018-01-08 07:08 EDT-------
A typo in my comment: There is still now passphrase prompt after  --> There is still no passphrase prompt after ...

------- Comment From chavez.com 2018-01-08 09:07 EDT-------
The complete update should have read as:

"I have tested with RHEL7.5-A1. There is still no passphrase prompt after installation and reboot on LPAR with LUKS encrypted root partition."

Hello,
This will not be able to make RHEL 7.5. This is now requested for RHEL 7.6.

Thank You
Joe Kachuck

------- Comment From Sebastian.Kentzler 2018-10-17 10:37 EDT-------
Still not fixed in RHEL7.6 RC.

Hello Hendrik,
Please let me know if this BZ should be closed?

Thank You
Joe Kachuck

------- Comment From tstaudt.com 2019-08-01 03:14 EDT-------
This also happens with RHEL 8.1 as of Pre-Beta (so I assume also with 8.0).

At least the kbase article https://access.redhat.com/solutions/2756891
should be extended to cover RHEL 8.

Hello IBM,
I have updated the kbase:
https://access.redhat.com/solutions/2756891

Please let me know if you would like any additional changes to the kbase.

Thank You
Joe Kachuck

------- Comment From Michael.Roesch2 2019-08-08 05:31 EDT-------
I'm not quite sure this procedure involving the editing of the zipl.conf will work as described with RHEL 8.x. To my knowledge, you have to change boot parameters for RHEL 8.x now in the /boot/entries config files. Which would mean Red Hat would have to adjust this section for RHEL 8.x

Hello Michael,
Would you be able to provide me the steps required to resolve this issue is RHEL 8?

Thank You
Joe Kachuck

------- Comment From Michael.Roesch2 2019-08-27 08:10 EDT-------
(In reply to comment #63)
> Hello Michael,
> Would you be able to provide me the steps required to resolve this issue is
> RHEL 8?
> Thank You
> Joe Kachuck

To my knowledge, the deviation for RHEL 8 would be that instead of changing the zipl.conf you have to change the file
/boot/loader/entries/(machine-id)-(runningkernel).conf
And afterwards zipl -V etc.

reassigning to Philipp as Hendrik has moved to a different role within IBM.

I'm closing this it has been open over 4 years and no customer has reported it.