Bug 1841089

Summary: RHEL 8.2 System takes long time appro 45 min to boot on 4 port adaptor with NPAR EP mode.
Product: Red Hat Enterprise Linux 8 Reporter: Manish Rangankar <manish.rangankar>
Component: dracutAssignee: Lukáš Nykrýn <lnykryn>
Status: CLOSED CANTFIX QA Contact: qe-baseos-daemons
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 8.2CC: dracut-maint-list, dtardon, njavali
Target Milestone: rcKeywords: Reopened
Target Release: 8.0   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-08-12 08:22:06 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Detailed logs attached
none
rd.debug log none

Description Manish Rangankar 2020-05-28 10:48:49 UTC
Created attachment 1693008 [details]
Detailed logs attached

Description of problem:
RHEL 8.2 System takes 45 min to boot on 4 port adaptor with NPAR EP mode.

Four ports are configured with iSCSI boot from SAN with same target.
All the port logged in to the target successfully.
Single disk is partition to have different /home, /root, swap partition
sdb               8:16   0   60G  0 disk
└─mpatha        253:0    0   60G  0 mpath
  ├─mpatha1     253:1    0  600M  0 part  /boot/efi
  ├─mpatha2     253:2    0    1G  0 part  /boot
  └─mpatha3     253:3    0 58.4G  0 part
    ├─rhel-home 253:4    0 17.2G  0 lvm   /home
    ├─rhel-root 253:5    0 35.2G  0 lvm   /
    └─rhel-swap 253:6    0    6G  0 lvm   [SWAP]

It has been observed that swap partition timesout and its dependency also fails. After which dracut take long time to boot or go to next step.

May 28 01:59:48 R7415-RH82-iSCSI systemd[1]: dev-mapper-rhel\x2dswap.device: Job dev-mapper-rhel\x2dswap.device/start timed out.
May 28 01:59:48 R7415-RH82-iSCSI systemd[1]: Timed out waiting for device dev-mapper-rhel\x2dswap.device.
May 28 01:59:48 R7415-RH82-iSCSI systemd[1]: Dependency failed for Resume from hibernation using device /dev/mapper/rhel-swap.
May 28 01:59:48 R7415-RH82-iSCSI systemd[1]: systemd-hibernate-resume@dev-mapper-rhel\x2dswap.service: Job systemd-hibernate-resume@dev-mapper-rhel\x2dswap.service/start failed with result 'dependency'.
May 28 01:59:48 R7415-RH82-iSCSI systemd[1]: dev-mapper-rhel\x2dswap.device: Job dev-mapper-rhel\x2dswap.device/start failed with result 'timeout'.
May 28 01:59:48 R7415-RH82-iSCSI systemd[1]: Reached target Local File Systems (Pre).
May 28 01:59:48 R7415-RH82-iSCSI systemd[1]: Reached target Local File Systems.
May 28 01:59:48 R7415-RH82-iSCSI systemd[1]: Starting Create Volatile Files and Directories...
May 28 01:59:48 R7415-RH82-iSCSI systemd[1]: Started Create Volatile Files and Directories.
May 28 01:59:48 R7415-RH82-iSCSI systemd[1]: Reached target System Initialization.
May 28 01:59:48 R7415-RH82-iSCSI systemd[1]: Reached target Basic System.


Version-Release number of selected component (if applicable):
Linux R7415-RH82-iSCSI 4.18.0-193.el8.x86_64 #1 SMP Fri Mar 27 14:35:58 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

How reproducible:
Configured four port for iSCSI boot from SAN configuration in NPAR EP mode i.e. fours port with 16 function out of which 4 are iSCSI functions.

Steps to Reproduce:
1. Configured iscsi four port for iSCSI BFS
2. INstall RHEL 82 OS
3. First boot

Actual results:
Takes long time to boot

Expected results:
Less time expected.

Additional info:
RHEL82_Logs
├── dmesg_rhel82.log
├── iscsiadm_fw_rhel82.log
├── iscsi_fail.txt
├── lsblk_rhel82.log
├── meminfo_rhel82.log
├── messages_rhel82.log
├── multipath_rhel82.log
├── multipath_v3_rhel82.log
├── pci_devices_rhel82.log
├── Ports_rhel82.log
├── qedi_interrupt_rhel82.log
├── session_rhel82.log
└── swaps_rhel82.log

0 directories, 13 files

Comment 1 David Tardon 2020-08-03 14:11:36 UTC
*** Bug 1841092 has been marked as a duplicate of this bug. ***

Comment 2 David Tardon 2020-08-03 14:12:37 UTC
Created attachment 1703298 [details]
rd.debug log

Comment 3 David Tardon 2020-08-07 15:40:50 UTC
I see many I/O errors in the log, like:

  [  842.724026] sd 10:0:0:0: rejecting I/O to offline device
  [  842.724111] sd 10:0:0:0: rejecting I/O to offline device
  [  842.724713] blk_update_request: I/O error, dev sdb, sector 53373096 op 0x0:(READ) flags 0x  84700 phys_seg 28 prio class 0
  [  842.725420] blk_update_request: I/O error, dev sdb, sector 52592648 op 0x0:(READ) flags 0x  84700 phys_seg 2 prio class 0
  [  842.725439] device-mapper: multipath: Failing path 8:16.

=> I think this is caused by HW malfunction.

Comment 4 Nilesh Javali (Marvell) 2020-08-12 05:29:50 UTC
Please suggest why this is HW malfunction.
The test scenario here is Boot from SAN and the swap partition which timesout is on the same remote LUN as the root partition.

Comment 5 David Tardon 2020-08-12 08:22:06 UTC
(In reply to Nilesh Javali (Marvell) from comment #4)
> Please suggest why this is HW malfunction.

I did. Please investigate the I/O errors first.