Bug 2002215 - Multipath day1 not working on s390x
Summary: Multipath day1 not working on s390x
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: RHCOS
Version: 4.9
Hardware: s390x
OS: Linux
high
medium
Target Milestone: ---
: 4.10.0
Assignee: jschinta
QA Contact: HuijingHei
URL:
Whiteboard:
Depends On: 2004596
Blocks: ocp-49-z-tracker 2009652
TreeView+ depends on / blocked
 
Reported: 2021-09-08 09:39 UTC by jschinta
Modified: 2022-05-17 07:09 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
Clone Of:
: 2009652 (view as bug list)
Environment:
Last Closed: 2022-03-10 16:08:32 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github coreos coreos-installer pull 613 0 None None None 2021-09-08 09:39:10 UTC
Red Hat Issue Tracker MULTIARCH-1668 0 None None None 2021-09-08 09:39:49 UTC
Red Hat Product Errata RHSA-2022:0056 0 None None None 2022-03-10 16:08:52 UTC

Description jschinta 2021-09-08 09:39:11 UTC
Description of problem:
When enabling multipath during firstboot with ignition-kargs, it fails and boots into the emergency shell instead.

Version-Release number of selected component (if applicable):
4.9

How reproducible:
Append multipath kargs through ignition during firstboot

Steps to Reproduce:
1. Create ignition with multipath kargs
2. Install rhcos with ignition-file
3.

Actual results:
Boot into emergency shell

Expected results:
Boot rhcos with multipath from day1

Additional info:
The Problem is a missing zipl helper script in the initramfs, preventing zipl to recognize the device-mapper and running correctly. The fix is already upstream.

Comment 1 jschinta 2021-09-09 13:19:16 UTC
When the fix lands in rhcos 4.9 we also need to enable the test again in kola https://github.com/openshift/os/pull/617

Comment 2 Prashanth Sundararaman 2021-09-13 16:23:54 UTC
the fix to coreos-installer has landed: https://github.com/coreos/coreos-installer/pull/613. there needs to be a new tag created in the coreos-installer and we need rpms built off of that in rhcos/fcos for this test to work.

Comment 3 Prashanth Sundararaman 2021-09-13 16:26:06 UTC
Jan,

Could you work with the coreos team to maybe bump coreos-installer by creating a new tag or by backporting the patch in the current release? 

Thanks
Prashanth

Comment 4 Dan Li 2021-09-13 16:34:49 UTC
Re-assigning to Jan per Comment 3, let me know if the assignment is incorrect.

Comment 5 Dan Li 2021-09-20 18:13:07 UTC
Hi @jschintag would it be okay to assign this bug under your name as the assignee per work stated in Comment 3? If so, do you think you will continue to work on this bug in the next sprint (starting September 27th)? If so, we might want to add "reviewed-in-sprint". Thanks!

Comment 6 jschinta 2021-09-21 08:49:38 UTC
Hi Dan, 
Sure, i was planning on doing the backports in the next Sprint.

Comment 7 jschinta 2021-09-27 08:47:28 UTC
I created the backports of the fix for coreos-installer to rhcos 4.10 and 4.9

4.10: https://src.osci.redhat.com/rpms/coreos-installer/pull-request/30
4.9: https://src.osci.redhat.com/rpms/coreos-installer/pull-request/31

Comment 8 jschinta 2021-09-30 12:48:15 UTC
The RPM is build and included in nightlys. Tested with rhcos-49.84.202109300002-0-qemu.s390x.qcow2
PRs to re-enable the test in rhcos:
https://github.com/openshift/os/pull/639
https://github.com/openshift/os/pull/640

Comment 10 Micah Abbott 2021-09-30 18:08:57 UTC
In order to follow OCP Backport policy, we must target this fix for 4.10 first, then create the necessary backport BZs for older versions.

RHCOS 410.84.202109292202-0 on s390x includes coreos-installer-0.10.0-2.rhaos4.10.el8 with the fix, so it should be simple enough to move this along.

Comment 11 RHCOS Bug Bot 2021-10-01 07:30:24 UTC
This bug has been reported fixed in a new RHCOS build.  Do not move this bug to MODIFIED until the fix has landed in a new bootimage.

Comment 12 Michael Nguyen 2021-10-12 18:17:12 UTC
Ran multipath.day1 test and it passed. Test details are here: https://github.com/coreos/coreos-assembler/blob/main/mantle/kola/tests/misc/multipath.go

[coreos-assembler]$ kola run --qemu-image rhcos-410.84.202110080602-0-qemu.s390x.qcow2 multipath.day1
⚠️  Skipping kola test pattern "fips.enable*":
  👉 https://bugzilla.redhat.com/show_bug.cgi?id=1782026
⚠️  Skipping kola test pattern "ext.config.var-mount":
  👉 https://github.com/ibm-s390-tools/s390-tools/pull/82
⚠️  Skipping kola test pattern "coreos.ignition.journald-log":
  👉 https://github.com/coreos/coreos-assembler/issues/1173
=== RUN   multipath.day1
--- PASS: multipath.day1 (330.88s)
PASS, output in tmp/kola/qemu-unpriv-2021-10-12-1804-37

Comment 13 RHCOS Bug Bot 2021-10-20 17:53:45 UTC
The fix for this bug has landed in a bootimage bump, as tracked in bug 2004596 (now in status MODIFIED).  Moving this bug to MODIFIED.

Comment 15 HuijingHei 2021-10-26 14:01:01 UTC
Verify passed with RHCOS 410.84.202110141003-0

$ sudo kola run --qemu-image rhcos-410.84.202110141003-0-qemu.s390x.qcow2 multipath.day1
=== RUN   multipath.day1
--- PASS: multipath.day1 (420.41s)
PASS, output in _kola_temp/qemu-unpriv-2021-10-26-1347-32

Comment 17 jschinta 2022-02-07 07:56:46 UTC
I can't set it myself, but Doc type for this Bug should be "No Doc Update"

Comment 18 HuijingHei 2022-02-07 08:08:04 UTC
(In reply to jschinta from comment #17)
> I can't set it myself, but Doc type for this Bug should be "No Doc Update"

Set Doc type to "No Doc Update"

Comment 21 errata-xmlrpc 2022-03-10 16:08:32 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.10.3 security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:0056


Note You need to log in before you can comment on or make changes to this bug.