Bug 1478448 - [RFE] Build RHEL .iso images with El-Torito boot information on s390x
[RFE] Build RHEL .iso images with El-Torito boot information on s390x
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: lorax (Show other bugs)
7.5
s390x Linux
medium Severity medium
: rc
: 7.5
Assigned To: Brian Lane
Release Test Team
Yehuda Zimmerman
: FutureFeature
Depends On: 1523487
Blocks: 1399177 1477926 1576733 1594179
  Show dependency treegraph
 
Reported: 2017-08-04 10:52 EDT by Thomas Huth
Modified: 2018-06-22 08:05 EDT (History)
11 users (show)

See Also:
Fixed In Version: lorax-19.6.97-1
Doc Type: Enhancement
Doc Text:
The Red Hat Enterprise Linux 7 ISO image can be used to create guests virtual machines on IBM z Systems With this release, you can create a bootable Red Hat Enterprise Linux ISO file for _KVM_ virtual machines on the IBM z Systems architecture. As a result, Red Hat Enterprise Linux guest virtual machines on IBM z Systems can boot from a `boot.iso` file.
Story Points: ---
Clone Of:
: 1523487 (view as bug list)
Environment:
Last Closed: 2018-04-10 13:39:42 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
mk-s390-cdboot utility (3.58 KB, text/plain)
2017-08-09 13:56 EDT, Brian Lane
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2018:0947 None None None 2018-04-10 13:40 EDT

  None (edit)
Description Thomas Huth 2017-08-04 10:52:58 EDT
Description of problem:
Currently it is not possible to boot a KVM guest on s390x from a RHEL CD-ROM ISO image. The firmware that bootstraps a KVM guest expects the so-called El-Torito boot information in an ISO image, but currently the RHEL ISO images are built without this El-Torito boot information on s390x. So please add support for El-Torito in the s390x RHEL ISO images.

Steps to Reproduce:
1. Get a s390x machine and provision it with the latest RHEL-ALT version
2. Install the qemu-kvm package
3. Create a hard disk image for the guest:
  qemu-img create -f qcow2 guest.qcow2 10G
4. Get the latest RHEL ISO image
5. Run qemu-kvm to start the guest with the ISO image, e.g.:
  /usr/libexec/qemu-kvm -nographic -m 768 -device virtio-scsi -drive \
    file=RHEL-7.4-20170711.0-Server-s390x-dvd1.iso,format=raw,if=none,id=c1 \
    -device scsi-cd,drive=c1 -hda guest.qcow2

Actual results:
Guest kernel is not started, and the boot firmware of the guest prints "Cannot IPL this ISO image".

Expected results:
Kernel is loaded from the ISO image, and the guest installation can be started.
Comment 2 Thomas Huth 2017-08-04 11:06:27 EDT
FWIW, I was able to manually create a bootable ISO image from an existing RHEL ISO image with these steps:

1. Get a tool that can glue an initrd to kernel for s390x, for example:
  wget https://raw.githubusercontent.com/rhinstaller/anaconda/rhel6-branch/utils/mk-s390-cdboot.c
  gcc mk-s390-cdboot.c -o mk-s390-cdboot

2. Loop-mount the existing RHEL ISO image:
  mount -o loop RHEL-7.4-20170711.0-Server-s390x-dvd1.iso /mnt

3. Create a (writable) folder with the new CD contents:
  mkdir cd
  cp -r /mnt/* cd/

4. Create a combined kernel+initrd file in that folder:
  echo ro > parm.txt
  ./mk-s390-cdboot -i /mnt/images/kernel.img -r /mnt/images/initrd.img \
                   -p parm.txt -o cd/images/bigkernel.img

5. Create the new ISO image with the El-Torito settings:
  genisoimage -r -l -no-emul-boot -eltorito-boot images/bigkernel.img \
              -o rhel-test.iso cd/
Comment 3 Brian Lane 2017-08-09 11:44:00 EDT
Thanks for the details! Did you run mk-s390-cdboot on a s390 or on x86 and copy it over?
Comment 4 Thomas Huth 2017-08-09 11:48:28 EDT
I ran it on a s390x machine. I haven't tried whether that tool also correctly runs on a x86 box.
Comment 5 Brian Lane 2017-08-09 12:07:07 EDT
(In reply to Thomas Huth from comment #4)
> I ran it on a s390x machine. I haven't tried whether that tool also
> correctly runs on a x86 box.

It probably doesn't, there's nothing there to handle the byte order differences. I'll rewrite it in python and make sure it can run everywhere.
Comment 6 Brian Lane 2017-08-09 13:56 EDT
Created attachment 1311325 [details]
mk-s390-cdboot utility

When you have a chance could you give this a try and make sure it works.
Comment 7 Brian Lane 2017-08-09 16:24:20 EDT
Here's a patch to lorax that *may* work, it hasn't been tested on s390 yet.

https://github.com/rhinstaller/lorax/pull/236
Comment 8 Thomas Huth 2017-08-10 01:32:15 EDT
(In reply to Brian Lane from comment #6)
> When you have a chance could you give this a try and make sure it works.

I'm sorry, but it does not work yet. I've debugged it, and it seems like it overwrites some required code at offset 0x11000 of the kernel image. Looks like the maximum length of the kernel parameters is less than 4096 here. It works fine if I use this value instead:

KERNEL_CMDLINE_SIZE = 0xb80

I think that should be the correct value here. Looking at the end of https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/s390/kernel/head.S you can see that the kernel command line starts at offset 0x10480 and ends at 0x11000.
Comment 9 Brian Lane 2017-08-10 11:26:00 EDT
(In reply to Thomas Huth from comment #8)
> (In reply to Brian Lane from comment #6)
> > When you have a chance could you give this a try and make sure it works.
> 
> I'm sorry, but it does not work yet. I've debugged it, and it seems like it
> overwrites some required code at offset 0x11000 of the kernel image. Looks
> like the maximum length of the kernel parameters is less than 4096 here. It
> works fine if I use this value instead:
> 
> KERNEL_CMDLINE_SIZE = 0xb80
> 
> I think that should be the correct value here. Looking at the end of
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/
> s390/kernel/head.S you can see that the kernel command line starts at offset
> 0x10480 and ends at 0x11000.

Thanks! It's interesting that doesn't match either of the values at https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/s390/include/uapi/asm/setup.h

I'm going to go with 896 just to be safe, really it just needs to be long enough to overwrite the default included in the kernel.
Comment 12 Thomas Huth 2017-08-18 06:17:41 EDT
Thanks for the new build. Using that, I was finally able to run lorax to a point where I got a boot.iso image. However, it still did not boot in my s390x guest yet. After doing quite a bit of debugging, I finally discovered that the beginning of the joint kernel image got destroyed and so some magic bytes are missing there and the firmware refuses to load the image.
The problem is the "-boot-info-table" parameter that you use for mkisofs: It rewrites some bytes at the beginning of the kernel image - but this is not needed for s390x. Could you please simply remove that parameter from the template?
Comment 13 Brian Lane 2017-08-18 11:53:30 EDT
(In reply to Thomas Huth from comment #12)

> The problem is the "-boot-info-table" parameter that you use for mkisofs: It
> rewrites some bytes at the beginning of the kernel image - but this is not
> needed for s390x. Could you please simply remove that parameter from the
> template?

Thanks for debugging this! Have you tried removing it yourself and confirmed that the boot.iso really boots?
Comment 14 Thomas Huth 2017-08-18 12:44:48 EDT
Yes, after removing "-boot-info-table" from /usr/share/lorax/s390.tmpl and running lorax again, the boot.iso file is finally bootable. So this seems to be the last missing piece to get it working!
Comment 15 Thomas Huth 2017-08-21 11:57:10 EDT
Thanks, I've checked with lorax-19.6.97-1 that the resulting boot.iso now boots fine in my KVM guest on s390x!
Comment 21 Jan Stodola 2017-12-06 09:19:16 EST
Moving back to ASSIGNED based on previous comment, dvd iso image is not set as bootable.
Comment 22 Brian Lane 2017-12-06 11:48:38 EST
The s390 boot.iso doesn't have the El Torito boot table, so running dumpet on it isn't expected to work. As indicated by comment 12 adding the table scrambles the s390 image and causes it not to boot. According to comment 15 they actually do work.
Comment 24 Thomas Huth 2017-12-06 13:09:07 EST
(In reply to Jan Stodola from comment #23)
> Thomas,
> could you please give it a try and confirm that you can boot both dvd.iso
> and boot.iso images from RHEL-7.5 Alpha compose?

Now this is weird. It works with the boot.iso image:

$ /usr/libexec/qemu-kvm -nographic -m 768 -device virtio-scsi -drive     file=RHEL-7.5-20171130.0-Server-s390x-boot.iso,format=raw,if=none,id=c1 -device scsi-cd,drive=c1 -hda /tmp/guest.qcow2 
LOADPARM=[........]
Using virtio-scsi.
SCSI CD-ROM detected.
ISO boot image size verified
[    0.084036] Initializing cgroup subsys cpuset
[    0.084039] Initializing cgroup subsys cpu
[    0.084041] Initializing cgroup subsys cpuacct
[    0.084045] Linux version 3.10.0-799.el7.s390x (mockbuild@s390-006.build.eng.bos.redhat.com) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-25) (GCC) ) #1 SMP Mon Nov 27 07:06:08 EST 2017
[    0.084051] setup: Linux is running under KVM in 64-bit mode
... etc. ...

But it seems like it does not work with the dvd.iso yet:

$ /usr/libexec/qemu-kvm -nographic -m 768 -device virtio-scsi -drive file=RHEL-7.5-20171130.0-Server-s390x-dvd1.iso,format=raw,if=none,id=c1 -device scsi-cd,drive=c1 -hda /tmp/guest.qcow2 
LOADPARM=[........]
Using virtio-scsi.
SCSI CD-ROM detected.
! Cannot IPL this ISO image !

Brian, any idea what might be wrong here?
Comment 25 Brian Lane 2017-12-06 14:36:23 EST
(In reply to Thomas Huth from comment #24)
> (In reply to Jan Stodola from comment #23)
> > Thomas,
> > could you please give it a try and confirm that you can boot both dvd.iso
> > and boot.iso images from RHEL-7.5 Alpha compose?
> 
> Now this is weird. It works with the boot.iso image:
> 
> $ /usr/libexec/qemu-kvm -nographic -m 768 -device virtio-scsi -drive    
> file=RHEL-7.5-20171130.0-Server-s390x-boot.iso,format=raw,if=none,id=c1
> -device scsi-cd,drive=c1 -hda /tmp/guest.qcow2 
> LOADPARM=[........]
> Using virtio-scsi.
> SCSI CD-ROM detected.
> ISO boot image size verified
> [    0.084036] Initializing cgroup subsys cpuset
> [    0.084039] Initializing cgroup subsys cpu
> [    0.084041] Initializing cgroup subsys cpuacct
> [    0.084045] Linux version 3.10.0-799.el7.s390x
> (mockbuild@s390-006.build.eng.bos.redhat.com) (gcc version 4.8.5 20150623
> (Red Hat 4.8.5-25) (GCC) ) #1 SMP Mon Nov 27 07:06:08 EST 2017
> [    0.084051] setup: Linux is running under KVM in 64-bit mode
> ... etc. ...
> 
> But it seems like it does not work with the dvd.iso yet:
> 
> $ /usr/libexec/qemu-kvm -nographic -m 768 -device virtio-scsi -drive
> file=RHEL-7.5-20171130.0-Server-s390x-dvd1.iso,format=raw,if=none,id=c1
> -device scsi-cd,drive=c1 -hda /tmp/guest.qcow2 
> LOADPARM=[........]
> Using virtio-scsi.
> SCSI CD-ROM detected.
> ! Cannot IPL this ISO image !
> 
> Brian, any idea what might be wrong here?

I think the dvd.iso is created using a different final process. IIRC they use lorax to make the tree, then add packages and run their own mkisofs on it, so that may need to be changed. I'm not sure who is in charge of that these days (used to be dmach) but Jan should know.
Comment 35 errata-xmlrpc 2018-04-10 13:39:42 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:0947

Note You need to log in before you can comment on or make changes to this bug.