Bug 1322235

One problem with making changes to the zpxe.rexx script is that it doesn't have any authoritative home anymore.

The only public versions I could find were in Cobbler's git, where it lived (and was maintained, to some extent) from 2009 until 2014 when Cobbler dropped S/390 support.

There also seems to be a copy floating around the s390-tools package from OpenSUSE, apparently with some modifications, but I didn't succeed in finding any proper revision history from OBS.

I have set up a Github repository for it, with the commit history extracted from Cobbler. Ultimately I am hoping that there is some better home for it, but in the meantime this is better than nothing...

https://github.com/beaker-project/zpxe

So my idea for this bug is to make zpxe.rexx understand ftp:// URLs in the profile:

https://github.com/beaker-project/zpxe/pull/1

and then when Beaker is generating the profile it will put ftp:// URLs pointing directly at the source images on the lab mirror:

http://gerrit.beaker-project.org/5303

alemay, jstancek, could you please take a look at the zpxe changes above and tell me if you think they are okay?

The downside to my solution for this bug is that we will have to update zpxe.rexx on *all* the guests (so that they can understand the ftp:// URLs) before we release the corresponding Beaker change which starts producing the ftp:// URLs. So we would need to be confident in the zpxe.rexx changes.

I have put my version of zpxe.rexx onto the 4 z/VM guests we have on beaker-devel, to give it some soak time (using plain TFTP paths, not ftp:// URLs though).

Looks good to me. I also like that we can go back to TFTP with "ZPXE_USE_FTP" beaker config option.

Why do you suppress FTP output? Is it producing too much output?

(In reply to Jan Stancek from comment #6)
> Why do you suppress FTP output? Is it producing too much output?

Yeah I almost didn't put that in... it matches the GetTFTP routine which is suppressing the TFTP output, so I kept the same in GetFTP to match.

It is pretty noisy (both the TFTP and FTP commands are), you see all the back and forth with the commands we are issuing and then going to the server. I guess the original idea of suppressing TFTP output was to keep the console nice and clean.

But it does mean if the server gives us an error we will never see it, which is not great.

I tried taking out the commands for output hiding, for both tftp and ftp.

In the TFTP case it looks like this on the console (with some CMS and zcon.py spew edited out):

    Connected to server 10.16.64.21

    Profile ibm-z-17.rhts.eng.bos.redhat.com found

    Downloading kernel �/images/ibm-z-17.rhts.eng.bos.redhat.com/kernel�...
    Command:
    Command:
    3839488 bytes transferred in 14.162 seconds. Transfer rate 271.114 Kbytes/sec.
    Command:
    Downloading initrd �/images/ibm-z-17.rhts.eng.bos.redhat.com/initrd�...
    Command:
    Command:
    28337936 bytes transferred in 61.481 seconds. Transfer rate 460.922 Kbytes/sec.
    Command:
    Starting install...

    00: 0000002 FILES PURGED
    [...]

which is not bad at all. And it gives some hints what went wrong if you mess up the arguments on the stack, like I did in one version of my patch. :-)

But the FTP version looks like this (with zcon and CMS junk stripped out)... It seems way too noisy to me:

    Connected to server 10.16.64.21

    Profile ibm-z-17.rhts.eng.bos.redhat.com found

    Downloading kernel �ftp://download.REDACTEDredhat.com/engarchive2/released/RHEL-
    7/7.2-RC-1/Server/s390x/os/images/kernel.img�...
    VM TCP/IP FTP Level 610
    Connecting to DOWNLOAD.REDACTEDREDHAT.COM 10.REDACTED, port 21
    220 Red Hat FTP server ready. All transfers are logged. �no EPSV�
    USER (identify yourself to the host):
    >>>USER anonymous
    331 Please specify the password.
    >>>PASS ********
    230 Login successful.
    Command:
    >>>TYPE i
    200 Switching to Binary mode.
    Command:
    >>>PORT 10,16,70,38,4,164
    200 PORT command successful. Consider using PASV.
    >>>RETR engarchive2/released/RHEL-7/7.2-RC-1/Server/s390x/os/images/kernel.img
    150 Opening BINARY mode data connection for engarchive2/released/RHEL-7/7.2-RC-1
    /Server/s390x/os/images/kernel.img (3839488 bytes).
    226 Transfer complete.
    3839488 bytes transferred in 0.482 seconds. Transfer rate 7965.74 Kbytes/sec.
    Command:
    >>>QUIT
    221 Goodbye.
    Downloading initrd �ftp://download.REDACTEDredhat.com/engarchive2/released/RHEL-
    7/7.2-RC-1/Server/s390x/os/images/initrd.img�...
    VM TCP/IP FTP Level 610
    Connecting to DOWNLOAD.REDACTEDREDHAT.COM 10.REDACTED, port 21
    220 Red Hat FTP server ready. All transfers are logged. �no EPSV�
    USER (identify yourself to the host):
    >>>USER anonymous
    331 Please specify the password.
    >>>PASS ********
    230 Login successful.
    Command:
    >>>TYPE i
    200 Switching to Binary mode.
    Command:
    >>>PORT 10,16,70,38,4,166
    200 PORT command successful. Consider using PASV.
    >>>RETR engarchive2/released/RHEL-7/7.2-RC-1/Server/s390x/os/images/initrd.img
    150 Opening BINARY mode data connection for engarchive2/released/RHEL-7/7.2-RC-1
    /Server/s390x/os/images/initrd.img (28337936 bytes).
    Command:
    >>>PORT 10,16,70,38,4,166
    200 PORT command successful. Consider using PASV.
    >>>RETR engarchive2/released/RHEL-7/7.2-RC-1/Server/s390x/os/images/initrd.img
    150 Opening BINARY mode data connection for engarchive2/released/RHEL-7/7.2-RC-1
    /Server/s390x/os/images/initrd.img (28337936 bytes).
    Command:
    >>>PORT 10,16,70,38,4,166
    200 PORT command successful. Consider using PASV.
    >>>RETR engarchive2/released/RHEL-7/7.2-RC-1/Server/s390x/os/images/initrd.img
    150 Opening BINARY mode data connection for engarchive2/released/RHEL-7/7.2-RC-1
    /Server/s390x/os/images/initrd.img (28337936 bytes).
    226 Transfer complete.
    28337936 bytes transferred in 9.068 seconds. Transfer rate 3125.05 Kbytes/sec.
    Command:
    >>>QUIT
    221 Goodbye.
    Starting install...
    [...]

I'm OK with non-verbose version. Since rexx script can be replaced at any time  to add extra verbosity we will be able to debug potential problems quite easily.

Hi Dan,
 Just checking in on this. Currently we have been blocked testing s/390x in Fedora since F22 because of this issue. Is there anyway we can get this in 24.0

Thanks in advance,
Jeff

This is ready to go on the Beaker side, but it requires the FTP change to be applied to zpxe.rexx in *all* z/VM guests before we can make the Beaker change, so we will have to coordinate it with Eng Ops.

Beaker 25.0 has been released.

Release notes are available upstream: https://beaker-project.org/docs/whats-new/release-25.html