Bug 103786

Summary: unable to ipl in LPAR
Product: Red Hat Enterprise Linux 3 Reporter: Jim Sibley <jlsibley>
Component: s390utilsAssignee: Phil Knirsch <pknirsch>
Status: CLOSED RAWHIDE QA Contact: David Lawrence <dkl>
Severity: medium Docs Contact:
Priority: high    
Version: 3.0CC: borgan, jlaska, karsten, laroche, mgrf, rvokal
Target Milestone: ---   
Target Release: ---   
Hardware: s390   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2003-10-02 17:26:28 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Jim Sibley 2003-09-04 21:38:55 UTC
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.3.1) Gecko/20030425

Description of problem:
I have built a system that IPLs under VM sucessfully. 

When I try to IPL the system in an LPAR, I get a disabled wait state code:

000A0000E0F00E00F 0008000080002008




Version-Release number of selected component (if applicable):
taroon beta2

How reproducible:
Always

Steps to Reproduce:
1. Build system under VM
2. IPL under LPAR
3.
    

Actual Results:  unsuccessful IPL - wait state code 000A0000E0F00E00F
0008000080002008



Expected Results:  IPL of system without any changes

Additional info:

All my volumes are real 3390-3 volumes, not minidisk, with CDL format and ext3
file system. It is setup as follows:

5601 /dev/dasda1     /
     /dev/md0        /usr
        5602 /dev/dasdb
        5611 /dev/dasdc
5612 /dev/dasdd1     swap

the /dev/md0 consists of /dev/dasdb and /dev/dasdc

After I was unsucessful in IPLing, I changed /etc/zipl.conf to include the
dasd=5601,5602,5611,5612 parameter, did a zipl and tried again. Same result.

Comment 1 Georg Markgraf 2003-09-05 10:33:00 UTC
Hello, 
we experienced that the devno from the DASD used for the initial installation 
is hard coded in the initrd. With that it is impossible to boot the system on 
different DASDs, respective LPARs etc. 
This could be the same reason. We did not open a bugzilla yet.
   

Comment 2 Georg Markgraf 2003-09-06 09:32:42 UTC
Jim I would propose to increase the severity to high as for the impact on your 
test progress:
Here is the comment you tried to add before completion of your access group 
def's   ciao Georg

"If what Georg says is true, this would have an enormous impact on
> maintaining systems for LPARs. Currently, we build a standard system under
> VM on real volumes.
> 
> "To clone them for LPAR use, we copy the volumes to new volumes, change the
> dasd volumes assigned in /etc/zipl.conf and do a chroot/zipl/exit. We then
> only have to edit HOSTNAME entries and TCP/IP entries.
> 
> "If the device address is in initrd, then this adds additional work in
> preparing new releases for LPARs. We currently have approximately 25-30
> LPARs active. So this would have a large impact on us."

Comment 3 Jim Sibley 2003-09-08 16:08:23 UTC
This is a show stopper for me because I cannot test under LPARs. In additon, I
need to have system that IPL both under VM and LPARs so that they can be used
interchangeablility for performance and functionality testing without changing
the testbed.

Comment 4 Jeremy Katz 2003-09-08 19:47:22 UTC
Instead of changing the zipl.conf, you need to change the dasd range in
/etc/modules.conf and rerun mkinitrd.  This is intentional because otherwise,
it's far too easy for people to lose needed kernel module options and break
their systems which would boot fine otherwise.

Comment 5 Jim Sibley 2003-09-08 21:08:31 UTC
I could not IPL in an LPAR BEFORE I attempted to change zipl.conf. I built the
system under VM to real volumes. I then attempted to ipl from the same addresses
that I ipl'd from VMin an LPAR and the system would not IPL at all, but went
into the wait state provided.

As to your comment
"Instead of changing the zipl.conf, you need to change the dasd range in
/etc/modules.conf and rerun mkinitrd.  This is intentional because otherwise,
it's far too easy for people to lose needed kernel module options and break
their systems which would boot fine otherwise."

We have been using this cloning procedure since zipl came out and it has NOT
cause us any problem. Where exactly is the change to modules.conf and mkinitrd
documented? 

If anything, the comment is only a side issue to the real problem. If I can IPL
the volume under VM, why can't I IPL it in an LPAR?

 


Comment 7 Karsten Hopp 2003-09-15 12:48:42 UTC
Just to make sure that I get it right: 
 
> I then attempted to ipl from the same addresses that I ipl'd from VMin an LPAR and the system would not > 
> IPL at all, but went into the wait state provided. 
 
You have installed on DASDs 5601,5602,5611,5612 in VM, copied those DASDs to an LPAR with exactly 
the same devicenumbers and an IPL doesn't work ? 

Comment 8 Jim Sibley 2003-09-15 16:22:08 UTC
NEGATIVE!!!!!!

I did the following:

1) On my VM guest, 

att 5601 * 5601
att 5602 * 5602
att 5611 * 5611
att 5612 * 5612

2) set my parameter file for the vmdrd install as

    dasd=5601,5602,5611,5612

2) did a vmrdr install to these volumes

     /dev/dasda1 = 5601   = /
     /dev/dasdb1 = 5602   = /usr
     /dev/dasdc1 = 5611   = /usr/share
     /dev/dasdd1 = 5621   = swap   

3) ipl'd 5601 to verify that it is iplable under VM, which is was.

4) Without any modification or change, I ipl'd 5601 in an LPAR. It got the
waitstate above.

I have gone back and rebuilt the system. I got the same results.

Comment 9 Florian La Roche 2003-09-15 16:25:32 UTC
Do we have a LPAR case inside Red Hat where we can reproduce this problem?

Thanks a lot,

Florian La Roche


Comment 10 Jim Sibley 2003-09-26 19:43:56 UTC
Apparently the dasd_eckd_mod is missing. 

I finally got it to work as follows:

1) you specify the dasd devices in /etc/modules.conf, i.e.

alias qeth0 qeth
options dasd_mod dasd=xxxx,yyyy,zzzz

2) Then you need to run mkinitrd:

a) do an "ls" on /lib/modules. The subdirectory name is what you need for the
mkinitrd. 
For example, /lib/modules/this-release

b) mkinitrd --preload dasd_eckd_mod new-initrd \
       this-release

3) change /etc/zipl so that
        ...
        ramdisk=/boot/new-initrd
        parameters="root=/dev/dasda1"        

    (also, other parameters, such as cio_ignore have to be coded here)

4) do a zipl

I tried this to clone a system, then use chroot to make the mods above. The
mkinitrd gave the following error messages, but it seemd to work:

/bin/bash: line 1: id: command not found
/bin/bash: line 1: id: command not found
/bin/bash: line 1: id: command not found
/sbin/mkinitrd: line 1: id: command not found
/sbin/mkinitrd: line 282: Y: !=: unary operator expected
/sbin/mkinitrd:line 1: tail: command not found

I could at least IPL the cloned system.

Comment 11 Karsten Hopp 2003-09-26 20:01:22 UTC
dasd_eckd_mod gets added to the initrd when the directory /proc/dasd exists. 
Can you please check which modules are in /lib of the original initrd ? 
 
It looks like you lost /usr/bin from your $PATH when you run chroot. 
 

Comment 12 Jim Sibley 2003-09-26 20:32:56 UTC
two different issues:

1) /proc/dasd: on the base system (no chroot involved), when I used the system
as distributed, I cannot IPL in an LPAR. When I do the mkinitrd
--preload=dasd_eckd_mod, I can ipl in an LPAR.

It appears that, in an LPAR, dasd_eckd_mod is needed by initrd well before
/proc/dasd exists.

2) chroot: Since I have a multi-pack system and /usr is on a different volume,
then I suppose the chroot is not finding /usr. (/usr is on a separate pack
because of the size of the dircetory and I am constrained to 3390-3 images).


Comment 13 Jim Sibley 2003-09-26 22:38:33 UTC
I built a "small" beta2 system to see what is happeninig. This is what I found

1) The system as built under vm will not IPL in an LPAR
2) If you do a mkinitrd (under vm), the /proc/dasd is present and the initrd is
corrected so that it will ipl in an lpar 
3) If you copy the respack pack to a new volume using "tar" or "cp" and change
the device addresses in modules.conf (and the root=/ to root/dev/dasda1 in
zipl.conf), then do the mkinitrd, the system will ipl in an LPAR (this is a bit
confusing because if you do a "ls" the chroot root does not show /proc/dasd).
4) With a small system, with /usr/bin on the root volume, the mkinitrd under
chroot, the miscellaneous error messages disappear.

I then went back to the beta2 cds and searched them for mkinitrd. There was no
mention of how to use mkinitrd or that you needed to run it to change the dasd
addresses.

How is this going to be handled on the candidate cd's? 

Comment 14 Jim Sibley 2003-10-02 16:45:49 UTC
Problem appears to be fixed in RC1. lsmod showed dasd_mod_eckd present after VM
build. I was able to ipl in an LPAR with NO CHANGES.

Comment 15 Florian La Roche 2003-10-02 17:26:28 UTC
That is very good news. Thanks a lot for your thorough testing!

greetings,

Florian La Roche