Bug 735054

Summary: minimal install fails because wget is not available
Product: [Retired] Beaker Reporter: Petr Beňas <pbenas>
Component: testsAssignee: Amit Saha <asaha>
Status: CLOSED CURRENTRELEASE QA Contact: Qixiang Wan <qwan>
Severity: high Docs Contact:
Priority: high    
Version: 0.7CC: asaha, bnater, bpeck, dcallagh, ebaak, jburke, jstancek, mbanas, mcsontos, mganisin, mishin, pfrields, pstehlik, qwan, rmancy, stl
Target Milestone: 0.10.0Keywords: TestBlocker
Target Release: ---   
Hardware: ppc64   
OS: Unspecified   
Whiteboard: Kickstart
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-11-22 06:44:14 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Console output when the bug occurs
none
kickstart file from job where bug did not occur, 'basic server' was selected instead of 'minimal' variant
none
simple implement of wget none

Description Petr Beňas 2011-09-01 10:09:31 UTC
Description of problem:
Unable to perform minimal RHEL6 installation via VNC on beaker host.
Probably not reporting against the right component, feel free to reasign.

Version-Release number of selected component (if applicable):


How reproducible:
100% on two different ppc64 machines, but don't think it's arch specific.

Steps to Reproduce:
1. Submit a bkr reserve job with manual/vnc options.
2. Connect via vnc, set 'Use whole diskspace' (probably not needed)
3. Select 'Minimal' variant
4. Use 'Customize Later' (probably not needed)
5. Start the installation and watch console for following error
  
Actual results:
/tmp/ks-script-K4Aser: line 205: wget: command not found
chmod: cannot access `/etc/rc.d/init.d/koan': No such file or directory
error reading information on service koan: No such file or directory
/tmp/ks-script-K4Aser: line 229: wget: command not found
/tmp/ks-script-K4Aser: line 230: wget: command not found
/tmp/ks-script-K4Aser: line 231: wget: command not found

Expected results:
Successful installation. Install wget or workaround the wget-dependent script.

Additional info:

Comment 1 Petr Beňas 2011-09-01 10:12:32 UTC
Forgot to mention Release number. Reproducible in:
RHEL6.2-20110822.5_nfs-Server-ppc64
RHEL6-6.1_nfs-Server-ppc64

Comment 3 David Cantrell 2011-09-01 13:28:38 UTC
Please attach the kickstart file you are using.  wget is present in the installation environment.  It's in /sbin.

Comment 4 Petr Beňas 2011-09-01 13:33:59 UTC
(In reply to comment #3)
> Please attach the kickstart file you are using.  wget is present in the
> installation environment.  It's in /sbin.

I am not using any custom kickstart. Just clicked Scheduler -> Reserve in Beaker WebUI, then selected arch and distro, clicked Auto pick system. Filled 'manual' to KickStart MetaData field and 'vnc' to Kernel Options (install) field. Submitted by clicking on Queue job button.

Comment 5 David Cantrell 2011-09-01 13:44:04 UTC
(In reply to comment #0)
> Actual results:
> /tmp/ks-script-K4Aser: line 205: wget: command not found
> chmod: cannot access `/etc/rc.d/init.d/koan': No such file or directory
> error reading information on service koan: No such file or directory
> /tmp/ks-script-K4Aser: line 229: wget: command not found
> /tmp/ks-script-K4Aser: line 230: wget: command not found
> /tmp/ks-script-K4Aser: line 231: wget: command not found

"/tmp/ks-script-K4Aser" means a kickstart file is in use.  Beaker generates these, I believe.

Log in as root to the host you are trying to install on once the installation is running (successful or not) and collect the log files and the /tmp/ks-script* file from /tmp and attach it to this bug report.  Please attach them individually as plain text files.

Comment 6 Petr Beňas 2011-09-01 15:24:20 UTC
(In reply to comment #5)
> Log in as root to the host you are trying to install on once the installation
> is running (successful or not) and collect the log files and the
> /tmp/ks-script* file from /tmp and attach it to this bug report.  Please attach
> them individually as plain text files.

How should I log in to the machine? I'm getting "Connection refused" on every ssh attempt to the machine during the installation, serial console seems to be read-only (excluding the language prompt at the beginning of the install process). 

Installation hangs with 'Running post-installation scripts' on the screen, errors from comment 0 on the serial console. When the host is rebooted, it begins new installation. 

I can attach the install process serial console output. 
I will also attach kickstart file from similar job, where 'Basic server' instead of 'Minimal' was selected. In that job this bug haven't occur, so I was able to get the kickstart file. 

How can be ssh access enabled during the install process? Or how should I get the kicstart file from the not-yet-installed machine?

thanks in advance

Comment 7 Petr Beňas 2011-09-01 15:27:49 UTC
Created attachment 521034 [details]
Console output when the bug occurs

Comment 8 Petr Beňas 2011-09-01 15:29:03 UTC
Created attachment 521035 [details]
kickstart file from job where bug did not occur, 'basic server' was selected instead of 'minimal' variant

Comment 9 David Cantrell 2011-09-01 17:29:04 UTC
It looks like Minimal does not include wget which this kickstart's %post depends on.  Does installation succeed if you do a minimal+wget install?

Comment 10 Petr Beňas 2011-09-05 09:20:07 UTC
The problem shows up when 'Customize now' is used instead 'Customize later'. When 'Basic server' selected, 'Customize now' shows several groups selected. 
It seems 'Minimal' does preselect nothing. Selecting 'Base' group solves this bug (wget is part of Base group).

Comment 11 Chris Lumens 2011-09-05 19:33:53 UTC
Either the beaker kickstart file needs to be changed to not use wget, or to include wget in the list of packages to install, or comps needs to include wget as part of the Minimal package set.

Comment 12 Bill Peck 2011-09-06 17:45:19 UTC
python will always be present right?  If so then we should be able implement the basics of wget in python itself.  

ryang,  Is this something you feel comfortable enough doing?

Comment 13 Yang Ren 2011-09-07 02:09:03 UTC
Created attachment 521783 [details]
simple implement of wget

attached a simple wget implement in python. It only can download files. Not support any other functions in wget.

@bpeck, is this enough for use?

Comment 14 Yang Ren 2011-09-07 05:54:14 UTC
We should also change cobbler snippet for auto provision kicktemplate.

Add something like this properly:

if [ -d /usr/bin/wget ] ; then
    ./wget.py XXXX
fi

rpm -q wget
if [ $? != 0 ]; then
    ./wget.py XXX
fi

Comment 15 Yang Ren 2011-09-07 06:00:31 UTC
The two snippet using wget are 

post_koan:wget -O /etc/rc.d/init.d/koan "http://$server/beaker/koan.init"
rhts_post:/usr/bin/wget -N http://$server/beaker/$yum

The problem now is where to put wget.py script. If know the script place in test machine. We can change the snippet. 

And in wget.py I use urllib and sys modules. They should all belong to basic python installation, right?

Comment 16 Petr Beňas 2011-09-12 08:06:38 UTC
And what about the issue described in comment 10? Isn't it necessary to pre-select the Base package group? 
Using 'different wget' may solve the original issue from bug description, but will the installation succeed when Base group is not selected?

Comment 17 Dan Callaghan 2011-09-14 05:42:10 UTC
Another possibility is to use /usr/bin/urlgrabber, which is part of python-urlgrabber (a dependency of yum).

Comment 18 Bill Peck 2011-09-14 13:06:05 UTC
(In reply to comment #17)
> Another possibility is to use /usr/bin/urlgrabber, which is part of
> python-urlgrabber (a dependency of yum).

Except we use wget to install yum on rhel3/4 :-)

Comment 19 Yang Ren 2011-09-15 14:54:01 UTC
@bpeck,

   I install successed with only @Base and @Core for packages setting in kickstart.
   But always got error 
"Checking dmesg for specific failures!
WARNING: CPU#0: NMI appears to be stuck (0->0)!
End of log.
"
   Strange the error will not occur if more package is selected.

   Find in runtest.sh of rh-tests-distribution-install-default
   https://bugzilla.redhat.com/show_bug.cgi?id=500845

   Is it means the test failed.

   And I have tried RHEL-6-GOLD-Client-x86_64, Is it necessary to test it in more distro.

   And I changed all wget in 6 snippet files, They are:
download_config_files
post_anamon
post_koan
pre_anamon
redhat_register
rhts_post

Not all of them can be find in beaker/LabController/snippets folder. Where can i submit new one?

Comment 20 Bill Peck 2011-09-15 15:03:39 UTC
That initial error is a RHEL bug on kvm systems I believe.  In fact I think if you install 6.1 you won't see it.

The other snippets using wget come from cobbler.  We'll have to get those fixed upstream.  In fact we should probably get wget.py upstream in cobbler.

Can you handle that?  If not I can take over and submit a patch to upstream cobbler.

Comment 21 Yang Ren 2011-09-16 02:08:32 UTC
I can do that. First I will push the beaker part patch to gerrit for review. If no problem I'll push the rest snippets to cobbler upstream. Got some change in wget.py.

http://beaker-gerrit.app.eng.bne.redhat.com/#change,678

Comment 22 Dan Callaghan 2012-09-27 21:53:23 UTC
*** Bug 861086 has been marked as a duplicate of this bug. ***

Comment 23 Dan Callaghan 2012-09-27 21:54:34 UTC
In bug 861086 mganisin suggests using curl. If that is always available, it might be simpler to switch to that, than trying to include our own minimal wget in Python.

Comment 24 Marian Ganisin 2012-10-01 06:35:52 UTC
Actually I think curl isn't present in RHEL5 minimal installation. However RHEL5 minimal is missing many critical packages (dhclient for example), so it should not be a blocker for this issue and usage of curl seems to be fine from el6 point of view and it should be fine also for future releases.

Comment 25 Martin Banas 2012-10-22 07:19:18 UTC
Hello,
any update on this bug? We really want this to be fixed ASAP since it breaks various tests in Beaker. Moreover, machines keep reinstalling, which also reduces our resources (tests lasts for 10 hours instead of 1, for example)..

Thank you

Comment 26 Marian Ganisin 2012-10-22 07:22:39 UTC
I don't see target milestone set unfortunately. However this bug is quite critical. Due to current state of most recent versions of installer just minimal package set is installed by default (without wget) and testing is effectively blocked in some cases. Custom provisioning is one of important affected areas. According to several proposed solutions implementation of bugfix seems to be quite simple.

Please look at this issue, consider its priority once more and try to fix it today. ;) Thank you very much in advance.

Comment 27 Dan Callaghan 2012-10-23 00:06:13 UTC
This is not trivial to fix. We already in the past switched from python to wget for making these calls, because some variant of RHEL3 s390x (I don't remember which) lacked python on the installed system. You have suggested using curl, but are you sure that it is available in all variants of all arches of RHEL3 upwards? I don't know of any nice automated way to find a list of installed packages for every kind of installation we have to support in Beaker, so there is a lot of testing involved.

The best fix is probably to switch back to making the calls with python using xmlrpclib but in %post --nochroot, since Anaconda is written in python so it must always be present in the installer image.

Comment 28 Dan Callaghan 2012-10-23 00:08:42 UTC
The other thing is we could change this for RHEL6 kickstarts only, and leave the other distros as is. That would reduce the amount of testing needed. Judging by the comments on this bug it is only RHEL6 minimal installs which people care about, not any earlier releases.

Comment 30 Amit Saha 2012-10-23 05:09:32 UTC
In gerrit: http://gerrit.beaker-project.org/#/c/1432/

Comment 31 Marian Ganisin 2012-10-23 08:54:26 UTC
(In reply to comment #27)
> This is not trivial to fix. We already in the past switched from python to
> wget for making these calls, because some variant of RHEL3 s390x (I don't
> remember which) lacked python on the installed system. You have suggested
> using curl, but are you sure that it is available in all variants of all
> arches of RHEL3 upwards?

Quick check made me quite sure. However I am almost sure wget isn't available in any minimal installation.

> The best fix is probably to switch back to making the calls with python
> using xmlrpclib but in %post --nochroot, since Anaconda is written in python
> so it must always be present in the installer image.

Yes, this sounds like a good idea.

Comment 34 Amit Saha 2012-10-23 23:38:58 UTC
(In reply to comment #31)
> (In reply to comment #27)
> > This is not trivial to fix. We already in the past switched from python to
> > wget for making these calls, because some variant of RHEL3 s390x (I don't
> > remember which) lacked python on the installed system. You have suggested
> > using curl, but are you sure that it is available in all variants of all
> > arches of RHEL3 upwards?
> 
> Quick check made me quite sure. However I am almost sure wget isn't
> available in any minimal installation.

Are you implying that 'curl' is present in RHEL 3/4/5 in the minimal install for all supported architectures?

Comment 35 Marian Ganisin 2012-10-24 06:38:23 UTC
(In reply to comment #34)
> (In reply to comment #31)
> > Quick check made me quite sure. However I am almost sure wget isn't
> > available in any minimal installation.
> 
> Are you implying that 'curl' is present in RHEL 3/4/5 in the minimal install
> for all supported architectures?

Unfortunately not. Anyway I would not care about older releases a lot. For example minimal installation of RHEL 5 is missing other important stuff like dhcp client or even yum. Present and future matters. ;)

Btw. yesterday in a bath I realized quite solid solution:

1. do wget,
2. if it fails, do curl,
3. if it fails do python
...and...
*  do not chroot!

I am almost sure, this will work in any situation on any distro. (I don't indicate that current proposal is wrong though)

Comment 37 Amit Saha 2012-10-26 00:23:07 UTC
The final solution uses 'wget' in the %pre install stages and 'curl' for all in %post for all RHEL 6+  and Fedora releases. 

Final patch: http://gerrit.beaker-project.org/#/c/1432/

Comment 38 Paul W. Frields 2012-11-08 14:02:42 UTC
What's the current status of this change?  RHEL 7.0 still shows this as a test blocker.

Comment 39 Amit Saha 2012-11-09 02:47:17 UTC
It will be included in the upcoming Beaker 0.10 scheduled to be released on November 21.

Comment 44 Raymond Mancy 2012-11-22 06:44:14 UTC
This has now been released