Bug 1303738

Summary:	Initial reboot of discovered host fails
Product:	Red Hat Satellite	Reporter:	Tasos Papaioannou <tpapaioa>
Component:	Discovery Image	Assignee:	Lukas Zapletal <lzap>
Status:	CLOSED NOTABUG	QA Contact:	Katello Bug Bin <katello-bugs>
Severity:	medium	Docs Contact:
Priority:	medium
Version:	6.1.3	CC:	tpapaioa
Target Milestone:	Unspecified
Target Release:	Unused
Hardware:	All
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2016-02-25 10:34:28 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:

Description Tasos Papaioannou 2016-02-01 19:50:43 UTC

Description of problem:

Initial reboot of discovered host fails.

Version-Release number of selected component (if applicable):

foreman-discovery-image-2.1.0-36.el7sat.noarch
ruby193-rubygem-foreman_discovery-2.0.0.19-1.el7sat.noarch

How reproducible:

100%

Steps to Reproduce:

This is an issue seen while testing the provisioning of libvirt guests with the Red Hat Cloud Infrastructure Unified Installer, which is currently built on Satellite 6.1.3.

1.) Create libvirt guest to pxe boot into foreman discovery:

# virt-install -n foreman-discovery-test --os-variant rhel7 --memory 1024 --pxe --vcpus 1 --cpu host,-invtsc --disk bus="virtio,size=10" --network "bridge:br1" --boot "network,hd" --graphics "vnc,listen=0.0.0.0,password=redhat" --noautoconsole

Starting install...
Allocating 'foreman-discovery-test.qcow2' | 10 GB 00:00:00
Creating domain... | 0 B 00:00:00
Domain installation still in progress. You can reconnect to
the console to complete the installation process.

2.) Verify that foreman-discovery-test is running, and that it appears under Hosts > Discovered Hosts on the Satellite.

# virsh list
Id Name State
----------------------------------------------------
106 satellite running
139 foreman-discovery-test running

3.) In the Satellite web UI, click the down arrow next to Provision for this discovered host, and click Reboot. See that the host shut down instead of rebooting:

# virsh list
Id Name State
----------------------------------------------------
106 satellite running

4.) Alternatively, with ssh enabled on the foreman discovery image (adding "fdi.ssh=1 fdi.rootpw=XXXX" to the APPEND line in the Satellite's /var/lib/tftpboot/pxelinux.cfg/default file), the same behavior can be seen by logging in to the guest and running "shutdown -r now" manually:

# ssh 192.168.0.127
# shutdown -r now
Connection to 192.168.0.127 closed by remote host.
Connection to 192.168.0.127 closed.

# virsh list
Id Name State
----------------------------------------------------
106 satellite running

5.) I only see this issue on the first reboot attempt after installing the guest. If I then restart the guest with "virsh start foreman-discovery-test", further reboots from the shell or Satellite web UI are successful.

Actual results:

Reboot of newly-created discovered host from Satellite web UI or shell fails to restart the guest.

Expected results:

Successful reboot.

Additional info:

Comment 1 Lukas Zapletal 2016-02-24 15:08:38 UTC

Hello, unable to reproduce. But I've already encountered similar behavior and it was caused by the hypervisor. Some of them, including libvirt, treats the initial start of the VM in a special way - they expect an OS installation. Then they need to switch BIOS configuration in order to boot from first drive and this causes the problem.

You need to provide options to libvirt not to actually do this. The key option is I think --boot for virt-install instead of --pxe (from its man page: Use the PXE boot protocol to load the initial ramdisk and kernel for starting the guest installation process.)

https://github.com/lzap/bin-public/blob/master/virt-install-discovery

Please confirm.

Comment 2 Lukas Zapletal 2016-02-24 15:09:52 UTC

One more thing - in Satellite 6 all hosts should be set to boot from network. Provisioned hosts boot PXELinux which chainloads from first drive, therefore there is no need to change the BIOS, even when you want to reprovision it.

Comment 3 Lukas Zapletal 2016-02-24 15:25:23 UTC

I am pretty sure this is the issue here, created documentation for this particular case: https://github.com/theforeman/theforeman.org/pull/545

Comment 4 Tasos Papaioannou 2016-02-24 15:39:33 UTC

Removing the --pxe option from virt-install (leaving "network" as the first --boot option) resolved the issue. Thanks!

Comment 5 Lukas Zapletal 2016-02-25 10:34:28 UTC

Thanks for info!