1303738 – Initial reboot of discovered host fails

Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1303738 - Initial reboot of discovered host fails

Summary: Initial reboot of discovered host fails

Keywords:
Status:	CLOSED NOTABUG
Alias:	None
Product:	Red Hat Satellite
Classification:	Red Hat
Component:	Discovery Image
Sub Component:
Version:	6.1.3
Hardware:	All
OS:	Linux
Priority:	medium
Severity:	medium
Target Milestone:	Unspecified
Assignee:	Lukas Zapletal
QA Contact:	Katello Bug Bin
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+	depends on / blocked

Reported:	2016-02-01 19:50 UTC by Tasos Papaioannou
Modified:	2016-02-25 10:34 UTC (History)
CC List:	1 user (show)
Fixed In Version:
Doc Type:	Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed:	2016-02-25 10:34:28 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Description Tasos Papaioannou 2016-02-01 19:50:43 UTC

Description of problem:

Initial reboot of discovered host fails.

Version-Release number of selected component (if applicable):

foreman-discovery-image-2.1.0-36.el7sat.noarch
ruby193-rubygem-foreman_discovery-2.0.0.19-1.el7sat.noarch

How reproducible:

100%

Steps to Reproduce:

This is an issue seen while testing the provisioning of libvirt guests with the Red Hat Cloud Infrastructure Unified Installer, which is currently built on Satellite 6.1.3.

1.) Create libvirt guest to pxe boot into foreman discovery:

# virt-install -n foreman-discovery-test --os-variant rhel7 --memory 1024 --pxe --vcpus 1 --cpu host,-invtsc --disk bus="virtio,size=10" --network "bridge:br1" --boot "network,hd" --graphics "vnc,listen=0.0.0.0,password=redhat" --noautoconsole

Starting install...
Allocating 'foreman-discovery-test.qcow2' | 10 GB 00:00:00
Creating domain... | 0 B 00:00:00
Domain installation still in progress. You can reconnect to
the console to complete the installation process.

2.) Verify that foreman-discovery-test is running, and that it appears under Hosts > Discovered Hosts on the Satellite.

# virsh list
Id Name State
----------------------------------------------------
106 satellite running
139 foreman-discovery-test running

3.) In the Satellite web UI, click the down arrow next to Provision for this discovered host, and click Reboot. See that the host shut down instead of rebooting:

# virsh list
Id Name State
----------------------------------------------------
106 satellite running

4.) Alternatively, with ssh enabled on the foreman discovery image (adding "fdi.ssh=1 fdi.rootpw=XXXX" to the APPEND line in the Satellite's /var/lib/tftpboot/pxelinux.cfg/default file), the same behavior can be seen by logging in to the guest and running "shutdown -r now" manually:

# ssh 192.168.0.127
# shutdown -r now
Connection to 192.168.0.127 closed by remote host.
Connection to 192.168.0.127 closed.

# virsh list
Id Name State
----------------------------------------------------
106 satellite running

5.) I only see this issue on the first reboot attempt after installing the guest. If I then restart the guest with "virsh start foreman-discovery-test", further reboots from the shell or Satellite web UI are successful.

Actual results:

Reboot of newly-created discovered host from Satellite web UI or shell fails to restart the guest.

Expected results:

Successful reboot.

Additional info:

Comment 1 Lukas Zapletal 2016-02-24 15:08:38 UTC

Hello, unable to reproduce. But I've already encountered similar behavior and it was caused by the hypervisor. Some of them, including libvirt, treats the initial start of the VM in a special way - they expect an OS installation. Then they need to switch BIOS configuration in order to boot from first drive and this causes the problem.

You need to provide options to libvirt not to actually do this. The key option is I think --boot for virt-install instead of --pxe (from its man page: Use the PXE boot protocol to load the initial ramdisk and kernel for starting the guest installation process.)

https://github.com/lzap/bin-public/blob/master/virt-install-discovery

Please confirm.

Comment 2 Lukas Zapletal 2016-02-24 15:09:52 UTC

One more thing - in Satellite 6 all hosts should be set to boot from network. Provisioned hosts boot PXELinux which chainloads from first drive, therefore there is no need to change the BIOS, even when you want to reprovision it.

Comment 3 Lukas Zapletal 2016-02-24 15:25:23 UTC

I am pretty sure this is the issue here, created documentation for this particular case: https://github.com/theforeman/theforeman.org/pull/545

Comment 4 Tasos Papaioannou 2016-02-24 15:39:33 UTC

Removing the --pxe option from virt-install (leaving "network" as the first --boot option) resolved the issue. Thanks!

Comment 5 Lukas Zapletal 2016-02-25 10:34:28 UTC

Thanks for info!

Note You need to log in before you can comment on or make changes to this bug.