Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1334048

Summary: Foreman discovery image proxy service fails to start when network is not yet initialized
Product: Red Hat Satellite Reporter: Paul Armstrong <parmstro>
Component: Discovery ImageAssignee: Lukas Zapletal <lzap>
Status: CLOSED ERRATA QA Contact: Sachin Ghai <sghai>
Severity: high Docs Contact:
Priority: unspecified    
Version: 6.1.8CC: bbuckingham, cwelton, lzap, sghai
Target Milestone: UnspecifiedKeywords: Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-07-27 11:15:39 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Paul Armstrong 2016-05-07 15:27:04 UTC 
Description of problem: foreman-proxy service fails to start, communication from Capsule to fdi then fails. 


Version-Release number of selected component (if applicable):
6.1.8

How reproducible:
One machine never fails. Other machine fails consistently. Machines are (theoretically) identical.

Steps to Reproduce:
1. Clean disk.
2. PXE boot.
3. Boot FDI

Actual results:
System boots, registers to Satellite, but due to foreman-proxy failure can not be provisioned without a manual reboot.

Expected results:
foreman-proxy should not fail to start.

Additional info:
SSH into the box and manually restart foreman-proxy and all is OK. foreman-proxy starts fine and can communicate with Satellite and can be provisioned remotely. ??foreman-proxy can't find satellite at initial boot?? 

May 06 18:57:31 fdi NetworkManager[790]: <info>  startup complete
May 06 18:57:31 fdi systemd[1]: Started Network Manager Wait Online.
May 06 18:57:31 fdi systemd[1]: Reached target Network.
May 06 18:57:31 fdi systemd[1]: Starting Network.
May 06 18:57:31 fdi systemd[1]: Starting Foreman Proxy...
May 06 18:57:31 fdi systemd[1]: Started Display interactive TUI on tty1.
May 06 18:57:31 fdi systemd[1]: Starting Display interactive TUI on tty1...
May 06 18:57:31 fdi systemd[1]: Starting /etc/rc.d/rc.local Compatibility...
May 06 18:57:31 fdi systemd[1]: Started /etc/rc.d/rc.local Compatibility.
May 06 18:57:31 fdi generate-proxy-cert[1107]: Generating a 2048 bit RSA private key
May 06 18:57:31 fdi generate-proxy-cert[1107]: ......................+++
May 06 18:57:31 fdi generate-proxy-cert[1107]: ...........+++
May 06 18:57:31 fdi generate-proxy-cert[1107]: writing new private key to '/etc/foreman-proxy/key.pem'
May 06 18:57:31 fdi generate-proxy-cert[1107]: -----
May 06 18:57:31 fdi generate-proxy-cert[1107]: end of string encountered while processing type of subject name element #0
May 06 18:57:31 fdi generate-proxy-cert[1107]: problems making Certificate Request
May 06 18:57:31 fdi systemd[1]: foreman-proxy.service: control process exited, code=exited status=1
May 06 18:57:31 fdi systemd[1]: Failed to start Foreman Proxy.
May 06 18:57:31 fdi systemd[1]: Unit foreman-proxy.service entered failed state.
May 06 18:57:31 fdi systemd[1]: foreman-proxy.service failed.
May 06 18:57:31 fdi systemd[1]: Reached target Multi-User System.
May 06 18:57:31 fdi systemd[1]: Starting Multi-User System.
May 06 18:57:31 fdi systemd[1]: Starting Update UTMP about System Runlevel Changes...
May 06 18:57:31 fdi systemd[1]: Started Stop Read-Ahead Data Collection 10s After Completed Startup.
May 06 18:57:31 fdi systemd[1]: Starting Stop Read-Ahead Data Collection 10s After Completed Startup.
May 06 18:57:31 fdi systemd[1]: Started Update UTMP about System Runlevel Changes.
May 06 18:57:31 fdi systemd[1]: Startup finished in 2.378s (kernel) + 1.926s (initrd) + 11.088s (userspace) = 15.393s.
May 06 18:57:31 fdi /usr/bin/discovery-menu[1108]: Kernel opts: initrd=boot/fdi-image-rhel_7-img rootflags=loop root=live:/fdi.iso rootfstype=auto ro rd.live.image acpi
May 06 18:57:31 fdi /usr/bin/discovery-menu[1108]: Entering screen_countdown
May 06 18:57:32 fdi NetworkManager[790]: <info>  (eno1): link connected
May 06 18:57:32 fdi NetworkManager[790]: <info>  (eno1): device state change: unavailable -> disconnected (reason 'carrier-changed') [20 30 40]
May 06 18:57:32 fdi kernel: e1000e: eno1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
May 06 18:57:32 fdi kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eno1: link becomes ready
May 06 18:57:32 fdi NetworkManager[790]: <info>  Auto-activating connection 'primary'.
May 06 18:57:32 fdi NetworkManager[790]: <info>  (eno1): Activation: starting connection 'primary' (5e84d9cc-13bc-11e6-8ce6-eca86bf29f9f)
May 06 18:57:32 fdi NetworkManager[790]: <info>  (eno1): device state change: disconnected -> prepare (reason 'none') [30 40 0]
May 06 18:57:32 fdi NetworkManager[790]: <info>  NetworkManager state is now CONNECTING
Comment 1 Lukas Zapletal 2016-05-09 11:36:54 UTC 
Analysis:

The proxy unit is missing After=basic.target network-online.target rule and can occassionally start when network is not yet online. The proxy certificate generator requires a valid IP address. For this reason, it fails to generate HTTPS cert therefore the service fails to start.

This was fixed in 6.2 BETA already. 0c18ba2a6d04e5105db1e2085fe69f091b6922c7

Workaround:

Use fdi image from 6.2 BETA repositories, upgrade it, restart.

Triage notes: We can backport this patch into 6.1 repository if needed.
Comment 5 Sachin Ghai 2016-06-28 07:45:46 UTC 
@Lzap: Could you please provide steps to verify this bz. In general I can discover host without any issue but just wanted to see if any extra verification required. thanks
Comment 6 Lukas Zapletal 2016-07-07 15:35:26 UTC 
One idea is: turn DHCP service off, start FDI, wait a bit and enable it. But you must not keep it turned off for more than 45 seconds, otherwise FDI will timeout.

https://bugzilla.redhat.com/show_bug.cgi?id=1262922
Comment 7 Sachin Ghai 2016-07-08 10:15:52 UTC 
Thanks Lukas. I tried to reproduce the issue with suggested steps in comment6 using sat62 GA snap19.

I turned off dhcpd and started the FDI and later switched it on. Host was registered successfully and foreman-proxy service was active.

Later, I tried FDI TUI, where I selected the "Discover via DHCP" and service was down for some time and then I restarted the dhcpd and host get the IP and discovered successfully.

In both cases, I see foreman-proxy service in active state. thanks
Comment 8 Bryan Kearney 2016-07-27 11:15:39 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:1501