Description of problem: foreman-proxy service fails to start, communication from Capsule to fdi then fails. Version-Release number of selected component (if applicable): 6.1.8 How reproducible: One machine never fails. Other machine fails consistently. Machines are (theoretically) identical. Steps to Reproduce: 1. Clean disk. 2. PXE boot. 3. Boot FDI Actual results: System boots, registers to Satellite, but due to foreman-proxy failure can not be provisioned without a manual reboot. Expected results: foreman-proxy should not fail to start. Additional info: SSH into the box and manually restart foreman-proxy and all is OK. foreman-proxy starts fine and can communicate with Satellite and can be provisioned remotely. ??foreman-proxy can't find satellite at initial boot?? May 06 18:57:31 fdi NetworkManager[790]: <info> startup complete May 06 18:57:31 fdi systemd[1]: Started Network Manager Wait Online. May 06 18:57:31 fdi systemd[1]: Reached target Network. May 06 18:57:31 fdi systemd[1]: Starting Network. May 06 18:57:31 fdi systemd[1]: Starting Foreman Proxy... May 06 18:57:31 fdi systemd[1]: Started Display interactive TUI on tty1. May 06 18:57:31 fdi systemd[1]: Starting Display interactive TUI on tty1... May 06 18:57:31 fdi systemd[1]: Starting /etc/rc.d/rc.local Compatibility... May 06 18:57:31 fdi systemd[1]: Started /etc/rc.d/rc.local Compatibility. May 06 18:57:31 fdi generate-proxy-cert[1107]: Generating a 2048 bit RSA private key May 06 18:57:31 fdi generate-proxy-cert[1107]: ......................+++ May 06 18:57:31 fdi generate-proxy-cert[1107]: ...........+++ May 06 18:57:31 fdi generate-proxy-cert[1107]: writing new private key to '/etc/foreman-proxy/key.pem' May 06 18:57:31 fdi generate-proxy-cert[1107]: ----- May 06 18:57:31 fdi generate-proxy-cert[1107]: end of string encountered while processing type of subject name element #0 May 06 18:57:31 fdi generate-proxy-cert[1107]: problems making Certificate Request May 06 18:57:31 fdi systemd[1]: foreman-proxy.service: control process exited, code=exited status=1 May 06 18:57:31 fdi systemd[1]: Failed to start Foreman Proxy. May 06 18:57:31 fdi systemd[1]: Unit foreman-proxy.service entered failed state. May 06 18:57:31 fdi systemd[1]: foreman-proxy.service failed. May 06 18:57:31 fdi systemd[1]: Reached target Multi-User System. May 06 18:57:31 fdi systemd[1]: Starting Multi-User System. May 06 18:57:31 fdi systemd[1]: Starting Update UTMP about System Runlevel Changes... May 06 18:57:31 fdi systemd[1]: Started Stop Read-Ahead Data Collection 10s After Completed Startup. May 06 18:57:31 fdi systemd[1]: Starting Stop Read-Ahead Data Collection 10s After Completed Startup. May 06 18:57:31 fdi systemd[1]: Started Update UTMP about System Runlevel Changes. May 06 18:57:31 fdi systemd[1]: Startup finished in 2.378s (kernel) + 1.926s (initrd) + 11.088s (userspace) = 15.393s. May 06 18:57:31 fdi /usr/bin/discovery-menu[1108]: Kernel opts: initrd=boot/fdi-image-rhel_7-img rootflags=loop root=live:/fdi.iso rootfstype=auto ro rd.live.image acpi May 06 18:57:31 fdi /usr/bin/discovery-menu[1108]: Entering screen_countdown May 06 18:57:32 fdi NetworkManager[790]: <info> (eno1): link connected May 06 18:57:32 fdi NetworkManager[790]: <info> (eno1): device state change: unavailable -> disconnected (reason 'carrier-changed') [20 30 40] May 06 18:57:32 fdi kernel: e1000e: eno1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None May 06 18:57:32 fdi kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eno1: link becomes ready May 06 18:57:32 fdi NetworkManager[790]: <info> Auto-activating connection 'primary'. May 06 18:57:32 fdi NetworkManager[790]: <info> (eno1): Activation: starting connection 'primary' (5e84d9cc-13bc-11e6-8ce6-eca86bf29f9f) May 06 18:57:32 fdi NetworkManager[790]: <info> (eno1): device state change: disconnected -> prepare (reason 'none') [30 40 0] May 06 18:57:32 fdi NetworkManager[790]: <info> NetworkManager state is now CONNECTING
Analysis: The proxy unit is missing After=basic.target network-online.target rule and can occassionally start when network is not yet online. The proxy certificate generator requires a valid IP address. For this reason, it fails to generate HTTPS cert therefore the service fails to start. This was fixed in 6.2 BETA already. 0c18ba2a6d04e5105db1e2085fe69f091b6922c7 Workaround: Use fdi image from 6.2 BETA repositories, upgrade it, restart. Triage notes: We can backport this patch into 6.1 repository if needed.
@Lzap: Could you please provide steps to verify this bz. In general I can discover host without any issue but just wanted to see if any extra verification required. thanks
One idea is: turn DHCP service off, start FDI, wait a bit and enable it. But you must not keep it turned off for more than 45 seconds, otherwise FDI will timeout. https://bugzilla.redhat.com/show_bug.cgi?id=1262922
Thanks Lukas. I tried to reproduce the issue with suggested steps in comment6 using sat62 GA snap19. I turned off dhcpd and started the FDI and later switched it on. Host was registered successfully and foreman-proxy service was active. Later, I tried FDI TUI, where I selected the "Discover via DHCP" and service was down for some time and then I restarted the dhcpd and host get the IP and discovered successfully. In both cases, I see foreman-proxy service in active state. thanks
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2016:1501