Bug 1194623
| Summary: | Bringing up the network on the cloud image is racy | ||||||
|---|---|---|---|---|---|---|---|
| Product: | [Fedora] Fedora | Reporter: | Sitsofe Wheeler <sitsofe> | ||||
| Component: | initscripts | Assignee: | Lukáš Nykrýn <lnykryn> | ||||
| Status: | CLOSED EOL | QA Contact: | Fedora Extras Quality Assurance <extras-qa> | ||||
| Severity: | unspecified | Docs Contact: | |||||
| Priority: | unspecified | ||||||
| Version: | 21 | CC: | dustymabe, jonathan, jzb, lnykryn, vpavlin, zbyszek | ||||
| Target Milestone: | --- | ||||||
| Target Release: | --- | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2015-12-02 09:16:58 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Attachments: |
|
||||||
|
Description
Sitsofe Wheeler
2015-02-20 11:58:18 UTC
Created attachment 993873 [details]
Attaching journal output for problem boot
Yep, this is common issue. But unfortunately there is nothing we can do, problem is that those device will appear after network initscript is run and this can't be fixed through ordering because we don't know that this device will eventually appear. There are some workarounds: 1) DEVTIMEOUT= this will tell initscripts to give that device some times to appear instead of priniting "Device eth0 does not seem to be present" 2) Don't use pasive network-scripts NetworkManager or networkd can react to events which happens after the boot of the system. 3) Write some custom udev rule You can set onboot=no for that device and write a udev rule which will run ifup on that device after it appears. In the past (and in rhel6), initscript had and hotplug script which was able to handle udev events after the system was boot up, but it was extremely racy and we have dropped it in favour of NM. Lukáš: I see - perhaps there could be a release note mentioning this explicitly for cloud images? I've been playing with DEVTIMEOUT=2 in /etc/sysconfig/network and it seems to help but I need the patch from Bug #1180837 for it to work. Could a "solution" be to load the network card drivers forcefully at initramfs time so the devices are given the best possible chance of existing before the initscript runs? This could be done only for cloud installs... CC'ing jzb. > Could a "solution" be to load the network card drivers forcefully at > initramfs time so the devices are given the best possible chance of existing > before the initscript runs? This could be done only for cloud installs... > CC'ing jzb. No, solution for this is not to use network-scripts at all. Even in rhel we recommend users to use them only for legacy stuff https://access.redhat.com/solutions/783533 If you want to have something small in your images just give a shot to networkd. It is installed anyway as a part of systemd package and it is really easy to configure. FOr basic setups you probably just need something like /etc/systemd/network/my.network [Match] Name=eth* [Network] DHCP=both But anyway I have build initscripts-9.56.1-7.fc21 with the mentioned patch Lukáš: OK what you're saying sounds pretty conclusive - initscripts should not be used on new systems for network bring up (unless you are in one the few situations mentioned in https://access.redhat.com/solutions/783533). jzb: Going forwards can Fedora cloud images stop defaulting to the use of network-scripts for network bring up and instead switch to networkd? There could be one hiccup with regards to using systemd-networkd. While not a common situation it is possible to statically configure networking using the network-interfaces directive of cloud-init [1]. Doing this populates the files in the network-scripts directory which is not compatible with systemd-networkd. This is something that should be considered before making such a move. [1] - http://cloudinit.readthedocs.org/en/latest/topics/datasources.html My plan is to write a generator which would convert ifcfg file to networkd configuration, so that should solve your problem. Lukas is there a tracker for this work that we can follow? Sitsofe, looks like work for systemd-networkd in cloud-init,etc.. won't be ready for Fedora 22. Have you managed to workaround at all? If so what are your workarounds? A few more questions: Does this only happen on Hyper-V and VMWare Fusion? Other options are KVM/Xen/VirtualBox. If you soft reboot the VM does networking come up fine on the 2nd try? Dusty: I have managed to workaround it by using the patch mentioned in #1180837 and adding DEVTIMEOUT=2 to /etc/sysconfig/network . DEVTIMEOUT=1 was enough to solve the problem on VMware Fusion but there were still issues on Hyper-V that went away with DEVTIMEOUT=2. I don't know if this happens on KVM/Xen/VirtualBox as my machines are currently all Hyper-V and VMware/ESXi based. If I soft reboot sometimes the networking comes up and sometimes it doesn't. This issue happens frequently (one in five - ten times) but you can boot from cold and things are fine then soft reboot and end up without networking. Likewise you can boot from cold, hit this problem and have no networking, reboot from within the VM and things are fine. I think for now a note saying "This is what can happen, workaround by doing this or that" should suffice. I suspect it only impacts people who have a comparatively fast and empty boot - if enough concurrent services are starting such that the networking script is always delayed by even a few tenths of a second I can well believe you will never see this issue. (In reply to Lukáš Nykrýn from comment #7) > My plan is to write a generator which would convert ifcfg file to networkd > configuration, so that should solve your problem. Hey Lukas, Did you make any progress on this? Is there a tracker for the work? https://github.com/lnykryn/ifcfg-generator But it is not usable yet, it supports only basic ethernet and I have just write a code and did not test it. (In reply to Lukáš Nykrýn from comment #11) > https://github.com/lnykryn/ifcfg-generator > But it is not usable yet, it supports only basic ethernet and I have just > write a code and did not test it. Progress! That is awesome. Do you think there is any chance we could have something in place for F23? We'd like to consider using systemd-networkd as the default for the F23 cloud image. This issue popped up again a month after I filed this but in a separate bug (Bug #1204612) and a default workaround was established there (cloud images from Fedora 22 default to DEVTIMEOUT=10). Adding NEEDINFO for comment #12 . This message is a reminder that Fedora 21 is nearing its end of life. Approximately 4 (four) weeks from now Fedora will stop maintaining and issuing updates for Fedora 21. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a Fedora 'version' of '21'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 21 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete. I've moved over to systemd-networkd which can react to network devices appearing so this is no longer an issue for me after customisation. Fedora 21 changed to end-of-life (EOL) status on 2015-12-01. Fedora 21 is no longer maintained, which means that it will not receive any further security or bug fix updates. As a result we are closing this bug. If you can reproduce this bug against a currently maintained version of Fedora please feel free to reopen this bug against that version. If you are unable to reopen this bug, please file a new report against the current release. If you experience problems, please add a comment to this bug. Thank you for reporting this bug and we are sorry it could not be fixed. |