Bug 1628599

Summary: very slow network configuration after upgrade to kernel-core >= 4.18.5-200.fc28.x86_64
Product: [Fedora] Fedora Reporter: Pavel Raiskup <praiskup>
Component: cloud-initAssignee: Garrett Holmstrom <gholms>
Status: CLOSED DUPLICATE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 28CC: adimania, apevec, eterrell, gholms, jlebon, lars, shardy, s
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-09-14 05:30:04 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
full diff none

Description Pavel Raiskup 2018-09-13 14:07:56 UTC 
In cloud-init.log I see:
2018-09-13 10:54:24,723 - __init__.py[DEBUG]: Looking for for data source in: ['NoCloud', 'ConfigDrive', 'OpenNebula', 'DigitalOcean', 'Azure', 'AltCloud', 'OVF', 'MAAS', 'GCE', 'OpenStack', 'AliYun', 'Ec2', 'CloudSigma', 'CloudStack', 'SmartOS', 'Bigstep', 'Scaleway', 'None'], via packages ['', 'cloudinit.sources'] that matches dependencies ['FILESYSTEM']
2018-09-13 11:17:14,109 - __init__.py[DEBUG]: Searching for local data source in: ['DataSourceNoCloud', 'DataSourceConfigDrive', 'D

See the gap between the two messages (23 minutes spent in list_sources()
method from cloud-init-17.1/cloudinit/sources/__init__.py).

Is there any easy way to debug such issue?
Comment 1 Jonathan Lebon 2018-09-13 14:10:57 UTC 
Likely dupe of #1572944.

> Is there any easy way to debug such issue?

Try `systemd-analyze blame | head`, e.g. see https://bugzilla.redhat.com/show_bug.cgi?id=1572944#c10.
Comment 2 Jonathan Lebon 2018-09-13 14:12:54 UTC 
(Well "dupe" might be wrong. The bug was initially opened against an older kernel, but judging from recent comments mentioning the same kernel versions as here, it looks like it might have regressed but AFAIK no new bugs have been created yet.)
Comment 3 Pavel Raiskup 2018-09-13 14:13:07 UTC 
Created attachment 1483069 [details]
full diff

I have working f28 and non-working f28 image, while these changes
might be related:

-selinux-policy-3.14.1-40.fc28.noarch
-selinux-policy-targeted-3.14.1-40.fc28.noarch
+selinux-policy-3.14.1-42.fc28.noarch
+selinux-policy-targeted-3.14.1-42.fc28.noarch

Full diff is attached.
Comment 4 Pavel Raiskup 2018-09-13 14:27:28 UTC 
# systemd-analyze blame | head
    4min 30.503s cloud-init-local.service
    4min 30.221s sshd-keygen
    4min 30.142s sshd-keygen
    4min 30.136s sshd-keygen
          8.866s cloud-init.service
           897ms initrd-switch-root.service
           526ms cloud-final.service
           505ms cloud-config.service
           241ms dnf-makecache.service
           216ms NetworkManager-wait-online.service

This is another box, which started faster (the gap was 12:37:47 -> 12:42:16).
Comment 5 Pavel Raiskup 2018-09-13 17:41:57 UTC 
Thanks Jonathan, you helped me a lot in the end - I used:

$ glance image-update --property hw_rng_model=virtio <THE_IMAGE>

... and the VMs boot fine again.
Comment 6 Pavel Raiskup 2018-09-14 05:30:04 UTC 

*** This bug has been marked as a duplicate of bug 1572944 ***