Bug 2040360

Created attachment 1850604 [details]
System Log from AWS

1. Please describe the problem:
After enabling enhanced networking in an AWS m4.large instance, the networking seems to stop working.   From CI runs, this does not seem to affect m5.large instances.

This bug was originally found in Fedora CoreOS which has enhanced networking enabled by default which preventing the installation of Fedora CoreOS.   See https://github.com/coreos/fedora-coreos-tracker/issues/1066


2. What is the Version-Release number of the kernel:

kernel-core-5.15.13-200.fc35.x86_64

3. Did it work previously in Fedora? If so, what kernel version did the issue
   *first* appear?  Old kernels are available for download at
   https://koji.fedoraproject.org/koji/packageinfo?packageID=8 :

From the CI runs in Fedora CoreOS, it was working on `5.15.10-200.fc35.x86_64` and stopped working on `5.15.10-200.fc35.x86_64`.

There were two CVEs fixed between these versions that seem to be related to networking:
- https://bugzilla.redhat.com/show_bug.cgi?id=2031199
- https://bugzilla.redhat.com/show_bug.cgi?id=2034940


4. Can you reproduce this issue? If so, please provide the steps to reproduce
   the issue below:

This problem happens intermittently.  It is sometimes successful but most of the time it does not.

- Launch AMI ami-08b4ee602f76bff79 with m4.large instance type on us-east-1 in AWS (latest from https://alt.fedoraproject.org/cloud/)
- SSH to instance and run `dnf -y update`
- Reboot and verify that SSH login to the instance is still possible.
- Stop the instance in AWS
- Run `aws ec2 modify-instance-attribute --instance-id <instance-id> --sriov-net-support simple` to enable enhance networking
- Start the instance in AWS
- Login is no longer possible through SSH

Note: There is no way to disable enhanced networking once it is enabled in AWS.

5. Does this problem occur with the latest Rawhide kernel? To install the
   Rawhide kernel, run ``sudo dnf install fedora-repos-rawhide`` followed by
   ``sudo dnf update --enablerepo=rawhide kernel``:

From testing it seems like this can be reproduced ~50%-60% of the time in the rawhide kernel


6. Are you running any modules that not shipped with directly Fedora's kernel?:

No

7. Please attach the kernel logs. You can get the complete kernel log
   for a boot with ``journalctl --no-hostname -k > dmesg.txt``. If the
   issue occurred on a previous boot, use the journalctl ``-b`` flag.

Since networking is down after starting, I can't run any additional commands.  I only have the system log from AWS which is attached.




Additional Notes:

The difference between enhanced networking and out of the box Fedora is the network driver--ixgbevf and vif, respectively.

Original Issue:
https://github.com/coreos/fedora-coreos-tracker/issues/1066