Bug 2481466 - systemd-imds-early-network.service fails on AWS EC2 - no IMDS auto-detection
Summary: systemd-imds-early-network.service fails on AWS EC2 - no IMDS auto-detection
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: systemd
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
Assignee: systemd-maint
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2026-05-26 14:20 UTC by Petr Sklenar
Modified: 2026-05-26 22:50 UTC (History)
10 users (show)

Fixed In Version: systemd-261~rc2-1.fc45
Clone Of:
Environment:
Last Closed: 2026-05-26 22:50:28 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
journalctl (1.06 MB, text/plain)
2026-05-26 15:08 UTC, Petr Sklenar
no flags Details

Description Petr Sklenar 2026-05-26 14:20:34 UTC
Description of problem:
The systemd-imds-early-network.service fails on AWS EC2 instances with error "No IMDS endpoint information provided or detected, cannot operate." The service expects systemd credentials (imds.vendor, imds.token_header_name, etc.) to be provided by bootloader/initrd, but these are not available on AWS EC2. The IMDS endpoint (http://169.254.169.254/) is accessible and the system correctly detects "Amazon EC2" via DMI (/sys/class/dmi/id/sys_vendor), but systemd-imdsd does not auto-detect the cloud platform.

Version-Release number of selected component:
systemd-261~rc1-1.fc45.x86_64
Fedora Linux 45 (Cloud Edition Prerelease)
Kernel: 7.1.0-0.rc5.35.fc45.x86_64

How reproducible:
Always (100%)

Steps to Reproduce:
1. Launch Fedora 45 Cloud Edition on AWS EC2 (tested on c6a.large)
2. Boot the system
3. Run: systemctl status systemd-imds-early-network.service

Actual Results:
Service fails with:
```
× systemd-imds-early-network.service - Enable Pre-IMDS Networking
     Active: failed (Result: exit-code)
   Main PID: 473 (code=exited, status=1/FAILURE)

May 26 14:08:42 systemd-imdsd[473]: Failed to read credential 'imds.vendor', ignoring: No such device or address
May 26 14:08:42 systemd-imdsd[473]: Failed to read credential 'imds.data_url', ignoring: No such device or address
[...60+ similar credential failures...]
May 26 14:08:42 systemd-imdsd[473]: No IMDS endpoint information provided or detected, cannot operate.
May 26 14:08:42 systemd[1]: systemd-imds-early-network.service: Failed with result 'exit-code'.
```

Expected Results:
Service should auto-detect AWS EC2 and successfully configure IMDS networking, OR gracefully skip if not needed without showing as failed.

Additional info:
- IMDS endpoint IS accessible: curl -s http://169.254.169.254/latest/meta-data/ works correctly
- DMI detection works: /sys/class/dmi/id/sys_vendor shows "Amazon EC2"
- Instance type: c6a.large
- systemctl --all --failed shows this as the only failed service

Root cause analysis:
systemd-imdsd requires systemd credentials to be set (imds.vendor, imds.token_header_name, imds.data_url, etc.) but:
1. No component in the boot chain sets these credentials on AWS EC2
2. systemd-imdsd lacks fallback auto-detection for common cloud platforms
3. The service is marked as "enabled-runtime" but has no mechanism to detect if it's actually needed

Suggested fixes:
1. Add auto-detection for AWS/Azure/GCP when credentials are missing (check DMI vendor string)
2. Have cloud-init or systemd-sysext provide credentials on cloud platforms
3. Change service to not fail if IMDS is genuinely unavailable (ConditionVirtualization= or similar)

Impact:
Visual clutter in systemctl --failed, but no functional impact observed - networking works correctly.

Comment 1 Petr Sklenar 2026-05-26 14:23:22 UTC
to reproduce:
https://fedoraproject.org/wiki/Test_Results:Fedora_45_Rawhide_20260523.n.0_Cloud#Amazon_EC2_(via_Testing_Farm)

with public api key:
testing-farm reserve --compose Fedora-Cloud-Base-AmazonEC2.x86_64-Rawhide-20260523.0  --arch x86_64

Comment 3 Frantisek Sumsal 2026-05-26 14:45:12 UTC
This is a duplicate of https://bugzilla.redhat.com/show_bug.cgi?id=2481466 and is already being resolved in upstream by https://github.com/systemd/systemd/pull/42308.

*** This bug has been marked as a duplicate of bug 2481304 ***

Comment 4 Frantisek Sumsal 2026-05-26 14:50:08 UTC
On second thought, the fail is not about an ordering cycle (at least from the snippet you sent) like the other issue I mentioned. Could you, please, provide a full journal from the machine where the unit fails?

Comment 5 Petr Sklenar 2026-05-26 15:08:29 UTC
Created attachment 2142962 [details]
journalctl

Comment 6 Petr Sklenar 2026-05-26 15:09:50 UTC
Comment on attachment 2142962 [details]
journalctl

May 26 15:04:06 ip-172-31-17-43.us-east-2.compute.internal systemd-imdsd[476]: Failed to read credential 'imds.vendor', ignoring: No such device or address

Comment 7 Frantisek Sumsal 2026-05-26 15:26:00 UTC
So, there _is_ the ordering cycle from https://bugzilla.redhat.com/show_bug.cgi?id=2481304 in the journal:

May 26 15:04:06 ip-172-31-17-43.us-east-2.compute.internal systemd[1]: sysinit.target: Found ordering cycle: systemd-imds-import.service/start after network-online.target/start after cloud-init-network.service/start after basic.target/start after sockets.target/start after sssd-kcm.socket/start after sysinit.target/start - after systemd-imds-import.service
May 26 15:04:06 ip-172-31-17-43.us-east-2.compute.internal systemd[1]: sysinit.target: Job systemd-imds-import.service/start deleted to break ordering cycle starting with sysinit.target/start

But the fail in systemd-imds-early-network.service is most likely a different issue:

...
May 26 15:04:06 ip-172-31-17-43.us-east-2.compute.internal systemd-imdsd[476]: Failed to read credential 'imds.key_userdata', ignoring: No such device or address
May 26 15:04:06 ip-172-31-17-43.us-east-2.compute.internal systemd-imdsd[476]: Failed to read credential 'imds.key_userdata_base', ignoring: No such device or address
May 26 15:04:06 ip-172-31-17-43.us-east-2.compute.internal systemd-imdsd[476]: Failed to read credential 'imds.key_userdata_base64', ignoring: No such device or address
May 26 15:04:06 ip-172-31-17-43.us-east-2.compute.internal systemd-imdsd[476]: No IMDS endpoint information provided or detected, cannot operate.
May 26 15:04:06 ip-172-31-17-43.us-east-2.compute.internal systemd[1]: Mounted sys-kernel-tracing.mount - Kernel Trace File System.
May 26 15:04:06 ip-172-31-17-43.us-east-2.compute.internal systemd[1]: Finished kmod-static-nodes.service - Create List of Static Device Nodes.
May 26 15:04:06 ip-172-31-17-43.us-east-2.compute.internal systemd[1]: Finished lvm2-monitor.service - Monitoring of LVM2 mirrors, snapshots etc. using dmeventd or progress polling.
May 26 15:04:06 ip-172-31-17-43.us-east-2.compute.internal systemd[1]: Mounted sys-fs-fuse-connections.mount - FUSE Control File System.
May 26 15:04:06 ip-172-31-17-43.us-east-2.compute.internal systemd[1]: systemd-imds-early-network.service: Main process exited, code=exited, status=1/FAILURE
May 26 15:04:06 ip-172-31-17-43.us-east-2.compute.internal systemd[1]: systemd-imds-early-network.service: Failed with result 'exit-code'.
May 26 15:04:06 ip-172-31-17-43.us-east-2.compute.internal systemd[1]: Failed to start systemd-imds-early-network.service - Enable Pre-IMDS Networking.
...

Comment 8 Zbigniew Jędrzejewski-Szmek 2026-05-26 16:28:57 UTC
https://github.com/systemd/systemd/pull/42318 is untested, but I think it should DTRT.

Comment 9 Fedora Update System 2026-05-26 21:34:18 UTC
FEDORA-2026-d51e462be4 (systemd-261~rc2-1.fc45) has been submitted as an update to Fedora 45.
https://bodhi.fedoraproject.org/updates/FEDORA-2026-d51e462be4

Comment 10 Fedora Update System 2026-05-26 22:50:28 UTC
FEDORA-2026-d51e462be4 (systemd-261~rc2-1.fc45) has been pushed to the Fedora 45 stable repository.
If problem still persists, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.