Bug 1364079
| Summary: | iPXE hangs with an infinite stream of different errors | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Dmitry Tantsur <dtantsur> | ||||||||
| Component: | ipxe | Assignee: | RHOS Maint <rhos-maint> | ||||||||
| Status: | CLOSED WORKSFORME | QA Contact: | Shai Revivo <srevivo> | ||||||||
| Severity: | unspecified | Docs Contact: | |||||||||
| Priority: | unspecified | ||||||||||
| Version: | 9.0 (Mitaka) | CC: | apevec, dtrainor, lhh, rlandy, srevivo | ||||||||
| Target Milestone: | --- | Keywords: | ZStream | ||||||||
| Target Release: | --- | ||||||||||
| Hardware: | Unspecified | ||||||||||
| OS: | Unspecified | ||||||||||
| Whiteboard: | |||||||||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |||||||||
| Doc Text: | Story Points: | --- | |||||||||
| Clone Of: | Environment: | ||||||||||
| Last Closed: | 2016-08-05 08:05:50 UTC | Type: | Bug | ||||||||
| Regression: | --- | Mount Type: | --- | ||||||||
| Documentation: | --- | CRM: | |||||||||
| Verified Versions: | Category: | --- | |||||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||||
| Embargoed: | |||||||||||
| Attachments: |
|
||||||||||
|
Description
Dmitry Tantsur
2016-08-04 12:49:05 UTC
Created attachment 1187475 [details]
iPXE versions as seen by the machine
We are seeing similar errors and hangs with introspection in CI jobs (happening on OVB and real baremetal hardware) - but only with master jobs - mitaka jobs are passing atm. The job logs show: Introspection completed with errors: 2a296b82-47d2-446f-961a-e8f34e0b21ea: Introspection timeout 678319e5-a270-49af-ab2f-f88aa52b9f7c: Introspection timeout ca5e6f69-32e8-4580-88ba-602685a3a2f1: Introspection timeout ad54e211-f5ad-41cf-833a-10a556f2181a: Introspection timeout Setting nodes for introspection to manageable... Starting introspection of manageable nodes Waiting for introspection to finish... Introspection for UUID 2a296b82-47d2-446f-961a-e8f34e0b21ea finished with error: Introspection timeout Introspection for UUID 678319e5-a270-49af-ab2f-f88aa52b9f7c finished with error: Introspection timeout Introspection for UUID ca5e6f69-32e8-4580-88ba-602685a3a2f1 finished with error: Introspection timeout Introspection for UUID ad54e211-f5ad-41cf-833a-10a556f2181a finished with error: Introspection timeout No nodes in manageable state found for introspection Looking at the console, I see repeated messages: 'Connection timeout...' Attaching screenshot of console Created attachment 1187565 [details]
Screenshot of OVB instance console during introspection
I'm sorry for the noise, my problem was due to some missing steps ended up in missing images. Ronelle, please double-check that the agent image is actually present in /httpboot. Dmitry, I checked the undercloud: [stack@undercloud ~]$ ls -la /httpboot total 339516 drwxr-xr-x. 2 ironic ironic 66 Aug 4 15:08 . dr-xr-xr-x. 19 root root 4096 Aug 4 14:50 .. -rwxr-xr-x. 1 root root 5158704 Aug 4 15:08 agent.kernel -rw-r--r--. 1 root root 342493754 Aug 4 15:08 agent.ramdisk -rw-r--r--. 1 ironic ironic 465 Aug 4 14:52 inspector.ipxe Watching the console during introspection, it gets inspector.ipxe just fine but times out on agent.kernel OK - I think this is due to the MTU values of the interface used for provisioning being set back to 1500 during undercloud install. This overwrites the value set in the OVB setup. Resetting value before introspection ... under test Confirmed it was an MTU issue. Thanks, so I'm closing it. Do you think we could update some documentation mentioning this potential MTU issue? The instructions to modify MTU are doc'ed. (and I did have those modifications made). The issue is that undercloud-install overwrites them - that could possibly be doc'ed. It's only an issue with OVB and some hardware platforms |