Bug 974222

Summary: Cloud-init various fails obtain instance-id
Product: Red Hat OpenStack Reporter: Jaroslav Henner <jhenner>
Component: distributionAssignee: RHOS Maint <rhos-maint>
Status: CLOSED WORKSFORME QA Contact: Ami Jeain <ajeain>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 3.0CC: markmc, yeylon
Target Milestone: ---   
Target Release: 4.0   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-06-13 19:11:14 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Jaroslav Henner 2013-06-13 17:35:08 UTC
Description of problem:
Cloud-init blocks the booting of instance about 2 minutes, spitting following errors to log.
20130613 12:59:00,910  url_helper.py[WARNING]: Calling 'http://169.254.169.254/20090404/metadata/instanceid' failed [3/120s]: url error [[Errno 113] No route to host

note there is a bug about missing dashes from log

20130613 12:59:00,910  url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [3/120s]: url error [[Errno 113] No route to host

Note that after login is available, there is no route to 169.254.169.254 in ip r, but curl http://169.254.169.254/2009-04-04/meta-data/instance-id returns the instance id, no problem. This is because the default route will deliver the packets to the router, where the port gets translated to some quantum-proxy port, which should proxy it to nova.



Version-Release number of selected component (if applicable):
cloud-init-0.7.1-2.el6.noarch
openstack-quantum-2013.1.1-10.el6ost.noarch
openstack-nova-common-2013.1.1-4.el6ost.noarch

How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 2 Jaroslav Henner 2013-06-13 17:40:49 UTC
Also note that during the time the cloud-init has problems, claiming "no route to host", the instance is pingable.

Comment 4 Jaroslav Henner 2013-06-13 18:06:19 UTC
The problem is that cloud-init adds temporary routing rule, saying that the 169.254.0.0/16 is local, therefore the packets are not routed to gateway from the sending machine, but ARP requests are being sent:

13:53:02.634932 ARP, Request who-has 169.254.169.254 tell 10.34.64.7, length 28

Nothing replies since there is no such listener (there is only NAT rule on the controller). The sender than realizes there is no route to host.

I believe this problem is not present when using nova-network, but I would need confirmation.

Comment 5 Jaroslav Henner 2013-06-13 19:11:14 UTC
I think I had some problem in my deployment.