Bug 1157776
| Summary: | ovs failures while setting gateway for a router | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Marko Myllynen <myllynen> | ||||
| Component: | openstack-neutron | Assignee: | Nir Magnezi <nmagnezi> | ||||
| Status: | CLOSED DUPLICATE | QA Contact: | Ofer Blaut <oblaut> | ||||
| Severity: | unspecified | Docs Contact: | |||||
| Priority: | unspecified | ||||||
| Version: | 6.0 (Juno) | CC: | chrisw, myllynen, nyechiel, yeylon | ||||
| Target Milestone: | --- | Keywords: | ZStream | ||||
| Target Release: | 6.0 (Juno) | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2015-05-07 12:48:14 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Attachments: |
|
||||||
Can you please attach full logs of the following: 1. neutron server 2. neutron l2 agent (ovs agent) 3. neutron l3 agent I reproduced this on OSP 6 and I'm attaching a tar ball containing all the requested logs from the ctrl/network node and two compute nodes. After installation I booted up the machines around 14:10 local time and then did the above steps listed in comment 0 starting around 14:15. On OSP 6 I also see these kinds of messages being flooded to logs: 2015-04-08T11:17:09.329Z|00003|dpif(handler_5)|WARN|system@ovs-system: failed to put[create][modify] (No such file or directory) key20(bad key length 4, expecte d -1)(00 00 00 00/(bad mask length 0, expected -1)(),key19(bad key length 4, exp ected -1)(00 00 00 00/(bad mask length 0, expected -1)(),skb_priority(0),in_port (1),skb_mark(0/0),eth(src=1e:b3:ff:f5:89:4c,dst=76:1b:43:d7:fc:4e),eth_type(0x08 00),ipv4(src=192.168.122.101/0.0.0.0,dst=192.168.122.103/0.0.0.0,proto=6/0,tos=0 /0,ttl=64/0,frag=no/0xff),tcp(src=5672/0,dst=37345/0),tcp_flags(0x018/0x000), ac tions:2 Thanks. Created attachment 1012174 [details]
osp-logs.tar
I managed to reproduce this with the latest upstream code. The error is triggered each time an interface moves across namespaces. Hence, other scenarios such as a creation of a dhcp port for a subnet will produce the same error. There's a broader issue with ovs here, which is fully described in bug #1160340 comment #3. In short: The net interface disappeared from the namespace where ovs-vswitchd is running. Due to the fact that ovs-vswitchd is unaware of namespaces, it just cannot access it anymore. Therefore it produces those warnings. According to Jiri, The kernel part of ovs is somehow namespace-aware so the packets keep flowing even after the interface is moved to a different namespace. *** This bug has been marked as a duplicate of bug 1160340 *** |
Description of problem: When configuring networks in a br-ex using setup with the stanzas below I see failures in /var/log/openvswitch/ovs-vswitchd.log (neutron output omitted for brevity): # . /root/keystonerc_admin # tid=$(keystone tenant-list | awk '/services/ {print $2}') # neutron net-create --tenant-id $tid --router:external=true net-ext0 # neutron subnet-create --tenant-id $tid --name subnet-ext0 --disable-dhcp --gateway 192.168.122.1 --allocation-pool start=192.168.122.200,end=192.168.122.254 net-ext0 192.168.122.0/24 # . /root/keystonerc_demo # neutron net-create net-demo0 # neutron subnet-create --name subnet-demo0 --enable-dhcp --gateway 192.168.1.1 --allocation-pool start=192.168.1.2,end=192.168.1.254 net-demo0 192.168.1.0/24 # neutron router-create router-demo0 # . /root/keystonerc_admin # neutron router-gateway-set router-demo0 net-ext0 # tail -n 3 /var/log/openvswitch/ovs-vswitchd.log 2014-10-27T16:05:46.537Z|00033|bridge|INFO|bridge br-ex: added interface qg-eb8b8a0d-42 on port 2 2014-10-27T16:05:47.413Z|00034|netdev_linux|INFO|ioctl(SIOCGIFHWADDR) on qg-eb8b8a0d-42 device failed: No such device 2014-10-27T16:05:51.254Z|00035|netdev_linux|WARN|ioctl(SIOCGIFINDEX) on qg-eb8b8a0d-42 device failed: No such device # ovs-vsctl list-ports br-ex eth0 qg-eb8b8a0d-42 And a bit later: # . /root/keystonerc_demo # neutron router-interface-add router-demo0 subnet-demo0 # tail -n 7 /var/log/openvswitch/ovs-vswitchd.log 2014-10-27T16:39:56.153Z|00036|bridge|INFO|bridge br-int: added interface qr-920e5c0b-62 on port 2 2014-10-27T16:39:56.153Z|00037|netdev_linux|WARN|qg-eb8b8a0d-42: removing policing failed: No such device 2014-10-27T16:39:56.693Z|00038|netdev_linux|INFO|ioctl(SIOCGIFHWADDR) on qr-920e5c0b-62 device failed: No such device 2014-10-27T16:39:59.569Z|00039|netdev_linux|WARN|ioctl(SIOCGIFINDEX) on qr-920e5c0b-62 device failed: No such device 2014-10-27T16:39:59.569Z|00040|netdev_linux|WARN|qr-920e5c0b-62: removing policing failed: No such device 2014-10-27T16:40:09.040Z|00041|ofproto|INFO|br-tun: 2 flow_mods 10 s ago (1 adds, 1 modifications) 2014-10-27T16:40:09.843Z|00042|ofproto|INFO|br-int: 1 flow_mods 10 s ago (1 deletes) Later on everything seems to work properly despite of the above. If the failures are expected perhaps they should be logged less prominently. Version-Release number of selected component (if applicable): openstack-neutron-2014.1.2-2.el7ost.noarch