Bug 1163783
| Summary: | HA | Neutron | Messaging error,openswitch agent stops working. | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Leonid Natapov <lnatapov> | ||||
| Component: | rhel-osp-installer | Assignee: | Mike Burns <mburns> | ||||
| Status: | CLOSED EOL | QA Contact: | Omri Hochman <ohochman> | ||||
| Severity: | high | Docs Contact: | |||||
| Priority: | unspecified | ||||||
| Version: | 5.0 (RHEL 7) | CC: | majopela, mark, oblaut, rhos-maint, sputhenp, srevivo, stuart.stent | ||||
| Target Milestone: | --- | Keywords: | ZStream | ||||
| Target Release: | Installer | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2016-09-29 13:45:43 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Bug Depends On: | 1129242 | ||||||
| Bug Blocks: | |||||||
| Attachments: |
|
||||||
I wonder if this is related either to: 1) several active neutron-servers 2) a messaging/HA problem, agent tries to attach to a fanout queue it's not there in previous messages (not posted in this log part) 3) a problem with neutron-server I will examine the logs ASAP. Also seeing the same problem in our deployment HA + VXLAN Closing list of bugs for RHEL OSP Installer since its support cycle has already ended [0]. If there is some bug closed by mistake, feel free to re-open. For new deployments, please, use RHOSP director (starting with version 7). -- Jaromir Coufal -- Sr. Product Manager -- Red Hat OpenStack Platform [0] https://access.redhat.com/support/policy/updates/openstack/platform |
Created attachment 957171 [details] logs I have an HA+Neutron VXLAN deployment. After some time,I get a lot of messaging errors and openswitch agent stops working. Restart openswitch agent solves the problem. openswitch-agent.log attached ,neutron-server logs form the controllers and rabbitmq logs. ----------------------- openstack-puppet-modules-2014.1-24.el6ost.noarch rhel-osp-installer-0.4.7-1.el6ost.noarch ruby193-rubygem-foreman_openstack_simplify-0.0.6-8.el6ost.noarch openstack-foreman-installer-2.0.32-1.el6ost.noarch ---------------------------- python-neutronclient-2.3.4-3.el7ost.noarch openstack-neutron-2014.1.3-7.el7ost.noarch openstack-neutron-openvswitch-2014.1.3-7.el7ost.noarch python-neutron-2014.1.3-7.el7ost.noarch ------------------------------------------------ 2014-11-13 07:17:48.704 2313 INFO neutron.plugins.openvswitch.agent.ovs_neutron_agent [-] Agent out of sync with plugin! 2014-11-13 07:17:48.952 2313 INFO neutron.agent.securitygroups_rpc [-] Preparing filters for devices set([u'78b0999e-a4b2-4e4c-ae38-37e43c48bdd7', u'36ccd710-ab1a-485d-b70b-6f57009562f2', u'139f0ef f-c4e6-43ca-88e2-e91a233c41f3', u'1bd4464c-6e26-452b-abed-51512e9fd2c9']) 2014-11-13 07:18:48.969 2313 ERROR neutron.plugins.openvswitch.agent.ovs_neutron_agent [-] Error while processing VIF ports 2014-11-13 07:18:48.969 2313 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent Traceback (most recent call last): 2014-11-13 07:18:48.969 2313 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent File "/usr/lib/python2.7/site-packages/neutron/plugins/openvswitch/agent/ovs_neutron_agent.py", line 1335, i n rpc_loop 2014-11-13 07:18:48.969 2313 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent ovs_restarted) 2014-11-13 07:18:48.969 2313 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent File "/usr/lib/python2.7/site-packages/neutron/plugins/openvswitch/agent/ovs_neutron_agent.py", line 1139, i n process_network_ports 2014-11-13 07:18:48.969 2313 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent port_info.get('updated', set())) 2014-11-13 07:18:48.969 2313 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent File "/usr/lib/python2.7/site-packages/neutron/agent/securitygroups_rpc.py", line 247, in setup_port_filters 2014-11-13 07:18:48.969 2313 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent self.prepare_devices_filter(new_devices) 2014-11-13 07:18:48.969 2313 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent File "/usr/lib/python2.7/site-packages/neutron/agent/securitygroups_rpc.py", line 161, in prepare_devices_fi lter 2014-11-13 07:18:48.969 2313 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent self.context, list(device_ids)) 2014-11-13 07:18:48.969 2313 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent File "/usr/lib/python2.7/site-packages/neutron/agent/securitygroups_rpc.py", line 86, in security_group_rule s_for_devices 2014-11-13 07:18:48.969 2313 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent topic=self.topic) 2014-11-13 07:18:48.969 2313 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/proxy.py", line 129, in call 2014-11-13 07:18:48.969 2313 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent exc.info, real_topic, msg.get('method')) 2014-11-13 07:18:48.969 2313 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent Timeout: Timeout while waiting on RPC response - topic: "q-plugin", RPC method: "security_group_rules_for_devi ces" info: "<unknown>" 2014-11-13 07:18:48.969 2313 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent