Bug 1533592
Summary: | Neutron ERROR DBDuplicateEntry ipallocations | |||
---|---|---|---|---|
Product: | Red Hat OpenStack | Reporter: | Marc Methot <mmethot> | |
Component: | openstack-neutron | Assignee: | Ihar Hrachyshka <ihrachys> | |
Status: | CLOSED ERRATA | QA Contact: | Toni Freger <tfreger> | |
Severity: | medium | Docs Contact: | ||
Priority: | medium | |||
Version: | 9.0 (Mitaka) | CC: | amuller, ccollett, chrisw, ihrachys, jlibosva, jraju, mbayer, nyechiel, oblaut, ragiman, srevivo | |
Target Milestone: | zstream | Keywords: | Triaged, ZStream | |
Target Release: | 9.0 (Mitaka) | |||
Hardware: | Unspecified | |||
OS: | Unspecified | |||
Whiteboard: | ||||
Fixed In Version: | openstack-neutron-8.4.0-17.el7ost | Doc Type: | Bug Fix | |
Doc Text: |
When multiple API requests that allocate IP addresses in the same subnet are issued, sometimes the IPAM layer raises a duplicate error. Previously, when this happened, the API layer would not retry the allocation; instead it bubbled up the error as HTTP error code 5xx to the API consumer. Consequently, API requests would sometimes fail with error code 5xx.
With this update, the Neutron API layer now retries all duplicate errors. This allows the IPAM layer to allocate a new, unused address, and successfully fulfil the API request. As a result, when the IPAM layer hits a duplicate error from allocating two addresses from the same subnet at the same time, no 5xx errors are bubbled up to the API consumers.
|
Story Points: | --- | |
Clone Of: | ||||
: | 1544561 (view as bug list) | Environment: | ||
Last Closed: | 2018-03-15 12:41:29 UTC | Type: | Bug | |
Regression: | --- | Mount Type: | --- | |
Documentation: | --- | CRM: | ||
Verified Versions: | Category: | --- | ||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
Cloudforms Team: | --- | Target Upstream Version: | ||
Embargoed: | ||||
Bug Depends On: | ||||
Bug Blocks: | 1544561 |
Description
Marc Methot
2018-01-11 17:19:59 UTC
Openstack installed rpms openstack-aodh-api-2.0.5-1.el7ost.noarch openstack-aodh-common-2.0.5-1.el7ost.noarch openstack-aodh-evaluator-2.0.5-1.el7ost.noarch openstack-aodh-listener-2.0.5-1.el7ost.noarch openstack-aodh-notifier-2.0.5-1.el7ost.noarch openstack-ceilometer-api-6.1.3-2.el7ost.noarch openstack-ceilometer-central-6.1.3-2.el7ost.noarch openstack-ceilometer-collector-6.1.3-2.el7ost.noarch openstack-ceilometer-common-6.1.3-2.el7ost.noarch openstack-ceilometer-compute-6.1.3-2.el7ost.noarch openstack-ceilometer-notification-6.1.3-2.el7ost.noarch openstack-ceilometer-polling-6.1.3-2.el7ost.noarch openstack-cinder-8.1.1-4.el7ost.noarch openstack-dashboard-9.0.1-3.el7ost.noarch openstack-dashboard-theme-9.0.1-3.el7ost.noarch openstack-glance-12.0.0-2.el7ost.noarch openstack-gnocchi-api-2.1.3-3.el7ost.noarch openstack-gnocchi-carbonara-2.1.3-3.el7ost.noarch openstack-gnocchi-common-2.1.3-3.el7ost.noarch openstack-gnocchi-indexer-sqlalchemy-2.1.3-3.el7ost.noarch openstack-gnocchi-metricd-2.1.3-3.el7ost.noarch openstack-gnocchi-statsd-2.1.3-3.el7ost.noarch openstack-heat-api-6.0.0-12.el7ost.noarch openstack-heat-api-cfn-6.0.0-12.el7ost.noarch openstack-heat-api-cloudwatch-6.0.0-12.el7ost.noarch openstack-heat-common-6.0.0-12.el7ost.noarch openstack-heat-engine-6.0.0-12.el7ost.noarch openstack-keystone-9.0.2-1.el7ost.noarch openstack-manila-2.0.0-6.el7ost.noarch openstack-manila-share-2.0.0-6.el7ost.noarch openstack-neutron-8.1.2-13.el7ost.noarch openstack-neutron-bigswitch-agent-2015.3.8-1.el7ost.noarch openstack-neutron-bigswitch-lldp-2015.3.8-1.el7ost.noarch openstack-neutron-common-8.1.2-13.el7ost.noarch openstack-neutron-lbaas-8.0.0-1.el7ost.noarch openstack-neutron-metering-agent-8.1.2-13.el7ost.noarch openstack-neutron-ml2-8.1.2-13.el7ost.noarch openstack-neutron-openvswitch-8.1.2-13.el7ost.noarch openstack-nova-api-13.1.2-9.el7ost.noarch openstack-nova-cert-13.1.2-9.el7ost.noarch openstack-nova-common-13.1.2-9.el7ost.noarch openstack-nova-compute-13.1.2-9.el7ost.noarch openstack-nova-conductor-13.1.2-9.el7ost.noarch openstack-nova-console-13.1.2-9.el7ost.noarch openstack-nova-novncproxy-13.1.2-9.el7ost.noarch openstack-nova-scheduler-13.1.2-9.el7ost.noarch openstack-puppet-modules-8.1.10-1.el7ost.noarch openstack-sahara-4.0.1-2.el7ost.noarch openstack-sahara-api-4.0.1-2.el7ost.noarch openstack-sahara-common-4.0.1-2.el7ost.noarch openstack-sahara-engine-4.0.1-2.el7ost.noarch openstack-sahara-ui-4.0.0-3.el7ost.noarch openstack-selinux-0.7.11-1.el7ost.noarch openstack-swift-2.7.0-2.el7ost.noarch openstack-swift-account-2.7.0-2.el7ost.noarch openstack-swift-container-2.7.0-2.el7ost.noarch openstack-swift-object-2.7.0-2.el7ost.noarch openstack-swift-plugin-swift3-1.10-1.el7ost.noarch openstack-swift-proxy-2.7.0-2.el7ost.noarch openstack-trove-api-5.0.1-1.el7ost.noarch openstack-trove-common-5.0.1-1.el7ost.noarch openstack-trove-conductor-5.0.1-1.el7ost.noarch openstack-trove-taskmanager-5.0.1-1.el7ost.noarch openstack-utils-2015.2-1.el7ost.noarch The patch I linked to should fix the issue. That being said, it's a very invasive patch that changes not only internal implementation (that would need significant regression testing) but also observed behavior of IPAM layer (the order in which addresses are allocated is now not fully predictable; the observed change is well captured by the following change to tempest that was needed to remove assumptions about the order from existing API tests: https://review.openstack.org/#/c/312771). We should consider whether the benefit of getting this fixed outweighs potential risks of breaking IP allocation layer. (Though on first look, the patch is more or less cleanly applicable to Mitaka codebase, so at least producing the backport shouldn't be that big of a deal, unless I miss some implicit dependencies of the fix on some other Newton+ work.) We will discuss applicability of the backport on next bug triage call. There is another patch worth a try: when DuplicateError is bubbled up to api layer, it should make it retry instead of spill 500 on user. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2018:0537 |