Bug 1567627
Summary: | DBAPI and DBDealock errors when trying to update agents | ||
---|---|---|---|
Product: | Red Hat OpenStack | Reporter: | Sai Sindhur Malleni <smalleni> |
Component: | openstack-neutron | Assignee: | Assaf Muller <amuller> |
Status: | CLOSED INSUFFICIENT_DATA | QA Contact: | Toni Freger <tfreger> |
Severity: | high | Docs Contact: | |
Priority: | unspecified | ||
Version: | 13.0 (Queens) | CC: | amuller, bcafarel, bhaley, chrisw, majopela, njohnston, nyechiel, pveiga, smalleni, srevivo, tfreger |
Target Milestone: | --- | Keywords: | Reopened |
Target Release: | --- | ||
Hardware: | Unspecified | ||
OS: | Linux | ||
Whiteboard: | |||
Fixed In Version: | Doc Type: | If docs needed, set a value | |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | 2019-04-29 13:53:11 UTC | Type: | Bug |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: |
Description
Sai Sindhur Malleni
2018-04-15 15:46:16 UTC
1) Can you please attach sosreport of all controllers during active load and the occurrence of these exceptions? 2) Can you please describe what kind of workload were you running to cause these errors? (Preferably a link to the Rally scenarios including parameters and how you ran it). Hey Assaf, I don't have the environment available now however the tests I ran are here using Browbeat (which uses Rally). https://github.com/openstack/browbeat/blob/master/browbeat-config.yaml#L144-L165 I ran the neutro ntests above with concurrencies of 8,16 and 32 and with times set to 500. I don't think a developer would be able to reproduce this easily, if you get the chance to reproduce this in the future on a environment could you ping a developer on the Networking DFG / Neutron squad so they'd be able to hop on the environment live? Hey Assaf, I did manage to reproduce it, here are the sosreports for the controllers (neutron was in debug) http://elk.browbeatproject.org:9090/~smalleni/sosreports/ I also have a hunch this could be triggering https://bugzilla.redhat.com/show_bug.cgi?id=1571499 Actually, that link now has sosreports for the 3 controllers as well as the 6 computes. If you are only interested in controllers, look for the following files in the above link sosreport-SaiMalleni-20180425124105.tar.xz sosreport-SaiMalleni-20180425124134.tar.xz sosreport-SaiMalleni-20180425124202.tar.xz We're trying to determine if this issue is related to 1571499, the one you linked in comment 4. Is it possible to workaround 1571499 and rerun the tests to see if you still see the Neutron DB exceptions? Maybe disable telemetry? This reminds me of issues we have seen in the past and that were recoverable messages of neutron talking to the database. (DB retries, etc..). But it deserves at least an investigation once we figure out what Assaf commented on 6. Sai - I have not looked through the sosreports yet, but do you know if this is still reproducible after disabling telemetry? Also, wouldn't https://review.opendev.org/#/c/326927/ fix it? Priscila, you reopened an existing bug that is for something unrelated. Please create a new bug. I am re-closing this bug now. (In reply to Nate Johnston from comment #13) > Priscila, you reopened an existing bug that is for something unrelated. > Please create a new bug. I am re-closing this bug now. Hi. I opened https://bugzilla.redhat.com/show_bug.cgi?id=1704497 |