Bug 1287621
Summary: | Horizon - Launch an instance on admin project return Internal Server Error | |||
---|---|---|---|---|
Product: | Red Hat OpenStack | Reporter: | Ido Ovadia <iovadia> | |
Component: | openstack-cinder | Assignee: | Eric Harney <eharney> | |
Status: | CLOSED ERRATA | QA Contact: | nlevinki <nlevinki> | |
Severity: | urgent | Docs Contact: | ||
Priority: | high | |||
Version: | 7.0 (Kilo) | CC: | aortega, athomas, ddomingo, eharney, geguileo, mcornea, mrunge, scohen, sgotliv, yeylon | |
Target Milestone: | z3 | Keywords: | Regression, ZStream | |
Target Release: | 7.0 (Kilo) | |||
Hardware: | x86_64 | |||
OS: | Linux | |||
Whiteboard: | ||||
Fixed In Version: | openstack-cinder-2015.1.2-5.el7ost | Doc Type: | Bug Fix | |
Doc Text: | Story Points: | --- | ||
Clone Of: | ||||
: | 1288652 (view as bug list) | Environment: | ||
Last Closed: | 2015-12-21 17:04:01 UTC | Type: | Bug | |
Regression: | --- | Mount Type: | --- | |
Documentation: | --- | CRM: | ||
Verified Versions: | Category: | --- | ||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
Cloudforms Team: | --- | Target Upstream Version: | ||
Embargoed: | ||||
Bug Depends On: | ||||
Bug Blocks: | 1288652, 1288654 |
Description
Ido Ovadia
2015-12-02 13:09:16 UTC
Cinder logs are showing: 2015-12-01 10:49:40.238 12951 ERROR cinder.api.middleware.fault [req-48f9277d-0ea8-4af9-b80c-a3e9eacdca6c 00074bde5412442499a7390d98c824d9 0788735cc9a445ac95611db9dffafebb - - -] Caught error: Entity '<class 'c inder.db.sqlalchemy.models.Snapshot'>' has no property 'tenant_id' 2015-12-01 10:49:40.239 12951 INFO cinder.api.middleware.fault [req-48f9277d-0ea8-4af9-b80c-a3e9eacdca6c 00074bde5412442499a7390d98c824d9 0788735cc9a445ac95611db9dffafebb - - -] http://cougar11.scl.lab.tlv.redhat.com:8776/v2/0788735cc9a445ac95611db9dffafebb/snapshots/detail?alltenants=1&tenant_id=0788735cc9a445ac95611db9dffafebb returned with HTTP 500 2015-12-01 10:49:40.240 12951 INFO eventlet.wsgi.server [req-48f9277d-0ea8-4af9-b80c-a3e9eacdca6c 00074bde5412442499a7390d98c824d9 0788735cc9a445ac95611db9dffafebb - - -] 10.35.160.111 - - [01/Dec/2015 10:49:40] "GET /v2/0788735cc9a445ac95611db9dffafebb/snapshots/detail?alltenants=1&tenant_id=0788735cc9a445ac95611db9dffafebb HTTP/1.1" 500 425 0.007403 Which makes me think, this is a cinder issue. *** Bug 1287664 has been marked as a duplicate of this bug. *** *** Bug 1288015 has been marked as a duplicate of this bug. *** There is a Cinder bug here, but I think it is only triggered by Horizon, due to the search_opts passed in to snapshot detail list. We'll fix up the Cinder side and then see if any corresponding changes are also needed in Horizon. Reproducer for cinder server problem: Edit cinderclient's v2/shell.py: change search_opts in snapshot_list to send "alltenants" instead of "all_tenants" as the name of the field. $ cinder snapshot-list --all-tenants 1 Reverting the patch e080aba7 Filter snapshots data on the DB side from Cinder causes Cinder to no longer throw HTTP 500 errors. (Added in bug 1279547.) Root of the problem is an horizon bug (even though Cinder should not return 500): https://bugs.launchpad.net/horizon/+bug/1469589 And it gets fixed in here: https://github.com/openstack/horizon/commit/faa44bfe1d6c4f342b8572bc4939771cda2039c7 As for Cinder, it should not return 500, so we need to we need to add filter validation (https://github.com/openstack/cinder/commit/8ba66fc5d2672bff69c21e36918e5e1882167649). Some of the validations introduced in this patch are already in place, though some are missing on 7.0. We need Horizon fix as well as the Cinder fix, because with only the Cinder fix Horizon will receive an empty list due to the malformed request. *** Bug 1287770 has been marked as a duplicate of this bug. *** Horizon backport is here https://review.openstack.org/253496 Tested python-cinder-2014.2.3-11.el7ost.noarch for OSP6: The Cinder server does not throw an HTTP 500. It reports: 2015-12-04 16:31:49.473 19854 INFO cinder.api.openstack.wsgi [req-6e320001-798b-47ad-b7ea-b0368e7c023d 5bfa55d0e4844a96b744cb311c8ac4ce 252783c016b54c21999d73d3ececa650 - - -] GET http://10.16.151.248:8776/v2/252783c016b54c21999d73d3ececa650/snapshots/detail?all_tenants=1&alltenants=1 ... 2015-12-04 16:31:49.477 19854 DEBUG cinder.db.sqlalchemy.api [req-6e320001-798b-47ad-b7ea-b0368e7c023d 5bfa55d0e4844a96b744cb311c8ac4ce 252783c016b54c21999d73d3ececa650 - - -] 'alltenants' filter key is not valid. snapshot_get_all /usr/lib/python2.7/site-packages/cinder/db/sqlalchemy/api.py:1742 It returns HTTP 200 with an empty list. This should probably be an HTTP 400, but seems much less harmful than the OSP7 behavior. Reverting this patch [1] doesn't make the situation any better. Yes, it addresses the immediate concern described in that case, but the idea is to move forward. I agree with comment#7 with one small correction, I think HTTP 400 is the better way to deal with the wrong filters than returning an empty list. 1. Horizon should stop sending faulty/non-existing filters, exactly as comment#7 says. 2. If Horizon fixes the problem then Cinder should start validating filters and respond with HTTP 400 if filter is wrong. If Horizon doesn't fix the problem then Cinder still has to validate filters but do that in the backward compatible way - means ignore faulty filters. [1] https://code.engineering.redhat.com/gerrit/#/c/61203/ I want to modify my previous comment: > If Horizon doesn't fix the problem then > Cinder still has to validate filters but do that in the backward compatible way > - means ignore faulty filters. If Horizon doesn't fix the problem then Cinder still has to validate filters and return an empty list if there are faulty filters as Cinder does in OSP 6.0, see comment #10. Just to make sure, I understand the issue on RHOS-6: I can not confirm an issue with and without having modified filters on horizon. I just tested openstack-cinder-2014.2.3-6.el7ost.noarch Is the patch appearing to make this issue worse being backported to rhos-6 too? Verified ======== openstack-cinder-2015.1.2-5.el7ost.noarch Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2015:2679 |