Bug 1508725 - Glance upload large image via dashboard fails
Summary: Glance upload large image via dashboard fails
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: python-django-horizon
Version: 10.0 (Newton)
Hardware: All
OS: Linux
urgent
urgent
Target Milestone: ---
: ---
Assignee: Radomir Dopieralski
QA Contact: Ido Ovadia
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-11-02 05:54 UTC by Anil Dhingra
Modified: 2021-03-11 16:10 UTC (History)
26 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-03-13 16:47:58 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Anil Dhingra 2017-11-02 05:54:07 UTC
Description of problem:

Glance upload large image via dashboard fails , Initially tried with 160GB which fails , than tried with 4 Gb ubuntu image which also fails .
not is logged in logs to find reason for failure /var/log/httpd/  or glance logs or /var/log/horizon/horizon.log which leads to identify issue.
- Browser used: Safari/Chrome. 
- sometimes it fails after completing 100% or sometimes before that
- following error on dashboard:
"TypeError: undefined is not an object (evaluating 'response.data')" 
"TypeError: Cannot read property 'data' of undefined" 

which may be related to below 

https://bugs.launchpad.net/horizon/+bug/1703708 and https://bugs.launchpad.net/horizon/+bug/1630833

Version-Release number of selected component (if applicable):
 

How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:

Horizon should be able to upload files or log reason for failure
Additional info:

using command line image upload works fine

Comment 1 Anil Dhingra 2017-11-02 08:40:24 UTC
 tried this on Chrome too. And  get the following error from the console:

horizon-gs.sg.myrepublic.net/ Failed to load resource: the server responded with a status of 500 (INTERNAL SERVER ERROR)
2acb6f038c7a.js:1230 

TypeError: Cannot read property 'data' of undefined
    at onCreateImage (f4df4c852c71.js:676)
    at processQueue (2acb6f038c7a.js:1326)
    at 2acb6f038c7a.js:1327
    at Scope.$eval (2acb6f038c7a.js:1364)
    at Scope.$digest (2acb6f038c7a.js:1360)
    at Scope.$apply (2acb6f038c7a.js:1365)
    at done (2acb6f038c7a.js:1142)
    at completeRequest (2acb6f038c7a.js:1159)
    at XMLHttpRequest.requestLoaded (2acb6f038c7a.js:1153)

Comment 3 Cyril Roelandt 2017-11-04 04:16:16 UTC
The upstream bugs you mentioned in your report seem solved by a patch to Horizon, not Glance. Also, the reference to "undefined" points to a Javascript issue, with an error not being handled properly.

Apparently, Beth from the UI team think this might be an issue in the Glance config. This also seems to be the case (hence the 403 return code), but this is a different issue.

@Beth: what do you think?

Comment 4 Beth White 2017-11-06 12:06:20 UTC
Hi Cyril,

As mentioned last week during testing, I was able to upload an image and my suggestion that this is an issue with the Glance config comes from there. With regard to the other issue now brought up as above, can you link the bz/launchpad/gerrit that you say might already fix this in later versions. It sounds like another bug might need to be opened to cover that and possibly do a backport for it.

Thanks!
Beth

Comment 5 Anil Dhingra 2017-11-07 01:26:33 UTC
Cu tried to upload the image as a non-admin user in private upload mode, the upload fails on 100% with the error:

(It fails in public mode, non-admin user too with the same error)

TypeError: Cannot read property 'data' of undefined

TypeError: Cannot read property 'data' of undefined 2acb6f038c7a.js:1230
    at onCreateImage (f4df4c852c71.js:676)
    at processQueue (2acb6f038c7a.js:1326)
    at 2acb6f038c7a.js:1327
    at Scope.$eval (2acb6f038c7a.js:1364)
    at Scope.$digest (2acb6f038c7a.js:1360)
    at Scope.$apply (2acb6f038c7a.js:1365)
    at done (2acb6f038c7a.js:1142)
    at completeRequest (2acb6f038c7a.js:1159)
    at XMLHttpRequest.requestLoaded (2acb6f038c7a.js:1153)

Comment 6 Pranali Deore 2017-11-07 05:57:21 UTC
Hi Beth,

According to the below bug's history, the fix is not backported in newton.

https://bugs.launchpad.net/horizon/+bug/1630833


So, that could be the reason behind this error in OSP 10.

Comment 7 Cyril Roelandt 2017-11-07 15:30:32 UTC
@Beth: So there might be an issue in the user's configuration, but I doubt it, since the same operation works perfectly when performed using the CLI.

The issue Anil is talking about is clearly a JS issue, probably fixed upstream (see the launchpad bug reports in the first message). Pranali is right: the patches have not been backported, which is why the user gets this weird stacktrace.

I'm moving this bug to python-django-horizon. Feel free to retarget to openstack-glance if you think I'm wrong. If, once the Horizon issue is fixed, Glance-related issues arise, please feel free to open a new bug.

Comment 24 Cyril Roelandt 2017-11-08 21:12:33 UTC
Radomir is probably right here. If a workaround is needed, can't the user just use the CLI, since this definitely works?

Comment 31 Erno Kuvaja 2017-11-09 12:30:24 UTC
Few quick remarks:
https://developer.openstack.org/api-ref/image/v2/index.html#create-an-image

Visibility is optional value but if it's passed, None is not one of the allowed ones (just noticed that been one of the errors seen in the horizon stack trace).

Glance API point of view the visibility has nothing to do with the image upload, the image creation likely fails if None is sent instead of allowed visibility value but it won't change the actual upload call, so fiddling around with that is just waste of time.

In non-cors environments uploading large images from Horizon has never really worked. I think it's even documented somewhere (I have no idea where, but I remember the discussion around documenting it) pretty clearly to be the expectation not to work.

Using CLI is perfectly viable solution (not just workaround) for the issue and the python-glanceclient should be able to deal with the token ttl.

If the direct upload method is used, just make sure that the cors middleware is also in the api pipeline (I don't remember if it's there by default in 10). `glance-api-paste.ini`

I have no idea if enabling cors will solve the issue on those data sizes. So far from the data available I have not found any indication of glance bug, maybe config issue, likely well known problem that has easy solution (use CLI instead of GUI for large image transfers).

Comment 39 Cyril Roelandt 2017-11-20 23:22:39 UTC
@Robin: In #34, Beth wrote:

/etc/glance/glance-api.conf 
uncomment and set
show_multiple_locations = true
allow_methods = GET,PUT,POST,DELETE,PATCH (* ensure to uncomment both of these - there are 2)


I can see that "allowed_methods" is set to the right value in the "cors" section, but is still commented out in the "cors.subdomain" section. Could you try and uncomment both of them and rerun your tests?

Comment 42 Cyril Roelandt 2017-11-22 23:18:20 UTC
Erno's comment in #31 still holds.

Your main problem here is that "data.get('visibility')" returns None. The request.DATA dictionary should have a valid value for the 'visibility' key. You could try to hardcode it for debugging purposes (it is a bit dirty), but really, the way the request is crafted in Horizon is probably wrong.

Comment 80 Radomir Dopieralski 2018-02-02 11:14:01 UTC
Perhaps a separate issue should be created for OSP9.


Note You need to log in before you can comment on or make changes to this bug.