Bug 1288841

Summary: Failed to upload an attached volume to Glance with --force True
Product: Red Hat OpenStack Reporter: Yogev Rabl <yrabl>
Component: openstack-cinderAssignee: Eric Harney <eharney>
Status: CLOSED NOTABUG QA Contact: nlevinki <nlevinki>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: unspecifiedCC: eharney, sgotliv, yeylon
Target Milestone: ---   
Target Release: 8.0 (Liberty)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-12-06 16:09:19 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Yogev Rabl 2015-12-06 13:35:07 UTC
Description of problem:
the command 
# cinder upload-to-image --force true ... 

allows to uploading attached volumes to Glance as images. But the command failed with an error in the client side: 
# cinder upload-to-image --container-format bare --disk-format raw --force True <volume uuid> image 
ERROR: Invalid volume: Volume status is in-use. (HTTP 400) (Request-ID: req-04848dff-90e4-4788-909d-3c850eb0c290)

Cinder API log shows: 

2015-12-06 15:03:45.103 21586 DEBUG eventlet.wsgi.server [-] (21586) accepted ('10.35.160.91', 38611) server /usr/lib/python2.7/site-packages/eventlet/wsgi.py:826
2015-12-06 15:03:45.108 21586 DEBUG keystoneclient.session [-] REQ: curl -g -i -X GET http://10.35.160.91:35357/v3/auth/tokens -H "X-Subject-Token: {SHA1}9a67ee224ec0e8214a1f3c2f5c432d6756308517" -H "User-Agent:
 python-keystoneclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}720f07164b61e9d1637c6e35d177035ee0410c6c" _http_log_request /usr/lib/python2.7/site-packages/keystoneclient/session.py:198
2015-12-06 15:03:45.189 21586 DEBUG keystoneclient.session [-] RESP: [200] content-length: 1886 content-encoding: gzip x-subject-token: {SHA1}9a67ee224ec0e8214a1f3c2f5c432d6756308517 vary: X-Auth-Token,Accept-En
coding server: Apache/2.4.6 (Red Hat Enterprise Linux) connection: close date: Sun, 06 Dec 2015 13:03:45 GMT content-type: application/json x-openstack-request-id: req-43b75f94-fa43-4cce-aba2-cec9e4d0b046 
RESP BODY: {"token": {"methods": ["password", "token"], "roles": [{"id": "c5e3b5a4c91242febcac3b873d179d47", "name": "admin"}, {"id": "9fe2ff9ee4384b1894a90878d3e92bab", "name": "_member_"}, {"id": "8797982c2d5c
4e54abf574899a36e6a8", "name": "heat_stack_owner"}], "expires_at": "2015-12-06T14:03:45.000000Z", "project": {"domain": {"id": "default", "name": "Default"}, "id": "a7c95c293a1441b394735bd4d75032e7", "name": "ad
min"}, "catalog": "<removed>", "extras": {}, "user": {"domain": {"id": "default", "name": "Default"}, "id": "eb74ae4564754c7d94fc8d4e0f21d5ca", "name": "admin"}, "audit_ids": ["9n92-5YUSQuD3q0-hJyKJw"], "issued_
at": "2015-12-06T13:03:45.077385"}}
 _http_log_response /usr/lib/python2.7/site-packages/keystoneclient/session.py:215
2015-12-06 15:03:45.193 21586 INFO cinder.api.openstack.wsgi [req-e6c8b92b-edb1-4b28-b738-86f51b262363 eb74ae4564754c7d94fc8d4e0f21d5ca a7c95c293a1441b394735bd4d75032e7 - - -] GET http://10.35.160.91:8776/v2/a7c
95c293a1441b394735bd4d75032e7/volumes/71663c02-b6a2-458e-9535-9a2f38067676
2015-12-06 15:03:45.194 21586 DEBUG cinder.api.openstack.wsgi [req-e6c8b92b-edb1-4b28-b738-86f51b262363 eb74ae4564754c7d94fc8d4e0f21d5ca a7c95c293a1441b394735bd4d75032e7 - - -] Empty body provided in request get
_body /usr/lib/python2.7/site-packages/cinder/api/openstack/wsgi.py:862
2015-12-06 15:03:45.386 21586 INFO cinder.volume.api [req-e6c8b92b-edb1-4b28-b738-86f51b262363 eb74ae4564754c7d94fc8d4e0f21d5ca a7c95c293a1441b394735bd4d75032e7 - - -] Volume info retrieved successfully.
2015-12-06 15:03:45.516 21586 INFO cinder.api.openstack.wsgi [req-e6c8b92b-edb1-4b28-b738-86f51b262363 eb74ae4564754c7d94fc8d4e0f21d5ca a7c95c293a1441b394735bd4d75032e7 - - -] http://10.35.160.91:8776/v2/a7c95c2
93a1441b394735bd4d75032e7/volumes/71663c02-b6a2-458e-9535-9a2f38067676 returned with HTTP 200
2015-12-06 15:03:45.517 21586 INFO eventlet.wsgi.server [req-e6c8b92b-edb1-4b28-b738-86f51b262363 eb74ae4564754c7d94fc8d4e0f21d5ca a7c95c293a1441b394735bd4d75032e7 - - -] 10.35.160.91 - - [06/Dec/2015 15:03:45] 
"GET /v2/a7c95c293a1441b394735bd4d75032e7/volumes/71663c02-b6a2-458e-9535-9a2f38067676 HTTP/1.1" 200 1648 0.411182
2015-12-06 15:03:45.534 21586 INFO cinder.api.openstack.wsgi [req-04848dff-90e4-4788-909d-3c850eb0c290 eb74ae4564754c7d94fc8d4e0f21d5ca a7c95c293a1441b394735bd4d75032e7 - - -] POST http://10.35.160.91:8776/v2/a7
c95c293a1441b394735bd4d75032e7/volumes/71663c02-b6a2-458e-9535-9a2f38067676/action
2015-12-06 15:03:45.535 21586 DEBUG cinder.api.openstack.wsgi [req-04848dff-90e4-4788-909d-3c850eb0c290 eb74ae4564754c7d94fc8d4e0f21d5ca a7c95c293a1441b394735bd4d75032e7 - - -] Action body: {"os-volume_upload_im
age": {"container_format": "bare", "force": "True", "image_name": "image02", "disk_format": "raw"}} get_method /usr/lib/python2.7/site-packages/cinder/api/openstack/wsgi.py:1088
2015-12-06 15:03:45.598 21586 INFO cinder.volume.api [req-04848dff-90e4-4788-909d-3c850eb0c290 eb74ae4564754c7d94fc8d4e0f21d5ca a7c95c293a1441b394735bd4d75032e7 - - -] Volume info retrieved successfully.
2015-12-06 15:03:45.599 21586 INFO cinder.volume.api [req-04848dff-90e4-4788-909d-3c850eb0c290 eb74ae4564754c7d94fc8d4e0f21d5ca a7c95c293a1441b394735bd4d75032e7 - - -] Force upload to image is disabled, Force option will be ignored.
2015-12-06 15:03:45.601 21586 INFO cinder.api.openstack.wsgi [req-04848dff-90e4-4788-909d-3c850eb0c290 eb74ae4564754c7d94fc8d4e0f21d5ca a7c95c293a1441b394735bd4d75032e7 - - -] HTTP exception thrown: Invalid volume: Volume status is in-use.

This error appears both when the volume is the root disk and when a disk is attached as vdb. 


Version-Release number of selected component (if applicable):
openstack-cinder-7.0.0-2.el7ost.noarch
python-cinderclient-1.4.0-1.el7ost.noarch
python-cinder-7.0.0-2.el7ost.noarch

How reproducible:
100% 

Steps to Reproduce:
1. Launch an instance 
# nova boot --flavor <uuid> --image <image uuid> --nic net-id <net uuid> VM01
2. Create a volume
# cinder create --display-name to-upload <size>
3. Attach the volume to the instance
# nova volume-attach <VM01 uuid> <to-upload uuid> auto 
4. Attempt to upload the volume to Glance 
# cinder upload-to-image --container-format bare --disk-format <format> --force True <to-upload uuid> image01

or 

1. Create an image
# glance image-create --container-format bare --disk-format <format> --name image01 --file <path to image>
2. Create a volume from image
# cinder create --image-id <image01 uuid> --display-name bootable-vol <size>
3. Launch an instance from the volume
# nova boot boot-volume <bootable-vol uuid> --flavor <flavor uuid> --nic net-id=<net uuid> --key-name <key> VM02
4. Upload the volume while the instance runs
#  cinder upload-to-image --container-format bare --disk-format <format> --force True <bootable-vol uuid> image02  

Actual results:
A client error when trying to upload the volume


Expected results:
The volume is been uploaded, and while is that the instance is paused/suspended.

Additional info:

Comment 2 Sergey Gotliv 2015-12-06 16:09:19 UTC
Yogev,

Not all backends support uploading of attached ("in-use") volumes to Glance, it potentially can lead to data corruption, that's why Cinder introduces a new configuration parameter "enable_force_upload" in addition to "force" command line parameter to enforce upload of attached volumes. This commit message provides a few more reasons [1] to use that additional configuration parameter. Please, configure "enable_force_upload=True" and rerun that test.

[1] https://review.openstack.org/#/c/176174/5//COMMIT_MSG