Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1368646

Summary: Cinder retype operation succeeds but the changes are not reflected in `cinder show` output.
Product: Red Hat OpenStack Reporter: Pratik Pravin Bandarkar <pbandark>
Component: openstack-cinderAssignee: Eric Harney <eharney>
Status: CLOSED ERRATA QA Contact: lkuchlan <lkuchlan>
Severity: high Docs Contact:
Priority: unspecified    
Version: 9.0 (Mitaka)CC: egafford, eharney, jjoyce, lkuchlan, srevivo
Target Milestone: asyncKeywords: ZStream
Target Release: 9.0 (Mitaka)   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: openstack-cinder-8.1.0-2.el7ost Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-10-05 19:14:46 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1365597    
Bug Blocks:    

Description Pratik Pravin Bandarkar 2016-08-20 10:53:48 UTC
Description of problem:


- Cinder retype operation successds but the changes are not reflected in `cinder show` output.

[root@dhcp200-208 ~(keystone_admin)]# cinder create --volume-type iscsi --name iscsi_vol1 1
+--------------------------------+--------------------------------------+
|            Property            |                Value                 |
+--------------------------------+--------------------------------------+
|          attachments           |                  []                  |
|       availability_zone        |                 nova                 |
|            bootable            |                false                 |
|      consistencygroup_id       |                 None                 |
|           created_at           |      2016-08-20T15:33:35.000000      |
|          description           |                 None                 |
|           encrypted            |                False                 |
|               id               | f0faecce-7b71-4d9b-89de-85bb90b850b0 |  <==
|            metadata            |                  {}                  |
|        migration_status        |                 None                 |
|          multiattach           |                False                 |
|              name              |              iscsi_vol1              |
|     os-vol-host-attr:host      |                 None                 |
| os-vol-mig-status-attr:migstat |                 None                 |
| os-vol-mig-status-attr:name_id |                 None                 |
|  os-vol-tenant-attr:tenant_id  |   84df9e408f454a74ac746c9e3af0abd6   |
|       replication_status       |               disabled               |
|              size              |                  1                   |
|          snapshot_id           |                 None                 |
|          source_volid          |                 None                 |
|             status             |               creating               |
|           updated_at           |                 None                 |
|            user_id             |   aa2bc39017904b7ea83b479af68ca1c7   |
|          volume_type           |                iscsi                 |  <==
+--------------------------------+--------------------------------------+


[root@dhcp200-208 ~(keystone_admin)]# lvs
  LV                                          VG             Attr       LSize Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  volume-f0faecce-7b71-4d9b-89de-85bb90b850b0 cinder-volumes -wi-a----- 1.00g     <===                                                
  volume-f6296b85-9c58-4eb6-a3d1-a61bdd0594b1 cinder-volumes -wi-a----- 1.00g   


[root@dhcp200-208 ~(keystone_admin)]# cinder extra-specs-list
+--------------------------------------+-------+-----------------------------------+
|                  ID                  |  Name |            extra_specs            |
+--------------------------------------+-------+-----------------------------------+
| 1ddc813e-bd65-4b93-adff-df562416e0fc | iscsi |   {'volume_backend_name': 'lvm'}  |
| 4a44b750-5937-4569-9c0e-1409e6106cef |  nfs  | {'volume_backend_name': 'nfs123'} |
| 561ca32b-a241-4d32-a809-ecb8338c6981 |  lvm  |   {'volume_backend_name': 'lvm'}  |
+--------------------------------------+-------+-----------------------------------+


[root@dhcp200-208 ~(keystone_admin)]# cinder retype f0faecce-7b71-4d9b-89de-85bb90b850b0 nfs --migration-policy on-demand

[root@dhcp200-208 ~(keystone_admin)]# cinder show f0faecce-7b71-4d9b-89de-85bb90b850b0
+--------------------------------+------------------------------------------------------+
|            Property            |                        Value                         |
+--------------------------------+------------------------------------------------------+
|          attachments           |                          []                          |
|       availability_zone        |                         nova                         |
|            bootable            |                        false                         |
|      consistencygroup_id       |                         None                         |
|           created_at           |              2016-08-20T15:33:35.000000              |
|          description           |                         None                         |
|           encrypted            |                        False                         |
|               id               |         f0faecce-7b71-4d9b-89de-85bb90b850b0         |
|            metadata            |                          {}                          |
|        migration_status        |                       success                        |
|          multiattach           |                        False                         |
|              name              |                      iscsi_vol1                      |
|     os-vol-host-attr:host      | dhcp200-208.gsslab.pnq.redhat.com@development#nfs123 |
| os-vol-mig-status-attr:migstat |                       success                        |
| os-vol-mig-status-attr:name_id |                         None                         |
|  os-vol-tenant-attr:tenant_id  |           84df9e408f454a74ac746c9e3af0abd6           |
|       replication_status       |                       disabled                       |
|              size              |                          1                           |
|          snapshot_id           |                         None                         |
|          source_volid          |                         None                         |
|             status             |                      available                       |
|           updated_at           |              2016-08-20T15:47:59.000000              |
|            user_id             |           aa2bc39017904b7ea83b479af68ca1c7           |
|          volume_type           |                        iscsi                         |  <===
+--------------------------------+------------------------------------------------------+



[root@dhcp200-208 ~(keystone_admin)]# lvs


[root@dhcp200-208 23bd9281402ef955f41e97f8acd037e0(keystone_admin)]# ls /var/lib/cinder/mnt/23bd9281402ef955f41e97f8acd037e0/volume-f0faecce-7b71-4d9b-89de-85bb90b850b0 
/var/lib/cinder/mnt/23bd9281402ef955f41e97f8acd037e0/volume-f0faecce-7b71-4d9b-89de-85bb90b850b0


[root@dhcp200-208 23bd9281402ef955f41e97f8acd037e0(keystone_admin)]# mount |grep -i 23bd9281402ef9
10.65.200.45:/home/rhos_export/rhos9_cinder1 on /var/lib/cinder/mnt/23bd9281402ef955f41e97f8acd037e0 type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,port=0,timeo=600,retrans=2,sec=sys,clientaddr=10.65.200.208,local_lock=none,addr=10.65.200.45)


+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++


2016-08-20 11:47:44.305 13273 DEBUG oslo_concurrency.processutils [req-7ed12831-5ec1-4c9d-ba40-98ac6867dfad aa2bc39017904b7ea83b479af68ca1c7 84df9e408f454a74ac746c9e3af0abd6 - - -] u'sudo cinder-rootwrap /etc/cinder/rootwrap.conf systool -c fc_host -v' failed. Not Retrying. execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:422


2016-08-20 11:47:42.939 13279 DEBUG cinder.volume.manager [req-7ed12831-5ec1-4c9d-ba40-98ac6867dfad aa2bc39017904b7ea83b479af68ca1c7 84df9e408f454a74ac746c9e3af0abd6 - - -] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (15917628-d5c5-4526-9dd0-6519a35bedac) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,consistencygroup=<?>,consistencygroup_id=None,created_at=2016-08-20T15:33:35Z,deleted=False,deleted_at=None,display_description=None,display_name='iscsi_vol1',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,host='dhcp200-208.gsslab.pnq.redhat.com@development#nfs123',id=bf3f1636-2ed5-40db-a2cc-68946ad17a3a,launched_at=2016-08-20T15:33:36Z,metadata={},migration_status='target:f0faecce-7b71-4d9b-89de-85bb90b850b0',multiattach=False,previous_status='available',project_id=84df9e408f454a74ac746c9e3af0abd6,provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status='disabled',scheduled_at=2016-08-20T15:33:35Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2016-08-20T15:47:42Z,user_id=aa2bc39017904b7ea83b479af68ca1c7,volume_attachment=<?>,volume_type=VolumeType(4a44b750-5937-4569-9c0e-1409e6106cef),volume_type_id=4a44b750-5937-4569-9c0e-1409e6106cef)' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:178


2016-08-20 11:47:42.950 13279 DEBUG cinder.volume.manager [req-7ed12831-5ec1-4c9d-ba40-98ac6867dfad aa2bc39017904b7ea83b479af68ca1c7 84df9e408f454a74ac746c9e3af0abd6 - - -] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (919f7fce-933e-4a03-a4e7-30bb4af0a259) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-bf3f1636-2ed5-40db-a2cc-68946ad17a3a', 'type': 'raw', 'volume_id': 'bf3f1636-2ed5-40db-a2cc-68946ad17a3a'}' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:178




2016-08-20 11:47:43.076 13279 INFO cinder.volume.flows.manager.create_volume [req-7ed12831-5ec1-4c9d-ba40-98ac6867dfad aa2bc39017904b7ea83b479af68ca1c7 84df9e408f454a74ac746c9e3af0abd6 - - -] Volume bf3f1636-2ed5-40db-a2cc-68946ad17a3a: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-bf3f1636-2ed5-40db-a2cc-68946ad17a3a'}


2016-08-20 11:47:43.579 13279 INFO cinder.volume.flows.manager.create_volume [req-7ed12831-5ec1-4c9d-ba40-98ac6867dfad aa2bc39017904b7ea83b479af68ca1c7 84df9e408f454a74ac746c9e3af0abd6 - - -] Volume volume-bf3f1636-2ed5-40db-a2cc-68946ad17a3a (bf3f1636-2ed5-40db-a2cc-68946ad17a3a): created successfully




2016-08-20 11:47:48.443 13273 DEBUG oslo_concurrency.processutils [req-7ed12831-5ec1-4c9d-ba40-98ac6867dfad aa2bc39017904b7ea83b479af68ca1c7 84df9e408f454a74ac746c9e3af0abd6 - - -] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf dd if=/dev/disk/by-path/ip-10.65.200.208:3260-iscsi-iqn.2010-10.org.openstack:volume-f0faecce-7b71-4d9b-89de-85bb90b850b0-lun-0 of=/var/lib/cinder/mnt/23bd9281402ef955f41e97f8acd037e0/volume-bf3f1636-2ed5-40db-a2cc-68946ad17a3a count=1024 bs=1M iflag=direct oflag=direct conv=sparse execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:344


2016-08-20 11:47:55.083 13273 DEBUG os_brick.initiator.linuxscsi [req-7ed12831-5ec1-4c9d-ba40-98ac6867dfad aa2bc39017904b7ea83b479af68ca1c7 84df9e408f454a74ac746c9e3af0abd6 - - -] Checking to see if SCSI volume /dev/disk/by-path/ip-10.65.200.208:3260-iscsi-iqn.2010-10.org.openstack:volume-f0faecce-7b71-4d9b-89de-85bb90b850b0-lun-0 has been removed. wait_for_volume_removal /usr/lib/python2.7/site-packages/os_brick/initiator/linuxscsi.py:82



Logout of [sid: 2, target: iqn.2010-10.org.openstack:volume-f0faecce-7b71-4d9b-89de-85bb90b850b0, portal: 10.65.200.208,3260] successful.


2016-08-20 11:47:59.695 13273 INFO cinder.volume.manager [req-7ed12831-5ec1-4c9d-ba40-98ac6867dfad aa2bc39017904b7ea83b479af68ca1c7 84df9e408f454a74ac746c9e3af0abd6 - - -] Complete-Migrate volume completed successfully.
2016-08-20 11:47:59.696 13273 INFO cinder.volume.manager [req-7ed12831-5ec1-4c9d-ba40-98ac6867dfad aa2bc39017904b7ea83b479af68ca1c7 84df9e408f454a74ac746c9e3af0abd6 - - -] Migrate volume completed successfully.


Version-Release number of selected component (if applicable):
RHOS9

How reproducible:
100%

Steps to Reproduce:
1.
2.
3.

Actual results:
Cinder retype operation succeeds but the changes are not reflected in `cinder show` output.

Expected results:
operation should update cinder db.

Additional info:

upstream bz: https://bugs.launchpad.net/cinder/+bug/1565133
patch: https://review.openstack.org/#/c/301366/

Comment 2 Pratik Pravin Bandarkar 2016-08-20 11:00:48 UTC
# rpm -qa|grep -i cinder
python-cinder-8.0.0-5.el7ost.noarch
openstack-cinder-8.0.0-5.el7ost.noarch
python-cinderclient-1.6.0-1.el7ost.noarch

Comment 4 Eric Harney 2016-09-06 15:21:51 UTC
This is fixed in the current OSP9 build.

Comment 6 lkuchlan 2016-09-28 12:34:05 UTC
Tested using:
openstack-cinder-8.1.1-1.el7ost.noarch
python-cinderclient-1.6.0-1.el7ost.noarch
python-cinder-8.1.1-1.el7ost.noarch

Verification flow:

[root@cougar09 ~(keystone_admin)]# cinder create 1 --volume-type iscsi --name vol1
+--------------------------------+--------------------------------------+
|            Property            |                Value                 |
+--------------------------------+--------------------------------------+
|          attachments           |                  []                  |
|       availability_zone        |                 nova                 |
|            bootable            |                false                 |
|      consistencygroup_id       |                 None                 |
|           created_at           |      2016-09-28T12:30:43.000000      |
|          description           |                 None                 |
|           encrypted            |                False                 |
|               id               | 3d49d42e-38ac-4ccc-9405-79c5c3fa03d7 |
|            metadata            |                  {}                  |
|        migration_status        |                 None                 |
|          multiattach           |                False                 |
|              name              |                 vol1                 |
|     os-vol-host-attr:host      |                 None                 |
| os-vol-mig-status-attr:migstat |                 None                 |
| os-vol-mig-status-attr:name_id |                 None                 |
|  os-vol-tenant-attr:tenant_id  |   14873d81838244f68f0d5c8ce904bcd9   |
|       replication_status       |               disabled               |
|              size              |                  1                   |
|          snapshot_id           |                 None                 |
|          source_volid          |                 None                 |
|             status             |               creating               |
|           updated_at           |                 None                 |
|            user_id             |   778719f70e21481bb2e4cfaacb4eebbd   |
|          volume_type           |                iscsi                 |
+--------------------------------+--------------------------------------+

[root@cougar09 ~(keystone_admin)]# cinder show 3d49d42e-38ac-4ccc-9405-79c5c3fa03d7
+--------------------------------+-----------------------------------------+
|            Property            |                  Value                  |
+--------------------------------+-----------------------------------------+
|          attachments           |                    []                   |
|       availability_zone        |                   nova                  |
|            bootable            |                  false                  |
|      consistencygroup_id       |                   None                  |
|           created_at           |        2016-09-28T12:30:43.000000       |
|          description           |                   None                  |
|           encrypted            |                  False                  |
|               id               |   3d49d42e-38ac-4ccc-9405-79c5c3fa03d7  |
|            metadata            |                    {}                   |
|        migration_status        |                   None                  |
|          multiattach           |                  False                  |
|              name              |                   vol1                  |
|     os-vol-host-attr:host      | cougar09.scl.lab.tlv.redhat.com@lvm#lvm |
| os-vol-mig-status-attr:migstat |                   None                  |
| os-vol-mig-status-attr:name_id |                   None                  |
|  os-vol-tenant-attr:tenant_id  |     14873d81838244f68f0d5c8ce904bcd9    |
|       replication_status       |                 disabled                |
|              size              |                    1                    |
|          snapshot_id           |                   None                  |
|          source_volid          |                   None                  |
|             status             |                available                |
|           updated_at           |        2016-09-28T12:30:45.000000       |
|            user_id             |     778719f70e21481bb2e4cfaacb4eebbd    |
|          volume_type           |                  iscsi                  |
+--------------------------------+-----------------------------------------+

[root@cougar09 ~(keystone_admin)]# cinder retype 3d49d42e-38ac-4ccc-9405-79c5c3fa03d7 netapp --migration-policy on-demand

[root@cougar09 ~(keystone_admin)]# cinder show 3d49d42e-38ac-4ccc-9405-79c5c3fa03d7
+--------------------------------+-----------------------------------------+
|            Property            |                  Value                  |
+--------------------------------+-----------------------------------------+
|          attachments           |                    []                   |
|       availability_zone        |                   nova                  |
|            bootable            |                  false                  |
|      consistencygroup_id       |                   None                  |
|           created_at           |        2016-09-28T12:30:43.000000       |
|          description           |                   None                  |
|           encrypted            |                  False                  |
|               id               |   3d49d42e-38ac-4ccc-9405-79c5c3fa03d7  |
|            metadata            |                    {}                   |
|        migration_status        |                   None                  |
|          multiattach           |                  False                  |
|              name              |                   vol1                  |
|     os-vol-host-attr:host      | cougar09.scl.lab.tlv.redhat.com@lvm#lvm |
| os-vol-mig-status-attr:migstat |                   None                  |
| os-vol-mig-status-attr:name_id |                   None                  |
|  os-vol-tenant-attr:tenant_id  |     14873d81838244f68f0d5c8ce904bcd9    |
|       replication_status       |                 disabled                |
|              size              |                    1                    |
|          snapshot_id           |                   None                  |
|          source_volid          |                   None                  |
|             status             |                available                |
|           updated_at           |        2016-09-28T12:32:01.000000       |
|            user_id             |     778719f70e21481bb2e4cfaacb4eebbd    |
|          volume_type           |                  netapp                 |
+--------------------------------+-----------------------------------------+

Comment 8 errata-xmlrpc 2016-10-05 19:14:46 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-2030.html