Description of problem: cinder-volume container needs to have libcgroup-tools for volume_copy_bps_limit The current cinder-volume container doesn't have a required command for volume_copy_bps_limit option. That means the user cannot use the functionality. The issue found at RHOSP13 but RHOSP16 is also affected. Version-Release number of selected component (if applicable): Current tripleo-common in OSP13 and OSP16. How reproducible: Always. Steps to Reproduce: 1. Create overcloud with infrared. 2. Add volume_copy_bps_limit in cinder.conf. 3. Create a volume from an image. Actual results: cinder-volume threw an exception due to the lack of cgcreate command provide by libcgroup-tools. Expected results: cinder-volume can use cgcreate command. Additional info:
In upstream discussion, Alex pointed out that it should be fixed in cinder side. I proposed a patch to cinder, https://review.opendev.org/c/openstack/cinder/+/795009 Once the patch is merged, I'll change the component to cinder.
(In reply to Keigo Noha from comment #3) > In upstream discussion, Alex pointed out that it should be fixed in cinder > side. > I proposed a patch to cinder, > https://review.opendev.org/c/openstack/cinder/+/795009 > Once the patch is merged, I'll change the component to cinder. The change is already cinder. Also, this needs to be fixed in the packaging (see the RDO patch).
Hi Luigi, Is it possible to backport the RDO patch into RHOSP13? Our customer would like to use the cinder's option but it cannot be usable now. It will be great if we can ship the fix into RHOSP13. Kind Regards, Keigo Noha
(In reply to Keigo Noha from comment #6) > Hi Luigi, > > Is it possible to backport the RDO patch into RHOSP13? > Our customer would like to use the cinder's option but it cannot be usable > now. > > It will be great if we can ship the fix into RHOSP13. I understand but I can't answer that. OSP13 is moving into the extended lifecycle support (ELS) phase in 3 days (June 27): https://access.redhat.com/support/policy/updates/openstack/platform After that, only selected bugs will be addressed, quoting from the page above: "Same as Maintenance Support, with a change of: Select Bug Fix Errata (RHBA) only. Qualified Critical and Important Security Advisories (RHSAs) only. Valid for existing environments only. Cannot be applied to new installations of the product on hardware or in combination with technologies which have not been certified with the version of the product, or its underlying RHEL components, as they were at the end of Maintenance Support. It is required to have the valid ELS subscriptions to receive support from Red Hat. "
Hi Luigi, I understand that point. On the other hand, the option cannot be used in RHOSP13. We haven't had a report our customer about it. But lacking this feature is severe to the customer who wants to reduce the I/O workload to conversion directory. It will be great if we can ship the feature in RHOSP13. Is there any way to put the fix into RHOSP13? Kind Regards, Keigo Noha
Verified on: openstack-cinder-15.6.1-2.20220112174913.c093eda.el8ost.noarch Confirming Cinder-volume container now contains libcgroup-tools [root@controller-2 ~]# podman exec -it openstack-cinder-volume-podman-0 rpm -qa | grep libcgroup-tools libcgroup-tools-0.41-19.el8.x86_64 On cinder.conf [tripleo_iscsi] backend_host=hostgroup volume_backend_name=tripleo_iscsi volume_driver=cinder.volume.drivers.lvm.LVMVolumeDriver target_ip_address=172.17.3.71 target_helper=lioadm volumes_dir=/var/lib/cinder/volumes target_protocol=iscsi volume_copy_bps_limit=104857600 ---> Added limit, restarted c-vol Created a volume from an image (overcloud) [stack@undercloud-0 ~]$ cinder create 4 --image cirros --name VolFromImageWithCopyBpsLimitSet +--------------------------------+--------------------------------------+ | Property | Value | +--------------------------------+--------------------------------------+ | attachments | [] | | availability_zone | nova | | bootable | false | | consistencygroup_id | None | | created_at | 2022-02-06T15:46:48.000000 | | description | None | | encrypted | False | | id | 50fc026c-5722-4f50-aa92-9df630bb2fa1 | | metadata | {} | | migration_status | None | | multiattach | False | | name | VolFromImageWithCopyBpsLimitSet | | os-vol-host-attr:host | None | | os-vol-mig-status-attr:migstat | None | | os-vol-mig-status-attr:name_id | None | | os-vol-tenant-attr:tenant_id | e07efcd0337e4d5eb28345c068695f84 | | replication_status | None | | size | 4 | | snapshot_id | None | | source_volid | None | | status | creating | | updated_at | None | | user_id | 1c242ea20ca040d5b4d3af171142119d | | volume_type | tripleo | +--------------------------------+--------------------------------------+ (overcloud) [stack@undercloud-0 ~]$ cinder list +--------------------------------------+-----------+---------------------------------+------+-------------+----------+--------------------------------------+ | ID | Status | Name | Size | Volume Type | Bootable | Attached to | +--------------------------------------+-----------+---------------------------------+------+-------------+----------+--------------------------------------+ | 50fc026c-5722-4f50-aa92-9df630bb2fa1 | available | VolFromImageWithCopyBpsLimitSet | 4 | tripleo | true | | This works, but lets retry which a larger image so that the limit has a chance to actually kick in. Uploaded rhel image which has a decent size of ~850mb (overcloud) [stack@undercloud-0 ~]$ glance image-create --name rhel7.9 --disk-format qcow2 --container-format bare --file rhel-server-7.9-update-9-x86_64-kvm.qcow2 --progress [=============================>] 100% +------------------+----------------------------------------------------------------------------------+ | Property | Value | +------------------+----------------------------------------------------------------------------------+ | checksum | 82dea14f2fac8b27009c5f926f885f5a | | container_format | bare | | created_at | 2022-02-06T16:07:38Z | | direct_url | swift+config://ref1/glance/11c53f83-31cf-43f1-b6aa-eca3d06f6732 | | disk_format | qcow2 | | id | 11c53f83-31cf-43f1-b6aa-eca3d06f6732 | | min_disk | 0 | | min_ram | 0 | | name | rhel7.9 | | os_hash_algo | sha512 | | os_hash_value | 5076869a5a0bed649fbfcb3373d1d85090bc315c1c9ccf714307dbe20b2b52f6af9ebc1657f0d3c0 | | | 2dd11775966d67165f3530a8fb176b0ddfccc850059fbb31 | | os_hidden | False | | owner | e07efcd0337e4d5eb28345c068695f84 | | protected | False | | size | 845833216 | | status | active | | stores | default_backend | | tags | [] | | updated_at | 2022-02-06T16:07:43Z | | virtual_size | Not available | | visibility | shared | +------------------+----------------------------------------------------------------------------------+ Now lets create a new volume from rhel image: (overcloud) [stack@undercloud-0 ~]$ cinder create 10 --image rhel7.9 --name volfromrhelimage +--------------------------------+--------------------------------------+ | Property | Value | +--------------------------------+--------------------------------------+ | attachments | [] | | availability_zone | nova | | bootable | false | | consistencygroup_id | None | | created_at | 2022-02-06T16:10:00.000000 | | description | None | | encrypted | False | | id | e3258b9e-804a-4dc5-9024-f9b73ba93150 | | metadata | {} | | migration_status | None | | multiattach | False | | name | volfromrhelimage | | os-vol-host-attr:host | None | | os-vol-mig-status-attr:migstat | None | | os-vol-mig-status-attr:name_id | None | | os-vol-tenant-attr:tenant_id | e07efcd0337e4d5eb28345c068695f84 | | replication_status | None | | size | 10 | | snapshot_id | None | | source_volid | None | | status | creating | | updated_at | None | | user_id | 1c242ea20ca040d5b4d3af171142119d | | volume_type | tripleo | +--------------------------------+--------------------------------------+ (overcloud) [stack@undercloud-0 ~]$ cinder list +--------------------------------------+-----------+------------------+------+-------------+----------+-------------+ | ID | Status | Name | Size | Volume Type | Bootable | Attached to | +--------------------------------------+-----------+------------------+------+-------------+----------+-------------+ | e3258b9e-804a-4dc5-9024-f9b73ba93150 | available | volfromrhelimage | 10 | tripleo | true | | +--------------------------------------+-----------+------------------+------+-------------+----------+-------------+ A volume is successfully created, plus we also noticed throttling on cinder volume log: [root@controller-2 ~]# grep -irn cgset /var/log/containers/cinder/cinder-volume.log 10963:2022-02-06 15:46:50.275 61 DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cgset -r blkio.throttle.read_bps_device=252:0 104857600 cinder-volume-copy execute /usr/lib/python3.6/site-packages/oslo_concurrency/processutils.py:379 10964:2022-02-06 15:46:50.285 61 DEBUG oslo_concurrency.processutils [-] CMD "cgset -r blkio.throttle.read_bps_device=252:0 104857600 cinder-volume-copy" returned: 0 in 0.011s execute /usr/lib/python3.6/site-packages/oslo_concurrency/processutils.py:416 10966:2022-02-06 15:46:50.287 61 DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cgset -r blkio.throttle.write_bps_device=253:6 104857600 cinder-volume-copy execute /usr/lib/python3.6/site-packages/oslo_concurrency/processutils.py:379 10967:2022-02-06 15:46:50.294 61 DEBUG oslo_concurrency.processutils [-] CMD "cgset -r blkio.throttle.write_bps_device=253:6 104857600 cinder-volume-copy" returned: 0 in 0.006s execute /usr/lib/python3.6/site-packages/oslo_concurrency/processutils.py:416 11400:2022-02-06 16:10:07.652 61 DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cgset -r blkio.throttle.read_bps_device=252:0 104857600 cinder-volume-copy execute /usr/lib/python3.6/site-packages/oslo_concurrency/processutils.py:379 11401:2022-02-06 16:10:07.657 61 DEBUG oslo_concurrency.processutils [-] CMD "cgset -r blkio.throttle.read_bps_device=252:0 104857600 cinder-volume-copy" returned: 0 in 0.005s execute /usr/lib/python3.6/site-packages/oslo_concurrency/processutils.py:416 11403:2022-02-06 16:10:07.659 61 DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cgset -r blkio.throttle.write_bps_device=253:4 104857600 cinder-volume-copy execute /usr/lib/python3.6/site-packages/oslo_concurrency/processutils.py:379 11404:2022-02-06 16:10:07.664 61 DEBUG oslo_concurrency.processutils [-] CMD "cgset -r blkio.throttle.write_bps_device=253:4 104857600 cinder-volume-copy" returned: 0 in 0.004s execute /usr/lib/python3.6/site-packages/oslo_concurrency/processutils.py:416 Good to verify, the previously missing packing is now included, which enables throttling of volume creation from image.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Release of components for Red Hat OpenStack Platform 16.2.2), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2022:1001