Bug 1562388

Summary: [RFE] Allow 'max_data_area_mb' property to be configured per-LUN
Product: [Red Hat Storage] Red Hat Ceph Storage Reporter: Jason Dillaman <jdillama>
Component: iSCSIAssignee: Mike Christie <mchristi>
Status: CLOSED ERRATA QA Contact: Madhavi Kasturi <mkasturi>
Severity: medium Docs Contact: Aron Gunn <agunn>
Priority: medium    
Version: 3.0CC: agunn, ceph-eng-bugs, ceph-qe-bugs, hnallurv, jbrier, jdillama, mchristi, sostapov, tchandra, vshankar
Target Milestone: rcKeywords: FutureFeature
Target Release: 3.1   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: ceph-iscsi-config-2.6-1.el7cp ceph-iscsi-cli-2.7-5.el7cp Doc Type: Enhancement
Doc Text:
.The `max_data_area_mb` option is configurable per-LUN Previously, the amount of memory the kernel used to pass SCSI command data to `tcmu-runner` was hard coded to 8MB. The hard coded limit was too small for many workloads and resulted in reduced throughput and/or TASK SET FULL errors filling initiator side logs. This can now be configured by setting the `max_data_area_mb` value with `gwcli`. Information on the new setting and command can be found in the {product} link:{block-dev-guide}#configuring_the_iscsi_target_using_the_command_line_interface[Block Device Guide].
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-09-26 18:19:46 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1584264    

Description Jason Dillaman 2018-03-30 13:37:21 UTC
Description of problem:
The 'max_data_area_mb' property controls the ring-buffer space allocated to a particular LUN. LUNs w/ high IOPS/throughput might require a larger maximum size so it should be configurable.

Comment 4 Harish NV Rao 2018-04-11 12:49:57 UTC
@Jason, can you please share any specific set of steps to verify the fix?

Comment 5 Jason Dillaman 2018-04-11 13:41:01 UTC
The configfs info path for the backing image should reflect the change under the "MaxDataAreaMB" property:

# cat /sys/kernel/config/target/core/user_0/block/info
Status: ACTIVATED Max Queue Depth: 0 SectorSize: 512 HwMaxSectors: 128
Config: rbd/XYZ Size: 1073741824 MaxDataAreaMB: 8

Comment 6 Mike Christie 2018-04-11 19:01:36 UTC
When it is done, we will have a gwcli command where you set the MaxDataAreaMB or pass it in during image/lun setup. I have not had time to redo my ceph-iscsi-cli/config patch for what went upstream in the rtslib link above. Will update with the final gwcli command when that is done.

You would then just check it is set like Jason wrote above.

Comment 12 Mike Christie 2018-08-08 15:23:49 UTC
This is working as designed. We cannot modify the max_data_area_mb setting while it is exported to a LUN.

Comment 13 Mike Christie 2018-08-08 15:26:16 UTC
@mkasturi

Oh yeah, this was the one issues that we were going to add documentation for in RHCS guide in the new sections about managing devices after setup, because the target ones you can reconfig on the fly but some devices ones you cannot due to how the kernel is managing the refcounts and buffers.

Comment 14 Mike Christie 2018-08-14 04:18:57 UTC
Talked to mkasturi and putting this back on me. We need this fix:

https://github.com/ceph/ceph-iscsi-cli/pull/114

to allow reconfig max_data_area to work.

Comment 15 Mike Christie 2018-08-14 04:25:52 UTC
@mkasturi

Here is how you use the command:

If you have not done disk add for rbd.esx-test1 then you can run:

/disks> reconfigure image_id=rbd.esx-test1 attribute=max_data_area_mb value=128
/disks/rbd.esx-test1> reconfigure attribute=max_data_area_mb value=128

any time.

If you have run disk add for that disk, then you have to log the initiator out then run reconfigure. You could also run the disk remove command, but in the docs we will advise users to not do that because it can cause the initiator to remove the LUN.

You can also set max_data_area_mb at creation time like:

/disks> create image=esx-test1 pool=rbd size=10G max_data_area 128

Comment 20 errata-xmlrpc 2018-09-26 18:19:46 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:2819