RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 2145225 - multipath doesn't verify the argument count in config option strings it passes to the kernel
Summary: multipath doesn't verify the argument count in config option strings it passe...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 9
Classification: Red Hat
Component: device-mapper-multipath
Version: 9.2
Hardware: x86_64
OS: Linux
unspecified
medium
Target Milestone: rc
: 9.2
Assignee: Ben Marzinski
QA Contact: Lin Li
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-11-23 14:56 UTC by Lin Li
Modified: 2023-05-09 10:15 UTC (History)
6 users (show)

Fixed In Version: device-mapper-multipath-0.8.7-17.el9
Doc Type: Bug Fix
Doc Text:
Cause: Multipath doesn't verify that the argument count in the "features", "path_selector", and "hardware_handler" option strings it passes to the kernel. If users don't match the count to the actual number of arguments, the kernel will fail to parse the device table Consequence: Multipath will be unable to create or reload devices with an incorrect argument count, and will not warn users what the problem is. Fix: multipath now verifies the argument count of configuration strings, and warns users when the count doesn't match the number of arguments. Result: Users now are notified why their devices aren't created or reloaded, when they have an invalid argument count in their configuration.
Clone Of:
Environment:
Last Closed: 2023-05-09 08:14:07 UTC
Type: Bug
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHELPLAN-140303 0 None None None 2022-11-23 15:04:04 UTC
Red Hat Product Errata RHSA-2023:2459 0 None None None 2023-05-09 08:14:30 UTC

Description Lin Li 2022-11-23 14:56:36 UTC 
Description of problem:
multipath does not print any error or warning for invalid features value

Version-Release number of selected component (if applicable):
device-mapper-multipath-0.8.7-15.el9

How reproducible:
always

Steps to Reproduce:
1.[root@storageqe-05 ~]# rpm -qa | grep multipath
device-mapper-multipath-libs-0.8.7-15.el9.x86_64
device-mapper-multipath-0.8.7-15.el9.x86_64

2.[root@storageqe-05 ~]# multipath -ll
360a98000324669436c2b45666c567865 dm-3 NETAPP,LUN
size=2.0G features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 alua' wp=rw
`-+- policy='service-time 0' prio=30 status=active
  |- 1:0:0:0 sdj 8:144 active ready running
  |- 1:0:1:0 sdn 8:208 active ready running
  |- 4:0:0:0 sdb 8:16  active ready running
  `- 4:0:1:0 sdf 8:80  active ready running
360a98000324669436c2b45666c567867 dm-4 NETAPP,LUN
size=2.0G features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 alua' wp=rw
`-+- policy='service-time 0' prio=30 status=active
  |- 1:0:0:1 sdk 8:160 active ready running
  |- 1:0:1:1 sdo 8:224 active ready running
  |- 4:0:0:1 sdc 8:32  active ready running
  `- 4:0:1:1 sdg 8:96  active ready running
360a98000324669436c2b45666c567869 dm-5 NETAPP,LUN
size=2.0G features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 alua' wp=rw
`-+- policy='service-time 0' prio=30 status=active
  |- 1:0:0:2 sdl 8:176 active ready running
  |- 1:0:1:2 sdp 8:240 active ready running
  |- 4:0:0:2 sdd 8:48  active ready running
  `- 4:0:1:2 sdh 8:112 active ready running
360a98000324669436c2b45666c56786b dm-6 NETAPP,LUN
size=2.0G features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 alua' wp=rw
`-+- policy='service-time 0' prio=30 status=active
  |- 1:0:0:3 sdm 8:192 active ready running
  |- 1:0:1:3 sdq 65:0  active ready running
  |- 4:0:0:3 sde 8:64  active ready running
  `- 4:0:1:3 sdi 8:128 active ready running

3.Edit /etc/multipath.conf
[root@storageqe-05 ~]# cat /etc/multipath.conf 
# device-mapper-multipath configuration file

# For a complete list of the default configuration values, run either:
# # multipath -t
# or
# # multipathd show config

# For a list of configuration options with descriptions, see the
# multipath.conf man page.

defaults {
	user_friendly_names yes
	find_multipaths yes
}

blacklist {
}

devices {
    device {
        vendor "NETAPP"
        product "LUN"
        path_grouping_policy multibus
        path_checker tur
        features "3 no_path_retry queue pg_init_retries 50"    <--------------
        hardware_handler 0
        prio ontap
        failback immediate
        rr_weight uniform
        rr_min_io 128
        flush_on_last_del yes
        dev_loss_tmo infinity
        retain_attached_hw_handler yes
        detect_prio yes
    }
}

4.[root@storageqe-05 ~]# service multipathd reload
Redirecting to /bin/systemctl reload multipathd.service

5.[root@storageqe-05 ~]# multipath -ll

Actual results:
The output is empty in step5. multipath does not print any error or warning for invalid features value

Expected results:
multipath prints an error or warning for invalid features value

Additional info:
Comment 3 Ben Marzinski 2022-12-21 01:11:07 UTC 
I'm not sure that multipath should validate the string arguments that it passes directly to the kernel.  This would cause multipath to be unable to use new device-mapper capabilities, without updating the userspace tools. These patches are unlikely to be acceptable upstream, where multipath has to work correctly on a variety of kernels. But what it should do is make sure that the argument count matches the actual number of arguments.  If this is off, it messes up the kernel's table parsing, and causes it fail with useless error messages.

I'm using this bug to make multipath validate the argument count in the "features", "hardware_handler", and "path_selector" configuration options. In the values for each of those options, multipath will make sure that the count argument matches the number of arguments that follow it.
Comment 9 Lin Li 2023-01-28 09:48:24 UTC 
Verified on device-mapper-multipath-0.8.7-17.el9
[root@storageqe-25 ~]# rpm -qa | grep multipath
device-mapper-multipath-libs-0.8.7-17.el9.x86_64
device-mapper-multipath-0.8.7-17.el9.x86_64

[root@storageqe-25 ~]# multipath -ll
360a98000324669436c2b45666c567858 dm-3 NETAPP,LUN
size=2.0G features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 alua' wp=rw
|-+- policy='service-time 0' prio=50 status=active
| |- 7:0:1:0 sdf 8:80  active ready running
| `- 8:0:1:0 sdn 8:208 active ready running
`-+- policy='service-time 0' prio=10 status=enabled
  |- 7:0:0:0 sdb 8:16  active ready running
  `- 8:0:0:0 sdj 8:144 active ready running
360a98000324669436c2b45666c56785a dm-4 NETAPP,LUN
size=2.0G features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 alua' wp=rw
|-+- policy='service-time 0' prio=50 status=active
| |- 7:0:1:1 sdg 8:96  active ready running
| `- 8:0:1:1 sdo 8:224 active ready running
`-+- policy='service-time 0' prio=10 status=enabled
  |- 7:0:0:1 sdc 8:32  active ready running
  `- 8:0:0:1 sdk 8:160 active ready running
360a98000324669436c2b45666c567861 dm-5 NETAPP,LUN
size=2.0G features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 alua' wp=rw
|-+- policy='service-time 0' prio=50 status=active
| |- 7:0:1:2 sdh 8:112 active ready running
| `- 8:0:1:2 sdp 8:240 active ready running
`-+- policy='service-time 0' prio=10 status=enabled
  |- 7:0:0:2 sdd 8:48  active ready running
  `- 8:0:0:2 sdl 8:176 active ready running
360a98000324669436c2b45666c567863 dm-6 NETAPP,LUN
size=2.0G features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 alua' wp=rw
|-+- policy='service-time 0' prio=50 status=active
| |- 7:0:1:3 sdi 8:128 active ready running
| `- 8:0:1:3 sdq 65:0  active ready running
`-+- policy='service-time 0' prio=10 status=enabled
  |- 7:0:0:3 sde 8:64  active ready running
  `- 8:0:0:3 sdm 8:192 active ready running

Edit /etc/multipath.conf 
[root@storageqe-25 ~]# cat /etc/multipath.conf 
# device-mapper-multipath configuration file

# For a complete list of the default configuration values, run either:
# # multipath -t
# or
# # multipathd show config

# For a list of configuration options with descriptions, see the
# multipath.conf man page.

defaults {
	user_friendly_names yes
	find_multipaths yes
}

blacklist {
}

devices {
    device {
        vendor "NETAPP"
        product "LUN"
        path_grouping_policy multibus
        path_checker tur
        features "3 no_path_retry queue pg_init_retries 50"    <--------------
        hardware_handler 0
        prio ontap
        failback immediate
        rr_weight uniform
        rr_min_io 128
        flush_on_last_del yes
        dev_loss_tmo infinity
        retain_attached_hw_handler yes
        detect_prio yes
    }
}

[root@storageqe-25 ~]# service multipathd reload
Redirecting to /bin/systemctl reload multipathd.service

[root@storageqe-25 ~]# multipath -ll
16455.179235 | /etc/multipath.conf line 25, invalid argument count for features:, got '3' expected '4'   <----------
360a98000324669436c2b45666c567858 dm-3 NETAPP,LUN
size=2.0G features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 alua' wp=rw
`-+- policy='service-time 0' prio=30 status=active
  |- 7:0:0:0 sdb 8:16  active ready running
  |- 7:0:1:0 sdf 8:80  active ready running
  |- 8:0:0:0 sdj 8:144 active ready running
  `- 8:0:1:0 sdn 8:208 active ready running
360a98000324669436c2b45666c56785a dm-4 NETAPP,LUN
size=2.0G features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 alua' wp=rw
`-+- policy='service-time 0' prio=30 status=active
  |- 7:0:0:1 sdc 8:32  active ready running
  |- 7:0:1:1 sdg 8:96  active ready running
  |- 8:0:0:1 sdk 8:160 active ready running
  `- 8:0:1:1 sdo 8:224 active ready running
360a98000324669436c2b45666c567861 dm-5 NETAPP,LUN
size=2.0G features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 alua' wp=rw
`-+- policy='service-time 0' prio=30 status=active
  |- 7:0:0:2 sdd 8:48  active ready running
  |- 7:0:1:2 sdh 8:112 active ready running
  |- 8:0:0:2 sdl 8:176 active ready running
  `- 8:0:1:2 sdp 8:240 active ready running
360a98000324669436c2b45666c567863 dm-6 NETAPP,LUN
size=2.0G features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 alua' wp=rw
`-+- policy='service-time 0' prio=30 status=active
  |- 7:0:0:3 sde 8:64  active ready running
  |- 7:0:1:3 sdi 8:128 active ready running
  |- 8:0:0:3 sdm 8:192 active ready running
  `- 8:0:1:3 sdq 65:0  active ready running


Test result: Users now are notified why their devices aren't created or reloaded, when they have an invalid argument count in their configuration.
Comment 11 errata-xmlrpc 2023-05-09 08:14:07 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: device-mapper-multipath security and bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2023:2459
Note You need to log in before you can comment on or make changes to this bug.