Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1883643

Summary: Using MD RAID on a cinder volume exposes the data to the compute host
Product: Red Hat OpenStack Reporter: David Vallee Delisle <dvd>
Component: openstack-tripleo-heat-templatesAssignee: Alan Bishop <abishop>
Status: CLOSED DEFERRED QA Contact: David Rosenfeld <drosenfe>
Severity: high Docs Contact:
Priority: high    
Version: 10.0 (Newton)CC: abishop, acanan, agk, akaris, astillma, broose, cschwede, eharney, gcharot, gfidente, jbrassow, jpittman, jraju, jvisser, jwaterwo, lmarsh, mabrams, marjones, mburns, nkshirsa, nsoffer, nwolf, pablo.iranzo, pgrist, pmorey, sclewis, scohen, sgotliv, spower, sputhenp, srevivo, teigland, tkajinam, tshefi, tvignaud, tvvcox, vcojot, vfarias, zkabelac
Target Milestone: ---Keywords: Triaged
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1261083 Environment:
Last Closed: 2020-11-04 15:20:04 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1261083, 1884225    
Bug Blocks: 1518969    

Description David Vallee Delisle 2020-09-29 18:51:32 UTC
Just like LVM in Bug #1261083 We have the same issue with MD Raids exposed via iSCSI block devices

Workaround is to echo "AUTO -ALL" | tee /etc/mdadm.conf or just add a valid DEVICE instruction.


From the case notes:
One of their tenant has a software RAID (md) on one of their cinder volume. The volume is connected to the hypervisor with iscsi/multipath.

The hypervisor was assembling the RAID as well as the instance because /etc/mdadm.conf had /dev/dm* as wildcard.

When the instance got stopped, we couldn't flush the multipath device because it was assembled my mdadm.

This led to a lot of faulty paths.


[1] instances with their block devices
[2] multipath status
[3] lsblk
[4] mdadm
[5] mdstat

We recommended to fix this issue like this:
root@compute-0 # mdadm --stop /dev/md125 /dev/md126 /dev/md127
root@compute-0 # multipath -f 3600601607b7036000aefd29a4aabe911
# Now let's validate, we shouldn't have anything here
root@compute-0 # multipath -ll

root@compute-1 # mdadm --stop /dev/md125 /dev/md126 /dev/md127
root@compute-1 # multipath -f 360060160aef0380067ec0d0f323eea11
# Now let's validate, we shouldn't have anything here
root@compute-1 # multipath -ll

If it doesn't clean the devices properly, it might be a good idea to simply reboot the compute node(s) instead of fighting the devices, we're going to save time.

After this, to prevent this issue from happening, on both computes:
echo "AUTO -all" | tee /etc/mdadm.conf 

Now their tenant is complaining that they can't assemble the RAID inside the VM [6]

We saw buffer errors and missing partition tables [7] on the hypervisor in the logs


[1]
~~~
overcloud-compute-0.localdomain/sos_commands/virsh/virsh_-r_dumpxml_instance-00004572:      <nova:name>INSTANCE-1</nova:name>
overcloud-compute-0.localdomain/sos_commands/virsh/virsh_-r_dumpxml_instance-00004572:      <source dev='/dev/dm-2'/>
overcloud-compute-0.localdomain/sos_commands/virsh/virsh_-r_dumpxml_instance-000049b8:      <nova:name>INSTANCE-2</nova:name>
overcloud-compute-0.localdomain/sos_commands/virsh/virsh_-r_dumpxml_instance-000049b8:      <source dev='/dev/dm-1'/>
overcloud-compute-1.localdomain/sos_commands/virsh/virsh_-r_dumpxml_instance-00004581:      <nova:name>INSTANCE-3</nova:name>
overcloud-compute-1.localdomain/sos_commands/virsh/virsh_-r_dumpxml_instance-00004581:      <source dev='/dev/dm-1'/>
overcloud-compute-1.localdomain/sos_commands/virsh/virsh_-r_dumpxml_instance-000049bb:      <nova:name>INSTANCE-4</nova:name>
overcloud-compute-1.localdomain/sos_commands/virsh/virsh_-r_dumpxml_instance-000049bb:      <source dev='/dev/dm-2'/>
~~~

[2]
~~~
$ grep -Pzo "[^\s]+ dm-[1-2] (.*|\n)*\nSep" */sos_commands/multipath/multipath_-v4_-ll
overcloud-compute-0.localdomain/sos_commands/multipath/multipath_-v4_-ll:3600601607b7036000aefd29a4aabe911 dm-1 DGC     ,VRAID           
size=1000G features='2 queue_if_no_path retain_attached_hw_handler' hwhandler='1 alua' wp=rw
|-+- policy='round-robin 0' prio=0 status=enabled
| |- 8:0:0:86  sdq  65:0   failed faulty running
| |- 5:0:0:86  sdn  8:208  failed faulty running
| |- 7:0:0:86  sdp  8:240  failed faulty running
| `- 7:0:0:85  sdt  65:48  failed faulty running
|-+- policy='round-robin 0' prio=50 status=active
| |- 5:0:0:43  sday 67:32  active ready running
| |- 8:0:0:225 sdas 66:192 failed faulty running
| |- 5:0:0:214 sdau 66:224 failed faulty running
| |- 8:0:0:207 sdao 66:128 failed faulty running
| |- 5:0:0:225 sdap 66:144 failed faulty running
| |- 8:0:0:63  sdag 66:0   failed faulty running
| |- 5:0:0:207 sdal 66:80  failed faulty running
| |- 8:0:0:85  sdu  65:64  failed faulty running
| |- 5:0:0:63  sdad 65:208 failed faulty running
| |- 8:0:0:132 sdy  65:128 failed faulty running
| |- 5:0:0:58  sdz  65:144 failed faulty running
| |- 8:0:0:58  sdac 65:192 failed faulty running
| |- 5:0:0:132 sdv  65:80  failed faulty running
| |- 8:0:0:214 sdax 67:16  failed faulty running
| |- 5:0:0:85  sdr  65:16  failed faulty running
| `- 8:0:0:43  sdbb 67:80  active ready running
`-+- policy='round-robin 0' prio=10 status=enabled
  |- 7:0:0:43  sdba 67:64  active ready running
  |- 6:0:0:43  sdaz 67:48  active ready running
  |- 7:0:0:225 sdar 66:176 failed faulty running
  |- 6:0:0:214 sdav 66:240 failed faulty running
  |- 7:0:0:207 sdan 66:112 failed faulty running
  |- 6:0:0:225 sdaq 66:160 failed faulty running
  |- 7:0:0:63  sdaf 65:240 failed faulty running
  |- 6:0:0:207 sdam 66:96  failed faulty running
  |- 7:0:0:132 sdx  65:112 failed faulty running
  |- 6:0:0:63  sdae 65:224 failed faulty running
  |- 7:0:0:58  sdab 65:176 failed faulty running
  |- 6:0:0:132 sdw  65:96  failed faulty running
  |- 7:0:0:214 sdaw 67:0   failed faulty running
  |- 6:0:0:86  sdo  8:224  failed faulty running
  |- 6:0:0:85  sds  65:32  failed faulty running
  `- 6:0:0:58  sdaa 65:160 failed faulty running
overcloud-compute-1.localdomain/sos_commands/multipath/multipath_-v4_-ll:360060160aef0380067ec0d0f323eea11 dm-2 DGC     ,VRAID           
size=1000G features='2 queue_if_no_path retain_attached_hw_handler' hwhandler='1 alua' wp=rw
|-+- policy='round-robin 0' prio=50 status=active
| |- 9:0:0:38  sdo 8:224 active ready running
| |- 10:0:0:20 sdn 8:208 failed faulty running
| |- 9:0:0:20  sdk 8:160 failed faulty running
| `- 10:0:0:38 sdr 65:16 active ready running
`-+- policy='round-robin 0' prio=10 status=enabled
  |- 12:0:0:38 sdq 65:0  active ready running
  |- 11:0:0:38 sdp 8:240 active ready running
  |- 12:0:0:20 sdm 8:192 failed faulty running
  `- 11:0:0:20 sdl 8:176 failed faulty running
~~~

[3]
~~~
$ cat */sos_commands/block/lsblk_-t
NAME                                ALIGNMENT MIN-IO  OPT-IO PHY-SEC LOG-SEC ROTA SCHED    RQ-SIZE   RA WSAME
sda                                         0 262144  262144     512     512    1 deadline     128 4096    0B
|-sda1                                      0 262144  262144     512     512    1 deadline     128 4096    0B
`-sda2                                      0 262144  262144     512     512    1 deadline     128 4096    0B
sdba                                        0   8192 4194304     512     512    1 deadline     128  128   32M
`-3600601607b7036000aefd29a4aabe911         0   8192 4194304     512     512    1 deadline     128  128   32M
sdbb                                        0   8192 4194304     512     512    1 deadline     128 4096   32M
`-3600601607b7036000aefd29a4aabe911         0   8192 4194304     512     512    1 deadline     128  128   32M
sday                                        0   8192 4194304     512     512    1 deadline     128  128   32M
`-3600601607b7036000aefd29a4aabe911         0   8192 4194304     512     512    1 deadline     128  128   32M
sdaz                                        0   8192 4194304     512     512    1 deadline     128  128   32M
`-3600601607b7036000aefd29a4aabe911         0   8192 4194304     512     512    1 deadline     128  128   32M
NAME                                ALIGNMENT MIN-IO  OPT-IO PHY-SEC LOG-SEC ROTA SCHED    RQ-SIZE   RA WSAME
sda                                         0 262144  262144     512     512    1 deadline     128 4096    0B
|-sda1                                      0 262144  262144     512     512    1 deadline     128 4096    0B
`-sda2                                      0 262144  262144     512     512    1 deadline     128 4096    0B
sdg                                         0   8192 4194304     512     512    1 deadline     128  128   32M
sdh                                         0   8192 4194304     512     512    1 deadline     128  128   32M
sdi                                         0   8192 4194304     512     512    1 deadline     128  128   32M
sdj                                         0   8192 4194304     512     512    1 deadline     128  128   32M
sdo                                         0   8192 4194304     512     512    1 deadline     128  128   32M
`-360060160aef0380067ec0d0f323eea11         0   8192 4194304     512     512    1 deadline     128  128   32M
sdp                                         0   8192 4194304     512     512    1 deadline     128  128   32M
`-360060160aef0380067ec0d0f323eea11         0   8192 4194304     512     512    1 deadline     128  128   32M
sdq                                         0   8192 4194304     512     512    1 deadline     128  128   32M
`-360060160aef0380067ec0d0f323eea11         0   8192 4194304     512     512    1 deadline     128  128   32M
sdr                                         0   8192 4194304     512     512    1 deadline     128  128   32M
`-360060160aef0380067ec0d0f323eea11         0   8192 4194304     512     512    1 deadline     128  128   32M
360060160aef038006010311de8c5e911           0    512       0     512     512    1 deadline     128  128    0B
`-md127                                  4096   8192 4194304     512     512    1              128 4096    0B
~~~

[4]
~~~
$ more */sos_commands/md/mdadm* | cat
::::::::::::::
overcloud-compute-0.localdomain/sos_commands/md/mdadm_-D_.dev.md
::::::::::::::
/dev/md125:
           Version : 1.1
        Raid Level : raid0
     Total Devices : 0
       Persistence : Superblock is persistent

             State : inactive

    Number   Major   Minor   RaidDevice
/dev/md126:
           Version : 1.1
        Raid Level : raid0
     Total Devices : 1
       Persistence : Superblock is persistent

             State : inactive
   Working Devices : 1

              Name : disc
              UUID : 5914375c:972d27e0:fad157e6:e2b33ef4
            Events : 125751

    Number   Major   Minor   RaidDevice

       -     253        1        -        /dev/dm-1
/dev/md127:
           Version : 1.1
        Raid Level : raid0
     Total Devices : 1
       Persistence : Superblock is persistent

             State : inactive
   Working Devices : 1

    Number   Major   Minor   RaidDevice

       -     253        0        -        /dev/dm-0
::::::::::::::
overcloud-compute-1.localdomain/sos_commands/md/mdadm_-D_.dev.md
::::::::::::::
mdadm: /dev/md does not appear to be an md device
/dev/md125:
           Version : 1.1
        Raid Level : raid0
     Total Devices : 1
       Persistence : Superblock is persistent

             State : inactive
   Working Devices : 1

              Name : disc
              UUID : be5856a7:2fd7fd53:50575d9c:7b870d57
            Events : 931568

    Number   Major   Minor   RaidDevice

       -     253        2        -        /dev/dm-2
/dev/md126:
           Version : 1.1
        Raid Level : raid0
     Total Devices : 0
       Persistence : Superblock is persistent

             State : inactive

    Number   Major   Minor   RaidDevice
/dev/md127:
           Version : 1.1
     Creation Time : Fri Aug 23 22:01:52 2019
        Raid Level : raid1
        Array Size : 1047851563 (999.31 GiB 1073.00 GB)
     Used Dev Size : 1047851563 (999.31 GiB 1073.00 GB)
      Raid Devices : 2
     Total Devices : 1
       Persistence : Superblock is persistent

     Intent Bitmap : Internal

       Update Time : Sat Aug 24 17:56:45 2019
             State : clean, degraded 
    Active Devices : 1
   Working Devices : 1
    Failed Devices : 0
     Spare Devices : 0

Consistency Policy : bitmap

    Number   Major   Minor   RaidDevice State
       0     253        0        0      active sync   /dev/dm-0
       -       0        0        1      removed
~~~

[5]
~~~
$ more */proc/mdstat| cat
::::::::::::::
overcloud-compute-0.localdomain/proc/mdstat
::::::::::::::
Personalities : 
md125 : inactive sdak[2](S)
      1048575868 blocks super 1.1
       
md126 : inactive dm-1[0](S)
      1048575868 blocks super 1.1
       
md127 : inactive dm-0[0](S)
      1048575868 blocks super 1.1
       
unused devices: <none>
::::::::::::::
overcloud-compute-1.localdomain/proc/mdstat
::::::::::::::
Personalities : [raid1] 
md125 : inactive dm-2[3](S)
      1048575868 blocks super 1.1
       
md126 : inactive sdf[2](S)
      1048575868 blocks super 1.1
       
md127 : active (auto-read-only) raid1 dm-0[0]
      1047851563 blocks super 1.1 [2/1] [U_]
      bitmap: 33/500 pages [132KB], 1024KB chunk

unused devices: <none>
~~~

[6]
~~~
[local]INSTANCE--00# show alarm outstanding
Thursday September 24 10:00:40 UTC 2020
Sev Object          Event
--- ----------      --------------------------------------------------------------------------------------------------------------------------------------------------------------------
CR  Card 1          RAID not available, reason 'failed to read RAID state'
[local]INSTANCE--00# 

 

[local]INSTANCE--00# show hd raid
Thursday September 24 10:00:10 UTC 2020
HD RAID:
  State                     : Not available
[local]INSTANCE--00# show hd raid verbose
Thursday September 24 10:00:18 UTC 2020
HD RAID:
  State                     : RAID not ready
[local]INSTANCE--00# 

 

 

Thu Sep 24 06:22:26 2020 Internal trap notification 1076 (StorageFailed) Storage device hd-local1 failed slot 1
Thu Sep 24 06:22:26 2020 Internal trap notification 1079 (RaidDegraded) Raid array hd-raid degraded; slot/device 1/hd-local1 unavailable
Thu Sep 24 06:24:04 2020 Internal trap notification 1313 (StatFilesizeClear) Bulkstat filesize threshold limit 6144000  measured value 274
Thu Sep 24 06:30:53 2020 Internal trap notification 52 (CLISessionStart) user staradmin privilege level Security Administrator ttyname /dev/pts/12
Thu Sep 24 06:30:55 2020 Internal trap notification 53 (CLISessionEnd) user staradmin privilege level Security Administrator ttyname /dev/pts/12
Thu Sep 24 06:47:24 2020 Internal trap notification 52 (CLISessionStart) user staradmin privilege level Security Administrator ttyname /dev/pts/12
Thu Sep 24 06:47:24 2020 Internal trap notification 1184 (TestModeEntered) context local user staradmin ttyname /dev/pts/12 address type IPV4 remote ip address 10.215.214.136
Thu Sep 24 06:48:02 2020 Internal trap notification 53 (CLISessionEnd) user staradmin privilege level Security Administrator ttyname /dev/pts/12
Thu Sep 24 06:56:09 2020 Internal trap notification 52 (CLISessionStart) user staradmin privilege level Security Administrator ttyname /dev/pts/12
Thu Sep 24 07:02:40 2020 Internal trap notification 1184 (TestModeEntered) context local user staradmin ttyname /dev/pts/12 address type IPV4 remote ip address 10.140.180.194
Thu Sep 24 07:03:13 2020 Internal trap notification 1077 (RaidFailed) Raid array hd-raid failed
Thu Sep 24 07:04:20 2020 Internal trap notification 1078 (RaidStarted) Raid array hd-raid started
Thu Sep 24 07:04:20 2020 Internal trap notification 1079 (RaidDegraded) Raid array hd-raid degraded; slot/device 1/hd-local1 unavailable
Thu Sep 24 07:06:05 2020 Internal trap notification 1312 (StatFilesizeExceeded) Bulkstat filesize threshold limit 6144000  measured value 6144049
Thu Sep 24 07:08:17 2020 Internal trap notification 1077 (RaidFailed) Raid array hd-raid failed
Thu Sep 24 07:08:17 2020 Internal trap notification 1077 (RaidFailed) Raid array hd-raid failed
Thu Sep 24 07:08:27 2020 Internal trap notification 1077 (RaidFailed) Raid array hd-raid failed
~~~

[7]
~~~
overcloud-compute-0.localdomain/sos_commands/logs/journalctl_--no-pager_--boot:Sep 23 20:29:46 overcloud-compute-0.localdomain kernel: buffer_io_error: 909 callbacks suppressed
overcloud-compute-0.localdomain/sos_commands/logs/journalctl_--no-pager_--boot:Sep 23 20:29:46 overcloud-compute-0.localdomain kernel: Buffer I/O error on dev dm-0, logical block 1, async page read
overcloud-compute-0.localdomain/sos_commands/logs/journalctl_--no-pager_--boot:Sep 23 20:29:47 overcloud-compute-0.localdomain kernel: Buffer I/O error on dev dm-0, logical block 1, async page read
overcloud-compute-0.localdomain/sos_commands/logs/journalctl_--no-pager_--boot:Sep 23 20:29:47 overcloud-compute-0.localdomain kernel: Buffer I/O error on dev dm-0, logical block 1, async page read
overcloud-compute-0.localdomain/sos_commands/logs/journalctl_--no-pager_--boot:Sep 23 20:29:47 overcloud-compute-0.localdomain kernel: Buffer I/O error on dev dm-0, logical block 1, async page read
overcloud-compute-0.localdomain/sos_commands/logs/journalctl_--no-pager_--boot:Sep 23 20:29:47 overcloud-compute-0.localdomain kernel: Buffer I/O error on dev dm-0, logical block 1, async page read
overcloud-compute-0.localdomain/sos_commands/logs/journalctl_--no-pager_--boot:Sep 23 20:29:47 overcloud-compute-0.localdomain kernel: Buffer I/O error on dev dm-0, logical block 1, async page read
overcloud-compute-0.localdomain/sos_commands/logs/journalctl_--no-pager_--boot:Sep 23 20:29:47 overcloud-compute-0.localdomain kernel: Buffer I/O error on dev dm-0, logical block 1, async page read
overcloud-compute-0.localdomain/sos_commands/logs/journalctl_--no-pager_--boot:Sep 23 20:29:47 overcloud-compute-0.localdomain kernel: Buffer I/O error on dev dm-0, logical block 1, async page read
overcloud-compute-0.localdomain/sos_commands/logs/journalctl_--no-pager_--boot:Sep 23 20:29:47 overcloud-compute-0.localdomain kernel: Buffer I/O error on dev dm-0, logical block 16, async page read
overcloud-compute-0.localdomain/sos_commands/logs/journalctl_--no-pager_--boot:Sep 23 20:29:47 overcloud-compute-0.localdomain kernel: Buffer I/O error on dev dm-0, logical block 8, async page read
overcloud-compute-1.localdomain/sos_commands/logs/journalctl_--no-pager_--boot:Sep 23 20:34:40 overcloud-compute-1.localdomain kernel: Buffer I/O error on dev sdl, logical block 0, async page read
overcloud-compute-1.localdomain/sos_commands/logs/journalctl_--no-pager_--boot:Sep 23 20:34:40 overcloud-compute-1.localdomain kernel: Buffer I/O error on dev sdl, logical block 0, async page read
overcloud-compute-1.localdomain/sos_commands/logs/journalctl_--no-pager_--boot:Sep 23 20:34:40 overcloud-compute-1.localdomain kernel: Buffer I/O error on dev sdm, logical block 0, async page read
overcloud-compute-1.localdomain/sos_commands/logs/journalctl_--no-pager_--boot:Sep 23 20:34:40 overcloud-compute-1.localdomain kernel: Buffer I/O error on dev sdm, logical block 0, async page read
overcloud-compute-1.localdomain/sos_commands/logs/journalctl_--no-pager_--boot:Sep 23 20:34:40 overcloud-compute-1.localdomain kernel: Buffer I/O error on dev md127, logical block 1047851456, async page read
overcloud-compute-1.localdomain/sos_commands/logs/journalctl_--no-pager_--boot:Sep 23 20:34:40 overcloud-compute-1.localdomain kernel: Buffer I/O error on dev md127, logical block 1047851457, async page read
overcloud-compute-1.localdomain/sos_commands/logs/journalctl_--no-pager_--boot:Sep 23 20:34:40 overcloud-compute-1.localdomain kernel: Buffer I/O error on dev md127, logical block 1047851458, async page read
overcloud-compute-1.localdomain/sos_commands/logs/journalctl_--no-pager_--boot:Sep 23 20:34:41 overcloud-compute-1.localdomain kernel: Buffer I/O error on dev md127, logical block 1047851459, async page read
overcloud-compute-1.localdomain/sos_commands/logs/journalctl_--no-pager_--boot:Sep 23 20:34:41 overcloud-compute-1.localdomain kernel: Buffer I/O error on dev sdk, logical block 262143984, async page read
overcloud-compute-1.localdomain/sos_commands/logs/journalctl_--no-pager_--boot:Sep 23 20:34:41 overcloud-compute-1.localdomain kernel: Buffer I/O error on dev sdl, logical block 262143984, async page read
overcloud-compute-1.localdomain/sos_commands/logs/journalctl_--no-pager_--boot:Sep 23 20:34:41 overcloud-compute-1.localdomain kernel:  md127: unable to read partition table
~~~

Comment 1 Alan Bishop 2020-10-08 19:50:35 UTC
I'm setting up an env where I can study the behavior.

Comment 4 Alan Bishop 2020-10-13 12:48:45 UTC
Why all the private comments?

Thanks for the advice, folks! I now have an env in which I can reproduce the problem, and the next step is to run a series of experiments to determine the most effective way of filtering devices without breaking other things.

Comment 5 David Vallee Delisle 2020-10-13 22:47:40 UTC
Hello,

A concern was raised by the fact that we documented some possible filesystem corruption after adding AUTO -all in /etc/mdadm.conf. I revisited the sosreport and here's the logs and the presumed timeline.

[1] We had recurring "tur checker reports path is down" messages returned by multipath
[2] At some point, during the troubleshooting of the case, we ended up with these kernel messages, like if the underlying block devices were pulled out without proper unmounting.

It's worth nothing that the second batch of errors happened at the time the sosreport was taken, so it might be caused by a read operation like listing the partitions.

Also, regarding the 20:29 errors, this happened during a troubleshooting session. We did a lot of operations to try to flush these devices like dmsetup remove, multipath -f.

One last thing, at the time of taking sosreport on that specific compute node, I don't see /etc/mdadm.conf in there, and mdadm still has the device mapper devices [3]. I can only presume that /etc/mdadm.conf didn't include AUTO -all at the time the sosreport was taken.

I'm no storage expert, but at this point, I'd be surprised if "AUTO -all" would have caused said corruption.

Thanks,

DVD

[1]
~~~
2020-09-23T20:29:42.818521+00:00 overcloud-compute-0 multipathd: 3600601607b703600d3008296faabe911: sdj - tur checker reports path is down
2020-09-23T20:29:42.818708+00:00 overcloud-compute-0 multipathd: 3600601607b703600d3008296faabe911: sdm - tur checker reports path is down
2020-09-23T20:29:42.818888+00:00 overcloud-compute-0 multipathd: 3600601607b703600d3008296faabe911: sdk - tur checker reports path is down
2020-09-23T20:29:42.819137+00:00 overcloud-compute-0 multipathd: 3600601607b7036000aefd29a4aabe911: sdn - tur checker reports path is down
2020-09-23T20:29:42.819380+00:00 overcloud-compute-0 multipathd: 3600601607b7036000aefd29a4aabe911: sdq - tur checker reports path is down
2020-09-23T20:29:42.819621+00:00 overcloud-compute-0 multipathd: 3600601607b7036000aefd29a4aabe911: sdt - tur checker reports path is down
2020-09-23T20:29:42.819865+00:00 overcloud-compute-0 multipathd: 3600601607b7036000aefd29a4aabe911: sdr - tur checker reports path is down
2020-09-23T20:29:42.820089+00:00 overcloud-compute-0 multipathd: 3600601607b7036000aefd29a4aabe911: sdv - tur checker reports path is down
2020-09-23T20:29:43.820706+00:00 overcloud-compute-0 multipathd: 3600601607b703600d3008296faabe911: sdf - tur checker reports path is down
2020-09-23T20:29:43.821015+00:00 overcloud-compute-0 multipathd: 3600601607b7036000aefd29a4aabe911: sdx - tur checker reports path is down
2020-09-23T20:29:43.821232+00:00 overcloud-compute-0 multipathd: 3600601607b7036000aefd29a4aabe911: sdab - tur checker reports path is down
2020-09-23T20:29:43.821448+00:00 overcloud-compute-0 multipathd: 3600601607b7036000aefd29a4aabe911: sdad - tur checker reports path is down
2020-09-23T20:29:43.822583+00:00 overcloud-compute-0 multipathd: 3600601607b7036000aefd29a4aabe911: sdaf - tur checker reports path is down
2020-09-23T20:29:43.822846+00:00 overcloud-compute-0 multipathd: 3600601607b7036000aefd29a4aabe911: sdal - tur checker reports path is down
2020-09-23T20:29:43.824198+00:00 overcloud-compute-0 multipathd: 3600601607b7036000aefd29a4aabe911: sdap - tur checker reports path is down
2020-09-23T20:29:43.825539+00:00 overcloud-compute-0 multipathd: 3600601607b7036000aefd29a4aabe911: sdar - tur checker reports path is down
2020-09-23T20:29:43.826730+00:00 overcloud-compute-0 multipathd: 3600601607b7036000aefd29a4aabe911: sdau - tur checker reports path is down
2020-09-23T20:29:43.826957+00:00 overcloud-compute-0 multipathd: 3600601607b7036000aefd29a4aabe911: sdaw - tur checker reports path is down
2020-09-23T20:29:44.827305+00:00 overcloud-compute-0 multipathd: 3600601607b7036000aefd29a4aabe911: sdae - tur checker reports path is down
2020-09-23T20:29:44.827588+00:00 overcloud-compute-0 multipathd: 3600601607b7036000aefd29a4aabe911: sdam - tur checker reports path is down
2020-09-23T20:29:44.827817+00:00 overcloud-compute-0 multipathd: 3600601607b7036000aefd29a4aabe911: sdan - tur checker reports path is down
2020-09-23T20:29:44.828023+00:00 overcloud-compute-0 multipathd: 3600601607b7036000aefd29a4aabe911: sdaq - tur checker reports path is down
2020-09-23T20:29:44.828237+00:00 overcloud-compute-0 multipathd: 3600601607b7036000aefd29a4aabe911: sdav - tur checker reports path is down
2020-09-23T20:29:45.830050+00:00 overcloud-compute-0 multipathd: 3600601607b7036000aefd29a4aabe911: sdu - tur checker reports path is down
2020-09-23T20:29:45.830378+00:00 overcloud-compute-0 multipathd: 3600601607b7036000aefd29a4aabe911: sdy - tur checker reports path is down
2020-09-23T20:29:45.830575+00:00 overcloud-compute-0 multipathd: 3600601607b7036000aefd29a4aabe911: sdw - tur checker reports path is down
2020-09-23T20:29:45.830768+00:00 overcloud-compute-0 multipathd: 3600601607b7036000aefd29a4aabe911: sdac - tur checker reports path is down
2020-09-23T20:29:45.830930+00:00 overcloud-compute-0 multipathd: 3600601607b7036000aefd29a4aabe911: sdaa - tur checker reports path is down
2020-09-23T20:29:45.831103+00:00 overcloud-compute-0 multipathd: 3600601607b7036000aefd29a4aabe911: sdag - tur checker reports path is down
2020-09-23T20:29:45.831264+00:00 overcloud-compute-0 multipathd: 3600601607b7036000aefd29a4aabe911: sdao - tur checker reports path is down
2020-09-23T20:29:45.831447+00:00 overcloud-compute-0 multipathd: 3600601607b7036000aefd29a4aabe911: sdas - tur checker reports path is down
2020-09-23T20:29:45.831622+00:00 overcloud-compute-0 multipathd: 3600601607b7036000aefd29a4aabe911: sdax - tur checker reports path is down
2020-09-23T20:29:46.831874+00:00 overcloud-compute-0 multipathd: 3600601607b703600d3008296faabe911: sdd - tur checker reports path is down
2020-09-23T20:29:46.832109+00:00 overcloud-compute-0 multipathd: 3600601607b703600d3008296faabe911: sdl - tur checker reports path is down
2020-09-23T20:29:46.832307+00:00 overcloud-compute-0 multipathd: 3600601607b7036000aefd29a4aabe911: sdp - tur checker reports path is down
2020-09-23T20:29:46.832485+00:00 overcloud-compute-0 multipathd: 3600601607b7036000aefd29a4aabe911: sdo - tur checker reports path is down
2020-09-23T20:29:46.832658+00:00 overcloud-compute-0 multipathd: 3600601607b7036000aefd29a4aabe911: sds - tur checker reports path is down
2020-09-23T20:29:46.832842+00:00 overcloud-compute-0 multipathd: 3600601607b7036000aefd29a4aabe911: sdz - tur checker reports path is down
~~~

[2]
~~~
2020-09-23T20:29:46.991585+00:00 overcloud-compute-0 kernel: blk_update_request: I/O error, dev dm-0, sector 8
2020-09-23T20:29:46.991733+00:00 overcloud-compute-0 kernel: buffer_io_error: 909 callbacks suppressed
2020-09-23T20:29:46.991833+00:00 overcloud-compute-0 kernel: Buffer I/O error on dev dm-0, logical block 1, async page read
2020-09-23T20:29:46.991889+00:00 overcloud-compute-0 kernel: blk_update_request: I/O error, dev dm-0, sector 8
2020-09-23T20:29:47.113830+00:00 overcloud-compute-0 kernel: Buffer I/O error on dev dm-0, logical block 1, async page read
2020-09-23T20:29:47.113898+00:00 overcloud-compute-0 kernel: blk_update_request: I/O error, dev dm-0, sector 8
2020-09-23T20:29:47.113924+00:00 overcloud-compute-0 kernel: Buffer I/O error on dev dm-0, logical block 1, async page read
2020-09-23T20:29:47.113973+00:00 overcloud-compute-0 kernel: blk_update_request: I/O error, dev dm-0, sector 8
2020-09-23T20:29:47.174240+00:00 overcloud-compute-0 kernel: Buffer I/O error on dev dm-0, logical block 1, async page read
2020-09-23T20:29:47.201817+00:00 overcloud-compute-0 kernel: blk_update_request: I/O error, dev dm-0, sector 8
2020-09-23T20:29:47.201881+00:00 overcloud-compute-0 kernel: Buffer I/O error on dev dm-0, logical block 1, async page read
2020-09-23T20:29:47.262283+00:00 overcloud-compute-0 kernel: blk_update_request: I/O error, dev dm-0, sector 8
2020-09-23T20:29:47.262334+00:00 overcloud-compute-0 kernel: Buffer I/O error on dev dm-0, logical block 1, async page read
2020-09-23T20:29:47.355168+00:00 overcloud-compute-0 kernel: blk_update_request: I/O error, dev dm-0, sector 8
2020-09-23T20:29:47.355212+00:00 overcloud-compute-0 kernel: Buffer I/O error on dev dm-0, logical block 1, async page read
2020-09-23T20:29:47.355237+00:00 overcloud-compute-0 kernel: blk_update_request: I/O error, dev dm-0, sector 8
2020-09-23T20:29:47.383049+00:00 overcloud-compute-0 kernel: Buffer I/O error on dev dm-0, logical block 1, async page read
2020-09-23T20:29:47.479099+00:00 overcloud-compute-0 kernel: blk_update_request: I/O error, dev dm-0, sector 128
2020-09-23T20:29:47.479170+00:00 overcloud-compute-0 kernel: Buffer I/O error on dev dm-0, logical block 16, async page read
2020-09-23T20:29:47.479211+00:00 overcloud-compute-0 kernel: blk_update_request: I/O error, dev dm-0, sector 64
2020-09-23T20:29:47.539936+00:00 overcloud-compute-0 kernel: Buffer I/O error on dev dm-0, logical block 8, async page read
2020-09-23T20:29:47.655758+00:00 overcloud-compute-0 kernel: sd 5:0:0:63: [sdad] FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
2020-09-23T20:29:47.656088+00:00 overcloud-compute-0 kernel: sd 5:0:0:63: [sdad] Sense Key : Illegal Request [current]
2020-09-23T20:29:47.656294+00:00 overcloud-compute-0 kernel: sd 5:0:0:63: [sdad] Add. Sense: Logical unit not supported
2020-09-23T20:29:47.656469+00:00 overcloud-compute-0 kernel: sd 5:0:0:63: [sdad] CDB: Read(10) 28 00 00 00 00 00 00 00 08 00
2020-09-23T20:29:47.656669+00:00 overcloud-compute-0 kernel: sd 5:0:0:63: [sdad] FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
2020-09-23T20:29:47.656862+00:00 overcloud-compute-0 kernel: sd 5:0:0:63: [sdad] Sense Key : Illegal Request [current]
2020-09-23T20:29:47.657021+00:00 overcloud-compute-0 kernel: sd 5:0:0:63: [sdad] Add. Sense: Logical unit not supported
2020-09-23T20:29:47.657181+00:00 overcloud-compute-0 kernel: sd 5:0:0:63: [sdad] CDB: Read(10) 28 00 00 00 00 00 00 00 08 00
2020-09-23T20:29:47.657313+00:00 overcloud-compute-0 kernel: sd 5:0:0:63: [sdad] FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
2020-09-23T20:29:47.657505+00:00 overcloud-compute-0 kernel: sd 5:0:0:63: [sdad] Sense Key : Illegal Request [current]
2020-09-23T20:29:47.657624+00:00 overcloud-compute-0 kernel: sd 5:0:0:63: [sdad] Add. Sense: Logical unit not supported
2020-09-23T20:29:47.657736+00:00 overcloud-compute-0 kernel: sd 5:0:0:63: [sdad] CDB: Read(10) 28 00 00 00 00 00 00 00 20 00
2020-09-23T20:29:47.657921+00:00 overcloud-compute-0 kernel: sd 5:0:0:63: [sdad] FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
2020-09-23T20:29:47.658063+00:00 overcloud-compute-0 kernel: sd 5:0:0:63: [sdad] Sense Key : Illegal Request [current]
2020-09-23T20:29:47.658227+00:00 overcloud-compute-0 kernel: sd 5:0:0:63: [sdad] Add. Sense: Logical unit not supported
2020-09-23T20:29:47.658384+00:00 overcloud-compute-0 kernel: sd 5:0:0:63: [sdad] CDB: Read(10) 28 00 00 00 00 00 00 00 08 00
2020-09-23T20:29:47.658558+00:00 overcloud-compute-0 kernel: sd 5:0:0:63: [sdad] FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
2020-09-23T20:29:47.658729+00:00 overcloud-compute-0 kernel: sd 5:0:0:63: [sdad] Sense Key : Illegal Request [current]
2020-09-23T20:29:47.658864+00:00 overcloud-compute-0 kernel: sd 5:0:0:63: [sdad] Add. Sense: Logical unit not supported
2020-09-23T20:29:47.659010+00:00 overcloud-compute-0 kernel: sd 5:0:0:63: [sdad] CDB: Read(10) 28 00 00 00 00 00 00 00 08 00
2020-09-23T20:29:47.659180+00:00 overcloud-compute-0 kernel: sd 5:0:0:63: [sdad] FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
2020-09-23T20:29:47.659302+00:00 overcloud-compute-0 kernel: sd 5:0:0:63: [sdad] Sense Key : Illegal Request [current]
2020-09-23T20:29:47.659433+00:00 overcloud-compute-0 kernel: sd 5:0:0:63: [sdad] Add. Sense: Logical unit not supported
2020-09-23T20:29:47.659538+00:00 overcloud-compute-0 kernel: sd 5:0:0:63: [sdad] CDB: Read(10) 28 00 00 00 00 00 00 00 08 00
2020-09-23T20:29:47.659645+00:00 overcloud-compute-0 kernel: sd 5:0:0:63: [sdad] FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
2020-09-23T20:29:47.659770+00:00 overcloud-compute-0 kernel: sd 5:0:0:63: [sdad] Sense Key : Illegal Request [current]
2020-09-23T20:29:47.659892+00:00 overcloud-compute-0 kernel: sd 5:0:0:63: [sdad] Add. Sense: Logical unit not supported
2020-09-23T20:29:47.660013+00:00 overcloud-compute-0 kernel: sd 5:0:0:63: [sdad] CDB: Read(10) 28 00 00 00 00 00 00 00 08 00
2020-09-23T20:29:47.660116+00:00 overcloud-compute-0 kernel: sd 5:0:0:63: [sdad] FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
2020-09-23T20:29:47.660219+00:00 overcloud-compute-0 kernel: sd 5:0:0:63: [sdad] Sense Key : Illegal Request [current]
2020-09-23T20:29:47.660334+00:00 overcloud-compute-0 kernel: sd 5:0:0:63: [sdad] Add. Sense: Logical unit not supported
2020-09-23T20:29:47.660469+00:00 overcloud-compute-0 kernel: sd 5:0:0:63: [sdad] CDB: Read(10) 28 00 00 00 00 00 00 00 08 00
2020-09-23T20:29:47.660578+00:00 overcloud-compute-0 kernel: sd 5:0:0:63: [sdad] FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
2020-09-23T20:29:47.660682+00:00 overcloud-compute-0 kernel: sd 5:0:0:63: [sdad] Sense Key : Illegal Request [current]
2020-09-23T20:29:47.660791+00:00 overcloud-compute-0 kernel: sd 5:0:0:63: [sdad] Add. Sense: Logical unit not supported
2020-09-23T20:29:47.660885+00:00 overcloud-compute-0 kernel: sd 5:0:0:63: [sdad] CDB: Read(10) 28 00 00 00 00 00 00 00 08 00
2020-09-23T20:29:47.661011+00:00 overcloud-compute-0 kernel: sd 5:0:0:63: [sdad] FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
2020-09-23T20:29:47.661108+00:00 overcloud-compute-0 kernel: sd 5:0:0:63: [sdad] Sense Key : Illegal Request [current]
2020-09-23T20:29:47.661264+00:00 overcloud-compute-0 kernel: sd 5:0:0:63: [sdad] Add. Sense: Logical unit not supported
2020-09-23T20:29:47.661448+00:00 overcloud-compute-0 kernel: sd 5:0:0:63: [sdad] CDB: Read(10) 28 00 00 00 00 00 00 00 08 00
2020-09-23T20:29:47.661592+00:00 overcloud-compute-0 kernel: sd 5:0:0:63: [sdad] Unit Not Ready
2020-09-23T20:29:47.661717+00:00 overcloud-compute-0 kernel: sd 5:0:0:63: [sdad] Sense Key : Illegal Request [current]
2020-09-23T20:29:47.661888+00:00 overcloud-compute-0 kernel: sd 5:0:0:63: [sdad] Add. Sense: Logical unit not supported
2020-09-23T20:29:47.662007+00:00 overcloud-compute-0 kernel: sd 5:0:0:63: [sdad] Read Capacity(10) failed: Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
2020-09-23T20:29:47.662135+00:00 overcloud-compute-0 kernel: sd 5:0:0:63: [sdad] Sense Key : Illegal Request [current]
2020-09-23T20:29:47.662236+00:00 overcloud-compute-0 kernel: sd 5:0:0:63: [sdad] Add. Sense: Logical unit not supported
2020-09-23T20:29:47.662281+00:00 overcloud-compute-0 kernel: sdad: detected capacity change from 1073741824000 to 0
2020-09-23T20:29:47.683541+00:00 overcloud-compute-0 kernel: sd 6:0:0:63: [sdae] Unit Not Ready
2020-09-23T20:29:47.683781+00:00 overcloud-compute-0 kernel: sd 6:0:0:63: [sdae] Sense Key : Illegal Request [current]
2020-09-23T20:29:47.683967+00:00 overcloud-compute-0 kernel: sd 6:0:0:63: [sdae] Add. Sense: Logical unit not supported
2020-09-23T20:29:47.684122+00:00 overcloud-compute-0 kernel: sd 6:0:0:63: [sdae] Read Capacity(10) failed: Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
2020-09-23T20:29:47.684311+00:00 overcloud-compute-0 kernel: sd 6:0:0:63: [sdae] Sense Key : Illegal Request [current]
2020-09-23T20:29:47.684451+00:00 overcloud-compute-0 kernel: sd 6:0:0:63: [sdae] Add. Sense: Logical unit not supported
2020-09-23T20:29:47.684500+00:00 overcloud-compute-0 kernel: sdae: detected capacity change from 1073741824000 to 0
2020-09-23T20:29:47.732545+00:00 overcloud-compute-0 kernel: sd 7:0:0:63: [sdaf] Unit Not Ready
2020-09-23T20:29:47.732710+00:00 overcloud-compute-0 kernel: sd 7:0:0:63: [sdaf] Sense Key : Illegal Request [current]
2020-09-23T20:29:47.732846+00:00 overcloud-compute-0 kernel: sd 7:0:0:63: [sdaf] Add. Sense: Logical unit not supported
2020-09-23T20:29:47.732976+00:00 overcloud-compute-0 kernel: sd 7:0:0:63: [sdaf] Read Capacity(10) failed: Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
2020-09-23T20:29:47.733125+00:00 overcloud-compute-0 kernel: sd 7:0:0:63: [sdaf] Sense Key : Illegal Request [current]
2020-09-23T20:29:47.733226+00:00 overcloud-compute-0 kernel: sd 7:0:0:63: [sdaf] Add. Sense: Logical unit not supported
2020-09-23T20:29:47.733287+00:00 overcloud-compute-0 kernel: sdaf: detected capacity change from 1073741824000 to 0
2020-09-23T20:29:47.739513+00:00 overcloud-compute-0 kernel: sd 8:0:0:63: [sdag] Unit Not Ready
2020-09-23T20:29:47.739681+00:00 overcloud-compute-0 kernel: sd 8:0:0:63: [sdag] Sense Key : Illegal Request [current]
2020-09-23T20:29:47.739849+00:00 overcloud-compute-0 kernel: sd 8:0:0:63: [sdag] Add. Sense: Logical unit not supported
2020-09-23T20:29:47.739982+00:00 overcloud-compute-0 kernel: sd 8:0:0:63: [sdag] Read Capacity(10) failed: Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
2020-09-23T20:29:47.740138+00:00 overcloud-compute-0 kernel: sd 8:0:0:63: [sdag] Sense Key : Illegal Request [current]
2020-09-23T20:29:47.740296+00:00 overcloud-compute-0 kernel: sd 8:0:0:63: [sdag] Add. Sense: Logical unit not supported
2020-09-23T20:29:47.740347+00:00 overcloud-compute-0 kernel: sdag: detected capacity change from 1073741824000 to 0
2020-09-23T20:29:47.763549+00:00 overcloud-compute-0 kernel: sd 5:0:0:207: [sdal] Unit Not Ready
2020-09-23T20:29:47.763756+00:00 overcloud-compute-0 kernel: sd 5:0:0:207: [sdal] Sense Key : Illegal Request [current]
2020-09-23T20:29:47.763939+00:00 overcloud-compute-0 kernel: sd 5:0:0:207: [sdal] Add. Sense: Logical unit not supported
2020-09-23T20:29:47.764065+00:00 overcloud-compute-0 kernel: sd 5:0:0:207: [sdal] Read Capacity(10) failed: Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
2020-09-23T20:29:47.764162+00:00 overcloud-compute-0 kernel: sd 5:0:0:207: [sdal] Sense Key : Illegal Request [current]
2020-09-23T20:29:47.764266+00:00 overcloud-compute-0 kernel: sd 5:0:0:207: [sdal] Add. Sense: Logical unit not supported
2020-09-23T20:29:47.764307+00:00 overcloud-compute-0 kernel: sdal: detected capacity change from 1073741824000 to 0
2020-09-23T20:29:47.811560+00:00 overcloud-compute-0 kernel: sd 6:0:0:207: [sdam] Unit Not Ready
2020-09-23T20:29:47.811726+00:00 overcloud-compute-0 kernel: sd 6:0:0:207: [sdam] Sense Key : Illegal Request [current]
2020-09-23T20:29:47.811879+00:00 overcloud-compute-0 kernel: sd 6:0:0:207: [sdam] Add. Sense: Logical unit not supported
2020-09-23T20:29:47.811991+00:00 overcloud-compute-0 kernel: sd 6:0:0:207: [sdam] Read Capacity(10) failed: Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
2020-09-23T20:29:47.812103+00:00 overcloud-compute-0 kernel: sd 6:0:0:207: [sdam] Sense Key : Illegal Request [current]
2020-09-23T20:29:47.812199+00:00 overcloud-compute-0 kernel: sd 6:0:0:207: [sdam] Add. Sense: Logical unit not supported
2020-09-23T20:29:47.812249+00:00 overcloud-compute-0 kernel: sdam: detected capacity change from 1073741824000 to 0
2020-09-23T20:29:47.834493+00:00 overcloud-compute-0 multipathd: 3600601607b703600d3008296faabe911: sdb - tur checker reports path is down
2020-09-23T20:29:47.834844+00:00 overcloud-compute-0 multipathd: 3600601607b703600d3008296faabe911: sde - tur checker reports path is down
2020-09-23T20:29:47.835025+00:00 overcloud-compute-0 multipathd: 3600601607b703600d3008296faabe911: sdc - tur checker reports path is down
2020-09-23T20:29:47.835214+00:00 overcloud-compute-0 multipathd: 3600601607b703600d3008296faabe911: sdg - tur checker reports path is down
2020-09-23T20:29:47.835396+00:00 overcloud-compute-0 multipathd: 3600601607b703600d3008296faabe911: sdh - tur checker reports path is down
2020-09-23T20:29:47.835593+00:00 overcloud-compute-0 multipathd: 3600601607b703600d3008296faabe911: sdi - tur checker reports path is down
2020-09-23T20:29:47.835785+00:00 overcloud-compute-0 multipathd: 3600601607b703600d3008296faabe911: sdj - tur checker reports path is down
2020-09-23T20:29:47.835957+00:00 overcloud-compute-0 multipathd: 3600601607b703600d3008296faabe911: sdm - tur checker reports path is down
2020-09-23T20:29:47.836122+00:00 overcloud-compute-0 multipathd: 3600601607b703600d3008296faabe911: sdk - tur checker reports path is down
2020-09-23T20:29:47.836303+00:00 overcloud-compute-0 multipathd: 3600601607b7036000aefd29a4aabe911: sdn - tur checker reports path is down
2020-09-23T20:29:47.836496+00:00 overcloud-compute-0 multipathd: 3600601607b7036000aefd29a4aabe911: sdq - tur checker reports path is down
2020-09-23T20:29:47.836676+00:00 overcloud-compute-0 multipathd: 3600601607b7036000aefd29a4aabe911: sdt - tur checker reports path is down
2020-09-23T20:29:47.836873+00:00 overcloud-compute-0 multipathd: 3600601607b7036000aefd29a4aabe911: sdr - tur checker reports path is down
2020-09-23T20:29:47.837045+00:00 overcloud-compute-0 multipathd: 3600601607b7036000aefd29a4aabe911: sdv - tur checker reports path is down
2020-09-23T20:29:47.857542+00:00 overcloud-compute-0 kernel: sd 7:0:0:207: [sdan] Unit Not Ready
2020-09-23T20:29:47.857765+00:00 overcloud-compute-0 kernel: sd 7:0:0:207: [sdan] Sense Key : Illegal Request [current]
2020-09-23T20:29:47.857929+00:00 overcloud-compute-0 kernel: sd 7:0:0:207: [sdan] Add. Sense: Logical unit not supported
2020-09-23T20:29:47.858079+00:00 overcloud-compute-0 kernel: sd 7:0:0:207: [sdan] Read Capacity(10) failed: Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
2020-09-23T20:29:47.858211+00:00 overcloud-compute-0 kernel: sd 7:0:0:207: [sdan] Sense Key : Illegal Request [current]
2020-09-23T20:29:47.858337+00:00 overcloud-compute-0 kernel: sd 7:0:0:207: [sdan] Add. Sense: Logical unit not supported
2020-09-23T20:29:47.858379+00:00 overcloud-compute-0 kernel: sdan: detected capacity change from 1073741824000 to 0
2020-09-23T20:29:47.880567+00:00 overcloud-compute-0 kernel: sd 8:0:0:207: [sdao] Unit Not Ready
2020-09-23T20:29:47.880798+00:00 overcloud-compute-0 kernel: sd 8:0:0:207: [sdao] Sense Key : Illegal Request [current]
2020-09-23T20:29:47.880930+00:00 overcloud-compute-0 kernel: sd 8:0:0:207: [sdao] Add. Sense: Logical unit not supported
2020-09-23T20:29:47.881040+00:00 overcloud-compute-0 kernel: sd 8:0:0:207: [sdao] Read Capacity(10) failed: Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
2020-09-23T20:29:47.881157+00:00 overcloud-compute-0 kernel: sd 8:0:0:207: [sdao] Sense Key : Illegal Request [current]
2020-09-23T20:29:47.881267+00:00 overcloud-compute-0 kernel: sd 8:0:0:207: [sdao] Add. Sense: Logical unit not supported
2020-09-23T20:29:47.881326+00:00 overcloud-compute-0 kernel: sdao: detected capacity change from 1073741824000 to 0
2020-09-23T20:29:47.903543+00:00 overcloud-compute-0 kernel: sd 5:0:0:225: [sdap] Unit Not Ready
2020-09-23T20:29:47.903747+00:00 overcloud-compute-0 kernel: sd 5:0:0:225: [sdap] Sense Key : Illegal Request [current]
2020-09-23T20:29:47.903888+00:00 overcloud-compute-0 kernel: sd 5:0:0:225: [sdap] Add. Sense: Logical unit not supported
2020-09-23T20:29:47.904018+00:00 overcloud-compute-0 kernel: sd 5:0:0:225: [sdap] Read Capacity(10) failed: Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
2020-09-23T20:29:47.904153+00:00 overcloud-compute-0 kernel: sd 5:0:0:225: [sdap] Sense Key : Illegal Request [current]
2020-09-23T20:29:47.904270+00:00 overcloud-compute-0 kernel: sd 5:0:0:225: [sdap] Add. Sense: Logical unit not supported
2020-09-23T20:29:47.904316+00:00 overcloud-compute-0 kernel: sdap: detected capacity change from 1073741824000 to 0
2020-09-23T20:29:47.926540+00:00 overcloud-compute-0 kernel: sd 6:0:0:225: [sdaq] Unit Not Ready
2020-09-23T20:29:47.926755+00:00 overcloud-compute-0 kernel: sd 6:0:0:225: [sdaq] Sense Key : Illegal Request [current]
2020-09-23T20:29:47.926929+00:00 overcloud-compute-0 kernel: sd 6:0:0:225: [sdaq] Add. Sense: Logical unit not supported
2020-09-23T20:29:47.927069+00:00 overcloud-compute-0 kernel: sd 6:0:0:225: [sdaq] Read Capacity(10) failed: Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
2020-09-23T20:29:47.927212+00:00 overcloud-compute-0 kernel: sd 6:0:0:225: [sdaq] Sense Key : Illegal Request [current]
2020-09-23T20:29:47.927330+00:00 overcloud-compute-0 kernel: sd 6:0:0:225: [sdaq] Add. Sense: Logical unit not supported
2020-09-23T20:29:47.927396+00:00 overcloud-compute-0 kernel: sdaq: detected capacity change from 1073741824000 to 0
2020-09-23T20:29:47.948536+00:00 overcloud-compute-0 kernel: sd 7:0:0:225: [sdar] Unit Not Ready
2020-09-23T20:29:47.948682+00:00 overcloud-compute-0 kernel: sd 7:0:0:225: [sdar] Sense Key : Illegal Request [current]
2020-09-23T20:29:47.948801+00:00 overcloud-compute-0 kernel: sd 7:0:0:225: [sdar] Add. Sense: Logical unit not supported
2020-09-23T20:29:47.948913+00:00 overcloud-compute-0 kernel: sd 7:0:0:225: [sdar] Read Capacity(10) failed: Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
2020-09-23T20:29:47.949035+00:00 overcloud-compute-0 kernel: sd 7:0:0:225: [sdar] Sense Key : Illegal Request [current]
2020-09-23T20:29:47.949138+00:00 overcloud-compute-0 kernel: sd 7:0:0:225: [sdar] Add. Sense: Logical unit not supported
2020-09-23T20:29:47.949189+00:00 overcloud-compute-0 kernel: sdar: detected capacity change from 1073741824000 to 0
2020-09-23T20:29:47.971558+00:00 overcloud-compute-0 kernel: sd 8:0:0:225: [sdas] Unit Not Ready
2020-09-23T20:29:47.971798+00:00 overcloud-compute-0 kernel: sd 8:0:0:225: [sdas] Sense Key : Illegal Request [current]
2020-09-23T20:29:47.971957+00:00 overcloud-compute-0 kernel: sd 8:0:0:225: [sdas] Add. Sense: Logical unit not supported
2020-09-23T20:29:47.972081+00:00 overcloud-compute-0 kernel: sd 8:0:0:225: [sdas] Read Capacity(10) failed: Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
2020-09-23T20:29:47.972210+00:00 overcloud-compute-0 kernel: sd 8:0:0:225: [sdas] Sense Key : Illegal Request [current]
2020-09-23T20:29:47.972327+00:00 overcloud-compute-0 kernel: sd 8:0:0:225: [sdas] Add. Sense: Logical unit not supported
2020-09-23T20:29:47.972414+00:00 overcloud-compute-0 kernel: sdas: detected capacity change from 1073741824000 to 0
2020-09-23T20:29:48.022572+00:00 overcloud-compute-0 kernel: sd 5:0:0:214: [sdau] Unit Not Ready
2020-09-23T20:29:48.022721+00:00 overcloud-compute-0 kernel: sd 5:0:0:214: [sdau] Sense Key : Illegal Request [current]
2020-09-23T20:29:48.022853+00:00 overcloud-compute-0 kernel: sd 5:0:0:214: [sdau] Add. Sense: Logical unit not supported
2020-09-23T20:29:48.022985+00:00 overcloud-compute-0 kernel: sd 5:0:0:214: [sdau] Read Capacity(10) failed: Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
2020-09-23T20:29:48.023119+00:00 overcloud-compute-0 kernel: sd 5:0:0:214: [sdau] Sense Key : Illegal Request [current]
2020-09-23T20:29:48.023207+00:00 overcloud-compute-0 kernel: sd 5:0:0:214: [sdau] Add. Sense: Logical unit not supported
2020-09-23T20:29:48.023251+00:00 overcloud-compute-0 kernel: sdau: detected capacity change from 1073741824000 to 0
2020-09-23T20:29:48.044557+00:00 overcloud-compute-0 kernel: sd 6:0:0:214: [sdav] Unit Not Ready
2020-09-23T20:29:48.044779+00:00 overcloud-compute-0 kernel: sd 6:0:0:214: [sdav] Sense Key : Illegal Request [current]
2020-09-23T20:29:48.044962+00:00 overcloud-compute-0 kernel: sd 6:0:0:214: [sdav] Add. Sense: Logical unit not supported
2020-09-23T20:29:48.045116+00:00 overcloud-compute-0 kernel: sd 6:0:0:214: [sdav] Read Capacity(10) failed: Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
2020-09-23T20:29:48.045292+00:00 overcloud-compute-0 kernel: sd 6:0:0:214: [sdav] Sense Key : Illegal Request [current]
2020-09-23T20:29:48.045436+00:00 overcloud-compute-0 kernel: sd 6:0:0:214: [sdav] Add. Sense: Logical unit not supported
2020-09-23T20:29:48.045492+00:00 overcloud-compute-0 kernel: sdav: detected capacity change from 1073741824000 to 0
2020-09-23T20:29:48.073542+00:00 overcloud-compute-0 kernel: sd 7:0:0:214: [sdaw] Unit Not Ready
2020-09-23T20:29:48.073740+00:00 overcloud-compute-0 kernel: sd 7:0:0:214: [sdaw] Sense Key : Illegal Request [current]
2020-09-23T20:29:48.073882+00:00 overcloud-compute-0 kernel: sd 7:0:0:214: [sdaw] Add. Sense: Logical unit not supported
2020-09-23T20:29:48.074063+00:00 overcloud-compute-0 kernel: sd 7:0:0:214: [sdaw] Read Capacity(10) failed: Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
2020-09-23T20:29:48.074207+00:00 overcloud-compute-0 kernel: sd 7:0:0:214: [sdaw] Sense Key : Illegal Request [current]
2020-09-23T20:29:48.074393+00:00 overcloud-compute-0 kernel: sd 7:0:0:214: [sdaw] Add. Sense: Logical unit not supported
2020-09-23T20:29:48.074449+00:00 overcloud-compute-0 kernel: sdaw: detected capacity change from 1073741824000 to 0
2020-09-23T20:29:48.093553+00:00 overcloud-compute-0 kernel: sd 8:0:0:214: [sdax] Unit Not Ready
2020-09-23T20:29:48.093702+00:00 overcloud-compute-0 kernel: sd 8:0:0:214: [sdax] Sense Key : Illegal Request [current]
2020-09-23T20:29:48.093842+00:00 overcloud-compute-0 kernel: sd 8:0:0:214: [sdax] Add. Sense: Logical unit not supported
[...]
2020-09-23T21:20:11.448513+00:00 overcloud-compute-0 kernel: blk_update_request: I/O error, dev dm-0, sector 8
2020-09-23T21:20:11.448584+00:00 overcloud-compute-0 kernel: buffer_io_error: 581 callbacks suppressed
2020-09-23T21:20:11.448614+00:00 overcloud-compute-0 kernel: Buffer I/O error on dev dm-0, logical block 1, async page read
2020-09-23T21:20:11.448636+00:00 overcloud-compute-0 kernel: blk_update_request: I/O error, dev dm-0, sector 8
2020-09-23T21:20:11.509030+00:00 overcloud-compute-0 kernel: Buffer I/O error on dev dm-0, logical block 1, async page read
2020-09-23T21:20:11.509086+00:00 overcloud-compute-0 kernel: blk_update_request: I/O error, dev dm-0, sector 8
2020-09-23T21:20:11.536946+00:00 overcloud-compute-0 kernel: Buffer I/O error on dev dm-0, logical block 1, async page read
2020-09-23T21:20:11.569442+00:00 overcloud-compute-0 kernel: blk_update_request: I/O error, dev dm-0, sector 8
2020-09-23T21:20:11.631674+00:00 overcloud-compute-0 kernel: Buffer I/O error on dev dm-0, logical block 1, async page read
2020-09-23T21:20:11.631727+00:00 overcloud-compute-0 kernel: blk_update_request: I/O error, dev dm-0, sector 8
2020-09-23T21:20:11.659570+00:00 overcloud-compute-0 kernel: Buffer I/O error on dev dm-0, logical block 1, async page read
2020-09-23T21:20:11.720400+00:00 overcloud-compute-0 kernel: blk_update_request: I/O error, dev dm-0, sector 8
2020-09-23T21:20:11.720472+00:00 overcloud-compute-0 kernel: Buffer I/O error on dev dm-0, logical block 1, async page read
2020-09-23T21:20:11.781559+00:00 overcloud-compute-0 kernel: blk_update_request: I/O error, dev dm-0, sector 8
2020-09-23T21:20:11.781599+00:00 overcloud-compute-0 kernel: Buffer I/O error on dev dm-0, logical block 1, async page read
2020-09-23T21:20:11.842215+00:00 overcloud-compute-0 kernel: blk_update_request: I/O error, dev dm-0, sector 8
2020-09-23T21:20:11.842264+00:00 overcloud-compute-0 kernel: Buffer I/O error on dev dm-0, logical block 1, async page read
2020-09-23T21:20:11.903762+00:00 overcloud-compute-0 kernel: blk_update_request: I/O error, dev dm-0, sector 128
2020-09-23T21:20:11.903813+00:00 overcloud-compute-0 kernel: Buffer I/O error on dev dm-0, logical block 16, async page read
2020-09-23T21:20:11.969855+00:00 overcloud-compute-0 kernel: Buffer I/O error on dev dm-0, logical block 8, async page read
~~~

[3]
~~~
02755983]$ cat sosreport-20200923-212117/overcloud-compute-0/sos_commands/md/mdadm_-D_.dev.md
/dev/md125:
           Version : 1.1
        Raid Level : raid0
     Total Devices : 0
       Persistence : Superblock is persistent

             State : inactive

    Number   Major   Minor   RaidDevice
/dev/md126:
           Version : 1.1
        Raid Level : raid0
     Total Devices : 1
       Persistence : Superblock is persistent

             State : inactive
   Working Devices : 1

              Name : disc
              UUID : 5914375c:972d27e0:fad157e6:e2b33ef4
            Events : 125751

    Number   Major   Minor   RaidDevice

       -     253        1        -        /dev/dm-1
/dev/md127:
           Version : 1.1
        Raid Level : raid0
     Total Devices : 1
       Persistence : Superblock is persistent

             State : inactive
   Working Devices : 1

    Number   Major   Minor   RaidDevice

       -     253        0        -        /dev/dm-0

Comment 10 Alan Bishop 2020-11-04 15:20:04 UTC
I'm closing this based on feedback received, and the details I outline in comment #7. Fixing the problem in older OSP releases (10 and 13) will be difficult, and later releases based on rhel-8 (e.g. OSP-16) are not vulnerable. Let's use a KCS to cover this one.