Bug 1627613 - After esx hosts rebooted , all path status is unused. the rebooted hosts can't use the iscsi storage.
Summary: After esx hosts rebooted , all path status is unused. the rebooted hosts...
Keywords:
Status: CLOSED DUPLICATE of bug 1623737
Alias: None
Product: Red Hat Ceph Storage
Classification: Red Hat Storage
Component: iSCSI
Version: 3.0
Hardware: x86_64
OS: Linux
urgent
urgent
Target Milestone: rc
: 3.*
Assignee: Mike Christie
QA Contact: Madhavi Kasturi
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-09-11 03:48 UTC by liuwei
Modified: 2021-12-10 17:26 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-09-12 15:29:09 UTC
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHCEPH-2699 0 None None None 2021-12-10 17:26:20 UTC

Description liuwei 2018-09-11 03:48:15 UTC 
Description of problem:

After esx hosts rebooted ,  all path status is unused.    the  hosts can't use the iscsi storage.

From the tcmu-runner log checking, the error is below:

For iscigw node osd16.

2018-09-05 12:59:31.656 1702 [ERROR] tcmu_notify_lock_lost:198 rbd/rbd.ceph-iscsi-gw-lun2100: Async lock drop. Old state 1
2018-09-05 12:59:32.554 1702 [WARN] tcmu_rbd_lock:446 rbd/rbd.ceph-iscsi-gw-lun2100: Acquired exclusive lock.
2018-09-05 12:59:35.766 1702 [ERROR] tcmu_notify_lock_lost:198 rbd/rbd.ceph-iscsi-gw-lun2106: Async lock drop. Old state 1
2018-09-05 12:59:36.558 1702 [WARN] tcmu_rbd_lock:446 rbd/rbd.ceph-iscsi-gw-lun2106: Acquired exclusive lock.
2018-09-05 12:59:36.881 1702 [ERROR] tcmu_notify_lock_lost:198 rbd/rbd.ceph-iscsi-gw-lun2107: Async lock drop. Old state 1
2018-09-05 12:59:37.645 1702 [WARN] tcmu_rbd_lock:446 rbd/rbd.ceph-iscsi-gw-lun2107: Acquired exclusive lock.
2018-09-06 19:20:38.979 1702 [ERROR] tcmu_notify_lock_lost:198 rbd/rbd.ceph-iscsi-gw-lun2200: Async lock drop. Old state 1
2018-09-06 19:20:40.119 1702 [WARN] tcmu_rbd_lock:446 rbd/rbd.ceph-iscsi-gw-lun2200: Acquired exclusive lock.
2018-09-06 19:35:42.209 1702 [ERROR] tcmu_notify_lock_lost:198 rbd/rbd.ceph-iscsi-gw-lun2200: Async lock drop. Old state 1
2018-09-06 19:35:44.325 1702 [WARN] tcmu_rbd_lock:446 rbd/rbd.ceph-iscsi-gw-lun2200: Acquired exclusive lock.
2018-09-06 19:56:58.534 1702 [ERROR] tcmu_notify_lock_lost:198 rbd/rbd.ceph-iscsi-gw-lun2200: Async lock drop. Old state 1
2018-09-06 19:57:00.441 1702 [WARN] tcmu_rbd_lock:446 rbd/rbd.ceph-iscsi-gw-lun2200: Acquired exclusive lock.


For osd24

2018-09-06 19:20:33.710 1687 [WARN] tcmu_rbd_lock:446 rbd/rbd.ceph-iscsi-gw-lun2200: Acquired exclusive lock.
2018-09-06 19:35:35.297 1687 [ERROR] tcmu_notify_lock_lost:198 rbd/rbd.ceph-iscsi-gw-lun2200: Async lock drop. Old state 1
2018-09-06 19:35:36.245 1687 [WARN] tcmu_rbd_lock:446 rbd/rbd.ceph-iscsi-gw-lun2200: Acquired exclusive lock.
2018-09-06 19:56:50.649 1687 [ERROR] tcmu_notify_lock_lost:198 rbd/rbd.ceph-iscsi-gw-lun2200: Async lock drop. Old state 1
2018-09-06 19:56:52.485 1687 [WARN] tcmu_rbd_lock:446 rbd/rbd.ceph-iscsi-gw-lun2200: Acquired exclusive lock.
2018-09-07 09:40:46.836 1687 [ERROR] tcmu_rbd_lock_break:397 rbd/rbd.ceph-iscsi-gw-lun3505: Could not break lock from 10.3.130.231:0/2540389405. (Err -16)
2018-09-07 09:40:47.837 1687 [ERROR] tcmu_rbd_lock_break:397 rbd/rbd.ceph-iscsi-gw-lun3505: Could not break lock from 10.3.130.231:0/2540389405. (Err -16)
2018-09-07 09:40:48.838 1687 [ERROR] tcmu_rbd_lock_break:397 rbd/rbd.ceph-iscsi-gw-lun3505: Could not break lock from 10.3.130.231:0/2540389405. (Err -16)
2018-09-07 09:40:49.840 1687 [ERROR] tcmu_rbd_lock_break:397 rbd/rbd.ceph-iscsi-gw-lun3505: Could not break lock from 10.3.130.231:0/2540389405. (Err -16)
2018-09-07 09:40:50.841 1687 [ERROR] tcmu_rbd_lock_break:397 rbd/rbd.ceph-iscsi-gw-lun3505: Could not break lock from 10.3.130.231:0/2540389405. (Err -16)
2018-09-07 09:40:52.368 1687 [ERROR] tcmu_rbd_lock_break:397 rbd/rbd.ceph-iscsi-gw-lun3505: Could not break lock from 10.3.130.231:0/2540389405. (Err -16)
2018-09-07 09:40:53.369 1687 [ERROR] tcmu_rbd_lock_break:397 rbd/rbd.ceph-iscsi-gw-lun3505: Could not break lock from 10.3.130.231:0/2540389405. (Err -16)
2018-09-07 09:40:54.370 1687 [ERROR] tcmu_rbd_lock_break:397 rbd/rbd.ceph-iscsi-gw-lun3505: Could not break lock from 10.3.130.231:0/2540389405. (Err -16)
2018-09-07 09:40:55.371 1687 [ERROR] tcmu_rbd_lock_break:397 rbd/rbd.ceph-iscsi-gw-lun3505: Could not break lock from 10.3.130.231:0/2540389405. (Err -16)
2018-09-07 09:40:56.372 1687 [ERROR] tcmu_rbd_lock_break:397 rbd/rbd.ceph-iscsi-gw-lun3505: Could not break lock from 10.3.130.231:0/2540389405. (Err -16)
2018-09-07 09:40:58.368 1687 [ERROR] tcmu_rbd_lock_break:397 rbd/rbd.ceph-iscsi-gw-lun3505: Could not break lock from 10.3.130.231:0/2540389405. (Err -16)
2018-09-07 09:40:59.369 1687 [ERROR] tcmu_rbd_lock_break:397 rbd/rbd.ceph-iscsi-gw-lun3505: Could not break lock from 10.3.130.231:0/2540389405. (Err -16)
2018-09-07 09:41:00.370 1687 [ERROR] tcmu_rbd_lock_break:397 rbd/rbd.ceph-iscsi-gw-lun3505: Could not break lock from 10.3.130.231:0/2540389405. (Err -16)
2018-09-07 09:41:01.371 1687 [ERROR] tcmu_rbd_lock_break:397 rbd/rbd.ceph-iscsi-gw-lun3505: Could not break lock from 10.3.130.231:0/2540389405. (Err -16)
2018-09-07 09:41:02.372 1687 [ERROR] tcmu_rbd_lock_break:397 rbd/rbd.ceph-iscsi-gw-lun3505: Could not break lock from 10.3.130.231:0/2540389405. (Err -16)
2018-09-07 09:41:04.368 1687 [ERROR] tcmu_rbd_lock_break:397 rbd/rbd.ceph-iscsi-gw-lun3505: Could not break lock from 10.3.130.231:0/2540389405. (Err -16)
2018-09-07 09:41:05.369 1687 [ERROR] tcmu_rbd_lock_break:397 rbd/rbd.ceph-iscsi-gw-lun3505: Could not break lock from 10.3.130.231:0/2540389405. (Err -16)
2018-09-07 09:41:06.370 1687 [ERROR] tcmu_rbd_lock_break:397 rbd/rbd.ceph-iscsi-gw-lun3505: Could not break lock from 10.3.130.231:0/2540389405. (Err -16)
2018-09-07 09:41:07.371 1687 [ERROR] tcmu_rbd_lock_break:397 rbd/rbd.ceph-iscsi-gw-lun3505: Could not break lock from 10.3.130.231:0/2540389405. (Err -16)
2018-09-07 09:41:08.372 1687 [ERROR] tcmu_rbd_lock_break:397 rbd/rbd.ceph-iscsi-gw-lun3505: Could not break lock from 10.3.130.231:0/2540389405. (Err -16)
2018-09-07 09:41:10.368 1687 [ERROR] tcmu_rbd_lock_break:397 rbd/rbd.ceph-iscsi-gw-lun3505: Could not break lock from 10.3.130.231:0/2540389405. (Err -16)
2018-09-07 09:41:11.369 1687 [ERROR] tcmu_rbd_lock_break:397 rbd/rbd.ceph-iscsi-gw-lun3505: Could not break lock from 10.3.130.231:0/2540389405. (Err -16)
2018-09-07 09:41:12.370 1687 [ERROR] tcmu_rbd_lock_break:397 rbd/rbd.ceph-iscsi-gw-lun3505: Could not break lock from 10.3.130.231:0/2540389405. (Err -16)
2018-09-07 09:41:13.371 1687 [ERROR] tcmu_rbd_lock_break:397 rbd/rbd.ceph-iscsi-gw-lun3505: Could not break lock from 10.3.130.231:0/2540389405. (Err -16)
2018-09-07 09:41:14.372 1687 [ERROR] tcmu_rbd_lock_break:397 rbd/rbd.ceph-iscsi-gw-lun3505: Could not break lock from 10.3.130.231:0/2540389405. (Err -16)




Version-Release number of selected component (if applicable):

 
ceph-common-12.2.4-10.el7cp.x86_64                          Sat Jun 16 02:08:00 2018
ceph-iscsi-cli-2.5-10.el7cp.noarch                          Sat Jun 16 08:38:12 2018
ceph-iscsi-config-2.3-13.el7cp.noarch                       Sat Jun 16 02:08:35 2018
libcephfs2-12.2.4-10.el7cp.x86_64                           Sat Jun 16 02:07:57 2018
python-cephfs-12.2.4-10.el7cp.x86_64                        Sat Jun 16 02:07:57 2018

tcmu-runner-1.3.0-0.7.el7.x86_64                            Sat Jun 16 18:28:56 2018


How reproducible:

100%reproduced

Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:

Fix this issue ASAP

Additional info:
Comment 7 Mike Christie 2018-09-11 15:44:38 UTC 
We have a fix or this in bz:

https://bugzilla.redhat.com/show_bug.cgi?id=1623737
Comment 8 Mike Christie 2018-09-12 15:29:09 UTC 

*** This bug has been marked as a duplicate of bug 1623737 ***
Note You need to log in before you can comment on or make changes to this bug.