| Summary: | Restart or reload of the rbd-target-gw service fails during alua configuration | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat Ceph Storage | Reporter: | Paul Cuzner <pcuzner> | ||||
| Component: | iSCSI | Assignee: | Paul Cuzner <pcuzner> | ||||
| Status: | CLOSED ERRATA | QA Contact: | ceph-qe-bugs <ceph-qe-bugs> | ||||
| Severity: | urgent | Docs Contact: | |||||
| Priority: | urgent | ||||||
| Version: | 3.0 | CC: | adeza, anharris, aschoen, ceph-eng-bugs, ceph-qe-bugs, gmeno, hyelloji, jdillama, kdreyer, nthomas, sankarshan, tchandra | ||||
| Target Milestone: | rc | ||||||
| Target Release: | 3.0 | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2017-12-05 23:32:37 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Attachments: |
|
||||||
patch currently being tested Followed the steps as mentioned in the description.
Created a gw configuration and on a co-located gw node reloaded the gw service and it is failing to reload..
[root@magna012 ~]# systemctl status rbd-target-gw
● rbd-target-gw.service - Setup system to export rbd images through LIO
Loaded: loaded (/usr/lib/systemd/system/rbd-target-gw.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2016-11-01 03:18:29 UTC; 1 weeks 1 days ago
Main PID: 161959 (rbd-target-gw)
CGroup: /system.slice/rbd-target-gw.service
└─161959 /usr/bin/python -u /usr/bin/rbd-target-gw
Nov 01 03:18:29 magna012 systemd[1]: Starting Setup system to export rbd images through LIO...
Nov 01 03:18:29 magna012 rbd-target-gw[161959]: Processing osd blacklist entries for this node
Nov 01 03:18:29 magna012 rbd-target-gw[161959]: No OSD blacklist entries found
Nov 01 03:18:29 magna012 rbd-target-gw[161959]: Reading the configuration object to update local LIO configuration
Nov 01 03:18:30 magna012 rbd-target-gw[161959]: Processing Gateway configuration
Nov 01 03:18:30 magna012 rbd-target-gw[161959]: (Gateway.load_config) successfully loaded existing target definition
Nov 01 03:18:30 magna012 rbd-target-gw[161959]: Processing LUN configuration
Nov 01 03:18:33 magna012 rbd-target-gw[161959]: (Gateway.load_config) successfully loaded existing target definition
Nov 01 03:18:34 magna012 rbd-target-gw[161959]: Processing client configuration
Nov 01 03:18:34 magna012 rbd-target-gw[161959]: iSCSI configuration load complete
[root@magna012 ~]# systemctl reload rbd-target-gw
[root@magna012 ~]# systemctl status rbd-target-gw
● rbd-target-gw.service - Setup system to export rbd images through LIO
Loaded: loaded (/usr/lib/systemd/system/rbd-target-gw.service; enabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Wed 2016-11-09 19:00:24 UTC; 1s ago
Process: 3030481 ExecReload=/usr/bin/kill -HUP $MAINPID (code=exited, status=0/SUCCESS)
Main PID: 161959 (code=exited, status=1/FAILURE)
Nov 09 19:00:24 magna012 rbd-target-gw[161959]: File "/usr/lib/python2.7/site-packages/rtslib_fb/tcm.py", line 677, in __init__
Nov 09 19:00:24 magna012 rbd-target-gw[161959]: super(BlockStorageObject, self).__init__(name, 'lookup')
Nov 09 19:00:24 magna012 rbd-target-gw[161959]: File "/usr/lib/python2.7/site-packages/rtslib_fb/tcm.py", line 52, in __init__
Nov 09 19:00:24 magna012 rbd-target-gw[161959]: self._backstore = _Backstore(name, type(self), mode)
Nov 09 19:00:24 magna012 rbd-target-gw[161959]: File "/usr/lib/python2.7/site-packages/rtslib_fb/tcm.py", line 900, in __init__
Nov 09 19:00:24 magna012 rbd-target-gw[161959]: (self._plugin, name))
Nov 09 19:00:24 magna012 rbd-target-gw[161959]: rtslib_fb.utils.RTSLibNotInCFS: Storage object block/rbd.test10 not found
Nov 09 19:00:24 magna012 systemd[1]: rbd-target-gw.service: main process exited, code=exited, status=1/FAILURE
Nov 09 19:00:24 magna012 systemd[1]: Unit rbd-target-gw.service entered failed state.
Nov 09 19:00:24 magna012 systemd[1]: rbd-target-gw.service failed.
[root@magna012 ~]# systemctl status rbd-target-gw
● rbd-target-gw.service - Setup system to export rbd images through LIO
Loaded: loaded (/usr/lib/systemd/system/rbd-target-gw.service; enabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Wed 2016-11-09 19:00:24 UTC; 3s ago
Process: 3030481 ExecReload=/usr/bin/kill -HUP $MAINPID (code=exited, status=0/SUCCESS)
Main PID: 161959 (code=exited, status=1/FAILURE)
Nov 09 19:00:24 magna012 rbd-target-gw[161959]: File "/usr/lib/python2.7/site-packages/rtslib_fb/tcm.py", line 677, in __init__
Nov 09 19:00:24 magna012 rbd-target-gw[161959]: super(BlockStorageObject, self).__init__(name, 'lookup')
Nov 09 19:00:24 magna012 rbd-target-gw[161959]: File "/usr/lib/python2.7/site-packages/rtslib_fb/tcm.py", line 52, in __init__
Nov 09 19:00:24 magna012 rbd-target-gw[161959]: self._backstore = _Backstore(name, type(self), mode)
Nov 09 19:00:24 magna012 rbd-target-gw[161959]: File "/usr/lib/python2.7/site-packages/rtslib_fb/tcm.py", line 900, in __init__
Nov 09 19:00:24 magna012 rbd-target-gw[161959]: (self._plugin, name))
Nov 09 19:00:24 magna012 rbd-target-gw[161959]: rtslib_fb.utils.RTSLibNotInCFS: Storage object block/rbd.test10 not found
Nov 09 19:00:24 magna012 systemd[1]: rbd-target-gw.service: main process exited, code=exited, status=1/FAILURE
Nov 09 19:00:24 magna012 systemd[1]: Unit rbd-target-gw.service entered failed state.
Nov 09 19:00:24 magna012 systemd[1]: rbd-target-gw.service failed.
==========================================
The gw service failed to reload
Issue has been fixed in a later release. Not seen in current ceph-iscsi-config 2.3 release Adding a needinfo for Tejas to confirm that the rbd-target-gw stop/start issue is resolved in the current release. Yes I agree Jason, this is a different issue. Also I am not seeing it anymore. The original issue is already resolved. Paul, could you please move this BZ to the necessary state. Thanks, Tejas Not seeing this issue in the latest builds: ceph-iscsi-config-2.3-5.el7cp.noarch ceph-iscsi-cli-2.5-4.el7cp.noarch Moving to verified. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2017:3387 |
Created attachment 1214146 [details] example python backtrace following a restart request Description of problem: During a service restart or a reload the service fails Version-Release number of selected component (if applicable): ceph-iscsi-config 1.3 How reproducible: Steps to Reproduce: 1. create a gateway configuration 2. issue a systemctl reload rbd-target-gw 3. Actual results: service enters a failed state Expected results: A reload should check the run time settings match the configuration Additional info: