Created attachment 1214146 [details] example python backtrace following a restart request Description of problem: During a service restart or a reload the service fails Version-Release number of selected component (if applicable): ceph-iscsi-config 1.3 How reproducible: Steps to Reproduce: 1. create a gateway configuration 2. issue a systemctl reload rbd-target-gw 3. Actual results: service enters a failed state Expected results: A reload should check the run time settings match the configuration Additional info:
patch currently being tested
Followed the steps as mentioned in the description. Created a gw configuration and on a co-located gw node reloaded the gw service and it is failing to reload.. [root@magna012 ~]# systemctl status rbd-target-gw ● rbd-target-gw.service - Setup system to export rbd images through LIO Loaded: loaded (/usr/lib/systemd/system/rbd-target-gw.service; enabled; vendor preset: disabled) Active: active (running) since Tue 2016-11-01 03:18:29 UTC; 1 weeks 1 days ago Main PID: 161959 (rbd-target-gw) CGroup: /system.slice/rbd-target-gw.service └─161959 /usr/bin/python -u /usr/bin/rbd-target-gw Nov 01 03:18:29 magna012 systemd[1]: Starting Setup system to export rbd images through LIO... Nov 01 03:18:29 magna012 rbd-target-gw[161959]: Processing osd blacklist entries for this node Nov 01 03:18:29 magna012 rbd-target-gw[161959]: No OSD blacklist entries found Nov 01 03:18:29 magna012 rbd-target-gw[161959]: Reading the configuration object to update local LIO configuration Nov 01 03:18:30 magna012 rbd-target-gw[161959]: Processing Gateway configuration Nov 01 03:18:30 magna012 rbd-target-gw[161959]: (Gateway.load_config) successfully loaded existing target definition Nov 01 03:18:30 magna012 rbd-target-gw[161959]: Processing LUN configuration Nov 01 03:18:33 magna012 rbd-target-gw[161959]: (Gateway.load_config) successfully loaded existing target definition Nov 01 03:18:34 magna012 rbd-target-gw[161959]: Processing client configuration Nov 01 03:18:34 magna012 rbd-target-gw[161959]: iSCSI configuration load complete [root@magna012 ~]# systemctl reload rbd-target-gw [root@magna012 ~]# systemctl status rbd-target-gw ● rbd-target-gw.service - Setup system to export rbd images through LIO Loaded: loaded (/usr/lib/systemd/system/rbd-target-gw.service; enabled; vendor preset: disabled) Active: failed (Result: exit-code) since Wed 2016-11-09 19:00:24 UTC; 1s ago Process: 3030481 ExecReload=/usr/bin/kill -HUP $MAINPID (code=exited, status=0/SUCCESS) Main PID: 161959 (code=exited, status=1/FAILURE) Nov 09 19:00:24 magna012 rbd-target-gw[161959]: File "/usr/lib/python2.7/site-packages/rtslib_fb/tcm.py", line 677, in __init__ Nov 09 19:00:24 magna012 rbd-target-gw[161959]: super(BlockStorageObject, self).__init__(name, 'lookup') Nov 09 19:00:24 magna012 rbd-target-gw[161959]: File "/usr/lib/python2.7/site-packages/rtslib_fb/tcm.py", line 52, in __init__ Nov 09 19:00:24 magna012 rbd-target-gw[161959]: self._backstore = _Backstore(name, type(self), mode) Nov 09 19:00:24 magna012 rbd-target-gw[161959]: File "/usr/lib/python2.7/site-packages/rtslib_fb/tcm.py", line 900, in __init__ Nov 09 19:00:24 magna012 rbd-target-gw[161959]: (self._plugin, name)) Nov 09 19:00:24 magna012 rbd-target-gw[161959]: rtslib_fb.utils.RTSLibNotInCFS: Storage object block/rbd.test10 not found Nov 09 19:00:24 magna012 systemd[1]: rbd-target-gw.service: main process exited, code=exited, status=1/FAILURE Nov 09 19:00:24 magna012 systemd[1]: Unit rbd-target-gw.service entered failed state. Nov 09 19:00:24 magna012 systemd[1]: rbd-target-gw.service failed. [root@magna012 ~]# systemctl status rbd-target-gw ● rbd-target-gw.service - Setup system to export rbd images through LIO Loaded: loaded (/usr/lib/systemd/system/rbd-target-gw.service; enabled; vendor preset: disabled) Active: failed (Result: exit-code) since Wed 2016-11-09 19:00:24 UTC; 3s ago Process: 3030481 ExecReload=/usr/bin/kill -HUP $MAINPID (code=exited, status=0/SUCCESS) Main PID: 161959 (code=exited, status=1/FAILURE) Nov 09 19:00:24 magna012 rbd-target-gw[161959]: File "/usr/lib/python2.7/site-packages/rtslib_fb/tcm.py", line 677, in __init__ Nov 09 19:00:24 magna012 rbd-target-gw[161959]: super(BlockStorageObject, self).__init__(name, 'lookup') Nov 09 19:00:24 magna012 rbd-target-gw[161959]: File "/usr/lib/python2.7/site-packages/rtslib_fb/tcm.py", line 52, in __init__ Nov 09 19:00:24 magna012 rbd-target-gw[161959]: self._backstore = _Backstore(name, type(self), mode) Nov 09 19:00:24 magna012 rbd-target-gw[161959]: File "/usr/lib/python2.7/site-packages/rtslib_fb/tcm.py", line 900, in __init__ Nov 09 19:00:24 magna012 rbd-target-gw[161959]: (self._plugin, name)) Nov 09 19:00:24 magna012 rbd-target-gw[161959]: rtslib_fb.utils.RTSLibNotInCFS: Storage object block/rbd.test10 not found Nov 09 19:00:24 magna012 systemd[1]: rbd-target-gw.service: main process exited, code=exited, status=1/FAILURE Nov 09 19:00:24 magna012 systemd[1]: Unit rbd-target-gw.service entered failed state. Nov 09 19:00:24 magna012 systemd[1]: rbd-target-gw.service failed. ========================================== The gw service failed to reload
Issue has been fixed in a later release. Not seen in current ceph-iscsi-config 2.3 release Adding a needinfo for Tejas to confirm that the rbd-target-gw stop/start issue is resolved in the current release.
Yes I agree Jason, this is a different issue. Also I am not seeing it anymore. The original issue is already resolved. Paul, could you please move this BZ to the necessary state. Thanks, Tejas
Not seeing this issue in the latest builds: ceph-iscsi-config-2.3-5.el7cp.noarch ceph-iscsi-cli-2.5-4.el7cp.noarch Moving to verified.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2017:3387