Red Hat Bugzilla – Bug 1633096
The "srp_daemon" is not starting automatically
Last modified: 2018-11-02 06:27:37 EDT
Description of problem: The "srp_daemon" is not starting automatically after rebooting the server. I installed the package, and enabled it, but after the reboot I have to manually start the service, as described below. Version-Release number of selected component (if applicable): Red Hat Enterprise Linux 7.6 beta x86_64 How reproducible: I enabled the service [root@dbt2 ~]# systemctl enable srp_daemon Created symlink from /etc/systemd/system/remote-fs-pre.target.wants/srp_daemon.service to /usr/lib/systemd/system/srp_daemon.service. [root@dbt2 ~]# Rebooted the machine, and the service is not working. First the output of "multipath" is showing only the local disk [root@dbt2 ~]# multipath -r Sep 24 14:33:19 | 3600605b00bba92901fc88b9112e47264: ignoring map [root@dbt2 ~]# [root@dbt2 ~]# systemctl status srp_daemon ● srp_daemon.service - Daemon that discovers and logs in to SRP target systems Loaded: loaded (/usr/lib/systemd/system/srp_daemon.service; enabled; vendor preset: disabled) Active: inactive (dead) Docs: man:srp_daemon file:/etc/srp_daemon.conf [root@dbt2 ~]# [root@dbt2 ~]# [root@dbt2 ~]# systemctl start srp_daemon [root@dbt2 ~]# [root@dbt2 ~]# systemctl status srp_daemon ● srp_daemon.service - Daemon that discovers and logs in to SRP target systems Loaded: loaded (/usr/lib/systemd/system/srp_daemon.service; enabled; vendor preset: disabled) Active: active (exited) since Mon 2018-09-24 14:33:55 UTC; 1s ago Docs: man:srp_daemon file:/etc/srp_daemon.conf Process: 14167 ExecStart=/usr/libexec/srp_daemon/start_on_all_ports (code=exited, status=0/SUCCESS) Main PID: 14167 (code=exited, status=0/SUCCESS) CGroup: /system.slice/srp_daemon.service Sep 24 14:33:55 dbt2 systemd[1]: Starting Daemon that discovers and logs in to SRP target systems... Sep 24 14:33:55 dbt2 systemd[1]: Started Daemon that discovers and logs in to SRP target systems. [root@dbt2 ~]# Actual results: The daemon is not working. Expected results: The output of "multipath" to show all my devices instead of only showing my local disk.
It looks like remote-fs-pre.target should start this up. What does # systemctl list-units --type target --all show. I assume that remote_fs.target is active. I'm reassigning this to rdma-core, since without paths, there is nothing for multipath to do.
srp issue, so I will take it.
Can you please provide the sosreport? https://access.redhat.com/solutions/3592
I can't reproduce this issue in-house. SRP daemon works for me. [root@rdma-dev-10 ~]$ cat /var/log/boot.log | grep -i srp Starting Daemon that discovers and logs in to SRP target systems... [ OK ] Started Daemon that discovers and logs in to SRP target systems. Starting Load RDMA modules from /etc/rdma/modules/srp_daemon.conf... [ OK ] Created slice system-srp_daemon_port.slice. [ OK ] Started Load RDMA modules from /etc/rdma/modules/srp_daemon.conf. [root@rdma-dev-10 ~]$ rpm -q kernel srp_daemon kernel-3.10.0-954.el7.x86_64 srp_daemon-17.2-3.el7.x86_64 [root@rdma-dev-10 ~]$ cat /etc/srp_daemon.conf | grep -v '^#' a ioc_guid=f4521403007be0e1 a ioc_guid=f4521403007be0e2 d [root@rdma-dev-10 ~]$ /sbin/service srp_daemon status Redirecting to /bin/systemctl status srp_daemon.service ● srp_daemon.service - Daemon that discovers and logs in to SRP target systems Loaded: loaded (/usr/lib/systemd/system/srp_daemon.service; enabled; vendor preset: disabled) Active: active (exited) since Fri 2018-09-28 21:28:46 EDT; 3min 24s ago Docs: man:srp_daemon file:/etc/srp_daemon.conf Process: 4907 ExecStart=/usr/libexec/srp_daemon/start_on_all_ports (code=exited, status=0/SUCCESS) Main PID: 4907 (code=exited, status=0/SUCCESS) CGroup: /system.slice/srp_daemon.service Sep 28 21:28:46 rdma-dev-10.lab.bos.redhat.com systemd[1]: Starting Daemon that discovers and logs in to SRP target systems... Sep 28 21:28:46 rdma-dev-10.lab.bos.redhat.com systemd[1]: Started Daemon that discovers and logs in to SRP target systems. [root@rdma-dev-10 ~]$ [root@rdma-dev-10 ~]$ multipath -r reload: mpatha (3600140568c40e3347164ad08419b5ca0) undef LIO-ORG ,ram1 size=100M features='0' hwhandler='0' wp=undef |-+- policy='service-time 0' prio=1 status=undef | `- 103:0:0:0 sdc 8:32 active ready running `-+- policy='service-time 0' prio=1 status=undef `- 92:0:0:0 sdb 8:16 active ready running reload: mpathb (3600140542a8899590fd4660a99019573) undef LIO-ORG ,ram2 size=100M features='0' hwhandler='0' wp=undef |-+- policy='service-time 0' prio=1 status=undef | `- 103:0:0:1 sde 8:64 active ready running `-+- policy='service-time 0' prio=1 status=undef `- 92:0:0:1 sdd 8:48 active ready running reload: mpathc (360014058bcc18cbaff24a6bb77a1a653) undef LIO-ORG ,ram11 size=100M features='0' hwhandler='0' wp=undef |-+- policy='service-time 0' prio=1 status=undef | `- 103:0:0:10 sdw 65:96 active ready running `-+- policy='service-time 0' prio=1 status=undef `- 92:0:0:10 sdv 65:80 active ready running reload: mpathd (360014056fe1d29d63d94801a4e89fbde) undef LIO-ORG ,ram12 size=100M features='0' hwhandler='0' wp=undef |-+- policy='service-time 0' prio=1 status=undef | `- 103:0:0:11 sdx 65:112 active ready running `-+- policy='service-time 0' prio=1 status=undef `- 92:0:0:11 sdy 65:128 active ready running reload: mpathe (36001405f499993e80ab471489966bfae) undef LIO-ORG ,ram13 size=100M features='0' hwhandler='0' wp=undef |-+- policy='service-time 0' prio=1 status=undef | `- 103:0:0:12 sdz 65:144 active ready running `-+- policy='service-time 0' prio=1 status=undef `- 92:0:0:12 sdaa 65:160 active ready running reload: mpathf (36001405920dc70b41f346188a89d4411) undef LIO-ORG ,ram14 size=100M features='0' hwhandler='0' wp=undef |-+- policy='service-time 0' prio=1 status=undef | `- 103:0:0:13 sdab 65:176 active ready running `-+- policy='service-time 0' prio=1 status=undef `- 92:0:0:13 sdac 65:192 active ready running reload: mpathg (36001405c114f6b0ba49497588802be78) undef LIO-ORG ,ram15 size=100M features='0' hwhandler='0' wp=undef |-+- policy='service-time 0' prio=1 status=undef | `- 103:0:0:14 sdad 65:208 active ready running `-+- policy='service-time 0' prio=1 status=undef `- 92:0:0:14 sdae 65:224 active ready running reload: mpathh (360014052a79eeda074749898ca0b1d2e) undef LIO-ORG ,ram16 size=100M features='0' hwhandler='0' wp=undef |-+- policy='service-time 0' prio=1 status=undef | `- 103:0:0:15 sdaf 65:240 active ready running `-+- policy='service-time 0' prio=1 status=undef `- 92:0:0:15 sdag 66:0 active ready running reload: mpathi (360014055d8f4bc5ad2243f8b9e3f6de0) undef LIO-ORG ,ram3 size=100M features='0' hwhandler='0' wp=undef |-+- policy='service-time 0' prio=1 status=undef | `- 103:0:0:2 sdf 8:80 active ready running `-+- policy='service-time 0' prio=1 status=undef `- 92:0:0:2 sdg 8:96 active ready running reload: mpathj (36001405b97f43ffce6e423885cdbe785) undef LIO-ORG ,ram4 size=100M features='0' hwhandler='0' wp=undef |-+- policy='service-time 0' prio=1 status=undef | `- 103:0:0:3 sdh 8:112 active ready running `-+- policy='service-time 0' prio=1 status=undef `- 92:0:0:3 sdi 8:128 active ready running reload: mpathk (36001405e77bf3879c5f4f96af5ebcdf0) undef LIO-ORG ,ram5 size=100M features='0' hwhandler='0' wp=undef |-+- policy='service-time 0' prio=1 status=undef | `- 103:0:0:4 sdk 8:160 active ready running `-+- policy='service-time 0' prio=1 status=undef `- 92:0:0:4 sdj 8:144 active ready running reload: mpathl (360014057ed2e9c2ce8347dbb20a32afb) undef LIO-ORG ,ram6 size=100M features='0' hwhandler='0' wp=undef |-+- policy='service-time 0' prio=1 status=undef | `- 103:0:0:5 sdl 8:176 active ready running `-+- policy='service-time 0' prio=1 status=undef `- 92:0:0:5 sdm 8:192 active ready running reload: mpathm (360014054df7e2e1bc3e42819f880b161) undef LIO-ORG ,ram7 size=100M features='0' hwhandler='0' wp=undef |-+- policy='service-time 0' prio=1 status=undef | `- 103:0:0:6 sdo 8:224 active ready running `-+- policy='service-time 0' prio=1 status=undef `- 92:0:0:6 sdn 8:208 active ready running reload: mpathn (360014058b31f6f9083d4c8986720a1b0) undef LIO-ORG ,ram8 size=100M features='0' hwhandler='0' wp=undef |-+- policy='service-time 0' prio=1 status=undef | `- 103:0:0:7 sdq 65:0 active ready running `-+- policy='service-time 0' prio=1 status=undef `- 92:0:0:7 sdp 8:240 active ready running reload: mpatho (360014053a1fec06a4514265a7686ab3a) undef LIO-ORG ,ram9 size=100M features='0' hwhandler='0' wp=undef |-+- policy='service-time 0' prio=1 status=undef | `- 103:0:0:8 sds 65:32 active ready running `-+- policy='service-time 0' prio=1 status=undef `- 92:0:0:8 sdr 65:16 active ready running reload: mpathp (360014054c5b1ef13cc64246910b42b0c) undef LIO-ORG ,ram10 size=100M features='0' hwhandler='0' wp=undef |-+- policy='service-time 0' prio=1 status=undef | `- 103:0:0:9 sdt 65:48 active ready running `-+- policy='service-time 0' prio=1 status=undef `- 92:0:0:9 sdu 65:64 active ready running [root@rdma-dev-10 ~]$
Sorry for not uploading the sosreports. At the moment, I can't test the machine. The server is dual boot, and the customer is using the production environment. Maybe later this week I have the chance of booting the server with Red Hat 7.6.
Created attachment 1492468 [details] SOS report from dbt2 server
Please boot with these parameters on the kernel command line: systemd.log_level=debug systemd.log_target=kmsg log_buf_len=16M printk.devkmsg=on And then feedback 'dmesg' output to us. Thanks
(In reply to Honggang LI from comment #8) > Please boot with these parameters on the kernel command line: > > systemd.log_level=debug systemd.log_target=kmsg log_buf_len=16M > printk.devkmsg=on > > > And then feedback 'dmesg' output to us. Thanks OK. As soon as I have a chance to reboot the server again, I'll get the "dmesg" to you. Today the customer is using the machine.
This case can be closed. The customer asked to uninstall the RHEL beta from the machine.