+++ This bug was initially created as a clone of Bug #2003151 +++

Description of problem:
When listing resource agents using 'crm_resource', agents of 'service' standard are listed which contain '@' in their name. Yet, crm_resource is unable to provide metadata for them.


Version-Release number of selected component (if applicable):
pacemaker-2.1.0-8.el8


How reproducible:
always, easily


Steps to Reproduce:
1. list 'service' agents, notice there are few with '@'
# crm_resource --list-agents service | grep @
autovt@
chrony-dnssrv@
chrony-dnssrv@.timer
container-getty@
getty@
lvm2-pvscan@
rdma-load-modules@
serial-getty@
sshd-keygen@
sshd@
systemd-backlight@
systemd-coredump@
systemd-fsck@
systemd-hibernate-resume@
teamd@
user-runtime-dir@
user@

2. try to fetch metadata of such an agent
# crm_resource --show-metadata service:sshd@
crm_resource: Metadata query for service:sshd@ failed: No such device or address
Error performing operation: No such object


Actual results:
Unable to fetch metadata.


Expected results:
Either be able to provide metadata for such agents, or don't list them.


Additional info:
The 'service' standard is an alias to lsb, systemd and other init systems. Agents with '@' in their name come most likely from systemd. It seems, however, that pacemaker fails in figuring out the actual service class providing the specified agent:

# crm_resource --show-metadata service:sshd@ -VVVV
(crm_log_args)  notice: Invoked: crm_resource --show-metadata service:sshd@ -VVVV
(expand_resource_class)         info: Assuming resource class lsb for agent sshd@ for sshd@
(lrmd_api_get_metadata_params)  error: Failed to retrieve meta-data for service:(null):sshd@
(lrmd_api_disconnect)   info: Disconnecting IPC local executor connection
crm_resource: Metadata query for service:sshd@ failed: No such device or address
Error performing operation: No such object
(crm_xml_cleanup)       info: Cleaning up memory from libxml2
(crm_exit)      info: Exiting crm_resource | with status 105

Note: expand_resource_class assumes lsb:sshd@ instead of systemd:sshd@ even though there seem to be no lsb initscripts installed

# ls -la /etc/init.d/
total 32
drwxr-xr-x.  2 root root  4096 May 24 14:07 .
drwxr-xr-x. 10 root root  4096 Feb 15  2021 ..
-rw-r--r--.  1 root root 18434 Feb 15  2021 functions
-rw-r--r--.  1 root root  1161 Mar 11  2021 README

--- Additional comment from Ken Gaillot on 2021-09-10 19:56:42 UTC ---

You're right, this is a bug in service handling. I think the issue is ultimately in systemd_unit_by_name() not handling names with "@" specially.

--- Additional comment from Ken Gaillot on 2021-09-20 21:38:11 UTC ---

It's worth noting that any of these forms already work for a systemd unit that uses "@":

    crm_resource --show-metadata service:$UNIT
    crm_resource --show-metadata service:$UNIT@$SUBUNIT
    crm_resource --show-metadata systemd:$UNIT@

so the problem is limited to the one case

    crm_resource --show-metadata service:$UNIT@

--- Additional comment from Chris Lumens on 2021-11-30 18:55:08 UTC ---

(In reply to Ken Gaillot from comment #2)
> It's worth noting that any of these forms already work for a systemd unit
> that uses "@":
> 
>     crm_resource --show-metadata service:$UNIT
>     crm_resource --show-metadata service:$UNIT@$SUBUNIT
>     crm_resource --show-metadata systemd:$UNIT@
> 
> so the problem is limited to the one case
> 
>     crm_resource --show-metadata service:$UNIT@

It's a little trickier than that.  I think in this case, the top three commands work only because there is both an sshd.service and an sshd@.service file.  If you try something that doesn't have both (like user@service), the result is:

[root@cluster01 system]# crm_resource --show-metadata service:user
crm_resource: Metadata query for service:user failed: No such device or address
Error performing operation: No such object
[root@cluster01 system]# crm_resource --show-metadata service:user@
crm_resource: Metadata query for service:user@ failed: No such device or address
Error performing operation: No such object

--- Additional comment from Chris Lumens on 2021-11-30 19:17:30 UTC ---

If you add enough -Vs, you'll see what is going on:

# crm_resource --show-metadata service:sshd@ -VVVVVVV
...
(pcmk_dbus_find_error)  trace: DBus reply indicated error 'org.freedesktop.DBus.Error.InvalidArgs' (Unit name sshd@.service is missing the instance name.)
(pcmk_dbus_find_error)  trace: DBus reply indicated error 'org.freedesktop.DBus.Error.InvalidArgs' (Unit name sshd@.service is missing the instance name.)
(expand_resource_class)         info: Assuming resource class lsb for agent sshd@ for sshd@
(services__get_lsb_metadata)    trace: Looking into /etc/rc.d/init.d/sshd@

And sure enough, if you try cramming some text after the @ sign (the instance name):
# crm_resource --show-metadata service:sshd@root
<?xml version="1.0"?>
...

It works.

--- Additional comment from Ken Gaillot on 2021-12-01 00:10:52 UTC ---

Fixed in upstream master branch by commit 09ef95a