Description of problem:
after installing pam_ssh_agent_auth and adding
auth sufficient pam_ssh_agent_auth.so file=~/.ssh/authorized_keys
line either to /etc/pam.d/sudo or via authselect system is authorizing properly in case sudoedit is invoked but there is no ssh-agent authentication in case of running sudo.


Version-Release number of selected component (if applicable):
0.10.4-5.1.fc36.1

How reproducible:
Every time. Tested with Fedora 36 only.

Steps to Reproduce:
1. sudo dnf -y install pam_ssh_agent_auth
2. add following line to /etc/pam.d/sudo or to /etc/authselect/system-auth: auth sufficient pam_ssh_agent_auth.so file=~/.ssh/authorized_keys
3. run sudo echo test and sudoedit test.file (regardless locally or from remote)

Actual results:
with sudo I am getting password prompt, with sudoedit only have to touch yubikey

Expected results:
consistent behaviour: system to wait for yubikey to be touched

Additional info:
sudo log (not working)

maj 12 13:15:36 host sudo[37577]: Beginning pam_ssh_agent_auth for user user
maj 12 13:15:36 host sudo[37577]: Attempting authentication: `user' as `user' using /home/user/.ssh/authorized_keys
maj 12 13:15:36 host sudo[37577]: No ssh-agent could be contacted
maj 12 13:15:36 host sudo[37577]: Failed Authentication: `user' as `user' using /home/user/.ssh/authorized_keys
maj 12 13:15:44 host sudo[37577]: pam_unix(sudo:auth): conversation failed
maj 12 13:15:44 host sudo[37577]: pam_unix(sudo:auth): auth could not identify password for [user]

sudoedit log (working)

maj 12 13:16:01 host sudoedit[37690]: Beginning pam_ssh_agent_auth for user user
maj 12 13:16:01 host sudoedit[37690]: Attempting authentication: `user' as `user' using /home/user/.ssh/authorized_keys
maj 12 13:16:01 host sudoedit[37690]: Contacted ssh-agent of user user (1001)
maj 12 13:16:01 host sudoedit[37690]: trying public key file /home/user/.ssh/authorized_keys
maj 12 13:16:01 host sudoedit[37690]: auth_secure_filename: checking for uid: 1001
maj 12 13:16:01 host sudoedit[37690]: secure_filename: checking '/home/user/.ssh'
maj 12 13:16:01 host sudoedit[37690]: secure_filename: checking '/home/user'
maj 12 13:16:01 host sudoedit[37690]: secure_filename: terminating check at '/home/user'
maj 12 13:16:01 host sudoedit[37690]: matching key found: file/command /home/user/.ssh/authorized_keys, line 1
maj 12 13:16:01 host sudoedit[37690]: Found matching ECDSA-SK key: /key/