Bug 1394977

Summary: VPN no longer works ("No agents were available for this request")
Product: [Fedora] Fedora Reporter: Andrey Yurovsky <yurovsky>
Component: NetworkManager-openconnectAssignee: Lubomir Rintel <lrintel>
Status: CLOSED EOL QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 25CC: christopherjabrams, dadreggors, dcbw, dwmw2, ivzave, jbirch, jonas, keirlawson, lrintel, marcin_stec, mvillarjuan, nekohayo, nmavrogi, robinlee.sysu, thaller
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-12-12 10:32:40 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
jbirch usr_lib__nm-openconnect-service.name none

Description Andrey Yurovsky 2016-11-14 23:05:36 UTC
Description of problem:

I had an openconnect VPN set up and upgraded to Fedora 25.  It's no longer possible to connect to the VPN and creating additional VPN connections results in the same failure.  journal shows:

Nov 14 14:45:10 carbon gnome-shell[1663]: JS LOG: Invalid VPN service type (cannot find authentication binary)
Nov 14 14:45:10 carbon gnome-shell[1663]: JS LOG: Invalid VPN service type (cannot find authentication binary)
Nov 14 14:45:10 carbon NetworkManager[926]: <error> [1479163510.5251] vpn-connection[0x55d4452b4520,eb7e99fc-7ef6-423b-862e-1998145590f6,"foo",0]: Failed to request VPN secrets #3: No agents were available for this request.


Version-Release number of selected component (if applicable):


How reproducible:

100%


Steps to Reproduce:
1. use Networking to add a VPN connection (openconnect)
2. journalctl -xf
3. try to connect via the NetworkManager UI

Actual results:

Above failure

Expected results:

Connection

Additional info:

Worked in Fedora 23 and 24

Comment 1 Thomas Haller 2016-11-15 08:20:50 UTC
do you have NetworkManager-openconnect-gnome installed?
do you use Gnome3, or nm-applet? Make sure one of them is running.
if you start nm-applet, does it work?

Comment 2 Andrey Yurovsky 2016-11-16 03:32:52 UTC
I do have NetworkManager-openconnect-gnome installed, I use the default Gnome3 desktop out of the box.

If I manually start nm-applet, things do work (thanks for suggesting that) -- so it seems that's something to do with it.

Comment 3 Thomas Haller 2016-11-16 09:18:29 UTC
when NetworkManager needs a password (secret), it requires a "secret-agent" to be around. That is a process that can ask for the password, or retrieve it somehow else.

nm-applet is such a secret agent. gnome-shell (from Gnome 3) too.


VPN secrets are a bit special, so it seems that gnome-shell has problems asking for nm-openconnect secrets...

Comment 4 Jason Birch 2016-11-23 10:25:20 UTC
I ran into this one upon upgrading from Fedora 24 to Fedora 25. It seems that both nmcli and the gnome-shell widget thingo lost the ability to get those secrets. Both using the GUI or nmcli to try and launch a VPN connection would just leave a `/usr/libexec/nm-openconnect-service --bus-name org.freedesktop.NetworkManager.openconnect.Connection_XX` process floating around.

I'm able to work around it for now by running `nmcli --ask c u <connection name>`.

Comment 5 Jason Birch 2016-11-23 10:31:31 UTC
I should add that trying to use the gnome widget to edit the connection shows "(Error: unable to load VPN connection editor)", along with the journalctl lines:

Nov 23 21:29:45 sleepygary gnome-shell[10653]: JS LOG: Invalid VPN service type (cannot find authentication binary)
Nov 23 21:29:45 sleepygary gnome-control-c[14322]: g_variant_unref: assertion 'value != NULL' failed

I don't know enough about the different bits in play here to know if this is related.

Comment 6 Thomas Haller 2016-11-23 12:53:52 UTC
you say in comment 4:

> It seems that both nmcli and the gnome-shell widget thingo lost the ability to get those secrets.

and

> I'm able to work around it for now by running `nmcli --ask c u <connection name>`.


could you clarify that?


Also: what means "leave ... floating around"? While the activation is in process, and secrets are requests, it would be expected. But after activation fails (due to missing secrets), the process should be terminated.

Comment 7 Lubomir Rintel 2016-11-23 13:20:43 UTC
(In reply to Jason Birch from comment #5)
> I should add that trying to use the gnome widget to edit the connection
> shows "(Error: unable to load VPN connection editor)", along with the
> journalctl lines:
> 
> Nov 23 21:29:45 sleepygary gnome-shell[10653]: JS LOG: Invalid VPN service
> type (cannot find authentication binary)

Both of these strongly indicate that the NetworkManager-openconnect-gnome package might be missing or not corresponding to your .name file.

I'm wondering if you could share your /etc/NetworkManager/VPN/nm-openconnect-service.name and /usr/lib/NetworkManager/VPN/nm-openconnect-service.name?

Also, please verify your packages are allright by running "rpm -V NetworkManager-openconnect NetworkManager-openconnect-gnome" (please share the output).

(In reply to Thomas Haller from comment #6)
> > I'm able to work around it for now by running `nmcli --ask c u <connection name>`.
> 
> could you clarify that?

Thomas, nmcli contains extra magic to deal with openconnect that does not need neither the plugin nor the auth dialog (it just rans openconnect directly to get the cookie).

Comment 8 Jason Birch 2016-11-24 01:01:58 UTC
(In reply to Thomas Haller from comment #6)

> could you clarify that?

Can do! At first I thought this was a gnome issue, so I wanted to try getting the VPN up the way I do on another machine, using nmcli. I tried invoking what I normally would - `nmci c u <connection name>`:

$ nmcli c u Atlassian\ AU\ Full 
A password is required to connect to 'Atlassian AU Full'.
Warning: password for 'vpn.secrets.gateway' not given in 'passwd-file' and nmcli cannot ask without '--ask' option.
Error: Connection activation failed: no valid VPN secrets.

This is what I mean by "lost the ability to get those secrets". I discovered `--ask` from this useful error message, which then will prompt for username and password (and second password in this case, because openconnect).

> Also: what means "leave ... floating around"? While the activation is in
> process, and secrets are requests, it would be expected. But after
> activation fails (due to missing secrets), the process should be terminated.

After running the above command (no --ask), or attempting to toggle the VPN via the GNOME UI, leaves a deparented process hanging, that doesn't look like it's doing anything. This process has an incrementing integer at the end of it:

$ nmcli c u Atlassian\ AU\ Full >/dev/null 2>&1
$ nmcli c u Atlassian\ AU\ Full >/dev/null 2>&1
$ nmcli c u Atlassian\ AU\ Full >/dev/null 2>&1
$ ps faux | grep nm-openconnect
jbirch   24172  0.0  0.0 119372   984 pts/2    S+   11:56   0:00  |       \_ grep --color=auto nm-openconnect
root     23279  0.0  0.0 328568  6788 ?        Sl   11:55   0:00 /usr/libexec/nm-openconnect-service --bus-name org.freedesktop.NetworkManager.openconnect.Connection_42
root     23354  0.0  0.0 328568  6816 ?        Sl   11:55   0:00 /usr/libexec/nm-openconnect-service --bus-name org.freedesktop.NetworkManager.openconnect.Connection_43
root     23483  0.0  0.0 328568  6832 ?        Sl   11:56   0:00 /usr/libexec/nm-openconnect-service --bus-name org.freedesktop.NetworkManager.openconnect.Connection_44

In the process of testing this, I noticed that these processes disappear after about 4 minutes. It looks like at some point the base process decides to end itself after hanging on on restart_syscall for a very long time:

$ sudo strace -fp 24150
[sudo] password for jbirch: 
strace: Process 24150 attached with 3 threads
[pid 24154] restart_syscall(<... resuming interrupted poll ...> <unfinished ...>
[pid 24153] restart_syscall(<... resuming interrupted poll ...> <unfinished ...>
[pid 24150] restart_syscall(<... resuming interrupted poll ...>) = 0
[pid 24150] write(3, "\1\0\0\0\0\0\0\0", 8) = 8
[pid 24150] futex(0x55a43921d920, FUTEX_WAKE_PRIVATE, 2147483647) = 0
[pid 24150] exit_group(0)               = ?
[pid 24154] +++ exited with 0 +++
[pid 24153] +++ exited with 0 +++
+++ exited with 0 +++

Comment 9 Jason Birch 2016-11-24 01:04:56 UTC
Created attachment 1223483 [details]
jbirch usr_lib__nm-openconnect-service.name

/usr/lib/NetworkManager/VPN/nm-openconnect-service.name attached.

Comment 10 Jason Birch 2016-11-24 01:11:53 UTC
(In reply to Lubomir Rintel from comment #7)

> I'm wondering if you could share your
> /etc/NetworkManager/VPN/nm-openconnect-service.name and
> /usr/lib/NetworkManager/VPN/nm-openconnect-service.name?

/etc/NetworkManager/VPN/nm-openconnect-service.name does not exist on my system - only corresponding files for openvpn, pptp, vpnc.

I have attached /usr/lib/NetworkManager/VPN/nm-openconnect-service.name. It lives alongside the same selection of files (for openvpn, pptp, vpnc).

> Also, please verify your packages are allright by running "rpm -V
> NetworkManager-openconnect NetworkManager-openconnect-gnome" (please share
> the output).

Ah this is interesting!

$ sudo rpm -V NetworkManager-openconnect NetworkManager-openconnect-gnome
package NetworkManager-openconnect-gnome is not installed

Taking a look at the dnf system-upgrade, it looks like the following generally related things happened (output from dnf history info)

    Upgraded    NetworkManager-openconnect-1.2.2-1.fc24.x86_64                     @updates-testing/24
    Upgrade                                1.2.3-0.20160606git5009f9.fc25.x86_64   @@commandline

Worth noting is that all of the aforementioned selection of services (openvpn, pptp, vpnc) have a corresponding NetworkManager-X-gnome package that was installed. openconnect does not.

I uh wonder how this ever worked. Shall I install NetworkManager-openconnect-gnome and see if things magically work?

Comment 11 Jason Birch 2016-11-24 04:28:33 UTC
Curiosity got the better of me; installing NetworkManager-openconnect-gnome and logging out and in again fixes the issue (for both gnome, and nmcli without the --ask option).

It also creates /etc/NetworkManager/VPN/nm-openconnect-service.name and the referenced /usr/libexec/nm-openconnect-auth-dialog which was also missing (I suspect these are just part of the package)

Comment 12 Thomas Haller 2016-11-24 12:16:13 UTC
Jason, if you didn't have NetworkManager-openconnect-gnome installed, it is expected that neither nm-applet nor gnome-shell is able to prompt for a password.

It is also expected, that nmcli only asks for a password when called with --ask.

So, your issue is different then the original reporter.


Also, a lingering nm-openconnect process is a separate issue.

Comment 13 Jason Birch 2016-11-24 18:12:06 UTC
Hm, apologies for muddling the issue. The symptoms were the same as the original reporter (IE, worked on Fedora 24, broke on update to Fedora 25, same error messages seen in journal).

Comment 14 Nikos Mavrogiannopoulos 2016-11-25 14:49:33 UTC
I can verify that after upgrading to F25 I saw the same issue. NetworkManager-openconnect-gnome disappeared. The error message on the NM gui was not very helpful.

Comment 15 Thomas Haller 2016-11-30 08:04:51 UTC
regarding the hanging nm-openconnect-service process from comment 8, that should be fixed upstream in NetworkManager: https://cgit.freedesktop.org/NetworkManager/NetworkManager/commit/?id=743d372d72878219a229ee420e477958939aa1fb


This bug is not about not having NetworkManager-openconnect-gnome package installed. That would be bug 1398425. So, comment 4 to 14 are unrelated to this bug.

Comment 16 David Dreggors 2017-02-22 02:15:21 UTC
I seem to be having the same issue in Fedora 25, I can no longer connect to openconnect vpn.

I keep getting this:

$ nmcli con up id Groupon passwd-file /tmp/.gvpn
A password is required to connect to 'Groupon'.
Warning: password for 'vpn.secrets.gateway' not given in 'passwd-file' and nmcli cannot ask without '--ask' option.
Error: Connection activation failed: no valid VPN secrets.


Note I gave it a password file and the contents have always been this:

vpn.secrets.Xauth password:XXXXXXXXXXX


I have tried deleting the connection and recreating with a different name and even tried connecting to new connections in Gnome 3 UI. This silently fails immediately.


As to installed packages, I have removed and reinstalled the packages and confirmed their files are found:

$ rpm -qa|grep NetworkManager-openconnect
NetworkManager-openconnect-gnome-1.2.4-3.fc25.x86_64
NetworkManager-openconnect-1.2.4-3.fc25.x86_64



$ rpm -V NetworkManager-openconnect NetworkManager-openconnect-gnome
$ echo $?
0

Comment 17 David Dreggors 2017-02-22 02:35:29 UTC
Update:

I installed network-manager-applet given that the bug reports I keep reading all mentioned trying Gnome or nm-applet.

Once installed and I logged out of Gnome and back in... I can not connect on command line or from Gnome NetworkWork Manager menu.

I am not sure why this is needed, but I did not require this before. Even in Fedora 25 after fresh install, all was well. After a couple updates now it is broken until I use nm-applet.

Comment 18 Robin Lee 2017-03-15 14:40:57 UTC
I met the same issue.

$ rpm -q NetworkManager-openconnect NetworkManager-openconnect-gnome
NetworkManager-openconnect-1.2.4-3.fc25.x86_64
NetworkManager-openconnect-gnome-1.2.4-3.fc25.x86_64

Comment 19 Jean-François Fortin Tam 2017-05-25 14:34:03 UTC
"dnf install NetworkManager-openconnect* " and relogging into gnome shell fixed the issue for me (no need for an "applet"). At least it should be installed by default (bug #1398425) but ideally, if some package is missing the GUIs should tell the user about it instead of having to dig into journalctl and bug reports.

Comment 20 Fedora End Of Life 2017-11-16 19:39:50 UTC
This message is a reminder that Fedora 25 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 25. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '25'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version'
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not
able to fix it before Fedora 25 is end of life. If you would still like
to see this bug fixed and are able to reproduce it against a later version
of Fedora, you are encouraged  change the 'version' to a later Fedora
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.

Comment 21 Fedora End Of Life 2017-12-12 10:32:40 UTC
Fedora 25 changed to end-of-life (EOL) status on 2017-12-12. Fedora 25 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.