Bug 1705054 - Security settings are missing after upgrade
Summary: Security settings are missing after upgrade
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: NetworkManager
Version: 30
Hardware: x86_64
OS: Linux
unspecified
unspecified
Target Milestone: ---
Assignee: Beniamino Galvani
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: 1714610
TreeView+ depends on / blocked
 
Reported: 2019-05-01 10:25 UTC by Matthias Summer
Modified: 2019-06-14 00:53 UTC (History)
19 users (show)

Fixed In Version: NetworkManager-1.16.2-1.fc30
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1714610 (view as bug list)
Environment:
Last Closed: 2019-06-14 00:53:25 UTC


Attachments (Terms of Use)
Output journalctl (573.45 KB, text/plain)
2019-05-10 08:27 UTC, Matthias Summer
no flags Details

Description Matthias Summer 2019-05-01 10:25:03 UTC
Description of problem:

After the upgrade from Fedora 29 to Fedora 30 the security-settings on the network-interfaces were missing.


How reproducible:
Every time (already had the issue 6 times).


Steps to Reproduce:
1. configure 802.1x security on any network interface
2. upgrade to Fedora 30

Actual results:
Security settings are off and had to be re-configured.

Expected results:
Security settings still exists.

Comment 1 Matthias Summer 2019-05-01 14:36:14 UTC
Short update - the 802.1x Security is disabled after every reboot.

Comment 2 Matthias Summer 2019-05-08 16:43:35 UTC
We use 802.1x with TLS as Authentication (CAcerfificate/Private key and password). These settings get lost after every reboot. This also happens to a network-profile.

When useing 802.1x for wifi it is the same. Thill a reboot the connection is available. New wifi-connections without 802.1x are still available after a restart.

Comment 3 Beniamino Galvani 2019-05-09 06:37:40 UTC
Can you please paste the content of the /etc/sysconfig/network-scripts/ifcfg-${connection_name} file, with sensitive data redacted?

Comment 4 Beniamino Galvani 2019-05-09 06:47:33 UTC
Also, do you see any warnings in the output of 'journalctl -u NetworkManager -b' about invalid connections?

Comment 5 Matthias Summer 2019-05-09 07:07:45 UTC
The content of the etc/sysconfig/network-scripts/ifcfg-Wired_connection_1-10 is


HWADDR=xx:xx:xx:xx:xx:xx
TYPE=Ethernet
KEY_MGMT=IEEE8021X
IEEE_8021X_EAP_METHODS=TLS
IEEE_8021X_IDENTITY=HOSTNAME.FQDN
IEEE_8021X_CA_CERT=/home/matthias/Documents/network-certs/root_ca2.pem
IEEE_8021X_PRIVATE_KEY=/home/matthias/Documents/network-certs/host.p12
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=dhcp
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME="Wired connection 1"
UUID=5c4d0f64-47d8-37de-ac6c-548e0c8bb037
ONBOOT=yes
AUTOCONNECT_PRIORITY=-999


In the /etc/sysconfig/network-scripts there is a ifcfg-Wired_connection_1-X file for every time connecting to that network:

-rw-r--r--. 1 root root   598 May 17  2018 ifcfg-Wired_connection_1
-rw-r--r--. 1 root root   532 May  1 09:35 ifcfg-Wired_connection_1-1
-rw-r--r--. 1 root root   611 May  9 08:20 ifcfg-Wired_connection_1-10
-rw-r--r--. 1 root root   616 May  1 16:28 ifcfg-Wired_connection_1-2
-rw-r--r--. 1 root root   611 May  1 16:37 ifcfg-Wired_connection_1-3
-rw-r--r--. 1 root root   626 May  1 16:42 ifcfg-Wired_connection_1-4
-rw-r--r--. 1 root root   611 May  2 09:29 ifcfg-Wired_connection_1-5
-rw-r--r--. 1 root root   578 May  3 08:41 ifcfg-Wired_connection_1-6
-rw-r--r--. 1 root root   611 May  3 08:49 ifcfg-Wired_connection_1-7
-rw-r--r--. 1 root root   611 May  7 13:55 ifcfg-Wired_connection_1-8
-rw-r--r--. 1 root root   611 May  8 08:39 ifcfg-Wired_connection_1-9



The warnings from 'journalctl -u NetworkManager -b' are:

May 09 08:18:48 XXXXXXX NetworkManager[1322]: <warn>  [1557382728.3648] ifcfg-rh: loading "/etc/sysconfig/network-scripts/ifcfg-Wired_connection_1-9" fails: Missing certificate for EAP method 'tls'.
May 09 08:18:48 XXXXXXX NetworkManager[1322]: <warn>  [1557382728.3655] ifcfg-rh: loading "/etc/sysconfig/network-scripts/ifcfg-Wired_connection_1-8" fails: Missing certificate for EAP method 'tls'.
May 09 08:18:48 XXXXXXX NetworkManager[1322]: <warn>  [1557382728.3662] ifcfg-rh: loading "/etc/sysconfig/network-scripts/ifcfg-Wired_connection_1-7" fails: Missing certificate for EAP method 'tls'.
May 09 08:18:48 XXXXXXX NetworkManager[1322]: <warn>  [1557382728.3681] ifcfg-rh: loading "/etc/sysconfig/network-scripts/ifcfg-Wired_connection_1-6" fails: Missing certificate for EAP method 'tls'.
May 09 08:18:48 XXXXXXX NetworkManager[1322]: <warn>  [1557382728.3686] ifcfg-rh: loading "/etc/sysconfig/network-scripts/ifcfg-Wired_connection_1-5" fails: Missing certificate for EAP method 'tls'.
May 09 08:18:48 XXXXXXX NetworkManager[1322]: <warn>  [1557382728.3689] ifcfg-rh: loading "/etc/sysconfig/network-scripts/ifcfg-enp0s25" fails: Missing certificate for EAP method 'tls'.
May 09 08:18:48 XXXXXXX NetworkManager[1322]: <warn>  [1557382728.3693] ifcfg-rh: loading "/etc/sysconfig/network-scripts/ifcfg-Wired_connection_1-4" fails: Missing certificate for EAP method 'tls'.
May 09 08:18:48 XXXXXXX NetworkManager[1322]: <warn>  [1557382728.3696] ifcfg-rh: loading "/etc/sysconfig/network-scripts/ifcfg-Wired_connection_1-3" fails: Missing certificate for EAP method 'tls'.
May 09 08:18:48 XXXXXXX NetworkManager[1322]: <warn>  [1557382728.3720] ifcfg-rh: loading "/etc/sysconfig/network-scripts/ifcfg-Wired_connection_1-2" fails: Missing certificate for EAP method 'tls'.
May 09 08:18:48 XXXXXXX NetworkManager[1322]: <warn>  [1557382728.3724] ifcfg-rh: loading "/etc/sysconfig/network-scripts/ifcfg-intern.cube.ch-1" fails: Missing certificate for EAP method 'tls'.
May 09 08:18:48 XXXXXXX NetworkManager[1322]: <warn>  [1557382728.3740] ifcfg-rh: loading "/etc/sysconfig/network-scripts/ifcfg-Wired_connection_1-1" fails: Missing certificate for EAP method 'tls'.
May 09 08:18:48 XXXXXXX NetworkManager[1322]: <warn>  [1557382728.3770] ifcfg-rh: loading "/etc/sysconfig/network-scripts/ifcfg-Wired_connection_1" fails: Missing certificate for EAP method 'tls'.
May 09 08:18:48 XXXXXXX NetworkManager[1322]: <warn>  [1557382728.3794] ifcfg-rh: loading "/etc/sysconfig/network-scripts/ifcfg-intern.cube.ch" fails: Missing certificate for EAP method 'tls'.

Comment 6 Beniamino Galvani 2019-05-09 07:31:31 UTC
How are you creating the connection? With nmcli, nm-connection-editor, gnome control-center ?

Comment 7 Matthias Summer 2019-05-09 07:36:55 UTC
We use the gnome-control-center.

Comment 8 Beniamino Galvani 2019-05-09 13:21:05 UTC
Does the following command create a working connection?

nmcli connection add \
        type ethernet \
        ifname '*' \
        ethernet.mac-address xx:xx:xx:xx:xx:xx \
        802-1x.eap tls \
        802-1x.identity HOSTNAME.FQDN \
        802-1x.ca-cert /home/matthias/Documents/network-certs/root_ca2.pem \
        802-1x.client-cert xxxxxxxxx \
        802-1x.private-key /home/matthias/Documents/network-certs/host.p12

I suspect the issue could be in gnome control-center.

Comment 9 Matthias Summer 2019-05-09 14:07:46 UTC
Yes, it creates a working connection. But after a reboot the created connection is not available.

When I executed the command a second time after the reboot a new ifcfg-ethernet-1 file was created under /etc/sysconfig/network-scripts and the connection was working again.

Comment 10 Beniamino Galvani 2019-05-09 14:32:00 UTC
I can't reproduce the problem. Please set level=TRACE in the [logging] section of /etc/NetworkManager/NetworkManager.conf, restart NM, reproduce the problem by adding the connection through nmcli, reboot, and then attach the output of 'journalctl -u NetworkManager -b -1; journalctl -u NetworkManager -b'. Thanks.

Comment 11 Matthias Summer 2019-05-10 08:27:54 UTC
Created attachment 1566596 [details]
Output journalctl

You will find the output in the attached file.

Comment 12 Beniamino Galvani 2019-05-10 09:37:08 UTC
Ok, I can reproduce the problem using control-center. If I add connection with EAP-TLS authentication and specify a p12 private key file, the client certificate is left empty and the connection fails validation. Perhaps NM should reject that connection. nmcli seems to work instead.

Comment 13 dirk 2019-05-10 10:16:03 UTC
@Beniamino Galvani

I have the very same problem. Can add a profile (with certificates) with NetworkManaager (gnome) or nm-connection-editor and connect, but after reboot profile is lost. 

What do you mean that "nmcli seems to work"? And if this would be a work around, HOW does it work?

Comment 14 Beniamino Galvani 2019-05-10 12:32:36 UTC
(In reply to dirk from comment #13)
> @Beniamino Galvani
> 
> I have the very same problem. Can add a profile (with certificates) with
> NetworkManaager (gnome) or nm-connection-editor and connect, but after
> reboot profile is lost. 
> 
> What do you mean that "nmcli seems to work"? And if this would be a work
> around, HOW does it work?

See comment 8. Does the following work for you?

nmcli connection add \
        type ethernet \
        ifname eth0 \
        con-name test+ \
        802-1x.eap tls \
        802-1x.identity id \
        802-1x.ca-cert ca.pem \
        802-1x.private-key client.p12

As a quick test, you can 'systemct restart NetworkManager' and check whether the connection is still there with 'nmcli connection'.

Comment 15 Christoph Sievers 2019-05-12 11:56:50 UTC
a previously working ifcfg- and keys- File stops working after updating from f29 (nm 1.14?) to f30 (nm 1.16?).

using ifcfg-rh (default for fedora) - when NetworkManager tries to read a formerly working configuration from ifcfg-blablanetwork:

Mai 11 19:24:40 lblabla NetworkManager[805]: <warn>  [1557595480.0829] ifcfg-rh: loading "/etc/sysconfig/network-scripts/ifcfg-blablanetwork" fails: Missing certificate for EAP method 'tls'.

investigating:

set NetworkManager.conf plugins= (to prevent ifcfg-rh from loading) and using nm-connection-editor I can create a working EAP-TLS configuration that survices a restart.

IMHO this is a ifcfg-rh problem since writing the ifcfg- file works but reading does not. Could this be something to ask thaller@redhat.com?

Comment 16 James Hewitt 2019-05-13 06:55:36 UTC
I think I identified the cause here:

https://gitlab.freedesktop.org/NetworkManager/NetworkManager/issues/173 (thaller@redhat.com has seen it)

Comment 17 James Hewitt 2019-05-13 07:01:52 UTC
This should only affect people using pkcs12 user certs.

The code detect a p12 file as the private key, and then doesn't save the cert config as the cert is included with the key. As a workaround, we can extract the key and use a separate file, to make it save both settings, then the load works.

openssl pkcs12 -in mywirelesscert.p12 -nocerts -out mywirelesscert-key.pem

Now use the p12 for the user cert in the configuration, and the pem for the private key, and the connection will survive reloads.

Comment 18 James Hewitt 2019-05-13 09:47:14 UTC
An easier workaround than splitting just the key out, which can still leave a problem with loading the p12 file as nm isn't great at saving the password for the p12 cert, is to just convert the p12 to pem entirely:

openssl pkcs12 -in mywirelesscert.p12 -out mywirelesscert.pem

And use that instead. The cert in the pem isn't password protected in the same way as in p12 so it works better with nm.

Comment 19 Christoph Sievers 2019-05-13 10:43:49 UTC
but also leaves the private key unprotected so...

Comment 20 James Hewitt 2019-05-13 13:47:11 UTC
The private key by default will still be protected. For an unprotected version, you would also need -nodes.

Comment 21 fedeb1995 2019-05-13 15:36:22 UTC
I can confirm this bug, I have been having it since when I clean installed fedora 30 (so no possible upgrade-related implications for me), and I have the same log messages about networkmanager in journalctl; I have this bug in both wifi and ethernet networks which use 802.1x authentication with cert file and password-protected private key (university network) both by configuring connections with gnome-control-center and by using eduroam's python configuration script (which basically automatically creates the same configuration in my place).

Comment 22 dirk 2019-05-15 11:43:23 UTC
@Christoph Sievers

For me your described workaround does NOT work.

Comment 23 Christoph Sievers 2019-05-15 11:48:54 UTC
@dirk

does your NetworkManager.conf look like this?

[main]
#plugins=ifcfg-rh,ibft
plugins=

also: Things are not going to reappear that way because that stops it from using ifcfg-rh which reads/writes stuff to sysconfig/network-scripts so you would have to create a new connection afert restarting NetworkManager with above config.

Comment 24 Thomas Haller 2019-05-15 19:32:34 UTC
Beniamino opened a merge-request, which is currently on review: https://gitlab.freedesktop.org/NetworkManager/NetworkManager/merge_requests/148

@Christoph, as the bug is in ifcfg-rh plugin, disabling the ifcfg-rh plugin (and only use "keyfile" setting plugin) does indeed work around the issue. But this work around is a bit drastic, as you don't use ifcfg-rh files anymore... Independent of this issue that may be a good idea.

Comment 25 Fedora Update System 2019-05-29 12:40:08 UTC
FEDORA-2019-b51ba86992 has been submitted as an update to Fedora 30. https://bodhi.fedoraproject.org/updates/FEDORA-2019-b51ba86992

Comment 26 Fedora Update System 2019-05-30 13:58:05 UTC
NetworkManager-1.16.2-1.fc30 has been pushed to the Fedora 30 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-b51ba86992

Comment 27 Fedora Update System 2019-06-14 00:53:25 UTC
NetworkManager-1.16.2-1.fc30 has been pushed to the Fedora 30 stable repository. If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.