Bug 1393997

Summary: nmcli duplicates a connection after a NetworkManager restart if DHCP_HOSTNAME is defined
Product: Red Hat Enterprise Linux 7 Reporter: João Avelino Bellomo Filho <jbellomo>
Component: NetworkManagerAssignee: Francesco Giudici <fgiudici>
Status: CLOSED ERRATA QA Contact: Desktop QE <desktop-qa-list>
Severity: medium Docs Contact: Mirek Jahoda <mjahoda>
Priority: high    
Version: 7.3CC: aloughla, atragler, bgalvani, fgiudici, fkrska, jbainbri, jbrouer, jmaxwell, jreznik, lrintel, mjahoda, pvarma, rkhan, rki, sukulkar, thaller, vbenes
Target Milestone: rcKeywords: ZStream
Target Release: ---   
Hardware: Unspecified   
OS: Linux   
Whiteboard:
Fixed In Version: NetworkManager-1.8.0-0.2.git20170215.1d40c5f4.el7 Doc Type: Bug Fix
Doc Text:
*NetworkManager* no longer duplicates a connection with already-set "DHCP_HOSTNAME" Previously, after a restart of the *NetworkManager* service, a connection with an already-set "DHCP_HOSTNAME" property was duplicated. Consequently, a DHCP lease was not always renewed upon its expiry. With this update, the connection is no longer duplicated, and a DHCP lease is correctly renewed in this scenario. Note that the fix includes ignoring the already-set hostname properties in the matching process. To avoid possible problems, remove all unused connections with an incorrect `ipv4.dhcp-hostname`. For more information, see https://access.redhat.com/articles/2948041.
Story Points: ---
Clone Of:
: 1421080 1421082 (view as bug list) Environment:
Last Closed: 2017-08-01 09:19:37 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1393481, 1421080, 1421082    
Attachments:
Description Flags
remove INFERRABLE flag from dhcp-hostname none

Description João Avelino Bellomo Filho 2016-11-10 19:27:41 UTC
Description of problem:

If the DHCP_HOSTNAME is defined on ifcfg file the restart of NetworkManager service causes the connection duplication on nmcli.

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1. Install a fresh RHEL 7.3.

2. Configure the DHCP_HOSTNAME of one of ifcfg file. For example:

[root@rhel73 ~]# grep DHCP /etc/sysconfig/network-scripts/ifcfg-ens3
DHCP_HOSTNAME=rhel73.redhat

3. Restart de system.

4. Login as 'root' and check the current connection list of NetworkManager service:

[root@rhel73 ~]# nmcli connection 
NAME  UUID                                  TYPE            DEVICE 
ens3  6b7e7365-af67-4e24-a71b-38a7b02b4684  802-3-ethernet  ens3   
ens8  fb832859-f896-4d7b-9b2e-01618084850d  802-3-ethernet  ens8   

5. Restart the NetworkManager service:

[root@rhel73 ~]# systemctl restart NetworkManager

6. Check the connection list again:

[root@rhel73 ~]# nmcli connection 

Actual results:

NAME  UUID                                  TYPE            DEVICE 
ens3  95ecc21a-0935-4891-ad4d-10435fbf7d24  802-3-ethernet  ens3   
ens8  fb832859-f896-4d7b-9b2e-01618084850d  802-3-ethernet  ens8   
ens3  6b7e7365-af67-4e24-a71b-38a7b02b4684  802-3-ethernet  --  


Expected results:

NAME  UUID                                  TYPE            DEVICE 
ens3  6b7e7365-af67-4e24-a71b-38a7b02b4684  802-3-ethernet  ens3   
ens8  fb832859-f896-4d7b-9b2e-01618084850d  802-3-ethernet  ens8 

Additional info:

There is two conenctions with the same name (ens3) but with different UUID. The original UUID seems to be not connected to a device.

The connections properties shows that the 'dhcp-hostname' is setted correctly on original connection. The UUID confirms that:

[root@rhel73 ~]# nmcli connection show ens3 | grep "\.id\|\.interface-name\|\.uuid\|dhcp-hostname"
connection.id:                          ens3
connection.uuid:                        95ecc21a-0935-4891-ad4d-10435fbf7d24
connection.interface-name:              ens3
ipv4.dhcp-hostname:                     --
ipv6.dhcp-hostname:                     --
connection.id:                          ens3
connection.uuid:                        6b7e7365-af67-4e24-a71b-38a7b02b4684
connection.interface-name:              ens3
ipv4.dhcp-hostname:                     rhel73.redhat
ipv6.dhcp-hostname:                     rhel73.redhat

If the system is rebooted the connection list returns to the correct entries.

Comment 2 Francesco Giudici 2016-11-18 15:55:49 UTC
Created attachment 1221903 [details]
remove INFERRABLE flag from dhcp-hostname

When NM is shut off, it will leave the connections up.
When it starts it will check if network devices are already configured: if so, it will assume the connection, mapping it to a new, on-the-fly generated connection retrieving the configuration data from the system.
Once this has been done, it will look among the connections saved on disk if there is one matching the properties of the assumed one.
Properties that should be considered in the match are flagged as INFERRABLE.

DHCP_HOSTNAME is mapped to the ip "dhcp-hostname" property, which is marked as "INFERRABLE". Anyway, the dhcp-hostname is not "inferred" from the system when NM starts.
So, the issue here is that when NM is restarted, the connection it "infers back" has empty dhcp-hostname property, triggering the failure with the on-disk one.

There are two options here:
1) try to retrieve the sent hostname from the dhcp client configuration file
2) remove the INFERRABLE flag from that property, so it will not be included in the checks between generated vs on-disk connection

I cannot envision a realistic scenario that could benefit from option 1), moreover option 2) is trivial: attaching a patch for option 2) here.

Comment 3 Beniamino Galvani 2016-11-18 16:02:35 UTC
(In reply to Francesco Giudici from comment #2)
> Created attachment 1221903 [details]
> remove INFERRABLE flag from dhcp-hostname

LGTM

Comment 6 Praveen Varma 2017-02-01 05:49:23 UTC
Case - 1733309 - On this case, after the Network Manager is restarted NetwrokManager creates new in-memory connection which have wrong parameters, which don't have  ipv4.dhcp-hostname parameter set, so after DHCP lease expire DNS server lost DNS client record.

Associated BZ - 1393997

Fransesco is assigned to this BZ and this is merged to upstream master at the moment. The status  - Modified

Customer mentioned that they does not want a hotfix and would like to see the fix on RHEL 7.2 EUS channel

@Chris & @Fransesco - Can we get an update on this and more importantly whether we would have a fix on the RHEL 7.2 EUS channel at the earliest?

Thanks,
Praveen
Escalation Manager

Comment 7 Francesco Giudici 2017-02-02 13:56:54 UTC
(In reply to Praveen Varma from comment #6)
> Case - 1733309 - On this case, after the Network Manager is restarted

May I ask why NetworkManager is restarted?
If it is just to update the connection list for newly added config files the proper way is:
"nmcli connection reload"
and the new connection can be applied with
'nmcli connection up $conn_name'

> NetwrokManager creates new in-memory connection which have wrong parameters,
> which don't have  ipv4.dhcp-hostname parameter set, so after DHCP lease
> expire DNS server lost DNS client record.

Appending '&& nmcli connection up $conn_name' to the NetworkManager restart command will allow NetworkManager to apply the stored connection overriding the one which has been found.
Anyway, my first advice would be to stick with the "reload" command above.

Please can you share why NetworkManager should be restart?
This should be avoided whenever possible to avoid side effects.

> 
> Associated BZ - 1393997
> 
> Fransesco is assigned to this BZ and this is merged to upstream master at
> the moment. The status  - Modified
> 
> Customer mentioned that they does not want a hotfix and would like to see
> the fix on RHEL 7.2 EUS channel
> 
> @Chris & @Fransesco - Can we get an update on this and more importantly
> whether we would have a fix on the RHEL 7.2 EUS channel at the earliest?
> 
> Thanks,
> Praveen
> Escalation Manager

Comment 9 Praveen Varma 2017-02-03 03:13:56 UTC
Thank you Fransesco for your comments.

1) I think we should be first talking to the customer about this and then decide if they are ready look at a backport option for their needs (after they are let known that the restart of NM is not advisable) - We would come back to let you know the decision

2) If the decision is to backport, can we make sure that this is taken care in 7.2 EUS? If so, by when can it be done?

Thanks again for your support Fransesco !!

Regards,
Praveen

Comment 10 Francesco Giudici 2017-02-03 08:46:04 UTC
(In reply to Praveen Varma from comment #9)
> Thank you Fransesco for your comments.
> 
> 1) I think we should be first talking to the customer about this and then
> decide if they are ready look at a backport option for their needs (after
> they are let known that the restart of NM is not advisable) - We would come
> back to let you know the decision

Great, thanks!

> 
> 2) If the decision is to backport, can we make sure that this is taken care
> in 7.2 EUS? If so, by when can it be done?

I guess to have this in 7.2 EUS the bug should be cloned by a PM for 7.3 and 7.2 EUS and should be accepted. Please, share the user scenario that requires NetworkManager to be restarted: this would help.
I'm not able to estimate the time this could take as it is not under my control.
Once the bugs are created and accepted then I can update the code for the RHEL packages quickly.

Thanks!

Francesco

> 
> Thanks again for your support Fransesco !!
> 
> Regards,
> Praveen

Comment 12 Praveen Varma 2017-02-06 03:48:47 UTC
Francesco - hope you have seen the comments from Robin which he received from the customer. Do you have some info to share on this?

Thanks in advance !!

Regards,
Praveen

Comment 26 errata-xmlrpc 2017-08-01 09:19:37 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2299