Bug 1963834

Summary: Bonding on VLAN with kickstart makes traceback: 'NoneType' object has no attribute 'get_mac_address'
Product: Red Hat Enterprise Linux 8 Reporter: Masahiro Matsuya <mmatsuya>
Component: anacondaAssignee: Anaconda Maintenance Team <anaconda-maint-list>
Status: CLOSED ERRATA QA Contact: Release Test Team <release-test-team-automation>
Severity: high Docs Contact:
Priority: high    
Version: 8.4CC: anaconda-maint-list, bilhar, jstodola, pamadio, peter.vreman, rvykydal, sbarcomb, zveleba
Target Milestone: betaKeywords: Regression, Reopened, Triaged
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: anaconda-33.16.5.3-1.el8 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-11-09 18:42:11 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Masahiro Matsuya 2021-05-24 07:14:18 UTC
Description of problem:

10:39:36,585 WARNING org.fedoraproject.Anaconda.Modules.Network:ERROR:anaconda.modules.common.task.task:Thread AnaTaskThread-ApplyKickstartTask-1 has failed: Traceback (most recent call last):
10:39:36,585 WARNING org.fedoraproject.Anaconda.Modules.Network:  File "/usr/lib64/python3.6/site-packages/pyanaconda/threading.py", line 280, in run
10:39:36,585 WARNING org.fedoraproject.Anaconda.Modules.Network:    threading.Thread.run(self)
10:39:36,585 WARNING org.fedoraproject.Anaconda.Modules.Network:  File "/usr/lib64/python3.6/threading.py", line 864, in run
10:39:36,585 WARNING org.fedoraproject.Anaconda.Modules.Network:    self._target(*self._args, **self._kwargs)
10:39:36,585 WARNING org.fedoraproject.Anaconda.Modules.Network:  File "/usr/lib64/python3.6/site-packages/pyanaconda/modules/common/task/task.py", line 97, in _task_run_callback
10:39:36,585 WARNING org.fedoraproject.Anaconda.Modules.Network:    self._set_result(self.run())
10:39:36,585 WARNING org.fedoraproject.Anaconda.Modules.Network:  File "/usr/lib64/python3.6/site-packages/pyanaconda/modules/network/utils.py", line 130, in wrapped
10:39:36,585 WARNING org.fedoraproject.Anaconda.Modules.Network:    return function(*args, **kwargs)
10:39:36,585 WARNING org.fedoraproject.Anaconda.Modules.Network:  File "/usr/lib64/python3.6/site-packages/pyanaconda/modules/network/initialization.py", line 129, in run
10:39:36,585 WARNING org.fedoraproject.Anaconda.Modules.Network:    device_name=device_name)
10:39:36,585 WARNING org.fedoraproject.Anaconda.Modules.Network:  File "/usr/lib64/python3.6/site-packages/pyanaconda/modules/network/nm_client.py", line 583, in update_connection_from_ksdata
10:39:36,585 WARNING org.fedoraproject.Anaconda.Modules.Network:    bind_connection(nm_client, connection, network_data.bindto, device_name)
10:39:36,585 WARNING org.fedoraproject.Anaconda.Modules.Network:  File "/usr/lib64/python3.6/site-packages/pyanaconda/modules/network/nm_client.py", line 765, in bind_connection
10:39:36,585 WARNING org.fedoraproject.Anaconda.Modules.Network:    modified = bind_settings_to_device(nm_client, s_con, s_wired, device_name, bind_exclusively)
10:39:36,585 WARNING org.fedoraproject.Anaconda.Modules.Network:  File "/usr/lib64/python3.6/site-packages/pyanaconda/modules/network/nm_client.py", line 712, in bind_settings_to_device
10:39:36,585 WARNING org.fedoraproject.Anaconda.Modules.Network:    mac_address = s_wired.get_mac_address()
10:39:36,585 WARNING org.fedoraproject.Anaconda.Modules.Network:AttributeError: 'NoneType' object has no attribute 'get_mac_address'
10:39:36,585 WARNING org.fedoraproject.Anaconda.Modules.Network:INFO:anaconda.threading:Thread Done: AnaTaskThread-ApplyKickstartTask-1 (140661604337408)
10:39:37,586 WARNING org.fedoraproject.Anaconda.Modules.Network:WARNING:dasbus.server.handler:The call org.fedoraproject.Anaconda.Task.Finish has failed with an exception:
10:39:37,586 WARNING org.fedoraproject.Anaconda.Modules.Network:Traceback (most recent call last):
10:39:37,586 WARNING org.fedoraproject.Anaconda.Modules.Network:  File "/usr/lib/python3.6/site-packages/dasbus/server/handler.py", line 421, in _method_callback
10:39:37,586 WARNING org.fedoraproject.Anaconda.Modules.Network:    *unwrap_variant(parameters)
10:39:37,586 WARNING org.fedoraproject.Anaconda.Modules.Network:  File "/usr/lib/python3.6/site-packages/dasbus/server/handler.py", line 234, in _handle_call
10:39:37,586 WARNING org.fedoraproject.Anaconda.Modules.Network:    return handler(*parameters)
10:39:37,586 WARNING org.fedoraproject.Anaconda.Modules.Network:  File "/usr/lib64/python3.6/site-packages/pyanaconda/modules/common/task/task_interface.py", line 114, in Finish
10:39:37,586 WARNING org.fedoraproject.Anaconda.Modules.Network:    self.implementation.finish()
10:39:37,586 WARNING org.fedoraproject.Anaconda.Modules.Network:  File "/usr/lib64/python3.6/site-packages/pyanaconda/modules/common/task/task.py", line 161, in finish
10:39:37,586 WARNING org.fedoraproject.Anaconda.Modules.Network:    threadMgr.raise_if_error(self._thread_name)
10:39:37,586 WARNING org.fedoraproject.Anaconda.Modules.Network:  File "/usr/lib64/python3.6/site-packages/pyanaconda/threading.py", line 171, in raise_if_error
10:39:37,586 WARNING org.fedoraproject.Anaconda.Modules.Network:    raise exc_info[0](exc_info[1]).with_traceback(exc_info[2])
10:39:37,586 WARNING org.fedoraproject.Anaconda.Modules.Network:  File "/usr/lib64/python3.6/site-packages/pyanaconda/threading.py", line 280, in run
10:39:37,586 WARNING org.fedoraproject.Anaconda.Modules.Network:    threading.Thread.run(self)
10:39:37,586 WARNING org.fedoraproject.Anaconda.Modules.Network:  File "/usr/lib64/python3.6/threading.py", line 864, in run
10:39:37,586 WARNING org.fedoraproject.Anaconda.Modules.Network:    self._target(*self._args, **self._kwargs)
10:39:37,586 WARNING org.fedoraproject.Anaconda.Modules.Network:  File "/usr/lib64/python3.6/site-packages/pyanaconda/modules/common/task/task.py", line 97, in _task_run_callback
10:39:37,586 WARNING org.fedoraproject.Anaconda.Modules.Network:    self._set_result(self.run())
10:39:37,586 WARNING org.fedoraproject.Anaconda.Modules.Network:  File "/usr/lib64/python3.6/site-packages/pyanaconda/modules/network/utils.py", line 130, in wrapped
10:39:37,586 WARNING org.fedoraproject.Anaconda.Modules.Network:    return function(*args, **kwargs)
10:39:37,586 WARNING org.fedoraproject.Anaconda.Modules.Network:  File "/usr/lib64/python3.6/site-packages/pyanaconda/modules/network/initialization.py", line 129, in run
10:39:37,586 WARNING org.fedoraproject.Anaconda.Modules.Network:    device_name=device_name)
10:39:37,586 WARNING org.fedoraproject.Anaconda.Modules.Network:  File "/usr/lib64/python3.6/site-packages/pyanaconda/modules/network/nm_client.py", line 583, in update_connection_from_ksdata
10:39:37,586 WARNING org.fedoraproject.Anaconda.Modules.Network:    bind_connection(nm_client, connection, network_data.bindto, device_name)
10:39:37,586 WARNING org.fedoraproject.Anaconda.Modules.Network:  File "/usr/lib64/python3.6/site-packages/pyanaconda/modules/network/nm_client.py", line 765, in bind_connection
10:39:37,586 WARNING org.fedoraproject.Anaconda.Modules.Network:    modified = bind_settings_to_device(nm_client, s_con, s_wired, device_name, bind_exclusively)
10:39:37,586 WARNING org.fedoraproject.Anaconda.Modules.Network:  File "/usr/lib64/python3.6/site-packages/pyanaconda/modules/network/nm_client.py", line 712, in bind_settings_to_device
10:39:37,587 WARNING org.fedoraproject.Anaconda.Modules.Network:    mac_address = s_wired.get_mac_address()
10:39:37,587 WARNING org.fedoraproject.Anaconda.Modules.Network:AttributeError: 'NoneType' object has no attribute 'get_mac_address'

The following upstream commit fixes this problem:
https://github.com/rhinstaller/anaconda/commit/3f17df57addf947b3e35fd3fa9298325172fc3c1

The following is optional, because NM_CONNECTION_TYPE_BOND definition is required. 
https://github.com/rhinstaller/anaconda/commit/2945b935c761a182114f8da758088fcc202cd0e0


This is not reproduced without the kickstart network command since the traceback includes "update_connection_from_ksdata" method call.

Version-Release number of selected component (if applicable):
Red Hat Enterprise Linux 8.4

How reproducible:
Always

Steps to Reproduce:
1. configure a HTTP server on VLAN (vlan10)

  http://192.168.120.100/rhel84      : RHEL8.4 install tree
  http://192.168.120.100/hostonly.ks : kickstart file contains the following line only.

      network --noipv6 --hostname=xyz.vlan10.example.com


2. configure dnsmasq for VLAN on the same host.

In /etc/dnsmasq.conf
  interface=vlan10
  domain=vlan10.example.com
  dhcp-range=192.168.120.101,192.168.120.102,12h


3. start installation with the following parameter.

inst.repo=http://192.168.120.100/rhel84 ipv6.disable=1 bond=bond0:enp1s0,enp2s0:mode=active-backup,primary=enp1s0,miimon=100 vlan=vlan10:bond0 ip=vlan10:dhcp inst.ks=http://192.168.120.100/hostonly.ks


Actual results:
The installation stopped with above python trackback.

Expected results:
The installation can proceed without any problem.

Additional info:

The customer said that this problem didn't happen in RHEL7.2, at least.
So, I set Regression keyword.

Comment 3 Peter Vreman 2021-06-09 07:34:33 UTC
In the associated case we found that the reproducer kickstart file the '--noipv6' shall be removed, because with that bond0 would be taken as active-link device and anaconda will force DHCP on it although that will not work because there is no untagged VLAN to the device. The result is that the bond0 dhcp times out after 45 seconds and networkmanager takes it down bond0 including the working vlan209. Honestly i doubt that behaviour is even correct. If the untagged bond0 is not working then the child VLANs can still be working valid.

The additonal info regression was that it worked with RHEL8.2 and RHEL7.x. Checking the log files i noticed that there is a massive rewrite between RHEL8.2 and RHEL8.3/8.4 in creating the ifcfg files, in RHEL8.2 it was created by dracut scripts directly, in RHEL8.4 it is all written by Networkmanager

Comment 4 Radek Vykydal 2021-06-10 14:02:17 UTC
If the point of the network command in kickstart file is just to configure hostname, there should contain just the --hostname option and --noipv6 should be removed.

If there are also other options then --hostname in the network command, Anaconda considers the command as a request to configure the device with implicit "--device link" specification, which means the first device with link (can be resolved to bond0 as in the case of this BZ). The option was targeted to environments with just single device being connected (having carrier) so that the device does not have to be known / specified for configuration via kickstart.

Comment 5 Peter Vreman 2021-06-11 06:19:51 UTC
Radek,

i already got the feedback that the --noipv6 had to be removed. But where is this feature documented that 'network --hostname <hostname>' is handled differently?

Second i opened yesterday another case where the 'network --hostname <hostname>' is breaking another use case i have where the network interface is now doing a DHCP/DNS lookup to get its hostname from DNS for that interface although i explcitly defined the hostname to be used.


Peter

Comment 6 Radek Vykydal 2021-06-11 07:05:03 UTC
(In reply to Peter Vreman from comment #5)
> Radek,
> 
> i already got the feedback that the --noipv6 had to be removed. But where is
> this feature documented that 'network --hostname <hostname>' is handled
> differently?

Good point, I think it is not documented and should be added to installation guide / kickstart reference (https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html-single/performing_an_advanced_rhel_installation/index/#network_kickstart-commands-for-network-configuration). What do you think Jan?

Comment 7 Jan Stodola 2021-06-11 17:53:01 UTC
Yes, I agree with documenting the network --hostname behavior.

Bilhar, could you please have a look at comment 4 and add a note to the installation guide?

Comment 13 Radek Vykydal 2021-06-14 12:27:33 UTC
PR with port of the patch from the description: https://github.com/rhinstaller/anaconda/pull/3441

Comment 17 Radek Vykydal 2021-06-29 07:48:49 UTC
We still want to apply https://github.com/rhinstaller/anaconda/pull/3441 as a resolution of this BZ.

Comment 20 Peter Vreman 2021-06-29 09:52:10 UTC
Feedback on the improvements:

I looked at the new text and i miss in the second paragraph 'with additonal options' what is going to be done with the value provided to 'hostname', how/where/when is it going to be used when 'network command configures a device using the options specified' is being applied?

~~~
If you only want to configure the target system’s host name, use the --hostname option in the network command and do not include any other option.

If you provide additional options when configuring the host name, the network command configures a device using the options specified. If you do not specify which device to configure using the --device option, the default --device link value is used. Additionally, if you do not specify the protocol using the --bootproto option, the device is configured to use DHCP by default.
~~~



The statement a bit lower about FQDN is not 100% correct and conflicts with SAP requirements of requiring a short hostname:
~~~
If your network does not provide a DHCP service, always use the FQDN as the system’s host name.
~~~

Comment 21 Radek Vykydal 2021-06-29 13:06:09 UTC
(In reply to Peter Vreman from comment #20)
> Feedback on the improvements:
> 
> I looked at the new text and i miss in the second paragraph 'with additonal
> options' what is going to be done with the value provided to 'hostname',
> how/where/when is it going to be used when 'network command configures a
> device using the options specified' is being applied?

It should work exactly the same as if only hostname is configured (the first paragraph of the --hostname option section) regarding the hostname configuration of the target system - which is the purpose of network --hostname command.

It seems that regarding the installer environment hostname the situation is different when a network device is re-activated (can happen for the second paragraph below) which might cause for example update of the current hostname by NM from dns (perhaps something you are commenting / hitting in bug 1975349). Still the value of such updated hostname is not related to the value of network --hostname command.
We may need to update the documentation with regard to hostname of installer environment but I am afraid the behaviour has not been scrictly specified or guaranteed. Maybe we can describe the current behaviour and consider RFE from https://bugzilla.redhat.com/show_bug.cgi?id=1975349#c4

> 
> ~~~
> If you only want to configure the target system’s host name, use the
> --hostname option in the network command and do not include any other option.
> 
> If you provide additional options when configuring the host name, the
> network command configures a device using the options specified. If you do
> not specify which device to configure using the --device option, the default
> --device link value is used. Additionally, if you do not specify the
> protocol using the --bootproto option, the device is configured to use DHCP
> by default.
> ~~~

Comment 22 Peter Vreman 2021-06-29 14:58:26 UTC
Radek,

My feedback was on the Documentation only. I split on purpose requests to different BZs
Honestly the documentation of 'network --hostname' also does not belong in this BZ about VLAN and Bonding

The fix for the BZ shall be https://github.com/rhinstaller/anaconda/pull/3441
I can confirm that the PR3441 fixes it, i already implemented it as a temporary workaround in the %pre using a sed-command.




Back to the documentation as we were on that topic also. Just adding a small remark that also the hostname will be set would be enough for me
~~~
If you provide additional options when configuring the host name, the network command configures a device using the options specified.
~~~

to 

~~~
If you provide additional options when configuring the host name, the network command configures ___the hostname and___ device using the options specified.
~~~


I also personally like to look at reference examples what is working and what not:

Maybe also an example might work, e.g. can you do split on several lines or combine it.
~~~
network --hostname blah
network --device ens192 --bootproto=x ...
~~~
is the same as
~~~
network --hostname blah --device ens192 --bootproto=x ...
~~~

And how does --hostname work (or fail) if multiple devices with multiple hostnames are set:
~~~
network --hostname blah.example.com --device ens192 --bootproto=x ...
network --hostname blah-management.example.com --device ens256 --bootproto=x ...
~~~

I am not asking for any code change. This feedback is only provided to improve the documentation on the behaviour

Comment 23 Radek Vykydal 2021-07-02 08:30:50 UTC
(In reply to Peter Vreman from comment #22)
> Radek,
> 
> My feedback was on the Documentation only. I split on purpose requests to
> different BZs
> Honestly the documentation of 'network --hostname' also does not belong in
> this BZ about VLAN and Bonding
> 
> The fix for the BZ shall be https://github.com/rhinstaller/anaconda/pull/3441
> I can confirm that the PR3441 fixes it, i already implemented it as a
> temporary workaround in the %pre using a sed-command.
> 
> 
> 
> 
> Back to the documentation as we were on that topic also. Just adding a small
> remark that also the hostname will be set would be enough for me
> ~~~
> If you provide additional options when configuring the host name, the
> network command configures a device using the options specified.
> ~~~
> 
> to 
> 
> ~~~
> If you provide additional options when configuring the host name, the
> network command configures ___the hostname and___ device using the options
> specified.
> ~~~
> 

Looks good to me.

> 
> I also personally like to look at reference examples what is working and
> what not:
> 
> Maybe also an example might work, e.g. can you do split on several lines or
> combine it.
> ~~~
> network --hostname blah
> network --device ens192 --bootproto=x ...
> ~~~
> is the same as
> ~~~
> network --hostname blah --device ens192 --bootproto=x ...
> ~~~
> 

I agree, good idea.

> And how does --hostname work (or fail) if multiple devices with multiple
> hostnames are set:
> ~~~
> network --hostname blah.example.com --device ens192 --bootproto=x ...
> network --hostname blah-management.example.com --device ens256 --bootproto=x
> ...
> ~~~

This is a bit of a grey zone but we should try to document it.

> 
> I am not asking for any code change. This feedback is only provided to
> improve the documentation on the behaviour

Thank you, these are all great suggestions. I think we should open a documentation ticket for them. Bilhar, could you giude us to the right place?

Comment 24 bilhar 2021-07-02 14:16:31 UTC
(In reply to Radek Vykydal from comment #23)
> (In reply to Peter Vreman from comment #22)
> > Radek,
> > 
> > My feedback was on the Documentation only. I split on purpose requests to
> > different BZs
> > Honestly the documentation of 'network --hostname' also does not belong in
> > this BZ about VLAN and Bonding
> > 
> > The fix for the BZ shall be https://github.com/rhinstaller/anaconda/pull/3441
> > I can confirm that the PR3441 fixes it, i already implemented it as a
> > temporary workaround in the %pre using a sed-command.
> > 
> > 
> > 
> > 
> > Back to the documentation as we were on that topic also. Just adding a small
> > remark that also the hostname will be set would be enough for me
> > ~~~
> > If you provide additional options when configuring the host name, the
> > network command configures a device using the options specified.
> > ~~~
> > 
> > to 
> > 
> > ~~~
> > If you provide additional options when configuring the host name, the
> > network command configures ___the hostname and___ device using the options
> > specified.
> > ~~~
> > 
> 
> Looks good to me.
> 
> > 
> > I also personally like to look at reference examples what is working and
> > what not:
> > 
> > Maybe also an example might work, e.g. can you do split on several lines or
> > combine it.
> > ~~~
> > network --hostname blah
> > network --device ens192 --bootproto=x ...
> > ~~~
> > is the same as
> > ~~~
> > network --hostname blah --device ens192 --bootproto=x ...
> > ~~~
> > 
> 
> I agree, good idea.
> 
> > And how does --hostname work (or fail) if multiple devices with multiple
> > hostnames are set:
> > ~~~
> > network --hostname blah.example.com --device ens192 --bootproto=x ...
> > network --hostname blah-management.example.com --device ens256 --bootproto=x
> > ...
> > ~~~
> 
> This is a bit of a grey zone but we should try to document it.
> 
> > 
> > I am not asking for any code change. This feedback is only provided to
> > improve the documentation on the behaviour
> 
> Thank you, these are all great suggestions. I think we should open a
> documentation ticket for them. Bilhar, could you giude us to the right place?

Hi Radek,

I think the best thing at this point would be to lodge a documentation BZ, which I think will automatically create an internal JIRA ticket. Otherwise, we can just track this work in that BZ.

Thanks,

Bilhar

Comment 25 bilhar 2021-07-02 14:16:53 UTC
(In reply to Radek Vykydal from comment #23)
> (In reply to Peter Vreman from comment #22)
> > Radek,
> > 
> > My feedback was on the Documentation only. I split on purpose requests to
> > different BZs
> > Honestly the documentation of 'network --hostname' also does not belong in
> > this BZ about VLAN and Bonding
> > 
> > The fix for the BZ shall be https://github.com/rhinstaller/anaconda/pull/3441
> > I can confirm that the PR3441 fixes it, i already implemented it as a
> > temporary workaround in the %pre using a sed-command.
> > 
> > 
> > 
> > 
> > Back to the documentation as we were on that topic also. Just adding a small
> > remark that also the hostname will be set would be enough for me
> > ~~~
> > If you provide additional options when configuring the host name, the
> > network command configures a device using the options specified.
> > ~~~
> > 
> > to 
> > 
> > ~~~
> > If you provide additional options when configuring the host name, the
> > network command configures ___the hostname and___ device using the options
> > specified.
> > ~~~
> > 
> 
> Looks good to me.
> 
> > 
> > I also personally like to look at reference examples what is working and
> > what not:
> > 
> > Maybe also an example might work, e.g. can you do split on several lines or
> > combine it.
> > ~~~
> > network --hostname blah
> > network --device ens192 --bootproto=x ...
> > ~~~
> > is the same as
> > ~~~
> > network --hostname blah --device ens192 --bootproto=x ...
> > ~~~
> > 
> 
> I agree, good idea.
> 
> > And how does --hostname work (or fail) if multiple devices with multiple
> > hostnames are set:
> > ~~~
> > network --hostname blah.example.com --device ens192 --bootproto=x ...
> > network --hostname blah-management.example.com --device ens256 --bootproto=x
> > ...
> > ~~~
> 
> This is a bit of a grey zone but we should try to document it.
> 
> > 
> > I am not asking for any code change. This feedback is only provided to
> > improve the documentation on the behaviour
> 
> Thank you, these are all great suggestions. I think we should open a
> documentation ticket for them. Bilhar, could you giude us to the right place?

Hi Radek,

I think the best thing at this point would be to lodge a documentation BZ, which I think will automatically create an internal JIRA ticket. Otherwise, we can just track this work in that BZ.

Thanks,

Bilhar

Comment 26 Zdenek Veleba 2021-07-27 08:29:29 UTC
Pre-verified with anaconda-33.16.5.3-1 in RHEL-8.5.0-20210718.d.2, manual reproducer and automated kickstart-test passed.

Comment 29 Jan Stodola 2021-08-02 15:34:40 UTC
naconda-33.16.5.3-1.el8 is present in RHEL-8.5.0-20210730.n.0, moving to VERIFIED.

Comment 31 errata-xmlrpc 2021-11-09 18:42:11 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (anaconda bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:4254