Setting severity to High, as inconsistent MAC means inconsistent dhcp-provided address, which may mean lost nodes.

Hello Gris, any chance this would make it to 1.0?

Hi guys,

When you did not define MAC address for a bond, you are expecting nmstate/NetworkManager/udev/kernel to choose whatever mac address fits.


The systemd-udev has its reputation on alter the algorithm which cause the MAC address of virtual device's MAC address altered.
So I would suggest you set the MAC address explicitly.

```yaml
---
interfaces:
- name: bond99
  type: bond
  state: up
  mac-address: ee:24:e8:8b:1b:ff
  link-aggregation:
    mode: balance-rr
    port:
    - eth1
    - eth2
```

Hello Gris,

specifying the MAC address explicitly indeed makes sure that the specified address is used, but this is not a solution for OCP Clusters.
We can't specify the same MAC for all nodes in an OCP cluster.
We cannot make users create a separate policy for each node in the cluster either. 

I believe we need a way to indicate, from which interface the MAC address should be used.
It makes the most sense to me to expose this feature/option at nmstate.

(In reply to Radim Hrazdil from comment #4)
> Hello Gris,
> 
> specifying the MAC address explicitly indeed makes sure that the specified
> address is used, but this is not a solution for OCP Clusters.
> We can't specify the same MAC for all nodes in an OCP cluster.
> We cannot make users create a separate policy for each node in the cluster
> either. 
> 
> I believe we need a way to indicate, from which interface the MAC address
> should be used.
> It makes the most sense to me to expose this feature/option at nmstate.


Is it make sense that nmstate will use the __first__ slave/port listed in
desire state for bond/bridge/etc interface?

> Is it make sense that nmstate will use the __first__ slave/port listed in
> desire state for bond/bridge/etc interface?


So if I take the following desiredState for example:

```yaml
---
interfaces:
- name: bond99
  type: bond
  state: up
  link-aggregation:
    mode: balance-rr
    port:
    - eth1
    - eth2
```

The mac address used for the bond interface would be taken from eth1 because it's at the top of the list?
If so, than I think this is a viable option.

The downside is that it's not self-explanatory, meaning that users need to consult documentation to learn about this behaviour.
On the other hand I understand that you probably don't want to introduce a new field for this, because that would deviate from the kernel driver API, right?

I prefer the idea of explicit, so let's go with `copy-mac-from: eth1`.

Does it work for you also, Radim?

Sounds great to me, thank you.

Patch posted to upstream. https://github.com/nmstate/nmstate/pull/1512

Please try `dnf copr enable packit/nmstate-nmstate-1512` to see whether it meet your needs.


Example yaml:

```yaml
---
interfaces:
- name: bond99
  type: bond
  state: up
  copy-mac-from: eth1
  link-aggregation:
    mode: balance-rr
    port:
    - eth2
    - eth1
```


Both bond and bridge is supporting this.

@ferferna I have tried to verify this, but the packit RPM you shared introduces a regression.

I used following config:

cat <<EOF | ./cluster/kubectl.sh apply -f -
apiVersion: nmstate.io/v1alpha1
kind: NodeNetworkConfigurationPolicy
metadata:
  name: test
spec:
  nodeSelector:
    kubernetes.io/hostname: node02
  desiredState:
    interfaces:
    - name: bond0
      type: bond
      state: up
      link-aggregation:
        mode: balance-rr
        options:
          miimon: '140'
        slaves:
        - eth1
        - eth2
EOF

It worked fine with 1.0.1. With the new build it failed with:

        libnmstate.error.NmstateLibnmError: Activate profile uuid:0f17245f-1bc1-4ab9-831f-9daa612e1be4 iface:calibe48e194813 type: ethernet failed: error=nm-manager-error-quark: Connection 'calibe48e194813' is not available on device calibe48e194813 because device is strictly unmanaged (2)

Full log is in the attachment.

Created attachment 1756642 [details]
latest nmstate failing on unmanaged iface

Verified with versions:
nmstate-1.0.2-0.3.el8.noarch
nispor-1.0.1-3.el8.x86_64
NetworkManager-1.30.0-0.10.el8.x86_64

Basically the feature works well, notice there are some expanded test issues:
#bz1928745
#bz1929132
#bz1929946

Hi Mingyu,

Thanks for the test feedback. I have made proper action plans for these bugs.

Hi Petr,

This sounds like a know issue of NetworkManager. Could you try it again with NetworkManager-1.30.0-0.10.el8.x86_64?

If still fails, please kindly provides:
 1. The network state before nmstate applies. The output of `npc` could be really helpfull.
 2. How to configure this unmanaged network?
 3. The type of `calibe48e194813`, veth?
 4. The desired state been passed to nmstate.


Thank you very much on performing regression test!

(In reply to Gris Ge from comment #23)
> Hi Petr,
> 
> This sounds like a know issue of NetworkManager. Could you try it again with
> NetworkManager-1.30.0-0.10.el8.x86_64?

I have tried it now with 1.30.0-0.7.el8, is it good enough?

> 
> If still fails, please kindly provides:
>  1. The network state before nmstate applies. The output of `npc` could be
> really helpfull.

I attached a tarball with this and other data obtained before the config.

>  2. How to configure this unmanaged network?

This is part of the Calico CNI that handles networking within my Kubernetes cluster. I don't know how they setup the network.

>  3. The type of `calibe48e194813`, veth?

I believe so.

>  4. The desired state been passed to nmstate.


This is my reproducer:

echo 'interfaces:
- name: bond0
  type: bond
  state: up
  link-aggregation:
    mode: balance-rr
    options:
      miimon: '140'
    slaves:
    - eth1
    - eth2' > config.yaml

nmstatectl show > nmstatectl-show.yaml
npc > npc.yaml
ip l > ip-l.txt
nmcli c > nmcli-c.txt
nmcli d > nmcli-c.txt

nmstatectl apply config.yaml > nmstatectl-apply.log


> 
> 
> Thank you very much on performing regression test!

Created attachment 1759905 [details]
Data collected before applying the config

Hi Petr,

I failed to find the root cause from the tarball.
Could you try NetworkManager-1.30.0-2.el8.x86_64.rpm and nmstate-1.0.2-2.el8?
If still fails, please open a new bug as current one is for `copy_mac_from` property.

Thank you very much!

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (nmstate bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:1748