Created attachment 1690640 [details]
kubemacpool-config

Description of problem:
When assigning MAC address to a secondary interface on a VM using KubeMacPool, only the secondary interface(s) should be assigned a MAC address from the MAC-pool range, while the primery interface should be assigned an arbitrary MAC address from Kubernetes. In reality, the primary interface MAC is also assigned form the MAC-pool range.


Version-Release number of selected component (if applicable):
KMP: registry.stage.redhat.io/container-native-virtualization/kubemacpool@sha256:7c27788e92ba6a17e7b3fd29e39ea2ec83e51051d6b8670b146b1305c2dce2de
CNAO: sha256_2ec373edc6a1307248e9d4e4276f26052aafcd6058e5ddc182de847f


How reproducible:
Always


Steps to Reproduce:
Run on a cluster with worker nodes with multi NICs:
1. Create a new namespace:
$ oc create ns kmp-opt-in-ns

2. Create a linux-bridge on one of the secondary NICs (on all worker nodes):
$ cat << EOF | oc create -f -
apiVersion: nmstate.io/v1alpha1
kind: NodeNetworkConfigurationPolicy
metadata:
  name: kmp-opt-br-nncp
spec:
  desiredState:
    interfaces:
    - name: kmp-opt-br
      description: Linux bridge with ens6 as a port
      type: linux-bridge
      state: up
      bridge:
        options:
          stp:
            enabled: false
        port:
        - name: ens6
EOF

If there's no interface on the workers which is named "ens6" - replace port.name value with the name of an actual secondary interface on the node.

3. Create a NetworkAttachmentDefinition which uses the bridge:
$ cat << EOF | oc create -f -
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: "kmp-opt-br"
  namespace: "kmp-opt-in-ns"
  annotations:
    k8s.v1.cni.cncf.io/resourceName: "bridge.network.kubevirt.io/kmp-opt-br"
spec:
  config: '{
    "cniVersion": "0.3.1",
    "name": "kmp-opt-br",
    "plugins": [{
      "type": "cnv-bridge",
      "bridge": "kmp-opt-br"
    }]
}'
EOF

4. Apply the KubeMacPool configuration:
$ oc apply -f kubemacpool-config.yaml

(kubemacpool-config.yaml is attached).
* NOTE: This configuration contains this MAC pool range:
  RANGE_END: 02:AA:AA:AA:AA:FF
  RANGE_START: "02:AA:AA:AA:AA:00"

5. Label the kmp-opt-in-ns namespace for KubeMacPool opt-in:
$ oc label namespace kmp-opt-in-ns mutatevirtualmachines.kubemacpool.io=allocateForAll
namespace/kmp-opt-in-ns labeled

6. Create a VM in kmp-opt-in-ns namespace, with a secondary interface using the NetworkAttachmentDefinition you created:
$ oc apply -f kmp-opt-vm.yaml
(kmp-opt-vm.yaml is attached)

7. Move to kmp-opt-in-ns:
$ oc project kmp-opt-in-ns

8. Start the VM:
$ virtctl start kmp-opt-vm

9. When VM is up and running - open its console:
$ virtctl console kmp-opt-vm

10. check the MAC addresses of the VM's interfaces:
$ ip link

Actual results:
$ ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
    link/ether 02:aa:aa:aa:aa:00 brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
    link/ether 02:aa:aa:aa:aa:01 brd ff:ff:ff:ff:ff:ff

<BUG> Pod interface eth0 MAC address is taken from the KubeMacPool range.

Expected results:
Pod interface eth0 MAC address should be an arbitrary value, assigned by k8s.
Only secondary interfaces (eth1 and on) should get their MAC address assigned from the KMP range.


Additional info: