Bug 2093996

Verified with virt-api:v4.12.0-253

1. Create a namespace and change the context to work in it:
$ oc create ns yoss-ns
namespace/yoss-ns created
$
$ oc project yoss-ns 
Now using project "yoss-ns" on server "https://api.net-ys-412-2.cnv-qe.rhcloud.com:6443".

2. Create a linux-bridge on a worker using this policy:
$ cat << EOF | oc apply -f -
apiVersion: nmstate.io/v1
kind: NodeNetworkConfigurationPolicy
metadata:
  name: linux-bridge
spec:
  desiredState:
    interfaces:
    - name: br1
      description: Linux bridge with eth1 as a port
      type: linux-bridge
      state: up
      ipv4:
        dhcp: true
        enabled: true
      bridge:
        options:
          stp:
            enabled: false
        port:
        - name: ens10
  nodeSelector:
    kubernetes.io/hostname: net-ys-412-2-rszl9-worker-0-hfnc2
EOF

3. Create the following NetworkAttachmentDefinition:
$ cat << EOF | oc apply -f -
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: br1-nad
  annotations:
    k8s.v1.cni.cncf.io/resourceName: bridge.network.kubevirt.io/br1
spec:
  config: '{
    "cniVersion": "0.3.1",
    "name": "br1-nad", 
    "type": "cnv-bridge", 
    "bridge": "br1", 
    "macspoofchk": true, 
    "vlan": 1 
  }'
EOF

4. Create the VM (spec attached), where the secondary interface is set first, and the default is set second:
...
          interfaces:
          - name: br1
            bridge: {}
          - masquerade: {}
            name: default
...
      networks:
      - name: default
        pod: {}
      - multus:                                                                                                                                                        
          networkName: br1-nad
        name: br1

5. Start the VM and wait for it to run:
$ virtctl start vmb
VM vmb was scheduled to start
$
$ oc get vmi -w
NAME   AGE   PHASE        IP    NODENAME   READY
vmb    2s    Scheduling                    False
vmb    10s   Scheduled          net-ys-412-2-rszl9-worker-0-hfnc2   False
vmb    15s   Scheduled          net-ys-412-2-rszl9-worker-0-hfnc2   False
vmb    15s   Running      10.129.2.131   net-ys-412-2-rszl9-worker-0-hfnc2   False
vmb    15s   Running      10.129.2.131   net-ys-412-2-rszl9-worker-0-hfnc2   True

6. Verify the interfaces order in the VMI is indeed as you set it, i.e. secondary first, primary second:
$ oc get vmi vmb -ojsonpath={.spec.domain.devices.interfaces} | jq
[
  {
    "bridge": {},
    "macAddress": "02:14:e5:00:00:02",
    "name": "br1"
  },
  {
    "macAddress": "02:14:e5:00:00:03",
    "masquerade": {},
    "name": "default"
  }
]

7. Verify the VM IP, as seen in the output of "oc get vmi" (10.129.2.131), is assigned to the primary (default) interface:
$ oc get vmi vmb -ojsonpath={.status.interfaces} | jq
[
  {
    "infoSource": "domain, guest-agent",
    "interfaceName": "eth1",
    "ipAddress": "10.129.2.131",
    "ipAddresses": [
      "10.129.2.131"
    ],
    "mac": "02:14:e5:00:00:03",
    "name": "default",
    "queueCount": 1
  },
  {
    "infoSource": "domain, guest-agent",
    "interfaceName": "eth0",
    "ipAddress": "fe80::14:e5ff:fe00:2",
    "ipAddresses": [
      "fe80::14:e5ff:fe00:2"
    ],
    "mac": "02:14:e5:00:00:02",
    "name": "br1",
    "queueCount": 1
  }
]

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: OpenShift Virtualization 4.12.0 Images security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2023:0408