Description of problem:

After enabled "MAC Address Changes" and "Forged Transmissions" on the dvSwitch as per stated on documentation [0] , Egress pod doesn't if promiscuous mode is disabled.

[0] https://docs.openshift.com/container-platform/3.4/admin_guide/managing_pods.html#admin-guide-limit-pod-access-important-deployment-considerations

Version-Release number of selected component (if applicable):

Environment:
Openshift 3.4
VMware



How reproducible:
100%

Steps to Reproduce:
1. Enable "MAC Address Changes" and "Forged Transmissions"
2. Deploy an egress router pod:

[root@bf-ocp-master2 ~]# cat egresstest.yaml
apiVersion: v1
kind: Pod
metadata:
  name: egress-1
  labels:
    name: egress-1
  annotations:
    pod.network.openshift.io/assign-macvlan: "true"
spec:
  containers:
  - name: egress-router
    image: openshift3/ose-egress-router
    securityContext:
      privileged: true
    env:
    - name: EGRESS_SOURCE 
      value: 10.23.155.13
    - name: EGRESS_GATEWAY 
      value: 10.23.155.1 
    - name: EGRESS_DESTINATION 
      value: 195.225.7.67 
  nodeSelector:
    type: node

[root@bf-ocp-master2 ~]#


3. "oc rsh" within the pod, try to ping the $EGRESS_DESTINATION, got "Destination Host Unreachable" message:

[root@bf-ocp-master2 ~]# oc rsh egress-1 
sh-4.2# ping 195.225.7.67
PING 195.225.7.67 (195.225.7.67) 56(84) bytes of data.
From 10.23.155.13 icmp_seq=1 Destination Host Unreachable
From 10.23.155.13 icmp_seq=2 Destination Host Unreachable
From 10.23.155.13 icmp_seq=3 Destination Host Unreachable
From 10.23.155.13 icmp_seq=4 Destination Host Unreachable

sh-4.2# ip n
10.23.155.1 dev macvlan0  FAILED

[root@bf-ocp-node3 ~]#


Actual results:
$EGRESS_DESTINATION is not reachable inside the pods

Expected results:
EGRESS_DESTINATION should "pingable" from egress router pod

Additional info:
As per [1]: vSwitches do not "learn" MAC addresses. To make this work,  promiscuous mode on the vSwitch might need to be enabled.

[1] https://communities.vmware.com/thread/320523?start=0&tstart=0