Description of problem:
LB service works well with 1 Windows node + n Windows pods, if scale up another Windows node, LB become unstable
e.g.
# oc get nodes -l kubernetes.io/os=windows
NAME              STATUS   ROLES    AGE     VERSION
winworker-5hjbf   Ready    worker   4h13m   v1.19.2-1005+e8f355a526fe47
winworker-wcvxf   Ready    worker   4h17m   v1.19.2-1005+e8f355a526fe47

# oc get pod -owide
NAME                             READY   STATUS    RESTARTS   AGE    IP           NODE              NOMINATED NODE   READINESS GATES
win-webserver-549cd7495d-86fx7   1/1     Running   0          85m    10.132.0.6   winworker-wcvxf   <none>           <none>
win-webserver-549cd7495d-8x4v9   1/1     Running   0          2m7s   10.132.1.7   winworker-5hjbf   <none>           <none>

# oc get service
NAME            TYPE           CLUSTER-IP       EXTERNAL-IP      PORT(S)        AGE
win-webserver   LoadBalancer   172.30.136.239   52.182.219.200   80:31231/TCP   148m

# curl 52.182.219.200
curl: (7) Failed connect to 52.182.219.200:80; Connection timed out


Version-Release number of selected component (if applicable):
WMCO 1.0.0
OCP version 4.6.0-0.nightly-2020-12-08-021151

How reproducible:
always

Steps to Reproduce:
1, Install OCP cluster with ovn-kubernetes network
2, Scale up 2 Windows node and 2 Windows pod
3, Check LB service
# curl 52.182.219.200

Actual results:
LB service unstable

Expected results:
LB service should work well

Additional info: