Bug 1989700 - Metallb: The controller fails to handout the available IP address if there is a service that is assigned a static address
Summary: Metallb: The controller fails to handout the available IP address if there is...
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Networking
Version: 4.9
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: ---
Assignee: Mohamed Mahmoud
QA Contact: Arti Sood
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-08-03 18:19 UTC by Arti Sood
Modified: 2021-08-04 17:19 UTC (History)
0 users

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-08-04 17:19:31 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Arti Sood 2021-08-03 18:19:36 UTC
Description of problem:
Metallb does not assign an IP address to the service if there is a service on the cluster with static IP address even though the address pool has available IP address.

Remove the service configured with static IP address, the loadbalancer service immediately gets the IP address assigned from the address pool.



Version-Release number of selected component (if applicable):
4.9

How reproducible:
Always

Steps to Reproduce:
1. Install the loadbalancer on vSphere cluster with metallb operator
2. Create a address pool with two ip address available to be assigned to services.

apiVersion: metallb.io/v1alpha1
kind: AddressPool
metadata:
  name: address-pool-silver
  namespace: metallb-system
spec:
  protocol: layer2
  addresses:
    - 172.31.249.160 - 172.31.249.160
    - 172.31.249.220 - 172.31.249.220

3. Create a loadbalancer service in project a1 with the following yaml (hello-world.yaml) to see it successfully get an IP address assigned to it.
---------------------
apiVersion: apps/v1
kind: ReplicaSet
metadata:
  name: hello-world
  labels:
    app: hello-world
spec:
  selector:
    matchLabels:
      app: hello-world
  replicas: 2
  template:
    metadata:
      labels:
        app: hello-world
    spec:
      containers:
      - name: hello-world
        image: gcr.io/google-samples/node-hello:1.0
        imagePullPolicy: Always
        ports:
        - containerPort: 8080
          protocol: TCP
 
---
apiVersion: v1
kind: Service
metadata:
  name: hello-world
spec:
  selector:
    app: hello-world
  ports:
  - port: 80
    targetPort: 8080
    protocol: TCP
  type: LoadBalancer

NAME          TYPE           CLUSTER-IP       EXTERNAL-IP      PORT(S)        AGE
hello-world   LoadBalancer   172.30.204.132   172.31.249.160   80:31317/TCP   14h

4. Edit network cluster object to provide allowed CIDR range.

oc edit network.config.openshift.io cluster -o yaml
spec:
  clusterNetwork:
  - cidr: 10.128.0.0/14
    hostPrefix: 23
  externalIP:
    policy:
      allowedCIDRs:
      - 172.31.249.10/24


5. Create another load balancer service with static Ip address in project a2 with YAML (hello.idle.yaml) below:-
------
apiVersion: v1
kind: List
items:
- apiVersion: v1
  kind: ReplicationController
  metadata:
    labels:
      name: hello-idle
    name: hello-idle
  spec:
    replicas: 2
    selector:
      name: hello-idle
    template:
      metadata:
        labels:
          name: hello-idle
      spec:
          containers:
          - image: quay.io/openshifttest/hello-pod@sha256:04b6af86b03c1836211be2589db870dba09b7811c197c47c07fbbe33c7f80ef7
            name: hello-idle
            ports:
            - containerPort: 8080
              protocol: TCP
            resources:
              limits:
                cpu: 200m
                memory: 256Mi
              requests:
                cpu: 100m
                memory: 256Mi
            terminationMessagePath: /dev/termination-log
          dnsPolicy: ClusterFirst
          restartPolicy: Always
          securityContext: {}
          terminationGracePeriodSeconds: 30
- apiVersion: v1
  kind: Service
  metadata:
    name: hello-idle
  spec:
    ports:
    - port: 80
      targetPort: 8080
      protocol: TCP
    selector:
      name: hello-idle
    type: LoadBalancer
    externalIPs: 
    - 172.31.249.10



6. Create another loadbalancer service with YAML in step#3 in project a3.

oc create -f hello-world.yaml


Actual results:
Service does not get IP address assigned.
NAME          TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)        AGE
hello-world   LoadBalancer   172.30.226.205   <pending>     80:30863/TCP   118m


Expected results:

Service should get an IP address assigned.

Additional info:
NAME            CSV                                   APPROVAL    APPROVED
install-8d5v2   metallb-operator.4.9.0-202108021529   Automatic   true
Server Version: 4.9.0-0.nightly-2021-08-02-145924
Kubernetes Version: v1.21.1+8268f88

Comment 1 Mohamed Mahmoud 2021-08-04 13:17:24 UTC
for statically assigned service IP we need to use 
spec:
  loadBalancerIP: 40.121.183.52

as described in https://metallb.universe.tf/usage/example/

Comment 2 Arti Sood 2021-08-04 14:47:16 UTC
Provided IP address in service spec as below:-
 spec:
    ports:
    - port: 80
      targetPort: 8080
      protocol: TCP
    selector:
      name: hello-idle
    type: LoadBalancer
    loadBalancerIP: 172.31.249.82

IP address is not assigned to service.

NAME         TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)        AGE
hello-idle   LoadBalancer   172.30.188.135   <pending>     80:32179/TCP   45m


MetalLB logs error message:-

{"caller":"main.go:76","event":"endUpdate","msg":"end of service update","service":"a1/hello-world","ts":"2021-08-04T13:58:51.149428959Z"}
{"caller":"main.go:49","event":"startUpdate","msg":"start of service update","service":"a2/hello-idle","ts":"2021-08-04T13:59:57.176240509Z"}
{"caller":"service.go:93","event":"clearAssignment","msg":"user requested a different IP than the one currently assigned","reason":"differentIPRequested","service":"a2/hello-idle","ts":"2021-08-04T13:59:57.176300106Z"}
{"caller":"service.go:106","error":"\"172.31.249.82\" is not allowed in config","msg":"IP allocation failed","op":"allocateIP","service":"a2/hello-idle","ts":"2021-08-04T13:59:57.176336044Z"}
{"caller":"main.go:75","event":"noChange","msg":"service converged, no change","service":"a2/hello-idle","ts":"2021-08-04T13:59:57.176452342Z"}
{"caller":"main.go:76","event":"endUpdate","msg":"end of service update","service":"a2/hello-idle","ts":"2021-08-04T13:59:57.176473491Z"}

Comment 3 Arti Sood 2021-08-04 15:38:55 UTC
To assign static ip address to a loadbalancer service.

1. Create address pool with IPs that could be used as static IPs.
apiVersion: metallb.io/v1alpha1
kind: AddressPool
metadata:
  name: address-pool-static
  namespace: metallb-system
spec:
  protocol: layer2
  addresses:
    - 172.31.249.82 - 172.31.249.82
    - 172.31.249.202 - 172.31.249.202
  autoAssign: false

2. Create a service with specific IP and annotation for the address pool.

apiVersion: v1
kind: List
items:
- apiVersion: v1
  kind: ReplicationController
  metadata:
    labels:
      name: hello-idle
    name: hello-idle
  spec:
    replicas: 2
    selector:
      name: hello-idle
    template:
      metadata:
        labels:
          name: hello-idle
      spec:
          containers:
          - image: quay.io/openshifttest/hello-pod@sha256:04b6af86b03c1836211be2589db870dba09b7811c197c47c07fbbe33c7f80ef7
            name: hello-idle
            ports:
            - containerPort: 8080
              protocol: TCP
            resources:
              limits:
                cpu: 200m
                memory: 256Mi
              requests:
                cpu: 100m
                memory: 256Mi
            terminationMessagePath: /dev/termination-log
          dnsPolicy: ClusterFirst
          restartPolicy: Always
          securityContext: {}
          terminationGracePeriodSeconds: 30
- apiVersion: v1
  kind: Service
  metadata:
    name: hello-idle
    annotations:
      metallb.universe.tf/address-pool: address-pool-static
  spec:
    ports:
    - port: 80
      targetPort: 8080
      protocol: TCP
    selector:
      name: hello-idle
    type: LoadBalancer
    loadBalancerIP: 172.31.249.82


The service will be assigned 172.31.249.82 from the pool of IPs in the address pool.
NAME         TYPE           CLUSTER-IP       EXTERNAL-IP     PORT(S)        AGE
hello-idle   LoadBalancer   172.30.191.103   172.31.249.82   80:31966/TCP   13m

curl http://172.31.249.82
Hello Pod!

Comment 4 Arti Sood 2021-08-04 15:45:58 UTC
I was able create another service that was obtained and assigned IP address dynamically from different address pool.


Note You need to log in before you can comment on or make changes to this bug.