Bug 1847368
| Summary: | API server crash loop on ARO | |||
|---|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Alberto <agarcial> | |
| Component: | kube-apiserver | Assignee: | Alberto <agarcial> | |
| Status: | CLOSED ERRATA | QA Contact: | Ke Wang <kewang> | |
| Severity: | urgent | Docs Contact: | ||
| Priority: | urgent | |||
| Version: | 4.3.z | CC: | aos-bugs, dcain, jhou, jminter, mfojtik, mjudeiki, nstielau, sttts, xxia | |
| Target Milestone: | --- | |||
| Target Release: | 4.5.0 | |||
| Hardware: | Unspecified | |||
| OS: | Unspecified | |||
| Whiteboard: | ||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | ||
| Doc Text: | Story Points: | --- | ||
| Clone Of: | 1847185 | |||
| : | 1847419 (view as bug list) | Environment: | ||
| Last Closed: | 2020-07-13 17:44:18 UTC | Type: | --- | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | 1847185 | |||
| Bug Blocks: | 1847419 | |||
|
Comment 1
Stefan Schimanski
2020-06-16 13:54:25 UTC
Talked to Alberto (https://coreos.slack.com/archives/CB48XQ4KZ/p1592388418137000?thread_ts=1592315769.107100&cid=CB48XQ4KZ). This is no regression in 4.5, but was preexisting. Moving out of blocker list. Verified with OCP build 4.5.0-0.nightly-2020-06-23-052343, steps see below,
- Creating one sc and pvc on non-zoned region,
$ cat sc-non-zoned.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
annotations:
labels:
kubernetes.io/cluster-service: "true"
name: managed-premium-nonzoned
parameters:
kind: Managed
storageaccounttype: Premium_LRS
zoned: "false"
provisioner: kubernetes.io/azure-disk
volumeBindingMode: WaitForFirstConsumer
$ oc apply -f sc-non-zoned.yaml
storageclass.storage.k8s.io/managed-premium-nonzoned created
$ oc get sc
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
managed-premium (default) kubernetes.io/azure-disk Delete WaitForFirstConsumer true 76m
managed-premium-nonzoned kubernetes.io/azure-disk Delete WaitForFirstConsumer false 36m
$ cat pvc-nonzoned.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: azure-managed-non
spec:
accessModes:
- ReadWriteOnce
storageClassName: managed-premium-nonzoned
resources:
requests:
storage: 5Gi
$ oc apply -f pvc-non-zoned.yaml
persistentvolumeclaim/azure-managed-non created
$ oc get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
azure-managed-non Pending managed-premium-nonzoned 76s
$ cat mypod-non-zoned.yaml
kind: Pod
apiVersion: v1
metadata:
name: mypod
spec:
containers:
- name: mypod
image: nginx:1.15.5
resources:
requests:
cpu: 100m
memory: 128Mi
limits:
cpu: 250m
memory: 256Mi
volumeMounts:
- mountPath: "/mnt/azure"
name: volume
volumes:
- name: volume
persistentVolumeClaim:
claimName: azure-managed-non
$ oc create -f mypod-non-zoned.yaml
pod/mypod created
Checked the created pod status,
$ oc get pod/mypod
NAME READY STATUS RESTARTS AGE
mypod 1/1 Running 0 27s
$ oc describe pod/mypod
Name: mypod
Namespace: default
...
Status: Running
...
Volumes:
volume:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: azure-managed-non
ReadOnly: false
default-token-rjg64:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-rjg64
Optional: false
...
- Creating one sc and pvc on zoned region,
Since one default zoned sc already existed, no need new one.
$ oc get sc
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
managed-premium (default) kubernetes.io/azure-disk Delete WaitForFirstConsumer true 76m
$ cat pvc-zoned.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: azure-managed-disk
spec:
accessModes:
- ReadWriteOnce
storageClassName: managed-premium
resources:
requests:
storage: 5Gi
$ oc apply -f pvc-zoned.yaml
persistentvolumeclaim/azure-managed-disk created
$ oc get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
azure-managed-disk Bound pvc-f86422fb-576f-455a-98d4-f864b4a7bf6f 5Gi RWO managed-premium 17m
...
$ cat mypod-zoned.yaml
kind: Pod
apiVersion: v1
metadata:
name: mypod1
spec:
containers:
- name: mypod1
image: nginx:1.15.5
resources:
requests:
cpu: 100m
memory: 128Mi
limits:
cpu: 250m
memory: 256Mi
volumeMounts:
- mountPath: "/mnt/azure"
name: volume
volumes:
- name: volume
persistentVolumeClaim:
claimName: azure-managed-disk
$ oc apply -f mypod-zoned.yaml
pod/mypod1 created
$ oc get pods
NAME READY STATUS RESTARTS AGE
mypod 1/1 Running 0 35m
mypod1 1/1 Running 0 19m
$ oc get pods -A | grep -E 'apiserver|NAME' | grep -vE 'installer|revision|catalog'
NAMESPACE NAME READY STATUS RESTARTS AGE
openshift-apiserver-operator openshift-apiserver-operator-9c88c497-gwtpm 1/1 Running 2 91m
openshift-apiserver apiserver-dfd78fb66-9f9bf 1/1 Running 0 80m
openshift-apiserver apiserver-dfd78fb66-cdz5c 1/1 Running 0 79m
openshift-apiserver apiserver-dfd78fb66-ntxgv 1/1 Running 0 81m
openshift-kube-apiserver-operator kube-apiserver-operator-6fc9948f46-sxqhb 1/1 Running 2 91m
openshift-kube-apiserver kube-apiserver-kewang23azure51-5whj2-master-0 4/4 Running 0 66m
openshift-kube-apiserver kube-apiserver-kewang23azure51-5whj2-master-1 4/4 Running 0 70m
openshift-kube-apiserver kube-apiserver-kewang23azure51-5whj2-master-2 4/4 Running 0 69m
From above test results, it doesn't matter if sc and pvc are created on zoned or non-zoned, the kube-apiservers won't crash, move the bug verified.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:2409 |