Description of problem: When deprovisioning kubevirt-apb it gets stuck on removing kuevirt-web-ui task. TASK [kubevirt_web_ui : include_tasks] ***************************************** included: /etc/ansible/roles/kubevirt-ansible/roles/kubevirt_web_ui/tasks/deprovision.yml for localhost TASK [kubevirt_web_ui : Remove kubevirt-web-ui project] ************************ Version-Release number of selected component (if applicable): kubevirt-apb-v3.11-14 How reproducible: 100% Steps to Reproduce: 1. Deploy kubevirt using APB 2. Deprovision kubevirt 3. observe logs of deprovisioning container Actual results: deprovision hangs on webconsole removal Expected results: it successfully deprovision kubevirt Additional info: [root@cnv-executor-ysegev-master1 ~]# oc logs -fn brew2-virtualization-depr-w9vht bundle-53f232b3-626d-41e5-8167-5aecbc1e3f34 DEPRECATED: APB playbooks should be stored at /opt/apb/project PLAY [Deprovision KubeVirt] **************************************************** TASK [ansible.kubernetes-modules : Install latest openshift client] ************ skipping: [localhost] TASK [ansibleplaybookbundle.asb-modules : debug] ******************************* skipping: [localhost] PLAY [all] ********************************************************************* TASK [Identify cluster] ******************************************************** ^[[Achanged: [localhost] TASK [Set cluster variable] **************************************************** [WARNING]: when statements should not include jinja2 templating delimiters such as {{ }} or {% %}. Found: {{ result.rc }} == 0 ok: [localhost] TASK [Login As Super User] ***************************************************** changed: [localhost] PLAY [masters[0]] ************************************************************** TASK [network-multus : include_tasks] ****************************************** included: /etc/ansible/roles/kubevirt-ansible/roles/network-multus/tasks/deprovision.yml for localhost TASK [network-multus : openshift cni config] *********************************** ok: [localhost] TASK [network-multus : kubernetes cni config] ********************************** skipping: [localhost] TASK [network-multus : Render multus deployment yaml] ************************** changed: [localhost] TASK [network-multus : Delete multus Resources] ******************************** changed: [localhost] TASK [network-multus : Render cni plugins deployment yaml] ********************* changed: [localhost] TASK [network-multus : Delete cni plugins Resources] *************************** changed: [localhost] TASK [network-multus : Render OVS plugin deployment yaml] ********************** changed: [localhost] TASK [network-multus : Delete OVS plugin Resources] **************************** changed: [localhost] TASK [network-multus : Render ovs-vsctl deployment yaml] *********************** changed: [localhost] TASK [network-multus : Delete ovs-vsctl Resources] ***************************** changed: [localhost] TASK [skydive : include_tasks] ************************************************* skipping: [localhost] PLAY [masters[0]] ************************************************************** TASK [kubevirt : include_tasks] ************************************************ included: /etc/ansible/roles/kubevirt-ansible/roles/kubevirt/tasks/deprovision.yml for localhost TASK [kubevirt : Check that demo-content.yaml still exists in /tmp] ************ ok: [localhost] TASK [kubevirt : Check for demo-content.yaml template in /etc/ansible/roles/kubevirt-ansible/roles/kubevirt/templates] *** ok: [localhost] TASK [kubevirt : Download Demo Content] **************************************** skipping: [localhost] TASK [kubevirt : Copy BYO Demo Content to /tmp] ******************************** changed: [localhost] TASK [kubevirt : Delete Demo Content] ****************************************** changed: [localhost] TASK [kubevirt : Check that kubevirt.yaml still exists in /tmp] **************** ok: [localhost] TASK [kubevirt : Check for kubevirt.yml.j2 template in /etc/ansible/roles/kubevirt-ansible/roles/kubevirt/templates] *** ok: [localhost] TASK [kubevirt : Download KubeVirt Template] *********************************** skipping: [localhost] TASK [kubevirt : Render KubeVirt template] ************************************* changed: [localhost] TASK [kubevirt : Delete KubeVirt Resources] ************************************ changed: [localhost] TASK [kubevirt : Delete Privileged Policy] ************************************* changed: [localhost] => (item=kubevirt-privileged) changed: [localhost] => (item=kubevirt-controller) changed: [localhost] => (item=kubevirt-infra) changed: [localhost] => (item=kubevirt-apiserver) TASK [kubevirt : Delete Hostmount-anyuid Policy] ******************************* changed: [localhost] PLAY [masters[0]] ************************************************************** TASK [cdi : include_tasks] ***************************************************** included: /etc/ansible/roles/kubevirt-ansible/roles/cdi/tasks/deprovision.yml for localhost TASK [cdi : Render kube-system ResourceQuota deprovision yaml] ***************** changed: [localhost] TASK [cdi : Delete kube-system ResourceQuota] ********************************** changed: [localhost] TASK [cdi : Check that cdi-provision.yml still exists in /tmp] ***************** ok: [localhost] TASK [cdi : Check for cdi-controller.yml.j2 template in /etc/ansible/roles/kubevirt-ansible/roles/cdi/templates] *** ok: [localhost] TASK [cdi : Download CDI Template] ********************************************* skipping: [localhost] TASK [cdi : Render CDI deprovision yaml] *************************************** changed: [localhost] TASK [cdi : Delete CDI Resources] ********************************************** changed: [localhost] PLAY [Kubevirt Web UI Install Checkpoint Start] ******************************** TASK [Set Console install 'In Progress'] *************************************** ok: [localhost] PLAY [Kubevirt Web UI] ********************************************************* TASK [kubevirt_web_ui : include_tasks] ***************************************** included: /etc/ansible/roles/kubevirt-ansible/roles/kubevirt_web_ui/tasks/required_params.yml for localhost TASK [kubevirt_web_ui : set_fact] ********************************************** skipping: [localhost] TASK [kubevirt_web_ui : set_fact] ********************************************** ok: [localhost] TASK [kubevirt_web_ui : set_fact] ********************************************** ok: [localhost] TASK [kubevirt_web_ui : Verify required kubevirt_web_ui_image_name variable is set] *** skipping: [localhost] TASK [kubevirt_web_ui : Discover public_master_hostname from openshift console deployment] *** changed: [localhost] TASK [kubevirt_web_ui : set_fact] ********************************************** ok: [localhost] TASK [kubevirt_web_ui : Discover openshift_master_default_subdomain from openshift console deployment] *** changed: [localhost] TASK [kubevirt_web_ui : set_fact] ********************************************** ok: [localhost] TASK [kubevirt_web_ui : set_fact] ********************************************** ok: [localhost] TASK [kubevirt_web_ui : set_fact] ********************************************** ok: [localhost] TASK [kubevirt_web_ui : set_fact] ********************************************** ok: [localhost] TASK [kubevirt_web_ui : set_fact] ********************************************** ok: [localhost] TASK [kubevirt_web_ui : include_tasks] ***************************************** included: /etc/ansible/roles/kubevirt-ansible/roles/kubevirt_web_ui/tasks/deprovision.yml for localhost TASK [kubevirt_web_ui : Remove kubevirt-web-ui project] ************************ [root@cnv-executor-ysegev-master1 ~]# oc get namespaces NAME STATUS AGE kubevirt-web-ui Terminating 7m [root@cnv-executor-ysegev-master1 ~]# oc get pods -n kubevirt-web-ui NAME READY STATUS RESTARTS AGE console-78fd8f977c-llbm6 1/1 Running 0 8m [root@cnv-executor-ysegev-master1 ~]# oc get pods -n kubevirt-web-ui -o yaml apiVersion: v1 items: - apiVersion: v1 kind: Pod metadata: annotations: k8s.v1.cni.cncf.io/networks-status: |- [{ "name": "openshift.1", "ips": [ "10.130.0.15" ], "default": true, "dns": {} }] openshift.io/scc: restricted creationTimestamp: 2018-11-20T15:46:57Z generateName: console-78fd8f977c- labels: app: kubevirt-web-ui component: ui pod-template-hash: "3498495337" name: console-78fd8f977c-llbm6 namespace: kubevirt-web-ui ownerReferences: - apiVersion: apps/v1 blockOwnerDeletion: true controller: true kind: ReplicaSet name: console-78fd8f977c uid: 8321ea76-ecdb-11e8-9dfa-fa163ef8ef62 resourceVersion: "6244" selfLink: /api/v1/namespaces/kubevirt-web-ui/pods/console-78fd8f977c-llbm6 uid: 83252e0a-ecdb-11e8-9dfa-fa163ef8ef62 spec: affinity: podAntiAffinity: preferredDuringSchedulingIgnoredDuringExecution: - podAffinityTerm: labelSelector: matchLabels: app: kubevirt-web-ui topologyKey: kubernetes.io/hostname weight: 100 containers: - command: - /opt/bridge/bin/bridge - --public-dir=/opt/bridge/static - --config=/var/console-config/console-config.yaml - --branding=okdvirt image: brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/cnv-tech-preview/kubevirt-web-ui:v1.3.0 imagePullPolicy: IfNotPresent livenessProbe: failureThreshold: 3 httpGet: path: /health port: 8443 scheme: HTTPS initialDelaySeconds: 30 periodSeconds: 10 successThreshold: 1 timeoutSeconds: 1 name: console ports: - containerPort: 8443 protocol: TCP readinessProbe: failureThreshold: 3 httpGet: path: /health port: 8443 scheme: HTTPS periodSeconds: 10 successThreshold: 1 timeoutSeconds: 1 resources: limits: cpu: 100m memory: 100Mi requests: cpu: 100m memory: 100Mi securityContext: capabilities: drop: - KILL - MKNOD - SETGID - SETUID runAsUser: 1000370000 terminationMessagePath: /dev/termination-log terminationMessagePolicy: File volumeMounts: - mountPath: /var/serving-cert name: serving-cert readOnly: true - mountPath: /var/oauth-config name: oauth-config readOnly: true - mountPath: /var/console-config name: console-config - mountPath: /var/run/secrets/kubernetes.io/serviceaccount name: default-token-t9wh2 readOnly: true dnsPolicy: ClusterFirst imagePullSecrets: - name: default-dockercfg-pzntn nodeName: cnv-executor-ysegev-node2.example.com nodeSelector: node-role.kubernetes.io/compute: "true" priority: 0 restartPolicy: Always schedulerName: default-scheduler securityContext: fsGroup: 1000370000 seLinuxOptions: level: s0:c19,c14 serviceAccount: default serviceAccountName: default terminationGracePeriodSeconds: 30 tolerations: - effect: NoSchedule key: node.kubernetes.io/memory-pressure operator: Exists volumes: - name: serving-cert secret: defaultMode: 288 secretName: console-serving-cert - name: oauth-config secret: defaultMode: 288 secretName: console-oauth-config - configMap: defaultMode: 288 name: console-config name: console-config - name: default-token-t9wh2 secret: defaultMode: 420 secretName: default-token-t9wh2 status: conditions: - lastProbeTime: null lastTransitionTime: 2018-11-20T15:46:58Z status: "True" type: Initialized - lastProbeTime: null lastTransitionTime: 2018-11-20T15:47:28Z status: "True" type: Ready - lastProbeTime: null lastTransitionTime: null status: "True" type: ContainersReady - lastProbeTime: null lastTransitionTime: 2018-11-20T15:46:57Z status: "True" type: PodScheduled containerStatuses: - containerID: cri-o://856e1e8dbf6fcd0696172ebe07e286bb5b72c75f01794c8ceabc5e9357616e5e image: brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/cnv-tech-preview/kubevirt-web-ui:v1.3.0 imageID: brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/cnv-tech-preview/kubevirt-web-ui@sha256:e2dec321bb3f36b9017af581747e3773caed8ad6b731abe2f3fa6efa8e5538be lastState: {} name: console ready: true restartCount: 0 state: running: startedAt: 2018-11-20T15:47:23Z hostIP: 172.16.0.13 phase: Running podIP: 10.130.0.15 qosClass: Guaranteed startTime: 2018-11-20T15:46:58Z kind: List metadata: resourceVersion: "" selfLink: "" [root@cnv-executor-ysegev-master1 ~]# oc describe pods -n kubevirt-web-ui Name: console-78fd8f977c-llbm6 Namespace: kubevirt-web-ui Priority: 0 PriorityClassName: <none> Node: cnv-executor-ysegev-node2.example.com/172.16.0.13 Start Time: Tue, 20 Nov 2018 10:46:58 -0500 Labels: app=kubevirt-web-ui component=ui pod-template-hash=3498495337 Annotations: k8s.v1.cni.cncf.io/networks-status=[{ "name": "openshift.1", "ips": [ "10.130.0.15" ], "default": true, "dns": {} }] openshift.io/scc=restricted Status: Running IP: 10.130.0.15 Controlled By: ReplicaSet/console-78fd8f977c Containers: console: Container ID: cri-o://856e1e8dbf6fcd0696172ebe07e286bb5b72c75f01794c8ceabc5e9357616e5e Image: brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/cnv-tech-preview/kubevirt-web-ui:v1.3.0 Image ID: brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/cnv-tech-preview/kubevirt-web-ui@sha256:e2dec321bb3f36b9017af581747e3773caed8ad6b731abe2f3fa6efa8e5538be Port: 8443/TCP Host Port: 0/TCP Command: /opt/bridge/bin/bridge --public-dir=/opt/bridge/static --config=/var/console-config/console-config.yaml --branding=okdvirt State: Running Started: Tue, 20 Nov 2018 10:47:23 -0500 Ready: True Restart Count: 0 Limits: cpu: 100m memory: 100Mi Requests: cpu: 100m memory: 100Mi Liveness: http-get https://:8443/health delay=30s timeout=1s period=10s #success=1 #failure=3 Readiness: http-get https://:8443/health delay=0s timeout=1s period=10s #success=1 #failure=3 Environment: <none> Mounts: /var/console-config from console-config (rw) /var/oauth-config from oauth-config (ro) /var/run/secrets/kubernetes.io/serviceaccount from default-token-t9wh2 (ro) /var/serving-cert from serving-cert (ro) Conditions: Type Status Initialized True Ready True ContainersReady True PodScheduled True Volumes: serving-cert: Type: Secret (a volume populated by a Secret) SecretName: console-serving-cert Optional: false oauth-config: Type: Secret (a volume populated by a Secret) SecretName: console-oauth-config Optional: false console-config: Type: ConfigMap (a volume populated by a ConfigMap) Name: console-config Optional: false default-token-t9wh2: Type: Secret (a volume populated by a Secret) SecretName: default-token-t9wh2 Optional: false QoS Class: Guaranteed Node-Selectors: node-role.kubernetes.io/compute=true Tolerations: node.kubernetes.io/memory-pressure:NoSchedule Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 8m default-scheduler Successfully assigned kubevirt-web-ui/console-78fd8f977c-llbm6 to cnv-executor-ysegev-node2.example.com Warning FailedMount 8m kubelet, cnv-executor-ysegev-node2.example.com MountVolume.SetUp failed for volume "serving-cert" : secrets "console-serving-cert" not found Normal Pulling 8m kubelet, cnv-executor-ysegev-node2.example.com pulling image "brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/cnv-tech-preview/kubevirt-web-ui:v1.3.0" Normal Pulled 8m kubelet, cnv-executor-ysegev-node2.example.com Successfully pulled image "brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/cnv-tech-preview/kubevirt-web-ui:v1.3.0" Normal Created 8m kubelet, cnv-executor-ysegev-node2.example.com Created container Normal Started 8m kubelet, cnv-executor-ysegev-node2.example.com Started container
This should work in the operator we will not fix the APB