Bug 1651971 - Logging Uninstall Unable to delete created logging pvc
Summary: Logging Uninstall Unable to delete created logging pvc
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Logging
Version: 3.9.0
Hardware: x86_64
OS: Linux
unspecified
medium
Target Milestone: ---
: 3.9.z
Assignee: Jeff Cantrill
QA Contact: Anping Li
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-11-21 10:35 UTC by Sri Vignesh Selvan
Modified: 2018-11-23 22:11 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-11-23 22:11:33 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Sri Vignesh Selvan 2018-11-21 10:35:15 UTC
Description of problem:
=======================
After Uninstalling logging through ansible-playbook, logging pvc should get deleted.

Version-Release number of selected component (if applicable):
=============================================================
oc v3.9.43
kubernetes v1.9.1+a0ce1bc657


How reproducible:
=================
Everytime

Steps to Reproduce:
===================
1. OCP 3.9 + OCS 3.11 setup with logging Installed
2. Uninstall logging by running ansible-playbook under openshift-logging, run config.yml
3. After uninstalling logging, check pvc. logging pvc should not get listed.

Actual results:
===============
logging pvc is present after uninstalling logging through ansible-playbook.

Expected results:
=================
logging pvc should get deleted after uninstalling logging through ansible-playbook.

Additional info:
================

[root@dhcp46-228 ~]# oc get pods -n logging
No resources found.

[root@dhcp46-228 ~]# oc get project
NAME                    DISPLAY NAME   STATUS
default                                Active
glusterfs                              Active
glusterfs-registry                     Active
kube-public                            Active
kube-system                            Active
management-infra                       Active
openshift                              Active
openshift-infra                        Active
openshift-node                         Active
openshift-web-console                  Active


[root@dhcp46-228 ~]# oc get pv
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS    CLAIM                                 STORAGECLASS               REASON    AGE
pvc-0204b02a-ec91-11e8-85e8-005056a593c6   50Gi       RWO            Delete           Bound     openshift-infra/metrics-cassandra-1   glusterfs-registry-block             1d
pvc-2ca3dd64-dce1-11e8-800f-005056a593c6   2Gi        RWO            Delete           Bound     glusterfs/claim3                      glusterfs-registry-block             21d
pvc-5d1969be-ed6a-11e8-bcd1-005056a593c6   80Gi       RWO            Delete           Bound     logging/logging-es-0                  glusterfs-registry-block             9m
pvc-6b6532fe-dcd2-11e8-800f-005056a593c6   2Gi        RWO            Delete           Bound     glusterfs/claim1                      glusterfs-storage-block              21d
pvc-a236c0e0-e66d-11e8-9974-005056a593c6   2Gi        RWO            Delete           Bound     glusterfs/claim003                    block-sc                             8d
pvc-aa231bee-e668-11e8-9974-005056a593c6   2Gi        RWO            Delete           Bound     glusterfs/claim002                    block-sc                             8d
pvc-d21680b2-dce0-11e8-800f-005056a593c6   2Gi        RWO            Delete           Bound     glusterfs/claim2                      glusterfs-storage-block              21d
pvc-e05e1b31-e66e-11e8-9974-005056a593c6   2Gi        RWO            Delete           Bound     glusterfs/claim005                    block-sc-invalid                     8d
registry-volume                            20Gi       RWX            Retain           Bound     default/registry-claim  


[root@dhcp46-228 ~]# oc describe pv pvc-5d1969be-ed6a-11e8-bcd1-005056a593c6
Name:            pvc-5d1969be-ed6a-11e8-bcd1-005056a593c6
Labels:          <none>
Annotations:     AccessKey=glusterblk-5b9056db-952a-4d96-9837-edb759e6e443-secret
                 AccessKeyNs=logging
                 Blockstring=url:http://heketi-registry.glusterfs-registry.svc:8080,user:admin,secret:heketi-registry-admin-secret-block,secretnamespace:glusterfs-registry
                 Description=Gluster-external: Dynamically provisioned PV
                 gluster.org/type=block
                 gluster.org/volume-id=8532e0b9d67636e72ca6abb2cb571492
                 glusterBlkProvIdentity=gluster.org/glusterblock
                 glusterBlockShare=blockvol_8532e0b9d67636e72ca6abb2cb571492
                 kubernetes.io/createdby=heketi
                 pv.kubernetes.io/provisioned-by=gluster.org/glusterblock
                 v2.0.0=v2.0.0
StorageClass:    glusterfs-registry-block
Status:          Bound
Claim:           logging/logging-es-0
Reclaim Policy:  Delete
Access Modes:    RWO
Capacity:        80Gi
Message:         
Source:
    Type:               ISCSI (an ISCSI Disk resource that is attached to a kubelet's host machine and then exposed to the pod)
    TargetPortal:       10.70.46.53
    IQN:                iqn.2016-12.org.gluster-block:5b9056db-952a-4d96-9837-edb759e6e443
    Lun:                0
    ISCSIInterface      default
    FSType:             xfs
    ReadOnly:           false
    Portals:            [10.70.46.190 10.70.46.51]
    DiscoveryCHAPAuth:  false
    SessionCHAPAuth:    true
    SecretRef:          &{glusterblk-5b9056db-952a-4d96-9837-edb759e6e443-secret }
    InitiatorName:      <none>
Events:                 <none>



[root@dhcp46-228 ~]# oc get pvc
NAME           STATUS    VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS               AGE
logging-es-0   Bound     pvc-5d1969be-ed6a-11e8-bcd1-005056a593c6   80Gi       RWO            glusterfs-registry-block   9m


[root@dhcp46-228 ~]# oc describe pvc logging-es-0
Name:          logging-es-0
Namespace:     logging
StorageClass:  glusterfs-registry-block
Status:        Bound
Volume:        pvc-5d1969be-ed6a-11e8-bcd1-005056a593c6
Labels:        logging-infra=support
Annotations:   control-plane.alpha.kubernetes.io/leader={"holderIdentity":"f6677ab6-ed58-11e8-966e-0a580a81041b","leaseDurationSeconds":15,"acquireTime":"2018-11-21T08:49:32Z","renewTime":"2018-11-21T08:49:45Z","lea...
               pv.kubernetes.io/bind-completed=yes
               pv.kubernetes.io/bound-by-controller=yes
               volume.beta.kubernetes.io/storage-provisioner=gluster.org/glusterblock
Finalizers:    []
Capacity:      80Gi
Access Modes:  RWO
Events:
  Type    Reason                 Age                From                                                           Message
  ----    ------                 ----               ----                                                           -------
  Normal  Provisioning           11m                gluster.org/glusterblock f6677ab6-ed58-11e8-966e-0a580a81041b  External provisioner is provisioning volume for claim "logging/logging-es-0"
  Normal  ExternalProvisioning   11m (x6 over 11m)  persistentvolume-controller                                    waiting for a volume to be created, either by external provisioner "gluster.org/glusterblock" or manually created by system administrator
  Normal  ProvisioningSucceeded  11m                gluster.org/glusterblock f6677ab6-ed58-11e8-966e-0a580a81041b  Successfully provisioned volume pvc-5d1969be-ed6a-11e8-bcd1-005056a593c6


From the logs I can see logging pvc objects deletion is skipped.


2018-11-21 14:27:36,823 p=23268 u=root |  TASK [openshift_logging : delete logging pvc objects] *************************************************************************************************************************************************************
2018-11-21 14:27:36,823 p=23268 u=root |  task path: /usr/share/ansible/openshift-ansible/roles/openshift_logging/tasks/delete_logging.yaml:18
2018-11-21 14:27:36,853 p=23268 u=root |  skipping: [dhcp46-228.lab.eng.blr.redhat.com] => {
    "changed": false,
    "skip_reason": "Conditional result was False",
    "skipped": true
}

Comment 2 Jeff Cantrill 2018-11-23 22:11:33 UTC
Logging roles intentionally do not remove storage unless explicitly setting the appropriate ansible variable 'openshift_logging_purge_logging'.

Ref: https://docs.openshift.com/container-platform/3.9/install_config/aggregate_logging.html#aggregate-logging-cleanup


Note You need to log in before you can comment on or make changes to this bug.