Red Hat Bugzilla – Bug 1395547
Provisioned PV logged an event for volume creation failure
Last modified: 2017-08-16 15:51 EDT
Description of problem: Sometimes, the provisioned PV logs an event for failed volume creation. PV and volume are proivisioned successfully, but an event for the error is left there. Version-Release number of selected component (if applicable): openshift v3.4.0.26+f7e109e kubernetes v1.4.0+776c994 etcd 3.1.0-rc.0 How reproducible: Sometimes Steps to Reproduce: 1. Create StorageClass for Ceph RBD provisioner 2. Create PVC 3. After PV is provisioned, run 'oc describe pv' Actual results: Name: cephrbdprovisioner Namespace: default StorageClass: cephrbdprovisioner Status: Bound Volume: pvc-e40020d8-abc9-11e6-9cd7-0eeaaf62bca4 Labels: <none> Capacity: 5Gi Access Modes: RWO Events: FirstSeen LastSeen Count From SubobjectPath Type Reason Message --------- -------- ----- ---- ------------- -------- ------ ------- 33m 33m 1 {persistentvolume-controller } Warning ProvisioningFailed Failed to provision volume with StorageClass "cephrbdprovisioner": rbd: create volume failed, err: exit status 1 Expected results: It seems reasonable that the provisioner failed at first try but later succeeded so an failure event was logged, not sure why sometimes it failed at provisioning. Reported this bug just in case customer get confused seeing this. Additional info:
jhou, do you have any logs from the kube-controller-manager (openshift-master) or ceph why it failed? We should log something better than "exit status 1". Also, adding an event "provisioned volume xxx using volume plugin yyy" would not harm either so the users can see that we recovered from the error.
(In reply to Jan Safranek from comment #1) > Also, adding an event "provisioned volume xxx using volume plugin yyy" would > not harm either so the users can see that we recovered from the error. Filled https://github.com/kubernetes/kubernetes/issues/37377 and https://trello.com/c/E1uljgPb/369-internal-provisioners-should-report-success-as-an-event Now the ambiguous "exit status 1" from RBD. We should make it nicer.
(In reply to Jan Safranek from comment #1) > We should log something better than "exit status 1". Pushed PR https://github.com/kubernetes/kubernetes/pull/37380, scheduled for 1.6. Please raise the priority if you want it earlier.
Almost missed this one, Origin PR: https://github.com/openshift/origin/pull/13017
This is not reproduced in latest ocp. Mark as verified.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2017:1716