Bug 1320113

Summary: Could not attach ebs volume in containered env for openshift
Product: OpenShift Container Platform Reporter: Chao Yang <chaoyang>
Component: StorageAssignee: Jan Safranek <jsafrane>
Status: CLOSED CURRENTRELEASE QA Contact: Jianwei Hou <jhou>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 3.2.0CC: aos-bugs, bchilds, tdawson
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-06-13 18:47:04 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Chao Yang 2016-03-22 10:39:59 UTC
Description of problem:
Could not attach ebs volume in containered env for openshift

Version-Release number of selected component (if applicable):
openshift v3.2.0.6
kubernetes v1.2.0-36-g4a3f9c5
etcd 2.2.5

How reproducible:
Always

Steps to Reproduce:
1. Create a dynamic provison pvc
{
  "kind": "PersistentVolumeClaim",
  "apiVersion": "v1",
  "metadata": {
    "name": "claim2",
    "labels": {
         "name": "testing"
     },
    "annotations": {
        "volume.alpha.kubernetes.io/storage-class": "foo"
    }
  },
  "spec": {
    "accessModes": [
      "ReadWriteOnce"
    ],
    "resources": {
      "requests": {
        "storage": "3Gi"
      }
    }
  }
}

2. NAME      STATUS    VOLUME         CAPACITY   ACCESSMODES   AGE
claim2    Bound     pv-aws-gc8n1   3Gi        RWO           1h

3. Create a pod using pvc claim2
4. Check pod status

Actual results:
1.volume  vol-f4798356 is in-use status from aws ebs web console
2. oc get event
24m         24m        1         awspd1                         Pod                   Normal    Scheduled               {default-scheduler }                        Successfully assigned awspd1 to ip-172-18-1-232.ec2.internal
14m         4m         2         awspd1                         Pod                   Warning   FailedMount             {kubelet ip-172-18-1-232.ec2.internal}      Unable to mount volumes for pod "awspd1_default(5a5163a1-f00d-11e5-83f4-0e8b1f5ad811)": Could not attach EBS Disk "aws://us-east-1d/vol-f4798356". Timeout waiting for mount paths to be created.
14m         4m         2         awspd1                         Pod                   Warning   FailedSync              {kubelet ip-172-18-1-232.ec2.internal}      Error syncing pod, skipping: Could not attach EBS Disk "aws://us-east-1d/vol-f4798356". Timeout waiting for mount paths to be created.




Expected results:
Pod should be running

Additional info:
Mar 22 09:06:24 ip-172-18-1-232.ec2.internal atomic-openshift-node[65494]: I0322 09:06:24.862321   65540 aws.go:1070] Assigned
 mount device bc -> volume vol-f4798356
Mar 22 09:06:25 ip-172-18-1-232.ec2.internal atomic-openshift-node[65494]: I0322 09:06:25.149716   65540 aws.go:1318] AttachVo
lume request returned {
Mar 22 09:06:25 ip-172-18-1-232.ec2.internal atomic-openshift-node[65494]: AttachTime: 2016-03-22 09:06:25.145 +0000 UTC,
Mar 22 09:06:25 ip-172-18-1-232.ec2.internal atomic-openshift-node[65494]: Device: "/dev/xvdbc",
Mar 22 09:06:25 ip-172-18-1-232.ec2.internal atomic-openshift-node[65494]: InstanceId: "i-e2e35379",
Mar 22 09:06:25 ip-172-18-1-232.ec2.internal atomic-openshift-node[65494]: State: "attaching",
Mar 22 09:06:25 ip-172-18-1-232.ec2.internal atomic-openshift-node[65494]: VolumeId: "vol-f4798356"
Mar 22 09:06:25 ip-172-18-1-232.ec2.internal atomic-openshift-node[65494]: }
Mar 22 09:06:25 ip-172-18-1-232.ec2.internal atomic-openshift-node[65494]: I0322 09:06:25.260695   65540 aws.go:1193] Waiting 
for volume state: actual=attaching, desired=attached
Mar 22 09:06:26 ip-172-18-1-232.ec2.internal atomic-openshift-node[65494]: I0322 09:06:26.311786   65540 aws.go:1193] Waiting 
for volume state: actual=attaching, desired=attached
Mar 22 09:06:27 ip-172-18-1-232.ec2.internal kernel: blkfront: xvdbc: barrier or flush: disabled; persistent grants: disabled;
 indirect descriptors: enabled;
Mar 22 09:06:27 ip-172-18-1-232.ec2.internal kernel:  xvdbc: unknown partition table

Comment 1 Jan Safranek 2016-03-23 11:03:44 UTC
Fixed by https://github.com/openshift/origin/pull/8182

Comment 2 Troy Dawson 2016-03-23 21:34:01 UTC
This should be in OSE v3.2.0.7 that was build and pushed to qe today.

Comment 3 Chao Yang 2016-03-24 05:24:02 UTC
Verification is failed on 
openshift v3.2.0.7
kubernetes v1.2.0-36-g4a3f9c5
etcd 2.2.5


Pods keeps status of "ContainerCreating"

1m        1m        1         test2      Pod                               Normal    Scheduled           {default-scheduler }                     Successfully assigned test2 to ip-172-18-3-242.ec2.internal
1m        8s        7         test2      Pod                               Warning   FailedMount         {kubelet ip-172-18-3-242.ec2.internal}   Unable to mount volumes for pod "test2_default(ca2f8bf0-f171-11e5-a9f0-0e0a208947f9)": exit status 32
1m        8s        7         test2      Pod                               Warning   FailedSync          {kubelet ip-172-18-3-242.ec2.internal}   Error syncing pod, skipping: exit status 32


Mar 24 03:36:04 ip-172-18-3-242.ec2.internal atomic-openshift-node[10519]: W0324 03:36:04.198738   10565 aws.go:1043] Got assignment call for already-assigned volume: ba@vol-b139f513
Mar 24 03:36:04 ip-172-18-3-242.ec2.internal atomic-openshift-node[10519]: E0324 03:36:04.301091   10565 aws.go:1081] endAttaching on non-allocated device
Mar 24 03:36:04 ip-172-18-3-242.ec2.internal atomic-openshift-node[10519]: I0324 03:36:04.301122   10565 aws_util.go:208] Successfully attached EBS Disk "aws://us-east-1d/vol-b139f513".
Mar 24 03:36:04 ip-172-18-3-242.ec2.internal atomic-openshift-node[10519]: I0324 03:36:04.304315   10565 nsenter_mount.go:179] Failed findmnt command: exit status 1
Mar 24 03:36:04 ip-172-18-3-242.ec2.internal atomic-openshift-node[10519]: E0324 03:36:04.331983   10565 kubelet.go:1796] Unable to mount volumes for pod "test2_default(27e92b1a-f171-11e5-a9f0-0e0a208947f9)": exit status 32; skipping pod
Mar 24 03:36:04 ip-172-18-3-242.ec2.internal atomic-openshift-node[10519]: E0324 03:36:04.332001   10565 pod_workers.go:138] Error syncing pod 27e92b1a-f171-11e5-a9f0-0e0a208947f9, skipping: exit status 32

Comment 4 Jan Safranek 2016-03-24 15:44:45 UTC
This is sort of duplicate of bug #1313210, I'll keep it open to track progress, but any discussion should happen in #1313210.

Comment 5 Jan Safranek 2016-04-14 08:27:55 UTC
Merged as origin PR: https://github.com/openshift/origin/pull/8501

Comment 6 Chao Yang 2016-05-04 09:02:01 UTC
I test this bug on ose, pod using aws ebs volume can be runing
openshift v3.2.0.20
kubernetes v1.2.0-36-g4a3f9c5
etcd 2.2.5

Comment 7 Chao Yang 2016-05-10 01:56:26 UTC
have tested and passed on 
openshift v3.2.0.20
kubernetes v1.2.0-36-g4a3f9c5
etcd 2.2.5

Comment 8 Scott Dodson 2016-06-13 18:47:04 UTC
Fixed in 3.2.0.20