This service will be undergoing maintenance at 00:00 UTC, 2017-10-23 It is expected to last about 30 minutes
Bug 1471062 - [RFE] etcd pod is in pending status due to pv is not created when deploying service catalog onto an existing cluster
[RFE] etcd pod is in pending status due to pv is not created when deploying s...
Status: NEW
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer (Show other bugs)
3.6.0
Unspecified Unspecified
low Severity low
: ---
: 3.7.0
Assigned To: Scott Dodson
Johnny Liu
: Reopened
Depends On: 1467574
Blocks:
  Show dependency treegraph
 
Reported: 2017-07-14 06:35 EDT by Johnny Liu
Modified: 2017-10-09 16:50 EDT (History)
5 users (show)

See Also:
Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2017-07-20 10:47:35 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Johnny Liu 2017-07-14 06:35:14 EDT
Description of problem:
On an existing cluster, enable service catalog deployment, at final, in openshift-ansible-service-broker project, asb and etcd pod are not running. due to no pv is created for etcd pvc.

Version-Release number of the following components:
openshift-ansible-3.6.144-1.git.0.50e12bf.el7.noarch
ansible-2.2.3.0-1.el7.noarch


How reproducible:
Always

Steps to Reproduce:
1. deploy a normal cluster
2. enable service catalog in inventory host file
openshift_hosted_etcd_storage_kind=nfs
openshift_hosted_etcd_storage_nfs_directory=/opt/osev3-etcd
openshift_hosted_etcd_storage_volume_name=etcd-vol3
openshift_hosted_etcd_storage_access_modes=['ReadWriteOnce']
openshift_hosted_etcd_storage_volume_size=10G
openshift_hosted_etcd_storage_host=10.8.183.187
openshift_hosted_etcd_storage_labels="{'storage': 'etcd'}"

openshift_enable_service_catalog=true
openshift_service_catalog_image_version=latest
openshift_service_catalog_image_prefix=openshift3/ose-
ansible_service_broker_image_prefix=openshift3/
ansible_service_broker_registry_url="registry.access.stage.redhat.com"

3. deploy service catalog
# ansible-playbook -i inventory_file ./playbooks/byo/openshift-cluster/service-catalog.yml

4. playbook is finished successfully, go to openshift-ansible-service-broker project to check pod status.


Actual results:
# oc get po
NAME                    READY     STATUS              RESTARTS   AGE
asb-281511082-q8s97     0/1       CrashLoopBackOff    14         54m
etcd-1487946270-46j9s   0/1       pending   

# oc describe po etcd-1487946270-46j9s
<--snip-->
Events:
  FirstSeen	LastSeen	Count	From						SubObjectPath		Type		Reason			Message
  ---------	--------	-----	----						-------------		--------	------			-------
  1h		1h		53	default-scheduler							Warning		FailedScheduling	SchedulerPredicates failed due to PersistentVolumeClaim is not bound: "etcd", which is unexpected. (repeated 2 times)
  1h		1h		1	default-scheduler							Warning		FailedScheduling	no nodes available to schedule pods
  1h		42m		124	default-scheduler							Warning		FailedScheduling	SchedulerPredicates failed due to PersistentVolumeClaim is not bound: "etcd", which is unexpected. (repeated 2 times)


# oc get pvc
NAME      STATUS    VOLUME    CAPACITY   ACCESSMODES   STORAGECLASS   AGE
etcd      Pending                                                     46m

# oc get pv
NAME               CAPACITY   ACCESSMODES   RECLAIMPOLICY   STATUS    CLAIM                                   STORAGECLASS   REASON    AGE
regpv-volume       17G        RWX           Retain          Bound     default/regpv-claim                                              1h

No etcd pv is created.

Expected results:
etcd pv is created created, and all pods are running well.


Additional info:
manually create etcd pv, pods are running well.
Comment 2 Scott Dodson 2017-07-20 10:47:35 EDT

*** This bug has been marked as a duplicate of bug 1467574 ***
Comment 3 Johnny Liu 2017-07-20 11:13:19 EDT
I do not think this bug is a dup of bug 1467574, the root cause of this bug is the following:
when running the separate service catalog playbook on an existing cluster, openshift_persistent_volumes is never called, so no pv is created. If running playbooks/byo/config.yml, no such issue.

While in BZ#1467574, it is running playbooks/byo/config.yml, but did not set openshift_hosted_etcd_storage to an existing nfs dir, want installer help create the nfs dir automatically. It is tracking installer does not create new nfs export dir on NFS server for etcd pv, the root cause is openshift_storage_nfs never take openshift_hosted_etcd_storage into consideration.
Comment 4 Scott Dodson 2017-07-20 11:22:07 EDT
ok, depends on it then, we'll work on this post 3.6 GA
Comment 5 Scott Dodson 2017-08-14 11:45:44 EDT
Need to add the persistent volumes playbooks from hosted into the service catalog playbook. Not going to get this for 3.6.1 but will be a 3.7 blocker.
Comment 11 ewolinet 2017-10-09 16:50:46 EDT
Dropping priority/severity and marking it as a RFE (but keeping it under installer).

At this time a PV can be created and then the service_catalog.yml playbook can be run which would allow the etcd PVC to bind.

Note You need to log in before you can comment on or make changes to this bug.