Bug 1371464 - [GCE]deployer status is error when DYNAMICALLY_PROVISION_STORAGE is 'true'
Summary: [GCE]deployer status is error when DYNAMICALLY_PROVISION_STORAGE is 'true'
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Hawkular
Version: 3.3.0
Hardware: Unspecified
OS: Unspecified
medium
low
Target Milestone: ---
: ---
Assignee: Matt Wringe
QA Contact: Peng Li
URL:
Whiteboard:
: 1425932 (view as bug list)
Depends On: 1392343
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-08-30 09:19 UTC by Peng Li
Modified: 2017-03-08 18:43 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: The scripts used to check if a deployment was successful did not properly handle the situation with dynamically provisioned storage and would cause an error message to be displayed after the metric components were deployed. Consequence: The deployer would exit in an error status and display an error message in the logs. The metrics components would still deploy and function properly, it did not affect any functionality. Fix: The scripts used to check if the deployment was correctly deployed have been updated to support dynamically provisioned volumes when using on GCE Result: New deployments to GCE with DYNAMICALLY_PROVISIONED_STORAGE set to true will no longer have this error message.
Clone Of:
Environment:
Last Closed: 2017-01-18 12:52:52 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
logs (21.21 KB, text/plain)
2016-08-30 09:19 UTC, Peng Li
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2017:0066 0 normal SHIPPED_LIVE Red Hat OpenShift Container Platform 3.4 RPM Release Advisory 2017-01-18 17:23:26 UTC

Description Peng Li 2016-08-30 09:19:03 UTC
Created attachment 1195798 [details]
logs

deployer status error when DYNAMICALLY_PROVISION_STORAGE is 'true'
 OCP 3.3.0 / metrics
 M
 M
Description of problem:
deploy metrics stack with parameter 'USE_PERSISTENT_STORAGE=true' and 'DYNAMICALLY_PROVISION_STORAGE=true' in a cloud-provider enabled env, wait until the deployment is finished. heapster,cassandra and metrics pods work well, and could get metrics from web ui, but the deployer pod's status is error.

Version-Release number of selected component (if applicable):
3.3.0

How reproducible:
always

Steps to Reproduce:
1.
oc project ${PROJECT}
oc create serviceaccount metrics-deployer

oadm policy add-cluster-role-to-user cluster-reader system:serviceaccount:openshift-infra:heapster

oc policy add-role-to-user edit system:serviceaccount:openshift-infra:metrics-deployer

oc secrets new metrics-deployer nothing=/dev/null

oc new-app metrics-deployer-template -p \
IMAGE_PREFIX=registry.ops.openshift.com/openshift3/,\
IMAGE_VERSION=3.3.0,\
MASTER_URL=${MASTER_URL},\
HAWKULAR_METRICS_HOSTNAME=hawkular-metrics.${SUBDOMAIN},\
MODE=deploy,\
USE_PERSISTENT_STORAGE=true,\
DYNAMICALLY_PROVISION_STORAGE=true,\
CASSANDRA_NODES=1,\
CASSANDRA_PV_SIZE=10,\
USER_WRITE_ACCESS=false

2. 
[penli@dhcp-137-185 33]$ oc get po
NAME                         READY     STATUS    RESTARTS   AGE
hawkular-cassandra-1-bi7nk   1/1       Running   0          1m
hawkular-metrics-t5bxo       1/1       Running   0          1m
heapster-qz34g               1/1       Running   0          1m
metrics-deployer-4wheq       0/1       Error     0          1m

3.
[penli@dhcp-137-185 33]$ oc logs metrics-deployer-4wheq
(...)
======== ERROR =========
validate_deployment_artifacts: 
scripts/validate.sh: line 277: line[3]: unbound variable
========================
--- validate_deployed_project ---

VALIDATION FAILED
(...)

Actual results:
deployer pod's status is error.

Expected results:
deployer pod's status is finished.

Additional info:
Full log is attached.

Comment 1 Xia Zhao 2016-08-31 02:21:58 UTC
Issue reproduced when USE_PERSISTENT_STORAGE=true and DYNAMICALLY_PROVISION_STORAGE=false.

Comment 3 Matt Wringe 2016-09-02 15:22:17 UTC
I have ran this multiple times on AWS without encountering this problem.

Can you please describe your setup a bit? Is this reproducible?

Comment 4 Peng Li 2016-09-05 08:48:21 UTC
Hi Matt, we ran the test on Google Cloud with Persistent Disk, cloud provier enabled, since Sep 5th is holiday in US, if you need, I can set up one Google Cloud for you on Sep 6th.

Comment 10 Matt Wringe 2016-10-31 13:58:44 UTC
Marking this as low as it doesn't actually affect the pods from being deployed, only our scripts which try and validate that the install happened correctly.

Comment 12 Peng Li 2016-11-04 07:13:50 UTC
There is some issue when use dynamic pvc on GCE, node will crash, this should be OCP issue not Metrics issue, I'll verify it once it's ready.

Comment 15 errata-xmlrpc 2017-01-18 12:52:52 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:0066

Comment 16 Matt Wringe 2017-02-22 19:58:13 UTC
*** Bug 1425932 has been marked as a duplicate of this bug. ***


Note You need to log in before you can comment on or make changes to this bug.