Bug 1540851 - redeploy metrics failed to create PersistentVolumeClaim
Summary: redeploy metrics failed to create PersistentVolumeClaim
Keywords:
Status: CLOSED DUPLICATE of bug 1540729
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer
Version: 3.9.0
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: 3.9.0
Assignee: Scott Dodson
QA Contact: Johnny Liu
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-02-01 07:11 UTC by Anping Li
Modified: 2018-02-02 05:31 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-02-02 05:31:57 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Anping Li 2018-02-01 07:11:44 UTC
Description of problem:
When upgrade to v3.9 using playbooks, it failed to apply PersistentVolumeClaim.

Version-Release number of selected component (if applicable):
ose-ansible: v3.9.0-0.35.0.0

How reproducible:
always

Steps to Reproduce:
1. deploy metrics v3.7 

openshift_metrics_cassandra_pvc_size=10Gi
openshift_metrics_cassandra_storage_type=dynamic
openshift_metrics_image_version=v3.7
openshift_metrics_image_prefix=registry.example.com/openshift3/
openshift_metrics_install_metrics=true


2. redeploy metrics v3.9
openshift_metrics_cassandra_pvc_size=10Gi
openshift_metrics_cassandra_storage_type=dynamic
openshift_metrics_image_version=v3.7
openshift_metrics_image_prefix=registry.example.com/openshift3/
openshift_metrics_install_metrics=true


Actual results:
TASK [openshift_metrics : Checking generation of PersistentVolumeClaim metrics-cassandra-1] ***
task path: /usr/share/ansible/openshift-ansible/roles/openshift_metrics/tasks/oc_apply.yaml:2
ok: [ec2-54-165-61-64.compute-1.amazonaws.com] => {"changed": false, "cmd": ["oc", "--config=/tmp/openshift-metrics-ansible-GFEOMb/admin.kubeconfig", "get", "PersistentVolumeClaim", "metrics-cassandra-1", "-o", "jsonpath={.metadata.resourceVersion}", "-n", "openshift-infra"], "delta": "0:00:00.289410", "end": "2018-02-01 01:30:46.837731", "failed_when_result": false, "rc": 0, "start": "2018-02-01 01:30:46.548321", "stderr": "", "stderr_lines": [], "stdout": "340407", "stdout_lines": ["340407"]}

TASK [openshift_metrics : Applying /tmp/openshift-metrics-ansible-GFEOMb/templates/hawkular-cassandra-pvc1.yaml] ***
task path: /usr/share/ansible/openshift-ansible/roles/openshift_metrics/tasks/oc_apply.yaml:13
fatal: [ec2-54-165-61-64.compute-1.amazonaws.com]: FAILED! => {"changed": false, "cmd": ["oc", "--config=/tmp/openshift-metrics-ansible-GFEOMb/admin.kubeconfig", "apply", "-f", "/tmp/openshift-metrics-ansible-GFEOMb/templates/hawkular-cassandra-pvc1.yaml", "-n", "openshift-infra"], "delta": "0:00:00.484469", "end": "2018-02-01 01:30:47.975929", "failed_when_result": true, "msg": "non-zero return code", "rc": 1, "start": "2018-02-01 01:30:47.491460", "stderr": "The PersistentVolumeClaim \"metrics-cassandra-1\" is invalid: spec: Forbidden: field is immutable after creation", "stderr_lines": ["The PersistentVolumeClaim \"metrics-cassandra-1\" is invalid: spec: Forbidden: field is immutable after creation"], "stdout": "", "stdout_lines": []}

PLAY RECAP *********************************************************************
ec2-184-72-79-104.compute-1.amazonaws.com : ok=24   changed=0    unreachable=0    failed=0   
ec2-34-233-136-214.compute-1.amazonaws.com : ok=25   changed=0    unreachable=0    failed=0   
ec2-52-90-155-141.compute-1.amazonaws.com : ok=28   changed=0    unreachable=0    failed=0   
ec2-52-90-239-166.compute-1.amazonaws.com : ok=24   changed=0    unreachable=0    failed=0   
ec2-54-165-61-64.compute-1.amazonaws.com : ok=183  changed=26   unreachable=0    failed=1   
ec2-54-226-169-154.compute-1.amazonaws.com : ok=24   changed=0    unreachable=0    failed=0   
localhost                  : ok=11   changed=0    unreachable=0    failed=0   




# cat /tmp/openshift-metrics-ansible-GFEOMb/templates/hawkular-cassandra-pvc1.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: "metrics-cassandra-1"
  labels:
    metrics-infra: hawkular-cassandra
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi
  storageClassName: 

Expected results:
Apply pvc should succeed

Additional info:

Comment 1 Junqi Zhao 2018-02-01 13:46:11 UTC
@anli
This defect is duplicate of BZ # 1539382, it is caused by the same issue.
after redeploy metrics with openshift-ansible-3.9.0-0.34, it throws out same error with this defect.

TASK [openshift_metrics : Applying /tmp/openshift-metrics-ansible-JduLci/templates/hawkular-cassandra-pvc1.yaml] **************************************************************************************************
fatal: [ec2-54-158-112-3.compute-1.amazonaws.com]: FAILED! => {"changed": false, "cmd": ["oc", "--config=/tmp/openshift-metrics-ansible-JduLci/admin.kubeconfig", "apply", "-f", "/tmp/openshift-metrics-ansible-JduLci/templates/hawkular-cassandra-pvc1.yaml", "-n", "openshift-infra"], "delta": "0:00:00.761062", "end": "2018-02-01 08:35:56.344560", "failed_when_result": true, "msg": "non-zero return code", "rc": 1, "start": "2018-02-01 08:35:55.583498", "stderr": "The PersistentVolumeClaim \"metrics-cassandra-1\" is invalid: spec: Forbidden: field is immutable after creation", "stderr_lines": ["The PersistentVolumeClaim \"metrics-cassandra-1\" is invalid: spec: Forbidden: field is immutable after creation"], "stdout": "", "stdout_lines": []}
	to retry, use: --limit @/usr/share/ansible/openshift-ansible/playbooks/openshift-metrics/config.retry

Comment 2 Junqi Zhao 2018-02-01 13:58:05 UTC
(In reply to Junqi Zhao from comment #1)
> @anli
> This defect is duplicate of BZ # 1539382, it is caused by the same issue.
> after redeploy metrics with openshift-ansible-3.9.0-0.34, it throws out same
> error with this defect.
> 
I mean BZ 1540851 is caused by the fix of BZ # 1539382, although the default openshift_metrics_cassandra_pvc_prefix is metrics-cassandra now.

Or shall I set VERIFIED to BZ # 1539382, and mention there is another defect BZ 1540851 was found


# grep -r "openshift_metrics_cassandra_pvc_prefix"
roles/openshift_metrics/defaults/main.yaml:openshift_metrics_cassandra_pvc_prefix: metrics-cassandra

Comment 4 Anping Li 2018-02-02 05:31:57 UTC

*** This bug has been marked as a duplicate of bug 1540729 ***


Note You need to log in before you can comment on or make changes to this bug.