Bug 1476195 - Deploy metrics via ansible was failed due to clusterrole "hawkular-metrics" was not found
Deploy metrics via ansible was failed due to clusterrole "hawkular-metrics" w...
Status: CLOSED ERRATA
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer (Show other bugs)
3.6.0
Unspecified Unspecified
high Severity high
: ---
: 3.7.0
Assigned To: ewolinet
Junqi Zhao
: Regression, TestBlocker
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2017-07-28 04:59 EDT by Junqi Zhao
Modified: 2017-11-28 17:06 EST (History)
4 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: The necessary role for a rolebinding in openshift_metrics was missing due to being processed out of order in the role. Consequence: The rolebinding creation would fail and the role would fail to install Fix: Updated so role was created right away so that rolebinding would correctly create. Result: The rolebinding is able to be created during installation every time.
Story Points: ---
Clone Of:
Environment:
Last Closed: 2017-11-28 17:06:30 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
ansilbe running log (833.01 KB, text/plain)
2017-07-28 04:59 EDT, Junqi Zhao
no flags Details
inventory file (443 bytes, text/plain)
2017-07-28 05:04 EDT, Junqi Zhao
no flags Details

  None (edit)
Description Junqi Zhao 2017-07-28 04:59:38 EDT
Created attachment 1305845 [details]
ansilbe running log

Description of problem:
Deploy metrics 3.6 via ansible failed at error: role.authorization.openshift.io "hawkular-metrics" not found.
We have one similar CLI defect: https://bugzilla.redhat.com/show_bug.cgi?id=1476166
 
fatal: [host-8-174-222.host.centralci.eng.rdu2.redhat.com]: FAILED! => {
    "changed": false, 
    "cmd": [
        "oc", 
        "--config=/tmp/openshift-metrics-ansible-UCPk2o/admin.kubeconfig", 
        "apply", 
        "-f", 
        "/tmp/openshift-metrics-ansible-UCPk2o/templates/hawkular-cluster-rolebinding.yaml", 
        "-n", 
        "openshift-infra"
    ], 
    "delta": "0:00:00.200168", 
    "end": "2017-07-28 04:27:52.029236", 
    "failed": true, 
    "failed_when_result": true, 
    "invocation": {
        "module_args": {
            "_raw_params": "oc --config=/tmp/openshift-metrics-ansible-UCPk2o/admin.kubeconfig apply -f /tmp/openshift-metrics-ansible-UCPk2o/templates/hawkular-cluster-rolebinding.yaml -n openshift-infra", 
            "_uses_shell": false, 
            "chdir": null, 
            "creates": null, 
            "executable": null, 
            "removes": null, 
            "warn": true
        }, 
        "module_name": "command"
    }, 
    "rc": 1, 
    "start": "2017-07-28 04:27:51.829068", 
    "warnings": []
}

STDERR:

Error from server (NotFound): error when creating "/tmp/openshift-metrics-ansible-UCPk2o/templates/hawkular-cluster-rolebinding.yaml": role.authorization.openshift.io "hawkular-metrics" not found
    to retry, use: --limit @/usr/share/ansible/openshift-ansible/playbooks/byo/openshift-cluster/openshift-metrics.retry


Version-Release number of selected component (if applicable):
# rpm -qa | grep openshift-ansible*
openshift-ansible-filter-plugins-3.6.172.0.1-1.git.0.5bd2286.el7.noarch
openshift-ansible-3.6.172.0.1-1.git.0.5bd2286.el7.noarch
openshift-ansible-lookup-plugins-3.6.172.0.1-1.git.0.5bd2286.el7.noarch
openshift-ansible-callback-plugins-3.6.172.0.1-1.git.0.5bd2286.el7.noarch
openshift-ansible-playbooks-3.6.172.0.1-1.git.0.5bd2286.el7.noarch
openshift-ansible-docs-3.6.172.0.1-1.git.0.5bd2286.el7.noarch
openshift-ansible-roles-3.6.172.0.1-1.git.0.5bd2286.el7.noarch

# openshift version
openshift v3.6.172.0.0
kubernetes v1.6.1+5115d708d7
etcd 3.2.1


How reproducible:
Always

Steps to Reproduce:
1. Deploy metrics 3.6 via ansible
2. 
3.

Actual results:
Deployment was failed due to clusterrole "hawkular-metrics" was not found

Expected results:
Deployment should be successfully.

Additional info:
Attached ansible running log and inventory file.

Description of problem:

Version-Release number of the following components:
rpm -q openshift-ansible
rpm -q ansible
ansible --version

How reproducible:

Steps to Reproduce:
1.
2.
3.

Actual results:
Please include the entire output from the last TASK line through the end of output if an error is generated

Expected results:

Additional info:
Please attach logs from ansible-playbook with the -vvv flag
Comment 1 Junqi Zhao 2017-07-28 05:04 EDT
Created attachment 1305847 [details]
inventory file
Comment 2 Junqi Zhao 2017-07-28 05:08:17 EDT
Metrics cases are all blocked.
Comment 3 ewolinet 2017-07-28 11:14:30 EDT
It looks like this is failing from the underlying oc apply command when creating the role binding. It seems the role template isn't being `oc apply`'d before this rolebinding one
Comment 5 Junqi Zhao 2017-07-30 21:23:41 EDT
Tested with 3.6.172.0.3-1 openshift-ansible playbooks, metrics can be deployed successfully now, please change the state to ON_QA, so we can close it. 
# rpm -qa | grep openshift-ansible
openshift-ansible-filter-plugins-3.6.172.0.3-1.git.0.8753f3b.el7.noarch
openshift-ansible-docs-3.6.172.0.3-1.git.0.8753f3b.el7.noarch
openshift-ansible-callback-plugins-3.6.172.0.3-1.git.0.8753f3b.el7.noarch
openshift-ansible-lookup-plugins-3.6.172.0.3-1.git.0.8753f3b.el7.noarch
openshift-ansible-playbooks-3.6.172.0.3-1.git.0.8753f3b.el7.noarch
openshift-ansible-3.6.172.0.3-1.git.0.8753f3b.el7.noarch
openshift-ansible-roles-3.6.172.0.3-1.git.0.8753f3b.el7.noarch

# oc get po
NAME                         READY     STATUS    RESTARTS   AGE
hawkular-cassandra-1-q14ls   1/1       Running   0          9m
hawkular-metrics-lj3xg       1/1       Running   0          9m
heapster-jsl3m               1/1       Running   0          9m
Comment 7 Junqi Zhao 2017-07-31 00:33:09 EDT
Set it to VERIFIED based on Comment 5
Comment 11 errata-xmlrpc 2017-11-28 17:06:30 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:3188

Note You need to log in before you can comment on or make changes to this bug.