Bug 1719362

Summary: Upgrade fails from 3.11.98 to 3.11.104 on task "Wait for /apis/metrics.k8s.io/v1beta1 when registered"
Product: OpenShift Container Platform Reporter: sperezto
Component: Cluster Version OperatorAssignee: Joseph Callen <jcallen>
Status: CLOSED ERRATA QA Contact: Weihua Meng <wmeng>
Severity: high Docs Contact:
Priority: high    
Version: 3.11.0CC: aos-bugs, dmoessne, jcallen, jokerman, mmccomas, szustkowski, vlaad, wmeng
Target Milestone: ---Keywords: Regression
Target Release: 3.11.z   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Cause: When openshift facts was recently modified the ipv4 dictionary item no longer existed Consequence: MTU was set incorrectly Fix: Remove the conditional with ipv4 Result: MTU set correctly.
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-07-23 19:56:23 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Comment 5 Weihua Meng 2019-06-15 02:41:25 UTC
meet same error when I upgrade from v3.10.149 to v3.11

openshift-ansible-3.11.119-1.git.0.c9a8ebf.el7

Failure summary:

  1. Hosts:    wmengahha310149-master-etcd-zone1-1.0614-ewv.qe.rhcloud.com
     Play:     Pre master upgrade - Upgrade all storage
     Task:     Wait for /apis/metrics.k8s.io/v1beta1 when registered
     Message:  non-zero return code

fatal: [wmengahha310149-master-etcd-zone1-1.0614-ewv.qe.rhcloud.com]: FAILED! => {
    "attempts": 30, 
    "changed": true, 
    "cmd": [
        "/usr/local/bin/oc", 
        "--config=/etc/origin/master/admin.kubeconfig", 
        "get", 
        "--raw", 
        "/apis/metrics.k8s.io/v1beta1"
    ], 
    "delta": "0:00:00.194761", 
    "end": "2019-06-14 11:06:16.282878", 
    "invocation": {
        "module_args": {
            "_raw_params": "/usr/local/bin/oc --config=/etc/origin/master/admin.kubeconfig get --raw /apis/metrics.k8s.io/v1beta1", 
            "_uses_shell": false, 
            "argv": null, 
            "chdir": null, 
            "creates": null, 
            "executable": null, 
            "removes": null, 
            "stdin": null, 
            "warn": true
        }
    }, 
    "msg": "non-zero return code", 
    "rc": 1, 
    "start": "2019-06-14 11:06:16.088117", 
    "stderr": "Error from server (ServiceUnavailable): the server is currently unable to handle the request", 
    "stderr_lines": [
        "Error from server (ServiceUnavailable): the server is currently unable to handle the request"
    ], 
    "stdout": "", 
    "stdout_lines": []
}


[root@wmengahha310149-master-etcd-zone1-1 ~]# oc get --raw /apis/metrics.k8s.io/v1beta1
Error from server (ServiceUnavailable): the server is currently unable to handle the request


[root@wmengahha310149-master-etcd-zone1-1 ~]# oc get apiservices
NAME                                   AGE
v1.                                    23h
v1.apps                                23h
v1.apps.openshift.io                   23h
v1.authentication.k8s.io               23h
v1.authorization.k8s.io                23h
v1.authorization.openshift.io          23h
v1.autoscaling                         23h
v1.batch                               23h
v1.build.openshift.io                  23h
v1.image.openshift.io                  23h
v1.network.openshift.io                23h
v1.networking.k8s.io                   23h
v1.oauth.openshift.io                  23h
v1.project.openshift.io                23h
v1.quota.openshift.io                  23h
v1.rbac.authorization.k8s.io           23h
v1.route.openshift.io                  23h
v1.security.openshift.io               23h
v1.storage.k8s.io                      23h
v1.template.openshift.io               23h
v1.user.openshift.io                   23h
v1alpha1.automationbroker.io           23h
v1beta1.admissionregistration.k8s.io   23h
v1beta1.apiextensions.k8s.io           23h
v1beta1.apps                           23h
v1beta1.authentication.k8s.io          23h
v1beta1.authorization.k8s.io           23h
v1beta1.batch                          23h
v1beta1.certificates.k8s.io            23h
v1beta1.events.k8s.io                  23h
v1beta1.extensions                     23h
v1beta1.metrics.k8s.io                 15h
v1beta1.policy                         23h
v1beta1.rbac.authorization.k8s.io      23h
v1beta1.servicecatalog.k8s.io          23h
v1beta1.storage.k8s.io                 23h
v1beta2.apps                           23h
v2beta1.autoscaling                    23h

Comment 8 Joseph Callen 2019-06-17 22:16:04 UTC
*** Bug 1713267 has been marked as a duplicate of this bug. ***

Comment 9 Joseph Callen 2019-06-17 22:16:38 UTC
*** Bug 1720982 has been marked as a duplicate of this bug. ***

Comment 22 Weihua Meng 2019-07-12 14:50:51 UTC
Verified.
openshift-ansible-3.11.129-1.git.0.11838de.el7
Upgrade success.

Comment 24 errata-xmlrpc 2019-07-23 19:56:23 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:1753