Bug 1631517 - [free-int] storage migration failures: the server does not allow this method on the requested resource [NEEDINFO]
Summary: [free-int] storage migration failures: the server does not allow this method ...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Master
Version: 3.11.0
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: 4.1.0
Assignee: Mo
QA Contact: zhou ying
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-09-20 19:06 UTC by Justin Pierce
Modified: 2019-06-04 10:40 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-06-04 10:40:35 UTC
Target Upstream Version:
yinzhou: needinfo? (mkhan)


Attachments (Terms of Use)
loglevel 8 migrate with errors (5.74 MB, application/x-gzip)
2018-09-20 19:06 UTC, Justin Pierce
no flags Details


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2019:0758 None None None 2019-06-04 10:40:45 UTC

Description Justin Pierce 2018-09-20 19:06:11 UTC
Created attachment 1485266 [details]
loglevel 8 migrate with errors

Description of problem:
During storage oc adm migrate storage, the tool is repeatedly reporting messages like:

E0920 18:39:49.045977 error:     -n hasha-pro1 pods/nodejs-ex-2-kmzrv: the server does not allow this method on the requested resource

Version-Release number of selected component (if applicable):
v3.11.7

How reproducible:
100%

Additional info:
High log level output attached.

Comment 2 Mo 2018-09-21 19:26:04 UTC
> openshift_metrics_server_install: "{{ openshift_metrics_install_metrics | default(False) }}"

Removing this as a release-3.11 blocker since it only occurs if you opt-in to installing the metrics server.

Comment 3 openshift-github-bot 2018-09-21 23:52:58 UTC
Commit pushed to master at https://github.com/openshift/origin

https://github.com/openshift/origin/commit/318343c755f9e47679c634216e2f25c8da4e8be3
migrate: ignore resources that cannot be listed and updated

This change updates FindAllCanonicalResources to check the verbs
asserted by discovery for listing and updating capability.  This
prevents us from trying to update aggregated resources such as pod
metrics.

Bug 1631517

Signed-off-by: Monis Khan <mkhan@redhat.com>

Comment 4 Mo 2018-09-25 16:21:37 UTC
https://github.com/openshift/origin/pull/21099 should fix the terrible error messages as well.

Comment 5 openshift-github-bot 2018-09-25 17:42:39 UTC
Commit pushed to master at https://github.com/openshift/origin

https://github.com/openshift/origin/commit/08e03204f63ce1d0dc2f23b21a5f5f05d41fb266
Use group resource in migrate reporting

This change updates the reporting during migration to use the group
resource instead of just the resource.

Thus the confusing messages with "the server does not allow this
method on the requested resource" for pod metrics will go from:

-n hasha-pro1 pods/nodejs-ex-2-kmzrv

to:

-n hasha-pro1 pods.metrics.k8s.io/nodejs-ex-2-kmzrv

This makes it clear what resource is being referenced.

Bug 1631517

Signed-off-by: Monis Khan <mkhan@redhat.com>

Comment 7 Xingxing Xia 2018-09-27 02:20:59 UTC
Checked the latest build v3.11.16. It includes the 1st PR #21071's code but not the 2nd PR yet. A confusion: saw target release is 3.11.z but advisory/36292 attaches this bug. Pls either include the 2nd PR into 3.11 and update 3.11.z to 3.11.0 for verifying it in 3.11, or drop the bug from the advisory. Thanks

Comment 9 Mo 2018-09-27 13:14:18 UTC
(In reply to Xingxing Xia from comment #7)
> Checked the latest build v3.11.16. It includes the 1st PR #21071's code but
> not the 2nd PR yet. A confusion: saw target release is 3.11.z but
> advisory/36292 attaches this bug. Pls either include the 2nd PR into 3.11
> and update 3.11.z to 3.11.0 for verifying it in 3.11, or drop the bug from
> the advisory. Thanks

2nd PR is just to improve error messaging in the future.  It will not be included in 3.11 or 3.11.z.  The 1st PR is the actual fix for this issue.  It is not a blocker for 3.11, hence the 3.11.z.  I have removed it from the advisory.

Comment 10 Xingxing Xia 2018-09-29 08:42:22 UTC
Now that the 1st PR goes into 3.11, VERIFIED the 1st PR with below steps.
Given env is installed with:
openshift_ansible_vars:
  openshift_metrics_server_install: true
(or if the env is not installed with that, add these vars in inventory file, then run `ansible-playbook -i hosts /usr/share/ansible/openshift-ansible/playbooks/metrics-server/config.yml`)

# oc get apiservice | grep metrics
v1beta1.metrics.k8s.io                 2018-09-29T06:31:55Z

Use old oc 3.11.12 to run:
$ oc adm migrate storage --loglevel=8 |& tee migrate.log
$ grep "the server does not allow this" migrate.log
There are many error logs.
In another env, use latest oc 3.11.17 to run above, no such error logs.
So the 1st PR is VERIFIED.

Comment 12 zhou ying 2018-11-02 06:14:19 UTC
Checked with the latest 'openshift v3.11.36', the PR still not merged with ocp3.11.x.

Comment 14 zhou ying 2018-11-07 09:02:56 UTC
Confirmed with openshift v4.0.0-0.49.0, the issue has fixed: 
[root@qe-yinzhou-master-etcd-1 ~]# openshift version
openshift v4.0.0-0.49.0

[root@qe-yinzhou-master-etcd-1 ~]# oc adm migrate storage --loglevel=8 |& tee migrate.log
[root@qe-yinzhou-master-etcd-1 ~]# cat migrate.log |grep unchanged  |grep images
I1107 02:43:28.376186 unchanged: -n zhouyt imagestreams.image.openshift.io/ruby-25-centos7
I1107 02:43:28.376496 unchanged: -n zhouy imagestreams.image.openshift.io/mytest

Comment 19 errata-xmlrpc 2019-06-04 10:40:35 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:0758


Note You need to log in before you can comment on or make changes to this bug.