Bug 1356530 - oc rollback meets error "cannot create deploymentconfigs/rollback in project" (compatibility issue with oc v3.3 VS openshift v3.2)
Summary: oc rollback meets error "cannot create deploymentconfigs/rollback in project"...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: openshift-controller-manager
Version: 3.3.0
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: ---
Assignee: Michail Kargakis
QA Contact: zhou ying
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-07-14 10:27 UTC by Xingxing Xia
Modified: 2017-03-08 18:26 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
oc rollback now uses a new endpoint for rolling back deployments. We had to gracefully fallback to the old endpoint for old servers.
Clone Of:
Environment:
Last Closed: 2016-09-27 09:40:11 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2016:1933 normal SHIPPED_LIVE Red Hat OpenShift Container Platform 3.3 Release Advisory 2016-09-27 13:24:36 UTC

Description Xingxing Xia 2016-07-14 10:27:23 UTC
Description of problem:
oc rollback meets error "cannot create deploymentconfigs/rollback in project" when testing oc v3.3 VS openshift v3.2. Other matrix of oc VS openshift don't have this issue:
oc   VS  openshift
3.3      3.3
3.2      3.2
3.2      3.3


Version-Release number of selected component (if applicable):
Server:
$ openshift version
openshift v3.2.1.7
kubernetes v1.2.0-36-g4a3f9c5
etcd 2.2.5

Client:
$ oc version
oc v3.3.0.5
kubernetes v1.3.0+57fb9ac
features: Basic-Auth GSSAPI Kerberos SPNEGO


How reproducible:
Always

Steps to Reproduce:
1. Create a dc
$ oc run hello --image openshift/hello-openshift
2. After hello-1 deployed, deploy again
$ oc deploy hello --latest
3. After hello-2 deployed, rollback
1> $ oc rollback hello-1
Check with loglevel:
2> $ oc rollback hello-1 --loglevel 6

Actual results:
3.
1> Error:
Error from server: User "bob" cannot create deploymentconfigs/rollback in project "bob-proj"

2> logs:
... GET https://<master>:8443/oapi/v1/namespaces/bob-proj/deploymentconfigs/hello 200 OK in 256 milliseconds
... POST https://<master>:8443/oapi/v1/namespaces/bob-proj/deploymentconfigs/hello/rollback 403 Forbidden in 323 milliseconds
... server response object: [{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {},
  "status": "Failure",
  "message": "User \"bob\" cannot create deploymentconfigs/rollback in project \"bob-proj\"",
  "reason": "Forbidden",
  "details": {
    "name": "hello",
    "kind": "deploymentconfigs/rollback"
  },
  "code": 403
}]

Expected results:
3. Should rollback successfully and output:
#3 rolled back to hello-1

Additional info:
Logs of other matrix shown by --loglevel 6:
oc  VS openshift
3.2      3.3
... POST https://<master>:8443/oapi/v1/namespaces/bob-proj/deploymentconfigrollbacks 201 Created in 11 milliseconds
... PUT https://<master>:8443/oapi/v1/namespaces/bob-proj/deploymentconfigs/hello 200 OK in 174 milliseconds

3.3      3.3
... POST https://<master>:8443/oapi/v1/namespaces/bob-proj/deploymentconfigs/hello/rollback 201 Created in 31 milliseconds
... PUT https://<master>:8443/oapi/v1/namespaces/bob-proj/deploymentconfigs/hello 200 OK in 8 milliseconds

Comment 1 openshift-github-bot 2016-07-19 12:06:54 UTC
Commit pushed to master at https://github.com/openshift/origin

https://github.com/openshift/origin/commit/4b3ec3a393976764a6caceab49e4832ebd21b5b0
Bug 1356530: handle 403 in oc rollback

Comment 2 Xingxing Xia 2016-07-20 06:28:47 UTC
First test the bug with oc compiled from latest Origin code, but the bug still exists:
Compile:
$ cd /data/src/github.com/openshift
$ mkdir origin-xxia
$ git clone origin-bare origin-xxia
$ cd origin-xxia
$ git pull https://github.com/openshift/origin
$ hack/build-cross.sh
$ _output/local/bin/linux/amd64/oc version
oc v1.3.0-alpha.2+2cbabda-dirty
kubernetes v1.3.0+57fb9ac
features: Basic-Auth GSSAPI Kerberos SPNEGO
$ git log --pretty="%h %cd - %s" 2cbabda | grep '#9889'
38a6fdb Tue Jul 19 08:06:51 2016 -0400 - Merge pull request #9889 from kargakis/bug-1356530

^ the compiled oc contains the fix.

Verify (against dev-preview-int (openshift v3.2.1.7)):
$ _output/local/bin/linux/amd64/oc rollback database-2
Error from server: User "xingxingxia" cannot create deploymentconfigs/rollback in project "xxia-proj"

Comment 3 Michail Kargakis 2016-07-20 08:37:02 UTC
Can you return the output of `oc rollback dc/hello --loglevel=8`? Here is mine (seems to work fine) : http://pastebin.com/B3s7MZtW

Comment 5 Xingxing Xia 2016-07-20 09:38:09 UTC
Please see the attachment. BTW, is your test against openshift v3.2?

Comment 6 Michail Kargakis 2016-07-20 09:46:34 UTC
> Please see the attachment. BTW, is your test against openshift v3.2?

Yes, see in the logs that I am getting a 403 when quering the new endpoint. Make sure you are running the latest version of oc that includes the fix. It should retry by quering the old endpoint when you get 403 from the new endpoint.

Comment 8 Xingxing Xia 2016-07-20 12:30:59 UTC
^^ Private comment not trigger email :)

Comment 9 Michail Kargakis 2016-07-20 12:35:53 UTC
Sorry but I don't have any time to test this in your env. Make sure you are using the correct binary.

Comment 10 Xingxing Xia 2016-07-22 05:06:34 UTC
Tested against dev-int-preview (openshift v3.2.1.10) with oc version:
$ oc version
oc v3.3.0.8
kubernetes v1.3.0+57fb9ac
features: Basic-Auth GSSAPI Kerberos SPNEGO

Now result is:
$ oc rollback database-3 --loglevel 6
...
I0722 13:01:28.616025     983 round_trippers.go:318] POST https://<dev-preview-int>:443/oapi/v1/namespaces/xxia-proj/deploymentconfigs/database/rollback 403 Forbidden in 262 milliseconds
I0722 13:01:28.923729     983 round_trippers.go:318] POST https://<dev-preview-int>:443/oapi/v1/namespaces/xxia-proj/deploymentconfigrollbacks 201 Created in 307 milliseconds
...
#5 rolled back to database-3

After "403 Forbidden", it retried and succeeded. The issue is fixed.

Comment 12 errata-xmlrpc 2016-09-27 09:40:11 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:1933


Note You need to log in before you can comment on or make changes to this bug.