Bug 1515703

Fabiano, I believe we disabled the use of cronjobs in free-int, which is why this error is being shown (when otherwise cronjobs would be shown as part of "all").

If the above assumption is correct, then this is not a bug, but rather the command failing to list cronjobs due to the current policy on the cluster.

If you try listing all (or even just cronjobs on a local cluster, this error does not happen).

Tagging as NOTABUG, but feel free to reopen if I have missed something.

Better to adjust and improve the UX for ONLINE, because:
For customer, such trailing printing is messy, no matter the project has app or not
$ oc new-project xxia21-proj
$ oc get all
Error from server (Forbidden): User "xxia-21" cannot list cronjobs.batch in the namespace "xxia21-proj": User "xxia-21" cannot list cronjobs.batch in project "xxia21-proj" (get cronjobs.batch)
$ echo $?
1
$ oc delete all --all
Error from server (Forbidden): User "xxia-21" cannot list cronjobs.batch in the namespace "xxia21-proj": User "xxia-21" cannot list cronjobs.batch in project "xxia21-proj" (get cronjobs.batch)
$ echo $?
1

And for QE, "1" will make existing automated scenarios fail which have used oc get/delete all and expected success of cmd execution status

Juan, Fabiano `oc get all` should never fail on access rights. It should silently ignore these errors (eventually appropriate message instead of empty list) should appear. I agree that this should be fixed, we can't expect every cluster have exactly same `all` resources but the command should just work.

Thanks Maciej,
Will go ahead and open a PR that hides permission errors from "oc get"

Upstream PR: https://github.com/kubernetes/kubernetes/pull/56801

When fixed upstream it will come with an upcoming rebase.

> How about just a warning next to the resources you can't see that clearly states you don't have access? 

Doesn't this already happen with the forbidden error that is show? Users are notified of the resource that could not be listed, and why

Yeah, we've talked about it with David and I also double checked on my own. This is how it should work.

Thanks Maciej, closing this as notabug

Saw above comments.
But considered UX and gathered use experience from other guys, the multiple error lines are not friendly. Some guys give feedback that they were confused when they saw the messages at first glance, they thought "I didn't create resource 'cronjob', why it shows me 'cannot list cronjobs.batch'".

As a solution suggestion in terms of UX, do you think it is feasible to fix as belows:
When going though each resource type included in alias `all`, first check whether it is configured "forbidden" by master, if yes, ignore this resource type, remember it, and continue to process next resource type. Finally, after all resource types in alias `all` are processed, rather than multiple error lines, print a single warning line that states the remembered types are configured-forbidden by master, and make command return '0' if no other error happens.

Thanks for long time discussion and patient explanation. OK, let it as it is

Could reproduce this bug only on starter clusters.
[1] api.free-int.openshift.com
[2] api.free-stg.openshift.com

In an empty project,try 'oc get all', show below error:
$ oc get all
Error from server (Forbidden): daemonsets.apps is forbidden: User "xiuwang-7" cannot list daemonsets.apps in the namespace "xiu1": User "xiuwang-7" cannot list daemonsets.apps in project "xiu1"
Error from server (Forbidden): replicasets.apps is forbidden: User "xiuwang-7" cannot list replicasets.apps in the namespace "xiu1": User "xiuwang-7" cannot list replicasets.apps in project "xiu1"
Error from server (Forbidden): cronjobs.batch is forbidden: User "xiuwang-7" cannot list cronjobs.batch in the namespace "xiu1": User "xiuwang-7" cannot list cronjobs.batch in project "xiu1"

(In reply to XiuJuan Wang from comment #19)
> Could reproduce this bug only on starter clusters.
> [1] api.free-int.openshift.com
> [2] api.free-stg.openshift.com
> 
> In an empty project,try 'oc get all', show below error:
> $ oc get all
> Error from server (Forbidden): daemonsets.apps is forbidden: User
> "xiuwang-7" cannot list daemonsets.apps in the namespace "xiu1": User
> "xiuwang-7" cannot list daemonsets.apps in project "xiu1"
> Error from server (Forbidden): replicasets.apps is forbidden: User
> "xiuwang-7" cannot list replicasets.apps in the namespace "xiu1": User
> "xiuwang-7" cannot list replicasets.apps in project "xiu1"
> Error from server (Forbidden): cronjobs.batch is forbidden: User "xiuwang-7"
> cannot list cronjobs.batch in the namespace "xiu1": User "xiuwang-7" cannot
> list cronjobs.batch in project "xiu1"

It fail for an empty project too. It fail for every where :(

@justin  still looks like permissions are messed up.  Why doesn't a user with access have read rights?

The procedure dakini defined for removing scheduledjobs/cronjobs/custom-host from starter is no longer accurate for 3.9. These roles are now split across multiple reources due to role aggregation. 

Assigning this back to me since a failure to reconcile is presently impacting CD deployments. I'm writing a script that will reconcile roles after an upgrade, but it is not complex enough preserve listing permissions. This may need to wait until 3.10.

Mo provided an idea to address this with role aggregation. I've run the following on free-int to eliminate the error:

cat <<EOF | oc apply -f -
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: system:openshift:cicd:aggregate-to-all-cronjobs-read-and-del
  labels:
    rbac.authorization.k8s.io/aggregate-to-admin: "true"
    rbac.authorization.k8s.io/aggregate-to-edit: "true"
rules:
- apiGroups:
  - batch
  resources:
  - cronjobs
  verbs:
  - get
  - list
  - watch
  - delete
  - deletecollection
EOF

cat <<EOF | oc apply -f -
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: system:openshift:cicd:aggregate-to-all-cronjobs-read
  labels:
    rbac.authorization.k8s.io/aggregate-to-view: "true"
rules:
- apiGroups:
  - batch
  resources:
  - cronjobs
  verbs:
  - get
  - list
  - watch
EOF

This bug has fixed in free-int (openshift v3.9.1 kubernetes v1.9.1+a0ce1bc657).

However, there is still a little issue in free-stg(openshift v3.9.1 kubernetes v1.9.1+a0ce1bc657).
In an empty project, try 'oc get all' and "oc delete all --all"
#oc get all       
Error from server (Forbidden): cronjobs.batch is forbidden: User "xiuwang-7" cannot list cronjobs.batch in the namespace "xiu1": User "xiuwang-7" cannot list cronjobs.batch in project "xiu1"

Sorry，my mistake，I should assign this bug back

Based on previous discussion I don't see what else needs fixing here. Moving back to QA.

https://api.free-stg.openshift.com:443
openshift v3.11.69
kubernetes v1.11.0+d4cacc0

Can't reproduce this issue in free-stg with current version.