Bug 1413848 - Router deployed by ansible installer does not use openshift-router.kubeconfig
Summary: Router deployed by ansible installer does not use openshift-router.kubeconfig
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: RFE
Version: 3.3.0
Hardware: x86_64
OS: Linux
Target Milestone: ---
: ---
Assignee: Marc Curry
QA Contact: Xiaoli Tian
Depends On:
TreeView+ depends on / blocked
Reported: 2017-01-17 06:40 UTC by Kenjiro Nakayama
Modified: 2020-02-14 18:28 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Last Closed: 2018-03-12 13:54:36 UTC
Target Upstream Version:

Attachments (Terms of Use)

System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 1282822 0 unspecified CLOSED [DOCS] Document how to create routes for specific labels/namespaces/projects 2021-02-22 00:41:40 UTC
Red Hat Knowledge Base (Solution) 2867371 0 None None None 2017-01-17 07:13:15 UTC

Internal Links: 1282822

Description Kenjiro Nakayama 2017-01-17 06:40:10 UTC
Description of problem:
  Ansible installer deploy router pods on region=infra by default. However, it seems that it doesn't deploy pod with /etc/origin/master/openshift-router.kubeconfig.
  bz#1282822 can provde this. The error is not produced without /etc/origin/master/openshift-router.kubeconfig.
    - a) router deployed by installer
    - b) router deployed by "oadm router" (without --credentials)
    - c) router deployed by "oadm router --credentials='/etc/origin/master/openshift-router.kubeconfig'"
  c) can produce error [1], but a) and b) doesn't prouduce the error.

NOTE: So, c) needs "oadm policy add-cluster-role-to-user cluster-reader system:openshift-router"

Version-Release number of selected component (if applicable):

How reproducible:
  Deploy router with ansible innstaller

  Steps to Reproduce:
  1. Run ansible installer and deploy router
  2. Configure "Router Shards" *without* "oadm policy add-cluster-role-to-user cluster-reader system:openshift-router"

Actual results:
  It should not work, since "system:openshift-router" doesn't have the permission to list namespace by default.

Expected results:
  - Router shards failed, but it will work after adding "oadm policy add-cluster-role-to-user cluster-reader system:openshift-router"

[1] E0116 09:55:49.535424       1 controller.go:75] unable to find namespaces for router: User "system:openshift-router" cannot list all namespaces in the cluster

[2] Router Shards

Comment 1 Scott Dodson 2017-01-17 14:27:26 UTC

I see that you wrote much of the documentation related to router shards. What's the best way for us to be installing the router such that it works for sharding and ideally works in normal scenarios?

Should we always grant the cluster-reader role to openshift-router?

Comment 2 Kenjiro Nakayama 2017-01-20 02:12:00 UTC
Could you please let us know what procedure is correct and how you are planning to fix this issue? The customer needs to standardize the steps.

Comment 3 Phil Cameron 2017-01-20 19:14:12 UTC
I am not up on the policy stuff. It appears that we need to have cluster-reader but I am not sure it should be set up by default. Maybe Clayton or Jordan could comment.

I am not sure sharding vs regular is all that different. Usually the routers are set up in default namespace and the routes are set up in some project namespace.

Comment 4 Scott Dodson 2017-01-20 19:48:01 UTC

Thoughts on granting the router SA cluster reader?

Comment 5 Jordan Liggitt 2017-01-20 19:50:07 UTC
we should not be deploying with --credentials any more

the router command grants required permissions to the router's service account

Comment 6 Scott Dodson 2017-01-20 20:43:40 UTC
We don't for 3.2 and later.

Comment 7 Kenjiro Nakayama 2017-01-21 02:40:49 UTC
> we should not be deploying with --credentials any more

So, all docs have to be updated..

Also, there are some mysterious remains... 
- Why you didn't remove --credentials options from "oadm router" command? 
- Is "/etc/origin/master/openshift-router.kubeconfig" necessary to deploy during the installation?

Comment 8 Kenjiro Nakayama 2017-01-21 03:05:15 UTC
> - Is "/etc/origin/master/openshift-router.kubeconfig" necessary to deploy during the installation?

I meant that openshift-router.kubeconfig should not be deployed. I think it is not used by any services.

Comment 9 Phil Cameron 2017-01-23 14:45:56 UTC
--credentials is there for customers that want to provide their own certificates.

Comment 10 Kenjiro Nakayama 2017-02-04 11:44:09 UTC
You mean that if we use own certs, we need to specify openshift-router.kubeconfig with --credentials? In that case, cluster-reader would be necessary to add.
So complicated... It is not possible to simplify this?

Comment 12 Ben Bennett 2017-02-08 19:51:42 UTC
In order to use NAMESPACE_LABELS you need to follow the instructions at https://docs.openshift.com/enterprise/3.2/install_config/install/deploy_router.html#creating-the-router-service-account to give the router sufficient privilege to read the namespace objects.

You then need to set the NAMESPACE_LABELS environment variable on the router dc to tell it what labels to read.

There are many other environment variables that it probably makes sense to allow people to set as well, so I think this request is larger in scope.  So we should probably work out how we want to handle setting arbitrary environment variables on the router (or whether we want to expose ansible settings for all of them) before rushing into changing this.

And if the NAMESPACE_LABELS are set, should we grant the permission automtically, or require them to specify that separately?

Comment 13 Ben Bennett 2017-02-13 18:36:23 UTC
We need to work out the strategy that we want to take for all of the router features.  I've added card https://trello.com/c/FUwFD0P3 to track this feature.

Comment 14 Eric Rich 2018-03-12 13:54:36 UTC
This bug has been identified as a dated (created more than 3 months ago) bug. 
This bug has been triaged (has a trello card linked to it), or reviewed by Engineering/PM and has been put into the product backlog, 
however this bug has not been slated for a currently planned release (3.9, 3.10 or 3.11), which cover our releases for the rest of the calendar year. 

As a result of this bugs age, state on the current roadmap and PM Score (being below 70), this bug is being Closed - Differed, 
as it is currently not part of the products immediate priorities.

Please see: https://docs.google.com/document/d/1zdqF4rB3ea8GmVIZ7qWCVYUaQ7-EexUrQEF0MTwdDkw/edit for more details.

Note You need to log in before you can comment on or make changes to this bug.