Bug 1452816

Summary: [DOCS] Permissions for Cloud Provider account
Product: OpenShift Container Platform Reporter: Eric Rich <erich>
Component: DocumentationAssignee: Gaurav Nelson <gnelson>
Status: CLOSED CURRENTRELEASE QA Contact: Chao Yang <chaoyang>
Severity: medium Docs Contact: Vikram Goyal <vigoyal>
Priority: medium    
Version: 3.5.0CC: aos-bugs, asolanas, chaoyang, eminguez, erich, gnelson, gwest, jhou, jokerman, mmccomas, pschiffe, rcook, snalawad, stwalter, tatanaka, tdrake, tparsons, tsedovic, vwalek, wehe, xtian, ykheddac
Target Milestone: ---Keywords: NeedsTestCase
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-03-07 00:57:28 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Eric Rich 2017-05-19 17:24:13 UTC
Document URL: 

https://docs.openshift.com/container-platform/3.5/install_config/configuring_aws.html
https://docs.openshift.com/container-platform/3.5/install_config/configuring_openstack.html
https://docs.openshift.com/container-platform/3.5/install_config/configuring_gce.html
https://docs.openshift.com/container-platform/3.5/install_config/configuring_azure.html

Section Number and Name: Multiple

Describe the issue: 

We don't document what permissions cloud-provider accounts need, in order to function or get capabilities on the stack working. 

Suggestions for improvement: 

Provide section / documentation on the permission the cloud integration components need so that limited access can be provided. 

Additional information:

Comment 1 Eric Rich 2017-07-05 19:50:55 UTC
https://github.com/kubernetes/kubernetes/blob/release-1.5/cluster/aws/templates/iam/kubernetes-master-policy.json may provide the docs answer for this with AWS.

Comment 2 Steven Walter 2017-08-11 13:39:24 UTC
My customer brings up a question about the actual required permissions. That template specifies:

      "Effect": "Allow",
      "Action": ["ec2:*"],
"Resource": ["*"]


Their concern is that if they need to set aws as the cloud provider but all they need is to dynamically assign Elastic Block Storage for persistent volumes, why should they need ec2:* and not just a smaller list of permissions?

Comment 5 Ryan Cook 2017-09-21 20:05:08 UTC
Masters need the following privileges 

                     "ec2:DescribeVolume*",
                     "ec2:CreateVolume",
                     "ec2:CreateTags",
                     "ec2:DescribeInstance*",
                     "ec2:AttachVolume",
                     "ec2:DetachVolume",
                     "ec2:DeleteVolume",
                     "ec2:DescribeSubnets",
                     "ec2:CreateSecurityGroup",
                     "ec2:DescribeSecurityGroups",
                     "elasticloadbalancing:DescribeTags",
                     "elasticloadbalancing:CreateLoadBalancerListeners",
                     "ec2:DescribeRouteTables",
                     "elasticloadbalancing:ConfigureHealthCheck",
                     "ec2:AuthorizeSecurityGroupIngress",
                     "elasticloadbalancing:DeleteLoadBalancerListeners",
                     "elasticloadbalancing:RegisterInstancesWithLoadBalancer",
                     "elasticloadbalancing:DescribeLoadBalancers",
                     "elasticloadbalancing:CreateLoadBalancer",
                     "elasticloadbalancing:DeleteLoadBalancer",
                     "elasticloadbalancing:ModifyLoadBalancerAttributes",
                     "elasticloadbalancing:DescribeLoadBalancerAttributes"


Nodes need the following privileges
                     "ec2:DescribeInstance*"



Masters and nodes all must have the tag of KubernetesCluster: value
All subnets must have the tag KubernetesCluster: value
One security group preferably the one linked to the nodes must have the tag KubernetesCluster: value

Note: Do not tag all security groups with the KubernetesCluster: value or the elb will not be able to create

Comment 6 Takayoshi Tanaka 2017-10-04 07:53:59 UTC
As of Azure, I wrote this KCS.

OpenShift Container Platform on Microsoft Azure with Azure Disk failed to create a pod with Persistent Volume
https://access.redhat.com/solutions/3003111

In this document, I require the "Contributor" role of the resource group. As far as I read the code, attach/detach VHD to the VM is only required, but I write a most privileged permission because it's not specified.

Comment 7 Gaurav Nelson 2017-10-09 02:07:38 UTC
I have created a PR to address this issue. Please check the PR at https://github.com/openshift/openshift-docs/pull/5333

Please add comments if I am missing anything, or if it require more details.

Comment 10 Gaurav Nelson 2017-10-19 01:06:56 UTC
*** Bug 1426755 has been marked as a duplicate of this bug. ***

Comment 13 Vikram Goyal 2017-12-28 09:47:17 UTC
*** Bug 1316357 has been marked as a duplicate of this bug. ***

Comment 14 Vikram Goyal 2017-12-30 23:30:31 UTC
*** Bug 1301669 has been marked as a duplicate of this bug. ***

Comment 16 Gaurav Nelson 2018-01-30 23:37:26 UTC
Thank you Ryan, Glenn, Peter, and Eduardo. 

In Google doc we have:
 - GCP, Azure, and AWS looks good.
 - Waiting on Tomas Sedovic to look at openStack permissions.

Comment 18 Gaurav Nelson 2018-02-15 01:46:55 UTC
Thanks for proving the info Tomas.

I have created PR with the agreed changes, PTAL https://github.com/openshift/openshift-docs/pull/7797

Comment 20 Eric Rich 2018-02-19 13:43:48 UTC
(In reply to Gaurav Nelson from comment #19)
> @Eric can you please review

Generally, I don't like that each document has a different flow (set of sections). In short, if we're going to list permissions and pre-requested, then it should be listed on all the cloud sections. 

In addition to this at the end of ever section we say: 

> "On each node host, restart the {product-title} service."

This should have an example command! 
In addition to that, we should have this process QE'ed! 

> AWS:
> https://github.com/gaurav-nelson/openshift-docs/blob/
> b098e51e10b320339898dc57d76a12fd1d2768f7/install_config/configuring_aws.
> adoc#configuring-aws-permissions

What is the difference between IAM and API credentals, it looks like were just removing options? Its not clear what we are trying to show as an example here. 

> 
> Azure:
> https://github.com/gaurav-nelson/openshift-docs/blob/
> b098e51e10b320339898dc57d76a12fd1d2768f7/install_config/configuring_azure.
> adoc#configuring-azure-permissions
> 
> GCE:
> https://github.com/gaurav-nelson/openshift-docs/blob/
> b098e51e10b320339898dc57d76a12fd1d2768f7/install_config/configuring_gce.
> adoc#configuring-gcp-permissions
> 
> OpenStack:
> https://github.com/gaurav-nelson/openshift-docs/blob/
> b098e51e10b320339898dc57d76a12fd1d2768f7/install_config/
> configuring_openstack.adoc#configuring-openstack-permissions

Comment 21 Gaurav Nelson 2018-02-21 02:10:17 UTC
Thank you @Eric

> Generally, I don't like that each document has a different flow (set of
> sections). In short, if we're going to list permissions and pre-requested,
> then it should be listed on all the cloud sections. 

I have updated Azure docs by removing the prerequisites section. All the things described in that section already appear in Azure configuration file section.

> In addition to this at the end of ever section we say: 
> 
> > "On each node host, restart the {product-title} service."
> 
> This should have an example command! 
> In addition to that, we should have this process QE'ed! 

Fixes for this are in PR https://github.com/openshift/openshift-docs/pull/7804

> > AWS:
> What is the difference between IAM and API credentals, it looks like were
> just removing options? Its not clear what we are trying to show as an
> example here. 

IAM bits because of https://bugzilla.redhat.com/show_bug.cgi?id=1462823 to answer the question "If you dont want to set the variables below, how do you use IAM role to authenticate your hosts with AWS?"

Comment 24 Gaurav Nelson 2018-02-26 00:20:39 UTC
Is there anything missing for this PR?

Comment 26 openshift-github-bot 2018-03-01 05:23:11 UTC
Commit pushed to master at https://github.com/openshift/openshift-docs

https://github.com/openshift/openshift-docs/commit/a3e3b7abc4c01561a98500c71102a8164160740e
Merge pull request #7797 from gaurav-nelson/bug1452816-fixes

Bug#1452816 - fixes - Added required roles and permissions for all cloudproviders

Comment 27 Wenqi He 2018-03-05 08:16:17 UTC
This is verified from Azure permission part.

Comment 28 Chao Yang 2018-03-05 09:25:19 UTC
I think it should add like "ec2:CreateRoute" and "ec2:ModifyVolume" according to https://github.com/kubernetes/kubernetes/blob/f4472b1a92877ed4b1576e7e44496b0de7a8efe2/pkg/cloudprovider/providers/aws/aws.go

@rcook what's your suggestion?

Comment 30 Gaurav Nelson 2018-03-06 10:53:56 UTC
Thank you for looking at this @chaoyang

Did the tests failed for AWS only? In that case I would like to close this bug and create a new bug just for AWS. Because from docs perspective the detailed work was carried out for all cloud providers.