Bug 1871712 - [AWS] Destroy cluster failed due to 'not-owned' role policy attached, error: DeleteConflict: Cannot delete entity, must detach all policies first.
Summary: [AWS] Destroy cluster failed due to 'not-owned' role policy attached, error: ...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer
Version: 4.6
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 4.6.0
Assignee: Russell Teague
QA Contact: Yunfei Jiang
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-08-24 05:40 UTC by Yunfei Jiang
Modified: 2020-10-27 16:31 UTC (History)
1 user (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-10-27 16:31:17 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift installer pull 4126 0 None closed Bug 1871712: pkg/destroy/aws: Detach attached policies during destroy 2020-10-27 07:21:45 UTC
Red Hat Product Errata RHBA-2020:4196 0 None None None 2020-10-27 16:31:20 UTC

Description Yunfei Jiang 2020-08-24 05:40:40 UTC
create a role and attach it to existing cluster. 
destroy cluster got errors:
level=debug msg="DeleteConflict: Cannot delete entity, must detach all policies first.\n\tstatus code: 409, request id: 7e88b46a-88a2-4086-b4d5-86d2ac742f35" arn="arn:aws:iam::301721915996:role/yunjiang-210641-2l66l-master-role"
level=debug msg="DeleteConflict: Cannot delete entity, must detach all policies first.\n\tstatus code: 409, request id: c6cd8f12-e5ac-4a36-a9de-f1ceadcb01cd" arn="arn:aws:iam::301721915996:role/yunjiang-210641-2l66l-worker-role"

Version-Release number of the following components: 
4.6.0-0.nightly-2020-08-18-165040
 
How reproducible: 
Always 
 
Steps to Reproduce: 
1. create a cluster: openshift-install create cluster --dir=demo
2. create policy
    aws iam --region us-east-2 create-policy --policy-name yunjiang-993197-725b2-testpolicy-1594977805-2 --policy-document '{"Version":"2012-10-17","Statement":[ {"Effect":"Allow","Action":["ec2:DescribeInstances","ec2:DescribeRegions"],"Resource":"*"}]}'
3. Attach policy to roles
    aws iam --region us-east-2 attach-role-policy --role-name yunjiang-993197-725b2-master-role --policy-arn arn:aws:iam::301721915996:policy/yunjiang-993197-725b2-testpolicy-1594977805-2
    aws iam --region us-east-2 attach-role-policy --role-name yunjiang-993197-725b2-worker-role --policy-arn arn:aws:iam::301721915996:policy/yunjiang-993197-725b2-testpolicy-1594977805-2
4. destroy cluster [failed]:
    level=debug msg="DeleteConflict: Cannot delete entity, must detach all policies first.\n\tstatus code: 409, request id: 7e88b46a-88a2-4086-b4d5-86d2ac742f35" arn="arn:aws:iam::301721915996:role/yunjiang-210641-2l66l-master-role"
    level=debug msg="DeleteConflict: Cannot delete entity, must detach all policies first.\n\tstatus code: 409, request id: c6cd8f12-e5ac-4a36-a9de-f1ceadcb01cd" arn="arn:aws:iam::301721915996:role/yunjiang-210641-2l66l-worker-role"


Actual results: 
destroy cluster failed

Expected results:
destroy cluster successfully

Additional info:

Comment 1 Russell Teague 2020-08-25 17:31:36 UTC
Reproduced:

DEBUG DeleteConflict: Cannot delete entity, must detach all policies first.
	status code: 409, request id: d2fd297b-22c1-4a8a-8495-84c4d5554793  arn="arn:aws:iam::269733383066:role/rteague-4a-9g567-master-role"
DEBUG DeleteConflict: Cannot delete entity, must detach all policies first.
	status code: 409, request id: 5fd81de6-f473-495c-ac1c-4a78229ddb99  arn="arn:aws:iam::269733383066:role/rteague-4a-9g567-worker-role"


User created policies are 'managed policies' instead of 'inline policies' as created by the installer.  Possibly something to look into when considering a fix.

Comment 3 Yunfei Jiang 2020-09-11 08:44:47 UTC
verified. PASS.
version: 4.6.0-0.nightly-2020-09-10-195619

>> create

# create role policy
09-11 16:13:14 INFO:root:creating policy: yunjiang-387607-4tfp2-testpolicy-1599811994-2
09-11 16:13:14 INFO:root:Command: aws iam --region us-east-2 create-policy --policy-name yunjiang-387607-4tfp2-testpolicy-1599811994-2 --policy-document '{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Action":["ec2:DescribeInstances","ec2:DescribeRegions"],"Resource":"*"}]}'

# attach role policy for master
09-11 16:13:16 INFO:root:attach policy
09-11 16:13:16 INFO:root:Command: aws iam --region us-east-2 attach-role-policy --role-name yunjiang-387607-4tfp2-master-role --policy-arn arn:aws:iam::301721915996:policy/yunjiang-387607-4tfp2-testpolicy-1599811994-2
09-11 16:13:16 INFO:root:Command: aws iam --region us-east-2 list-attached-role-policies --role-name yunjiang-387607-4tfp2-master-role | jq -r .AttachedPolicies[].PolicyArn

# attach role policy for worker
09-11 16:13:17 INFO:root:attach policy
09-11 16:13:17 INFO:root:Command: aws iam --region us-east-2 attach-role-policy --role-name yunjiang-387607-4tfp2-worker-role --policy-arn arn:aws:iam::301721915996:policy/yunjiang-387607-4tfp2-testpolicy-1599811994-2
09-11 16:13:18 INFO:root:Command: aws iam --region us-east-2 list-attached-role-policies --role-name yunjiang-387607-4tfp2-worker-role | jq -r .AttachedPolicies[].PolicyArn

>> destroy

level=info msg=Deleted arn="arn:aws:iam::301721915996:role/yunjiang-387607-4tfp2-master-role" id=yunjiang-387607-4tfp2-master-role name=yunjiang-387607-4tfp2-master-role policy=yunjiang-387607-4tfp2-master-policy
level=info msg=Detached arn="arn:aws:iam::301721915996:role/yunjiang-387607-4tfp2-master-role" id=yunjiang-387607-4tfp2-master-role name=yunjiang-387607-4tfp2-master-role policy=yunjiang-387607-4tfp2-testpolicy-1599811994-2
level=info msg=Deleted arn="arn:aws:iam::301721915996:role/yunjiang-387607-4tfp2-master-role" id=yunjiang-387607-4tfp2-master-role name=yunjiang-387607-4tfp2-master-role


level=info msg=Deleted arn="arn:aws:iam::301721915996:role/yunjiang-387607-4tfp2-worker-role" id=yunjiang-387607-4tfp2-worker-role name=yunjiang-387607-4tfp2-worker-role policy=yunjiang-387607-4tfp2-worker-policy
level=info msg=Detached arn="arn:aws:iam::301721915996:role/yunjiang-387607-4tfp2-worker-role" id=yunjiang-387607-4tfp2-worker-role name=yunjiang-387607-4tfp2-worker-role policy=yunjiang-387607-4tfp2-testpolicy-1599811994-2
level=info msg=Deleted arn="arn:aws:iam::301721915996:role/yunjiang-387607-4tfp2-worker-role" id=yunjiang-387607-4tfp2-worker-role name=yunjiang-387607-4tfp2-worker-role

Comment 5 errata-xmlrpc 2020-10-27 16:31:17 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.6 GA Images), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:4196


Note You need to log in before you can comment on or make changes to this bug.