Bug 1738606 - Calls to GCP API hang forever when metadata access is blocked
Summary: Calls to GCP API hang forever when metadata access is blocked
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Cloud Credential Operator
Version: 4.2.0
Hardware: All
OS: All
medium
medium
Target Milestone: ---
: 4.2.0
Assignee: Cesar Wong
QA Contact: Oleg Nesterov
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-08-07 14:43 UTC by Cesar Wong
Modified: 2019-10-16 06:35 UTC (History)
1 user (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-10-16 06:35:15 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift cloud-credential-operator pull 107 0 None closed Bug 1738606 [gcp] set timeout on cloud API calls 2020-04-07 13:52:10 UTC
Red Hat Product Errata RHBA-2019:2922 0 None None None 2019-10-16 06:35:25 UTC

Description Cesar Wong 2019-08-07 14:43:32 UTC
Description of problem:
With access to metadata port 53 blocked, calls inside the cloud credential operator would hang a goroutine forever. 

Version-Release number of selected component (if applicable):
4.2

How reproducible:
Always

Steps to Reproduce:
1. In a control plane node running cloud credential operator, block access to metadata addresses, by using iptables rules:
  - oc debug node/[name-of-control-plane-node]
    # chroot /host
    # iptables -A OUTPUT  -d 169.254.169.254 -j REJECT
    # iptables -A FORWARD -d 169.254.169.254 -j REJECT
2. Watch logs of the cloud credential operator pod running on that node.

Actual results:
The logs stop at the cloud credential operator trying to access the gcp API

Expected results:
The cloud credential operator reports that there is an error connecting to gcp and keeps trying.


Additional info:

Comment 1 Joel Diaz 2019-08-23 14:55:07 UTC
fixed with https://github.com/openshift/cloud-credential-operator/pull/107

Comment 3 Oleg Nesterov 2019-08-30 14:04:44 UTC
Verified on 4.2.0-0.nightly-2019-08-28-235925

Comment 4 errata-xmlrpc 2019-10-16 06:35:15 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:2922


Note You need to log in before you can comment on or make changes to this bug.