Note: This bug is displayed in read-only format because
the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
How reproducible:
fence_gce --action=reboot --zone=<zone-name> --project=<Google Project> -n <Nodename> -v
or manually execute the python code from fence_gce
Steps to Reproduce:
1) Add the following to your /root/.bash_profile
export GOOGLE_APPLICATION_CREDENTIALS="/root/account.json"
2) Close terminal and start a new session.
3) Verify that variable exists
# echo $GOOGLE_APPLICATION_CREDENTIALS
4) Manually test fencing to verify that fencing works with no errors.
# fence_gce --action=reboot --zone=<zone-details> --project=<Google Project> -n <Nodename> -v
5) If fencing works with no errors, then verify that it works within the pacemaker
by fencing a cluster node which should reboot the node.
# pcs stonith fence <node_name>
also here is the python snippet from fence_gce which can simulate the failure situation
import googleapiclient.discovery
credentials = None
conn = googleapiclient.discovery.build('compute', 'v1', credentials=credentials) ====> This should fail
instance = conn.instances().get(project="<Enter Google project details>",zone="<Enter you zone details>",instance="<Enter node to fence>").execute()
print(json.dumps(instance.get('status'),indent=4, sort_keys=True))
Here is the python snippet from fence_gce which works
import os
import googleapiclient.discovery
os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = "<Filename /location for the file for account.json>"
credentials = None
conn = googleapiclient.discovery.build('compute', 'v1', credentials=credentials)
conn.instances().stop(project="<Enter Google project details>",zone="<Enter you zone details>",instance="<Enter node to fence>").execute()
instance = conn.instances().get(project="<Enter Google project details>",zone="<Enter you zone details>",instance="<Enter node to fence>").execute()
instance = conn.instances().get(project="<Enter Google project details>",zone="<Enter you zone details>",instance="<Enter node to fence>").execute()
print(json.dumps(instance.get('status'),indent=4, sort_keys=True)) ==>This should result in terminated for successfull fence
Actual results:
Insufficient privilege, 403 error while accessing googleapis for node resets
Expected results:
It should reset the node by successfully authenticating using the GOOGLE_APPLICATION_CREDENTIALS or get successful authorization against the Oauth , There is no good documentation on this
or while configuring the stonith there is no way that we could mention from where to source the variable GOOGLE_APPLICATION_CREDENTIALS for authentication except setting them manually in bash profile or hardcoding the environment variable in fence_gce
Additional info:
Reference internal redhat case 02326309
Comment 12Oyvind Albrigtsen
2020-04-21 13:36:05 UTC
Moving to RHEL8, as oauth2client uses a new format for load_pem_private_key(), which doesnt work the with the python-cryptography version available in RHEL7.
Planning to add a parameter to specify the keyfile.
Comment 16Oyvind Albrigtsen
2021-06-09 14:22:21 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory (fence-agents bug fix and enhancement update), and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.
https://access.redhat.com/errata/RHBA-2021:4148
How reproducible: fence_gce --action=reboot --zone=<zone-name> --project=<Google Project> -n <Nodename> -v or manually execute the python code from fence_gce Steps to Reproduce: 1) Add the following to your /root/.bash_profile export GOOGLE_APPLICATION_CREDENTIALS="/root/account.json" 2) Close terminal and start a new session. 3) Verify that variable exists # echo $GOOGLE_APPLICATION_CREDENTIALS 4) Manually test fencing to verify that fencing works with no errors. # fence_gce --action=reboot --zone=<zone-details> --project=<Google Project> -n <Nodename> -v 5) If fencing works with no errors, then verify that it works within the pacemaker by fencing a cluster node which should reboot the node. # pcs stonith fence <node_name> also here is the python snippet from fence_gce which can simulate the failure situation import googleapiclient.discovery credentials = None conn = googleapiclient.discovery.build('compute', 'v1', credentials=credentials) ====> This should fail instance = conn.instances().get(project="<Enter Google project details>",zone="<Enter you zone details>",instance="<Enter node to fence>").execute() print(json.dumps(instance.get('status'),indent=4, sort_keys=True)) Here is the python snippet from fence_gce which works import os import googleapiclient.discovery os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = "<Filename /location for the file for account.json>" credentials = None conn = googleapiclient.discovery.build('compute', 'v1', credentials=credentials) conn.instances().stop(project="<Enter Google project details>",zone="<Enter you zone details>",instance="<Enter node to fence>").execute() instance = conn.instances().get(project="<Enter Google project details>",zone="<Enter you zone details>",instance="<Enter node to fence>").execute() instance = conn.instances().get(project="<Enter Google project details>",zone="<Enter you zone details>",instance="<Enter node to fence>").execute() print(json.dumps(instance.get('status'),indent=4, sort_keys=True)) ==>This should result in terminated for successfull fence Actual results: Insufficient privilege, 403 error while accessing googleapis for node resets Expected results: It should reset the node by successfully authenticating using the GOOGLE_APPLICATION_CREDENTIALS or get successful authorization against the Oauth , There is no good documentation on this or while configuring the stonith there is no way that we could mention from where to source the variable GOOGLE_APPLICATION_CREDENTIALS for authentication except setting them manually in bash profile or hardcoding the environment variable in fence_gce Additional info: Reference internal redhat case 02326309