Bug 2089137

Summary: "ceph-external-cluster-details-exporter.py" ( 4.11 ) incompatible with python 2.x
Product: [Red Hat Storage] Red Hat OpenShift Data Foundation Reporter: Vijay Avuthu <vavuthu>
Component: rookAssignee: Parth Arora <paarora>
Status: CLOSED WONTFIX QA Contact: Shivam Durgbuns <sdurgbun>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 4.11CC: amohan, bkunal, clacroix, ebenahar, kramdoss, mbukatov, muagarwa, ocs-bugs, odf-bz-bot, paarora, pbalogh, rcyriac, sdurgbun, srai, tnielsen
Target Milestone: ---Flags: paarora: needinfo-
kramdoss: needinfo+
kramdoss: needinfo+
kramdoss: needinfo+
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
RHEL 7.x cluster has a dependency to run the external script with python2, so make the external script compatible so it can run with python2.
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-04-28 07:14:21 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Vijay Avuthu 2022-05-23 06:07:13 UTC
Description of problem (please be detailed as possible and provide log
snippests):

"ceph-external-cluster-details-exporter.py" incompatible with python 2.x

Version of all relevant components (if applicable):

ocs-registry:4.11.0-75


Does this issue impact your ability to continue to work with the product
(please explain in detail what is the user impact)?

Yes

Is there any workaround available to the best of your knowledge?
No

Rate from 1 - 5 the complexity of the scenario you performed that caused this
bug (1 - very simple, 5 - very complex)?
1

Can this issue reproducible?
Yes

Can this issue reproduce from the UI?
Not tried

If this is a regression, please provide more details to justify this:
Yes

Steps to Reproduce:
1. install odf ( external mode ) using ocs-ci
2. verify exporter script ran succesfully
3.


Actual results:

ceph-external-cluster-details-exporter.py should run without any errors


Expected results:

2022-05-21 21:23:28  15:53:28 - MainThread - ocs_ci.deployment.helpers.external_cluster_helpers - ERROR - Failed to run /tmp/external-cluster-details-exporter-54lmdp2j.py with parameters --rbd-data-pool-name rbd --rgw-endpoint 10.70.39.7:8080. Error: Traceback (most recent call last):
2022-05-21 21:23:28    File "/tmp/external-cluster-details-exporter-54lmdp2j.py", line 18, in <module>
2022-05-21 21:23:28      from inspect import signature
2022-05-21 21:23:28  ImportError: cannot import name signature


Additional info:

job: https://ocs4-jenkins-csb-odf-qe.apps.ocp-c1.prod.psi.redhat.com/job/qe-deploy-ocs-cluster-prod/4273//console

must gather: http://magna002.ceph.redhat.com/ocsci-jenkins/openshift-clusters/j-029vu1ce33-t4a/j-029vu1ce33-t4a_20220521T152333/logs/failed_testcase_ocs_logs_1653146889/test_deployment_ocs_logs/ocs_must_gather/

Comment 2 Subham Rai 2022-05-23 07:50:59 UTC
```
from inspect import signature
```
signature is not present in python 2.x  we can use ```from funcsigs import signature``` for python 2.x

one question, since python 2.x is EOL, till when we are planning to support python 2.x?

Comment 3 Martin Bukatovic 2022-05-23 13:04:18 UTC
(In reply to Subham Rai from comment #2)
> one question, since python 2.x is EOL, till when we are planning to support
> python 2.x?

The documentation states that one should use python3, with an exception of rhel7:

> Use python instead of python3 if the Red Hat Ceph Storage 4.x cluster
> is deployed on Red Hat Enterprise Linux 7.x (RHEL 7.x) cluster.

Comment 4 Subham Rai 2022-05-30 10:37:57 UTC
Also, will QE need help in testing since we don't have the required setup. Thanks

Comment 5 Subham Rai 2022-05-30 10:39:19 UTC
(In reply to Subham Rai from comment #4)
> Also, will need QE help in testing since we don't have the required setup.
> Thanks

Comment 8 Mudit Agarwal 2022-06-07 09:12:54 UTC
Please backport the fix to 4.11

Comment 9 Subham Rai 2022-06-07 09:29:42 UTC
(In reply to Mudit Agarwal from comment #8)
> Please backport the fix to 4.11

added

Comment 11 Coady LaCroix 2022-08-22 19:00:20 UTC
In verifying this bug I'm seeing another ImportError:

2022-08-22 11:25:06  18:25:06 - MainThread - ocs_ci.utility.connection - INFO  - Executing cmd: python /tmp/external-cluster-details-exporter-81d5q9qq.py --rbd-data-pool-name rbd --rgw-endpoint 10.70.39.7:8080 on 10.70.39.4
2022-08-22 11:25:07  18:25:07 - MainThread - ocs_ci.deployment.helpers.external_cluster_helpers - ERROR  - Failed to run /tmp/external-cluster-details-exporter-81d5q9qq.py with parameters --rbd-data-pool-name rbd --rgw-endpoint 10.70.39.7:8080. Error: Traceback (most recent call last):
2022-08-22 11:25:07    File "/tmp/external-cluster-details-exporter-81d5q9qq.py", line 29, in <module>
2022-08-22 11:25:07      import urllib.parse
2022-08-22 11:25:07  ImportError: No module named parse

This is from line 29 and the original error is line 18 so it looks like the original error has been resolved however we still can't execute the exporter script due to this.


Version: quay.io/rhceph-dev/ocs-registry:4.11.0-137
Job: https://ocs4-jenkins-csb-odf-qe.apps.ocp-c1.prod.psi.redhat.com/job/qe-deploy-ocs-cluster-prod/5154/console

Comment 12 Elad 2022-08-23 08:28:28 UTC
Moving back to ASSIGNED, based on Coady's input from the above comment as well as an offline discussion I had with Subham

Comment 13 Mudit Agarwal 2022-08-23 08:36:31 UTC
Not a 4.11.0 blocker, moving it out

Comment 19 Subham Rai 2022-09-15 08:26:22 UTC
moving to Modified PR merged downstream yesterday.

Comment 23 Coady LaCroix 2022-12-20 17:33:08 UTC
Any info on the above error?

Comment 24 Subham Rai 2022-12-28 07:40:09 UTC
Parth could look at the error. Thanks!

Comment 28 Parth Arora 2023-01-03 16:06:43 UTC
`execErr.stderr.decode("utf-8")` would not be present for subprocess in python2 so we need to change it accordingly,
I see it was changed from exec to execErr earlier but seems not working still https://github.com/rook/rook/pull/8623

Will raise a PR for it, or arun if you any immediate response to it you can proceed.

Comment 29 arun kumar mohan 2023-01-05 14:52:36 UTC
Hi Parth, nothing much to add. Please go ahead with your change, will review the PR.
Thanks,
Arun

Comment 30 Subham Rai 2023-01-11 08:32:37 UTC
removing my need info, @paarora feel free to open a pr. Thanks

Comment 31 Vijay Avuthu 2023-01-24 06:30:26 UTC
Update:
========

Failed with the same error which was mentioned in comment #22

ODF operator: ocs-registry:4.12.0-167


2023-01-23 22:34:25  17:04:24 - MainThread - ocs_ci.utility.connection - INFO  - Executing cmd: python /tmp/external-cluster-details-exporter-zaa_ifvb.py --rbd-data-pool-name rbd --rgw-endpoint 10.0.210.136:8080 on 10.0.210.160
2023-01-23 22:34:26  17:04:26 - MainThread - ocs_ci.deployment.helpers.external_cluster_helpers - ERROR  - Failed to run /tmp/external-cluster-details-exporter-zaa_ifvb.py with parameters --rbd-data-pool-name rbd --rgw-endpoint 10.0.210.136:8080. Error: Traceback (most recent call last):
2023-01-23 22:34:26    File "/tmp/external-cluster-details-exporter-zaa_ifvb.py", line 1682, in <module>
2023-01-23 22:34:26      rjObj.main()
2023-01-23 22:34:26    File "/tmp/external-cluster-details-exporter-zaa_ifvb.py", line 1662, in main
2023-01-23 22:34:26      generated_output = self.gen_json_out()
2023-01-23 22:34:26    File "/tmp/external-cluster-details-exporter-zaa_ifvb.py", line 1387, in gen_json_out
2023-01-23 22:34:26      self._gen_output_map()
2023-01-23 22:34:26    File "/tmp/external-cluster-details-exporter-zaa_ifvb.py", line 1370, in _gen_output_map
2023-01-23 22:34:26      ) = self.create_rgw_admin_ops_user()
2023-01-23 22:34:26    File "/tmp/external-cluster-details-exporter-zaa_ifvb.py", line 1155, in create_rgw_admin_ops_user
2023-01-23 22:34:26      in execErr.stderr.decode("utf-8")
2023-01-23 22:34:26  AttributeError: 'CalledProcessError' object has no attribute 'stderr'

job link: https://url.corp.redhat.com/6455dab

must gather: https://url.corp.redhat.com/f30c6af

Comment 32 Parth Arora 2023-01-24 08:04:43 UTC
Fixed by PR: https://github.com/red-hat-storage/rook/pull/445

Comment 37 Parth Arora 2023-01-24 08:15:16 UTC
(In reply to Martin Bukatovic from comment #3)
> (In reply to Subham Rai from comment #2)
> > one question, since python 2.x is EOL, till when we are planning to support
> > python 2.x?
> 
> The documentation states that one should use python3, with an exception of
> rhel7:
> 
> > Use python instead of python3 if the Red Hat Ceph Storage 4.x cluster
> > is deployed on Red Hat Enterprise Linux 7.x (RHEL 7.x) cluster.

So only cluster 4.11 and below would be only affected, as the 4.12 cluster would be using RHEL 8.x.
Then no need to fix & test it on the 4.12 cluster, right?

Comment 38 Travis Nielsen 2023-01-24 15:25:34 UTC
Per previous comment, moving back to consider for 4.11 and older

Comment 49 Parth Arora 2023-02-28 12:11:40 UTC
Travis any plans when are going to take this in https://github.com/red-hat-storage/rook/pull/446

And Bipin probably it is good to backport to 4.10 too as it also supports (RHEL 7.x)

Comment 65 Subham Rai 2023-04-20 06:22:10 UTC
no info required, since Parth already create bp pr.