Bug 2080729

Summary: fence_apc, fence_ilo_moonshot, fence_lpar: missing "import logging"
Product: Red Hat Enterprise Linux 8 Reporter: Patrick Hibbs <hibbsncc1701>
Component: fence-agentsAssignee: Oyvind Albrigtsen <oalbrigt>
Status: CLOSED ERRATA QA Contact: cluster-qe <cluster-qe>
Severity: medium Docs Contact:
Priority: medium    
Version: CentOS StreamCC: bugs, cfeist, cluster-maint, emesika, mjuricek, mperina
Target Milestone: rcKeywords: ZStream
Target Release: ---Flags: pm-rhel: mirror+
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: fence-agents-4.2.1-95.el8 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 2086559 2086838 (view as bug list) Environment:
Last Closed: 2022-11-08 09:26:47 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Infra RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 2086559, 2086838    

Description Patrick Hibbs 2022-05-01 11:28:47 UTC
Description of problem:
My setup uses a APC master switch (Specificly a RackPDU 7800.) to fence it's hosts. The APC unit configured for telnet access. The apc fence type is used by oVirt to handle this fencing type.

When /usr/bin/fence_apc is invoked by the user, the command succeeds. When the engine invokes the command, we see that the it has appended an extra string, (in this case a " -6"), to the open command given to the telnet binary. The extra string results in failure to fence the host. Although an IP address is used below, the same result occurs if the engine is given a DNS hostname.

---snip---
May 01 07:02:05 azura.codenet /fence_apc[18208]: Timeout exceeded.
                                                 <fencing.fspawn object at 0x7f6ffd503048>
                                                 command: /usr/bin/telnet
                                                 args: [b'/usr/bin/telnet']
                                                 buffer (last 100 chars): 'net> open 192.168.9.50 -6\r\ntelnet: 192.168.9.50: Address family for hostname not supported\r\nt>
                                                 before (last 100 chars): 'net> open 192.168.9.50 -6\r\ntelnet: 192.168.9.50: Address family for hostname not supported\r\nt>
                                                 after: <class 'pexpect.exceptions.TIMEOUT'>
                                                 match: None
                                                 match_index: None
                                                 exitstatus: None
                                                 flag_eof: False
                                                 pid: 18210
                                                 child_fd: 6
                                                 closed: False
                                                 timeout: 30
                                                 delimiter: <class 'pexpect.exceptions.EOF'>
                                                 logfile: None
                                                 logfile_read: None
                                                 logfile_send: None
                                                 maxread: 2000
                                                 ignorecase: False
                                                 searchwindowsize: None
                                                 delaybeforesend: 0.05
                                                 delayafterclose: 0.1
                                                 delayafterterminate: 0.1
                                                 searcher: searcher_re:
                                                     0: re.compile('(login\\s*: )|((?!Last )Login Name:  )|(username: )|(User Name :)')
May 01 07:02:05 azura.codenet /fence_apc[18208]: Unable to connect/login to fencing device

Version-Release number of selected component (if applicable):
ovirt-host-4.5.0-3.el8.x86_64
vdsm-4.50.0.13-1.el8.x86_64
fence-agents-apc-4.2.1-89.el8.noarch

How reproducible:
Always.

Steps to Reproduce:
1. Set up power management for a host in oVirt 4.5 to use the "apc" fence type, do not enable the secure option.

2. Attempt to fence the host. Or test the fence agent's configuration in the admin UI.

Actual results:
Failure to fence the host. Failure to get power management status. Test configuration fails.

Expected results:
Host can be fenced, power management status is retrievable, test configuration passes.

Additional info:
There seems to be a similar issue described in an old discussion on the oVirt users mailing list: https://lists.ovirt.org/archives/list/users@ovirt.org/thread/GBD7MUJKXDZE56TYUCBJNDE7TWS7CHTZ/

Comment 1 Patrick Hibbs 2022-05-01 13:48:37 UTC
In the process of debugging this, I've found that the logging python library wasn't included by the fence_apc script despite it's use. Altering script to include the missing library seems to have fixed the issue.

I.e. adding "import logging" to the other import lines at the top of /usr/sbin/fence_apc.

Comment 2 Eli Mesika 2022-05-15 08:27:10 UTC
I understood that you workaround that manually by adding the missing import to the agent python file.

Which version of fence-agents package are installed on the host that serves as a proxy to the fencing operation.
We have to check the sources and open a bug if the problem is a missing import in the Python code

Comment 3 Patrick Hibbs 2022-05-16 02:20:39 UTC
According to the rpm -qa output on the host the fence-agents packages have these version numbers:

$ rpm -qa | grep fence-agent
fence-agents-brocade-4.2.1-93.el8.noarch
fence-agents-heuristics-ping-4.2.1-93.el8.noarch
fence-agents-intelmodular-4.2.1-93.el8.noarch
fence-agents-sbd-4.2.1-93.el8.noarch
fence-agents-apc-4.2.1-93.el8.noarch
fence-agents-eaton-snmp-4.2.1-93.el8.noarch
fence-agents-ilo-mp-4.2.1-93.el8.noarch
fence-agents-rhevm-4.2.1-93.el8.noarch
fence-agents-common-4.2.1-93.el8.noarch
fence-agents-apc-snmp-4.2.1-93.el8.noarch
fence-agents-cisco-ucs-4.2.1-93.el8.noarch
fence-agents-emerson-4.2.1-93.el8.noarch
fence-agents-ibmblade-4.2.1-93.el8.noarch
fence-agents-ilo-ssh-4.2.1-93.el8.noarch
fence-agents-ipmilan-4.2.1-93.el8.noarch
fence-agents-rsa-4.2.1-93.el8.noarch
fence-agents-vmware-rest-4.2.1-93.el8.noarch
fence-agents-kdump-4.2.1-93.el8.x86_64
fence-agents-bladecenter-4.2.1-93.el8.noarch
fence-agents-compute-4.2.1-93.el8.noarch
fence-agents-eps-4.2.1-93.el8.noarch
fence-agents-ifmib-4.2.1-93.el8.noarch
fence-agents-ilo2-4.2.1-93.el8.noarch
fence-agents-mpath-4.2.1-93.el8.noarch
fence-agents-rsb-4.2.1-93.el8.noarch
fence-agents-vmware-soap-4.2.1-93.el8.noarch
fence-agents-all-4.2.1-93.el8.x86_64
fence-agents-amt-ws-4.2.1-93.el8.noarch
fence-agents-drac5-4.2.1-93.el8.noarch
fence-agents-ilo-moonshot-4.2.1-93.el8.noarch
fence-agents-redfish-4.2.1-93.el8.x86_64
fence-agents-wti-4.2.1-93.el8.noarch
fence-agents-cisco-mds-4.2.1-93.el8.noarch
fence-agents-hpblade-4.2.1-93.el8.noarch
fence-agents-ipdu-4.2.1-93.el8.noarch
fence-agents-scsi-4.2.1-93.el8.noarch

Comment 4 Eli Mesika 2022-05-16 07:41:51 UTC
Checked latest fence-agents code on github
I found all scripts affected 

I have opened an issue in fence-agents github [1] 

workaround :

Either downgrade to version 4.2.1-53 or fix manually until a new release fixing that is available and distributed 


[1] https://github.com/ClusterLabs/fence-agents/issues/485

Comment 5 Martin Perina 2022-05-16 07:55:57 UTC
Based on above findings moving to the platform

Comment 7 Oyvind Albrigtsen 2022-05-16 09:06:00 UTC
Fix: https://github.com/ClusterLabs/fence-agents/pull/486

fence_lpar patch that makes it use fail_usage() intead: https://github.com/ClusterLabs/fence-agents/pull/474.

Comment 8 Martin Perina 2022-05-16 12:36:18 UTC
RHV 4.4 is staying on RHEL 8.6 LTS, so could this fix be also backported to RHEL 8.6.z?

Comment 17 errata-xmlrpc 2022-11-08 09:26:47 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (fence-agents bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:7520