Bug 1114977

Summary:	Class cast exception when fence_apc_snmp fails
Product:	[Retired] oVirt	Reporter:	Juan Hernández <juan.hernandez>
Component:	ovirt-engine-core	Assignee:	Eli Mesika <emesika>
Status:	CLOSED CURRENTRELEASE	QA Contact:	sefi litmanovich <slitmano>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	3.5	CC:	bugs, emesika, gklein, iheim, oourfali, rbalakri, yeylon
Target Milestone:	---
Target Release:	3.5.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:	infra
Fixed In Version:	ovirt-3.5.0_rc1	Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2014-10-17 12:22:17 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	Infra	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Juan Hernández 2014-07-01 11:25:21 UTC

Description of problem:

When the fencing agent fence_apc_snmp fails to connect to the fencing device the result returned by VDSM causes a class cast exception inside the engine.

Version-Release number of selected component (if applicable):

How reproducible:

ovirt-engine-3.5.0-0.0.master.20140629172303.git0b16ed7.fc20.noarch
vdsm-4.16.0-3.git601f786.fc19.x86_64
fence-agents-apc-snmp-4.0.4-3.fc19.x86_64

Steps to Reproduce:
1. Configure a host with the fence_snmp_apc agent.
2. Block IP access from the fencing proxy. I did with "iptables filter -I OUTPUT -d 10.35.23.25 -j REJECT".
3. Test fencing using the "Test" button of the host edit dialog.


Actual results:

The fencing fails with a ClassCastException:

2014-07-01 13:10:18,649 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand] (DefaultQuartzScheduler_Worker-4) Command FenceVdsVDSCommand(HostName = the_host, HostId = 081d0678-3728-486c-b6fc-eb264963b9cb, targetVdsId = b1aa88ed-63cd-4866-8a73-54de0d59c31a, action = Status, ip = the_ip, port = , type = apc_snmp, user = the_user, password = ******, options = 'port=15') execution failed. Exception: ClassCastException: [Ljava.lang.Object; cannot be cast to java.lang.String


Expected results:

The fenciong should fail, but not with a ClassCastException.

Additional info:

The following from the VDSM log of the fencing proxy may be relevant:

Thread-16::DEBUG::2014-07-01 13:11:49,627::API::1153::vds::(fenceNode) fenceNode(addr=the_ip,port=,agent=apc_snmp,user=the_user,passwd=XXXX,action=status,secure=,options=port=15)
Thread-16::DEBUG::2014-07-01 13:11:49,627::utils::738::root::(execCmd) /usr/sbin/fence_apc_snmp (cwd None)
Thread-16::DEBUG::2014-07-01 13:11:49,771::utils::758::root::(execCmd) FAILED: <err> = "Returned 1: snmpwalk: Failure in sendto (Operation not permitted)\r\n\nPlease use '-h' for usage\n"; <rc> = 1
Thread-16::DEBUG::2014-07-01 13:11:49,771::API::1140::vds::(fence) rc 1 inp agent=fence_apc_snmp
ipaddr=the_ip
login=the_user
action=status
passwd=XXXX
port=15 out [] err ['Returned 1: snmpwalk: Failure in sendto (Operation not permitted)', '', "Please use '-h' for usage"]
Thread-16::DEBUG::2014-07-01 13:11:49,771::API::1176::vds::(fenceNode) rc 1 in agent=fence_apc_snmp
ipaddr=rack05-pdu01-lab4.tlv.redhat.com
login=emesika
action=status
passwd=XXXX
port=15 out [] err ['Returned 1: snmpwalk: Failure in sendto (Operation not permitted)', '', "Please use '-h' for usage"]

Comment 1 sefi litmanovich 2014-09-04 08:31:38 UTC

Verified with ovirt-engine-3.5.0-0.0.master.20140821064931.gitb794d66.el6.noarch.
vdsm-4.16.2-1.gite8cba75.el6.x86_64.
fence-agents-3.1.5-35.el6_5.4.x86_64.

Verified according to description. fence failed as expected.

2014-09-04 11:23:44,050 WARN  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-7) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Power Management test failed for Host my_host.[Returned 1: No log handling enabled - turning on stderr logging, snmpwalk: Failure in sendto (Operation not permitted), , Please use '-h' for usage]
2014-09-04 11:23:44,052 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand] (ajp--127.0.0.1-8702-7) FINISH, FenceVdsVDSCommand, return: Test Failed, [Returned 1: No log handling enabled - turning on stderr logging, snmpwalk: Failure in sendto (Operation not permitted), , Please use '-h' for usage], log id: 17de5770
2014-09-04 11:23:44,053 WARN  [org.ovirt.engine.core.bll.FenceExecutor] (ajp--127.0.0.1-8702-7) Fencing operation failed with proxy host 0b143354-e959-42ef-b055-289603b06107, trying another proxy...
2014-09-04 11:23:44,181 ERROR [org.ovirt.engine.core.bll.FenceExecutor] (ajp--127.0.0.1-8702-7) Failed to run Power Management command on Host aqua-vds7, no running proxy Host was found.
2014-09-04 11:23:44,182 WARN  [org.ovirt.engine.core.bll.FenceExecutor] (ajp--127.0.0.1-8702-7) Failed to find other proxy to re-run failed fence operation, retrying with the same proxy...

Comment 2 Sandro Bonazzola 2014-10-17 12:22:17 UTC

oVirt 3.5 has been released and should include the fix for this issue.