RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1198554 - ilo2 firmware +2.27 does not work with fence_ilo2
Summary: ilo2 firmware +2.27 does not work with fence_ilo2
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: fence-agents
Version: 6.8
Hardware: All
OS: Linux
high
high
Target Milestone: rc
: ---
Assignee: Marek Grac
QA Contact: cluster-qe@redhat.com
URL:
Whiteboard:
Depends On: 1199966
Blocks: 1199970 1199971 1208751 1214411
TreeView+ depends on / blocked
 
Reported: 2015-03-04 11:11 UTC by Gerwin Krist
Modified: 2019-08-15 04:19 UTC (History)
8 users (show)

Fixed In Version: fence-agents-4.0.15-8.el6
Doc Type: Bug Fix
Doc Text:
Prior to this update, when using an Integrated Lights-Out 2 (iLO2) device with firmware version 2.27, iLO2 negotiation of the TLS protocol did not work properly. Consequently, the user could not connect or login to the fencing device. With this update, the "--tls1.0" option has been added, and the user can now set the aforementioned protocol without negotiation.
Clone Of:
: 1199970 1208751 1214411 (view as bug list)
Environment:
Last Closed: 2015-07-22 06:58:50 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
Proposed fence agent (12.74 KB, application/octet-stream)
2015-03-04 14:04 UTC, Marek Grac
no flags Details
Proposed patch to fence_ilo (2.04 KB, patch)
2015-04-01 10:21 UTC, Marek Grac
no flags Details | Diff


Links
System ID Private Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 1374953 0 None None None Never
Red Hat Product Errata RHBA-2015:1350 0 normal SHIPPED_LIVE fence-agents bug fix and enhancement update 2015-07-20 17:59:44 UTC

Description Gerwin Krist 2015-03-04 11:11:55 UTC
Description of problem:
Since we have updated our blades to ilo v2.27 (jan 2015) the always working fence_ilo2 tool cannot login anymore on ilo2. For our RHEV environment this is mandatory.


Version-Release number of selected component (if applicable):
3.1.5-48

How reproducible:

Steps to Reproduce:
1. update the ilo2 firmware to 2.27 and then try:
2. /usr/sbin/fence_ilo2 -o status -a X.X.X.X -l <user> -p <password> -v


Actual results:
Says "Unable to connect/login to fencing device" even with -v it does not show more 

Expected results:
XML output and the power state


Additional info:
My guess is that fence_ilo2 is trying to use SSLv3. Which seems to be disabled in ilo v2.27. Our work-around is to enable sslv3 at the ilo2 interface with:
set /map1/config1 oemhp_ssl_v3_enable=yes 
After the ilo2 controller is reset it works again!

Comment 2 Marek Grac 2015-03-04 14:04:08 UTC
Created attachment 997855 [details]
Proposed fence agent

@Gerwin:

can you try to run fence agent (with current library) on your machine. It should be enough to use new option --notls which should solve this problem.

Comment 3 Gerwin Krist 2015-03-04 14:45:12 UTC
Hi Marek,

Does not work here:

./fence_ilo -o status -a 172.19.1.40 -l XXXXX -p XXXXXXX --notls -v
Running command: /usr/bin/gnutls-cli --priority "NORMAL:-VERS-TLS1.2:-VERS-TLS1.1:-VERS-TLS1.0:+VERS-SSL3.0" --insecure --crlf -p 443 172.19.1.40
Sent: <?xml version="1.0"?>

Unable to connect/login to fencing device

btw fence_ilo is same as fence_ilo2?

Comment 4 Marek Grac 2015-03-04 15:02:51 UTC
Hi,

can you take a look what will happend when you run that gnutls command directly? possibly with --verbose, so we can see where the problem is

Comment 5 Gerwin Krist 2015-03-04 15:34:58 UTC
output ilo2 2.27 - with oemhp_ssl_v3_enable=no (default in 2.27!)
=================================================================
openssl s_client -connect  x.x.x.x:443 -no_tls1_2 -no_tls1_1 -no_tls1:
SSL handshake has read 0 bytes and written 124 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NON

output ilo2 2.27 - with oemhp_ssl_v3_enable=yes
================================================
openssl s_client -connect  x.x.x.x:443 -no_tls1_2 -no_tls1_1 -no_tls1:

SSL handshake has read 1566 bytes and written 338 bytes
---
New, TLSv1/SSLv3, Cipher is DHE-RSA-AES256-SHA
Server public key is 2048 bit
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol  : SSLv3
    Cipher    : DHE-RSA-AES256-SHA
    Session-ID: 03F64F841BC33C593C4016866E59209389762EFD5701168EC57CA23C9E46E90E
    Session-ID-ctx:
    Master-Key: 05BC5524F3C97CD7EDDCB71A691C734301702F0DA1F5D86714EC8A9C5E660450D9327673F667DFD4ACEC998532BF783C
    Key-Arg   : None
    Krb5 Principal: None
    PSK identity: None
    PSK identity hint: None
    Start Time: 1425483129
    Timeout   : 300 (sec)
    Verify return code: 21 (unable to verify the first certificate)

Comment 6 Gerwin Krist 2015-03-04 15:36:10 UTC
Resolving '172.19.1.40'...
Connecting to '172.19.1.40:443'...
*** Fatal error: A TLS packet with unexpected length was received.
*** Handshake has failed
GNUTLS ERROR: A TLS packet with unexpected length was received.

Comment 7 Marek Grac 2015-03-04 16:00:35 UTC
Thanks, I will try to obtain a machine with that firmware and fix the issue.

Comment 8 Gerwin Krist 2015-03-05 08:37:27 UTC
If you have problems finding one. I can can help you with that :-) Just let me know.

Comment 9 Marek Grac 2015-03-05 08:39:00 UTC
@Gerwin: that will be great; I hope that I will need it just for a while.

Comment 10 Marek Grac 2015-03-06 14:27:06 UTC
@Gerwin:

thanks a lot, you can close connection.

as official fix will follow later, hotfix is

change line with 'gnutls_opts = ' in fencing.py to

gnutls_opts = "--priority \"NORMAL:-VERS-TLS1.2:-VERS-TLS1.1:+VERS-TLS1.0:%LATEST_RECORD_VERSION\""

that way, if you will use --notls, it should work as expected. 

Problem is that HP iLO2 supports only TLS1.0

Comment 12 Marek Grac 2015-03-09 12:07:18 UTC
In gnutls-cli (v2.8) there is not a required option. Bug against gnutls was created and if it will be approved, we will fix this issue.

Comment 13 John Ruemker 2015-03-13 20:19:46 UTC
Raising priority/severity based on recent customer case.

Comment 14 Marek Grac 2015-03-20 18:55:42 UTC
We have fix that does not need change in gnutls.

Comment 20 Marek Grac 2015-04-01 10:21:04 UTC
Created attachment 1009609 [details]
Proposed patch to fence_ilo

@John:

Proposed patch to add gnutls-cli support. In order to not break anything, fence agent will:

Try to connect using nss_wrapper as before and only if this fail, we will fallback to gnutls-cli.

This way, we can be sure that existing functionality will not be broken. Patch is quite simple, so code review by someone else will be fast.

Comment 21 John Ruemker 2015-04-01 13:37:43 UTC
Great, I'll see if the customer can test this.  

Thanks

Comment 23 Gerwin Krist 2015-04-01 14:20:20 UTC
@marek
Do you have the complete patched script? I will test it here then.

Comment 24 John Ruemker 2015-04-02 21:58:11 UTC
One small issue, one with your test build that's not in your patch posted at https://bugzilla.redhat.com/attachment.cgi?id=1009609&action=diff.  path specified was /usr/bin/gnutls, not gnutls-cli.  In RHEL 7 its provided by autoconf so we didn't have to worry about it, so I think the manual entry just led to a typo here. I built a new package with the correct path and tested it against your ilo and it works (status only, at least).  Customer is confirming in their environment, and we'll report back. 

I'm going to go ahead and start the ball rolling on the z-stream request now.

Justification: this issue requires customers that rely on iLO devices to maintain a widely-publicized security hole on these devices for the sake of compatibility with our fence agent.  Releasing a fix for this will allow our product to work with the iLO firmware version that closes that security hole, and so it is not only important for this customer, but we will surely see a number of others looking for this in 6.6.z in the months to come.

Comment 29 errata-xmlrpc 2015-07-22 06:58:50 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-1350.html


Note You need to log in before you can comment on or make changes to this bug.