Bug 579859

Summary: [Cisco 5.6 feat] Include Cisco UCS Fencing Agent
Product: Red Hat Enterprise Linux 5 Reporter: Dave <dave.costakos>
Component: cmanAssignee: Marek Grac <mgrac>
Status: CLOSED ERRATA QA Contact: Cluster QE <mspqa-list>
Severity: high Docs Contact:
Priority: high    
Version: 5.5CC: andriusb, cluster-maint, cww, djansa, ericwill, green, iannis, joshua, jwest, liko, plsmith, sbradley, scofeldm, sghosh, shrijeet, ssaha, Stuart.Kirk, tao, zbrown
Target Milestone: ---Keywords: FutureFeature, Triaged
Target Release: 5.6   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: cman-2.0.115-61.el5 Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of:
: 580492 637222 (view as bug list) Environment:
Last Closed: 2011-01-13 22:32:59 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 637830    
Bug Blocks: 554476, 557597, 580492, 637222, 637223    
Attachments:
Description Flags
Proposed fence agent
none
Fencing library
none
Proposed fence agent
none
Fence library #2 none

Description Dave 2010-04-06 18:43:46 UTC
Description of problem:
We are using some Cisco UCS systems for RHCS and RHEV and I'd like to request a UCS-specific fencing agent.  We can use ipmilan to do UCS fencing today, but there is currently no way to tie the BMC IP address to a UCS Service Profile.  Therefore, there is not way to permanently gaurantee that we know the IMPI address of a UCS blade for fencing.  I'd love to see a fencing agent written that uses the Cisco UCS XML API for this.  I have tested such functionality, but I'm no python programmer.  I am however willing to help


Version-Release number of selected component (if applicable):
Cisco UCS All


How reproducible:
Anytime you move a physical Cisco blade into a different slot, it may get a new BMC IP address

Steps to Reproduce:
1. Create a UCS Service Profile in UCS Manager
2. Assign to server
3. Verify the BMC address by doing a "ssh to BMC for SoL" in UCS Manager
4. Move blade to a new slot
5. Verify that BMC address has changed by repeating step 3
  
Actual results:
BMC address moves, therefore UCS fencing may not work in production.

Expected results:
Repeatable method for UCS fencing.


Additional info:

Comment 1 Dave 2010-04-20 16:47:02 UTC
Opened SR 2014406 to track this request.

Comment 2 Chris Williams 2010-05-11 17:50:02 UTC
*** Bug 591238 has been marked as a duplicate of this bug. ***

Comment 6 Andrius Benokraitis 2010-06-29 17:27:11 UTC
Hi Scott, I've had a heck of a time trying to find someone within Cisco that can help assist test/verify this. Would you happen to know who I can start with to find the right person?

Comment 8 Stuart R. Kirk 2010-07-27 18:30:55 UTC
I can help tee you up with someone who can work with the XML API on this.  This
is definitely something that would benefit us mutually.  Please e-mail me, we
can coordinate a resource, and then add them to the BZ for further updates.

Comment 9 Eric Williams 2010-07-27 19:41:04 UTC
Andrius / Stuart:

I would be the resource that can help test / verify this.  

/Eric

Comment 10 Perry Myers 2010-08-09 16:04:59 UTC
(In reply to comment #9)
> Andrius / Stuart:
> 
> I would be the resource that can help test / verify this.  

Eric, we need a simulator of some sort to develop this.  Lon H said he talked to you about getting one.  Right now we're blocked on this, and with the code freeze date imminent for RHEL5.6 this may need to get pushed to 5.7

Comment 11 Eric Williams 2010-08-09 22:37:07 UTC
Perry:

I can get one to you today.  Would you like me to put it out on my Dropbox account and send you a public link?  Or would you like me to upload it to a ftp server you have?

Let me know either way.  I can help you out with this no problem.

Thanks,
Eric

Comment 12 Perry Myers 2010-08-12 12:21:05 UTC
(In reply to comment #11)
> Perry:
> 
> I can get one to you today.  Would you like me to put it out on my Dropbox
> account and send you a public link?  Or would you like me to upload it to a ftp
> server you have?

Dropbox would work.  The developer who'd be working on this is out for a week or so anyhow, so we're sort of stuck for the short term anyhow.  But if he can hit the ground running with the simulator from your Dropbox link when he gets back, that would be good.  Thanks!

Comment 13 Eric Williams 2010-08-13 14:11:47 UTC
OK.  I'll upload it later today.  I'll post the link here when it is done.

Thanks,
Eric

Comment 14 Eric Williams 2010-08-16 21:30:33 UTC
Here is the download information for the UCS Platform Emulator.

http://dl.dropbox.com/u/5820028/Cisco_UCS_Platform_Emulator_v1.3.55965.355989.zip

Comment 15 Marek Grac 2010-08-25 14:01:36 UTC
@Eric:

Thanks for emulator. 

Is there a possibility to send several commands through one open connection? e.g. login, get status of machine, power off, ..., logout

Comment 16 Marek Grac 2010-08-26 15:09:43 UTC
Created attachment 441246 [details]
Proposed fence agent

You will need this fence agent + fence library (attached as next file) and curl. 

Agent was tested against emulator where user/pass identification is not working, so I was unable to test it. It works for operations status/on/off/reboot. 

Missing features: 
* timeout options
* print debug to file
* it is questionable if we should use curl or implement it by ourselves.

Comment 17 Marek Grac 2010-08-26 15:10:16 UTC
Created attachment 441247 [details]
Fencing library

Comment 18 Marek Grac 2010-08-26 15:11:28 UTC
SSL version is not working correctly for me - no response for login

how to use it:

./fence_ucs.py -a 192.168.192.128 -l config -p config -o status -n aqua_ri-us

Comment 19 Eric Williams 2010-08-27 14:31:12 UTC
(In reply to comment #15)
> @Eric:
> 
> Thanks for emulator. 
> 
> Is there a possibility to send several commands through one open connection?
> e.g. login, get status of machine, power off, ..., logout

Yes you can send several commands through one open connection.  Once you do the login command, you can query for power status, power machines on / off, query again for power status, etc. and then logoff.  

Best practices do say to reuse a session as much as possible, but do logout when finished so that session is deleted.  If you have questions, please feel free to call me if you want to talk in real time.

Thanks,
Eric

Comment 20 Eric Williams 2010-08-27 14:32:06 UTC
(In reply to comment #18)
> SSL version is not working correctly for me - no response for login
> 
> how to use it:
> 
> ./fence_ucs.py -a 192.168.192.128 -l config -p config -o status -n aqua_ri-us

We will need to test SSL version on a real system.  HTTPS XMLAPI communication is not supported in our emulator just yet.  Looking to do that in the next rev of our emulator.

Comment 21 Sayan Saha 2010-08-30 17:27:06 UTC
Eric,

Can you please test the entire fence_agent on a real system as soon as we have it available for you and let us know the results?

Thanks
Sayan

Comment 22 Eric Williams 2010-08-31 17:00:28 UTC
(In reply to comment #21)
> Eric,
> 
> Can you please test the entire fence_agent on a real system as soon as we have
> it available for you and let us know the results?
> 
> Thanks
> Sayan

Sayan:

yes I can do that no problem.  I'm out in San Francisco at VMworld until Thursday night.  Earliest I can do a full blown test is on Friday.  Just send me the test plan you would like done, and I can test it against my systems in the lab.

Eric

Comment 24 Marek Grac 2010-09-01 15:26:28 UTC
Created attachment 442435 [details]
Proposed fence agent

New version of fence agent. It do not use curl but pycurl. 

Each command is issued in new connection (cookie is reused) as it will be more portable for other fence agents if needed. This version was tested against real machine (list, status, reboot, on, off) and it works as expected.

Timeouts, better error handling still have to be added. There will be some rewriting after fence agent for RHEV will be ready as they surely will have common code.

Comment 25 Marek Grac 2010-09-01 15:27:42 UTC
Created attachment 442436 [details]
Fence library #2

Comment 36 joshua 2011-01-12 17:51:05 UTC
How about from RHEL6 ?

Comment 37 errata-xmlrpc 2011-01-13 22:32:59 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2011-0036.html