Bug 1304418

Summary: Type conversion problem with Nori in Automate, data structure corrupted
Product: Red Hat CloudForms Management Engine Reporter: Martin Welk <mwelk>
Component: AutomateAssignee: mkanoor
Status: CLOSED WORKSFORME QA Contact: Dave Johnson <dajohnso>
Severity: medium Docs Contact:
Priority: medium    
Version: 5.5.0CC: jhardy, mkanoor, mwelk, obarenbo, psavage, tfitzger
Target Milestone: GA   
Target Release: 5.6.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-02-23 16:06:05 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Martin Welk 2016-02-03 14:40:47 UTC
Description of problem:
I am doing CFME automate integration with an IPAM (Bluecat, if it matters). I'm using Savon to interact with the Blucat API. It seems I get the strings from Savon back as Nori::StringWithAttributes types.
If I use the result of this type for a provisiong object, the data structure is broken. Since I see the data from the structure passed to further states in the state machine, it appears that you might able to pass data through or add attributes in a way from a custom method, that it makes other methods crash.

Version-Release number of selected component (if applicable):
5.5.0.13.20151201120956_653c0d4 

How reproducible:
for me, 100%

Steps to Reproduce:
I can only give some example: we register an IP address in IPAM. I am adding the debug code from the SOAP request in [1]
From that result, I need the <properties></properties>, I get it via
properties = ipam_response.body[:assign_next_available_ip4_address_response][:return][:properties]

This is a list of key1=value1|key2=value2|... pairs.

I separate that and put it in hash:

  propertieshash = Hash.new
  p=properties.split(/\|/)
  for q in p do
    key, value = q.split(/\=/)
    propertieshash[key] = value
  end

and finally, I get the IP with

ip = propertieshash["address"]

I also need the id of the record:

id = ipam_response.body[:assign_next_available_ip4_address_response][:return][:id]

later, I put these in the provisioning object

prov.set_option(:vmipaddr, ip)
prov.set_option(:ipam_vmid, ip)

Later on, I do 

@prov = $evm.root["miq_provision"]
log("info", "#{prov.inspect}")

that gives me [2]

A part of it looks like: ipam_vmid=>#<DRb::DRbUnknown:0x00000009070630 @name="Nori", @buf="\x04\bIC:\x1FNori::StringWithAttributes\"\r12534279\a:\x06ET:\x10@attributes{\x00"
>, :vmipaddr=>"141.201.106.111"

Provisioning won't work, before the call goes out to the VMware infrastructure provider, I see another ruby exception in the logs.

Expected results:
Honestly, I don't know. I'm not a developer. From the view of developing small methods for integration, I would say, that data corruption and all the following troubles shouldn't happen.

Additional info:

[1] [----] I, [2016-02-03T13:58:57.548893 #2932:6522b18]  INFO -- : Q-task_id([miq_provision_1000000002012]) Method STDOUT: D, [2016-02-03T13:58:57.253150 #30468] DEBUG -- : <env:Envelope xmlns:env='http://schemas.xmlsoap.org/soap/envelope/'><env:Header></env:Header><env:Body><tns:assignNextAvailableIP4AddressResponse xmlns:tns='http://api.proteus.bluecatnetworks.com'><return><id>12534384</id><name xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:nil="true"/><properties>firewall=false|contact=Auto-generated from CloudForms|address=141.201.106.112|state=DHCP_RESERVED|macAddress=57-C6-E2-7D-2E-F2|</properties><type>IP4Address</type></return></tns:assignNextAvailableIP4AddressResponse></env:Body></env:Envelope>
[----] I, [2016-02-03T13:58:57.548958 #2932:6522b18]  INFO -- : Q-task_id([miq_provision_1000000002012]) Method STDOUT: ipam_response = #<Savon::Response:0x000000020b0318 @http=#<HTTPI::Response:0x000000020b0a70 @code=200, @headers={"Content-Type"=>"text/xml;charset=UTF-8", "Transfer-Encoding"=>"chunked", "Date"=>"Wed, 03 Feb 2016 12:58:57 GMT", "Server"=>"BAM"}, @raw_body="<env:Envelope xmlns:env='http://schemas.xmlsoap.org/soap/envelope/'><env:Header></env:Header><env:Body><tns:assignNextAvailableIP4AddressResponse xmlns:tns='http://api.proteus.bluecatnetworks.com'><return><id>12534384</id><name xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xsi:nil=\"true\"/><properties>firewall=false|contact=Auto-generated from CloudForms|address=141.201.106.112|state=DHCP_RESERVED|macAddress=57-C6-E2-7D-2E-F2|</properties><type>IP4Address</type></return></tns:assignNextAvailableIP4AddressResponse></env:Body></env:Envelope>", @body="<env:Envelope xmlns:env='http://schemas.xmlsoap.org/soap/envelope/'><env:Header></env:Header><env:Body><tns:assignNextAvailableIP4AddressResponse xmlns:tns='http://api.proteus.bluecatnetworks.com'><return><id>12534384</id><name xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xsi:nil=\"true\"/><properties>firewall=false|contact=Auto-generated from CloudForms|address=141.201.106.112|state=DHCP_RESERVED|macAddress=57-C6-E2-7D-2E-F2|</properties><type>IP4Address</type></return></tns:assignNextAvailableIP4AddressResponse></env:Body></env:Envelope>">, @globals=#<Savon::GlobalOptions:0x000000021ce8f8 @option_type=:global, @options={:encoding=>"UTF-8", :soap_version=>1, :namespaces=>{}, :logger=>#<Logger:0x000000021ce880 @progname=nil, @level=0, @default_formatter=#<Logger::Formatter:0x000000021ce858 @datetime_format=nil>, @formatter=nil, @logdev=#<Logger::LogDevice:0x000000021ce808 @shift_size=nil, @shift_age=nil, @filename=nil, @dev=#<IO:<STDOUT>>, @mutex=#<Logger::LogDevice::LogDeviceMutex:0x000000021ce7e0 @mon_owner=nil, @mon_count=0, @mon_mutex=#<Mutex:0x000000021ce740>>>>, :log=>true, :filters=>[], :pretty_print_xml=>false, :raise_errors=>true, :strip_namespaces=>true, :convert_response_tags_to=>#<Proc:0x000000021ce6c8@/opt/rh/cfme-gemset/gems/savon-2.2.0/lib/savon/options.rb:56 (lambda)>, :wsdl=>"https://bluecat.mgmt.sbg.ac.at:443/Services/API?wsdl", :open_timeout=>20, :read_timeout=>20, :ssl_verify_mode=>:none}>, @locals=#<Savon::LocalOptions:0x00000002089240 @option_type=:local, @options={:advanced_typecasting=>true, :response_parser=>:nokogiri, :cookies=>[#<HTTPI::Cookie:0x00000001cff458 @cookie="JSESSIONID=86A9946D01AEFD6DA208CCFE0F89261C; Path=/Services; Secure">], :message=>{:configuration_id=>"5", :mac_address=>"57-C6-E2-7D-2E-F2", :parent_id=>"159191", :host_info=>"cf000117.sbg.ac.at,10,true,false", :action=>"MAKE_DHCP_RESERVED", :properties=>"contact=Auto-generated from CloudForms"}}>>


[2] ialog_vm_config_network=>"141.201.106.0/23 sbg.ac.at", :os_name=>"rhel7-64", :dialog_os_name=>"rhel7-64", :vm_owner_name=>"rht", :dialog_vm_owner_name=>"rht", :vm_owner_username=>"pl
us", :dialog_vm_owner_username=>"plus", :"password::vm_owner_password"=>"v2:{ukv8ucMWFSGYs5LrWrBpCw==}", :"password::dialog_vm_owner_password"=>"v2:{ukv8ucMWFSGYs5LrWrBpCw==}", :vm_owner_pa
ssword_readable=>"geheim", :networks=>[{:network=>"VLAN106", :devicetype=>"VirtualVmxnet3", :is_dvs=>true}], :disk_scsi=>[{:bus=>0, :pos=>0, :sizeInMB=>51200, :backing=>{:thinprovisioned=>t
rue}}], :vm_fqdn=>"cf000116.sbg.ac.at", :ipam_vmid=>#<DRb::DRbUnknown:0x00000009070630 @name="Nori", @buf="\x04\bIC:\x1FNori::StringWithAttributes\"\r12534279\a:\x06ET:\x10@attributes{\x00"
>, :vmipaddr=>"141.201.106.111", :vm_notes=>"Owner:  \nEmail: patrick.schwabl.at\nSource: Template_RHEL6_x64", :dest_host=>[1000000000003, "esx04.mgmt.sbg.ac.at"], :dest_storage=>[10
00000000060, "Hosting_DS1_Cdot"], :requested_network_adapter_count=>1}, created_on: "2016-02-03 12:49:26", updated_on: "2016-02-03 12:51:33", message: "Enter Foreman Parameters", status: "O
k", type: "ManageIQ::Providers::Vmware::InfraManager::Provisi...", miq_request_id: 1000000000705, source

Comment 2 mkanoor 2016-02-03 15:03:51 UTC
What portion of the ipam_vmid do you need for other states in Automate.
If you only need specific attributes you can save them individually using the prov.set_option.

If you need the entire thing then you would have to convert that whole Nori::StringWithAttributes into a YAML string. It might support a .to_yaml method which will become like a regular string, and that string can be stored in the prov.set_option and then once you need it you would have to convert back from the YAML string to Nori::StringWithAttributes

Comment 3 mkanoor 2016-02-03 15:04:04 UTC
What portion of the ipam_vmid do you need for other states in Automate.
If you only need specific attributes you can save them individually using the prov.set_option.

If you need the entire thing then you would have to convert that whole Nori::StringWithAttributes into a YAML string. It might support a .to_yaml method which will become like a regular string, and that string can be stored in the prov.set_option and then once you need it you would have to convert back from the YAML string to Nori::StringWithAttributes

Comment 4 Greg McCullough 2016-02-03 15:22:22 UTC
Saving custom object types in the provision hash (like Nori::StringWithAttributes) will cause the issue since later automate methods may not be able to properly convert them.

To avoid these errors you likely want to call ".to_s" on the strings being used with the set_options calls.

Something like this:

prov.set_option(:vmipaddr, ip.to_s)
prov.set_option(:ipam_vmid, ip.to_s)

That should give you a normal ruby string object instead of a custom class.

Comment 5 Martin Welk 2016-02-23 16:06:05 UTC
Sorry for the late reply, a consultants life is always to busy. Yes, the explicit type conversion solves my problem, and I learned a lot about Ruby that day. Thank you for your support, it was good to be able to show my customer our responsiveness!