Bug 1304418 - Type conversion problem with Nori in Automate, data structure corrupted
Summary: Type conversion problem with Nori in Automate, data structure corrupted
Keywords:
Status: CLOSED WORKSFORME
Alias: None
Product: Red Hat CloudForms Management Engine
Classification: Red Hat
Component: Automate
Version: 5.5.0
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: GA
: 5.6.0
Assignee: mkanoor
QA Contact: Dave Johnson
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-02-03 14:40 UTC by Martin Welk
Modified: 2017-12-05 15:51 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-02-23 16:06:05 UTC
Category: ---
Cloudforms Team: ---
Target Upstream Version:


Attachments (Terms of Use)

Description Martin Welk 2016-02-03 14:40:47 UTC
Description of problem:
I am doing CFME automate integration with an IPAM (Bluecat, if it matters). I'm using Savon to interact with the Blucat API. It seems I get the strings from Savon back as Nori::StringWithAttributes types.
If I use the result of this type for a provisiong object, the data structure is broken. Since I see the data from the structure passed to further states in the state machine, it appears that you might able to pass data through or add attributes in a way from a custom method, that it makes other methods crash.

Version-Release number of selected component (if applicable):
5.5.0.13.20151201120956_653c0d4 

How reproducible:
for me, 100%

Steps to Reproduce:
I can only give some example: we register an IP address in IPAM. I am adding the debug code from the SOAP request in [1]
From that result, I need the <properties></properties>, I get it via
properties = ipam_response.body[:assign_next_available_ip4_address_response][:return][:properties]

This is a list of key1=value1|key2=value2|... pairs.

I separate that and put it in hash:

  propertieshash = Hash.new
  p=properties.split(/\|/)
  for q in p do
    key, value = q.split(/\=/)
    propertieshash[key] = value
  end

and finally, I get the IP with

ip = propertieshash["address"]

I also need the id of the record:

id = ipam_response.body[:assign_next_available_ip4_address_response][:return][:id]

later, I put these in the provisioning object

prov.set_option(:vmipaddr, ip)
prov.set_option(:ipam_vmid, ip)

Later on, I do 

@prov = $evm.root["miq_provision"]
log("info", "#{prov.inspect}")

that gives me [2]

A part of it looks like: ipam_vmid=>#<DRb::DRbUnknown:0x00000009070630 @name="Nori", @buf="\x04\bIC:\x1FNori::StringWithAttributes\"\r12534279\a:\x06ET:\x10@attributes{\x00"
>, :vmipaddr=>"141.201.106.111"

Provisioning won't work, before the call goes out to the VMware infrastructure provider, I see another ruby exception in the logs.

Expected results:
Honestly, I don't know. I'm not a developer. From the view of developing small methods for integration, I would say, that data corruption and all the following troubles shouldn't happen.

Additional info:

[1] [----] I, [2016-02-03T13:58:57.548893 #2932:6522b18]  INFO -- : Q-task_id([miq_provision_1000000002012]) Method STDOUT: D, [2016-02-03T13:58:57.253150 #30468] DEBUG -- : <env:Envelope xmlns:env='http://schemas.xmlsoap.org/soap/envelope/'><env:Header></env:Header><env:Body><tns:assignNextAvailableIP4AddressResponse xmlns:tns='http://api.proteus.bluecatnetworks.com'><return><id>12534384</id><name xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:nil="true"/><properties>firewall=false|contact=Auto-generated from CloudForms|address=141.201.106.112|state=DHCP_RESERVED|macAddress=57-C6-E2-7D-2E-F2|</properties><type>IP4Address</type></return></tns:assignNextAvailableIP4AddressResponse></env:Body></env:Envelope>
[----] I, [2016-02-03T13:58:57.548958 #2932:6522b18]  INFO -- : Q-task_id([miq_provision_1000000002012]) Method STDOUT: ipam_response = #<Savon::Response:0x000000020b0318 @http=#<HTTPI::Response:0x000000020b0a70 @code=200, @headers={"Content-Type"=>"text/xml;charset=UTF-8", "Transfer-Encoding"=>"chunked", "Date"=>"Wed, 03 Feb 2016 12:58:57 GMT", "Server"=>"BAM"}, @raw_body="<env:Envelope xmlns:env='http://schemas.xmlsoap.org/soap/envelope/'><env:Header></env:Header><env:Body><tns:assignNextAvailableIP4AddressResponse xmlns:tns='http://api.proteus.bluecatnetworks.com'><return><id>12534384</id><name xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xsi:nil=\"true\"/><properties>firewall=false|contact=Auto-generated from CloudForms|address=141.201.106.112|state=DHCP_RESERVED|macAddress=57-C6-E2-7D-2E-F2|</properties><type>IP4Address</type></return></tns:assignNextAvailableIP4AddressResponse></env:Body></env:Envelope>", @body="<env:Envelope xmlns:env='http://schemas.xmlsoap.org/soap/envelope/'><env:Header></env:Header><env:Body><tns:assignNextAvailableIP4AddressResponse xmlns:tns='http://api.proteus.bluecatnetworks.com'><return><id>12534384</id><name xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xsi:nil=\"true\"/><properties>firewall=false|contact=Auto-generated from CloudForms|address=141.201.106.112|state=DHCP_RESERVED|macAddress=57-C6-E2-7D-2E-F2|</properties><type>IP4Address</type></return></tns:assignNextAvailableIP4AddressResponse></env:Body></env:Envelope>">, @globals=#<Savon::GlobalOptions:0x000000021ce8f8 @option_type=:global, @options={:encoding=>"UTF-8", :soap_version=>1, :namespaces=>{}, :logger=>#<Logger:0x000000021ce880 @progname=nil, @level=0, @default_formatter=#<Logger::Formatter:0x000000021ce858 @datetime_format=nil>, @formatter=nil, @logdev=#<Logger::LogDevice:0x000000021ce808 @shift_size=nil, @shift_age=nil, @filename=nil, @dev=#<IO:<STDOUT>>, @mutex=#<Logger::LogDevice::LogDeviceMutex:0x000000021ce7e0 @mon_owner=nil, @mon_count=0, @mon_mutex=#<Mutex:0x000000021ce740>>>>, :log=>true, :filters=>[], :pretty_print_xml=>false, :raise_errors=>true, :strip_namespaces=>true, :convert_response_tags_to=>#<Proc:0x000000021ce6c8@/opt/rh/cfme-gemset/gems/savon-2.2.0/lib/savon/options.rb:56 (lambda)>, :wsdl=>"https://bluecat.mgmt.sbg.ac.at:443/Services/API?wsdl", :open_timeout=>20, :read_timeout=>20, :ssl_verify_mode=>:none}>, @locals=#<Savon::LocalOptions:0x00000002089240 @option_type=:local, @options={:advanced_typecasting=>true, :response_parser=>:nokogiri, :cookies=>[#<HTTPI::Cookie:0x00000001cff458 @cookie="JSESSIONID=86A9946D01AEFD6DA208CCFE0F89261C; Path=/Services; Secure">], :message=>{:configuration_id=>"5", :mac_address=>"57-C6-E2-7D-2E-F2", :parent_id=>"159191", :host_info=>"cf000117.sbg.ac.at,10,true,false", :action=>"MAKE_DHCP_RESERVED", :properties=>"contact=Auto-generated from CloudForms"}}>>


[2] ialog_vm_config_network=>"141.201.106.0/23 sbg.ac.at", :os_name=>"rhel7-64", :dialog_os_name=>"rhel7-64", :vm_owner_name=>"rht", :dialog_vm_owner_name=>"rht", :vm_owner_username=>"pl
us", :dialog_vm_owner_username=>"plus", :"password::vm_owner_password"=>"v2:{ukv8ucMWFSGYs5LrWrBpCw==}", :"password::dialog_vm_owner_password"=>"v2:{ukv8ucMWFSGYs5LrWrBpCw==}", :vm_owner_pa
ssword_readable=>"geheim", :networks=>[{:network=>"VLAN106", :devicetype=>"VirtualVmxnet3", :is_dvs=>true}], :disk_scsi=>[{:bus=>0, :pos=>0, :sizeInMB=>51200, :backing=>{:thinprovisioned=>t
rue}}], :vm_fqdn=>"cf000116.sbg.ac.at", :ipam_vmid=>#<DRb::DRbUnknown:0x00000009070630 @name="Nori", @buf="\x04\bIC:\x1FNori::StringWithAttributes\"\r12534279\a:\x06ET:\x10@attributes{\x00"
>, :vmipaddr=>"141.201.106.111", :vm_notes=>"Owner:  \nEmail: patrick.schwabl.at\nSource: Template_RHEL6_x64", :dest_host=>[1000000000003, "esx04.mgmt.sbg.ac.at"], :dest_storage=>[10
00000000060, "Hosting_DS1_Cdot"], :requested_network_adapter_count=>1}, created_on: "2016-02-03 12:49:26", updated_on: "2016-02-03 12:51:33", message: "Enter Foreman Parameters", status: "O
k", type: "ManageIQ::Providers::Vmware::InfraManager::Provisi...", miq_request_id: 1000000000705, source

Comment 2 mkanoor 2016-02-03 15:03:51 UTC
What portion of the ipam_vmid do you need for other states in Automate.
If you only need specific attributes you can save them individually using the prov.set_option.

If you need the entire thing then you would have to convert that whole Nori::StringWithAttributes into a YAML string. It might support a .to_yaml method which will become like a regular string, and that string can be stored in the prov.set_option and then once you need it you would have to convert back from the YAML string to Nori::StringWithAttributes

Comment 3 mkanoor 2016-02-03 15:04:04 UTC
What portion of the ipam_vmid do you need for other states in Automate.
If you only need specific attributes you can save them individually using the prov.set_option.

If you need the entire thing then you would have to convert that whole Nori::StringWithAttributes into a YAML string. It might support a .to_yaml method which will become like a regular string, and that string can be stored in the prov.set_option and then once you need it you would have to convert back from the YAML string to Nori::StringWithAttributes

Comment 4 Greg McCullough 2016-02-03 15:22:22 UTC
Saving custom object types in the provision hash (like Nori::StringWithAttributes) will cause the issue since later automate methods may not be able to properly convert them.

To avoid these errors you likely want to call ".to_s" on the strings being used with the set_options calls.

Something like this:

prov.set_option(:vmipaddr, ip.to_s)
prov.set_option(:ipam_vmid, ip.to_s)

That should give you a normal ruby string object instead of a custom class.

Comment 5 Martin Welk 2016-02-23 16:06:05 UTC
Sorry for the late reply, a consultants life is always to busy. Yes, the explicit type conversion solves my problem, and I learned a lot about Ruby that day. Thank you for your support, it was good to be able to show my customer our responsiveness!


Note You need to log in before you can comment on or make changes to this bug.