Description of problem: We are able to deploy heat stack from CF as a single catalog items. However we need to deploy VMs to multiple site (different Openstack) and we are trying to deploy them as bundle catalog items which is failing. I think the due a bad/invalid json format for the Active record Object the server is not accepting it and thereby the update of AR object ServiceTemplateProvisionTask fails. Need to check as to how they are populating the object. The following error occurred during instance method <update_and_notify_parent> for AR object <#<ServiceTemplateProvisionTask id: 1000000000684, description: "Provisioning [Multisite-OS-Test] for Service [Multi...", state: "finished", request_type: "clone_to_service", userid: "admin", options: {:dialog=>{"dialog_stack_name"=>"MultisiteOSTest", "dialog_stack_onfailure"=>"ROLLBACK", "dialog_stack_timeout"=>"", "dialog_option_param_name"=>"MultisiteOSTestVM", "request"=>"clone_to_service"}, :workflow_settings=>{:resource_action_id=>1000000000577, :dialog_id=>1000000000023}, :src_id=>1000000000022, :delivered_on=>2015-09-06 05:24:58 UTC, :pass=>0, :parsed_dialog_options=>"---\n0:\n :stack_name: MultisiteOSTest\n :dialog_stack_name: MultisiteOSTest\n :stack_onfailure: ROLLBACK\n :dialog_stack_onfailure: ROLLBACK\n :option_param_name: MultisiteOSTestVM\n :dialog_option_param_name: MultisiteOSTestVM\n", :parsed_dialog_tags=>"--- {}\n"}, created_on: "2015-09-06 05:24:58", updated_on: "2015-09-06 05:26:23", message: "Service Provisioned Successfully", status: "Warn", type: "ServiceTemplateProvisionTask", miq_request_id: 1000000000265, source_id: 1000000000022, source_type: "ServiceTemplate", destination_id: 1000000000310, destination_type: "Service", miq_request_task_id: nil, phase: nil, phase_context: {}>> ============ [----] W, [2015-09-06T15:25:12.376647 #5483:5c3e9c] WARN -- : Q-task_id([service_template_provision_task_1000000000684]) MIQ(MiqQueue.put) /var/www/miq/vmdb/lib/miq_automation_engine/engine/miq_ae_engine.rb:49 called with large payload (args: 575 bytes, data: 0 bytes) Message id: [1000000628209], id: [], Zone: [default], Role: [automate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [service_template_provision_task_1000000000684], Command: [MiqAeEngine.deliver], Timeout: [3600], Priority: [100], State: [ready], Deliver On: [2015-09-06 05:26:12 UTC], Data: [], Args: [{:object_type=>"ServiceTemplateProvisionTask", :object_id=>1000000000684, :namespace=>"Service/Provisioning/StateMachines", :class_name=>"ServiceProvision_Template", :instance_name=>"CatalogBundleInitialization", :automate_message=>"create", :attrs=>{"dialog_stack_name"=>"MultisiteOSTest", "dialog_stack_onfailure"=>"ROLLBACK", "dialog_stack_timeout"=>"", "dialog_option_param_name"=>"MultisiteOSTestVM", "request"=>"clone_to_service"}, :user_id=>1000000000001, :state=>"checkprovisioned", :ae_fsm_started=>nil, :ae_state_started=>"2015-09-06 05:25:11 UTC", :ae_state_retries=>1}] .... [----] E, [2015-09-06T15:26:24.182385 #5483:9d35a00] ERROR -- : Q-task_id([service_template_provision_task_1000000000684]) MIQ(abstract_adapter) Name: [], Message: [PGError: ERROR: value too long for type character varying(255) ...: UPDATE "miq_requests" SET "request_state" = 'finished', "status" = 'Warn', "message" = 'Expected(201) <=> Actual(400 Bad Request) excon.error.response :body => "{\"explanation\": \"The server could not comply with the request since it is either malformed or otherwise incorrect.\", \"code\": 400, \"error\": {\"message\": \"The server could not comply with the request since it is either malformed or otherwise incorrect.\", \"traceback\": null, \"type\": \"HTTPBadRequest\"}, \"title\": \"Bad Request\"}" :headers => { "Content-Length" => "321" "Content-Type" => "application/json; charset=UTF-8" "Date" => "Sun, 06 Sep 2015 05:25:30 GMT" "X-Openstack-Request-Id" => "req-de6da573-51b1-45a8-a735-55e69a87b18c" } :local_address => "10.0.14.10" :local_port => 46871 :reason_phrase => "Bad Request" :remote_ip => "10.0.14.20" :status => 400 :status_line => "HTTP/1.1 400 Bad Request\r\n" (MiqException...] [----] E, [2015-09-06T15:26:24.189066 #5483:9d35a00] ERROR -- : Q-task_id([service_template_provision_task_1000000000684]) <AutomationEngine> The following error occurred during instance method <update_and_notify_parent> for AR object <#<ServiceTemplateProvisionTask id: 1000000000684, description: "Provisioning [Multisite-OS-Test] for Service [Multi...", state: "finished", request_type: "clone_to_service", userid: "admin", options: {:dialog=>{"dialog_stack_name"=>"MultisiteOSTest", "dialog_stack_onfailure"=>"ROLLBACK", "dialog_stack_timeout"=>"", "dialog_option_param_name"=>"MultisiteOSTestVM", "request"=>"clone_to_service"}, :workflow_settings=>{:resource_action_id=>1000000000577, :dialog_id=>1000000000023}, :src_id=>1000000000022, :delivered_on=>2015-09-06 05:24:58 UTC, :pass=>0, :parsed_dialog_options=>"---\n0:\n :stack_name: MultisiteOSTest\n :dialog_stack_name: MultisiteOSTest\n :stack_onfailure: ROLLBACK\n :dialog_stack_onfailure: ROLLBACK\n :option_param_name: MultisiteOSTestVM\n :dialog_option_param_name: MultisiteOSTestVM\n", :parsed_dialog_tags=>"--- {}\n"}, created_on: "2015-09-06 05:24:58", updated_on: "2015-09-06 05:26:23", message: "Service Provisioned Successfully", status: "Warn", type: "ServiceTemplateProvisionTask", miq_request_id: 1000000000265, source_id: 1000000000022, source_type: "ServiceTemplate", destination_id: 1000000000310, destination_type: "Service", miq_request_task_id: nil, phase: nil, phase_context: {}>> [----] E, [2015-09-06T15:26:24.189959 #5483:9d35a00] ERROR -- : Q-task_id([service_template_provision_task_1000000000684]) <AutomationEngine> MiqAeServiceModelBase.ar_method raised: <ActiveRecord::StatementInvalid>: <Database statement error encountered> [----] E, [2015-09-06T15:26:24.190133 #5483:9d35a00] ERROR -- : Q-task_id([service_template_provision_task_1000000000684]) <AutomationEngine> /opt/rh/cfme-gemset/bundler/gems/rails-4842a8377644/activerecord/lib/active_record/connection_adapters/postgresql_adapter.rb:1327:in `async_exec' ..... [----] E, [2015-09-06T15:26:24.193173 #5483:96d7168] ERROR -- : Q-task_id([service_template_provision_task_1000000000684]) <AutomationEngine> <AEMethod task_finished> The following error occurred during method evaluation: [----] E, [2015-09-06T15:26:24.193762 #5483:96d7168] ERROR -- : Q-task_id([service_template_provision_task_1000000000684]) <AutomationEngine> <AEMethod task_finished> DRb::DRbUnknownError: ActiveRecord:: [----] E, [2015-09-06T15:26:24.194689 #5483:96d7168] ERROR -- : Q-task_id([service_template_provision_task_1000000000684]) <AutomationEngine> <AEMethod task_finished> /opt/rh/ruby200/root/usr/share/ruby/drb/drb.rb:1117:in `method_missing' [----] E, [2015-09-06T15:26:24.208948 #5483:5c3e9c] ERROR -- : Q-task_id([service_template_provision_task_1000000000684]) <AutomationEngine> Method STDERR: /opt/rh/ruby200/root/usr/share/ruby/drb/drb.rb:1117:in `method_missing': ActiveRecord:: (DRb::DRbUnknownError) [----] E, [2015-09-06T15:26:24.209161 #5483:5c3e9c] ERROR -- : Q-task_id([service_template_provision_task_1000000000684]) <AutomationEngine> Method STDERR: from <code: task.finished($evm.inputs['message'])>:6:in `<main>' [----] I, [2015-09-06T15:26:24.212412 #5483:5c3e9c] INFO -- : Q-task_id([service_template_provision_task_1000000000684]) <AutomationEngine> <AEMethod [/ManageIQ/System/CommonMethods/StateMachineMethods/task_finished]> Ending [----] E, [2015-09-06T15:26:24.213754 #5483:5c3e9c] ERROR -- : Q-task_id([service_template_provision_task_1000000000684]) <AutomationEngine> Aborting instantiation (unknown method return code) because [Method exited with rc=Unknown RC: [1]] [----] E, [2015-09-06T15:26:24.214248 #5483:5c3e9c] ERROR -- : Q-task_id([service_template_provision_task_1000000000684]) <AutomationEngine> State=<Finished> running raised exception: <Method exited with rc=Unknown RC: [1]> [----] W, [2015-09-06T15:26:24.214416 #5483:5c3e9c] WARN -- : Q-task_id([service_template_provision_task_1000000000684]) <AutomationEngine> Error in State=[Finished] [----] E, [2015-09-06T15:26:24.215168 #5483:5c3e9c] ERROR -- : Q-task_id([service_template_provision_task_1000000000684]) MIQ(MiqAeEngine.deliver) Error delivering {"dialog_stack_name"=>"MultisiteOSTest", "dialog_stack_onfailure"=>"ROLLBACK", "dialog_stack_timeout"=>"", "dialog_option_param_name"=>"MultisiteOSTestVM", "request"=>"clone_to_service"} for object [ServiceTemplateProvisionTask.1000000000684] with state [checkprovisioned] to Automate: [----] I, [2015-09-06T15:26:24.215268 #5483:5c3e9c] INFO -- : Q-task_id([service_template_provision_task_1000000000684]) MIQ(ServiceTemplateProvisionTask.after_ae_delivery) ae_result="error" ================================================================================================================ Version-Release number of selected component (if applicable): 5.4.1.0 How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info:
The error about the message too long (255) haas been resolved in this issue https://bugzilla.redhat.com/show_bug.cgi?id=1252849.
Created attachment 1071452 [details] Automate domain changes for Orchestration provisioning with a bundle We have determined the issue reported here is because the CatalogBundleInitialization/CatalogItemInitialization methods used with the service state machines were originally designed to work with VM provisioning, before Orchestration provisioning was introduced into the product. These helper methods were designed to assist in passing parameters from a bundle service down to individual VM provision tasks. Orchestration provisioning uses a different configuration of tasks to perform provisioning which is not supported by the current methods. Attached is an updated domain containing a small change to the CatalogItemInitialization method that will allow for the dialog fields to be passed to the Orchestration provision item when used with a bundle. Additionally, it includes the fix from Bug 1252849 which should be applied to the environment as well to protect against Finally, the domain includes a CatalogItemInitialization state-machine instance for the /Cloud/Orchestration/Provisioning which needs to be used for the Orchestration Catalog Item when it is being provisioned as part of a bundled service. This instance runs the CatalogItemInitialization before running the standard orchestration state-machine steps. We have tested these changes locally and would like the customer to validate them in their environment.
Do we have access to the full appliance logs? The automation log above only shows the bundle service and we also want to see the catalog item. The fog error with "unknown protocol" is normal.
the archive is available on the dropbox ftp as 01503978-Current_region_1_default_1000000000001_EVM_1000000000001_20150909_031005_20150909_140938.zip
In addition to Comment 22 I wanted to add that after importing the automate domain you need to perform the following steps: 1. Ensure the domain where you import the changes is enabled. (If you import into a new domain it will be disabled by default.) 2. The import includes a CatalogItemInitialization state-machine instance for the /Cloud/Orchestration/Provisioning which needs to be used for the Orchestration Catalog Item when it is being provisioned as part of a bundled service. This instance runs the CatalogItemInitialization before running the standard orchestration state-machine steps. This change is what does the work of passing the parameters down to the catalog items.
Created attachment 1071848 [details] (Updated) Automate domain changes for Orchestration provisioning with a bundle Looking into this further we realized that the service dialog was modified to use the "option_0_" prefix on the fields which is used in conjunction with the catalog initialization automate methods to control how field values are pass down to sub-services. For example: Fields that start with "option_0_" are passed to all services. Fields starting with "option_1_" would only be passed to catalog items in a bundle with a provision index of 1. In both cases the field name is stripped of the "option_#_" before passing it down. If a field does not begin with a prefix like this it will be passed to all services with an unmodified field name. This is the different between what we were testing and what the customer was using. The customer had "option_0_stack_name" for the field and we were testing with just "stack_name". The export has been updated to deal with this difference. Please have the customer test with the latest export provided with this comment. Also note, the dialog field could be reverted back to use the original field names without the "option_0_" prefix which would work without these latest changes.
New commit detected on ManageIQ/manageiq/master: https://github.com/ManageIQ/manageiq/commit/1c94f4b608530e0870e46be8f7a7e3484614ce2a commit 1c94f4b608530e0870e46be8f7a7e3484614ce2a Author: Tina Fitzgerald <tfitzger> AuthorDate: Thu Sep 17 15:17:31 2015 -0400 Commit: Tina Fitzgerald <tfitzger> CommitDate: Thu Sep 17 15:17:31 2015 -0400 Changed CatalogItemInitialization automate method to pass dialog fields to the Orchestration provision item when used with a bundle. CatalogBundleInitialization/CatalogItemInitialization methods were designed to work with VM provisioning, before Orchestration provisioning was introduced. These methods were designed to assist in passing parameters from a bundle service down to individual VM provision tasks. Orchestration provisioning uses a different configuration of tasks to perform provisioning which is not supported by the current methods. https://bugzilla.redhat.com/show_bug.cgi?id=1260436 .../Methods.class/__methods__/catalogiteminitialization.rb | 13 +++++++++++++ 1 file changed, 13 insertions(+)
New commit detected on cfme/5.4.z: https://code.engineering.redhat.com/gerrit/gitweb?p=cfme.git;a=commitdiff;h=68ff7034a90e798a1deac12a221a095c84a79084 commit 68ff7034a90e798a1deac12a221a095c84a79084 Author: Tina Fitzgerald <tfitzger> AuthorDate: Thu Sep 17 15:17:31 2015 -0400 Commit: Tina Fitzgerald <tfitzger> CommitDate: Mon Sep 28 09:56:30 2015 -0400 Changed CatalogItemInitialization automate method to pass dialog fields to the Orchestration provision item when used with a bundle. CatalogBundleInitialization/CatalogItemInitialization methods were designed to work with VM provisioning, before Orchestration provisioning was introduced. These methods were designed to assist in passing parameters from a bundle service down to individual VM provision tasks. Orchestration provisioning uses a different configuration of tasks to perform provisioning which is not supported by the current methods. https://bugzilla.redhat.com/show_bug.cgi?id=1260436 .../Methods.class/__methods__/catalogiteminitialization.rb | 13 +++++++++++++ 1 file changed, 13 insertions(+)
New commit detected on cfme/5.4.z: https://code.engineering.redhat.com/gerrit/gitweb?p=cfme.git;a=commitdiff;h=cba77ec2888c0f59b594795ffc862bc3b5419f7d commit cba77ec2888c0f59b594795ffc862bc3b5419f7d Merge: b421bf0 8e836ec Author: Greg McCullough <gmccullo> AuthorDate: Tue Sep 29 12:03:48 2015 -0400 Commit: Greg McCullough <gmccullo> CommitDate: Tue Sep 29 12:03:48 2015 -0400 Merge branch 'bz_1261087' into '5.4.z' Automate | Services | Changed CatalogItemInitialization automate method for Orchestration provisioning Applied Orchestration automate fixes. https://bugzilla.redhat.com/show_bug.cgi?id=1261087 Original: https://github.com/ManageIQ/manageiq/pull/4416 https://bugzilla.redhat.com/show_bug.cgi?id=1260436 Clean cherry-pick except for spec test. Applying spec test patch wasn't successful since test was created after vmdb was rerooted. See merge request !266 .../Provision.class/catalogiteminitization.yaml | 14 +++ .../__methods__/catalogiteminitialization.rb | 15 +++ .../catalog_item_initialization_spec.rb | 105 +++++++++++++++++++++ vmdb/spec/support/service_template_helper.rb | 96 +++++++++++++++++++ 4 files changed, 230 insertions(+)
New commit detected on cfme_productization/master: https://code.engineering.redhat.com/gerrit/gitweb?p=cfme_productization.git;a=commit;h=68ff7034a90e798a1deac12a221a095c84a79084 commit 68ff7034a90e798a1deac12a221a095c84a79084 Author: Tina Fitzgerald <tfitzger> AuthorDate: Thu Sep 17 15:17:31 2015 -0400 Commit: Tina Fitzgerald <tfitzger> CommitDate: Mon Sep 28 09:56:30 2015 -0400 Changed CatalogItemInitialization automate method to pass dialog fields to the Orchestration provision item when used with a bundle. CatalogBundleInitialization/CatalogItemInitialization methods were designed to work with VM provisioning, before Orchestration provisioning was introduced. These methods were designed to assist in passing parameters from a bundle service down to individual VM provision tasks. Orchestration provisioning uses a different configuration of tasks to perform provisioning which is not supported by the current methods. https://bugzilla.redhat.com/show_bug.cgi?id=1260436 .../Methods.class/__methods__/catalogiteminitialization.rb | 13 +++++++++++++ 1 file changed, 13 insertions(+)
New commit detected on cfme_productization/master: https://code.engineering.redhat.com/gerrit/gitweb?p=cfme_productization.git;a=commit;h=cba77ec2888c0f59b594795ffc862bc3b5419f7d commit cba77ec2888c0f59b594795ffc862bc3b5419f7d Merge: b421bf0 8e836ec Author: Greg McCullough <gmccullo> AuthorDate: Tue Sep 29 12:03:48 2015 -0400 Commit: Greg McCullough <gmccullo> CommitDate: Tue Sep 29 12:03:48 2015 -0400 Merge branch 'bz_1261087' into '5.4.z' Automate | Services | Changed CatalogItemInitialization automate method for Orchestration provisioning Applied Orchestration automate fixes. https://bugzilla.redhat.com/show_bug.cgi?id=1261087 Original: https://github.com/ManageIQ/manageiq/pull/4416 https://bugzilla.redhat.com/show_bug.cgi?id=1260436 Clean cherry-pick except for spec test. Applying spec test patch wasn't successful since test was created after vmdb was rerooted. See merge request !266 .../Provision.class/catalogiteminitization.yaml | 14 +++ .../__methods__/catalogiteminitialization.rb | 15 +++ .../catalog_item_initialization_spec.rb | 105 +++++++++++++++++++++ vmdb/spec/support/service_template_helper.rb | 96 +++++++++++++++++++ 4 files changed, 230 insertions(+)
To verify.... Added two RHOS providers, one RHOS6 one RHOS7 Added a stack template Created a catalog item for each provider and pointed them to the orchestration entrypoint Created a bundle and pointed it to the service entrypoint Deployed service Verified working in 5.5.0.11
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2015:2551