Bug 1732782

Summary: Metrics Capture of instance cause an Internal Server Error
Product: Red Hat CloudForms Management Engine Reporter: Ido Ovadia <iovadia>
Component: ProvidersAssignee: Marek Aufart <maufart>
Status: NEW --- QA Contact: Ido Ovadia <iovadia>
Severity: high Docs Contact: Red Hat CloudForms Documentation <cloudforms-docs>
Priority: medium    
Version: 5.11.0CC: jfrey, jhardy, maufart, mshriver, obarenbo, smallamp
Target Milestone: GA   
Target Release: 5.11.z   
Hardware: x86_64   
OS: All   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: Bug
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: Openstack Target Upstream Version:
Attachments:
Description Flags
evm.log none

Description Ido Ovadia 2019-07-24 11:06:12 UTC
Created attachment 1593112 [details]
evm.log

Description of problem:
=======================
Instance Utilization is grayed out  
and evm.log show Internal Server Error  
ERROR -- : MIQ(ManageIQ::Providers::Openstack::CloudManager::MetricsCapture#perf_collect_metrics) [realtime] for: [ManageIQ::Providers::Openstack::CloudManager::Vm], [11000000000013], [instance-adm] Unhandled exception during perf data collection: [Expected(200) <=> Actual(500 InternalServerError)

Version-Release number of selected component:
=============================================
CFME 5.11.0.14

RHOS_TRUNK-15.0-RHEL-8-20190701.n.0

How reproducible:
=================
100%

Steps to Reproduce:
===================
1. Deploy RHOS 15 HA SSL
2. Add OSP as cloud provider
3. Create an instance (from Cirros image) 
4. Wait for target refresh

Actual results:
===============
Instance Utilization is grayed out


[----] E, [2019-07-24T06:38:06.179259 #30881:2ae5dfb925b8] ERROR -- : MIQ(ManageIQ::Providers::Openstack::CloudManager::MetricsCapture#perf_collect_metrics) [realtime] for: [ManageIQ::Providers::Openstack::CloudManager::Vm], [11000000000013], [instance-adm] Unhandled exception during perf data collection: [Expected(200) <=> Actual(500 InternalServerError)
excon.error.response
  :body          => "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML 2.0//EN\">\n<html><head>\n<title>500 Internal Server Error</title>\n</head><body>\n<h1>Internal Server Error</h1>\n<p>The server encountered an internal error or\nmisconfiguration and was unable to complete\nyour request.</p>\n<p>Please contact the server administrator at \n [no address given] to inform them of the time this error occurred,\n and the actions you performed just before this error.</p>\n<p>More information about this error may be available\nin the server error log.</p>\n</body></html>\n"
  :cookies       => [
  ]
  :headers       => {
    "Content-Length" => "531"
    "Content-Type"   => "text/html; charset=iso-8859-1"
    "Date"           => "Wed, 24 Jul 2019 10:38:06 GMT"
    "Server"         => "Apache"
  }
  :host          => "10.0.0.101"
  :local_address => "10.0.0.81"
  :local_port    => 46298
  :path          => "/v1/resource/generic/f4631c67-83fd-42a4-9a68-4ec50fa9a2c9/metric/memory.usage/measures"
  :port          => 13041
  :reason_phrase => "Internal Server Error"
  :remote_ip     => "10.0.0.101"
  :status        => 500
  :status_line   => "HTTP/1.1 500 Internal Server Error\r\n"
], class: [Excon::Error::InternalServerError]
[----] E, [2019-07-24T06:38:06.179491 #30881:2ae5dfb925b8] ERROR -- : MIQ(ManageIQ::Providers::Openstack::CloudManager::MetricsCapture#perf_collect_metrics) [realtime] for: [ManageIQ::Providers::Openstack::CloudManager::Vm], [11000000000013], [instance-adm]   Timings at time of error: {:capture_state=>0.0014889240264892578, :connect=>1.3005270957946777, :capture_counters=>1.0493550300598145, :capture_counter_values=>0.06941938400268555}
[----] E, [2019-07-24T06:38:06.179742 #30881:2ae5dfb925b8] ERROR -- : [Excon::Error::InternalServerError]: Expected(200) <=> Actual(500 InternalServerError)
excon.error.response
  :body          => "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML 2.0//EN\">\n<html><head>\n<title>500 Internal Server Error</title>\n</head><body>\n<h1>Internal Server Error</h1>\n<p>The server encountered an internal error or\nmisconfiguration and was unable to complete\nyour request.</p>\n<p>Please contact the server administrator at \n [no address given] to inform them of the time this error occurred,\n and the actions you performed just before this error.</p>\n<p>More information about this error may be available\nin the server error log.</p>\n</body></html>\n"
  :cookies       => [
  ]
  :headers       => {
    "Content-Length" => "531"
    "Content-Type"   => "text/html; charset=iso-8859-1"
    "Date"           => "Wed, 24 Jul 2019 10:38:06 GMT"
    "Server"         => "Apache"
  }
  :host          => "10.0.0.101"
  :local_address => "10.0.0.81"
  :local_port    => 46298
  :path          => "/v1/resource/generic/f4631c67-83fd-42a4-9a68-4ec50fa9a2c9/metric/memory.usage/measures"
  :port          => 13041
  :reason_phrase => "Internal Server Error"
  :remote_ip     => "10.0.0.101"
  :status        => 500
  :status_line   => "HTTP/1.1 500 Internal Server Error\r\n"
  Method:[block (2 levels) in <class:LogProxy>]


Expected results:
=================
Utilization is enabled, no errors

Additional info:
================
evm.log is enclosed

Comment 1 Sudhir Mallamprabhakara 2019-08-12 02:01:08 UTC
Does this scenario work on previous OSP versions?

Comment 2 Ido Ovadia 2019-08-12 10:38:07 UTC
(In reply to Sudhir Mallamprabhakara from comment #1)
> Does this scenario work on previous OSP versions?

I cat reproduce it because of: https://bugzilla.redhat.com/show_bug.cgi?id=1727275#c9

Comment 3 Ido Ovadia 2019-08-12 10:39:55 UTC
(In reply to Ido Ovadia from comment #2)
> (In reply to Sudhir Mallamprabhakara from comment #1)
> > Does this scenario work on previous OSP versions?
> 


I can't reproduce it because of: https://bugzilla.redhat.com/show_bug.cgi?id=1727275#c9

Comment 4 Sudhir Mallamprabhakara 2019-08-16 18:57:07 UTC
Ido, Blocker BZ is verified. Can you try now pls?

Comment 5 Ido Ovadia 2019-08-19 09:51:23 UTC
(In reply to Sudhir Mallamprabhakara from comment #1)
> Does this scenario work on previous OSP versions?

Works fine on OSP 10 and CFME 5.11.0.19