Bug 1427653

Summary:	Rhev inventory refresh fails after rhev upgrade from 3.6 to 4.0
Product:	Red Hat CloudForms Management Engine	Reporter:	Josh Carter <jocarter>
Component:	Providers	Assignee:	Boriso <bodnopoz>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Jan Zmeskal <jzmeskal>
Severity:	medium	Docs Contact:
Priority:	high
Version:	5.7.0	CC:	adahms, agrare, bodnopoz, cpelland, ikaur, jfrey, jhardy, jocarter, juan.hernandez, jzmeskal, mgoldboi, mperina, obarenbo, rspagnol, sacpatil, simaishi, wdh
Target Milestone:	GA	Keywords:	TestOnly
Target Release:	5.9.2
Hardware:	All
OS:	All
Whiteboard:	rhev:ems_refresh
Fixed In Version:	5.9.0.1	Doc Type:	Known Issue
Doc Text:	At current, Red Hat CloudForms is unable to correctly collect inventory details from Red Hat Virtualization environments that have been upgraded from Red Hat Enterprise Virtualization 3.X to Red Hat Virtualization 4.X. This is caused by Red Hat CloudForms attempting to collect inventory details from the old FQDN for the environment and the new FQDN for the environment after the FQDN has been updated for that provider. As a workaround, restart the evmserverd on the appliance. This issue will be addressed in a future release of Red Hat CloudForms 4.6.	Story Points:	---
Clone Of:
Clones:	1442768 1442769 (view as bug list)		Environment:
Last Closed:	2018-05-09 13:11:55 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	CFME Core	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1442768, 1442769

Description Josh Carter 2017-02-28 20:20:18 UTC

Description of problem:

After upgrading rhevm from 3.6 to 4.0 CloudForms is unable to collect inventory

[----] D, [2017-02-28T16:03:07.673741 #13288:e27d04] DEBUG -- : MIQ(Rbac::Authorizer#role_allows?) Auth successful for user 'cfs', role 'EvmRole-super_administrator', feature identifier 'ems_infra_edit'
[----] I, [2017-02-28T16:03:07.940981 #13288:e27d04]  INFO -- : MIQ(ManageIQ::Providers::Redhat::InfraManager#with_provider_connection) Connecting through ManageIQ::Providers::Redhat::InfraManager: [GVA RHEV]
[----] I, [2017-02-28T16:03:07.941190 #13288:e27d04]  INFO -- : MIQ(ManageIQ::Providers::Redhat::InfraManager#connect) Using stored API path '/api'.
[----] D, [2017-02-28T16:03:07.941575 #13288:e27d04] DEBUG -- : MIQ(VMDB::Config#initialize) VMDB::Config.new is deprecated.  Prefer using Settings directly.
[----] D, [2017-02-28T16:03:07.941628 #13288:e27d04] DEBUG -- : MIQ(VMDB::Config#config) VMDB::Config#config is deprecated.  Prefer using Settings directly.
[----] E, [2017-02-28T16:03:07.969314 #13288:e27d04] ERROR -- : <RHEVM> Ovirt::Service#parse_error_response Return from URL: <https://lxvgvarhvprd01p.corp.ubp.ch/api> Data:<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">

<html><head>
<title>404 Not Found</title>
</head><body>
<h1>Not Found</h1>
<p>The requested URL /api was not found on this server.</p>
</body></html>

[----] W, [2017-02-28T16:03:07.969666 #13288:e27d04]  WARN -- : MIQ(ManageIQ::Providers::Redhat::InfraManager#authentication_check_no_validation) type: ["default"] for [42000000000005] [GVA RHEV] Validation failed: unreachable, Invalid URI specified for the server.
[----] E, [2017-02-28T16:03:07.969825 #13288:e27d04] ERROR -- : MIQ(ems_infra_controller-update): Credential validation was not successful: Invalid URI specified for the server.
^C


Version-Release number of selected component (if applicable): 5.7.*


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 2 Juan Hernández 2017-03-02 18:27:51 UTC

The support for checking the version of oVirt and automatically updating the endpoint path was added in 5.7.0.13:

  Add features based on supported_api_versions to RedHat provider
  https://github.com/ManageIQ/manageiq/pull/11418/files

I suggest that we verify that verify that the upgrade of RHV works correctly with the latest version 5.7.z version of CFME, and then close this bug as CURRENTRELEASE.

Meanwhile, users that have an older version of CFME can workaround the issue manually updating the 'path' column in the database:

  update endpoints set path='/ovirt-engine/api' where path='/api/ and id='...'

Makes sense?

Comment 4 Juan Hernández 2017-03-06 16:20:45 UTC

I did verify this with 5.7.0.17, and it worked correctly for me, that is why I assumed that the customer was using an older version. Note that the API path is stored in the database, and updated only when running a full refresh. Maybe that is what the customer didn't do. I will re-test.

Comment 5 Juan Hernández 2017-03-06 18:01:38 UTC

I repeated my tests and I can see that CFME doesn't automatically upgrade the stored 'path' to work correctly with oVirt 4. So we have two options:

1. Document this behavior and tell users to run the following when upgrading oVirt from 3.6 to 4.0 *if* it is combined with CFME 5.7.z:

  update endpoints set path='/ovirt-engine/api' where path='/api'

2. Change the provider so that it does never use the 'path' stored in the 'endpoints' table. Initially we didn't want to do that because it means an additional HTTP request for each connection between CFME and oVirt.

Which option do we want?

Comment 19 Juan Hernández 2017-03-15 09:13:13 UTC

The proposed solution to this issue is to add a database migration that automatically changes the 'path' column of the 'endpoints' table from '/api' to '/ovirt-engine/api':

  Migrate oVirt API path from /api to /ovirt-engine/api
  https://github.com/ManageIQ/manageiq/pull/14336

Comment 25 Juan Hernández 2017-04-03 09:51:26 UTC

As the data migration doesn't help users of version 5.7, we will change the provider so that it always uses /ovirt-engine/api:

  Drop support for oVirt /api, always use /ovirt-engine/api 
  https://github.com/ManageIQ/manageiq/pull/14469

Comment 35 Juan Hernández 2018-01-10 16:18:02 UTC

The use of the `/api` path to collectsevents for version 3.x of RHV is expected, and won't cause problems.

What is important to verify in this bug is that the `/api` path isn't used after upgrading to RHV 4.x, not for inventory collection neither for event collection, as it would not work.

A side effect of the fixes is that even for RHV 3.x the provider will use the `/ovirt-engine/api` path for inventory collection, but that isn't an issue and doesn't need to be verified.

Comment 36 Jan Zmeskal 2018-01-26 09:06:19 UTC

Verification steps:
1. I had machine A with RHEL6 and RHV 3.6.12.3
2. I got machine B with RHEL7
3. I got CFME appliance version 5.9.0.17

4. Added RHV 3.6 from machine A to the appliance
5. Backed-up config of RHV 3.6 (engine-backup)
6. Installed RHV 4.0.7.5 on machine B
7. Restored config from machine A RHV 3.6 to machine B RHV 4.0
8. Renamed RHV 4.0 on machine B (ovirt-engine-rename)
9. Edited the original provider in CFME appliance to have the new hostname
10. Started refresh for the Provider
11. Inspected rhevm.log on the appliance

Expected result:
<engine_url>/api should not be used after upgrading to 4.0. Only <engine_url>/ovirt-engine/api should be used.

Actual result:
/api path is still used. For example:

Ovirt::Service#resource_get: Sending URL: <engine_url/api/events?from=1137>

For details see the attached "rhevm.log from verification" and search for ".com/api". All content in rhevm.log was logged after upgrade from RHV 3.6 to RHV 4.0.

Comment 49 Andrew Dahms 2018-02-05 05:59:42 UTC

Updated the doc text and doc type fields.

Comment 51 Martin Perina 2018-02-14 11:31:11 UTC

Andrew, we don't have anything work there, this bug should be closed once we have updated the documentation. So can we close it?

Comment 52 Andrew Dahms 2018-02-14 22:49:04 UTC

Hi Martin,

Thank you for your needinfo request.

The known issue doc text has been added to the release notes, so if that is alright, there is nothing on the documentation side that should prevent this bug from being closed.

Kind regards,

Andrew

Comment 53 Martin Perina 2018-02-15 08:42:49 UTC

Thanks Andrew, so based on above moving to VERIFIED

Comment 54 Martin Perina 2018-02-26 08:54:54 UTC

(In reply to Andrew Dahms from comment #52)
> Hi Martin,
> 
> Thank you for your needinfo request.
> 
> The known issue doc text has been added to the release notes, so if that is
> alright, there is nothing on the documentation side that should prevent this
> bug from being closed.
> 
> Kind regards,
> 
> Andrew

Andrew, just small targeting question: is it contained in 5.9.0 release notes or 5.9.1 only? Just asking if we should not retarget to 5.9.0 ...

Comment 55 Andrew Dahms 2018-03-02 01:42:04 UTC

Hi Martin,

Thank you for your needinfo request.

To confirm - this release note is included in the 5.9.0 version of the Release Notes document.

If it makes more sense to align the target milestone with that, then I agree that 5.9.0 sounds more appropriate.

Kind regards,

Andrew

Comment 56 Martin Perina 2018-03-02 09:05:23 UTC

Unfortunately I'm no longer able to move the bug to 5.9.0, but the important thing is that the message is included in 5.9.0 release notes. Thanks Andrew.