Bug 1400995 - Central Admin - Azure instance Provision fails
Summary: Central Admin - Azure instance Provision fails
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat CloudForms Management Engine
Classification: Red Hat
Component: Provisioning
Version: 5.7.0
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: GA
: cfme-future
Assignee: Brandon Dunne
QA Contact: Alex Newman
URL:
Whiteboard: ca:provision:azure
Depends On:
Blocks: 1401000
TreeView+ depends on / blocked
 
Reported: 2016-12-02 14:14 UTC by Leo Khomenko
Modified: 2017-02-15 05:15 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-01-20 21:20:43 UTC
Category: ---
Cloudforms Team: Azure
Target Upstream Version:


Attachments (Terms of Use)
Remote region evm.log (4.40 MB, text/plain)
2016-12-02 14:14 UTC, Leo Khomenko
no flags Details

Description Leo Khomenko 2016-12-02 14:14:07 UTC
Created attachment 1227347 [details]
Remote region evm.log

Description of problem:


Version-Release number of selected component (if applicable):5.7.0.13


How reproducible:100%


Steps to Reproduce:
1.preconfigure 2 appliances to enable Central Admin
2.provision Azure Instance


Actual results:[EVM] VM [test_lkhom_ca] Step [CheckProvisioned] Status [[MiqPassword::MiqPasswordError]: can not decrypt v2_key encrypted string] Message [[MiqPassword::MiqPasswordError]: can not decrypt v2_key encrypted string]


Expected results:Instance Provisioned



Additional info:attaching logs from Remote Region with DEBUG level

Comment 2 Gregg Tanzillo 2016-12-02 15:57:59 UTC
Hi Leo, that error indicates that the encryption key of the remote region is not known to the global region. The encryption key is needed for the server to server authentication that central admin uses when forwarding operations to the remote regions.

There is a step that you may have missed on the global region to retrieve the keys from each of the regions that are being replicated.

You need to go to the replication settings in the global region ("Configuration" select the region and click the "Replication" tab). Once there set "Central Admin Enabled" to "Yes". Here's a screen shot http://screencast.com/t/1ZE8rZnBBZ4 then enter the credentials necessary for making an ssh connection to the remote region as in this screenshot http://screencast.com/t/sslxb2GF. The username should be "root" for the ssh connection.

This will retrieve the encryption key of the remote region and store it in the global region so that it can be user to authenticate to the remote region.

Comment 3 Gregg Tanzillo 2016-12-02 17:24:46 UTC
In looking at this a bit further, it looks like the issue is not what I described above. Based on the attached log I can assume that it got beyond the server to server authentication. The error seems to be happening trying to decrypt the providers root password.

I have a couple of questions about that -
1. Is the attached log from the remote region or the global? I would expect that it is the remote.
2. Was the provider created on the remote region? Could the encryption key have been changed after the provider was created?
3. Can we get access to the environment where this is happening - both remote and global regions - so that we can inspect?

Thanks.

Comment 8 CFME Bot 2017-01-12 14:26:04 UTC
New commit detected on ManageIQ/manageiq/master:
https://github.com/ManageIQ/manageiq/commit/511e666f6274285dfdc709486cd55a94cddd4dc8

commit 511e666f6274285dfdc709486cd55a94cddd4dc8
Author:     Nick Carboni <ncarboni>
AuthorDate: Fri Dec 9 09:32:22 2016 -0500
Commit:     Nick Carboni <ncarboni>
CommitDate: Fri Dec 9 09:50:01 2016 -0500

    Expose a method for encrypting using a remote v2_key
    
    These keys are saved as a part of configuring central admin.
    When encrypted data must be send to a remote region, that data
    has to be encrypted using the remote region's encryption key.
    
    This allows callers to encrypt the data so that the remote region
    can use it properly.
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1400995

 app/models/miq_region.rb       | 18 +++++++++++-------
 spec/models/miq_region_spec.rb | 18 ++++++++++++++++++
 2 files changed, 29 insertions(+), 7 deletions(-)

Comment 9 Nick Carboni 2017-01-12 14:30:05 UTC
My PR to add a method to encrypt a value using a particular region's v2_key has been merged. Re-assigning this bug to bdunne for the provisioning side.

Comment 11 Brandon Dunne 2017-01-20 21:20:43 UTC
Based on the documentation [1], the same encryption key is supposed to be used in all regions where Central Admin is expected to provision anything.  tools/fix_auth.rb can be used to change the encryption key if needed.


Note You need to log in before you can comment on or make changes to this bug.