Bug 1602883

Summary: Custom Buttons - When using protected fields, variables are not decrypted when passed to playbook
Product: Red Hat CloudForms Management Engine Reporter: David Luong <dluong>
Component: AutomateAssignee: Lucy Fu <lufu>
Status: CLOSED ERRATA QA Contact: Satyajit Bulage <sbulage>
Severity: medium Docs Contact:
Priority: medium    
Version: 5.9.0CC: cpelland, dluong, duhlmann, gmccullo, greartes, mkanoor, obarenbo, sbulage, simaishi, tfitzger, ytale
Target Milestone: GAKeywords: Reopened
Target Release: 5.10.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: 5.10.0.18 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-02-07 23:03:30 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: Bug
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: CFME Core Target Upstream Version:
Bug Depends On: 1640533    
Bug Blocks:    

Description David Luong 2018-07-18 17:37:17 UTC
Description of problem:
When using a protected field, for something, say a password, it doesn't seem to be decrypted

Version-Release number of selected component (if applicable):
5.9.3.4

How reproducible:
Always

Steps to Reproduce:
1.  Pass a var through a protected field in service dialog
2.  Submit and run playbook
3.

Actual results:
Fails due to incorrect var

Expected results:
Succeeds task requiring password

Additional info:

Comment 5 Greg McCullough 2018-07-18 18:03:51 UTC
Bill - Please review.

Comment 6 Bill Wei 2018-07-18 19:35:47 UTC
It is a service dialog issue. For a protected field the encrypt string is a result of *******, not of the actual text the user input.

Comment 7 Tina Fitzgerald 2018-07-19 22:05:46 UTC
Hi David,

I believe this issue has been resolved in 5.9.3.4 cfme-gemset hotfix -2 as described below:  

===  cfme-gemset  ====
 5.9.3.4-2 build
  - Patch1 BZ 1595776 (cloudforms/cfme-api@1e73337)
    Pass option to retain dialog values so they're not rerun

I added a "tina_password_test" Generic Service to your reproducer environment.
You can order the Service and type a password into the text box field.  I have a dynamic dialog as the second field which executes an Automate method that dumps the root object and decrypts the test box(password) contents.
I see the decrypted value is correct the first time the method is executed, but the method is executed a second time and that is when I see the password decrypted value as "********".

Can you apply the hot fix to the reproducer environment and test with my Service?

Let me know if you have any questions.

Thanks,
Tina

Comment 8 David Luong 2018-07-19 22:48:06 UTC
Hello Tina,

Applying the hotfix seems to have broken both of my buttons.  

It is no longer targeting the "Target Machines" and is now always choosing localhost.

As for your service, I can't see anything populate the 2nd textbox.

Comment 9 Tina Fitzgerald 2018-07-19 23:40:50 UTC
Hi David,

Nothing is supposed to populate the 2nd dialog. I have it there only to execute a dynamic method to dump the root object and decrypt the dialog password.


I can see the decrypted value in the Automate method whereas before the fixpack was applied, the decrypted value was "*********" regardless of the actual value specified.

Automation.log before fixpack
----] I, [2018-07-19T19:22:21.669583 #22667:16efc98]  INFO -- : <AEMethod inspectme>   Attribute - vmdb_object_type: service_template
[----] I, [2018-07-19T19:22:21.670928 #22667:1251c7c]  INFO -- : <AEMethod inspectme> Root:<$evm.root> Attributes - End
[----] I, [2018-07-19T19:22:21.671868 #22667:1251c7c]  INFO -- : <AEMethod inspectme>
[----] I, [2018-07-19T19:22:21.674600 #22667:1251c7c]  INFO -- : <AEMethod inspectme> TINA password is ********


Automation.log after fixpack:
----] I, [2018-07-19T19:22:28.350987 #22667:16f7510]  INFO -- : <AEMethod inspectme>   Attribute - vmdb_object_type: service_template
[----] I, [2018-07-19T19:22:28.352583 #22667:1251c7c]  INFO -- : <AEMethod inspectme> Root:<$evm.root> Attributes - End
[----] I, [2018-07-19T19:22:28.353939 #22667:1251c7c]  INFO -- : <AEMethod inspectme>
[----] I, [2018-07-19T19:22:28.356834 #22667:1251c7c]  INFO -- : <AEMethod inspectme> TINA password is testpwd


I'm sorry that it caused issues with your buttons. Are you able to manually specify the target machines?

Since this is fix pack and is QE not tested, it could have unintended side affects. The sole reason for applying the fix here would be to confirm that the password issue is resolved.

Can you test it to see if you are able to decrypt the password in your playbook?

Thanks,
Tina

Comment 10 David Luong 2018-07-20 15:04:40 UTC
Hello Tina,

After looking at the automate logs, your service works as intended.

However, as it pertains to playbooks, it looks like it's not taking in ANYTHING from the service dialog.  It's only passing default values from whatever is in the Catalog Item, so I cannot confirm that this works with Playbooks.

Comment 12 Tina Fitzgerald 2018-07-20 18:22:36 UTC
Hi David,

I launched the "test" custom button on your reproducer environment and I noticed a few things.
 
If you want to pass an extra_var to a playbook through a Service dialog entry, the dialog entry name must start with "param_". Dialog entries without the "param_" prefix will not be sent to the playbook.

I changed your "test" Service dialog to use the "param_" prefix for the "host" and "credential" dialog entries. 

The log snippets below show the before and after affect of that change. 

Before: Notice only the "rhn_password" extra_var.
[----] I, [2018-07-20T13:37:52.987019 #14663:1185118]  INFO -- : MIQ(MiqQueue#delivered) Message id: [18148], State: [ok], Delivered in [0.004890112] seconds
[----] I, [2018-07-20T13:37:55.129305 #22473:121bde8]  INFO -- : MIQ(ServiceAnsiblePlaybook#execute) Launching Ansible Tower job with options:
[----] I, [2018-07-20T13:37:55.132992 #22473:121bde8]  INFO -- :
---
extra_vars:
  rhn_password: [FILTERED]
  manageiq:
    api_url: https://10.10.181.137
    api_token: 52d3cc2ddf21a8f370a9b5b0ee363d47
    service: services/26
    user: users/1
    group: groups/2
    action: Provision
    X_MIQ_Group: EvmGroup-super_administrator
  manageiq_connection:
    url: https://10.10.181.137
    token: 52d3cc2ddf21a8f370a9b5b0ee363d47
    X_MIQ_Group: EvmGroup-super_administrator


After: Notice the "hosts" and "credentials" extra_vars.
---
extra_vars:
  credential: ''
  hosts: localhost
  rhn_password: [FILTERED]
  manageiq:
    api_url: https://10.10.181.137
    api_token: 938d818eaa454531655ebbfb54a12c5c
    service: services/28
    user: users/1
    group: groups/2
    action: Provision
    X_MIQ_Group: EvmGroup-super_administrator
  manageiq_connection:
    url: https://10.10.181.137
    token: 938d818eaa454531655ebbfb54a12c5c
    X_MIQ_Group: EvmGroup-super_administrator
@                                                          

My advice for for using this type of custom button is to first ensure the Service provisions properly when ordered through the Service catalog.
Behavior changes/issues would be more obvious when ordering the Service through the custom button once we know the Service provision works properly.

Can you order the Service through Service provisioning and report back if the password value is correct?

Any/all other issues can be addressed in a separate ticket and likely will have already been addressed, but maybe not included in your reproducer environment build.

Let me know if you have any questions.

Thanks,
Tina

Comment 13 David Luong 2018-07-20 19:03:48 UTC
Hello Tina,

The two fields you mentioned, host and credentials, were not generated by myself, but by the UI when creating a catalog item out of an Ansible Playbook.  I can confirm that those fields are correct without your param modifications, as:
1.  We transfer it through the host field just fine, as evidenced through this service provision through catalog, this is without the param_hosts modification.  It passes through as dialog_hosts just fine:
[----] I, [2018-07-20T14:38:53.604050 #25424:1185118]  INFO -- : Q-task_id([service_template_provision_task_29]) Updated namespace [/Service/Generic/StateMachines/GenericLifecycle/provision?MiqServer%3A%3Amiq_server=1&Service%3A%3AService=29&ServiceTemplateProvisionTask%3A%3Aservice_template_provision_task=29&User%3A%3Auser=1&ae_state=check_completed&ae_state_previous=---%0A%22%2FManageIQ%2FService%2FGeneric%2FStateMachines%2FGenericLifecycle%2Fprovision%22%3A%0A%20%20ae_state%3A%20check_completed%0A%20%20ae_state_retries%3A%201%0A%20%20ae_state_started%3A%202018-07-20%2018%3A37%3A44%20UTC%0A%20%20ae_state_max_retries%3A%20100%0A&ae_state_retries=1&ae_state_started=2018-07-20%2018%3A37%3A44%20UTC&dialog_credential=9&dialog_hosts=10.10.181.137&object_name=provision&password%3A%3Adialog_param_password=v2%3A%7Bx17OlVEw0EVzJBx%2FLY%2BYOf8yCZygjD3s8xILZFZC4xc%3D%7D&request=clone_to_service&service_action=Provision&vmdb_object_type=service_template_provision_task  ManageIQ/Service/Generic/StateMachines]

2.  I don't see how or why we would pass Credentials through to the playbook?  From what I understand, that is the credentials that CloudForms utilizes to connect when using ansible, not something that the playbook itself utilizes.

It was working before the hotfix, and now these fields are no longer populating correctly when making use of the button, which I suspect means we broke our buttons that utilize ansible playbooks, which I agree is a separate bug. However, the decryption does seem to work now minus if just using service catalog.

What it is doing when we order the catalog item through the button is that it is only taking the defaults that we supply to the Catalog Item itself.  This can be evidenced by the output of this play here:

SSH password: 

PLAY [all] *********************************************************************

TASK [Gathering Facts] *********************************************************
ok: [localhost]

TASK [debug] *******************************************************************
ok: [localhost] => {
    "msg": "This password is password"
}

PLAY RECAP *********************************************************************
localhost                  : ok=2    changed=0    unreachable=0    failed=0 

If the button worked as intended instead of utilizing defaults, it wouldn't show localhost and it also the password would've been different.  It seems like the defaults are overwriting whatever is supposed to be in there when using buttons, which I'll put in a new bug.

Comment 14 Tina Fitzgerald 2018-07-20 20:27:03 UTC
Hi David,

Yes, sorry about that, it was my mistake.  Host and credentials don't need to be passed to the playbook, and don't need the param prefix.

I suspect the custom button issue you're experiencing ash already been resolved, but is not included in the fix pack. I'll watch for your new ticket and will see if we can get it validated against a later build.

Can we close this ticket since password decryption is working properly?

Thanks,
Tina

Comment 15 David Luong 2018-07-20 21:52:06 UTC
Is there an existing bug I can link this to?  If so, I'd be comfortable with closing, if not, we should keep it open since it's not in the actual 5.9.3 errata, shouldn't we?

Here's the link to the new bug: https://bugzilla.redhat.com/show_bug.cgi?id=1606831

Comment 16 Tina Fitzgerald 2018-07-23 09:49:46 UTC
Hi David,
 
Thanks for opening the new bug. 

Closing this ticket as decryption is working properly.

Thanks,
Tina

Comment 17 Greg McCullough 2018-07-25 21:14:40 UTC
See bug 1595776 which addresses this issue and contains a 5.9.3 hotfix.

Comment 27 CFME Bot 2018-09-28 16:06:10 UTC
New commit detected on ManageIQ/manageiq/master:

https://github.com/ManageIQ/manageiq/commit/2496b7892a585bada8df53bf9f4ec5fdfab6d5ec
commit 2496b7892a585bada8df53bf9f4ec5fdfab6d5ec
Author:     Erik Clarizio <eclarizi@redhat.com>
AuthorDate: Thu Sep 27 17:57:21 2018 -0400
Commit:     Erik Clarizio <eclarizi@redhat.com>
CommitDate: Thu Sep 27 17:57:21 2018 -0400

    Do not double encrypt a protected password field

    https://bugzilla.redhat.com/show_bug.cgi?id=1602883

 app/models/dialog_field_text_box.rb | 6 +-
 spec/models/dialog_field_text_box_spec.rb | 12 +
 2 files changed, 17 insertions(+), 1 deletion(-)

Comment 29 CFME Bot 2018-09-28 20:51:14 UTC
New commit detected on ManageIQ/manageiq/master:

https://github.com/ManageIQ/manageiq/commit/d7366c9bb57b7f313d9d086fe510cf52e524625f
commit d7366c9bb57b7f313d9d086fe510cf52e524625f
Author:     Erik Clarizio <eclarizi@redhat.com>
AuthorDate: Thu Sep 27 17:57:21 2018 -0400
Commit:     Erik Clarizio <eclarizi@redhat.com>
CommitDate: Thu Sep 27 17:57:21 2018 -0400

    Do not double encrypt a protected password field

    https://bugzilla.redhat.com/show_bug.cgi?id=1602883

 app/models/dialog_field_text_box.rb | 6 +-
 spec/models/dialog_field_text_box_spec.rb | 12 -
 2 files changed, 1 insertion(+), 17 deletions(-)

Comment 30 CFME Bot 2018-10-01 16:57:14 UTC
New commit detected on ManageIQ/manageiq/hammer:

https://github.com/ManageIQ/manageiq/commit/0a94e25b14fd8534e45b32518d1e841a17ef44a4
commit 0a94e25b14fd8534e45b32518d1e841a17ef44a4
Author:     Greg McCullough <gmccullo@redhat.com>
AuthorDate: Fri Sep 28 12:04:02 2018 -0400
Commit:     Greg McCullough <gmccullo@redhat.com>
CommitDate: Fri Sep 28 12:04:02 2018 -0400

    Merge pull request #18031 from eclarizio/BZ1602883-Related

    Do not double encrypt a protected password dialog text field

    (cherry picked from commit 27c9fe355f36d372ec30ccd0eb26495421db1aa8)

    https://bugzilla.redhat.com/show_bug.cgi?id=1602883

 app/models/dialog_field_text_box.rb | 6 +-
 spec/models/dialog_field_text_box_spec.rb | 12 +
 2 files changed, 17 insertions(+), 1 deletion(-)

Comment 31 CFME Bot 2018-10-01 20:19:51 UTC
New commit detected on ManageIQ/manageiq-automation_engine/master:

https://github.com/ManageIQ/manageiq-automation_engine/commit/488dfa1b3c00426694f432a06d2a9f2a1e8f3d68
commit 488dfa1b3c00426694f432a06d2a9f2a1e8f3d68
Author:     Lucy Fu <lufu@redhat.com>
AuthorDate: Fri Sep 28 11:12:20 2018 -0400
Commit:     Lucy Fu <lufu@redhat.com>
CommitDate: Fri Sep 28 11:12:20 2018 -0400

    Add some help methods for MiqAeObject.

    https://bugzilla.redhat.com/show_bug.cgi?id=1602883

 lib/miq_automation_engine/engine/miq_ae_method_service/miq_ae_service_object_common.rb | 8 +
 spec/engine/miq_ae_method_service/miq_ae_service_object_common_spec.rb | 44 +
 2 files changed, 52 insertions(+)

Comment 32 CFME Bot 2018-10-01 20:37:45 UTC
New commit detected on ManageIQ/manageiq-content/master:

https://github.com/ManageIQ/manageiq-content/commit/5b95076f785d30f2cc429a1d9671aa91258bb7c0
commit 5b95076f785d30f2cc429a1d9671aa91258bb7c0
Author:     Lucy Fu <lufu@redhat.com>
AuthorDate: Thu Sep 27 17:52:01 2018 -0400
Commit:     Lucy Fu <lufu@redhat.com>
CommitDate: Thu Sep 27 17:52:01 2018 -0400

    Keep the encrypted value as is when creating service provision request.

    Currently the encrypted value is sent to service provision request as "********" when called via custom button.

    https://bugzilla.redhat.com/show_bug.cgi?id=1602883

 content/automate/ManageIQ/System/Request.class/__methods__/order_ansible_playbook.rb | 3 +-
 spec/content/automate/ManageIQ/System/Request.class/__methods__/order_ansible_playbook_spec.rb | 25 +-
 2 files changed, 26 insertions(+), 2 deletions(-)

Comment 33 CFME Bot 2018-10-02 12:28:00 UTC
New commit detected on ManageIQ/manageiq-providers-ansible_tower/master:

https://github.com/ManageIQ/manageiq-providers-ansible_tower/commit/45f6fe7dac4eef238c63d85c5b640d0c7bff2102
commit 45f6fe7dac4eef238c63d85c5b640d0c7bff2102
Author:     Lucy Fu <lufu@redhat.com>
AuthorDate: Wed Sep 26 15:23:47 2018 -0400
Commit:     Lucy Fu <lufu@redhat.com>
CommitDate: Wed Sep 26 15:23:47 2018 -0400

    Decrypt extra_vars before sending over to tower gem.

    https://bugzilla.redhat.com/show_bug.cgi?id=1602883

 app/models/manageiq/providers/ansible_tower/shared/automation_manager/configuration_script.rb | 6 +-
 spec/support/ansible_shared/automation_manager/configuration_script.rb | 6 +
 2 files changed, 11 insertions(+), 1 deletion(-)

Comment 34 CFME Bot 2018-10-02 13:33:28 UTC
New commit detected on ManageIQ/manageiq-automation_engine/hammer:

https://github.com/ManageIQ/manageiq-automation_engine/commit/febc71ffec2d1f7f67b1765f8fa77bba9105f324
commit febc71ffec2d1f7f67b1765f8fa77bba9105f324
Author:     Madhu Kanoor <mkanoor@redhat.com>
AuthorDate: Mon Oct  1 16:17:56 2018 -0400
Commit:     Madhu Kanoor <mkanoor@redhat.com>
CommitDate: Mon Oct  1 16:17:56 2018 -0400

    Merge pull request #237 from lfu/encrypted_password_in_playbook_1602883

    Add some helper methods for MiqAeObject.

    (cherry picked from commit 28fe61df70bdd738d781cf2409e4a07af2dc8391)

    https://bugzilla.redhat.com/show_bug.cgi?id=1602883

 lib/miq_automation_engine/engine/miq_ae_method_service/miq_ae_service_object_common.rb | 8 +
 spec/engine/miq_ae_method_service/miq_ae_service_object_common_spec.rb | 44 +
 2 files changed, 52 insertions(+)

Comment 35 CFME Bot 2018-10-02 13:37:10 UTC
New commit detected on ManageIQ/manageiq-content/hammer:

https://github.com/ManageIQ/manageiq-content/commit/cbee9c91ab7d26cb59954d254990f8ac405a9c79
commit cbee9c91ab7d26cb59954d254990f8ac405a9c79
Author:     Madhu Kanoor <mkanoor@redhat.com>
AuthorDate: Mon Oct  1 16:36:06 2018 -0400
Commit:     Madhu Kanoor <mkanoor@redhat.com>
CommitDate: Mon Oct  1 16:36:06 2018 -0400

    Merge pull request #435 from lfu/encrypted_password_in_playbook_1602883

    Keep the encrypted value as is when creating service provision request.

    (cherry picked from commit 09ad2ef2f24cbef3463f86fb460573149b98a6c3)

    https://bugzilla.redhat.com/show_bug.cgi?id=1602883

 content/automate/ManageIQ/System/Request.class/__methods__/order_ansible_playbook.rb | 3 +-
 spec/content/automate/ManageIQ/System/Request.class/__methods__/order_ansible_playbook_spec.rb | 25 +-
 2 files changed, 26 insertions(+), 2 deletions(-)

Comment 36 CFME Bot 2018-10-02 13:37:59 UTC
New commit detected on ManageIQ/manageiq-providers-ansible_tower/hammer:

https://github.com/ManageIQ/manageiq-providers-ansible_tower/commit/73b132b92635abdf17d903edcc44e417cacbc810
commit 73b132b92635abdf17d903edcc44e417cacbc810
Author:     Greg McCullough <gmccullo@redhat.com>
AuthorDate: Tue Oct  2 08:25:35 2018 -0400
Commit:     Greg McCullough <gmccullo@redhat.com>
CommitDate: Tue Oct  2 08:25:35 2018 -0400

    Merge pull request #127 from lfu/encrypted_password_in_playbook_1602883

    Decrypt extra_vars before sending over to tower gem.

    (cherry picked from commit 028981410c8d7fb9b8109149c944b31fa665472b)

    https://bugzilla.redhat.com/show_bug.cgi?id=1602883

 app/models/manageiq/providers/ansible_tower/shared/automation_manager/configuration_script.rb | 6 +-
 spec/support/ansible_shared/automation_manager/configuration_script.rb | 6 +
 2 files changed, 11 insertions(+), 1 deletion(-)

Comment 39 Satyajit Bulage 2018-11-14 14:57:57 UTC
Verified Version: 5.10.0.23.20181106165157_92dd189

Comment 40 errata-xmlrpc 2019-02-07 23:03:30 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2019:0212