Created attachment 1589510 [details] evm.log after disabling role. Description of problem: Enable Embedded Ansible server role. Add repository, create and order service. Disable Embedded Ansible server role. OR Simply enable role wait for notification. Disable role. Watch evm.log for this string: [StandardError]: Embedded ansible is disabled Version-Release number of selected component (if applicable): 5.10.7.0.20190709151852_68f0bf9 How reproducible: 100% Steps to Reproduce: 1. 2. 3. Actual results: [StandardError]: Embedded ansible is disabled Expected results: There should be error present after role is disabled. Additional info: Works in previous version of 5.10.x See Attached evm.log
Hi Satyajit, Can you help me understand how it "worked" in 5.10.x? What do you mean "There should be error present after role is disabled". Where is the error in 5.10.x? Can you send me a screenshot or explain what you mean? "Expected results: There should be error present after role is disabled. Additional info: Works in previous version of 5.10.x See Attached evm.log"
(In reply to Joe Rafaniello from comment #4) > Hi Satyajit, > > Can you help me understand how it "worked" in 5.10.x? What do you mean > "There should be error present after role is disabled". Where is the error > in 5.10.x? Can you send me a screenshot or explain what you mean? > > > "Expected results: There should be error present after role is disabled. Typo :( "Expected results: There should not be error present after role is disabled. > > > Additional info: > > Works in previous version of 5.10.x > See Attached evm.log"
Do you have an example logs with 5.10.x? Things to check for: ems_inventory role is enabled in 5.10.x ems_refresh is queued for embedded ansible (it's possible we avoid queueing if the embedded ansible role is disabled, therefore it could be a timing issue (queue it before the role is disabled)) What do we want to solve here? 1) Don't queue refreshes for a disabled provider 2) Don't log a backtrace for a refresh of a disabled provider 3) Exit the refresh worker for a disabled provider Note, we don't pick up role changes instantly so there's always going to be some delay between the role being disabled and workers exiting, queue items no longer being added, etc. Please advise what you we want to solve here. Also, can you provide the full evm log on 5.11.x. I would like to see how queueing and exiting of the worker performed in 5.11.x Thanks!
Please provide the full log so I can see which of the items below are a problem in 5.11.x. 1) Don't queue refreshes for a disabled provider 2) Don't log a backtrace for a refresh of a disabled provider / continually refresh 3) Exit the refresh worker for a disabled provider 4) Something else... I honestly still don't know what the regression is here. Please send me a link to a 5.10.x appliance/logs where this doesn't happen. I'm leaning towards treating this as an enhancement/bugfix without a specific regression that can be identified.
Looking at the 197.111 reproducer environment on 5.10.7.0, I'm seeing the same error: [root@dhcp-8-197-111 vmdb]# grep -A 10 "StandardError" log/evm.log [----] E, [2019-07-11T09:16:37.433726 #37934:97af54] ERROR -- : [StandardError]: Embedded ansible is disabled Method:[block (2 levels) in <class:LogProxy>] [----] E, [2019-07-11T09:16:37.433921 #37934:97af54] ERROR -- : /var/www/miq/vmdb/app/models/manageiq/providers/embedded_ansible/provider.rb:14:in `raw_connect' /opt/rh/cfme-gemset/bundler/gems/cfme-providers-ansible_tower-719c9e2619ed/app/models/manageiq/providers/ansible_tower/shared/provider.rb:41:in `connect' /var/www/miq/vmdb/app/models/provider.rb:56:in `with_provider_connection' /opt/rh/cfme-gemset/bundler/gems/cfme-providers-ansible_tower-719c9e2619ed/app/models/manageiq/providers/ansible_tower/shared/automation_manager.rb:5:in `with_provider_connection' /opt/rh/cfme-gemset/bundler/gems/cfme-providers-ansible_tower-719c9e2619ed/app/models/manageiq/providers/ansible_tower/shared/automation_manager/refresher.rb:3:in `collect_inventory_for_targets' /var/www/miq/vmdb/app/models/manageiq/providers/base_manager/refresher.rb:97:in `block in refresh_targets_for_ems' /opt/rh/cfme-gemset/bundler/gems/cfme-gems-pending-7300bf53e177/lib/gems/pending/util/extensions/miq-benchmark.rb:11:in `realtime_store' /opt/rh/cfme-gemset/bundler/gems/cfme-gems-pending-7300bf53e177/lib/gems/pending/util/extensions/miq-benchmark.rb:28:in `realtime_block' /var/www/miq/vmdb/app/models/manageiq/providers/base_manager/refresher.rb:96:in `refresh_targets_for_ems' /var/www/miq/vmdb/app/models/manageiq/providers/base_manager/refresher.rb:41:in `block (2 levels) in refresh' -- [----] E, [2019-07-11T09:31:28.676589 #37934:97af54] ERROR -- : [StandardError]: Embedded ansible is disabled Method:[block (2 levels) in <class:LogProxy>] [----] E, [2019-07-11T09:31:28.676807 #37934:97af54] ERROR -- : /var/www/miq/vmdb/app/models/manageiq/providers/embedded_ansible/provider.rb:14:in `raw_connect' /opt/rh/cfme-gemset/bundler/gems/cfme-providers-ansible_tower-719c9e2619ed/app/models/manageiq/providers/ansible_tower/shared/provider.rb:41:in `connect' /var/www/miq/vmdb/app/models/provider.rb:56:in `with_provider_connection' /opt/rh/cfme-gemset/bundler/gems/cfme-providers-ansible_tower-719c9e2619ed/app/models/manageiq/providers/ansible_tower/shared/automation_manager.rb:5:in `with_provider_connection' /opt/rh/cfme-gemset/bundler/gems/cfme-providers-ansible_tower-719c9e2619ed/app/models/manageiq/providers/ansible_tower/shared/automation_manager/refresher.rb:3:in `collect_inventory_for_targets' /var/www/miq/vmdb/app/models/manageiq/providers/base_manager/refresher.rb:97:in `block in refresh_targets_for_ems' /opt/rh/cfme-gemset/bundler/gems/cfme-gems-pending-7300bf53e177/lib/gems/pending/util/extensions/miq-benchmark.rb:11:in `realtime_store' /opt/rh/cfme-gemset/bundler/gems/cfme-gems-pending-7300bf53e177/lib/gems/pending/util/extensions/miq-benchmark.rb:28:in `realtime_block' /var/www/miq/vmdb/app/models/manageiq/providers/base_manager/refresher.rb:96:in `refresh_targets_for_ems' /var/www/miq/vmdb/app/models/manageiq/providers/base_manager/refresher.rb:41:in `block (2 levels) in refresh' -- [----] E, [2019-07-11T09:46:30.126763 #37934:97af54] ERROR -- : [StandardError]: Embedded ansible is disabled Method:[block (2 levels) in <class:LogProxy>] [----] E, [2019-07-11T09:46:30.126936 #37934:97af54] ERROR -- : /var/www/miq/vmdb/app/models/manageiq/providers/embedded_ansible/provider.rb:14:in `raw_connect' /opt/rh/cfme-gemset/bundler/gems/cfme-providers-ansible_tower-719c9e2619ed/app/models/manageiq/providers/ansible_tower/shared/provider.rb:41:in `connect' /var/www/miq/vmdb/app/models/provider.rb:56:in `with_provider_connection' /opt/rh/cfme-gemset/bundler/gems/cfme-providers-ansible_tower-719c9e2619ed/app/models/manageiq/providers/ansible_tower/shared/automation_manager.rb:5:in `with_provider_connection' /opt/rh/cfme-gemset/bundler/gems/cfme-providers-ansible_tower-719c9e2619ed/app/models/manageiq/providers/ansible_tower/shared/automation_manager/refresher.rb:3:in `collect_inventory_for_targets' /var/www/miq/vmdb/app/models/manageiq/providers/base_manager/refresher.rb:97:in `block in refresh_targets_for_ems' /opt/rh/cfme-gemset/bundler/gems/cfme-gems-pending-7300bf53e177/lib/gems/pending/util/extensions/miq-benchmark.rb:11:in `realtime_store' /opt/rh/cfme-gemset/bundler/gems/cfme-gems-pending-7300bf53e177/lib/gems/pending/util/extensions/miq-benchmark.rb:28:in `realtime_block' /var/www/miq/vmdb/app/models/manageiq/providers/base_manager/refresher.rb:96:in `refresh_targets_for_ems' /var/www/miq/vmdb/app/models/manageiq/providers/base_manager/refresher.rb:41:in `block (2 levels) in refresh'
Please verify if this behavior is the same with 5.11.0
Hi Satyajit, I have opened a pull request for this issue, please see: https://github.com/ManageIQ/manageiq/pull/19229 Summary: * error is still printed as ERROR, telling you that you have the role disabled * the ruby backtrace is no longer printed Note, this is a 5.10 hammer only fix as 5.11/ivanchuk has the original code removed since we went to ansible runner.
New commit detected on ManageIQ/manageiq/hammer: https://github.com/ManageIQ/manageiq/commit/1b4010fae6a6a9cb01f45d48069c66a60957f152 commit 1b4010fae6a6a9cb01f45d48069c66a60957f152 Author: Joe Rafaniello <jrafanie> AuthorDate: Thu Aug 29 16:28:03 2019 -0400 Commit: Joe Rafaniello <jrafanie> CommitDate: Thu Aug 29 16:28:03 2019 -0400 log_backtrace won't print backtrace for MiqException errors See: https://github.com/ManageIQ/manageiq-gems-pending/blob/hammer/lib/gems/pending/util/vmdb-logger.rb#L106 This will still print as an ERROR level but the backtrace will not be printed to the log as this is a known error condition where the backtrace is not needed. Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1729166 Note, embedded ansible was converted to use runner starting in ivanchuk, therefore this is a hammer only change. See: https://github.com/ManageIQ/manageiq/pull/18687 app/models/manageiq/providers/embedded_ansible/provider.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
I'm not sure what to do about this. The original location is no longer logging the backtrace, so that part is fixed... but the summary refresh code is still logging the backtrace as it does with all exceptions. We could teach refresh to not hit this exception by filtering out the embedded ansible providers if the role is disabled but this provider has been removed on master so any changes we make should be minimal as it's a hammer only change.
Considering comment 18, comment 19 and comment 20, Able to see this BZ is fixed and cannot see the failure after disabling Embedded Ansible error in the logs. Verified Version: 5.10.12.0.20191007204014_0d55c84
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:3268