Bug 1460263
| Summary: | shutdown_and_exit messages get marked as error and never removed from miq_queue table | ||
|---|---|---|---|
| Product: | Red Hat CloudForms Management Engine | Reporter: | Tasos Papaioannou <tpapaioa> |
| Component: | Appliance | Assignee: | Yuri Rudman <yrudman> |
| Status: | CLOSED ERRATA | QA Contact: | Tasos Papaioannou <tpapaioa> |
| Severity: | medium | Docs Contact: | |
| Priority: | medium | ||
| Version: | 5.8.0 | CC: | abellott, jhardy, obarenbo, simaishi, smallamp, yrudman |
| Target Milestone: | GA | ||
| Target Release: | 5.10.0 | ||
| Hardware: | All | ||
| OS: | All | ||
| Whiteboard: | appliance | ||
| Fixed In Version: | 5.10.0.0 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2019-02-07 23:02:34 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
New commit detected on ManageIQ/manageiq/master: https://github.com/ManageIQ/manageiq/commit/87cbb0cc69bf7cfad3c96ace92ae41e6f699f290 commit 87cbb0cc69bf7cfad3c96ace92ae41e6f699f290 Author: Yuri Rudman <yrudman> AuthorDate: Tue May 1 14:12:18 2018 -0400 Commit: Yuri Rudman <yrudman> CommitDate: Tue May 1 14:12:18 2018 -0400 remove shutdown_and_exit message from queue on server start if message dequeued# Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1460263 app/models/miq_server/at_startup.rb | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-) New commit detected on ManageIQ/manageiq/master: https://github.com/ManageIQ/manageiq/commit/87cbb0cc69bf7cfad3c96ace92ae41e6f699f290 commit 87cbb0cc69bf7cfad3c96ace92ae41e6f699f290 Author: Yuri Rudman <yrudman> AuthorDate: Tue May 1 14:12:18 2018 -0400 Commit: Yuri Rudman <yrudman> CommitDate: Tue May 1 14:12:18 2018 -0400 remove shutdown_and_exit message from queue on server start if message dequeued# Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1460263 app/models/miq_server/at_startup.rb | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-) Verified on 5.10.0.1. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2019:0212 |
Description of problem: From appliance_console, select Stop EVM Server Processes, then Start EVM Server Processes. The processes stop and start as expected, but the shutdown_and_exit messages stick around in the database w/ a state of 'error': vmdb_production=# select id, method_name, state, created_on, updated_on, queue_name, class_name, instance_id from miq_queue; id | method_name | state | created_on | updated_on | queue_name | class_name | instance_id -------+-------------------+-------+----------------------------+----------------------------+------------+------------+------------- 21775 | shutdown_and_exit | error | 2017-06-08 17:45:03.548581 | 2017-06-08 17:47:04.980664 | miq_server | MiqServer | 2 23188 | shutdown_and_exit | error | 2017-06-08 19:57:34.96574 | 2017-06-08 19:58:49.018568 | miq_server | MiqServer | 2 23094 | shutdown_and_exit | error | 2017-06-08 19:51:55.54475 | 2017-06-08 19:53:23.717701 | miq_server | MiqServer | 2 22578 | shutdown_and_exit | error | 2017-06-08 19:01:42.738824 | 2017-06-08 19:04:35.980748 | miq_server | MiqServer | 2 21704 | shutdown_and_exit | error | 2017-06-08 17:41:32.127746 | 2017-06-08 17:43:04.36108 | miq_server | MiqServer | 2 21587 | shutdown_and_exit | error | 2017-06-08 17:25:42.082366 | 2017-06-08 17:35:38.89939 | miq_server | MiqServer | 2 (6 rows) Version-Release number of selected component (if applicable): 5.8.0.17 How reproducible: 100% Steps to Reproduce: 1.) Stop then start EVM processes from appliance_console. 2.) View miq_queue entries in psql or rails console. 3.) See all historical shutdown_and_exit messages, marked as 'error'. Actual results: shutdown_and_exit messages not cleaned out of db after startup. Expected results: shutdown_and_exit messages cleaned out of db after startup. Additional info: In /var/www/miq/vmdb/app/models/miq_queue.rb, the atStartup method finds all queue messages with a state of 'dequeue' handled by the server, and updates their state to 'error'. **** def self.atStartup _log.info("Cleaning up queue messages...") MiqQueue.where(:state => STATE_DEQUEUE).each do |message| if message.handler.nil? _log.warn("Cleaning message in dequeue state without worker: #{format_full_log_msg(message)}") else handler_server = message.handler if message.handler.kind_of?(MiqServer) handler_server = message.handler.miq_server if message.handler.kind_of?(MiqWorker) next unless handler_server == MiqServer.my_server _log.warn("Cleaning message: #{format_full_log_msg(message)}") end message.update_attributes(:state => STATE_ERROR) rescue nil end _log.info("Cleaning up queue messages... Complete") end **** It's not clear that the shutdown_and_exit msg should be marked as 'error', since it did succeed. But either way, nothing cleans up the msg after it's marked as 'error', such as by calling the delivered method.