Red Hat Bugzilla – Bug 1248663
Mongod fails to start due to bad configuration of pid file
Last modified: 2017-02-23 14:48:44 EST
Description of problem: Mongod fails to start due to bad configuration of pid file in init script after upgrade of Satellite Server from 6.0 to 6.1 Version-Release number of selected component (if applicable): mongodb-2.4.9-3.el7sat.x86_64 python-pymongo-2.5.2-3.el7sat.x86_64 python-pymongo-gridfs-2.5.2-3.el7sat.x86_64 python-mongoengine-0.7.10-2.el7sat.noarch mongodb-server-2.4.9-3.el7sat.x86_64 How reproducible: Have only tested on one server. Steps to Reproduce: 1. katello-installer --upgrade Actual results: Upgrading... Upgrade Step: stop_services... Upgrade Step: start_mongo... Upgrade Step: migrate_pulp... Upgrade Step: migrate_candlepin... Upgrade Step: migrate_foreman... Upgrade Step: Running installer... /Stage[main]/Mongodb::Server::Service/Service[mongodb]: Failed to call refresh: Could not restart Service[mongodb]: Execution of '/usr/share/katello-installer/modules/service_wait/bin/service-wait restart mongod' returned 1: Redirecting to /bin/systemctl restart mongod.service /Stage[main]/Mongodb::Server::Service/Service[mongodb]: Could not restart Service[mongodb]: Execution of '/usr/share/katello-installer/modules/service_wait/bin/service-wait restart mongod' returned 1: Redirecting to /bin/systemctl restart mongod.service Installing Debug: Unable to connect to mongodb server (127.0. [73%] [............................................................................................. ] Expected results: Upgrading... Upgrade Step: stop_services... Upgrade Step: start_mongo... Upgrade Step: migrate_pulp... Upgrade Step: migrate_candlepin... Upgrade Step: migrate_foreman... Upgrade Step: Running installer... Installing Done [100%] [..............................................................................................................................] The full log is at /var/log/katello-installer/katello-installer.log Additional info: I was able to correct the problem by digging in and recognizing that the pid file in /var/run/mongodb/ was different than what is specified in /etc/systemd/system/multi-user.target.wants/mongod.service after noticing that the service would start, then get signal 15 sent to the process after a timeout. Further triage included correcting the line in /etc/systemd/system/multi-user.target.wants/mongod.service, then running `katello-installer --upgrade` again would overwrite it back to the way it was before, ultimately breaking it again. I then went to /usr/share/katello-installer/modules/mongodb/manifests/params.pp to update line 52 from: " $pidfilepath = '/var/run/mongodb/mongod.pid'" to: " $pidfilepath = '/var/run/mongodb/mongodb.pid'". After this, katello-installer --upgrade correctly generates /etc/systemd/system/multi-user.target.wants/mongod.service and completes successfully.
*** Bug 1252974 has been marked as a duplicate of this bug. ***
Verified with sat6.1.2 snap1. Before upgrade on sat6.0.8, the installed version of mongod package: [root@cloud-qe-20 ~]# rpm -qa | grep mongo mongodb-2.4.9-3.el7sat.x86_64 python-pymongo-2.5.2-3.el7sat.x86_64 mongodb-server-2.4.9-3.el7sat.x86_64 [root@cloud-qe-20 ~]# cat /usr/share/katello-installer/modules/mongodb/manifests/params.pp | grep pidfilepath $pidfilepath = '/var/run/mongodb/mongodb.pid'
Upgrade started and break it by "ctrl + C" and restarted mongod. [root@cloud-qe-20 yum.repos.d]# katello-installer --upgrade Upgrading... Upgrade Step: stop_services... Upgrade Step: start_mongo... Upgrade Step: migrate_pulp... Upgrade Step: start_httpd... ^C/usr/share/katello-installer/hooks/boot/01-helpers.rb:24:in ``': Interrupt from /usr/share/katello-installer/hooks/boot/01-helpers.rb:24:in `block in execute' from /usr/share/katello-installer/hooks/boot/01-helpers.rb:23:in `each' from /usr/share/katello-installer/hooks/boot/01-helpers.rb:23:in `execute' from /usr/share/katello-installer/hooks/pre/30-upgrade.rb:13:in `start_httpd' from /usr/share/katello-installer/hooks/pre/30-upgrade.rb:39:in `upgrade_step' from /usr/share/katello-installer/hooks/pre/30-upgrade.rb:56:in `block (4 levels) in load' from /usr/share/gems/gems/kafo-0.6.5.9/lib/kafo/hooking.rb:32:in `instance_eval' from /usr/share/gems/gems/kafo-0.6.5.9/lib/kafo/hooking.rb:32:in `block (4 levels) in load' from /usr/share/gems/gems/kafo-0.6.5.9/lib/kafo/hook_context.rb:13:in `instance_exec' from /usr/share/gems/gems/kafo-0.6.5.9/lib/kafo/hook_context.rb:13:in `execute' from /usr/share/gems/gems/kafo-0.6.5.9/lib/kafo/hooking.rb:49:in `block in execute' from /usr/share/gems/gems/kafo-0.6.5.9/lib/kafo/hooking.rb:47:in `each' from /usr/share/gems/gems/kafo-0.6.5.9/lib/kafo/hooking.rb:47:in `execute' from /usr/share/gems/gems/kafo-0.6.5.9/lib/kafo/kafo_configure.rb:301:in `run_installation' from /usr/share/gems/gems/kafo-0.6.5.9/lib/kafo/kafo_configure.rb:109:in `execute' from /usr/share/gems/gems/clamp-0.6.2/lib/clamp/command.rb:67:in `run' from /usr/share/gems/gems/clamp-0.6.2/lib/clamp/command.rb:125:in `run' from /usr/share/gems/gems/kafo-0.6.5.9/lib/kafo/kafo_configure.rb:116:in `run' from /usr/sbin/katello-installer:46:in `<main>' [root@cloud-qe-20 yum.repos.d]# service mongod restart Redirecting to /bin/systemctl restart mongod.service
Re-ran the upgrade and upgrade was completed successfully. [root@cloud-qe-20 yum.repos.d]# katello-installer --upgrade Upgrading... Upgrade Step: stop_services... Upgrade Step: start_mongo... Upgrade Step: migrate_pulp... Upgrade Step: start_httpd... Upgrade Step: migrate_candlepin... Upgrade Step: migrate_foreman... Upgrade Step: Running installer... Installing Done [100%] [..................................................................] The full log is at /var/log/katello-installer/katello-installer.log Upgrade Step: restart_services... Upgrade Step: db_seed... Upgrade Step: errata_import (this may take a while) ... Upgrade Step: update_gpg_urls (this may take a while) ... Upgrade Step: update_repository_metadata (this may take a while) ... Katello upgrade completed!
this file doesn't have pid file change under "migrate_pulp" section: /usr/share/katello-installer/hooks/pre/30-upgrade.rb <snippet> def migrate_pulp # Start mongo if not running unless Kafo::Helpers.execute('pgrep mongod') Kafo::Helpers.execute('service-wait mongod start') end Based on above comments, moving this to verified.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2015:1786