RDO tickets are now tracked in Jira https://issues.redhat.com/projects/RDO/issues/
Bug 1236775 - rdo kilo mongo fails to start
Summary: rdo kilo mongo fails to start
Keywords:
Status: CLOSED EOL
Alias: None
Product: RDO
Classification: Community
Component: openstack-puppet-modules
Version: Kilo
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: Kilo
Assignee: Ivan Chavero
QA Contact: Shai Revivo
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-06-30 01:22 UTC by wes hayutin
Modified: 2016-05-19 15:37 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-05-19 15:37:10 UTC
Embargoed:


Attachments (Terms of Use)

Description wes hayutin 2015-06-30 01:22:56 UTC
Description of problem:

/var/log/mongodb.log

2015-06-29T22:16:21.565+0000 SEVERE: terminate() called, printing stack (if implemented for platform):
0x7ff3418a3d69 0x7ff3418a3572 0x7ff33e276926 0x7ff33e276953 0x7ff33e276b73 0x7ff340199fff 0x7ff34189ffe5 0x7ff34189ea63 0x7ff34189d5e9 0x7ff3412668ec 0x7ff3412742fa 0x7ff341274e1f 0x7ff3412316a1 0x7ff33d951af5 0x7ff3412617f9 
 /usr/bin/mongod(_ZN5mongo15printStackTraceERSo+0x39) [0x7ff3418a3d69]
 /usr/bin/mongod(+0x8c3572) [0x7ff3418a3572]
 /lib64/libstdc++.so.6(+0x5e926) [0x7ff33e276926]
 /lib64/libstdc++.so.6(+0x5e953) [0x7ff33e276953]
 /lib64/libstdc++.so.6(+0x5eb73) [0x7ff33e276b73]
 /lib64/libboost_filesystem-mt.so.1.53.0(_ZN5boost10filesystem6detail6statusERKNS0_4pathEPNS_6system10error_codeE+0x21f) [0x7ff340199fff]
 /usr/bin/mongod(_ZN5mongo14LinuxSysHelper14getLinuxDistroERSsS1_+0x3f5) [0x7ff34189ffe5]
 /usr/bin/mongod(_ZN5mongo11ProcessInfo10SystemInfo17collectSystemInfoEv+0x193) [0x7ff34189ea63]
 /usr/bin/mongod(_ZN5mongo36_mongoInitializerFunction_SystemInfoEPNS_18InitializerContextE+0xb9) [0x7ff34189d5e9]
 /usr/bin/mongod(_ZN5boost6detail8function17function_invoker1IPFN5mongo6StatusEPNS3_18InitializerContextEES4_S6_E6invokeERNS1_15function_bufferES6_+0xc) [0x7ff3412668ec]
 /usr/bin/mongod(_ZNK5mongo11Initializer7executeERKSt6vectorISsSaISsEERKSt3mapISsSsSt4lessISsESaISt4pairIKSsSsEEE+0x23a) [0x7ff3412742fa]
 /usr/bin/mongod(_ZN5mongo21runGlobalInitializersEiPKPKcS3_+0x32f) [0x7ff341274e1f]
 /usr/bin/mongod(main+0x91) [0x7ff3412316a1]
 /lib64/libc.so.6(__libc_start_main+0xf5) [0x7ff33d951af5]
 /usr/bin/mongod(+0x2817f9) [0x7ff3412617f9]




======== END OF STDOUT ========
2015-06-29 22:16:23::DEBUG::sequences::52::root:: Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/packstack/installer/core/sequences.py", line 50, in run
    self.function(config, messages)
  File "/usr/lib/python2.7/site-packages/packstack/plugins/puppet_950.py", line 210, in apply_puppet_manifest
    wait_for_puppet(currently_running, messages)
  File "/usr/lib/python2.7/site-packages/packstack/plugins/puppet_950.py", line 124, in wait_for_puppet
    validate_logfile(log)
  File "/usr/lib/python2.7/site-packages/packstack/modules/puppet.py", line 107, in validate_logfile
    raise PuppetError(message)
PuppetError: Error appeared during Puppet run: 172.16.0.167_mongodb.pp
Error: Could not start Service[mongodb]: Execution of '/usr/bin/systemctl start mongod' returned 1: Job for mongod.service failed. See 'systemctl status mongod.service' and 'journalctl -xn' for details.
You will find full trace in log /var/tmp/packstack/20150629-214736-OB5p5v/manifests/172.16.0.167_mongodb.pp.log

2015-06-29 22:16:23::ERROR::run_setup::1018::root:: Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/packstack/installer/run_setup.py", line 1013, in main
    _main(options, confFile, logFile)
  File "/usr/lib/python2.7/site-packages/packstack/installer/run_setup.py", line 655, in _main
    runSequences()
  File "/usr/lib/python2.7/site-packages/packstack/installer/run_setup.py", line 626, in runSequences
    controller.runAllSequences()
  File "/usr/lib/python2.7/site-packages/packstack/installer/setup_controller.py", line 81, in runAllSequences
    sequence.run(config=self.CONF, messages=self.MESSAGES)
  File "/usr/lib/python2.7/site-packages/packstack/installer/core/sequences.py", line 109, in run
    step.run(config=config, messages=messages)
  File "/usr/lib/python2.7/site-packages/packstack/installer/core/sequences.py", line 50, in run
    self.function(config, messages)
  File "/usr/lib/python2.7/site-packages/packstack/plugins/puppet_950.py", line 210, in apply_puppet_manifest
    wait_for_puppet(currently_running, messages)
  File "/usr/lib/python2.7/site-packages/packstack/plugins/puppet_950.py", line 124, in wait_for_puppet
    validate_logfile(log)
  File "/usr/lib/python2.7/site-packages/packstack/modules/puppet.py", line 107, in validate_logfile
    raise PuppetError(message)
PuppetError: Error appeared during Puppet run: 172.16.0.167_mongodb.pp
Error: Could not start Service[mongodb]: Execution of '/usr/bin/systemctl start mongod' returned 1: Job for mongod.service failed. See 'systemctl status mongod.service' and 'journalctl -xn' for details.
You will find full trace in log /var/tmp/packstack/20150629-214736-OB5p5v/manifests/172.16.0.167_mongodb.pp.log

Comment 2 wes hayutin 2015-06-30 01:31:42 UTC
Seems like this could just be a mongo issue

Comment 3 Javier Peña 2015-06-30 16:33:12 UTC
I have found the following avc in /var/log/audit/audit.log:

type=AVC msg=audit(1435669284.687:13409): avc:  denied  { read } for  pid=22312 comm="mongod" name="system-release" dev="vda1" ino=158 scontext=system_u:system_r:mongod_t:s0 tcontext=system_u:object_r:unlabeled_t:s0 tclass=lnk_file

This seems to match the mongod process stack trace, since it was trying to determine the Linux distribution when it failed.

The strange thing about this is that /etc/system-release should have a different SELinux label (etc_t instead of unlabeled_t).

Comment 5 Chandan Kumar 2016-05-19 15:37:10 UTC
This bug is against a Version which has reached End of Life.
If it's still present in supported release (http://releases.openstack.org), please update Version and reopen.


Note You need to log in before you can comment on or make changes to this bug.