Bug 1035937

Summary:	Packstack fails on Ceilometer db sync because it expects mongodb instance which is not running
Product:	Red Hat OpenStack	Reporter:	Rami Vaknin <rvaknin>
Component:	openstack-packstack	Assignee:	Ivan Chavero <ichavero>
Status:	CLOSED NOTABUG	QA Contact:	Nir Magnezi <nmagnezi>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	4.0	CC:	aortega, derekh, hateya, ichavero, oblaut, rvaknin, yeylon
Target Milestone:	---	Keywords:	Regression, Triaged
Target Release:	4.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2013-12-02 22:59:40 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:

Description Rami Vaknin 2013-11-28 22:54:53 UTC

Version
=======
rhos 4.0 on rhel 6.5 with puddle 2013-11-26.3
openstack-packstack-2013.2.1-0.11.dev847.el6ost

# rpm -qa | grep -i mongo
mongodb24-runtime-1-8.el6_3.x86_64
mongodb-2.2.4-3.el6ost.x86_64
mongodb24-libunwind-1.1-2.el6.x86_64
libmongodb-2.2.4-3.el6ost.x86_64
mongodb-server-2.2.4-3.el6ost.x86_64
python-pymongo-2.5.2-3.el6ost.x86_64
mongodb24-gperftools-libs-2.0-12.el6.x86_64


Description
===========
When Ceilometer is enabled, Packstack is trying to run db sync to a mongodb port which nobody listen on.
No running mongodb instances on the system, **neither iptables port open for such connections** (future gotcha).


From packstack's log
====================

ESC[0;32mInfo: /Stage[main]/Ceilometer::Db/Ceilometer_config[DEFAULT/database_connection]: Scheduling refresh of Exec[ceilometer-dbsync
ESC[0;36mDebug: Exec[ceilometer-dbsync](provider=posix): Executing 'ceilometer-dbsync --config-file=/etc/ceilometer/ceilometer.conf
ESC[0;36mDebug: Executing 'ceilometer-dbsync --config-file=/etc/ceilometer/ceilometer.conf
ESC[mNotice: /Stage[main]/Ceilometer::Db/Exec[ceilometer-dbsync]/returns: 2013-11-29 00:21:20.657 11269 CRITICAL ceilometer [-] could not connect to localhost:27017: [Errno 111]
 ECONNREFUSED
ESC[1;31mError: /Stage[main]/Ceilometer::Db/Exec[ceilometer-dbsync]: Failed to call refresh: ceilometer-dbsync --config-file=/etc/ceilometer/ceilometer.conf returned 1 instead o
f one of [0]
ESC[1;31mError: /Stage[main]/Ceilometer::Db/Exec[ceilometer-dbsync]: ceilometer-dbsync --config-file=/etc/ceilometer/ceilometer.conf returned 1 instead of one of [0]



Running the command manually
============================

# ceilometer-dbsync --config-file=/etc/ceilometer/ceilometer.conf
2013-11-29 00:38:57.715 15227 INFO ceilometer.storage.impl_mongodb [-] connecting to MongoDB on mongodb://localhost:27017/ceilometer
2013-11-29 00:38:57.742 15227 CRITICAL ceilometer [-] could not connect to localhost:27017: [Errno 111] ECONNREFUSED
2013-11-29 00:38:57.742 15227 TRACE ceilometer Traceback (most recent call last):
2013-11-29 00:38:57.742 15227 TRACE ceilometer   File "/usr/bin/ceilometer-dbsync", line 10, in <module>
2013-11-29 00:38:57.742 15227 TRACE ceilometer     sys.exit(dbsync())
2013-11-29 00:38:57.742 15227 TRACE ceilometer   File "/usr/lib/python2.6/site-packages/ceilometer/storage/__init__.py", line 139, in dbsync
2013-11-29 00:38:57.742 15227 TRACE ceilometer     get_connection(cfg.CONF).upgrade()
2013-11-29 00:38:57.742 15227 TRACE ceilometer   File "/usr/lib/python2.6/site-packages/ceilometer/storage/__init__.py", line 81, in get_connection
2013-11-29 00:38:57.742 15227 TRACE ceilometer     return get_engine(conf).get_connection(conf)
2013-11-29 00:38:57.742 15227 TRACE ceilometer   File "/usr/lib/python2.6/site-packages/ceilometer/storage/impl_mongodb.py", line 76, in get_connection
2013-11-29 00:38:57.742 15227 TRACE ceilometer     return Connection(conf)
2013-11-29 00:38:57.742 15227 TRACE ceilometer   File "/usr/lib/python2.6/site-packages/ceilometer/storage/impl_mongodb.py", line 311, in __init__
2013-11-29 00:38:57.742 15227 TRACE ceilometer     self.conn = self.CONNECTION_POOL.connect(url)
2013-11-29 00:38:57.742 15227 TRACE ceilometer   File "/usr/lib/python2.6/site-packages/ceilometer/storage/impl_mongodb.py", line 161, in connect
2013-11-29 00:38:57.742 15227 TRACE ceilometer     safe=True)
2013-11-29 00:38:57.742 15227 TRACE ceilometer   File "/usr/lib64/python2.6/site-packages/pymongo/mongo_client.py", line 337, in __init__
2013-11-29 00:38:57.742 15227 TRACE ceilometer     raise ConnectionFailure(str(e))
2013-11-29 00:38:57.742 15227 TRACE ceilometer ConnectionFailure: could not connect to localhost:27017: [Errno 111] ECONNREFUSED
2013-11-29 00:38:57.742 15227 TRACE ceilometer 



# ps -elf | grep -i mongo | grep -v grep
<empty_output>

Comment 1 Ivan Chavero 2013-11-29 02:32:28 UTC

Can you confirm with a yum info mongodb if mongodb is installed, apparently the ceilometer module is assuming that python-pymongo installs mongodb as dependency which its not true:

yum deplist python-pymongo

Loading mirror speeds from cached hostfile
 * fedora: mirror.hmc.edu
 * rpmfusion-free-rawhide: mirror.web-ster.com
 * rpmfusion-nonfree-rawhide: mirror.web-ster.com
 * updates: mirrors.usc.edu
 * updates-testing: mirrors.usc.edu
paquete: python-pymongo.x86_64 2.5.2-3.fc20
 dependencia: libc.so.6(GLIBC_2.14)(64bit)
   provider: glibc.x86_64 2.18-11.fc20
 dependencia: libpthread.so.0()(64bit)
   provider: glibc.x86_64 2.18-11.fc20
 dependencia: libpython2.7.so.1.0()(64bit)
   provider: python-libs.x86_64 2.7.5-9.fc20
 dependencia: python(abi) = 2.7
   provider: python.x86_64 2.7.5-9.fc20
   provider: python.i686 2.7.5-9.fc20
 dependencia: python-bson = 2.5.2-3.fc20
   provider: python-bson.x86_64 2.5.2-3.fc20
 dependencia: rtld(GNU_HASH)
   provider: glibc.x86_64 2.18-11.fc20
   provider: glibc.i686 2.18-11.fc20

Comment 2 Ivan Chavero 2013-11-29 02:36:51 UTC

The packstack ceilometer puppet manifest calls a mongodb class,
this should install it.

Comment 3 Ivan Chavero 2013-11-29 07:38:09 UTC

Tested the package in a clean environment of RHOS 4.0 in RHEL 6.5 and it worked correctly, mongodb is running and the command: ceilometer-dbsync --config-file=/etc/ceilometer/ceilometer.conf does not give the error mentioned above.

Can you test in a clean environment?

Confirmed that ceilometer is working correctly:

# ceilometer meter-list
+----------------------------+-------+-----------+--------------------------------------+---------+----------------------------------+
| Name                       | Type  | Unit      | Resource ID                          | User ID | Project ID                       |
+----------------------------+-------+-----------+--------------------------------------+---------+----------------------------------+
| image                      | gauge | image     | 48c97b30-97e1-4818-98c9-81b44b353888 | None    | 70fdad4f0df7421382e16830ec047aec |
| image.size                 | gauge | B         | 48c97b30-97e1-4818-98c9-81b44b353888 | None    | 70fdad4f0df7421382e16830ec047aec |
| storage.objects            | gauge | object    | 3547751a0f4e41349ccc35fd4a2c4217     | None    | 3547751a0f4e41349ccc35fd4a2c4217 |
| storage.objects            | gauge | object    | 5ec3b6f1b67c49f29b783474ea6cfe46     | None    | 5ec3b6f1b67c49f29b783474ea6cfe46 |
| storage.objects            | gauge | object    | 70fdad4f0df7421382e16830ec047aec     | None    | 70fdad4f0df7421382e16830ec047aec |
| storage.objects            | gauge | object    | 874779f5eddb4e91b9802a0956ab5960     | None    | 874779f5eddb4e91b9802a0956ab5960 |
| storage.objects.containers | gauge | container | 3547751a0f4e41349ccc35fd4a2c4217     | None    | 3547751a0f4e41349ccc35fd4a2c4217 |
| storage.objects.containers | gauge | container | 5ec3b6f1b67c49f29b783474ea6cfe46     | None    | 5ec3b6f1b67c49f29b783474ea6cfe46 |
| storage.objects.containers | gauge | container | 70fdad4f0df7421382e16830ec047aec     | None    | 70fdad4f0df7421382e16830ec047aec |
| storage.objects.containers | gauge | container | 874779f5eddb4e91b9802a0956ab5960     | None    | 874779f5eddb4e91b9802a0956ab5960 |
| storage.objects.size       | gauge | B         | 3547751a0f4e41349ccc35fd4a2c4217     | None    | 3547751a0f4e41349ccc35fd4a2c4217 |
| storage.objects.size       | gauge | B         | 5ec3b6f1b67c49f29b783474ea6cfe46     | None    | 5ec3b6f1b67c49f29b783474ea6cfe46 |
| storage.objects.size       | gauge | B         | 70fdad4f0df7421382e16830ec047aec     | None    | 70fdad4f0df7421382e16830ec047aec |
| storage.objects.size       | gauge | B         | 874779f5eddb4e91b9802a0956ab5960     | None    | 874779f5eddb4e91b9802a0956ab5960 |
+----------------------------+-------+-----------+--------------------------------------+---------+----------------------------------+


confirm that mongodb is working properly:

# curl http://localhost:27017/ceilometer
You are trying to access MongoDB on the native driver port. For http diagnostic access, add 1000 to the port number

Comment 4 Alvaro Lopez Ortega 2013-11-29 21:19:15 UTC

It doesn't seem to be a valid bug. rvaknin, could you please confirm it?

Comment 5 Rami Vaknin 2013-11-30 06:37:08 UTC

(In reply to Alvaro Lopez Ortega from comment #4)
> It doesn't seem to be a valid bug. rvaknin, could you please confirm it?

I definitely can NOT confirm that, I got it reproduced 2 out of 2 trials.
mongodb is installed, that's not the issue, the issue is mongodb is not running so no one listen on 27017.

The reason:
/usr/bin/mongod: error while loading shared libraries: libtcmalloc.so.4: cannot open shared object file: No such file or directory

More debugging (the env with reproduction is {still} available, please contact me for more info):

[root@puma10 ~]# ceilometer-dbsync --config-file=/etc/ceilometer/ceilometer.conf
2013-11-30 08:25:18.661 15056 INFO ceilometer.storage.impl_mongodb [-] connecting to MongoDB on mongodb://localhost:27017/ceilometer
2013-11-30 08:25:18.676 15056 CRITICAL ceilometer [-] could not connect to localhost:27017: [Errno 111] ECONNREFUSED
2013-11-30 08:25:18.676 15056 TRACE ceilometer Traceback (most recent call last):
2013-11-30 08:25:18.676 15056 TRACE ceilometer   File "/usr/bin/ceilometer-dbsync", line 10, in <module>
2013-11-30 08:25:18.676 15056 TRACE ceilometer     sys.exit(dbsync())
2013-11-30 08:25:18.676 15056 TRACE ceilometer   File "/usr/lib/python2.6/site-packages/ceilometer/storage/__init__.py", line 139, in dbsync
2013-11-30 08:25:18.676 15056 TRACE ceilometer     get_connection(cfg.CONF).upgrade()
2013-11-30 08:25:18.676 15056 TRACE ceilometer   File "/usr/lib/python2.6/site-packages/ceilometer/storage/__init__.py", line 81, in get_connection
2013-11-30 08:25:18.676 15056 TRACE ceilometer     return get_engine(conf).get_connection(conf)
2013-11-30 08:25:18.676 15056 TRACE ceilometer   File "/usr/lib/python2.6/site-packages/ceilometer/storage/impl_mongodb.py", line 76, in get_connection
2013-11-30 08:25:18.676 15056 TRACE ceilometer     return Connection(conf)
2013-11-30 08:25:18.676 15056 TRACE ceilometer   File "/usr/lib/python2.6/site-packages/ceilometer/storage/impl_mongodb.py", line 311, in __init__
2013-11-30 08:25:18.676 15056 TRACE ceilometer     self.conn = self.CONNECTION_POOL.connect(url)
2013-11-30 08:25:18.676 15056 TRACE ceilometer   File "/usr/lib/python2.6/site-packages/ceilometer/storage/impl_mongodb.py", line 161, in connect
2013-11-30 08:25:18.676 15056 TRACE ceilometer     safe=True)
2013-11-30 08:25:18.676 15056 TRACE ceilometer   File "/usr/lib64/python2.6/site-packages/pymongo/mongo_client.py", line 337, in __init__
2013-11-30 08:25:18.676 15056 TRACE ceilometer     raise ConnectionFailure(str(e))
2013-11-30 08:25:18.676 15056 TRACE ceilometer ConnectionFailure: could not connect to localhost:27017: [Errno 111] ECONNREFUSED
2013-11-30 08:25:18.676 15056 TRACE ceilometer 
[root@puma10 ~]# lsof -i -Pn | grep 270
[root@puma10 ~]# yum info mongodb
Loaded plugins: priorities, product-id, rhnplugin, security, subscription-manager
This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
This system is receiving updates from RHN Classic or RHN Satellite.
Installed Packages
Name        : mongodb
Arch        : x86_64
Version     : 2.2.4
Release     : 3.el6ost
Size        : 79 M
Repo        : installed
From repo   : puddle
Summary     : High-performance, schema-free document-oriented database
URL         : http://www.mongodb.org
License     : AGPLv3 and zlib and ASL 2.0
Description : Mongo (from "humongous") is a high-performance, open source, schema-free
            : document-oriented database. MongoDB is written in C++ and offers the following
            : features:
            :     * Collection oriented storage: easy storage of object/JSON-style data
            :     * Dynamic queries
            :     * Full index support, including on inner objects and embedded arrays
            :     * Query profiling
            :     * Replication and fail-over support
            :     * Efficient storage of binary data including large objects (e.g. photos
            :     and videos)
            :     * Auto-sharding for cloud-level scalability (currently in early alpha)
            :     * Commercial Support Available
            : 
            : A key goal of MongoDB is to bridge the gap between key/value stores (which are
            : fast and highly scalable) and traditional RDBMS systems (which are deep in
            : functionality).

[root@puma10 ~]# rpm -qa | grep -i mongo
mongodb24-runtime-1-8.el6_3.x86_64
mongodb-2.2.4-3.el6ost.x86_64
mongodb24-libunwind-1.1-2.el6.x86_64
libmongodb-2.2.4-3.el6ost.x86_64
mongodb-server-2.2.4-3.el6ost.x86_64
python-pymongo-2.5.2-3.el6ost.x86_64
mongodb24-gperftools-libs-2.0-12.el6.x86_64
[root@puma10 ~]# yum deplist python-pymongo
Loaded plugins: priorities, product-id, rhnplugin, security, subscription-manager
This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
This system is receiving updates from RHN Classic or RHN Satellite.
Finding dependencies: 
package: python-pymongo.x86_64 2.5.2-3.el6ost
  dependency: libc.so.6()(64bit)
   provider: glibc.x86_64 2.12-1.130.el6
   provider: glibc.x86_64 2.12-1.7.el6
   provider: glibc.x86_64 2.12-1.7.el6_0.3
   provider: glibc.x86_64 2.12-1.7.el6_0.4
   provider: glibc.x86_64 2.12-1.7.el6_0.5
   provider: glibc.x86_64 2.12-1.25.el6
   provider: glibc.x86_64 2.12-1.25.el6_1.3
   provider: glibc.x86_64 2.12-1.47.el6
   provider: glibc.x86_64 2.12-1.47.el6_2.5
   provider: glibc.x86_64 2.12-1.47.el6_2.9
   provider: glibc.x86_64 2.12-1.47.el6_2.12
   provider: glibc.x86_64 2.12-1.80.el6
   provider: glibc.x86_64 2.12-1.80.el6_3.3
   provider: glibc.x86_64 2.12-1.80.el6_3.4
   provider: glibc.x86_64 2.12-1.80.el6_3.5
   provider: glibc.x86_64 2.12-1.80.el6_3.6
   provider: glibc.x86_64 2.12-1.80.el6_3.7
   provider: glibc.x86_64 2.12-1.107.el6
   provider: glibc.x86_64 2.12-1.107.el6_4.2
   provider: glibc.x86_64 2.12-1.107.el6_4.4
   provider: glibc.x86_64 2.12-1.107.el6_4.5
   provider: glibc.x86_64 2.12-1.132.el6
  dependency: rtld(GNU_HASH)
   provider: glibc.x86_64 2.12-1.130.el6
   provider: glibc.i686 2.12-1.130.el6
   provider: glibc.i686 2.12-1.7.el6
   provider: glibc.x86_64 2.12-1.7.el6
   provider: glibc.i686 2.12-1.7.el6_0.3
   provider: glibc.x86_64 2.12-1.7.el6_0.3
   provider: glibc.i686 2.12-1.7.el6_0.4
   provider: glibc.x86_64 2.12-1.7.el6_0.4
   provider: glibc.i686 2.12-1.7.el6_0.5
   provider: glibc.x86_64 2.12-1.7.el6_0.5
   provider: glibc.x86_64 2.12-1.25.el6
   provider: glibc.i686 2.12-1.25.el6
   provider: glibc.i686 2.12-1.25.el6_1.3
   provider: glibc.x86_64 2.12-1.25.el6_1.3
   provider: glibc.i686 2.12-1.47.el6
   provider: glibc.x86_64 2.12-1.47.el6
   provider: glibc.i686 2.12-1.47.el6_2.5
   provider: glibc.x86_64 2.12-1.47.el6_2.5
   provider: glibc.i686 2.12-1.47.el6_2.9
   provider: glibc.x86_64 2.12-1.47.el6_2.9
   provider: glibc.i686 2.12-1.47.el6_2.12
   provider: glibc.x86_64 2.12-1.47.el6_2.12
   provider: glibc.x86_64 2.12-1.80.el6
   provider: glibc.i686 2.12-1.80.el6
   provider: glibc.i686 2.12-1.80.el6_3.3
   provider: glibc.x86_64 2.12-1.80.el6_3.3
   provider: glibc.i686 2.12-1.80.el6_3.4
   provider: glibc.x86_64 2.12-1.80.el6_3.4
   provider: glibc.i686 2.12-1.80.el6_3.5
   provider: glibc.x86_64 2.12-1.80.el6_3.5
   provider: glibc.i686 2.12-1.80.el6_3.6
   provider: glibc.x86_64 2.12-1.80.el6_3.6
   provider: glibc.i686 2.12-1.80.el6_3.7
   provider: glibc.x86_64 2.12-1.80.el6_3.7
   provider: glibc.i686 2.12-1.107.el6
   provider: glibc.x86_64 2.12-1.107.el6
   provider: glibc.i686 2.12-1.107.el6_4.2
   provider: glibc.x86_64 2.12-1.107.el6_4.2
   provider: glibc.i686 2.12-1.107.el6_4.4
   provider: glibc.x86_64 2.12-1.107.el6_4.4
   provider: glibc.i686 2.12-1.107.el6_4.5
   provider: glibc.x86_64 2.12-1.107.el6_4.5
   provider: glibc.x86_64 2.12-1.132.el6
   provider: glibc.i686 2.12-1.132.el6
  dependency: python(abi) = 2.6
   provider: python.x86_64 2.6.6-51.el6
   provider: python.x86_64 2.6.5-3.el6
   provider: python.i686 2.6.5-3.el6
   provider: python.i686 2.6.5-3.el6_0.2
   provider: python.x86_64 2.6.5-3.el6_0.2
   provider: python.x86_64 2.6.6-20.el6
   provider: python.x86_64 2.6.6-29.el6
   provider: python.x86_64 2.6.6-29.el6_2.2
   provider: python.x86_64 2.6.6-29.el6_3.3
   provider: python.x86_64 2.6.6-36.el6
   provider: python.x86_64 2.6.6-37.el6_4
   provider: python.x86_64 2.6.6-51.el6
  dependency: libpython2.6.so.1.0()(64bit)
   provider: python-libs.x86_64 2.6.6-51.el6
   provider: python-libs.x86_64 2.6.5-3.el6
   provider: python-libs.x86_64 2.6.5-3.el6_0.2
   provider: python-libs.x86_64 2.6.6-20.el6
   provider: python-libs.x86_64 2.6.6-29.el6
   provider: python-libs.x86_64 2.6.6-29.el6_2.2
   provider: python-libs.x86_64 2.6.6-29.el6_3.3
   provider: python-libs.x86_64 2.6.6-36.el6
   provider: python-libs.x86_64 2.6.6-37.el6_4
   provider: python-libs.x86_64 2.6.6-51.el6
  dependency: python-bson = 2.5.2-3.el6ost
   provider: python-bson.x86_64 2.5.2-3.el6ost
  dependency: libpthread.so.0()(64bit)
   provider: glibc.x86_64 2.12-1.130.el6
   provider: glibc.x86_64 2.12-1.7.el6
   provider: glibc.x86_64 2.12-1.7.el6_0.3
   provider: glibc.x86_64 2.12-1.7.el6_0.4
   provider: glibc.x86_64 2.12-1.7.el6_0.5
   provider: glibc.x86_64 2.12-1.25.el6
   provider: glibc.x86_64 2.12-1.25.el6_1.3
   provider: glibc.x86_64 2.12-1.47.el6
   provider: glibc.x86_64 2.12-1.47.el6_2.5
   provider: glibc.x86_64 2.12-1.47.el6_2.9
   provider: glibc.x86_64 2.12-1.47.el6_2.12
   provider: glibc.x86_64 2.12-1.80.el6
   provider: glibc.x86_64 2.12-1.80.el6_3.3
   provider: glibc.x86_64 2.12-1.80.el6_3.4
   provider: glibc.x86_64 2.12-1.80.el6_3.5
   provider: glibc.x86_64 2.12-1.80.el6_3.6
   provider: glibc.x86_64 2.12-1.80.el6_3.7
   provider: glibc.x86_64 2.12-1.107.el6
   provider: glibc.x86_64 2.12-1.107.el6_4.2
   provider: glibc.x86_64 2.12-1.107.el6_4.4
   provider: glibc.x86_64 2.12-1.107.el6_4.5
   provider: glibc.x86_64 2.12-1.132.el6
  dependency: libc.so.6(GLIBC_2.2.5)(64bit)
   provider: glibc.x86_64 2.12-1.130.el6
   provider: glibc.x86_64 2.12-1.7.el6
   provider: glibc.x86_64 2.12-1.7.el6_0.3
   provider: glibc.x86_64 2.12-1.7.el6_0.4
   provider: glibc.x86_64 2.12-1.7.el6_0.5
   provider: glibc.x86_64 2.12-1.25.el6
   provider: glibc.x86_64 2.12-1.25.el6_1.3
   provider: glibc.x86_64 2.12-1.47.el6
   provider: glibc.x86_64 2.12-1.47.el6_2.5
   provider: glibc.x86_64 2.12-1.47.el6_2.9
   provider: glibc.x86_64 2.12-1.47.el6_2.12
   provider: glibc.x86_64 2.12-1.80.el6
   provider: glibc.x86_64 2.12-1.80.el6_3.3
   provider: glibc.x86_64 2.12-1.80.el6_3.4
   provider: glibc.x86_64 2.12-1.80.el6_3.5
   provider: glibc.x86_64 2.12-1.80.el6_3.6
   provider: glibc.x86_64 2.12-1.80.el6_3.7
   provider: glibc.x86_64 2.12-1.107.el6
   provider: glibc.x86_64 2.12-1.107.el6_4.2
   provider: glibc.x86_64 2.12-1.107.el6_4.4
   provider: glibc.x86_64 2.12-1.107.el6_4.5
   provider: glibc.x86_64 2.12-1.132.el6
[root@puma10 ~]# curl http://localhost:27017/ceilometer
curl: (7) couldn't connect to host
[root@puma10 ~]# /etc/init.d/mongod status
mongod dead but subsys locked
[root@puma10 ~]# cat /var/log/mongodb/mongodb.log
/usr/bin/mongod: error while loading shared libraries: libtcmalloc.so.4: cannot open shared object file: No such file or directory
[root@puma10 ~]# find / 2> /dev/null | grep libtcmalloc.so.4
/opt/rh/mongodb24/root/usr/lib64/libtcmalloc.so.4
/opt/rh/mongodb24/root/usr/lib64/libtcmalloc.so.4.1.0
[root@puma10 ~]# rpm -qf /opt/rh/mongodb24/root/usr/lib64/libtcmalloc.so.4
mongodb24-gperftools-libs-2.0-12.el6.x86_64
[root@puma10 ~]#

Comment 6 Ivan Chavero 2013-12-02 22:59:40 UTC

According to the latest information you privided we conclude this is not a packstack or openstack-puppet-modules bug. If anything this would be a MongoDB packaging issue. Thus, I'm closing it. Please, do not hesitate to reopen it if you believe I missed anything.

Comment 7 Rami Vaknin 2013-12-03 01:21:28 UTC

I tend to agree this might count as a packaging issue - maybe wrong yum repository file, however, I don't see how connections from ceilometer to mongodb would be able pass anyway (assuming iptables is enforced, although lately I saw a bug that iptables is not enforced), I couldn't find any iptables rule allowing such connections, I assume they should be added after this installation step, could you please confirm you have such rule(s) in your env?

Comment 8 Ivan Chavero 2014-06-09 15:26:54 UTC

the firewall rules for mongodb are created by packstack