Description of problem: Upon updating from openstack-swift-1.8.0-3.el6ost to openstack-swift-1.8.0-4.el6ost using 'yum update -y', the following error starts occurring: Message from syslogd@localhost at Jun 17 09:43:15 ... �<130>proxy-server UNCAUGHT EXCEPTION#012Traceback (most recent call last):#012 File "/usr/bin/swift-proxy-server", line 22, in <module>#012 run_wsgi(conf_file, 'proxy-server', default_port=8080, **options)#012 File "/usr/lib/python2.6/site-packages/swift/common/wsgi.py", line 187, in run_wsgi#012 run_server()#012 File "/usr/lib/python2.6/site-packages/swift/common/wsgi.py", line 149, in run_server#012 global_conf={'log_name': log_name})#012 File "/usr/lib/python2.6/site-packages/PasteDeploy-1.5.0-py2.6.egg/paste/deploy/loadwsgi.py", line 247, in loadapp#012 return loadobj(APP, uri, name=name, **kw)#012 File "/usr/lib/python2.6/site-packages/PasteDeploy-1.5.0-py2.6.egg/paste/deploy/loadwsgi.py", line 271, in loadobj#012 global_conf=global_conf)#012 File "/usr/lib/python2.6/site-packages/PasteDeploy-1.5.0-py2.6.egg/paste/deploy/loadwsgi.py", line 296, in loadcontext#012 global_conf=global_conf)#012 File "/usr/lib/python2.6/site-packages/PasteDeploy-1.5.0-py2.6.egg/paste/deploy/loadwsgi.py", line 317, in _loadconfig#012 loader = ConfigLoader(path)#012 File "/usr/lib/python2.6/site-packages/PasteDeploy-1.5.0-py2.6.egg/paste/deploy/loadwsgi.py", line 393, in __init__#012 with open(filename) as f:#012IOError: [Errno 13] Permission denied: '/etc/swift/proxy-server.conf' The problem is two fold: - an update process probably should be a bit cleaner and not blow up like this, and - this should not be at LOG_EMERG level, or if it is, it should be tracked and quiesced after the first one or two Due to the nature that this is logged at LOG_EMERG (or perhaps LOG_ALERT) and logged in a VERY tight loop, my system is presently entirely unusable - all PTYs receive this message.
ssh $HOST "killall -9 swift-proxy-server" made the madness end.
*** This bug has been marked as a duplicate of bug 967631 ***
Lon, please don't touch anything, if you can. I was unable to extract the following two things from Jaroslav - How the wrong permissions occur in the first place. He alluded that "Swift changed the permissions", which sounds too crazy. A spec that specifies that in %dir? I can't find anything like that! So I need your /var/log/yum.log (I know that you mentioned precise rpm versions in the report, thanks a lot) - How the heck the console spam happens to begin with. We never dump at LOG_EMERG. Very puzzling! I thought his rsyslog config was screwed, but perhaps not?
If swift is not changing anything, how do you explain the following? [root@folsom-rhel6 ~]# ll -d /etc/swift drwx------. 4 swift swift 4096 16. čen 10.21 /etc/swift [root@folsom-rhel6 ~]# chmod u-r /etc/swift [root@folsom-rhel6 ~]# ll -d /etc/swift d-wx------. 4 swift swift 4096 16. čen 10.21 /etc/swift [root@folsom-rhel6 ~]# /etc/init.d/openstack-swift-proxy restart Stopping openstack-swift-proxy: [ OK ] Starting openstack-swift-proxy: [ OK ] [root@folsom-rhel6 ~]# ll -d /etc/swift drwx------. 4 swift swift 4096 16. čen 10.21 /etc/swift Yes, It is quite crazy, I agree.
Day's result 6/19, again on Lon's VM: rsyslog receives message with BOM, looking like this by strace: recvmsg(0, {msg_name(0)=NULL, msg_iov(1)=[{"\357\273\277<146>proxy-server UNCAUGHT EX"..., 2048}] ...... Perhaps there's a unicode-type string in the traceback, causing this behaviour. Still no clue why or how Keystone middleware triggers it. URL: http://docs.python.org/2/howto/logging-cookbook.html#inserting-a-bom-into-messages-sent-to-a-sysloghandler