Description of problem: In bug #1020449, the basic Swift services (proxy, account, container, object) didn't start up because of an error. Yet Packstack didn't warn about this and finished successfully. I didn't even find a related error or warning message in the log file. Version-Release number of selected component (if applicable): The above mentioned error happened on RHEL 6.5 with openstack-packstack-2013.2.1-0.6.dev763.el6ost.noarch, openstack-swift-1.10.0-0.2.rc1.el6ost.noarch How reproducible: always Steps to Reproduce: 1. use RDO Havana or RHOS4 where Swift fails to start (openstack-swift-1.10.0-0.2) 2. packstack --allinone 3. try using swift - everything will fail with "connection refused" or look at "swift-init status all" - you'll see that the proxy server isn't running. Actual results: packstack finishes without a complaint, but the main Swift services won't be running: # swift-init status all container-updater running (20439 - /etc/swift/container-server.conf) account-auditor running (19937 - /etc/swift/account-server.conf) object-replicator running (20411 - /etc/swift/object-server.conf) No proxy-server running container-replicator running (20191 - /etc/swift/container-server.conf) object-auditor running (19965 - /etc/swift/object-server.conf) No object-expirer running container-auditor running (19869 - /etc/swift/container-server.conf) No container-server running No account-server running account-reaper running (19897 - /etc/swift/account-server.conf) No container-sync running account-replicator running (20277 - /etc/swift/account-server.conf) object-updater running (20124 - /etc/swift/object-server.conf) No object-server running Expected results: Packstack should fail with an error message about this or at least warn about it. Additional info:
I think this happened to me previously with a different error, but I no longer have the logs for that. However, in this case, it's probably Swift's fault: # swift-init proxy start; echo $? Starting proxy-server...(/etc/swift/proxy-server.conf) Error trying to load config from /etc/swift/proxy-server.conf: xattr>=0.4 0 # swift-init proxy status; echo $? No proxy-server running 1 The Swift service didn't start, yet returned 0. Maybe it would make sense to check the status of the services and not just the exit code of the 'start' command.
Created attachment 828046 [details] Possible fix
ok, so this won't help with sysv initscripts, same as other openstack initscripts ( see bug 1036515 comment 10 ) in openstack-swift-functions swift_start() we have: daemon --pidfile $pid_file \ "/usr/bin/swift-$long_name $conf_file &>/var/log/swift-startup.log & echo \$! > $pid_file" which returns immediately.
That's true, however even if we, for instance, switch initscripts from doing shell background with '&' to relying on daemonization (Swift has that like most other daemons), then we would need the return code fixed. What Martina pointed out in comment #2 is symptom of the same problem.
Failed QA: I removed the proxy server configuration and tried restarting it. It failed, but exited with 0. # mv /etc/swift/proxy-server.{conf,conf-bak} # swift-init proxy restart Signal proxy-server pid: 26929 signal: 15 proxy-server (26929) appears to have stopped Unable to locate config for proxy-server # echo $? 0 # rpm -qa|grep swift openstack-swift-1.10.0-3.el6ost.noarch openstack-swift-plugin-swift3-1.0.0-0.20120711git.1.el6ost.noarch python-swiftclient-2.0.2-2.el6ost.noarch openstack-swift-account-1.10.0-3.el6ost.noarch openstack-swift-object-1.10.0-3.el6ost.noarch openstack-swift-proxy-1.10.0-3.el6ost.noarch openstack-swift-container-1.10.0-3.el6ost.noarch Are you sure that the patch is in openstack-swift-1.10.0-3? The package is a few months old.
> Are you sure that the patch is in openstack-swift-1.10.0-3? Yes, 0005-Return-an-exit-code-for-configuration-errors.patch is in openstack-swift-1.10.0-3.el6ost so it must be something else, looking...
Alan, Martina - sorry I did not think of restart. So the restart function looks like this in openstack-swift-proxy.init: restart() { stop start } With my fix, start function returns the error in exit code, but restart function ignores it. I thought in bash the return value was the last exit code, but perhaps not.
Never mind, I know what's up. The bug was originally filed for the config file being invalid and not loading. That was fixed. However, if the config file is plain missing, then... magic happens. The swift.common.manager.Manager.launch() returns [], roundly ignored by the start command, and we get where we are. Bash does not get into picture at all, because Martina used swift-init directly.
I tried this with existing but invalid configuration. Swift init works as expected, but the initscripts say everything is alright. I tried this with the proxy and object server, with the same results. I damaged the configuration by removing the line "use = egg:swift#proxy" # service openstack-swift-proxy start Starting openstack-swift-proxy: [ OK ] # echo $? 0 # service openstack-swift-proxy status openstack-swift-proxy dead but pid file exists # swift-init proxy start Starting proxy-server...(/etc/swift/proxy-server.conf) Error trying to load config from /etc/swift/proxy-server.conf: No loader given in section 'app:proxy-server' # echo $? 1 Should I open a new bug for this, given that this one is about swift-init?
I've found that is caused by the bug #1036515. As for the scope of the original report, this is fixed. Though I'm not very happy it returns 0 even if the config file doesn't exist.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHBA-2014-0577.html