Red Hat Bugzilla – Bug 976081
signing_dir must not be /etc/swift
Last modified: 2014-01-12 19:57:06 EST
Description of problem:
According to data captured for bug 967631, packstack --allinone creates
the following configuration in proxy-server.conf:
signing_dir = /etc/swift
This results in Swift denying access to its own processes and failing
with a looping crash.
Version-Release number of selected component (if applicable):
Steps to Reproduce:
1. packstack --allinone
2. grep signing_dir /etc/swift/proxy-server.conf
Swift blows up
One insiduous problem here is that the fateful chmod occurs when Swift
starts. Therefore, you can start it ONCE, test that it works. But if
you restart it, system becomes almost inaccessible with consoles
flooded by looping crash tracebacks.
The openstack-swift RPM ships with signing_dir /tmp/something-something.
Perhaps someone thought it was not secure enough. If that is a concern,
we must package a separate directory in /var/run and use /etc/tmpfiles.d
to establish proper permissions. Note that this is different on systemd
and Upstart systems like RHEL.
Alan pointed out that this is a problem not in Packstack as such, but in
one of upstream Puppet modules, here:
Proper fix is not to set signing_dir at all so upstream default applies (I didn't see any justification in puppet-swift why siging_dir
In keystoneclinet master that was recently changed to random tempdir:
but in RHOS3 "stable/grizzly" we still have old default ~/keystone-signing:
$HOME for swift account is /var/lib/swift but this folder is not include in the RPM, so you get:
File "/usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py", line 306, in __init__
File "/usr/lib64/python2.6/os.py", line 150, in makedirs
File "/usr/lib64/python2.6/os.py", line 157, in makedirs
OSError: [Errno 13] Permission denied: '/var/lib/swift'
/var/lib/swift should be included in the openstack-swift RPM and/or backport https://github.com/openstack/python-keystoneclient/commit/03012e641d6c2a98fbfe3780102e28a65d11a887 to "stable/grizzly".
(In reply to Alan Pevec from comment #3)
> Proper fix is not to set signing_dir at all so upstream default applies (I
> didn't see any justification in puppet-swift why siging_dir
... was set at all).
So what do you suggest? Remove "signing_dir = /etc/swift" from authtoken.conf.erb?
(In reply to Martin Magr from comment #5)
> So what do you suggest? Remove "signing_dir = /etc/swift" from
Yes, but then also need to include /var/lib/swift in the openstack-swift RPM or update keystoneclient to include upstream fix mentioned in comment 3.
Created attachment 763330 [details]
> /var/lib/swift should be included in the openstack-swift RPM
fixed in openstack-swift-1.8.0-6.el6ost
Created attachment 763432 [details]
Signing patch mk2
Removed removing last empty line
Comment on attachment 763330 [details]
Needinfo: Martin, could you pls read Doc Text and let me know if that's OK. Thx.
verified on openstack-packstack-2013.1.1-0.20.dev632.el6ost
[root@vm-161-67 ~]# grep signing_dir /etc/swift/proxy-server.conf
swift is running:
[root@vm-161-67 ~]# ps aux | grep swift
root 1144 0.0 0.0 103236 832 pts/0 R+ 09:15 0:00 grep swift
swift 29649 0.0 0.4 237896 19580 ? Ss 09:01 0:00 /usr/bin/python /usr/bin/swift-proxy-server /etc/swift/proxy-server.conf
swift 29684 0.0 0.4 238760 18384 ? S 09:01 0:00 /usr/bin/python /usr/bin/swift-proxy-server /etc/swift/proxy-server.conf
swift 29776 0.0 0.3 228932 14192 ? Ss 09:01 0:00 /usr/bin/python /usr/bin/swift-account-auditor /etc/swift/account-server.conf
swift 29804 0.0 0.4 230076 15792 ? Ss 09:01 0:00 /usr/bin/python /usr/bin/swift-account-replicator /etc/swift/account-server.conf
swift 29835 0.0 0.3 229388 14796 ? Ss 09:01 0:00 /usr/bin/python /usr/bin/swift-account-server /etc/swift/account-server.conf
swift 29854 0.0 0.3 230008 13380 ? S 09:01 0:00 /usr/bin/python /usr/bin/swift-account-server /etc/swift/account-server.conf
swift 29857 0.0 0.3 228960 14360 ? Ss 09:01 0:00 /usr/bin/python /usr/bin/swift-account-reaper /etc/swift/account-server.conf
swift 29958 0.0 0.3 222288 14416 ? Ss 09:02 0:00 /usr/bin/python /usr/bin/swift-object-replicator /etc/swift/object-server.conf
swift 29988 0.0 0.3 228704 14524 ? Ss 09:02 0:00 /usr/bin/python /usr/bin/swift-object-server /etc/swift/object-server.conf
swift 30000 0.0 0.3 228836 11848 ? S 09:02 0:00 /usr/bin/python /usr/bin/swift-object-server /etc/swift/object-server.conf
swift 30011 0.0 0.3 229144 15044 ? Ss 09:02 0:00 /usr/bin/python /usr/bin/swift-object-updater /etc/swift/object-server.conf
swift 30044 0.0 0.3 227988 13900 ? Ss 09:02 0:00 /usr/bin/python /usr/bin/swift-object-auditor /etc/swift/object-server.conf
swift 30053 0.0 0.2 227988 11084 ? S 09:02 0:00 /usr/bin/python /usr/bin/swift-object-auditor /etc/swift/object-server.conf
swift 30142 0.0 0.3 230092 15480 ? Ss 09:02 0:00 /usr/bin/python /usr/bin/swift-container-updater /etc/swift/container-server.conf
swift 30173 0.0 0.3 229400 14812 ? Ss 09:02 0:00 /usr/bin/python /usr/bin/swift-container-server /etc/swift/container-server.conf
swift 30192 0.0 0.3 229740 12152 ? S 09:02 0:00 /usr/bin/python /usr/bin/swift-container-server /etc/swift/container-server.conf
swift 30207 0.0 0.3 228980 14200 ? Ss 09:02 0:00 /usr/bin/python /usr/bin/swift-container-auditor /etc/swift/container-server.conf
swift 30238 0.0 0.3 230084 15484 ? Ss 09:02 0:00 /usr/bin/python /usr/bin/swift-container-replicator /etc/swift/container-server.conf
Docs are OK, thanks Bruce.
BTW looks like the origin of signing_dir = /etc/swift was a bad advice in one upstream bug comment: https://bugs.launchpad.net/keystone/+bug/1036847/comments/7
From Adam's comment  it seems that correct directory should be /var/cache/swift. So what now? Correct that in packstack and let swift rpm to create such directory?
I think Adam is right. I never was enthusiastic about throwing signing_dir
into automatically created directories, although it works as a stopgap
measure against the major screwup with directing it to /etc/swift.
I'm going to open yet another bug against Swift. At worst we'll close it
Filed bug 978408 so Fedora is consistent with RHOS going forward.
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory, and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.