Bug 976081 - signing_dir must not be /etc/swift
signing_dir must not be /etc/swift
Status: CLOSED ERRATA
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-packstack (Show other bugs)
3.0
Unspecified Unspecified
high Severity high
: rc
: 3.0
Assigned To: Martin Magr
Haim
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-06-19 16:42 EDT by Pete Zaitcev
Modified: 2014-01-12 19:57 EST (History)
11 users (show)

See Also:
Fixed In Version: openstack-packstack-2013.1.1-0.20.dev632.el6ost
Doc Type: Bug Fix
Doc Text:
Previously, after using PackStack to install OpenStack, one could start Swift successfully the first time. However, attempting to restart Swift after the first start failed. This has been fixed and Swift restarts correctly.
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-06-27 13:18:34 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
RHOS patch (710 bytes, patch)
2013-06-20 05:06 EDT, Martin Magr
derekh: review+
Details | Diff
Signing patch mk2 (557 bytes, patch)
2013-06-20 07:34 EDT, Martin Magr
no flags Details | Diff

  None (edit)
Description Pete Zaitcev 2013-06-19 16:42:24 EDT
Description of problem:

According to data captured for bug 967631, packstack --allinone creates
the following configuration in proxy-server.conf:

[filter:authtoken]
signing_dir = /etc/swift

This results in Swift denying access to its own processes and failing
with a looping crash.

Version-Release number of selected component (if applicable):

openstack-packstack-2013.1.1-0.18.dev631.el6ost.noarch

How reproducible:

Synchronous, apparently

Steps to Reproduce:
1. packstack --allinone
2. grep signing_dir /etc/swift/proxy-server.conf

Actual results:

Swift blows up

Expected results:

Swift works

Additional info:

One insiduous problem here is that the fateful chmod occurs when Swift
starts. Therefore, you can start it ONCE, test that it works. But if
you restart it, system becomes almost inaccessible with consoles
flooded by looping crash tracebacks.

The openstack-swift RPM ships with signing_dir /tmp/something-something.
Perhaps someone thought it was not secure enough. If that is a concern,
we must package a separate directory in /var/run and use /etc/tmpfiles.d
to establish proper permissions. Note that this is different on systemd
and Upstart systems like RHEL.
Comment 2 Pete Zaitcev 2013-06-19 17:17:02 EDT
Alan pointed out that this is a problem not in Packstack as such, but in
one of upstream Puppet modules, here:
 https://github.com/stackforge/puppet-swift/blob/ee4a9d48599bce332d0d7bdf4f8c0bbb6d9c6f2e/templates/proxy/authtoken.conf.erb
Comment 3 Alan Pevec 2013-06-19 18:54:26 EDT
Proper fix is not to set signing_dir at all so upstream default applies (I didn't see any justification in puppet-swift why siging_dir

In keystoneclinet master that was recently changed to random tempdir:
 https://github.com/openstack/python-keystoneclient/blob/master/keystoneclient/middleware/auth_token.py#L296
but in RHOS3 "stable/grizzly" we still have old default ~/keystone-signing:
 https://github.com/redhat-openstack/python-keystoneclient/blob/stable/grizzly/keystoneclient/middleware/auth_token.py#L214

$HOME for swift account is /var/lib/swift but this folder is not include in the RPM, so you get:
...
  File "/usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py", line 306, in __init__
    os.makedirs(self.signing_dirname)
  File "/usr/lib64/python2.6/os.py", line 150, in makedirs
    makedirs(head, mode)
  File "/usr/lib64/python2.6/os.py", line 157, in makedirs
    mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/var/lib/swift'

/var/lib/swift should be included in the openstack-swift RPM and/or backport https://github.com/openstack/python-keystoneclient/commit/03012e641d6c2a98fbfe3780102e28a65d11a887 to "stable/grizzly".
Comment 4 Alan Pevec 2013-06-20 04:26:26 EDT
(In reply to Alan Pevec from comment #3)
> Proper fix is not to set signing_dir at all so upstream default applies (I
> didn't see any justification in puppet-swift why siging_dir

... was set at all).
Comment 5 Martin Magr 2013-06-20 04:31:27 EDT
So what do you suggest? Remove "signing_dir = /etc/swift" from authtoken.conf.erb?
Comment 6 Alan Pevec 2013-06-20 04:34:44 EDT
(In reply to Martin Magr from comment #5)
> So what do you suggest? Remove "signing_dir = /etc/swift" from
> authtoken.conf.erb?

Yes, but then also need to include /var/lib/swift in the openstack-swift RPM or update keystoneclient to include upstream fix mentioned in comment 3.
Comment 8 Martin Magr 2013-06-20 05:06:17 EDT
Created attachment 763330 [details]
RHOS patch
Comment 9 Alan Pevec 2013-06-20 07:25:15 EDT
> /var/lib/swift should be included in the openstack-swift RPM

fixed in openstack-swift-1.8.0-6.el6ost
Comment 10 Martin Magr 2013-06-20 07:34:05 EDT
Created attachment 763432 [details]
Signing patch mk2

Removed removing last empty line
Comment 11 Derek Higgins 2013-06-20 08:17:38 EDT
Comment on attachment 763330 [details]
RHOS patch

lgtm
Comment 13 Bruce Reeler 2013-06-21 00:01:33 EDT
Needinfo: Martin, could you pls read Doc Text and let me know if that's OK. Thx.
Comment 14 Haim 2013-06-24 02:15:20 EDT
verified on openstack-packstack-2013.1.1-0.20.dev632.el6ost

[root@vm-161-67 ~]# grep signing_dir /etc/swift/proxy-server.conf 
[root@vm-161-67 ~]# 

swift is running: 

[root@vm-161-67 ~]# ps aux | grep swift
root      1144  0.0  0.0 103236   832 pts/0    R+   09:15   0:00 grep swift
swift    29649  0.0  0.4 237896 19580 ?        Ss   09:01   0:00 /usr/bin/python /usr/bin/swift-proxy-server /etc/swift/proxy-server.conf
swift    29684  0.0  0.4 238760 18384 ?        S    09:01   0:00 /usr/bin/python /usr/bin/swift-proxy-server /etc/swift/proxy-server.conf
swift    29776  0.0  0.3 228932 14192 ?        Ss   09:01   0:00 /usr/bin/python /usr/bin/swift-account-auditor /etc/swift/account-server.conf
swift    29804  0.0  0.4 230076 15792 ?        Ss   09:01   0:00 /usr/bin/python /usr/bin/swift-account-replicator /etc/swift/account-server.conf
swift    29835  0.0  0.3 229388 14796 ?        Ss   09:01   0:00 /usr/bin/python /usr/bin/swift-account-server /etc/swift/account-server.conf
swift    29854  0.0  0.3 230008 13380 ?        S    09:01   0:00 /usr/bin/python /usr/bin/swift-account-server /etc/swift/account-server.conf
swift    29857  0.0  0.3 228960 14360 ?        Ss   09:01   0:00 /usr/bin/python /usr/bin/swift-account-reaper /etc/swift/account-server.conf
swift    29958  0.0  0.3 222288 14416 ?        Ss   09:02   0:00 /usr/bin/python /usr/bin/swift-object-replicator /etc/swift/object-server.conf
swift    29988  0.0  0.3 228704 14524 ?        Ss   09:02   0:00 /usr/bin/python /usr/bin/swift-object-server /etc/swift/object-server.conf
swift    30000  0.0  0.3 228836 11848 ?        S    09:02   0:00 /usr/bin/python /usr/bin/swift-object-server /etc/swift/object-server.conf
swift    30011  0.0  0.3 229144 15044 ?        Ss   09:02   0:00 /usr/bin/python /usr/bin/swift-object-updater /etc/swift/object-server.conf
swift    30044  0.0  0.3 227988 13900 ?        Ss   09:02   0:00 /usr/bin/python /usr/bin/swift-object-auditor /etc/swift/object-server.conf
swift    30053  0.0  0.2 227988 11084 ?        S    09:02   0:00 /usr/bin/python /usr/bin/swift-object-auditor /etc/swift/object-server.conf
swift    30142  0.0  0.3 230092 15480 ?        Ss   09:02   0:00 /usr/bin/python /usr/bin/swift-container-updater /etc/swift/container-server.conf
swift    30173  0.0  0.3 229400 14812 ?        Ss   09:02   0:00 /usr/bin/python /usr/bin/swift-container-server /etc/swift/container-server.conf
swift    30192  0.0  0.3 229740 12152 ?        S    09:02   0:00 /usr/bin/python /usr/bin/swift-container-server /etc/swift/container-server.conf
swift    30207  0.0  0.3 228980 14200 ?        Ss   09:02   0:00 /usr/bin/python /usr/bin/swift-container-auditor /etc/swift/container-server.conf
swift    30238  0.0  0.3 230084 15484 ?        Ss   09:02   0:00 /usr/bin/python /usr/bin/swift-container-replicator /etc/swift/container-server.conf
Comment 15 Martin Magr 2013-06-24 04:52:15 EDT
Docs are OK, thanks Bruce.
Comment 16 Alan Pevec 2013-06-25 17:39:09 EDT
BTW looks like the origin of signing_dir = /etc/swift was a bad advice in one upstream bug comment: https://bugs.launchpad.net/keystone/+bug/1036847/comments/7
Comment 17 Martin Magr 2013-06-26 03:42:44 EDT
From Adam's comment [1] it seems that correct directory should be /var/cache/swift. So what now? Correct that in packstack and let swift rpm to create such directory?

[1] https://bugs.launchpad.net/keystone/+bug/1036847/comments/10
Comment 18 Pete Zaitcev 2013-06-26 10:17:54 EDT
I think Adam is right. I never was enthusiastic about throwing signing_dir
into automatically created directories, although it works as a stopgap
measure against the major screwup with directing it to /etc/swift.

I'm going to open yet another bug against Swift. At worst we'll close it
with wontfix.
Comment 19 Pete Zaitcev 2013-06-26 10:32:14 EDT
Filed bug 978408 so Fedora is consistent with RHOS going forward.
Comment 21 errata-xmlrpc 2013-06-27 13:18:34 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-0968.html

Note You need to log in before you can comment on or make changes to this bug.