Description of problem: I installed Kolla with CentOS-binary mitaka with swift enabled, and noticed one of the swift container does not start properly. Swift container "swift_object_expirer" fails to start and tries restart again and again. Here's the docker logs of this container. Running command: 'swift-object-expirer /etc/swift/object-expirer.conf --verbose' /usr/local/bin/kolla_start: line 24: exec: swift-object-expirer: not found INFO:__main__:Kolla config strategy set to: COPY_ALWAYS INFO:__main__:Loading config file at /var/lib/kolla/config_files/config.json INFO:__main__:Validating config file INFO:__main__:Copying service configuration files INFO:__main__:Removing existing destination: /etc/swift/account.ring.gz INFO:__main__:Copying /var/lib/kolla/swift/account.ring.gz to /etc/swift/account.ring.gz INFO:__main__:Setting permissions for /etc/swift/account.ring.gz INFO:__main__:Removing existing destination: /etc/swift/container.ring.gz INFO:__main__:Copying /var/lib/kolla/swift/container.ring.gz to /etc/swift/container.ring.gz INFO:__main__:Setting permissions for /etc/swift/container.ring.gz INFO:__main__:Removing existing destination: /etc/swift/object.ring.gz INFO:__main__:Copying /var/lib/kolla/swift/object.ring.gz to /etc/swift/object.ring.gz INFO:__main__:Setting permissions for /etc/swift/object.ring.gz INFO:__main__:Removing existing destination: /etc/swift/swift.conf INFO:__main__:Copying /var/lib/kolla/config_files/swift.conf to /etc/swift/swift.conf INFO:__main__:Setting permissions for /etc/swift/swift.conf INFO:__main__:Removing existing destination: /etc/swift/object-expirer.conf INFO:__main__:Copying /var/lib/kolla/config_files/object-expirer.conf to /etc/swift/object-expirer.conf INFO:__main__:Setting permissions for /etc/swift/object-expirer.conf INFO:__main__:Writing out command to execute Running command: 'swift-object-expirer /etc/swift/object-expirer.conf --verbose' /usr/local/bin/kolla_start: line 24: exec: swift-object-expirer: not found Actual results: The command "swift-object-expirer" is provided with openstack-swift-proxy.rpm. Expected results: The command "swift-object-expirer" to be provided with openstack-swift-object.rpm.
The object-expirer service is usually run on proxies. It uses the internal client to access storage nodes, so it is isomorphic with the proxy. It is not advisable to run it on object storage nodes. Therefore, it is co-packaged with the proxy. If object-exporer were to packaged with object-storage service, then it would be impossible to run on proxies that do not have object-storage RPM installed, and that is not acceptable. The bug appears to be in whatever orchestration is yielding the above result. If the orchestration is too hard to fix, we could pull the binary into python-swift, I suppose.
The place where it is usually used should not determine where it belongs. As the utility name says, it is related to the objects and not to the proxy. In fact for a even more flexible usage, maybe it could have a own package? people then can decide if install it on proxy or not.
I believe this should be closed as 'invalid' or 'won't fix'. I can understand how the name may seem to imply that this is a service that it is typically run like the other background object services, but this is not necessarily the case. What I have found from talking to various operators involved in the Swift community is that there's not really one right approach to running the object expirer. In my very unscientific survey of asking people on IRC, I realized that it was pretty evenly split how ops are running the expirer. About half run on the proxies nodes and the other half run on the storage nodes. There are also those, that actually run the expirer on a separate node (together with other auxiliary services like statsd server). Sure, we could break the proxy rpm down into 3 rpms, one for expirer, one for reconciler (which has the exactly same issue) and one for the proxy itself, but I think this is a minor enough problem that does not warrant the overhead of two more rpms to track for Swift. @Otavio, I think you took the correct approach of creating a kolla image for the expirer, that way, operators can choose to run that container wherever they see fit.
Closing based on the comments from Pete & Thiago.