Hide Forgot
Description of problem: OSP11 -> OSP12 upgrade: openstack-swift-object-expirer.service is still running under systemd after upgrading to docker architecture: During upgrade, at step overcloud-AllNodesDeploySteps-74yhsp77to5a-AllNodesPostUpgradeSteps-tl7b4sbb2xft-ControllerDeployment_Step3-o2ybgewtnni3 we can see on the overcloud node: [root@controller-0 heat-admin]# systemctl status openstack-swift-object-expirer.service ● openstack-swift-object-expirer.service - OpenStack Object Storage (swift) - Object Expirer Loaded: loaded (/usr/lib/systemd/system/openstack-swift-object-expirer.service; enabled; vendor preset: disabled) Active: active (running) since Tue 2017-07-11 18:01:56 UTC; 15h ago Main PID: 86168 (swift-object-ex) Memory: 18.8M CGroup: /system.slice/openstack-swift-object-expirer.service └─86168 /usr/bin/python2 /usr/bin/swift-object-expirer /etc/swift/object-expirer.conf Jul 12 09:03:42 controller-0 swift[86168]: ERROR with Account server 172.17.4.10:6002/d1 re: Trying to HEAD /v1/.expiring_objects: Connection refused (txn: tx318ebe42c99f4cb89bfee-005965e5ee) Jul 12 09:03:42 controller-0 swift[86168]: Account HEAD returning 503 for [] (txn: tx318ebe42c99f4cb89bfee-005965e5ee) Jul 12 09:03:42 controller-0 object-expirer[86168]: Unhandled exception: #012Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/swift/obj/expirer.py", line 175, in run_once#012 self.swift.g...nal_client.py", Jul 12 09:06:23 controller-0 swift[86168]: ERROR with Account server 172.17.4.10:6002/d1 re: Trying to HEAD /v1/.expiring_objects: Connection refused (txn: txe269afd014e14839afc15-005965e68f) Jul 12 09:06:23 controller-0 swift[86168]: Account HEAD returning 503 for [] (txn: txe269afd014e14839afc15-005965e68f) Jul 12 09:06:25 controller-0 swift[86168]: ERROR with Account server 172.17.4.10:6002/d1 re: Trying to HEAD /v1/.expiring_objects: Connection refused (txn: txbd99e615031340c5a04ce-005965e691) Jul 12 09:06:25 controller-0 swift[86168]: Account HEAD returning 503 for [] (txn: txbd99e615031340c5a04ce-005965e691) Jul 12 09:06:29 controller-0 swift[86168]: ERROR with Account server 172.17.4.10:6002/d1 re: Trying to HEAD /v1/.expiring_objects: Connection refused (txn: tx3ef221a9ab354f0ab23bd-005965e695) Jul 12 09:06:29 controller-0 swift[86168]: Account HEAD returning 503 for [] (txn: tx3ef221a9ab354f0ab23bd-005965e695) Jul 12 09:06:29 controller-0 object-expirer[86168]: Unhandled exception: #012Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/swift/obj/expirer.py", line 175, in run_once#012 self.swift.g...nal_client.py", Hint: Some lines were ellipsized, use -l to show in fu Version-Release number of selected component (if applicable): openstack-tripleo-heat-templates-7.0.0-0.20170628002128.el7ost.noarch How reproducible: 100% Steps to Reproduce: 1. Deploy OSP11 2. Upgrade to OSP12 with containerized architecture Actual results: openstack-swift-object-expirer.service is running under systemd. Expected results: openstack-swift-object-expirer.service should be running only inside containers. Additional info:
Pushed an upstream fix.
Tested now: [heat-admin@overcloud-controller-0 ~]$ systemctl status openstack-swift-object-expirer.service ● openstack-swift-object-expirer.service - OpenStack Object Storage (swift) - Object Expirer Loaded: loaded (/usr/lib/systemd/system/openstack-swift-object-expirer.service; disabled; vendor preset: disabled) Active: inactive (dead) Aug 01 20:27:01 overcloud-controller-0 object-expirer[30485]: Pass completed in 0s; 0 objects expired (txn: txc0c81742b566471dbbc8d-005980e415) Aug 01 20:28:17 overcloud-controller-0 object-expirer[30485]: Pass beginning; 0 possible containers; 0 possible objects (txn: txa10f16e5281b45aabd6dd-005980e461) Aug 01 20:28:18 overcloud-controller-0 object-expirer[30485]: Pass completed in 0s; 0 objects expired (txn: tx6ed12ad96997487185568-005980e461) Aug 01 20:32:23 overcloud-controller-0 object-expirer[30485]: Pass beginning; 0 possible containers; 0 possible objects (txn: txfbd9df99089d457badd2d-005980e557) Aug 01 20:32:23 overcloud-controller-0 object-expirer[30485]: Pass completed in 0s; 0 objects expired (txn: tx1d40d73e1893471dabbd8-005980e557) Aug 01 20:35:21 overcloud-controller-0 object-expirer[30485]: Pass beginning; 0 possible containers; 0 possible objects (txn: txfc65897df33f49a1b69b7-005980e609) Aug 01 20:35:21 overcloud-controller-0 object-expirer[30485]: Pass completed in 0s; 0 objects expired (txn: txaeb37ab856e4483dbcf7e-005980e609) Aug 01 20:40:05 overcloud-controller-0 systemd[1]: Stopping OpenStack Object Storage (swift) - Object Expirer... Aug 01 20:40:05 overcloud-controller-0 object-expirer[30485]: SIGTERM received Aug 01 20:40:05 overcloud-controller-0 systemd[1]: Stopped OpenStack Object Storage (swift) - Object Expirer.
After upgrade: [root@controller-0 heat-admin]# systemctl status openstack-swift-object-expirer.service ● openstack-swift-object-expirer.service - OpenStack Object Storage (swift) - Object Expirer Loaded: loaded (/usr/lib/systemd/system/openstack-swift-object-expirer.service; disabled; vendor preset: disabled) Active: inactive (dead) Nov 08 14:53:39 controller-0 object-expirer[87639]: Pass beginning; 0 possible containers; 0 possible objects (txn: txa8b456db27f94a3991694-005a031a73) Nov 08 14:53:39 controller-0 object-expirer[87639]: Pass completed in 0s; 0 objects expired (txn: tx37e3bb8b9c3443cfb1f29-005a031a73) Nov 08 14:55:48 controller-0 object-expirer[87639]: Pass beginning; 0 possible containers; 0 possible objects (txn: tx1bbc3bfc894d40bfa25be-005a031af4) Nov 08 14:55:48 controller-0 object-expirer[87639]: STDERR: ERROR:root:Error talking to memcached: 172.17.1.18:11211#012Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/swift/common/memcache...ackages/eventlet Nov 08 14:55:48 controller-0 object-expirer[87639]: STDERR: ERROR:root:Error connecting to memcached: 172.17.1.22:11211#012Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/swift/common/memca...on2.7/site-packa Nov 08 14:55:48 controller-0 object-expirer[87639]: STDERR: ERROR:root:Error connecting to memcached: 172.17.1.23:11211#012Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/swift/common/memca...on2.7/site-packa Nov 08 14:55:48 controller-0 object-expirer[87639]: Pass completed in 0s; 0 objects expired (txn: tx5784b3edc4514c04be260-005a031af4) Nov 08 14:56:42 controller-0 systemd[1]: Stopping OpenStack Object Storage (swift) - Object Expirer... Nov 08 14:56:42 controller-0 object-expirer[87639]: SIGTERM received Nov 08 14:56:42 controller-0 systemd[1]: Stopped OpenStack Object Storage (swift) - Object Expirer. Hint: Some lines were ellipsized, use -l to show in full.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2017:3462