Bug 1470005 - OSP11 -> OSP12 upgrade: openstack-swift-object-expirer.service is still running under systemd after upgrading to docker architecture
OSP11 -> OSP12 upgrade: openstack-swift-object-expirer.service is still runni...
Status: CLOSED ERRATA
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates (Show other bugs)
12.0 (Pike)
Unspecified Unspecified
high Severity urgent
: beta
: 12.0 (Pike)
Assigned To: Emilien Macchi
Marius Cornea
: Triaged
Depends On:
Blocks: 1399762
  Show dependency treegraph
 
Reported: 2017-07-12 05:10 EDT by Marius Cornea
Modified: 2017-12-13 16:40 EST (History)
7 users (show)

See Also:
Fixed In Version: openstack-tripleo-heat-templates-7.0.0-0.20170805163046.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2017-12-13 16:40:04 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Launchpad 1699444 None None None 2017-07-12 05:10 EDT
OpenStack gerrit 484711 None None None 2017-07-18 06:21 EDT

  None (edit)
Description Marius Cornea 2017-07-12 05:10:57 EDT
Description of problem:
OSP11 -> OSP12 upgrade: openstack-swift-object-expirer.service is still running under systemd after upgrading to docker architecture:

During upgrade, at step overcloud-AllNodesDeploySteps-74yhsp77to5a-AllNodesPostUpgradeSteps-tl7b4sbb2xft-ControllerDeployment_Step3-o2ybgewtnni3
we can see on the overcloud node:

[root@controller-0 heat-admin]# systemctl status openstack-swift-object-expirer.service
● openstack-swift-object-expirer.service - OpenStack Object Storage (swift) - Object Expirer
   Loaded: loaded (/usr/lib/systemd/system/openstack-swift-object-expirer.service; enabled; vendor preset: disabled)
   Active: active (running) since Tue 2017-07-11 18:01:56 UTC; 15h ago
 Main PID: 86168 (swift-object-ex)
   Memory: 18.8M
   CGroup: /system.slice/openstack-swift-object-expirer.service
           └─86168 /usr/bin/python2 /usr/bin/swift-object-expirer /etc/swift/object-expirer.conf

Jul 12 09:03:42 controller-0 swift[86168]: ERROR with Account server 172.17.4.10:6002/d1 re: Trying to HEAD /v1/.expiring_objects: Connection refused (txn: tx318ebe42c99f4cb89bfee-005965e5ee)
Jul 12 09:03:42 controller-0 swift[86168]: Account HEAD returning 503 for [] (txn: tx318ebe42c99f4cb89bfee-005965e5ee)
Jul 12 09:03:42 controller-0 object-expirer[86168]: Unhandled exception: #012Traceback (most recent call last):#012  File "/usr/lib/python2.7/site-packages/swift/obj/expirer.py", line 175, in run_once#012    self.swift.g...nal_client.py", 
Jul 12 09:06:23 controller-0 swift[86168]: ERROR with Account server 172.17.4.10:6002/d1 re: Trying to HEAD /v1/.expiring_objects: Connection refused (txn: txe269afd014e14839afc15-005965e68f)
Jul 12 09:06:23 controller-0 swift[86168]: Account HEAD returning 503 for [] (txn: txe269afd014e14839afc15-005965e68f)
Jul 12 09:06:25 controller-0 swift[86168]: ERROR with Account server 172.17.4.10:6002/d1 re: Trying to HEAD /v1/.expiring_objects: Connection refused (txn: txbd99e615031340c5a04ce-005965e691)
Jul 12 09:06:25 controller-0 swift[86168]: Account HEAD returning 503 for [] (txn: txbd99e615031340c5a04ce-005965e691)
Jul 12 09:06:29 controller-0 swift[86168]: ERROR with Account server 172.17.4.10:6002/d1 re: Trying to HEAD /v1/.expiring_objects: Connection refused (txn: tx3ef221a9ab354f0ab23bd-005965e695)
Jul 12 09:06:29 controller-0 swift[86168]: Account HEAD returning 503 for [] (txn: tx3ef221a9ab354f0ab23bd-005965e695)
Jul 12 09:06:29 controller-0 object-expirer[86168]: Unhandled exception: #012Traceback (most recent call last):#012  File "/usr/lib/python2.7/site-packages/swift/obj/expirer.py", line 175, in run_once#012    self.swift.g...nal_client.py", 
Hint: Some lines were ellipsized, use -l to show in fu

Version-Release number of selected component (if applicable):
openstack-tripleo-heat-templates-7.0.0-0.20170628002128.el7ost.noarch

How reproducible:
100%
Steps to Reproduce:
1. Deploy OSP11
2. Upgrade to OSP12 with containerized architecture

Actual results:
openstack-swift-object-expirer.service is running under systemd.

Expected results:
openstack-swift-object-expirer.service should be running only inside containers.

Additional info:
Comment 1 Carlos Camacho 2017-07-19 05:20:15 EDT
Pushed an upstream fix.
Comment 2 Carlos Camacho 2017-08-02 04:46:06 EDT
Tested now:

[heat-admin@overcloud-controller-0 ~]$ systemctl status openstack-swift-object-expirer.service
● openstack-swift-object-expirer.service - OpenStack Object Storage (swift) - Object Expirer
   Loaded: loaded (/usr/lib/systemd/system/openstack-swift-object-expirer.service; disabled; vendor preset: disabled)
   Active: inactive (dead)

Aug 01 20:27:01 overcloud-controller-0 object-expirer[30485]: Pass completed in 0s; 0 objects expired (txn: txc0c81742b566471dbbc8d-005980e415)
Aug 01 20:28:17 overcloud-controller-0 object-expirer[30485]: Pass beginning; 0 possible containers; 0 possible objects (txn: txa10f16e5281b45aabd6dd-005980e461)
Aug 01 20:28:18 overcloud-controller-0 object-expirer[30485]: Pass completed in 0s; 0 objects expired (txn: tx6ed12ad96997487185568-005980e461)
Aug 01 20:32:23 overcloud-controller-0 object-expirer[30485]: Pass beginning; 0 possible containers; 0 possible objects (txn: txfbd9df99089d457badd2d-005980e557)
Aug 01 20:32:23 overcloud-controller-0 object-expirer[30485]: Pass completed in 0s; 0 objects expired (txn: tx1d40d73e1893471dabbd8-005980e557)
Aug 01 20:35:21 overcloud-controller-0 object-expirer[30485]: Pass beginning; 0 possible containers; 0 possible objects (txn: txfc65897df33f49a1b69b7-005980e609)
Aug 01 20:35:21 overcloud-controller-0 object-expirer[30485]: Pass completed in 0s; 0 objects expired (txn: txaeb37ab856e4483dbcf7e-005980e609)
Aug 01 20:40:05 overcloud-controller-0 systemd[1]: Stopping OpenStack Object Storage (swift) - Object Expirer...
Aug 01 20:40:05 overcloud-controller-0 object-expirer[30485]: SIGTERM received
Aug 01 20:40:05 overcloud-controller-0 systemd[1]: Stopped OpenStack Object Storage (swift) - Object Expirer.
Comment 6 Marius Cornea 2017-11-08 11:20:32 EST
After upgrade:

[root@controller-0 heat-admin]# systemctl status openstack-swift-object-expirer.service
● openstack-swift-object-expirer.service - OpenStack Object Storage (swift) - Object Expirer
   Loaded: loaded (/usr/lib/systemd/system/openstack-swift-object-expirer.service; disabled; vendor preset: disabled)
   Active: inactive (dead)

Nov 08 14:53:39 controller-0 object-expirer[87639]: Pass beginning; 0 possible containers; 0 possible objects (txn: txa8b456db27f94a3991694-005a031a73)
Nov 08 14:53:39 controller-0 object-expirer[87639]: Pass completed in 0s; 0 objects expired (txn: tx37e3bb8b9c3443cfb1f29-005a031a73)
Nov 08 14:55:48 controller-0 object-expirer[87639]: Pass beginning; 0 possible containers; 0 possible objects (txn: tx1bbc3bfc894d40bfa25be-005a031af4)
Nov 08 14:55:48 controller-0 object-expirer[87639]: STDERR: ERROR:root:Error talking to memcached: 172.17.1.18:11211#012Traceback (most recent call last):#012  File "/usr/lib/python2.7/site-packages/swift/common/memcache...ackages/eventlet
Nov 08 14:55:48 controller-0 object-expirer[87639]: STDERR: ERROR:root:Error connecting to memcached: 172.17.1.22:11211#012Traceback (most recent call last):#012  File "/usr/lib/python2.7/site-packages/swift/common/memca...on2.7/site-packa
Nov 08 14:55:48 controller-0 object-expirer[87639]: STDERR: ERROR:root:Error connecting to memcached: 172.17.1.23:11211#012Traceback (most recent call last):#012  File "/usr/lib/python2.7/site-packages/swift/common/memca...on2.7/site-packa
Nov 08 14:55:48 controller-0 object-expirer[87639]: Pass completed in 0s; 0 objects expired (txn: tx5784b3edc4514c04be260-005a031af4)
Nov 08 14:56:42 controller-0 systemd[1]: Stopping OpenStack Object Storage (swift) - Object Expirer...
Nov 08 14:56:42 controller-0 object-expirer[87639]: SIGTERM received
Nov 08 14:56:42 controller-0 systemd[1]: Stopped OpenStack Object Storage (swift) - Object Expirer.
Hint: Some lines were ellipsized, use -l to show in full.
Comment 9 errata-xmlrpc 2017-12-13 16:40:04 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:3462

Note You need to log in before you can comment on or make changes to this bug.