Bug 927423 - Graceful restart of swift services via initscripts
Summary: Graceful restart of swift services via initscripts
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-swift
Version: 2.1
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: snapshot2
: 3.0
Assignee: Pete Zaitcev
QA Contact: Martina Kollarova
URL:
Whiteboard:
Depends On:
Blocks: 971881
TreeView+ depends on / blocked
 
Reported: 2013-03-25 21:13 UTC by Lon Hohberger
Modified: 2016-04-26 17:44 UTC (History)
6 users (show)

Fixed In Version: openstack-swift-1.8.0-3.el6ost
Doc Type: Enhancement
Doc Text:
Clone Of:
: 971881 (view as bug list)
Environment:
Last Closed: 2013-06-11 18:39:54 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
Candidate 2 (4.07 KB, patch)
2013-04-26 06:23 UTC, Pete Zaitcev
no flags Details | Diff


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2013:0934 0 normal SHIPPED_LIVE openstack-swift enhancement update 2013-06-11 22:37:55 UTC

Description Lon Hohberger 2013-03-25 21:13:46 UTC
Description of problem:  Attila noted that swift can be restarted gracefully. 

https://bugzilla.redhat.com/show_bug.cgi?id=885888#c4

It would be trivial to make swift's initscript 'reload' section call the correct swift-init commands to do a graceful restart.

Comment 1 Lon Hohberger 2013-03-25 21:14:39 UTC
I meant 'restart', not 'reload'.

Comment 4 Pete Zaitcev 2013-03-27 17:28:28 UTC
The patch has merit. I think it makes sense to restart server instances
this way to reduce the risk of inconsistencies.

Alan pointed out that folding it into "reload" is misleading, since it's
not actually. It's a soft shutdown, then restart. In this case it's the
terminology that Swift itself uses, e.g. "swift-init reload object-server"
is what the patch does. But in RHEL this may violate the principle of
least surprise. If this is really objectionable, we may throw out the
"reload" and keep the "shutdown" at least, that one is surely good.

Just to make it more fun, Swift servers reload rings when they detect
the timestamp change. So you only need this "reload" if configuration
changes in material ways. Which is never.

Comment 6 Pete Zaitcev 2013-04-26 06:23:57 UTC
Created attachment 740237 [details]
Candidate 2

I talked to Greg Holt about the practices of shutdown, and he said that
in RAX they do not do the shutdown as a part of a scripted restart.
Instead, they do the HUP thing and leave the system alone for a day,
then kill all the stuck processes, if any.

So, I propose this:
 - do not implement the "reload". This way we stick with KISS principle
   and no user thinks that configurationg or ring is reloaded.
 - implement shutdown without waiting. Atilla needed it in order for
   reload to work.

Hope this does the trick. Opinions?

P.S. patch goes on top of the patch for Jaroslav's bug 917657, pay it
no mind. It's trivial to adapt to 2.1 async if needed.

Comment 9 Lon Hohberger 2013-05-09 20:24:06 UTC
ACK on design from me.  Attila?

Comment 11 Martina Kollarova 2013-06-05 14:43:18 UTC
It would be nice if it was more explicit what is a soft and what is hard shutdown. Maybe add some note about it in the help message?

Also, I'm not getting a newline, so the output gets mangled up. This does not happen with the 'restart' command, for example.
[root@server ~]# service openstack-swift-proxy shutdown 
[root@server ~]# ack-swift-proxy:                       [  OK  ]

Any ideas about how to test if something really does a hard/soft stop of the service?

Comment 13 Martina Kollarova 2013-06-07 13:39:54 UTC
As requested, I created another bug #971881 for the cosmetic issues.

Comment 15 errata-xmlrpc 2013-06-11 18:39:54 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHEA-2013-0934.html

Comment 16 Pete Zaitcev 2013-06-24 22:12:38 UTC
(In reply to Martina Kollarova from comment #11)
> Any ideas about how to test if something really does a hard/soft stop of the
> service?

To test the condition one has to start a longish transfer and shut down services that are involved, such as proxy. The transfer should complete successfully. Sorry, this came too late, but in case.


Note You need to log in before you can comment on or make changes to this bug.