Bug 1652561 - Run tendrl-ansible during update
Summary: Run tendrl-ansible during update
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat Storage
Component: doc-RHGS_Web_Administration
Version: rhgs-3.4
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: RHGS 3.4.z Batch Update 2
Assignee: Rakesh
QA Contact: Filip Balák
URL:
Whiteboard:
Depends On: 1650557
Blocks: 1650394
TreeView+ depends on / blocked
 
Reported: 2018-11-22 11:25 UTC by Filip Balák
Modified: 2018-12-28 10:21 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-12-28 10:21:53 UTC
Embargoed:


Attachments (Terms of Use)
screenshot with documentation, highlighting options where to add the additional step (131.24 KB, image/png)
2018-12-04 15:57 UTC, Martin Bukatovic
no flags Details

Description Filip Balák 2018-11-22 11:25:57 UTC
Description of problem:
There should be a step that describes execution of tendrl-ansible command in upgrade guide [1] as part of solution for bz 1650557. tendrl-ansible should be probably called after `# yum update` is called on WA server or at the end of upgrade. There should be also a link to section with tendrl-ansible and mention how to run it.


[1] https://access.qa.redhat.com/documentation/en-us/red_hat_gluster_storage/3.4/html-single/quick_start_guide/#red_hat_gluster_storage_web_administration_3_4_x_to_3_4_y

Comment 2 Daniel Horák 2018-11-27 09:28:27 UTC
It would be better to get ack from dev team, but my opinion is, that the best place for this additional step is between step 3 (Run the tendrl-upgrade script) and step 4 (If updates to the kernel package occurred, reboot the server system. If not, restart the Web Administration services).

And the new step should look somehow like this:
Run tendrl-ansible playbooks with the same configuration as for initial installation. For more information see section 3.3. Web Administration Installation[1], steps 8 and 9.

[1] https://access.qa.redhat.com/documentation/en-us/red_hat_gluster_storage/3.4/html-single/quick_start_guide/#web_administration_installation

Comment 3 Daniel Horák 2018-11-27 09:39:05 UTC
One small change for the new step:
Run tendrl-ansible playbooks with the same configuration as for initial installation. For more information see section 3.3. Web Administration Installation[1], steps 2 to 9.

Comment 4 Nishanth Thomas 2018-11-29 06:13:50 UTC
@Daniel, Running tendrl-ansible can be the last step. Also note that we can remove the step of restarting the tendrl services(barring httpd and collectd) as the tendrl-ansible will take care of the same. But I don't think httpd and collecd are restarted by tendrl-ansible.

What do you think?

Comment 5 Daniel Horák 2018-11-29 07:21:39 UTC
It might be possible to drop the steps for restarting Tendrl related services
in the future and let tendrl-ansible do that task completely. But for now I
would rather keep it as it is - manually instruct user to restart all the
related services at the end of the update process. The reasons are follows:

* tendrl-ansible ensure, that the required services are running, they are not
  explicitly restarted (except following case)

* tendrl-ansible restart only those services where configuration was changed by
  tendrl-ansible

* tendrl-ansible doesn't manage tendrl-gluster-integration and collectd on
  Gluster Storage Servers

If we would like to drop the steps with manual restart of Tendrl services, we
have to extensively test multiple related scenarios to ensure, that there is
no possibility, that some of the services will get to unexpected state.
I think, we shouldn't take the risk of breaking something so late in the
schedule of this release. Restarting of all the related services at the end of
the update process should be safe, without any side effect.

But for the future releases, it is worth to consider enhancement of
tendrl-ansible to take care of some or all of the additional update steps.

Comment 6 Martin Bukatovic 2018-12-04 15:43:00 UTC
To sum this up: because of BZ 1650557, we need to enhance our documentation
related to update procedure.

In section "4.2. Red Hat Gluster Storage Web Administration 3.4.x to 3.4.y",
we need add a step like this (drafted in comment 2 and comment 3):

> Run tendrl-ansible playbooks with the same configuration as for initial
> installation. For more information see section 3.3. Web Administration
> Installation[1], steps 2 to 9.
>
> [1] https://access.qa.redhat.com/documentation/en-us/red_hat_gluster_storage/3.4/html-single/quick_start_guide/#web_administration_installation

Now, there are 2 places where we could add it (as Daniel and Nishanth point
out):

a) after step #2 (yum update) and before step #3 (tendrl-upgrade)
b) after step #3 (tendrl-upgrade) and before step #4 (restarting services)
c) after step #4 (restarting services)

Option a is original proposal from Filip, option b is suggested by Daniel in
comment 2, option c by Nishanth in comment 4.

I'm asking Nishanth to verify this description, and indicate if he agrees with
Daniel to go with option b or would like to go with c.

## Additional Details, not important for the work which needs to be done here

(doc team can safely ignore this section)

As Daniel points out, there is no way to delegate service restart into tendrl
ansible now, as that would require changes in tendrl-ansible.

The best way would actually be to:

* move installation of gluster integration from import job to tendrl-ansible
  (eg. https://github.com/Tendrl/tendrl-ansible/issues/126)
* actually care about the update and allocate the dev effort for it, which
  includes update playbook

But this is outside of scope of this bugzilla or BU2.

Comment 7 Martin Bukatovic 2018-12-04 15:57:18 UTC
Created attachment 1511377 [details]
screenshot with documentation, highlighting options where to add the additional step

I'm attaching a screenshot with documentation, highlighting options where to add
the additional step.

Comment 8 gowtham 2018-12-05 06:25:32 UTC
I will suggest running tendrl-ansible between 3 and 4.

In initial steps, we are suggesting to stop all tendrl related services So I feel running ansible between 3 and 4 will definitely start the services again. (already httpd also started by tendrl-ansible).

and in step 4 we have to consider two scenarios:
  1. managed cluster: if the cluster is managed then we have to worry about starting gluster-integration and collectd services. So we will suggest step only for the already managed clusters.

  2. Unmanaged cluster: if the cluster is not managed yet but if the user doing the update then step 4 is not required, because we are going to run tendrl-ansible. If they run collectd and tendrl-ansible as per that downstream document then import flow will be screwed up. (if it is a fresh machine then it is fine, but if it is already unmanaged cluster and ready to import then it is a problem, they have to unmanage again to solve this problem).

Comment 9 gowtham 2018-12-05 06:27:49 UTC
As per Daniels comment if tendrl-ansible not start all stopped services then we have to modify step 4 like what are all the services should be started for managed cluster and what are all the services required to start for an unmanaged cluster.

Comment 10 Nishanth Thomas 2018-12-05 09:06:17 UTC
Agree to go forward with option (b) now.

(In reply to Martin Bukatovic from comment #6)
> To sum this up: because of BZ 1650557, we need to enhance our documentation
> related to update procedure.
> 
> In section "4.2. Red Hat Gluster Storage Web Administration 3.4.x to 3.4.y",
> we need add a step like this (drafted in comment 2 and comment 3):
> 
> > Run tendrl-ansible playbooks with the same configuration as for initial
> > installation. For more information see section 3.3. Web Administration
> > Installation[1], steps 2 to 9.
> >
> > [1] https://access.qa.redhat.com/documentation/en-us/red_hat_gluster_storage/3.4/html-single/quick_start_guide/#web_administration_installation
> 
> Now, there are 2 places where we could add it (as Daniel and Nishanth point
> out):
> 
> a) after step #2 (yum update) and before step #3 (tendrl-upgrade)
> b) after step #3 (tendrl-upgrade) and before step #4 (restarting services)
> c) after step #4 (restarting services)
> 
> Option a is original proposal from Filip, option b is suggested by Daniel in
> comment 2, option c by Nishanth in comment 4.
> 
> I'm asking Nishanth to verify this description, and indicate if he agrees
> with
> Daniel to go with option b or would like to go with c.
> 
> ## Additional Details, not important for the work which needs to be done here
> 
> (doc team can safely ignore this section)
> 
> As Daniel points out, there is no way to delegate service restart into tendrl
> ansible now, as that would require changes in tendrl-ansible.
> 
> The best way would actually be to:
> 
> * move installation of gluster integration from import job to tendrl-ansible
>   (eg. https://github.com/Tendrl/tendrl-ansible/issues/126)
> * actually care about the update and allocate the dev effort for it, which
>   includes update playbook
> 
> But this is outside of scope of this bugzilla or BU2.

Comment 12 Filip Balák 2018-12-05 15:51:53 UTC
Looks good. --> VERIFIED


Note You need to log in before you can comment on or make changes to this bug.