Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1992655

Summary: STF | FFU | Configuration of collectd ceph plugin must be removed from the tripleo template prior upgrading overcloud.
Product: Red Hat OpenStack Reporter: Leonid Natapov <lnatapov>
Component: documentationAssignee: Leif Madsen <lmadsen>
Status: CLOSED CURRENTRELEASE QA Contact: Leonid Natapov <lnatapov>
Severity: high Docs Contact: Joanne O'Flynn <joflynn>
Priority: high    
Version: 16.2 (Train)CC: astillma, lmadsen, lmarsh
Target Milestone: z4Keywords: Documentation, Triaged
Target Release: 16.2 (Train on RHEL 8.4)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Release Note
Doc Text:
In previous releases, administrators had to add the `ceph` plugin to the `CollectdExtraPlugins` parameter in their custom environment files. + With this release, the `ceph` plugin loads automatically on Ceph Storage nodes. Therefore, before you upgrade from Red Hat OpenStack Platform 13 to 16.2, you must remove the `ceph` plugin from the `CollectdExtraPlugins` parameter in your custom environment files.
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-10-25 14:10:04 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1989660    
Bug Blocks:    

Description Leonid Natapov 2021-08-11 13:35:29 UTC
STF | FFU | Configuration of collectd ceph plugin must be removed from the tripleo template prior upgrading overcloud.

Scenario:
OSP13 deployed with ceph nodes and running collectd that configured with collectd ceph plugin.

If user running OSP13 and has ceph nodes that run collectd and configured collectd ceph plugin ,the ceph plugin must be removed before overcloud upgrade.


In OSP13 in order to load ceph plugin it should be explicitly specified in the tripleo template but in OSP16+  ceph plugin loads automatically on the ceph nodes. It results the situation when ceph plugin defined two times. One time by user in a template file and second time by the puppet.  

In such a case upgrade of the ceph nodes fails because collectd is unable to start giving: Evaluation Error: Error while evaluating a Resource Statement, Duplicate declaration.

Soluition:
collectd ceph plugin must be removed from the template prior overcloud upgrade.

Comment 6 Vlada Grosu 2021-09-03 08:38:13 UTC
Hello Leonid,

Is there another engineering BZ that is connected to the issue described in this BZ that you can point me to, please?
I am looking for more details to estimate how likely this issue is to occur during FFU. If the likelihood of that is high, I would also recommend adding this information to the FFU guides:

* Framework for Upgrades (13 to 16.1) https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/16.1/html-single/framework_for_upgrades_13_to_16.1/index
* Framework for Upgrades (13 to 16.2) https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/16.2-beta/html-single/framework_for_upgrades_13_to_16.2/index

Depending on the details, this ought to be called out in the "Known issues that might block an upgrade" topic (at least) or add a new procedure prior to the overcloud upgrade to explicitly request operators to edit the `CollectdExtraPlugins` parameter in their custom environment files.

I am happy to help with any FFU docs changes. Please let me know what you think.

Many thanks,
Vlada

Comment 10 Leif Madsen 2021-09-13 17:40:52 UTC
I have responded to the linked document asked for review. Let me know if you need something else from me.

Comment 12 Vlada Grosu 2021-10-01 16:08:00 UTC
Hi Leif,

Can you please review comment #6 as well? I wonder if there any manual steps that need to be added to the Framework for Upgrades document, if the upgrade (13 -> 16.2 and 13 -> 16.1) is affected by this issue?


Many thanks,
Vlada

(In reply to Leif Madsen from comment #10)
> I have responded to the linked document asked for review. Let me know if you
> need something else from me.

Comment 13 Leonid Natapov 2021-10-03 12:26:43 UTC
(In reply to Vlada Grosu from comment #12)
> Hi Leif,
> 
> Can you please review comment #6 as well? I wonder if there any manual steps
> that need to be added to the Framework for Upgrades document, if the upgrade
> (13 -> 16.2 and 13 -> 16.1) is affected by this issue?
> 
> 
> Many thanks,
> Vlada
> 
> (In reply to Leif Madsen from comment #10)
> > I have responded to the linked document asked for review. Let me know if you
> > need something else from me.

Hey Valda.
If there is a ceph configuration that was added by user in OSP13 it should  be removed before the FFU.