Bug 2179284

Summary:	[RHOSP 16.2] Rsyslog container looking pacemaker log file in the wrong directory
Product:	Red Hat OpenStack	Reporter:	Eric Nothen <enothen>
Component:	openstack-tripleo-heat-templates	Assignee:	Yadnesh Kulkarni <ykulkarn>
Status:	CLOSED ERRATA	QA Contact:	Leonid Natapov <lnatapov>
Severity:	medium	Docs Contact:	mgeary <mgeary>
Priority:	high
Version:	16.2 (Train)	CC:	jelynch, lmadsen, mariel, mburns, ykulkarn
Target Milestone:	z6	Keywords:	Triaged, ZStream
Target Release:	16.2 (Train on RHEL 8.4)
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	openstack-tripleo-heat-templates-11.6.1-2.20230717085025.1608f56.el8ost	Doc Type:	Bug Fix
Doc Text:	Before this update, host services, such as Pacemaker, were mounted under `/var/log/host/` in the rsyslog container. However, the configuration path was the same as the host path `/var/log/pacemaker/`, and the rsyslog service could not retrieve Pacemaker log files. With this update, the Pacemaker log path has been updated to `/var/log/host/pacemaker/`.	Story Points:	---
Clone Of:		Environment:
Last Closed:	2023-11-08 19:18:31 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	2180933
Bug Blocks:	2179343

Description Eric Nothen 2023-03-17 08:27:10 UTC

This bug was initially created as a copy of Bug #1877475

I am copying this bug because: 

The same issue is still present in 16.2.4


Description of problem:
Rsyslog heat template point to wrong pacemaker log file. 


Version-Release number of selected component (if applicable):
RHOSP-16.1

How reproducible:
Steps to Reproduce:

Deploy RHOSP-16.1 enabling rsyslog template "/usr/share/openstack-tripleo-heat-templates/environments/logging-environment-rsyslog.yaml" following the URL : https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/16.1/html-single/logging_monitoring_and_troubleshooting_guide/index#testing_the_connection. After deployment, as per the rsyslog configuration, it looks for pacemaker logs in location "/var/log/host/pacemaker.log" which actually does not exist.

Actual results:
root@overcloud-controller-0 ~]# grep -A5 pacemaker_system.pacemaker /var/lib/config-data/puppet-generated/rsyslog/etc/rsyslog.d/50_openstack_logs.conf
# pacemaker_system.pacemaker
input(type="imfile"
  file=" "
  startmsg.regex="^[a-zA-Z]{3} [0-9]{2} [:0-9]{8}"
  tag="system.pacemaker"
)
[root@overcloud-controller-0 ~]#
Log file points to "/var/log/host/pacemaker.log" which does not exist.
[root@overcloud-controller-0 ~]# ls -l /var/log/host/pacemaker.log
ls: cannot access '/var/log/host/pacemaker.log': No such file or directory
[root@overcloud-controller-0 ~]# 

Expected results:


The log file to be pointed to "/var/log/pacemaker/pacemaker.log"
[root@overcloud-controller-0 ~]# ls -l /var/log/pacemaker/pacemaker.log 
-rw-rw----. 1 hacluster haclient 14313545 Sep  9 16:55 /var/log/pacemaker/pacemaker.log
[root@overcloud-controller-0 ~]#

Comment 2 Eric Nothen 2023-03-17 08:28:30 UTC

(overcloud) [stack.lab ~]$ rpm -q openstack-tripleo-heat-templates
openstack-tripleo-heat-templates-11.6.1-2.20221010235135.el8ost.noarch
(overcloud) [stack.lab ~]$ 
(overcloud) [stack.lab ~]$ ssh heat-admin@overcloud-controller-0 sudo grep -A5 pacemaker_system.pacemaker /var/lib/config-data/puppet-generated/rsyslog/etc/rsyslog.d/50_openstack_logs.conf
# pacemaker_system.pacemaker
input(type="imfile"
  file="/var/log/pacemaker/pacemaker.log"
  startmsg.regex="^[a-zA-Z]{3} [0-9]{2} [:0-9]{8}"
  tag="system.pacemaker"
  reopenOnTruncate="on"
(overcloud) [stack.lab ~]$ 
(overcloud) [stack.lab ~]$ ssh heat-admin@overcloud-controller-0 sudo podman exec rsyslog ls -l /var/log/{,host/}pacemaker/pacemaker.log
-rw-rw----. 1 189 189 48126262 Mar 17 08:26 /var/log/host/pacemaker/pacemaker.log
ls: cannot access '/var/log/pacemaker/pacemaker.log': No such file or directory
(overcloud) [stack.lab ~]$

Comment 3 Leif Madsen 2023-03-22 14:04:38 UTC

We think we might see this in 17.1 as well, so we'll need to get that resolved in Wallaby upstream and then backported to Train.

Comment 10 Leonid Natapov 2023-09-30 09:17:01 UTC

Fixed. openstack-tripleo-heat-templates-11.6.1-2.20230808225213.9adcac6.el8ost.noarch


[root@ctrl-2-16-2 tripleo-admin]# grep -A5 pacemaker_system.pacemaker /var/lib/config-data/puppet-generated/rsyslog/etc/rsyslog.d/50_openstack_logs.conf
# pacemaker_system.pacemaker
input(type="imfile"
  file="/var/log/host/pacemaker/pacemaker.log"
  startmsg.regex="^[a-zA-Z]{3} [0-9]{2} [:0-9]{8}"
  tag="system.pacemaker"
)

Comment 17 errata-xmlrpc 2023-11-08 19:18:31 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat OpenStack Platform 16.2.6 (Train) bug fix and enhancement advisory), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2023:6307

Comment 18 Red Hat Bugzilla 2024-03-08 04:25:32 UTC

The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days