Bug 2179284

Summary:	[RHOSP 16.2] Rsyslog container looking pacemaker log file in the wrong directory
Product:	Red Hat OpenStack	Reporter:	Eric Nothen <enothen>
Component:	openstack-tripleo-heat-templates	Assignee:	Yadnesh Kulkarni <ykulkarn>
Status:	ON_QA ---	QA Contact:	Leonid Natapov <lnatapov>
Severity:	medium	Docs Contact:	mgeary <mgeary>
Priority:	high
Version:	16.2 (Train)	CC:	lmadsen, mburns, tkajinam, ykulkarn
Target Milestone:	z6	Keywords:	Triaged, ZStream
Target Release:	16.2 (Train on RHEL 8.4)
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	openstack-tripleo-heat-templates-11.6.1-2.20230717085025.1608f56.el8ost	Doc Type:	Bug Fix
Doc Text:	Host services such as pacemaker are mounted under "/var/log/host/" inside rsyslog container however the path held by configuration was the same as path on the host "/var/log/pacemaker/" which is why pacemaker logs couldn't be found by rsyslog service. This fix rectifies the pacemaker log path from "/var/log/pacemaker/" to "/var/log/host/pacemaker/".	Story Points:	---
Clone Of:		Environment:
Last Closed:		Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	2180933
Bug Blocks:	2179343

Description Eric Nothen 2023-03-17 08:27:10 UTC

This bug was initially created as a copy of Bug #1877475

I am copying this bug because: 

The same issue is still present in 16.2.4


Description of problem:
Rsyslog heat template point to wrong pacemaker log file. 


Version-Release number of selected component (if applicable):
RHOSP-16.1

How reproducible:
Steps to Reproduce:

Deploy RHOSP-16.1 enabling rsyslog template "/usr/share/openstack-tripleo-heat-templates/environments/logging-environment-rsyslog.yaml" following the URL : https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/16.1/html-single/logging_monitoring_and_troubleshooting_guide/index#testing_the_connection. After deployment, as per the rsyslog configuration, it looks for pacemaker logs in location "/var/log/host/pacemaker.log" which actually does not exist.

Actual results:
root@overcloud-controller-0 ~]# grep -A5 pacemaker_system.pacemaker /var/lib/config-data/puppet-generated/rsyslog/etc/rsyslog.d/50_openstack_logs.conf
# pacemaker_system.pacemaker
input(type="imfile"
  file=" "
  startmsg.regex="^[a-zA-Z]{3} [0-9]{2} [:0-9]{8}"
  tag="system.pacemaker"
)
[root@overcloud-controller-0 ~]#
Log file points to "/var/log/host/pacemaker.log" which does not exist.
[root@overcloud-controller-0 ~]# ls -l /var/log/host/pacemaker.log
ls: cannot access '/var/log/host/pacemaker.log': No such file or directory
[root@overcloud-controller-0 ~]# 

Expected results:


The log file to be pointed to "/var/log/pacemaker/pacemaker.log"
[root@overcloud-controller-0 ~]# ls -l /var/log/pacemaker/pacemaker.log 
-rw-rw----. 1 hacluster haclient 14313545 Sep  9 16:55 /var/log/pacemaker/pacemaker.log
[root@overcloud-controller-0 ~]#

Comment 2 Eric Nothen 2023-03-17 08:28:30 UTC

(overcloud) [stack.lab ~]$ rpm -q openstack-tripleo-heat-templates
openstack-tripleo-heat-templates-11.6.1-2.20221010235135.el8ost.noarch
(overcloud) [stack.lab ~]$ 
(overcloud) [stack.lab ~]$ ssh heat-admin@overcloud-controller-0 sudo grep -A5 pacemaker_system.pacemaker /var/lib/config-data/puppet-generated/rsyslog/etc/rsyslog.d/50_openstack_logs.conf
# pacemaker_system.pacemaker
input(type="imfile"
  file="/var/log/pacemaker/pacemaker.log"
  startmsg.regex="^[a-zA-Z]{3} [0-9]{2} [:0-9]{8}"
  tag="system.pacemaker"
  reopenOnTruncate="on"
(overcloud) [stack.lab ~]$ 
(overcloud) [stack.lab ~]$ ssh heat-admin@overcloud-controller-0 sudo podman exec rsyslog ls -l /var/log/{,host/}pacemaker/pacemaker.log
-rw-rw----. 1 189 189 48126262 Mar 17 08:26 /var/log/host/pacemaker/pacemaker.log
ls: cannot access '/var/log/pacemaker/pacemaker.log': No such file or directory
(overcloud) [stack.lab ~]$

Comment 3 Leif Madsen 2023-03-22 14:04:38 UTC

We think we might see this in 17.1 as well, so we'll need to get that resolved in Wallaby upstream and then backported to Train.