Bug 2160772
| Summary: | Amphora log missing from default set of forwarded logs | ||
|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Eric Nothen <enothen> |
| Component: | openstack-tripleo-heat-templates | Assignee: | Gregory Thiemonge <gthiemon> |
| Status: | CLOSED ERRATA | QA Contact: | Bruna Bonguardo <bbonguar> |
| Severity: | low | Docs Contact: | |
| Priority: | low | ||
| Version: | 16.1 (Train) | CC: | gregraka, gthiemon, mariel, mburns, njohnston, tweining, ykarel |
| Target Milestone: | z6 | Keywords: | Triaged |
| Target Release: | 16.2 (Train on RHEL 8.4) | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | openstack-tripleo-heat-templates-11.6.1-2.20230323164846.5a8404a.el8ost | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2023-11-08 19:18:30 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Red Hat OpenStack Platform 16.2.6 (Train) bug fix and enhancement advisory), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2023:6307 |
Description of problem: When setting up log forwarding [0] and octavia log offload [1] at the same time, the log file in which octavia dumps amphora logs is not included on the default set of files that are forwarded to Splunk/Elastic/etc. Similar to what I reported earlier for Horizon [2] and HAproxy, but the difference here is that the amphora log is optional and only exists if log offloading is enabled. Version-Release number of selected component (if applicable): 16.x How reproducible: Always reproducible. Steps to Reproduce: 1. Deploy RHOSP 16 with log forwarding and octavia log offloading 2. Check that the octavia_rsyslog container is running: ~~~ (overcloud) [stack@director ~]$ ansible -i inventory.yaml -m shell -a 'podman ps -f name=octavia_rsyslog' -b Controller overcloud-controller-0 | CHANGED | rc=0 >> CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 16ba34bd959e director.ctlplane.localdomain:8787/rhosp-rhel8/openstack-rsyslog:16.2 kolla_start About an hour ago Up About an hour ago octavia_rsyslog overcloud-controller-1 | CHANGED | rc=0 >> CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES f5c0f9a4cdf2 director.ctlplane.localdomain:8787/rhosp-rhel8/openstack-rsyslog:16.2 kolla_start About an hour ago Up About an hour ago octavia_rsyslog overcloud-controller-2 | CHANGED | rc=0 >> CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 84c4d5df210f director.ctlplane.localdomain:8787/rhosp-rhel8/openstack-rsyslog:16.2 kolla_start About an hour ago Up About an hour ago octavia_rsyslog ~~~ 3. Create an LB ~~~ (overcloud) [stack@director ~]$ openstack loadbalancer create --flavor activestandby --name testlb-stdby-01 --vip-subnet-id external ~~~ 3. After some minutes, the amphora log file shows some content: ~~~ (overcloud) [stack@director ~]$ ansible -i inventory.yaml -m shell -a 'ls -l /var/log/containers/octavia-amphorae/' -b Controller overcloud-controller-2 | CHANGED | rc=0 >> total 0 overcloud-controller-0 | CHANGED | rc=0 >> total 28 -rw-r--r--. 1 root root 28385 Jan 13 16:05 octavia-amphora.log overcloud-controller-1 | CHANGED | rc=0 >> total 0 ~~~ 4. However, this file is not included on the default set of files that are forwarded (but the rest of the octavia files are): ~~~ [root@overcloud-controller-0 ~]# egrep file= /var/lib/config-data/puppet-generated/rsyslog/etc/rsyslog.d/50_openstack_logs.conf file="/var/log/containers/cinder/cinder-scheduler.log" file="/var/log/containers/cinder/cinder-volume.log" file="/var/log/containers/glance/api.log" file="/var/log/containers/haproxy/haproxy.log" file="/var/log/containers/heat/heat_api_cfn.log" file="/var/log/containers/heat/heat_api.log" file="/var/log/containers/heat/heat-engine.log" file="/var/log/containers/horizon/horizon.log" file="/var/log/containers/keystone/keystone.log" file="/var/log/containers/neutron/server.log" file="/var/log/containers/nova/nova-api.log" file="/var/log/containers/nova/nova-conductor.log" file="/var/log/containers/nova/nova-metadata-api.log" file="/var/log/containers/nova/nova-scheduler.log" file="/var/log/containers/nova/nova-novncproxy.log" file="/var/log/containers/octavia/api.log" file="/var/log/containers/octavia/health-manager.log" file="/var/log/containers/octavia/housekeeping.log" file="/var/log/containers/octavia/worker.log" file="/var/log/pacemaker/pacemaker.log" file="/var/log/host/messages" errorfile="/var/log/rsyslog/omelasticsearch.log" [root@overcloud-controller-0 ~]# Actual results: /var/log/containers/octavia-amphorae/octavia-amphora.log is not on the list of files to be forwarded by rsyslog. Expected results: /var/log/containers/octavia-amphorae/octavia-amphora.log comes from a container deployed by TripleO, so it should be forwarded in the same way that they other logs from the rest of the containers, especially since the rest of the octavia files are already included. Additional info: [0] https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/16.1/html-single/logging_monitoring_and_troubleshooting_guide/index#enabling-centralized-logging-during-deployment_installing-and-configuring-the-logs-service [1] https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/16.1/html-single/using_octavia_for_load_balancing-as-a-service/index#enable-lb-service-instance-admin-log-offload_manage-lb-service-instance-logs [1] https://bugzilla.redhat.com/show_bug.cgi?id=2034730 [2] https://bugzilla.redhat.com/show_bug.cgi?id=2026290