+++ This bug is a downstream clone. The original bug is: +++ +++ bug 1685034 +++ ====================================================================== Description of problem: Another kind of bug where the /var/log/messages is flooded with: ovs-vsctl: ovs|00001|db_ctl_base|ERR|unix:/var/run/openvswitch/db.sock: database connection failed (No such file or directory) This time most likely the culprit is: /usr/libexec/vdsm/hooks/after_get_caps/*openstacknet* those hooks are querying even if services ovs-vswitchd.service and ovsdb-server.service are disabled. Not configured services are fine, it is the result of the question during "engine-setup" DEBUG otopi.plugins.otopi.dialog.human dialog.__logString:204 DIALOG:SEND Configure ovirt-provider-ovn (Yes, No) [Yes]: DEBUG otopi.plugins.otopi.dialog.human dialog.__logString:204 DIALOG:SEND Set up ovirt-provider-ovn : False Version-Release number of selected component (if applicable): 4.2.8 How reproducible: always Steps to Reproduce: 1. upgrade from 4.1.z 2. during "engine-setup" disable "Set up ovirt-provider-ovn" Actual results: /var/log/messages flooded with error messages Expected results: If ovs services are not configured, hooks shouldn't query Additional info: I had problem to find out who produces those messages. I found using SELinx and watch rule: auditctl -w /usr/libexec/vdsm/hooks/after_get_caps/ -p x -k bug (Originally by Olimp Bockowski)
Dan, what do you expect to see in the logfile, if openvswitch is not running? (Originally by Dominik Holler)
We can log this to vdsm (probably only once when vdsm starts) We clearly should not spam /var/log/message (Originally by danken)
*** Bug 1694232 has been marked as a duplicate of this bug. *** (Originally by Dominik Holler)
vdsm's hook log was already hiding this since commit 5d585ba0ada0758eafa87225fe8213efe83cbea8, which aims to avoid log noise *whenever* the odl key is missing. Looks like that commit takes things a bit too far, and removes all syslog noise. Not sure that's what we want, but, from my perspective, this bug can be verified (on the vdsm + ovn driver side the logs are no longer spammed). (Originally by Miguel Duarte de Mora Barroso)
Changing to ovirt-provider-ovn, because the issue is fixed in vdsm in 4.2.8-3 by bug 1673765 already. This should not change the verification of the bug, because /var/log/messages of a host should not contain the line ovs-vsctl: ovs|00001|db_ctl_base|ERR|unix:/var/run/openvswitch/db.sock: database connection failed (No such file or directory) at all. (Originally by Dominik Holler)
(In reply to Dominik Holler from comment #9) > Changing to ovirt-provider-ovn, because the issue is fixed in vdsm in > 4.2.8-3 by bug 1673765 already. > This should not change the verification of the bug, because > /var/log/messages of a host should not contain the line > ovs-vsctl: ovs|00001|db_ctl_base|ERR|unix:/var/run/openvswitch/db.sock: > database connection failed (No such file or directory) > at all. I don't understand. BZ 1673765 doesn't related to this bug. Also this fix is also on vdsm + ovn provider. (Originally by Michael Burman)
(In reply to Michael Burman from comment #10) > > Changing to ovirt-provider-ovn, because the issue is fixed in vdsm in > > 4.2.8-3 by bug 1673765 already. > > This should not change the verification of the bug, because > > /var/log/messages of a host should not contain the line > > ovs-vsctl: ovs|00001|db_ctl_base|ERR|unix:/var/run/openvswitch/db.sock: > > database connection failed (No such file or directory) > > at all. > I don't understand. BZ 1673765 doesn't related to this bug. Also this fix is also on vdsm + ovn provider. The fix to BZ 1673765 disables logging to var log messages from the vdsm openstack hook, which means that the vdsm change was not strictly needed, despite making the flow more correct, since the hook bails out when the OVS services are not available. That change was *not* backported into 4.3, and is only available in master (it'll eventually find its way to 4.4). (Originally by Miguel Duarte de Mora Barroso)
(In reply to Miguel Duarte Barroso from comment #11) > (In reply to Michael Burman from comment #10) > > > > Changing to ovirt-provider-ovn, because the issue is fixed in vdsm in > > > 4.2.8-3 by bug 1673765 already. > > > This should not change the verification of the bug, because > > > /var/log/messages of a host should not contain the line > > > ovs-vsctl: ovs|00001|db_ctl_base|ERR|unix:/var/run/openvswitch/db.sock: > > > database connection failed (No such file or directory) > > > at all. > > > I don't understand. BZ 1673765 doesn't related to this bug. Also this fix is also on vdsm + ovn provider. > > The fix to BZ 1673765 disables logging to var log messages from the vdsm > openstack hook, which means that the vdsm change was > not strictly needed, despite making the flow more correct, since the hook > bails out when the OVS services are not available. > > That change was *not* backported into 4.3, and is only available in master > (it'll eventually find its way to 4.4). I don't understand. This comment is even more confusing. So rhv 4.3 won't have the fix? only 4.4 you say (Originally by Michael Burman)
If this is the case this should be re targeted to 4.4 (Originally by Michael Burman)
VDSM of RHV 4.3 won't have the issue. This bug fixed in vdsm in 4.2.8-3 by the fix for bug 1673765 already, without our awareness. So we just have to fix in ovirt-provider-ovn-driver for 4.3.4. If the bug would be verified on a host with ovirt-provider-ovn-driver or the opentstack hook installed on a 4.3.4 host by checking that /var/log/messages does not contain ovs-vsctl: ovs|00001|db_ctl_base|ERR|unix:/var/run/openvswitch/db.sock: database connection failed (No such file or directory) both issues, the one in ovirt-provider-ovn-driver and the one in the opentstack hook, are covered. (Originally by Dominik Holler)
(In reply to Dominik Holler from comment #14) > VDSM of RHV 4.3 won't have the issue. > This bug fixed in vdsm in 4.2.8-3 by the fix for bug 1673765 already, > without our awareness. > So we just have to fix in ovirt-provider-ovn-driver for 4.3.4. > > If the bug would be verified on a host with ovirt-provider-ovn-driver or the > opentstack hook installed on a 4.3.4 host by checking that > /var/log/messages does not contain > ovs-vsctl: ovs|00001|db_ctl_base|ERR|unix:/var/run/openvswitch/db.sock: > database connection failed (No such file or directory) > both issues, the one in ovirt-provider-ovn-driver and the one in the > opentstack hook, are covered. Dominik, this can't be 4.3.4 if it has no patches pushed to 4.3.4 code. You doing me a mess here (Originally by Michael Burman)
> Dominik, this can't be 4.3.4 if it has no patches pushed to 4.3.4 code. The fix for ovirt-provider-ovn-driver will be pushed to 4.3.4, this is why I changed the component to ovirt-provider-ovn. (Originally by Dominik Holler)
Dominik, in Comment 14 you state that this bug has been fixed in 4.2.8-3; but we're currently on 4.2.8.7-0 and are still seeing /var/log/messages flooded with these messages. Was the fix you mentioned reverted, or was it not actually fixed? (Originally by paulds)
(In reply to Paul Stauffer, comment#17) > Dominik, in Comment 14 you state that this bug has been fixed in 4.2.8-3; but we're currently on 4.2.8.7-0 and are still seeing /var/log/messages flooded with these messages. Was the fix you mentioned reverted, or was it not actually fixed? There were two sources of ovs errors, one in the driver hook, and another in vdsm. The messages originating from vdsm were capped, and that fix is available (as Dominik says) since 4.2.8-3. Unfortunately, the driver hook is still spamming the var log messages. The fix for the driver hook is available on ovirt-provider-ovn-driver version 1.2.21, which will be released in ovirt-4.3.4 (Originally by Miguel Duarte de Mora Barroso)
Michel, can you please ack this bug for 4.3.4? (Originally by Dominik Holler)
Verified on - ovirt-provider-ovn-1.2.22-1.el7ev.noarch with rhvm-4.3.4-0.1.el7.noarch ovirt-provider-ovn-driver-1.2.22-1.el7ev.noarch vdsm-4.30.15-1.el7ev.x86_64 rhv-openvswitch-ovn-host-2.10-6.el7ev.noarch rhv-openvswitch-ovn-common-2.10-6.el7ev.noarch openvswitch2.10-ovn-common-2.10.0-28.el7fdp.3.x86_64 openvswitch2.10-ovn-host-2.10.0-28.el7fdp.3.x86_64 (Originally by Michael Burman)
*** Bug 1715077 has been marked as a duplicate of this bug. *** (Originally by Dominik Holler)
Verified according to comment #21
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:1557