Hide Forgot
Description of problem: When systemd-journald restarts, the console connection from Horizon web interface to the VM does not work anymore until nova-novavncproxy is restarted (with command pcs resource restart openstack-nova-novncproxy-clone.) Version-Release number of selected component (if applicable): RHEL OSP 7 RHEL OSP 8 RHEL OSP 9 How reproducible: Everytime. Steps to Reproduce: 1. Check the status of "nova-novncproxy" and "systemd-journald" services. Verify that instance console is accessible. 2. Restart the "systemd-journald" service using systemd command. 3. "nova-novncproxy" is still running but instance console is not accessible. 4. Restart "nova-novncproxy" now instance console is accessible. Actual results: instance doesn't remain functional after restarting "systemd-journald" service. Expected results: instance console should remain functional after restarting "systemd-journald" service. Additional info: Opening this Bug with OSP 9 as latest version is also impacted with this issue. This isssue is also seen in OSP 8 and OSP 7.
Workaround : One way to fix is to modify shared by Cu : /lib/systemd/system/openstack-nova-novncproxy.service so that it does not use the default redirecting to systemd-journald by adding these 2 lines to it StandardOutput=null StandardError=null For example this way [root@controller-1 ~]# cat /lib/systemd/system/openstack-nova-novncproxy.service [Unit] Description=OpenStack Nova NoVNC Proxy Server After=syslog.target network.target [Service] Type=simple User=nova EnvironmentFile=-/etc/sysconfig/openstack-nova-novncproxy StandardOutput=null StandardError=null ExecStart=/usr/bin/nova-novncproxy --web /usr/share/novnc/ $OPTIONS [Install] WantedBy=multi-user.target
I'm pretty sure that restarting of the systemd-journald process is *not* supported by the systemd project. The journal holds the other end of the stdout/stderr FDs for processes connected to it, and when you restart journald those FDs get closed, breaking stdout/err for every process.
(In reply to Daniel Berrange from comment #2) > I'm pretty sure that restarting of the systemd-journald process is *not* > supported by the systemd project. The journal holds the other end of the > stdout/stderr FDs for processes connected to it, and when you restart > journald those FDs get closed, breaking stdout/err for every process. According to the last comment from Daniel, can Cu explain why there are restarting 'systemd-journald' ?
(In reply to Daniel Berrange from comment #2) > I'm pretty sure that restarting of the systemd-journald process is *not* > supported by the systemd project. The journal holds the other end of the > stdout/stderr FDs for processes connected to it, and when you restart > journald those FDs get closed, breaking stdout/err for every process. In NCIO16.5 there is one puppet module, which modifies journal configuration. To make that setting active, the systemd-journald must be restarted. I do not know if there is any other way. I think the idea here is to do that restart after all controller and compute nodes have been installed.. The change to journald configuration is to make journal logs storage persistent. Othewise the journald log information will be lost after rebooting the node. The fix creates /etc/systemd/journald.conf.d/persistent_storage.conf That has setting [Journal] Storage=persistent I heard that there should be a RedHat ticket for that problem (journald information will be lost after reboot).
WONTFIX/NOTABUG therefore QE Won't automate