Bug 1554283

Summary:	[downstream clone - 4.1.11] hosted-engine --get-shared-config rewrites all the hosted-engine configuration files loosing spm id
Product:	Red Hat Enterprise Virtualization Manager	Reporter:	RHV bug bot <rhv-bugzilla-bot>
Component:	ovirt-hosted-engine-ha	Assignee:	Martin Sivák <msivak>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Nikolai Sednev <nsednev>
Severity:	urgent	Docs Contact:
Priority:	urgent
Version:	unspecified	CC:	bugs, dfediuck, eedri, eheftman, lsurette, lsvaty, mavital, mgoldboi, msivak, stirabos, ykaul
Target Milestone:	ovirt-4.1.11	Keywords:	Triaged, ZStream
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	ovirt-hosted-engine-ha-2.1.11-1.el7ev	Doc Type:	Rebase: Bug Fixes and Enhancements
Doc Text:	Previously, /etc/hosted-engine/hosted-engine.conf was overwritten each time hosted-engine --get-shared-config was used due to a problem distinguishing between reload and refresh. In this release, hosted-engine --get-shared-config and hosted-engine --set-shared-config contain new options that enable local values to be correctly distinguished from shared values. The following configuration "types" (--type) were changed/added: - he_conf is now represented by he_local (local /etc) - he_shared (shared version of he_local used for newly deployed hosts)	Story Points:	---
Clone Of:	1543988	Environment:
Last Closed:	2019-02-22 08:35:05 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	Integration	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1543988
Bug Blocks:

Description RHV bug bot 2018-03-12 10:18:10 UTC

+++ This bug is an upstream to downstream clone. The original bug is: +++
+++   bug 1543988 +++
======================================================================

Description of problem:

hosted-engine --get-shared-config will write vm.conf with wrong ownership if not present 

[root@c74he20180108h1 ~]# ls -l /var/run/ovirt-hosted-engine-ha/vm.conf 
-rw-r--r--. 1 vdsm kvm 8113  9 feb 18.56 /var/run/ovirt-hosted-engine-ha/vm.conf
[root@c74he20180108h1 ~]# rm -f /var/run/ovirt-hosted-engine-ha/vm.conf; hosted-engine --get-shared-config gateway

gateway : 192.168.1.1, type : he_conf

[root@c74he20180108h1 ~]# ls -l /var/run/ovirt-hosted-engine-ha/vm.conf 
-rw-r--r--. 1 root root 8113  9 feb 18.56 /var/run/ovirt-hosted-engine-ha/vm.conf

ovirt-ha-agent runs as vdsm and so it will fail trying to refresh vm.conf

MainThread::INFO::2018-02-09 18:58:59,561::ovf_store::118::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) Found OVF_STORE: imgUUID:7482540b-ea01-4f8c-93d4-a9d46e93e6eb, volUUID:b7130665-8e41-44a3-970d-f97809bda7e7
MainThread::INFO::2018-02-09 18:59:00,132::ovf_store::118::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) Found OVF_STORE: imgUUID:c78da68d-fc54-48a1-9e99-bcd1f9712b21, volUUID:37e76796-21f3-4531-85a1-97e213fb534f
MainThread::INFO::2018-02-09 18:59:01,372::ovf_store::149::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF) OVF_STORE volume path: /var/run/vdsm/storage/997acd46-4433-4154-b6ea-99ff26da6fe9/c78da68d-fc54-48a1-9e99-bcd1f9712b21/37e76796-21f3-4531-85a1-97e213fb534f
MainThread::ERROR::2018-02-09 18:59:01,385::agent::144::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent
    return action(he)
  File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper
    return he.start_monitoring()
  File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 424, in start_monitoring
    self._config.refresh_vm_conf()
  File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 523, in refresh_vm_conf
    if self._publish_local_conf_file(VM, content):
  File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 409, in _publish_local_conf_file
    with open(localcopy_filename, 'w') as target:
IOError: [Errno 13] Permission denied: '/var/run/ovirt-hosted-engine-ha/vm.conf'



Version-Release number of selected component (if applicable):
2.2.8

How reproducible:
100%

Steps to Reproduce:
1. rm -f /var/run/ovirt-hosted-engine-ha/vm.conf; hosted-engine --get-shared-config gateway (as root)
2. ls -l /var/run/ovirt-hosted-engine-ha/vm.conf 
3.

Actual results:
/var/run/ovirt-hosted-engine-ha/vm.conf is owned by root:root

Expected results:
/var/run/ovirt-hosted-engine-ha/vm.conf is owned by vdsm:kvm

Additional info:

(Originally by Simone Tiraboschi)

Comment 1 RHV bug bot 2018-03-12 10:18:16 UTC

This looks by far worst than I though.
/etc/ovirt-hosted-engine/hosted-engine.conf got also rewrote by the copy on the shared storage and so all the host where hosted-engine --get-shared-config get executed will end with host_id=1 causing an spm collision almost for sure.

(Originally by Simone Tiraboschi)

Comment 3 RHV bug bot 2018-03-12 10:18:23 UTC

This looks by far worst than I though.
/etc/ovirt-hosted-engine/hosted-engine.conf got also rewrote by the copy on the shared storage and so all the host where hosted-engine --get-shared-config get executed will end with host_id=1 causing an spm collision almost for sure.

(Originally by Simone Tiraboschi)

Comment 4 RHV bug bot 2018-03-12 10:18:27 UTC

I think it affects 4.1 as well.

(Originally by Simone Tiraboschi)

Comment 5 Simone Tiraboschi 2018-03-19 13:09:41 UTC

All the patches are included in ovirt-hosted-engine-ha-2.1.11-1.el7ev and https://brewweb.engineering.redhat.com/brew/buildinfo?buildID=661953 has this in its changelog.
ovirt-hosted-engine-ha-2.1.11-1.el7ev is in https://errata.devel.redhat.com/advisory/32648 but it skipped this bug, maybe something wrong in the changelog syntax.

Comment 9 Sandro Bonazzola 2018-03-19 14:28:33 UTC

Postponing to 4.1.11 for allowing QE time to properly verifying this, despite the fix is already including in 4.1.10.

Comment 11 Nikolai Sednev 2018-03-21 12:56:06 UTC

Looks like its too early for this bug to reach QA, forth to our conversation with Anton, 4.1.11 is not yet scheduled.
Moving back to assigned.

Comment 12 Nikolai Sednev 2018-03-21 12:56:31 UTC

To modified.

Comment 13 Martin Sivák 2018-03-21 14:07:06 UTC

Nikolai, have you seen comment #9?

Comment 14 Nikolai Sednev 2018-03-21 14:43:48 UTC

(In reply to Martin Sivák from comment #13)
> Nikolai, have you seen comment #9?

No, I didn't. Anyway I will test it on 4.1.11 instead of 4.1.10 forth to that comment.

Comment 15 Simone Tiraboschi 2018-04-11 09:05:26 UTC

Moving now to ON_QA: no other builds are required for 4.1.11 since the fix was already in 4.1.10.

Comment 16 Nikolai Sednev 2018-04-12 15:34:19 UTC

Forth to discussion with Simone, moving this bug to verified.
Tested on these components:
ovirt-hosted-engine-setup-2.1.4.2-1.el7ev.noarch
ovirt-hosted-engine-ha-2.1.11-1.el7ev.noarch
rhvm-appliance-4.1.20180125.0-1.el7.noarch
Red Hat Enterprise Linux Server release 7.5 (Maipo)
Linux 3.10.0-862.el7.x86_64 #1 SMP Wed Mar 21 18:14:51 EDT 2018 x86_64 x86_64 x86_64 GNU/Linux

1.Deployed 2 ha-hosts.
2.Set global maintenance.
3.SHE-VM is running on first host.
4."cat /etc/ovirt-hosted-engine/hosted-engine.conf | grep host_id"
host_id=2
5."hosted-engine --set-shared-config gateway <some pingable IP> --type=he_shared"
6."rm -f /var/run/ovirt-hosted-engine-ha/vm.conf".
7."ls -l /var/run/ovirt-hosted-engine-ha/vm.conf"
-rw-r--r--. 1 vdsm kvm 2369 Apr 12 18:29 /var/run/ovirt-hosted-engine-ha/vm.conf
8."cat /etc/ovirt-hosted-engine/hosted-engine.conf | grep host_id"
host_id=2.