Yuval, is this due to the issue we had in appliance about wrong permissions?

Sandro, probably not - the ovirt-imageio-daemon is installed in RHVH and looks like the packaging of this rpm is wrong (unless a chown is expeted somewhere):

[root@node-6740 ovirt_imageio_daemon]# rpm -q --dump ovirt-imageio-daemon|grep "daemon.log "
/var/log/ovirt-imageio-daemon/daemon.log 0 1521368643 e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 0100644 root root 0 0 0 X

(In reply to Yuval Turgeman from comment #4)
> Sandro, probably not - the ovirt-imageio-daemon is installed in RHVH and
> looks like the packaging of this rpm is wrong (unless a chown is expeted
> somewhere):
> 
> [root@node-6740 ovirt_imageio_daemon]# rpm -q --dump
> ovirt-imageio-daemon|grep "daemon.log "
> /var/log/ovirt-imageio-daemon/daemon.log 0 1521368643
> e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 0100644
> root root 0 0 0 X

The permissions of daemon.log file should be vdsm:kvm. It is set in 
ovirt-imageio-daemon.spec.in: "%dir %attr(755, vdsm, kvm) %{logdir}".
How is the daemon installed in HE env? Any difference than regular host deployment? Which permissions the appliance gets? Perhaps something changed in HE deployment recently (as I'm not aware of such issue in previous builds).

I'm not sure which version of ovirt-imageio-daemon we are talking about, but `rpm -q --dump` queries the rpm db itself.
I checked the latest version as well:

[yturgema@piggie ~/Downloads]$ rpm -qp --dump ovirt-imageio-daemon-1.3.1-0.el7ev.noarch.rpm|grep "daemon.log "
/var/log/ovirt-imageio-daemon/daemon.log 0 1523781570 e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 0100644 root root 0 0 0 X

As for the spec, I think the problem is this:

touch %{buildroot}%{logdir}/daemon.log

since it's created as root

Try to change the %ghost line for daemon.log to something like:
%ghost %attr(644, vdsm, kvm) %{logdir}/daemon.log*

Daniel, can you take a look?

(In reply to Yuval Turgeman from comment #6)
> I'm not sure which version of ovirt-imageio-daemon we are talking about, but
> `rpm -q --dump` queries the rpm db itself.
> I checked the latest version as well:
> 
> [yturgema@piggie ~/Downloads]$ rpm -qp --dump
> ovirt-imageio-daemon-1.3.1-0.el7ev.noarch.rpm|grep "daemon.log "
> /var/log/ovirt-imageio-daemon/daemon.log 0 1523781570
> e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 0100644
> root root 0 0 0 X
> 
> As for the spec, I think the problem is this:
> 
> touch %{buildroot}%{logdir}/daemon.log
> 
> since it's created as root
> 
> Try to change the %ghost line for daemon.log to something like:
> %ghost %attr(644, vdsm, kvm) %{logdir}/daemon.log*

Added - https://gerrit.ovirt.org/#/c/90699/

Seems fine now:

$ rpm -qp --dump ovirt-imageio-daemon-1.3.2-0.fc26.noarch.rpm | grep "daemon.log "

/var/log/ovirt-imageio-daemon/daemon.log 0 1524746121 e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 0100644 vdsm kvm 0 0 0 X

(In reply to Daniel Erez from comment #8)
> (In reply to Yuval Turgeman from comment #6)
> > I'm not sure which version of ovirt-imageio-daemon we are talking about, but
> > `rpm -q --dump` queries the rpm db itself.
> > I checked the latest version as well:
> > 
> > [yturgema@piggie ~/Downloads]$ rpm -qp --dump
> > ovirt-imageio-daemon-1.3.1-0.el7ev.noarch.rpm|grep "daemon.log "
> > /var/log/ovirt-imageio-daemon/daemon.log 0 1523781570
> > e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 0100644
> > root root 0 0 0 X
> > 
> > As for the spec, I think the problem is this:
> > 
> > touch %{buildroot}%{logdir}/daemon.log
> > 
> > since it's created as root
> > 
> > Try to change the %ghost line for daemon.log to something like:
> > %ghost %attr(644, vdsm, kvm) %{logdir}/daemon.log*
> 
> Added - https://gerrit.ovirt.org/#/c/90699/
> 
> Seems fine now:
> 
> $ rpm -qp --dump ovirt-imageio-daemon-1.3.2-0.fc26.noarch.rpm | grep
> "daemon.log "
> 
> /var/log/ovirt-imageio-daemon/daemon.log 0 1524746121
> e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 0100644
> vdsm kvm 0 0 0 X

@Simone - would it be good enough for fixing the HE deployment?

@Amar - is this issue reproducible only on a specific version? Can you please try it in latest 4.2 build?

It's probably not enough since it's %ghost, it would just fix the `rpm --verify`.  What versions are you running exactly ?

(In reply to Daniel Erez from comment #9)
> @Simone - would it be good enough for fixing the HE deployment?

Yes, I think so

(In reply to Simone Tiraboschi from comment #12)
> (In reply to Daniel Erez from comment #9)
> > @Simone - would it be good enough for fixing the HE deployment?
> 
> Yes, I think so

I'll release a new build of imageio then, so we'll see if it gets reproduced.

what are reproduction steps?
Is this a RHEVH specific?

Not being reproduced on RHEL systems:
alma03 ~]# ls -laZ /var/log/ovirt-imageio-daemon
drwxr-xr-x. vdsm kvm  system_u:object_r:var_log_t:s0   .
drwxr-xr-x. root root system_u:object_r:var_log_t:s0   ..
-rw-r--r--. vdsm kvm  system_u:object_r:var_log_t:s0   daemon.log

All deployments on all types of storages except from FC were successful for vintage and Node 0 using CLI.

Works for me on these components:
ovirt-engine-4.2.3.3-0.1.el7.noarch
rhvm-appliance-4.2-20180427.0.el7.noarch
ovirt-hosted-engine-setup-2.2.19-1.el7ev.noarch
ovirt-hosted-engine-ha-2.2.11-1.el7ev.noarch
Linux 3.10.0-862.el7.x86_64 #1 SMP Wed Mar 21 18:14:51 EDT 2018 x86_64 x86_64 x86_64 GNU/Linux
Red Hat Enterprise Linux Server release 7.5 (Maipo)

Please retest on RHVH as it turns to be RHVH specific bug.

You can reproduce this on RHEL as follows:

1. Make sure ovirt-imageio-daemon service is stopped and /var/log/ovirt-imageio-daemon/daemon.log doesn't exist.
2. systemctl start ovirt-imageio-daemon

3. Make sure ownership for /var/log/ovirt-imageio-daemon/daemon.log is correct:

[root@node-7140 ~]# ls -l /var/log/ovirt-imageio-daemon/daemon.log
-rw-r--r--. 1 vdsm kvm 1210 Apr 30 18:29 /var/log/ovirt-imageio-daemon/daemon.log

4. rpm -V ovirt-imageio-daemon will show wrong user/group:

[root@node-7140 ~]# rpm -V ovirt-imageio-daemon
.....UG..  g /var/log/ovirt-imageio-daemon/daemon.log

5. Ask rpm to fix the files according to its packaging:

[root@node-7140 ~]# rpm --setugids ovirt-imageio-daemon

6. Check again

[root@node-7140 ~]# ls -l /var/log/ovirt-imageio-daemon/daemon.log
-rw-r--r--. 1 root root 1362 May  2 09:15 /var/log/ovirt-imageio-daemon/daemon.log

Reproduction in this way will probably work, but the issue is in that on RHEL the file being created with correct UID and UG, thus deployment will be successful, so its solely RHVH specific issue.

(In reply to Nikolai Sednev from comment #28)
> Reproduction in this way will probably work, but the issue is in that on
> RHEL the file being created with correct UID and UG, thus deployment will be
> successful, so its solely RHVH specific issue.

Right, unless a RHEL-H user would run for some reason "rpm --setugids"...  Anyway, I just thought it was important to document the steps :)

Released fix to ovirt-imageio-daemon-1.3.1.1.

Daniel, can we respin the errata to get this included in the latest 4.2.3 compose?

(In reply to Allon Mureinik from comment #34)
> Daniel, can we respin the errata to get this included in the latest 4.2.3
> compose?

Yes, already added imageio-1.3.1.2 to 4.2.3.

hmm... The ET is already on REL_PREP, moving to ON_QA.

VERIFIED the bug on build redhat-virtualization-host-4.2-20180507.0(ovirt-imageio-daemon-1.3.1.2-0.el7ev.noarch)

Pre-check: Reproduced the issue with released build redhat-virtualization-host-4.1-20180314.0 upgrade to redhat-virtualization-host-4.1-20180410.1. After upgrading, the ownership for /var/log/ovirt-imageio-daemon/daemon.log was incorrect as the following. Host deployment failed.

# ls -al /var/log/ovirt-imageio-daemon/daemon.log
-rw-r--r--. 1 root root 302 May  8 09:50 /var/log/ovirt-imageio-daemon/daemon.log

Steps to verify this bug:
1. Installed the version redhat-virtualization-host-4.1-20180314.0
2. Made the daemon ovirt-imageio-daemon is running to generate the daemon.log before upgrading.(start the daemon manually or host deployment process and stop.)
3. Checked the ownership of /var/log/ovirt-imageio-daemon/daemon.log was vdsm:kvm
4. Upgraded to the version redhat-virtualization-host-4.2-20180507.0 successful by yum update.
5. Checked the ownership of /var/log/ovirt-imageio-daemon/daemon.log, the ownership is correct with vdsm:kvm after upgrading.
# ls -al /var/log/ovirt-imageio-daemon/daemon.log 
-rw-r--r--. 1 vdsm kvm 1096 May  8 13:28 /var/log/ovirt-imageio-daemon/daemon.log
6. Host deployment on the environment successful on nfs storage.

# hosted-engine --vm-status
--== Host 1 status ==--

conf_on_shared_storage             : True
Status up-to-date                  : True
Hostname                           : ibm-x3650m5-06.lab.eng.pek2.redhat.com
Host ID                            : 1
Engine status                      : {"health": "good", "vm": "up", "detail": "Up"}
Score                              : 3400
stopped                            : False
Local maintenance                  : False
crc32                              : 2cd86815
local_conf_timestamp               : 10967
Host timestamp                     : 10967
Extra metadata (valid at timestamp):
	metadata_parse_version=1
	metadata_feature_version=1
	timestamp=10967 (Tue May  8 14:22:43 2018)
	host-id=1
	score=3400
	vm_conf_refresh_time=10967 (Tue May  8 14:22:44 2018)
	conf_on_shared_storage=True
	maintenance=False
	state=EngineUp
	stopped=False

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:1520

BZ<2>Jira Resync

sync2jira

sync2jira