Description of problem: MainThread::ERROR::2016-01-04 10:23:27,916::agent::205::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Error: 'unhashable type: 'dict'' - trying to restart agent MainThread::WARNING::2016-01-04 10:23:32,921::agent::208::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Restarting agent, attempt '1' Uff, it is ERROR level and it has no information value for an user observing agent.log. Either make it different level or extend the message to contain useful value. Thx. Version-Release number of selected component (if applicable): ovirt-hosted-engine-ha-1.3.3.3-1.el7ev.noarch How reproducible: no idea Steps to Reproduce: 1. no idea 2. 3. Actual results: MainThread::ERROR::2016-01-04 10:23:27,916::agent::205::ovirt_hosted_engine_ha.agent.agent.Agent:: Expected results: ERROR level msg should have information value as it is ERROR, serious issue Additional info: Either make it different level or extend the message to contain useful value.
Simone, this is related to the upgrade code according to the log. jbelka: It is a language error the user can do nothing about. Can you retest with the latest version? I believe we released 1.3.3.5 or 6 before Christmas.
The hosted engine-storage domain was created inside a custom folder within the NFS export which is not the typical configuration: the export was 10.34.63.202:/mnt/export/nfs/lv2 while hosted-engine was deployed in 10.34.63.202:/mnt/export/nfs/lv2/_brq-setup and _brq-setup that was owned by root:root It seams that current VDSM complains about that and unmount the storage so the upgrade was failing. MainProcess|Thread-15::WARNING::2016-01-04 14:17:03,320::fileUtils::84::Storage.fileUtils::(validateAccess) Permission denied for directory: /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv2___brq-setup with permissions: 7 MainProcess|Thread-15::ERROR::2016-01-04 14:17:03,321::supervdsmServer::118::SuperVdsm.ServerCallback::(wrapper) Error in validateAccess Traceback (most recent call last): File "/usr/share/vdsm/supervdsmServer", line 116, in wrapper res = func(*args, **kwargs) File "/usr/share/vdsm/supervdsmServer", line 302, in validateAccess kwargs=kwargs) File "/usr/share/vdsm/supervdsmServer", line 290, in _runAs raise err OSError: [Errno 13] Permission denied The same configuration was working with VDSM 4.16. Fixing folder ownership to root:root and upgrading ovirt-hosted-engine-ha to 1.3.3.6 allowed to conclude the upgrade.
(In reply to Simone Tiraboschi from comment #3) > Fixing folder ownership to root:root Sorry, to vdsm:kvm
OK, looking into this the user manually added _brq-setup subdirectory with the wrong ownership (root:root) and entered the whole '10.34.63.202:/mnt/export/nfs/lv2/_brq-setup' at hosted-engine setup. This is something which wasn't properly checked previously. Specifically vdsm 4.16 was working with that configuration while vdsm 4.17 explicitly or better check NFS folder permission. Based on the above this is not a bug and closing accordingly.