Red Hat Bugzilla – Bug 1295381
ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Error: 'unhashable type: 'dict'' - trying to restart agent
Last modified: 2016-01-25 06:05:00 EST
Description of problem:
MainThread::ERROR::2016-01-04 10:23:27,916::agent::205::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Error: 'unhashable type: 'dict'' - trying to restart agent
MainThread::WARNING::2016-01-04 10:23:32,921::agent::208::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Restarting agent, attempt '1'
Uff, it is ERROR level and it has no information value for an user observing agent.log.
Either make it different level or extend the message to contain useful value. Thx.
Version-Release number of selected component (if applicable):
Steps to Reproduce:
1. no idea
ERROR level msg should have information value as it is ERROR, serious issue
Either make it different level or extend the message to contain useful value.
Simone, this is related to the upgrade code according to the log.
jbelka: It is a language error the user can do nothing about. Can you retest with the latest version? I believe we released 220.127.116.11 or 6 before Christmas.
The hosted engine-storage domain was created inside a custom folder within the NFS export which is not the typical configuration:
the export was 10.34.63.202:/mnt/export/nfs/lv2 while hosted-engine was deployed in 10.34.63.202:/mnt/export/nfs/lv2/_brq-setup and _brq-setup that was owned by root:root
It seams that current VDSM complains about that and unmount the storage so the upgrade was failing.
MainProcess|Thread-15::WARNING::2016-01-04 14:17:03,320::fileUtils::84::Storage.fileUtils::(validateAccess) Permission denied for directory: /rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv2___brq-setup with permissions: 7
MainProcess|Thread-15::ERROR::2016-01-04 14:17:03,321::supervdsmServer::118::SuperVdsm.ServerCallback::(wrapper) Error in validateAccess
Traceback (most recent call last):
File "/usr/share/vdsm/supervdsmServer", line 116, in wrapper
res = func(*args, **kwargs)
File "/usr/share/vdsm/supervdsmServer", line 302, in validateAccess
File "/usr/share/vdsm/supervdsmServer", line 290, in _runAs
OSError: [Errno 13] Permission denied
The same configuration was working with VDSM 4.16.
Fixing folder ownership to root:root and upgrading ovirt-hosted-engine-ha to 18.104.22.168 allowed to conclude the upgrade.
(In reply to Simone Tiraboschi from comment #3)
> Fixing folder ownership to root:root
Sorry, to vdsm:kvm
looking into this the user manually added _brq-setup subdirectory with the wrong ownership (root:root) and entered the whole '10.34.63.202:/mnt/export/nfs/lv2/_brq-setup' at hosted-engine setup.
This is something which wasn't properly checked previously. Specifically vdsm 4.16 was working with that configuration while vdsm 4.17 explicitly or better check NFS folder permission.
Based on the above this is not a bug and closing accordingly.