Bug 1295381 - ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Error: 'unhashable type: 'dict'' - trying to restart agent
ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Error: 'unhashable typ...
Product: ovirt-hosted-engine-ha
Classification: oVirt
Component: Agent (Show other bugs)
Unspecified Unspecified
unspecified Severity medium (vote)
: ---
: ---
Assigned To: Martin Sivák
Ilanit Stein
Depends On:
Blocks: RHEV3.6Upgrade
  Show dependency treegraph
Reported: 2016-01-04 04:59 EST by Jiri Belka
Modified: 2016-01-25 06:05 EST (History)
5 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2016-01-25 06:05:00 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: SLA
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
rule-engine: planning_ack?
rule-engine: devel_ack?
rule-engine: testing_ack?

Attachments (Terms of Use)

  None (edit)
Description Jiri Belka 2016-01-04 04:59:06 EST
Description of problem:

MainThread::ERROR::2016-01-04 10:23:27,916::agent::205::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Error: 'unhashable type: 'dict'' - trying to restart agent
MainThread::WARNING::2016-01-04 10:23:32,921::agent::208::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Restarting agent, attempt '1'

Uff, it is ERROR level and it has no information value for an user observing agent.log.

Either make it different level or extend the message to contain useful value. Thx.

Version-Release number of selected component (if applicable):

How reproducible:
no idea

Steps to Reproduce:
1. no idea

Actual results:
MainThread::ERROR::2016-01-04 10:23:27,916::agent::205::ovirt_hosted_engine_ha.agent.agent.Agent::

Expected results:
ERROR level msg should have information value as it is ERROR, serious issue

Additional info:
Either make it different level or extend the message to contain useful value.
Comment 2 Martin Sivák 2016-01-04 05:04:11 EST
Simone, this is related to the upgrade code according to the log.

jbelka: It is a language error the user can do nothing about. Can you retest with the latest version? I believe we released or 6 before Christmas.
Comment 3 Simone Tiraboschi 2016-01-04 11:09:29 EST
The hosted engine-storage domain was created inside a custom folder within the NFS export which is not the typical configuration:
the export was while hosted-engine was deployed in and _brq-setup that was owned by root:root

It seams that current VDSM complains about that and unmount the storage so the upgrade was failing.

MainProcess|Thread-15::WARNING::2016-01-04 14:17:03,320::fileUtils::84::Storage.fileUtils::(validateAccess) Permission denied for directory: /rhev/data-center/mnt/ with permissions: 7
MainProcess|Thread-15::ERROR::2016-01-04 14:17:03,321::supervdsmServer::118::SuperVdsm.ServerCallback::(wrapper) Error in validateAccess
Traceback (most recent call last):
  File "/usr/share/vdsm/supervdsmServer", line 116, in wrapper
    res = func(*args, **kwargs)
  File "/usr/share/vdsm/supervdsmServer", line 302, in validateAccess
  File "/usr/share/vdsm/supervdsmServer", line 290, in _runAs
    raise err
OSError: [Errno 13] Permission denied

The same configuration was working with VDSM 4.16.

Fixing folder ownership to root:root and upgrading ovirt-hosted-engine-ha to allowed to conclude the upgrade.
Comment 4 Simone Tiraboschi 2016-01-05 05:09:56 EST
(In reply to Simone Tiraboschi from comment #3)
> Fixing folder ownership to root:root

Sorry, to vdsm:kvm
Comment 5 Doron Fediuck 2016-01-25 06:05:00 EST
looking into this the user manually added _brq-setup subdirectory with the wrong ownership (root:root) and entered the whole '' at hosted-engine setup.

This is something which wasn't properly checked previously. Specifically vdsm 4.16 was working with that configuration while vdsm 4.17 explicitly or better check NFS folder permission.

Based on the above this is not a bug and closing accordingly.

Note You need to log in before you can comment on or make changes to this bug.