Bug 1314486

Summary:

dbus errors in skynet.log during an execution of a create cluster task

Product:

[Red Hat Storage] Red Hat Storage Console

Reporter:

Martin Bukatovic <mbukatov>

Component:

unclassified

Assignee:

Timothy Asir <tjeyasin>

Status:

CLOSED WONTFIX

QA Contact:

sds-qe-bugs

Severity:

medium

Docs Contact:

Priority:

unspecified

Version:

CC:

mkudlej, nthomas, sankarshan, tjeyasin

Target Milestone:

---

Target Release:

Hardware:

Unspecified

OS:

Unspecified

Whiteboard:

Fixed In Version:

rhscon-core-0.0.34-1.el7scon.x86_64 rhscon-ceph-0.0.33-1.el7scon.x86_64 rhscon-ui-0.0.47-1.el7scon.noarch

Doc Type:

Bug Fix

Doc Text:

Story Points:

---

Clone Of:

Environment:

Last Closed:

2017-03-23 04:12:38 UTC

Type:

Bug

Regression:

---

Mount Type:

---

Documentation:

---

CRM:

Verified Versions:

Category:

---

oVirt Team:

---

RHEL 7.3 requirements from Atomic Host:

Cloudforms Team:

---

Target Upstream Version:

Embargoed:

Attachments:

Description	Flags
skynet.log.bz2	none

Description Martin Bukatovic 2016-03-03 18:00:23 UTC

Created attachment 1132882 [details]
skynet.log.bz2

Description of problem
======================

During *Create Cluster* task, I see lot of ERROR messages in skynet.log file
complaining about dbus handlers.

Version-Release number of selected component
============================================

rhscon-core-0.0.8-10.el7.x86_64
rhscon-ui-0.0.19-1.el7.noarch
rhscon-ceph-0.0.6-10.el7.x86_64

rhscon-agent-0.0.3-3.el7.noarch

How reproducible
================

5% (not sure how to reproduce)

Steps to Reproduce
==================

Here are the steps I did:

1. Install usm following the documentation.
2. Create cluster 1st cluster, but the task failed
3. Removed the 1st cluster
4. Created 2nd cluster, the task finished this time
5. Look into skynet.log on node machines

Actual results
==============

There are lot of error messages in skynet.log files on all node machines:

~~~
2016-03-02 12:00:52,162 - INFO - skynetd: Initializing the daemon
2016-03-02 12:48:34,173 - INFO - skynetd: Initializing the daemon
2016-03-02 12:48:35,076 - INFO - skynetd: Started the daemon
2016-03-02 13:40:48,228 - ERROR - skynetd: Error in handler: block_add | Reason: 'NoneType' object is not iterable | Args: (<skynetd.callback.Callback object at 0x21b7150>, dbus.ObjectPath('/org/storaged/Storaged/block_devices/vdc'), dbus.Dictionary({dbus.String(u'org.storaged.Storaged.PartitionTable'): dbus.Dictionary({dbus.String(u'Type'): dbus.String(u'gpt', variant_level=1)}, signature=dbus.Signature('sv'))}, signature=dbus.Signature('sa{sv}'))) | Kwargs: {'path': dbus.ObjectPath('/org/storaged/Storaged')}
2016-03-02 13:40:48,230 - ERROR - skynetd: Error in handler: drive_add | Reason: 'NoneType' object is not iterable | Args: (<skynetd.callback.Callback object at 0x21b7150>, dbus.ObjectPath('/org/storaged/Storaged/block_devices/vdc'), dbus.Dictionary({dbus.String(u'org.storaged.Storaged.PartitionTable'): dbus.Dictionary({dbus.String(u'Type'): dbus.String(u'gpt', variant_level=1)}, signature=dbus.Signature('sv'))}, signature=dbus.Signature('sa{sv}'))) | Kwargs: {'path': dbus.ObjectPath('/org/storaged/Storaged')}
2016-03-02 13:40:50,886 - ERROR - skynetd: Error in handler: drive_add | Reason: 'NoneType' object is not iterable | Args: (<skynetd.callback.Callback object at 0x21b7150>, dbus.ObjectPath('/org/storaged/Storaged/block_devices/vdc2'), dbus.Dictionary({dbus.String(u'org.storaged.Storaged.Block'): dbus.Dictionary({dbus.String(u'IdType'): dbus.String(u'', variant_level=1), dbus.String(u'PreferredDevice'): dbus.Array([dbus.Byte(47), dbus.Byte(100), dbus.Byte(101), dbus.Byte(118), dbus.Byte(47), dbus.Byte(118), dbus.Byte(100), dbus.Byte(99), dbus.Byte(50), dbus.Byte(0)], signature=dbus.Signature('y'), variant_level=1), dbus.String(u'IdLabel'): dbus.String(u'', variant_level=1), dbus.String(u'HintSymbolicIconName'): dbus.String(u'', variant_level=1), dbus.String(u'HintPartitionable'): dbus.Boolean(True, variant_level=1), dbus.String(u'IdUUID'): dbus.String(u'', variant_level=1), dbus.String(u'MDRaid'): dbus.ObjectPath('/', variant_level=1), dbus.String(u'IdVersion'): dbus.String(u'', variant_level=1), dbus.String(u'ReadOnly'): dbus.Boolean(False, variant_level=1), dbus.String(u'CryptoBackingDevice'): dbus.ObjectPath('/', variant_level=1), dbus.String(u'Device'): dbus.Array([dbus.Byte(47), dbus.Byte(100), dbus.Byte(101), dbus.Byte(118), dbus.Byte(47), dbus.Byte(118), dbus.Byte(100), dbus.Byte(99), dbus.Byte(50), dbus.Byte(0)], signature=dbus.Signature('y'), variant_level=1), dbus.String(u'Configuration'): dbus.Array([], signature=dbus.Signature('(sa{sv})'), variant_level=1), dbus.String(u'Symlinks'): dbus.Array([dbus.Array([dbus.Byte(47), dbus.Byte(100), dbus.Byte(101), dbus.Byte(118), dbus.Byte(47), dbus.Byte(100), dbus.Byte(105), dbus.Byte(115), dbus.Byte(107), dbus.Byte(47), dbus.Byte(98), dbus.Byte(121), dbus.Byte(45), dbus.Byte(105), dbus.Byte(100), dbus.Byte(47), dbus.Byte(118), dbus.Byte(105), dbus.Byte(114), dbus.Byte(116), dbus.Byte(105), dbus.Byte(111), dbus.Byte(45), dbus.Byte(56), dbus.Byte(100), dbus.Byte(48), dbus.Byte(101), dbus.Byte(97), dbus.Byte(101), dbus.Byte(52), dbus.Byte(97), dbus.Byte(45), dbus.Byte(100), dbus.Byte(55), dbus.Byte(49), dbus.Byte(50), dbus.Byte(45), dbus.Byte(52), dbus.Byte(99), dbus.Byte(100), dbus.Byte(102), dbus.Byte(45), dbus.Byte(56), dbus.Byte(45), dbus.Byte(112), dbus.Byte(97), dbus.Byte(114), dbus.Byte(116), dbus.Byte(50), dbus.Byte(0)], signature=dbus.Signature('y')), dbus.Array([dbus.Byte(47), dbus.Byte(100), dbus.Byte(101), dbus.Byte(118), dbus.Byte(47), dbus.Byte(100), dbus.Byte(105), dbus.Byte(115), dbus.Byte(107), dbus.Byte(47), dbus.Byte(98), dbus.Byte(121), dbus.Byte(45), dbus.Byte(112), dbus.Byte(97), dbus.Byte(114), dbus.Byte(116), dbus.Byte(108), dbus.Byte(97), dbus.Byte(98), dbus.Byte(101), dbus.Byte(108), dbus.Byte(47), dbus.Byte(99), dbus.Byte(101), dbus.Byte(112), dbus.Byte(104), dbus.Byte(92), dbus.Byte(120), dbus.Byte(50), dbus.Byte(48), dbus.Byte(106), dbus.Byte(111), dbus.Byte(117), dbus.Byte(114), dbus.Byte(110), dbus.Byte(97), dbus.Byte(108), dbus.Byte(0)], signature=dbus.Signature('y')), dbus.Array([dbus.Byte(47), dbus.Byte(100), dbus.Byte(101), dbus.Byte(118), dbus.Byte(47), dbus.Byte(100), dbus.Byte(105), dbus.Byte(115), dbus.Byte(107), dbus.Byte(47), dbus.Byte(98), dbus.Byte(121), dbus.Byte(45), dbus.Byte(112), dbus.Byte(97), dbus.Byte(114), dbus.Byte(116), dbus.Byte(116), dbus.Byte(121), dbus.Byte(112), dbus.Byte(101), dbus.Byte(117), dbus.Byte(117), dbus.Byte(105), dbus.Byte(100), dbus.Byte(47), dbus.Byte(52), dbus.Byte(53), dbus.Byte(98), dbus.Byte(48), dbus.Byte(57), dbus.Byte(54), dbus.Byte(57), dbus.Byte(101), dbus.Byte(45), dbus.Byte(57), dbus.Byte(98), dbus.Byte(48), dbus.Byte(51), dbus.Byte(45), dbus.Byte(52), dbus.Byte(102), dbus.Byte(51), dbus.Byte(48), dbus.Byte(45), dbus.Byte(98), dbus.Byte(52), dbus.Byte(99), dbus.Byte(54), dbus.Byte(45), dbus.Byte(98), dbus.Byte(52), dbus.Byte(98), dbus.Byte(56), dbus.Byte(48), dbus.Byte(99), dbus.Byte(101), dbus.Byte(102), dbus.Byte(102), dbus.Byte(49), dbus.Byte(48), dbus.Byte(54), dbus.Byte(46), dbus.Byte(53), dbus.Byte(52), dbus.Byte(102), dbus.Byte(48), dbus.Byte(97), dbus.Byte(53), dbus.Byte(102), dbus.Byte(50), dbus.Byte(45), dbus.Byte(98), dbus.Byte(51), dbus.Byte(98), dbus.Byte(97), dbus.Byte(45), dbus.Byte(52), dbus.Byte(101), dbus.Byte(52), dbus.Byte(100), dbus.Byte(45), dbus.Byte(98), dbus.Byte(54), dbus.Byte(51), dbus.Byte(56), dbus.Byte(45), dbus.Byte(52), dbus.Byte(49), dbus.Byte(55), dbus.Byte(56), dbus.Byte(98), dbus.Byte(56), dbus.Byte(54), dbus.Byte(100), dbus.Byte(98), dbus.Byte(57), dbus.Byte(101), dbus.Byte(52), dbus.Byte(0)], signature=dbus.Signature('y')), dbus.Array([dbus.Byte(47), dbus.Byte(100), dbus.Byte(101), dbus.Byte(118), dbus.Byte(47), dbus.Byte(100), dbus.Byte(105), dbus.Byte(115), dbus.Byte(107), dbus.Byte(47), dbus.Byte(98), dbus.Byte(121), dbus.Byte(45), dbus.Byte(112), dbus.Byte(97), dbus.Byte(114), dbus.Byte(116), dbus.Byte(117), dbus.Byte(117), dbus.Byte(105), dbus.Byte(100), dbus.Byte(47), dbus.Byte(53), dbus.Byte(52), dbus.Byte(102), dbus.Byte(48), dbus.Byte(97), dbus.Byte(53), dbus.Byte(102), dbus.Byte(50), dbus.Byte(45), dbus.Byte(98), dbus.Byte(51), dbus.Byte(98), dbus.Byte(97), dbus.Byte(45), dbus.Byte(52), dbus.Byte(101), dbus.Byte(52), dbus.Byte(100), dbus.Byte(45), dbus.Byte(98), dbus.Byte(54), dbus.Byte(51), dbus.Byte(56), dbus.Byte(45), dbus.Byte(52), dbus.Byte(49), dbus.Byte(55), dbus.Byte(56), dbus.Byte(98), dbus.Byte(56), dbus.Byte(54), dbus.Byte(100), dbus.Byte(98), dbus.Byte(57), dbus.Byte(101), dbus.Byte(52), dbus.Byte(0)], signature=dbus.Signature('y'))], signature=dbus.Signature('ay'), variant_level=1), dbus.String(u'HintIconName'): dbus.String(u'', variant_level=1), dbus.String(u'Drive'): dbus.ObjectPath('/org/storaged/Storaged/drives/VirtIO_Disk_8d0eae4a_d712_4cdf_8', variant_level=1), dbus.String(u'IdUsage'): dbus.String(u'', variant_level=1), dbus.String(u'MDRaidMember'): dbus.ObjectPath('/', variant_level=1), dbus.String(u'DeviceNumber'): dbus.UInt64(64802L, variant_level=1), dbus.String(u'HintSystem'): dbus.Boolean(True, variant_level=1), dbus.String(u'HintName'): dbus.String(u'', variant_level=1), dbus.String(u'HintIgnore'): dbus.Boolean(False, variant_level=1), dbus.String(u'Id'): dbus.String(u'by-id-virtio-8d0eae4a-d712-4cdf-8-part2', variant_level=1), dbus.String(u'HintAuto'): dbus.Boolean(False, variant_level=1), dbus.String(u'Size'): dbus.UInt64(1072693760L, variant_level=1)}, signature=dbus.Signature('sv')), dbus.String(u'org.storaged.Storaged.Partition'): dbus.Dictionary({dbus.String(u'Name'): dbus.String(u'ceph journal', variant_level=1), dbus.String(u'Number'): dbus.UInt32(2L, variant_level=1), dbus.String(u'IsContainer'): dbus.Boolean(False, variant_level=1), dbus.String(u'UUID'): dbus.String(u'54f0a5f2-b3ba-4e4d-b638-4178b86db9e4', variant_level=1), dbus.String(u'Flags'): dbus.UInt64(0L, variant_level=1), dbus.String(u'Offset'): dbus.UInt64(1048576L, variant_level=1), dbus.String(u'IsContained'): dbus.Boolean(False, variant_level=1), dbus.String(u'Table'): dbus.ObjectPath('/org/storaged/Storaged/block_devices/vdc', variant_level=1), dbus.String(u'Type'): dbus.String(u'45b0969e-9b03-4f30-b4c6-b4b80ceff106', variant_level=1), dbus.String(u'Size'): dbus.UInt64(1072693760L, variant_level=1)}, signature=dbus.Signature('sv'))}, signature=dbus.Signature('sa{sv}'))) | Kwargs: {'path': dbus.ObjectPath('/org/storaged/Storaged')}

~~~

Full log file attached.

Expected results
================

There are no dbus errors.

Additional info
===============

On March 2 I discussed this with ndarshan who confirmed it as a valid issue.

Comment 2 Timothy Asir 2016-06-23 09:17:52 UTC

Patch sent to upstream for review: https://review.gerrithub.io/#/c/281518/

Comment 3 Martin Kudlej 2016-08-03 14:30:45 UTC

I still see this issue with
server:
ceph-ansible-1.0.5-31.el7scon.noarch
ceph-installer-1.0.14-1.el7scon.noarch
rhscon-ceph-0.0.39-1.el7scon.x86_64
rhscon-core-0.0.39-1.el7scon.x86_64
rhscon-core-selinux-0.0.39-1.el7scon.noarch
rhscon-ui-0.0.51-1.el7scon.noarch

node:
ceph-base-10.2.2-33.el7cp.x86_64
ceph-common-10.2.2-33.el7cp.x86_64
ceph-osd-10.2.2-33.el7cp.x86_64
ceph-selinux-10.2.2-33.el7cp.x86_64
libcephfs1-10.2.2-33.el7cp.x86_64
python-cephfs-10.2.2-33.el7cp.x86_64
rhscon-agent-0.0.16-1.el7scon.noarch
rhscon-core-selinux-0.0.39-1.el7scon.noarch

Comment 5 Nishanth Thomas 2016-08-05 06:56:07 UTC

Analysed this as per the discussion in the bug scrub. This won't create any issues as such. Hence moving this to next release.