Bug 1508618 - Hyperconver single host - failed to create single Brick
Summary: Hyperconver single host - failed to create single Brick
Keywords:
Status: CLOSED WORKSFORME
Alias: None
Product: ovirt-engine
Classification: oVirt
Component: BLL.Storage
Version: 4.2.0
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: ---
Assignee: Sahina Bose
QA Contact: Raz Tamir
URL:
Whiteboard:
Depends On: 1491771
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-11-01 19:19 UTC by Kevin Alon Goldblatt
Modified: 2017-12-06 14:35 UTC (History)
2 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2017-12-06 14:35:34 UTC
oVirt Team: Gluster
Embargoed:


Attachments (Terms of Use)
vdsm, server, engine logs (1.03 MB, application/x-gzip)
2017-11-01 19:23 UTC, Kevin Alon Goldblatt
no flags Details
vdsm, server, engine logs, supervdsm logs (1.62 MB, application/x-gzip)
2017-11-06 14:46 UTC, Kevin Alon Goldblatt
no flags Details

Description Kevin Alon Goldblatt 2017-11-01 19:19:33 UTC
Description of problem:
Fails to create single brick

Version-Release number of selected component (if applicable):
ovirt-engine-4.2.0-0.0.master.20171029154613.git19686f3.el7.centos.noarch
vdsm-4.20.5-5.git3d47978.el7.centos.x86_64
glusterfs-api-3.12.2-2.el7.x86_64
glusterfs-fuse-3.12.2-2.el7.x86_64
glusterfs-server-3.12.2-2.el7.x86_64
glusterfs-events-3.12.2-2.el7.x86_64
glusterfs-cli-3.12.2-2.el7.x86_64
glusterfs-geo-replication-3.12.2-2.el7.x86_64
glusterfs-3.12.2-2.el7.x86_64
glusterfs-libs-3.12.2-2.el7.x86_64
glusterfs-client-xlators-3.12.2-2.el7.x86_64
glusterfs-rdma-3.12.2-2.el7.x86_64


How reproducible:
100%

Steps to Reproduce:
1. After enabling gluster services on cluster host is up
2. Hosts -> Storge Device tab -> Create brick - select None for raid group, press enter >>>> Fails with None type has not attribute size
3.Try using raid type 0 >>>>

Actual results:
Fails to create brick

Expected results:
Should succeed

Additional info:
Engine log:
-----------
2017-11-01 18:43:30,939+02 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler2) [7d831a81] START, GlusterVolumesListVDSCommand(HostName = vm-83-162.scl.lab.tlv.r
edhat.com, GlusterVolumesListVDSParameters:{hostId='38368e80-a831-476b-b354-86e4f07637bf'}), log id: f10e2be
2017-11-01 18:43:31,893+02 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler2) [7d831a81] FINISH, GlusterVolumesListVDSCommand, return: {}, log id: f10e2be
2017-11-01 18:43:33,177+02 WARN  [org.ovirt.engine.core.vdsbroker.gluster.CreateBrickVDSCommand] (default task-13) [92223ebf-13d7-4fb2-89e0-12dd1f178af5] Unexpected return value: Status [code=-32603, message=Int
ernal JSON-RPC error: {'reason': "'NoneType' object has no attribute 'size'"}]
2017-11-01 18:43:33,177+02 WARN  [org.ovirt.engine.core.vdsbroker.gluster.CreateBrickVDSCommand] (default task-13) [92223ebf-13d7-4fb2-89e0-12dd1f178af5] Unexpected return value: Status [code=-32603, message=Int
ernal JSON-RPC error: {'reason': "'NoneType' object has no attribute 'size'"}]
2017-11-01 18:43:33,178+02 ERROR [org.ovirt.engine.core.vdsbroker.gluster.CreateBrickVDSCommand] (default task-13) [92223ebf-13d7-4fb2-89e0-12dd1f178af5] Failed in 'CreateBrickVDS' method
2017-11-01 18:43:33,178+02 WARN  [org.ovirt.engine.core.vdsbroker.gluster.CreateBrickVDSCommand] (default task-13) [92223ebf-13d7-4fb2-89e0-12dd1f178af5] Unexpected return value: Status [code=-32603, message=Int
ernal JSON-RPC error: {'reason': "'NoneType' object has no attribute 'size'"}]
2017-11-01 18:43:33,188+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-13) [92223ebf-13d7-4fb2-89e0-12dd1f178af5] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM
 vm-83-162.scl.lab.tlv.redhat.com command CreateBrickVDS failed: Internal JSON-RPC error: {'reason': "'NoneType' object has no attribute 'size'"}
2017-11-01 18:43:33,189+02 ERROR [org.ovirt.engine.core.vdsbroker.gluster.CreateBrickVDSCommand] (default task-13) [92223ebf-13d7-4fb2-89e0-12dd1f178af5] Command 'CreateBrickVDSCommand(HostName = vm-83-162.scl.l
ab.tlv.redhat.com, CreateBrickVDSParameters:{hostId='38368e80-a831-476b-b354-86e4f07637bf', lvName='brick_on', mountPoint='/gluster-bricks/brick_on', storageDevices='[StorageDevice:{id='c17abef3-498e-4414-8c78-a
7646e4595ee', name='vdb', devUuid='null', fsUuid='null', vdsId='38368e80-a831-476b-b354-86e4f07637bf', description='disk', devType='null', devPath='/dev/vdb', fsType='null', mountPoint='null', size='102400', can
CreateBrick='true', isGlusterBrick='false'}]', raidParams='[]', fsType='xfs'})' execution failed: VDSGenericException: VDSErrorException: Failed to CreateBrickVDS, error = Internal JSON-RPC error: {'reason': "'N
oneType' object has no attribute 'size'"}, code = -32603
2017-11-01 18:43:33,189+02 INFO  [org.ovirt.engine.core.vdsbroker.gluster.CreateBrickVDSCommand] (default task-13) [92223ebf-13d7-4fb2-89e0-12dd1f178af5] FINISH, CreateBrickVDSCommand, log id: 6e3fce01
2017-11-01 18:43:33,189+02 ERROR [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-13) [92223ebf-13d7-4fb2-89e0-12dd1f178af5] Command 'org.ovirt.engine.core.bll.gluster.CreateBrickCommand' fai
led: EngineException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed to CreateBrickVDS, error = Internal JSON-RPC error: {'reason': "'NoneType' object
 has no attribute 'size'"}, code = -32603 (Failed with error unexpected and code 16)

Comment 1 Kevin Alon Goldblatt 2017-11-01 19:23:55 UTC
Created attachment 1346708 [details]
vdsm, server, engine logs

Added logs

Comment 2 Yaniv Kaul 2017-11-02 07:57:54 UTC
Relevant log is from VDSM (no idea why you put engine log above in comment 1 ?):
2017-11-01 18:43:33,165+0200 ERROR (jsonrpc/0) [jsonrpc.JsonRpcServer] Internal server error (__init__:668)
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 663, in _handle_request
    res = method(**params)
  File "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py", line 197, in _dynamicMethod
    result = fn(*methodArgs)
  File "/usr/lib/python2.7/site-packages/vdsm/gluster/apiwrapper.py", line 87, in createBrick
    devList, fsType, raidParams)
  File "/usr/lib/python2.7/site-packages/vdsm/gluster/api.py", line 90, in wrapper
    rv = func(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/vdsm/gluster/api.py", line 701, in createBrick
    raidParams)
  File "/usr/lib/python2.7/site-packages/vdsm/supervdsm.py", line 53, in __call__
    return callMethod()
  File "/usr/lib/python2.7/site-packages/vdsm/supervdsm.py", line 51, in <lambda>
    **kwargs)
  File "<string>", line 2, in glusterCreateBrick
  File "/usr/lib64/python2.7/multiprocessing/managers.py", line 773, in _callmethod
    raise convert_to_error(kind, result)
AttributeError: 'NoneType' object has no attribute 'size'
2017-11-01 18:43:33,172+0200 INFO  (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC call GlusterHost.createBrick failed (error -32603) in 5.67 seconds (__init__:630)
2017-11-01 18:43:34,939+0200 INFO  (jsonrpc/2) [api.host] START getAllVmStats() from=::ffff:10.35.163.161,46068 (api:46)


Would you please attach supervdsm log as well?

Comment 3 Kevin Alon Goldblatt 2017-11-06 14:45:32 UTC
(In reply to Yaniv Kaul from comment #2)
> Relevant log is from VDSM (no idea why you put engine log above in comment 1
> ?):
> 2017-11-01 18:43:33,165+0200 ERROR (jsonrpc/0) [jsonrpc.JsonRpcServer]
> Internal server error (__init__:668)
> Traceback (most recent call last):
>   File "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 663,
> in _handle_request
>     res = method(**params)
>   File "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py", line 197, in
> _dynamicMethod
>     result = fn(*methodArgs)
>   File "/usr/lib/python2.7/site-packages/vdsm/gluster/apiwrapper.py", line
> 87, in createBrick
>     devList, fsType, raidParams)
>   File "/usr/lib/python2.7/site-packages/vdsm/gluster/api.py", line 90, in
> wrapper
>     rv = func(*args, **kwargs)
>   File "/usr/lib/python2.7/site-packages/vdsm/gluster/api.py", line 701, in
> createBrick
>     raidParams)
>   File "/usr/lib/python2.7/site-packages/vdsm/supervdsm.py", line 53, in
> __call__
>     return callMethod()
>   File "/usr/lib/python2.7/site-packages/vdsm/supervdsm.py", line 51, in
> <lambda>
>     **kwargs)
>   File "<string>", line 2, in glusterCreateBrick
>   File "/usr/lib64/python2.7/multiprocessing/managers.py", line 773, in
> _callmethod
>     raise convert_to_error(kind, result)
> AttributeError: 'NoneType' object has no attribute 'size'
> 2017-11-01 18:43:33,172+0200 INFO  (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC
> call GlusterHost.createBrick failed (error -32603) in 5.67 seconds
> (__init__:630)
> 2017-11-01 18:43:34,939+0200 INFO  (jsonrpc/2) [api.host] START
> getAllVmStats() from=::ffff:10.35.163.161,46068 (api:46)
> 
> 
> Would you please attach supervdsm log as well?

System is back up. Below is the supervdsm output. Adding logs

From supervdsm.log:
------------------------
MainProcess|jsonrpc/0::DEBUG::2017-11-01 18:43:33,159::devicetree::2773::blivet::(resolveDevice) resolved '/dev/vda1' to 'vda1' (partition)
MainProcess|jsonrpc/0::DEBUG::2017-11-01 18:43:33,161::storage_log::39::blivet::(log_method_call)            DeviceTree.getDeviceByName: hidden: False ; name: vg-brick_on ; incomplete: False ;
MainProcess|jsonrpc/0::DEBUG::2017-11-01 18:43:33,163::storage_log::47::blivet::(log_method_return)            DeviceTree.getDeviceByName returned None
MainProcess|jsonrpc/0::ERROR::2017-11-01 18:43:33,164::supervdsm_server::94::SuperVdsm.ServerCallback::(wrapper) Error in createBrick
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/vdsm/supervdsm_server.py", line 92, in wrapper
    res = func(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/vdsm/gluster/storagedev.py", line 275, in createBrick
    vgSizeKib = int(_getDeviceSize(vg, 'KiB'))
  File "/usr/lib/python2.7/site-packages/vdsm/gluster/storagedev.py", line 82, in _getDeviceSize
    return device.size.convertTo(spec=unitType)
AttributeError: 'NoneType' object has no attribute 'size'

Comment 4 Kevin Alon Goldblatt 2017-11-06 14:46:22 UTC
Created attachment 1348623 [details]
vdsm, server, engine logs, supervdsm logs

Adding logs

Comment 5 Yaniv Kaul 2017-11-06 14:57:33 UTC
(In reply to Kevin Alon Goldblatt from comment #4)
> Created attachment 1348623 [details]
> vdsm, server, engine logs, supervdsm logs
> 
> Adding logs

Comment 6 Kevin Alon Goldblatt 2017-11-16 17:12:08 UTC
Re-ran the scenario with the latest code:
-----------------------------------------------
ovirt-engine-4.2.0-0.0.master.20171114111003.git7aa1b91.el7.centos.noarch
vdsm-4.20.7-1.gitc9cf1ee.el7.centos.x86_64


Ran the scenario again:
----------------------------------------------
Steps to Reproduce:
1. After enabling gluster services on cluster host is up
2. Hosts -> Storge Device tab -> Create brick - select None for raid group, press enter >>>> Brick is created

Comment 7 Sahina Bose 2017-11-24 12:25:40 UTC
Can this bug be closed based on Comment 6?

Comment 8 Kevin Alon Goldblatt 2017-12-06 09:42:35 UTC
(In reply to Sahina Bose from comment #7)
> Can this bug be closed based on Comment 6?

Yes it is working now. Ran this scenario again with the latest code and it is working

Comment 9 Kevin Alon Goldblatt 2017-12-06 14:35:34 UTC
Works now with latest code as per comment 6

Moving to CLOSED


Note You need to log in before you can comment on or make changes to this bug.