Bug 1294473

Summary: RHSC sync job fails when cluster has tiered volumes
Product: Red Hat Gluster Storage Reporter: Triveni Rao <trao>
Component: rhscAssignee: Ramesh N <rnachimu>
Status: CLOSED ERRATA QA Contact: Triveni Rao <trao>
Severity: high Docs Contact:
Priority: high    
Version: rhgs-3.1CC: asrivast, byarlaga, divya, nlevinki, rhs-bugs, rnachimu, sabose, sankarshan, sashinde
Target Milestone: ---Keywords: ZStream
Target Release: RHGS 3.1.2   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: rhsc-3.1.2-0.70 Doc Type: Bug Fix
Doc Text:
Previously, when you import a cluster with tiered volumes and other volumes, Red Hat Gluster Storage Console threw an exception and failed without displaying a proper error message. With this fix, when importing a cluster, Red Hat Gluster Storage ignores the tiered volume and unknown volume types that are not supported. Now, all supported volume types are synced correctly. Tiered volume topology is not displayed in Red Hat Gluster Storage Console and tiered volume actions are not supported.
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-03-01 06:13:02 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Triveni Rao 2015-12-28 11:30:53 UTC
Description of problem:
==========================
With tired volume console is not able to sync other volumes. sync job fails on console.


Version-Release number of selected component (if applicable):
=============================================================
rhsc-3.1.2-0.69.el6.noarch

How reproducible:
==================
easily

Steps to Reproduce:
1. create a cluster with 2 nodes. create some normal volumes and tier volumes.
2. add one of the nodes to console.
3. import cluster, check that volumes are not synced.

Actual results:
===============
Volumes are not shown.

Expected results:
===================
Other normal volumes must be shown.

Additional info:
================

Console engine logs.

5b-44c2-a6c3-409a607193f2), log id: 61647214
2015-12-28 16:34:59,902 WARN  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc] (DefaultQuartzScheduler_Worker-66) [6daa57d6] Could not add brick 10.70.35.105:/rhs/brick1/v0 to volume 6a0d5b7c-359c-413f-9e70-90a1b498ded6 - server uuid 25383562-5a3c-4f25-a071-843f54ea26ef not found in cluster 99514e2d-320e-4947-8578-84f0d8baca7c
2015-12-28 16:34:59,905 WARN  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc] (DefaultQuartzScheduler_Worker-66) [6daa57d6] Could not add brick 10.70.35.105:/rhs/brick2/v0 to volume 6a0d5b7c-359c-413f-9e70-90a1b498ded6 - server uuid 25383562-5a3c-4f25-a071-843f54ea26ef not found in cluster 99514e2d-320e-4947-8578-84f0d8baca7c
2015-12-28 16:34:59,906 ERROR [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-66) [6daa57d6] Failed in GlusterVolumesListVDS method, for vds: host_rhel6.7_node1; host: 10.70.35.240
2015-12-28 16:34:59,906 ERROR [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-66) [6daa57d6] Command GlusterVolumesListVDSCommand(HostName = host_rhel6.7_node1, HostId = 34b59f2a-255b-44c2-a6c3-409a607193f2) execution failed. Exception: NullPointerException: 
2015-12-28 16:34:59,906 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-66) [6daa57d6] FINISH, GlusterVolumesListVDSCommand, log id: 61647214
2015-12-28 16:34:59,906 ERROR [org.ovirt.engine.core.bll.gluster.GlusterSyncJob] (DefaultQuartzScheduler_Worker-66) [6daa57d6] Error while refreshing Gluster lightweight data of cluster cluster2!: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: java.lang.NullPointerException (Failed with error ENGINE and code 5001)
        at org.ovirt.engine.core.bll.VdsHandler.handleVdsResult(VdsHandler.java:117) [bll.jar:]
        at org.ovirt.engine.core.bll.VDSBrokerFrontendImpl.RunVdsCommand(VDSBrokerFrontendImpl.java:33) [bll.jar:]
        at org.ovirt.engine.core.bll.gluster.GlusterJob.runVdsCommand(GlusterJob.java:68) [bll.jar:]
        at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.fetchVolumes(GlusterSyncJob.java:506) [bll.jar:]
        at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.fetchVolumes(GlusterSyncJob.java:492) [bll.jar:]
        at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshVolumeData(GlusterSyncJob.java:463) [bll.jar:]
        at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshClusterData(GlusterSyncJob.java:120) [bll.jar:]
        at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:99) [bll.jar:]
        at sun.reflect.GeneratedMethodAccessor21.invoke(Unknown Source) [:1.7.0_91]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_91]
:

   at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.fetchVolumes(GlusterSyncJob.java:492) [bll.jar:]
        at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshVolumeData(GlusterSyncJob.java:463) [bll.jar:]
        at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshClusterData(GlusterSyncJob.java:120) [bll.jar:]
        at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:99) [bll.jar:]
        at sun.reflect.GeneratedMethodAccessor21.invoke(Unknown Source) [:1.7.0_91]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_91]
        at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_91]
        at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:]
        at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:]
        at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:]

2015-12-28 16:35:04,937 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-47) START, GlusterServersListVDSCommand(HostName = 10.70.35.3, HostId = 22049403-047f-4e4a-b0a5-ebcef0d36a91), log id: 79600f8
2015-12-28 16:35:05,162 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-47) FINISH, GlusterServersListVDSCommand, return: [10.70.35.3:CONNECTED, dhcp35-42.lab.eng.blr.redhat.com:CONNECTED], log id: 79600f8
2015-12-28 16:35:05,169 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-47) START, GlusterVolumesListVDSCommand(HostName = 10.70.35.3, HostId = 22049403-047f-4e4a-b0a5-ebcef0d36a91), log id: 427fb483
2015-12-28 16:35:05,299 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-47) FINISH, GlusterVolumesListVDSCommand, return: {1fe7c229-25ad-4efe-88e0-79517ac5cab1=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@4bdfddc6}, log id: 427fb483
2015-12-28 16:35:05,320 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-47) START, GlusterServersListVDSCommand(HostName = host_rhel6.7_node1, HostId = 34b59f2a-255b-44c2-a6c3-409a607193f2), log id: 7fbaf0a7
2015-12-28 16:35:05,545 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-47) FINISH, GlusterServersListVDSCommand, return: [10.70.35.240:CONNECTED, 10.70.35.105:CONNECTED], log id: 7fbaf0a7
2015-12-28 16:35:05,550 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-47) START, GlusterVolumesListVDSCommand(HostName = host_rhel6.7_node1, HostId = 34b59f2a-255b-44c2-a6c3-409a607193f2), log id: 41c89ed5
:

Comment 2 Ramesh N 2015-12-29 12:18:35 UTC
Tiered volumes are not handled correctly in RHGSC and it results in Nullpointer exception during volume sync. Either we have to ignore these tiered volumes or sync them properly and show them at least in the UI.

Comment 6 Triveni Rao 2016-01-06 07:51:26 UTC
The bug was verified with the build rhsc-3.1.2-0.70 and no issues found.

Steps Followed:
1. create a cluster with 2 nodes. create some normal volumes and tier volumes.
2. add one of the nodes to console.
3. import cluster, volumes are synced properly. Tier volumes were displayed.
4. Event log displayed warning message that ;Detected a volume Test_tier with type TIER on cluster cluster1 and it is not fully supported by engine

Few observations:
1. Tier volume type was shown as Tier.
2. Bricks sub-tab of Tier volumes showed empty.
3. Any volume management actions(snapshot,geo-rep,add bricks etc) are not supported by RHSC UI for Tiered Volume type.

Comment 7 Divya 2016-01-28 09:51:58 UTC
Sahina,

Please review and sign-off the edited doc text.

Comment 8 Sahina Bose 2016-01-29 10:53:06 UTC
Looks good to me

Comment 10 errata-xmlrpc 2016-03-01 06:13:02 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-0310.html