Bug 1294473 - RHSC sync job fails when cluster has tiered volumes
Summary: RHSC sync job fails when cluster has tiered volumes
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat Storage
Component: rhsc
Version: rhgs-3.1
Hardware: x86_64
OS: Linux
high
high
Target Milestone: ---
: RHGS 3.1.2
Assignee: Ramesh N
QA Contact: Triveni Rao
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-12-28 11:30 UTC by Triveni Rao
Modified: 2016-05-16 04:39 UTC (History)
9 users (show)

Fixed In Version: rhsc-3.1.2-0.70
Doc Type: Bug Fix
Doc Text:
Previously, when you import a cluster with tiered volumes and other volumes, Red Hat Gluster Storage Console threw an exception and failed without displaying a proper error message. With this fix, when importing a cluster, Red Hat Gluster Storage ignores the tiered volume and unknown volume types that are not supported. Now, all supported volume types are synced correctly. Tiered volume topology is not displayed in Red Hat Gluster Storage Console and tiered volume actions are not supported.
Clone Of:
Environment:
Last Closed: 2016-03-01 06:13:02 UTC
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2016:0310 0 normal SHIPPED_LIVE Red Hat Gluster Storage Console 3.1 update 2 bug fixes 2016-03-01 10:31:05 UTC
oVirt gerrit 51156 0 master MERGED gluster: handle unsupported gluster volume types 2015-12-31 09:25:05 UTC
oVirt gerrit 51191 0 ovirt-engine-3.5-gluster MERGED gluster: handle unsupported gluster volume types 2015-12-31 11:14:58 UTC

Description Triveni Rao 2015-12-28 11:30:53 UTC
Description of problem:
==========================
With tired volume console is not able to sync other volumes. sync job fails on console.


Version-Release number of selected component (if applicable):
=============================================================
rhsc-3.1.2-0.69.el6.noarch

How reproducible:
==================
easily

Steps to Reproduce:
1. create a cluster with 2 nodes. create some normal volumes and tier volumes.
2. add one of the nodes to console.
3. import cluster, check that volumes are not synced.

Actual results:
===============
Volumes are not shown.

Expected results:
===================
Other normal volumes must be shown.

Additional info:
================

Console engine logs.

5b-44c2-a6c3-409a607193f2), log id: 61647214
2015-12-28 16:34:59,902 WARN  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc] (DefaultQuartzScheduler_Worker-66) [6daa57d6] Could not add brick 10.70.35.105:/rhs/brick1/v0 to volume 6a0d5b7c-359c-413f-9e70-90a1b498ded6 - server uuid 25383562-5a3c-4f25-a071-843f54ea26ef not found in cluster 99514e2d-320e-4947-8578-84f0d8baca7c
2015-12-28 16:34:59,905 WARN  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc] (DefaultQuartzScheduler_Worker-66) [6daa57d6] Could not add brick 10.70.35.105:/rhs/brick2/v0 to volume 6a0d5b7c-359c-413f-9e70-90a1b498ded6 - server uuid 25383562-5a3c-4f25-a071-843f54ea26ef not found in cluster 99514e2d-320e-4947-8578-84f0d8baca7c
2015-12-28 16:34:59,906 ERROR [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-66) [6daa57d6] Failed in GlusterVolumesListVDS method, for vds: host_rhel6.7_node1; host: 10.70.35.240
2015-12-28 16:34:59,906 ERROR [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-66) [6daa57d6] Command GlusterVolumesListVDSCommand(HostName = host_rhel6.7_node1, HostId = 34b59f2a-255b-44c2-a6c3-409a607193f2) execution failed. Exception: NullPointerException: 
2015-12-28 16:34:59,906 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-66) [6daa57d6] FINISH, GlusterVolumesListVDSCommand, log id: 61647214
2015-12-28 16:34:59,906 ERROR [org.ovirt.engine.core.bll.gluster.GlusterSyncJob] (DefaultQuartzScheduler_Worker-66) [6daa57d6] Error while refreshing Gluster lightweight data of cluster cluster2!: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: java.lang.NullPointerException (Failed with error ENGINE and code 5001)
        at org.ovirt.engine.core.bll.VdsHandler.handleVdsResult(VdsHandler.java:117) [bll.jar:]
        at org.ovirt.engine.core.bll.VDSBrokerFrontendImpl.RunVdsCommand(VDSBrokerFrontendImpl.java:33) [bll.jar:]
        at org.ovirt.engine.core.bll.gluster.GlusterJob.runVdsCommand(GlusterJob.java:68) [bll.jar:]
        at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.fetchVolumes(GlusterSyncJob.java:506) [bll.jar:]
        at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.fetchVolumes(GlusterSyncJob.java:492) [bll.jar:]
        at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshVolumeData(GlusterSyncJob.java:463) [bll.jar:]
        at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshClusterData(GlusterSyncJob.java:120) [bll.jar:]
        at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:99) [bll.jar:]
        at sun.reflect.GeneratedMethodAccessor21.invoke(Unknown Source) [:1.7.0_91]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_91]
:

   at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.fetchVolumes(GlusterSyncJob.java:492) [bll.jar:]
        at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshVolumeData(GlusterSyncJob.java:463) [bll.jar:]
        at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshClusterData(GlusterSyncJob.java:120) [bll.jar:]
        at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:99) [bll.jar:]
        at sun.reflect.GeneratedMethodAccessor21.invoke(Unknown Source) [:1.7.0_91]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_91]
        at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_91]
        at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:]
        at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:]
        at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:]

2015-12-28 16:35:04,937 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-47) START, GlusterServersListVDSCommand(HostName = 10.70.35.3, HostId = 22049403-047f-4e4a-b0a5-ebcef0d36a91), log id: 79600f8
2015-12-28 16:35:05,162 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-47) FINISH, GlusterServersListVDSCommand, return: [10.70.35.3:CONNECTED, dhcp35-42.lab.eng.blr.redhat.com:CONNECTED], log id: 79600f8
2015-12-28 16:35:05,169 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-47) START, GlusterVolumesListVDSCommand(HostName = 10.70.35.3, HostId = 22049403-047f-4e4a-b0a5-ebcef0d36a91), log id: 427fb483
2015-12-28 16:35:05,299 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-47) FINISH, GlusterVolumesListVDSCommand, return: {1fe7c229-25ad-4efe-88e0-79517ac5cab1=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@4bdfddc6}, log id: 427fb483
2015-12-28 16:35:05,320 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-47) START, GlusterServersListVDSCommand(HostName = host_rhel6.7_node1, HostId = 34b59f2a-255b-44c2-a6c3-409a607193f2), log id: 7fbaf0a7
2015-12-28 16:35:05,545 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler_Worker-47) FINISH, GlusterServersListVDSCommand, return: [10.70.35.240:CONNECTED, 10.70.35.105:CONNECTED], log id: 7fbaf0a7
2015-12-28 16:35:05,550 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-47) START, GlusterVolumesListVDSCommand(HostName = host_rhel6.7_node1, HostId = 34b59f2a-255b-44c2-a6c3-409a607193f2), log id: 41c89ed5
:

Comment 2 Ramesh N 2015-12-29 12:18:35 UTC
Tiered volumes are not handled correctly in RHGSC and it results in Nullpointer exception during volume sync. Either we have to ignore these tiered volumes or sync them properly and show them at least in the UI.

Comment 6 Triveni Rao 2016-01-06 07:51:26 UTC
The bug was verified with the build rhsc-3.1.2-0.70 and no issues found.

Steps Followed:
1. create a cluster with 2 nodes. create some normal volumes and tier volumes.
2. add one of the nodes to console.
3. import cluster, volumes are synced properly. Tier volumes were displayed.
4. Event log displayed warning message that ;Detected a volume Test_tier with type TIER on cluster cluster1 and it is not fully supported by engine

Few observations:
1. Tier volume type was shown as Tier.
2. Bricks sub-tab of Tier volumes showed empty.
3. Any volume management actions(snapshot,geo-rep,add bricks etc) are not supported by RHSC UI for Tiered Volume type.

Comment 7 Divya 2016-01-28 09:51:58 UTC
Sahina,

Please review and sign-off the edited doc text.

Comment 8 Sahina Bose 2016-01-29 10:53:06 UTC
Looks good to me

Comment 10 errata-xmlrpc 2016-03-01 06:13:02 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-0310.html


Note You need to log in before you can comment on or make changes to this bug.