Bug 1578257

Summary: Unable to schedule a Snapshot of a Gluster volume
Product: [oVirt] ovirt-engine Reporter: Mark Betham <mark.betham>
Component: BLL.GlusterAssignee: Sahina Bose <sabose>
Status: CLOSED CURRENTRELEASE QA Contact: bipin <bshetty>
Severity: high Docs Contact:
Priority: high    
Version: 4.2.3.5CC: amureini, bshetty, bugs, lveyde, sabose
Target Milestone: ovirt-4.2.4Flags: rule-engine: ovirt-4.2?
bshetty: planning_ack?
rule-engine: devel_ack+
bshetty: testing_ack+
Target Release: 4.2.4.1   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: ovirt-engine-4.2.4.1 Doc Type: Bug Fix
Doc Text:
Cause: Quartz scheduled jobs failed to fire due to issue with dependency classes Consequence: Snapshot scheduling did not work Fix: Dependency classes are instantiated in the job class Result: Snapshot scheduling works
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-06-26 08:46:19 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Gluster RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Engine Log File
none
Additional log info with debug flag as requested
none
Screenshot1_Verified
none
Screenshot2_Verified none

Description Mark Betham 2018-05-15 06:42:49 UTC
Created attachment 1436699 [details]
Engine Log File

Description of problem:
After creating a schedule for the snapshotting of a Gluster volume no snapshots are taken.

Version-Release number of selected component (if applicable):
Ovirt version "4.2.3.5-1.el7.centos", Gluster version "glusterfs-3.13.2-2.el7" on a base OS of "CentOS Linux release 7.4.1708 (Core)", Kernel "3.10.0 - 693.21.1.el7.x86_64", VDSM version "vdsm-4.20.23-1.el7.centos"

How reproducible:
The entire Ovirt and Gluster environment have been rebuilt a total of 3 times and each time the fault exists.

Steps to Reproduce:
To configure the snapshot schedule I did the following;
1. Login to Ovirt WebUI
2. From left hand menu select ‘Storage’ and ‘Volumes'
3. I then selected the volume I wanted to snapshot by clicking on the link within the ‘Name’ column
4. From here I selected the ‘Snapshots’ tab
5. From the top menu options I selected the drop down ‘Snapshot’
6. From the drop down options I selected ‘New’
7. A new window appeared titled ‘Create/Schedule Snapshot’
8. I entered a snapshot prefix and description into the available fields and selected the ‘Schedule’ page
9. On the schedule page I selected ‘Minute’ from the ‘Recurrence’ drop down
10. Set ‘Interval’ to every ’30’ minutes
11. Changed timezone to ‘Europe/London=(GMT+00:00) London Standard Time’
12. Left value in ‘Start Schedule by’ at default value
13. Set schedule to ‘No End Date’
14. Click 'OK'
15. Wait for defined period for snapshots to be created

Actual results:
No scheduled snapshots are taken

Expected results:
A snapshot of the gluster volume should be taken at the defined intervals as defined by the schedule.

Additional info:
I see the following messages in logs: 
2018-05-14 04:30:00,018Z ERROR [org.ovirt.engine.core.utils.timer.JobWrapper] (QuartzOvirtDBScheduler9) [d0c31a9] Failed to invoke scheduled method onTimer: null

I have attached the engine log file.

Comment 1 Sahina Bose 2018-05-15 10:13:24 UTC
Adding debug logs, I see below stacktrace

Caused by: java.lang.NullPointerException
        at org.ovirt.engine.core.bll.gluster.GlusterSnapshotScheduleJob.onTimer(GlusterSnapshotScheduleJob.java:56) [bll.jar:]
        ... 13 more

pointing to line 
 final GlusterVolumeEntity volume = glusterVolumeDao.getById(new Guid(volumeId));

It looks like the dependencies are not injected correctly here

Allon, anything missing in the patchset to inject dependencies?

Comment 2 Mark Betham 2018-05-15 11:39:59 UTC
Created attachment 1436773 [details]
Additional log info with debug flag as requested

Hi Sahina,
Please find attached the engine log with debug enabled as requested.  Apologies for the delay in getting this to you.
Thanks,
Mark Betham

Comment 3 bipin 2018-06-22 09:13:39 UTC
Verified the bug in the version ovirt-engine-4.2.4.4.

Steps:
Followed the Steps 1-15 as mentioned in the Description.
Attaching the  UI screenshot of scheduled snapshot.

Engine log:
2018-06-22 12:12:03,753+05 INFO  [org.ovirt.engine.core.bll.gluster.ScheduleGlusterVolumeSnapshotCommand] (default task-5) [77ea1b98-9f8e-4591-ab67-c95dac543872] Running command: ScheduleGlusterVolumeSnapshotCommand internal: false. Entities affected :  ID: 84b8adff-113d-45c2-972d-f325585ddd06 Type: GlusterVolumeAction group MANIPULATE_GLUSTER_VOLUME with role type ADMIN
2018-06-22 12:12:04,016+05 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-5) [77ea1b98-9f8e-4591-ab67-c95dac543872] EVENT_ID: GLUSTER_VOLUME_SNAPSHOT_SCHEDULED(4,134), Snapshots scheduled on volume vmstore of cluster Default.
<...>
2018-06-22 12:21:47,404+05 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVolumeSnapshotInfoVDSCommand] (DefaultQuartzScheduler8) [2a59ef8d] FINISH, GetGlusterVolumeSnapshotInfoVDSCommand, return: [], log id: 1a11f698
<...>
2018-06-22 12:21:47,632+05 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVolumeSnapshotConfigInfoVDSCommand] (DefaultQuartzScheduler8) [2a59ef8d] FINISH, GetGlusterVolumeSnapshotConfigInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.gluster.GlusterSnapshotConfigInfo@f2c8f7e4, log id: 50a7feb6
<...>
2018-06-22 12:30:00,111+05 INFO  [org.ovirt.engine.core.bll.gluster.CreateGlusterVolumeSnapshotCommand] (QuartzOvirtDBScheduler1) [1729c24] Running command: CreateGlusterVolumeSnapshotCommand internal: true. Entities affected :  ID: 84b8adff-113d-45c2-972d-f325585ddd06 Type: GlusterVolumeAction group MANIPULATE_GLUSTER_VOLUME with role type ADMIN
2018-06-22 12:30:04,692+05 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzOvirtDBScheduler1) [1729c24] EVENT_ID: GLUSTER_VOLUME_SNAPSHOT_CREATED(4,132), Snapshot Snap1_GMT-2018.06.22-07.00.00 created for volume vmstore of cluster Default.

Comment 5 bipin 2018-06-22 09:15:34 UTC
Created attachment 1453674 [details]
Screenshot2_Verified

Comment 6 Sandro Bonazzola 2018-06-26 08:46:19 UTC
This bugzilla is included in oVirt 4.2.4 release, published on June 26th 2018.

Since the problem described in this bug report should be
resolved in oVirt 4.2.4 release, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.