Bug 1578257 - Unable to schedule a Snapshot of a Gluster volume
Summary: Unable to schedule a Snapshot of a Gluster volume
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: ovirt-engine
Classification: oVirt
Component: BLL.Gluster
Version: 4.2.3.5
Hardware: x86_64
OS: Linux
high
high
Target Milestone: ovirt-4.2.4
: 4.2.4.1
Assignee: Sahina Bose
QA Contact: bipin
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-05-15 06:42 UTC by Mark Betham
Modified: 2018-06-26 08:46 UTC (History)
5 users (show)

Fixed In Version: ovirt-engine-4.2.4.1
Doc Type: Bug Fix
Doc Text:
Cause: Quartz scheduled jobs failed to fire due to issue with dependency classes Consequence: Snapshot scheduling did not work Fix: Dependency classes are instantiated in the job class Result: Snapshot scheduling works
Clone Of:
Environment:
Last Closed: 2018-06-26 08:46:19 UTC
oVirt Team: Gluster
Embargoed:
rule-engine: ovirt-4.2?
bshetty: planning_ack?
rule-engine: devel_ack+
bshetty: testing_ack+


Attachments (Terms of Use)
Engine Log File (720.42 KB, application/zip)
2018-05-15 06:42 UTC, Mark Betham
no flags Details
Additional log info with debug flag as requested (169.67 KB, application/zip)
2018-05-15 11:39 UTC, Mark Betham
no flags Details
Screenshot1_Verified (70.69 KB, image/png)
2018-06-22 09:14 UTC, bipin
no flags Details
Screenshot2_Verified (63.06 KB, image/png)
2018-06-22 09:15 UTC, bipin
no flags Details


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 91304 0 master MERGED gluster: Fixing the snapshot schedule DI 2021-02-17 20:40:25 UTC
oVirt gerrit 91675 0 ovirt-engine-4.2 MERGED gluster: Fixing the snapshot schedule DI 2021-02-17 20:40:25 UTC

Description Mark Betham 2018-05-15 06:42:49 UTC
Created attachment 1436699 [details]
Engine Log File

Description of problem:
After creating a schedule for the snapshotting of a Gluster volume no snapshots are taken.

Version-Release number of selected component (if applicable):
Ovirt version "4.2.3.5-1.el7.centos", Gluster version "glusterfs-3.13.2-2.el7" on a base OS of "CentOS Linux release 7.4.1708 (Core)", Kernel "3.10.0 - 693.21.1.el7.x86_64", VDSM version "vdsm-4.20.23-1.el7.centos"

How reproducible:
The entire Ovirt and Gluster environment have been rebuilt a total of 3 times and each time the fault exists.

Steps to Reproduce:
To configure the snapshot schedule I did the following;
1. Login to Ovirt WebUI
2. From left hand menu select ‘Storage’ and ‘Volumes'
3. I then selected the volume I wanted to snapshot by clicking on the link within the ‘Name’ column
4. From here I selected the ‘Snapshots’ tab
5. From the top menu options I selected the drop down ‘Snapshot’
6. From the drop down options I selected ‘New’
7. A new window appeared titled ‘Create/Schedule Snapshot’
8. I entered a snapshot prefix and description into the available fields and selected the ‘Schedule’ page
9. On the schedule page I selected ‘Minute’ from the ‘Recurrence’ drop down
10. Set ‘Interval’ to every ’30’ minutes
11. Changed timezone to ‘Europe/London=(GMT+00:00) London Standard Time’
12. Left value in ‘Start Schedule by’ at default value
13. Set schedule to ‘No End Date’
14. Click 'OK'
15. Wait for defined period for snapshots to be created

Actual results:
No scheduled snapshots are taken

Expected results:
A snapshot of the gluster volume should be taken at the defined intervals as defined by the schedule.

Additional info:
I see the following messages in logs: 
2018-05-14 04:30:00,018Z ERROR [org.ovirt.engine.core.utils.timer.JobWrapper] (QuartzOvirtDBScheduler9) [d0c31a9] Failed to invoke scheduled method onTimer: null

I have attached the engine log file.

Comment 1 Sahina Bose 2018-05-15 10:13:24 UTC
Adding debug logs, I see below stacktrace

Caused by: java.lang.NullPointerException
        at org.ovirt.engine.core.bll.gluster.GlusterSnapshotScheduleJob.onTimer(GlusterSnapshotScheduleJob.java:56) [bll.jar:]
        ... 13 more

pointing to line 
 final GlusterVolumeEntity volume = glusterVolumeDao.getById(new Guid(volumeId));

It looks like the dependencies are not injected correctly here

Allon, anything missing in the patchset to inject dependencies?

Comment 2 Mark Betham 2018-05-15 11:39:59 UTC
Created attachment 1436773 [details]
Additional log info with debug flag as requested

Hi Sahina,
Please find attached the engine log with debug enabled as requested.  Apologies for the delay in getting this to you.
Thanks,
Mark Betham

Comment 3 bipin 2018-06-22 09:13:39 UTC
Verified the bug in the version ovirt-engine-4.2.4.4.

Steps:
Followed the Steps 1-15 as mentioned in the Description.
Attaching the  UI screenshot of scheduled snapshot.

Engine log:
2018-06-22 12:12:03,753+05 INFO  [org.ovirt.engine.core.bll.gluster.ScheduleGlusterVolumeSnapshotCommand] (default task-5) [77ea1b98-9f8e-4591-ab67-c95dac543872] Running command: ScheduleGlusterVolumeSnapshotCommand internal: false. Entities affected :  ID: 84b8adff-113d-45c2-972d-f325585ddd06 Type: GlusterVolumeAction group MANIPULATE_GLUSTER_VOLUME with role type ADMIN
2018-06-22 12:12:04,016+05 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-5) [77ea1b98-9f8e-4591-ab67-c95dac543872] EVENT_ID: GLUSTER_VOLUME_SNAPSHOT_SCHEDULED(4,134), Snapshots scheduled on volume vmstore of cluster Default.
<...>
2018-06-22 12:21:47,404+05 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVolumeSnapshotInfoVDSCommand] (DefaultQuartzScheduler8) [2a59ef8d] FINISH, GetGlusterVolumeSnapshotInfoVDSCommand, return: [], log id: 1a11f698
<...>
2018-06-22 12:21:47,632+05 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVolumeSnapshotConfigInfoVDSCommand] (DefaultQuartzScheduler8) [2a59ef8d] FINISH, GetGlusterVolumeSnapshotConfigInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.gluster.GlusterSnapshotConfigInfo@f2c8f7e4, log id: 50a7feb6
<...>
2018-06-22 12:30:00,111+05 INFO  [org.ovirt.engine.core.bll.gluster.CreateGlusterVolumeSnapshotCommand] (QuartzOvirtDBScheduler1) [1729c24] Running command: CreateGlusterVolumeSnapshotCommand internal: true. Entities affected :  ID: 84b8adff-113d-45c2-972d-f325585ddd06 Type: GlusterVolumeAction group MANIPULATE_GLUSTER_VOLUME with role type ADMIN
2018-06-22 12:30:04,692+05 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzOvirtDBScheduler1) [1729c24] EVENT_ID: GLUSTER_VOLUME_SNAPSHOT_CREATED(4,132), Snapshot Snap1_GMT-2018.06.22-07.00.00 created for volume vmstore of cluster Default.

Comment 5 bipin 2018-06-22 09:15:34 UTC
Created attachment 1453674 [details]
Screenshot2_Verified

Comment 6 Sandro Bonazzola 2018-06-26 08:46:19 UTC
This bugzilla is included in oVirt 4.2.4 release, published on June 26th 2018.

Since the problem described in this bug report should be
resolved in oVirt 4.2.4 release, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.