Bug 1226450 - Unable to import large sparse templates from GlusterFS
Summary: Unable to import large sparse templates from GlusterFS
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: oVirt
Classification: Retired
Component: ovirt-engine-core
Version: 3.6
Hardware: x86_64
OS: Linux
unspecified
unspecified
Target Milestone: m1
: 3.6.0
Assignee: Liron Aravot
QA Contact: Aharon Canan
URL:
Whiteboard: storage
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-05-29 19:46 UTC by Christopher Pereira
Modified: 2016-03-10 06:17 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-07-07 21:23:17 UTC
oVirt Team: Storage
Embargoed:


Attachments (Terms of Use)
Engine and VDSM Logs (1.25 MB, application/x-rar)
2015-05-29 19:46 UTC, Christopher Pereira
no flags Details

Description Christopher Pereira 2015-05-29 19:46:35 UTC
Created attachment 1032282 [details]
Engine and VDSM Logs

Template import task fails.

To reproduce, import a (big) sparse file using GlusterFS.
Since GlusterFS doesn't support sparse files efficiently [1], the import task will take days and be cleared by Engine.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1220173

== Engine Events ==
	
2015-May-24, 16:03
Starting to import Template CentOS7-2000GB to Data Center Default, Cluster Default
	
2015-May-26, 18:04
Failed to complete copy of Template CentOS7-2000GB to Domain <UNKNOWN>.

2015-May-26, 18:04
Clearing asynchronous task ImportVmTemplate that started at Sun May 24 16:03:48 CLT 2015

2015-May-26, 18:03
Stopping async task ImportVmTemplate that started at Sun May 24 16:03:48 CLT 2015

== Relevant Engine Logs ==

2015-05-26 18:03:51,489 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-39) [] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Stopping async task ImportVmTemplate that started at Sun May 24 16:03:48 CLT 2015
2015-05-26 18:03:51,489 INFO  [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-39) [] Cleaning zombie tasks: Stopping async task 'ImportVmTemplate' that started at 'Sun May 24 16:03:48 CLT 2015'
2015-05-26 18:03:51,490 INFO  [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-39) [] SPMAsyncTask::StopTask: Attempting to stop task 'f2114ddd-0362-4cb1-811f-80912dbe2779' (Parent Command 'ImportVmTemplate', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters').
2015-05-26 18:03:51,491 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.SPMStopTaskVDSCommand] (DefaultQuartzScheduler_Worker-39) [] START, SPMStopTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-00000000037d', ignoreFailoverLimit='false', taskId='f2114ddd-0362-4cb1-811f-80912dbe2779'}), log id: 733d4369
2015-05-26 18:03:51,493 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMStopTaskVDSCommand] (DefaultQuartzScheduler_Worker-39) [] START, HSMStopTaskVDSCommand(HostName = H1, HSMTaskGuidBaseVDSCommandParameters:{runAsync='true', hostId='d0ad0910-698d-429d-b6f6-842d794ae9e1', taskId='f2114ddd-0362-4cb1-811f-80912dbe2779'}), log id: 3b14dd9a
2015-05-26 18:03:51,542 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMStopTaskVDSCommand] (DefaultQuartzScheduler_Worker-39) [] FINISH, HSMStopTaskVDSCommand, log id: 3b14dd9a
2015-05-26 18:03:51,542 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.SPMStopTaskVDSCommand] (DefaultQuartzScheduler_Worker-39) [] FINISH, SPMStopTaskVDSCommand, log id: 733d4369

2015-05-26 18:04:01,979 ERROR [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-50) [] BaseAsyncTask::logEndTaskFailure: Task 'f2114ddd-0362-4cb1-811f-80912dbe2779' (Parent Command 'ImportVmTemplate', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended with failure:
-- Result: 'success'
-- Message: '',
-- Exception: '[null]'

2015-05-26 18:04:01,982 INFO  [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-50) [] CommandAsyncTask::endActionIfNecessary: All tasks of command '0740ee3e-718c-4299-8cdc-adfb55881406' has ended -> executing 'endAction'
2015-05-26 18:04:01,983 INFO  [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-50) [] CommandAsyncTask::endAction: Ending action for '1' tasks (command ID: '0740ee3e-718c-4299-8cdc-adfb55881406'): calling endAction '.
2015-05-26 18:04:01,983 INFO  [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-12-thread-12) [] CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction 'ImportVmTemplate', executionIndex: '0'
2015-05-26 18:04:02,014 INFO  [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-50) [] BaseAsyncTask::onTaskEndSuccess: Task 'f2114ddd-0362-4cb1-811f-80912dbe2779' (Parent Command 'ImportVmTemplate', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully.
2015-05-26 18:04:02,015 INFO  [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-50) [] Task with DB Task ID 'bd162de9-ed30-4772-80d8-5eaa60c73119' and VDSM Task ID 'f2114ddd-0362-4cb1-811f-80912dbe2779' is in state AttemptingEndAction. End action for command 0740ee3e-718c-4299-8cdc-adfb55881406 will proceed when all the entity's tasks are completed.

== Unvalidated hint ==

I'm not sure, but it seems the qemu-img process continued running during a previous test (I missed to check this time).

Comment 1 Liron Aravot 2015-05-31 16:56:32 UTC
Hi Christopher,
The engine currently uses the 'AsyncTaskZombieTaskLifeInMinutes' config value to indicate the maximum duration of a task - it can be increased to prevent tasks from being stops during their execution.

Comment 2 Christopher Pereira 2015-07-07 21:23:17 UTC
Thanks Liron.
The main issue was slow sparse files transfer due to gluster not supporting SEEK_HOLE/SEEK_DATA.
This is being implemented now:
https://bugzilla.redhat.com/show_bug.cgi?id=1220173

Closing with "wont fix".


Note You need to log in before you can comment on or make changes to this bug.