Bug 1952916 - [CBT] [VEEAM] Download VM backup disk fails if user deleted the snapshot that contains the backup on a RAW disk with enabled incremental backup
Summary: [CBT] [VEEAM] Download VM backup disk fails if user deleted the snapshot that...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: ovirt-engine
Classification: oVirt
Component: Backup-Restore.VMs
Version: 4.4.5.11
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ovirt-4.4.9
: ---
Assignee: Eyal Shenitzky
QA Contact: Amit Sharir
bugs@ovirt.org
URL:
Whiteboard:
: 1990854 (view as bug list)
Depends On:
Blocks: 1954427
TreeView+ depends on / blocked
 
Reported: 2021-04-23 14:43 UTC by Yury.Panchenko
Modified: 2021-12-23 16:28 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-10-21 07:27:14 UTC
oVirt Team: Storage
Embargoed:
pm-rhel: ovirt-4.4+
asharir: testing_plan_complete+


Attachments (Terms of Use)
engine logs (418.10 KB, application/zip)
2021-04-23 14:46 UTC, Yury.Panchenko
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 1915025 1 high CLOSED Unable to backup VM with raw disk after snapshot deletion 2021-11-04 19:28:55 UTC
oVirt gerrit 116154 0 master MERGED core: expose NBD server bitmap only for QCOW disks 2021-08-11 14:33:18 UTC

Internal Links: 1954447

Description Yury.Panchenko 2021-04-23 14:43:20 UTC
Description of problem:
image transfer fails if user deleted snapshot on a RAW disk where incremental backup was been enabled

Version-Release number of selected component (if applicable):
ovirt-4.4.5
vdsm-4.40.50.10-1.el8ev.x86_64
ovirt-imageio-daemon-2.1.1-1.el8ev.x86_64
libvirt-daemon-6.6.0-13.2.module+el8.3.1+10483+85317cf0.x86_64
qemu-kvm-5.1.0-21.module+el8.3.1+10464+8ad18d1a.x86_64

How reproducible:
Create vm with QCOW and RAW disks
take a snapshot
Enable incremental backup option on both disks
Make full backup
Make incremental backup
Delete the snapshot
start incremental backup

Actual results:
backup crushed

Expected results:
RAW disk sucessfully backuped in full mode

Additional info:
2021-04-23 15:07:09,806+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.StartNbdServerVDSCommand] (default task-276) [e1d58a42-e58b-4e65-8e56-4e385b45d404] Failed in 'StartNbdServerVDS' method
2021-04-23 15:07:09,806+02 WARN  [org.ovirt.engine.core.vdsbroker.vdsbroker.StartNbdServerVDSCommand] (default task-276) [e1d58a42-e58b-4e65-8e56-4e385b45d404] Unexpected return value: Status [code=2009, message=Unsupported operation: 'reason=Cannot export bitmap from RAW volume, context={}']
2021-04-23 15:07:09,815+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-276) [e1d58a42-e58b-4e65-8e56-4e385b45d404] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM pan-rhv44rel1 command StartNbdServerVDS failed: Unsupported operation: 'reason=Cannot export bitmap from RAW volume, context={}'

after unsucessfull backup engine trys to close the ticket, but can't do that

Failed to stop image transfer session. Ticket does not exist for image '4c23c18c-7804-4850-a5d1-41391fdc4482'

Comment 1 Yury.Panchenko 2021-04-23 14:46:09 UTC
Created attachment 1774850 [details]
engine logs

2021-04-23 15:07:09,817+02 ERROR [org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand] (default task-276) [e1d58a42-e58b-4e65-8e56-4e385b45d404] Failed to prepare image for transfer session: EngineException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed to StartNbdServerVDS, error = Unsupported operation: 'reason=Cannot export bitmap from RAW volume, context={}', code = 2009 (Failed with error unexpected and code 16): org.ovirt.engine.core.common.errors.EngineException: EngineException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed to StartNbdServerVDS, error = Unsupported operation: 'reason=Cannot export bitmap from RAW volume, context={}', code = 2009 (Failed with error unexpected and code 16)
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.VdsHandler.handleVdsResult(VdsHandler.java:114)
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.VDSBrokerFrontendImpl.runVdsCommand(VDSBrokerFrontendImpl.java:33)
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.runVdsCommand(CommandBase.java:2119)
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand.prepareImage(TransferDiskImageCommand.java:155)
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand.startImageTransferSession(TransferDiskImageCommand.java:985)
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand.handleImageIsReadyForTransfer(TransferDiskImageCommand.java:602)
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand.executeCommand(TransferDiskImageCommand.java:432)
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1174)
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1332)
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:2008)
	at org.ovirt.engine.core.utils//org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:140)
	at org.ovirt.engine.core.utils//org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:79)
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1392)
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:424)
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.executor.DefaultBackendActionExecutor.execute(DefaultBackendActionExecutor.java:13)
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.Backend.runAction(Backend.java:442)
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:424)
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.Backend.runAction(Backend.java:379)
	at jdk.internal.reflect.GeneratedMethodAccessor550.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.jboss.as.ee.1.GA-redhat-00003//org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52)
	at org.jboss.invocation.2.Final-redhat-00001//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
	at org.jboss.invocation.2.Final-redhat-00001//org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509)
	at org.jboss.as.weld.ejb.1.GA-redhat-00003//org.jboss.as.weld.ejb.DelegatingInterceptorInvocationContext.proceed(DelegatingInterceptorInvocationContext.java:92)
	at org.jboss.weld.core.4.Final-redhat-00001//org.jboss.weld.interceptor.proxy.WeldInvocationContextImpl.interceptorChainCompleted(WeldInvocationContextImpl.java:107)
	at org.jboss.weld.core.4.Final-redhat-00001//org.jboss.weld.interceptor.proxy.WeldInvocationContextImpl.proceed(WeldInvocationContextImpl.java:126)
	at org.ovirt.engine.core.common//org.ovirt.engine.core.common.di.interceptor.LoggingInterceptor.apply(LoggingInterceptor.java:12)
	at jdk.internal.reflect.GeneratedMethodAccessor57.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.jboss.weld.core.4.Final-redhat-00001//org.jboss.weld.interceptor.reader.SimpleInterceptorInvocation$SimpleMethodInvocation.invoke(SimpleInterceptorInvocation.java:73)
	at org.jboss.weld.core.4.Final-redhat-00001//org.jboss.weld.interceptor.proxy.WeldInvocationContextImpl.invokeNext(WeldInvocationContextImpl.java:92)
	at org.jboss.weld.core.4.Final-redhat-00001//org.jboss.weld.interceptor.proxy.WeldInvocationContextImpl.proceed(WeldInvocationContextImpl.java:124)
	at org.jboss.weld.core.4.Final-redhat-00001//org.jboss.weld.bean.InterceptorImpl.intercept(InterceptorImpl.java:105)
	at org.jboss.as.weld.ejb.1.GA-redhat-00003//org.jboss.as.weld.ejb.DelegatingInterceptorInvocationContext.proceed(DelegatingInterceptorInvocationContext.java:82)
	at org.jboss.as.weld.ejb.1.GA-redhat-00003//org.jboss.as.weld.interceptors.EjbComponentInterceptorSupport.delegateInterception(EjbComponentInterceptorSupport.java:60)
	at org.jboss.as.weld.common.1.GA-redhat-00003//org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:77)
	at org.jboss.as.weld.common.1.GA-redhat-00003//org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:89)
	at org.jboss.as.weld.common.1.GA-redhat-00003//org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:102)
	at org.jboss.as.ee.1.GA-redhat-00003//org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63)
	at org.jboss.invocation.2.Final-redhat-00001//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
	at org.jboss.invocation.2.Final-redhat-00001//org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509)
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.interceptors.CorrelationIdTrackerInterceptor.aroundInvoke(CorrelationIdTrackerInterceptor.java:13)
	at jdk.internal.reflect.GeneratedMethodAccessor79.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.jboss.as.ee.1.GA-redhat-00003//org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptor.java:89)
	at org.jboss.invocation.2.Final-redhat-00001//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
	at org.jboss.as.ejb3.1.GA-redhat-00003//org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43)
	at org.jboss.invocation.2.Final-redhat-00001//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
	at org.jboss.as.ee.1.GA-redhat-00003//org.jboss.as.ee.concurrent.ConcurrentContextInterceptor.processInvocation(ConcurrentContextInterceptor.java:45)
	at org.jboss.invocation.2.Final-redhat-00001//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
	at org.jboss.invocation.2.Final-redhat-00001//org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:40)
	at org.jboss.invocation.2.Final-redhat-00001//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
	at org.jboss.invocation.2.Final-redhat-00001//org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:53)
	at org.jboss.as.ee.1.GA-redhat-00003//org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:52)
	at org.jboss.invocation.2.Final-redhat-00001//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
	at org.jboss.as.ejb3.1.GA-redhat-00003//org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53)
	at org.jboss.invocation.2.Final-redhat-00001//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
	at org.jboss.as.ejb3.1.GA-redhat-00003//org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:228)
	at org.jboss.as.ejb3.1.GA-redhat-00003//org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:442)
	at org.jboss.as.ejb3.1.GA-redhat-00003//org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:160)
	at org.jboss.invocation.2.Final-redhat-00001//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
	at org.jboss.invocation.2.Final-redhat-00001//org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509)
	at org.jboss.weld.core.4.Final-redhat-00001//org.jboss.weld.module.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:81)
	at org.jboss.as.weld.common.1.GA-redhat-00003//org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:89)
	at org.jboss.invocation.2.Final-redhat-00001//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
	at org.jboss.as.ejb3.1.GA-redhat-00003//org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41)
	at org.jboss.invocation.2.Final-redhat-00001//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
	at org.jboss.as.ejb3.1.GA-redhat-00003//org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47)
	at org.jboss.invocation.2.Final-redhat-00001//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
	at org.jboss.as.ejb3.1.GA-redhat-00003//org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100)
	at org.jboss.invocation.2.Final-redhat-00001//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
	at org.jboss.as.ejb3.1.GA-redhat-00003//org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22)
	at org.jboss.invocation.2.Final-redhat-00001//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
	at org.jboss.as.ejb3.1.GA-redhat-00003//org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64)
	at org.jboss.invocation.2.Final-redhat-00001//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
	at org.jboss.as.ejb3.1.GA-redhat-00003//org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67)
	at org.jboss.invocation.2.Final-redhat-00001//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
	at org.jboss.as.ee.1.GA-redhat-00003//org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50)
	at org.jboss.invocation.2.Final-redhat-00001//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
	at org.jboss.invocation.2.Final-redhat-00001//org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:60)
	at org.jboss.invocation.2.Final-redhat-00001//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
	at org.jboss.invocation.2.Final-redhat-00001//org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:438)
	at org.wildfly.security.elytron-private.6.Final-redhat-00001//org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:627)
	at org.jboss.invocation.2.Final-redhat-00001//org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:57)
	at org.jboss.invocation.2.Final-redhat-00001//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
	at org.jboss.invocation.2.Final-redhat-00001//org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:53)
	at org.jboss.as.ee.1.GA-redhat-00003//org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198)
	at org.jboss.as.ee.1.GA-redhat-00003//org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185)
	at org.jboss.as.ee.1.GA-redhat-00003//org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:81)
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.common.interfaces.BackendLocal$$$view3.runAction(Unknown Source)
	at org.ovirt.engine.api.restapi-jaxrs//org.ovirt.engine.api.restapi.resource.BackendResource.doAction(BackendResource.java:251)
	at org.ovirt.engine.api.restapi-jaxrs//org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.doCreateEntity(AbstractBackendCollectionResource.java:244)
	at org.ovirt.engine.api.restapi-jaxrs//org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.performCreate(AbstractBackendCollectionResource.java:144)
	at org.ovirt.engine.api.restapi-jaxrs//org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.performCreate(AbstractBackendCollectionResource.java:134)
	at org.ovirt.engine.api.restapi-jaxrs//org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.performCreate(AbstractBackendCollectionResource.java:153)
	at org.ovirt.engine.api.restapi-jaxrs//org.ovirt.engine.api.restapi.resource.BackendImageTransfersResource.performCreate(BackendImageTransfersResource.java:79)
	at org.ovirt.engine.api.restapi-jaxrs//org.ovirt.engine.api.restapi.resource.BackendImageTransfersResource.addForImage(BackendImageTransfersResource.java:41)
	at jdk.internal.reflect.GeneratedMethodAccessor1993.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.ovirt.engine.api.restapi-definition//org.ovirt.engine.api.resource.ImageTransfersResource.add(ImageTransfersResource.java:101)
	at jdk.internal.reflect.GeneratedMethodAccessor1657.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.jboss.resteasy.resteasy-jaxrs.2.Final-redhat-00002//org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:138)
	at org.jboss.resteasy.resteasy-jaxrs.2.Final-redhat-00002//org.jboss.resteasy.core.ResourceMethodInvoker.internalInvokeOnTarget(ResourceMethodInvoker.java:535)
	at org.jboss.resteasy.resteasy-jaxrs.2.Final-redhat-00002//org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTargetAfterFilter(ResourceMethodInvoker.java:424)
	at org.jboss.resteasy.resteasy-jaxrs.2.Final-redhat-00002//org.jboss.resteasy.core.ResourceMethodInvoker.lambda$invokeOnTarget$0(ResourceMethodInvoker.java:385)
	at org.jboss.resteasy.resteasy-jaxrs.2.Final-redhat-00002//org.jboss.resteasy.core.interception.PreMatchContainerRequestContext.filter(PreMatchContainerRequestContext.java:356)
	at org.jboss.resteasy.resteasy-jaxrs.2.Final-redhat-00002//org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:387)
	at org.jboss.resteasy.resteasy-jaxrs.2.Final-redhat-00002//org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:356)
	at org.jboss.resteasy.resteasy-jaxrs.2.Final-redhat-00002//org.jboss.resteasy.core.ResourceLocatorInvoker.invokeOnTargetObject(ResourceLocatorInvoker.java:150)
	at org.jboss.resteasy.resteasy-jaxrs.2.Final-redhat-00002//org.jboss.resteasy.core.ResourceLocatorInvoker.invoke(ResourceLocatorInvoker.java:104)
	at org.jboss.resteasy.resteasy-jaxrs.2.Final-redhat-00002//org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:440)
	at org.jboss.resteasy.resteasy-jaxrs.2.Final-redhat-00002//org.jboss.resteasy.core.SynchronousDispatcher.lambda$invoke$4(SynchronousDispatcher.java:229)
	at org.jboss.resteasy.resteasy-jaxrs.2.Final-redhat-00002//org.jboss.resteasy.core.SynchronousDispatcher.lambda$preprocess$0(SynchronousDispatcher.java:135)
	at org.jboss.resteasy.resteasy-jaxrs.2.Final-redhat-00002//org.jboss.resteasy.core.interception.PreMatchContainerRequestContext.filter(PreMatchContainerRequestContext.java:356)
	at org.jboss.resteasy.resteasy-jaxrs.2.Final-redhat-00002//org.jboss.resteasy.core.SynchronousDispatcher.preprocess(SynchronousDispatcher.java:138)
	at org.jboss.resteasy.resteasy-jaxrs.2.Final-redhat-00002//org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:215)
	at org.jboss.resteasy.resteasy-jaxrs.2.Final-redhat-00002//org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:227)
	at org.jboss.resteasy.resteasy-jaxrs.2.Final-redhat-00002//org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56)
	at org.jboss.resteasy.resteasy-jaxrs.2.Final-redhat-00002//org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51)
	at javax.servlet.api.0.Final-redhat-00001//javax.servlet.http.HttpServlet.service(HttpServlet.java:590)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:74)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:81)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.handlers.ServletChain$1.handleRequest(ServletChain.java:68)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
	at io.undertow.core.30.SP3-redhat-00001//io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.handlers.RedirectDirHandler.handleRequest(RedirectDirHandler.java:68)
	at io.undertow.core.30.SP3-redhat-00001//io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
	at io.undertow.core.30.SP3-redhat-00001//io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:251)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.handlers.ServletInitialHandler.dispatchToPath(ServletInitialHandler.java:186)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.spec.RequestDispatcherImpl.forwardImpl(RequestDispatcherImpl.java:227)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.spec.RequestDispatcherImpl.forwardImplSetup(RequestDispatcherImpl.java:149)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.spec.RequestDispatcherImpl.forward(RequestDispatcherImpl.java:111)
	at org.ovirt.engine.api.restapi-jaxrs//org.ovirt.engine.api.restapi.invocation.VersionFilter.doFilter(VersionFilter.java:167)
	at org.ovirt.engine.api.restapi-jaxrs//org.ovirt.engine.api.restapi.invocation.VersionFilter.doFilter(VersionFilter.java:87)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
	at org.ovirt.engine.api.restapi-jaxrs//org.ovirt.engine.api.restapi.invocation.CurrentFilter.doFilter(CurrentFilter.java:106)
	at org.ovirt.engine.api.restapi-jaxrs//org.ovirt.engine.api.restapi.invocation.CurrentFilter.doFilter(CurrentFilter.java:61)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
	at org.ovirt.engine.core.aaa//org.ovirt.engine.core.aaa.filters.RestApiSessionMgmtFilter.doFilter(RestApiSessionMgmtFilter.java:78)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
	at org.ovirt.engine.core.aaa//org.ovirt.engine.core.aaa.filters.EnforceAuthFilter.doFilter(EnforceAuthFilter.java:42)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
	at org.ovirt.engine.core.aaa//org.ovirt.engine.core.aaa.filters.SsoRestApiNegotiationFilter.doFilter(SsoRestApiNegotiationFilter.java:84)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
	at org.ovirt.engine.core.aaa//org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter.doFilter(SsoRestApiAuthFilter.java:46)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
	at org.ovirt.engine.core.aaa//org.ovirt.engine.core.aaa.filters.SessionValidationFilter.doFilter(SessionValidationFilter.java:59)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
	at org.ovirt.engine.core.aaa//org.ovirt.engine.core.aaa.filters.RestApiSessionValidationFilter.doFilter(RestApiSessionValidationFilter.java:35)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
	at org.ovirt.engine.api.restapi-jaxrs//org.ovirt.engine.api.restapi.security.CSRFProtectionFilter.doFilter(CSRFProtectionFilter.java:100)
	at org.ovirt.engine.api.restapi-jaxrs//org.ovirt.engine.api.restapi.security.CSRFProtectionFilter.doFilter(CSRFProtectionFilter.java:91)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
	at org.ovirt.engine.core.utils//org.ovirt.engine.core.utils.servlet.CORSSupportFilter.doFilter(CORSSupportFilter.java:272)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.handlers.ServletChain$1.handleRequest(ServletChain.java:68)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
	at org.wildfly.extension.undertow.1.GA-redhat-00003//org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
	at io.undertow.core.30.SP3-redhat-00001//io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.handlers.RedirectDirHandler.handleRequest(RedirectDirHandler.java:68)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:132)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
	at io.undertow.core.30.SP3-redhat-00001//io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
	at io.undertow.core.30.SP3-redhat-00001//io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53)
	at io.undertow.core.30.SP3-redhat-00001//io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59)
	at io.undertow.core.30.SP3-redhat-00001//io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77)
	at io.undertow.core.30.SP3-redhat-00001//io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
	at io.undertow.core.30.SP3-redhat-00001//io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
	at io.undertow.core.30.SP3-redhat-00001//io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
	at org.wildfly.extension.undertow.1.GA-redhat-00003//org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
	at io.undertow.core.30.SP3-redhat-00001//io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
	at io.undertow.core.30.SP3-redhat-00001//io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:269)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:78)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:133)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:130)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)
	at org.wildfly.extension.undertow.1.GA-redhat-00003//org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105)
	at org.wildfly.extension.undertow.1.GA-redhat-00003//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1541)
	at org.wildfly.extension.undertow.1.GA-redhat-00003//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1541)
	at org.wildfly.extension.undertow.1.GA-redhat-00003//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1541)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:249)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:78)
	at io.undertow.servlet.30.SP3-redhat-00001//io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:99)
	at io.undertow.core.30.SP3-redhat-00001//io.undertow.server.Connectors.executeRootHandler(Connectors.java:376)
	at io.undertow.core.30.SP3-redhat-00001//io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:830)
	at org.jboss.threads.3.Final-redhat-00001//org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
	at org.jboss.threads.3.Final-redhat-00001//org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982)
	at org.jboss.threads.3.Final-redhat-00001//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486)
	at org.jboss.threads.3.Final-redhat-00001//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377)
	at java.base/java.lang.Thread.run(Thread.java:834)

Comment 2 Eyal Shenitzky 2021-04-25 11:39:33 UTC
Yury, the error seems OK.

You can't download the incremental backup that was taken for the RAW disk while there was a snapshot for it and it was removed.
If the snapshot removed, the backup removed also since the disk format is back to RAW now.

So the error looks fine, is the bug refers to the problem with removing the ticket?

Comment 3 Yury.Panchenko 2021-04-27 12:08:59 UTC
Hello, Eyal.

> You can't download the incremental backup that was taken for the RAW disk while there was a snapshot for it and it was removed.
> If the snapshot removed, the backup removed also since the disk format is back to RAW now.
I expect more user friendly exception, like incremental backup doesn't supported, or start full backup in this case.

> So the error looks fine, is the bug refers to the problem with removing the ticket?
It's second problem.

Comment 4 Eyal Shenitzky 2021-04-28 07:31:06 UTC
This bug will refer to the first problem of failing to download the missing backup without proper error.
I've open bug 1954427 to track the missing ticket issue.

Comment 5 Eyal Shenitzky 2021-05-05 08:29:51 UTC
Yury, can you please clarify the steps to reproduce this bug?

First, is the backup done when the VM is running?
Second, when the snapshot removed? after the backup started but before starting to transfer the backup disk?

Please include the VDSM and the engine log that contains the snapshots and backup operations.

Comment 6 Yury.Panchenko 2021-05-05 12:37:42 UTC
Hello, Eyal.
> First, is the backup done when the VM is running?
Yes

> Second, when the snapshot removed? after the backup started but before starting to transfer the backup disk?
Between two incremental backups. I deleted the snapshot and only after that started new backup

> Please include the VDSM and the engine log that contains the snapshots and backup operations.
In attached logs those operations should be exist

Comment 7 Eyal Shenitzky 2021-05-05 13:03:26 UTC
(In reply to Yury.Panchenko from comment #6)
> Hello, Eyal.
> > First, is the backup done when the VM is running?
> Yes
> 
> > Second, when the snapshot removed? after the backup started but before starting to transfer the backup disk?
> Between two incremental backups. I deleted the snapshot and only after that
> started new backup
> 
> > Please include the VDSM and the engine log that contains the snapshots and backup operations.
> In attached logs those operations should be exist

This is strange because in this case, the incremental backup that was taken after the snapshot removal should fail.
The disk is now back in RAW format and doesn't contain the previous bitmaps and checkpoints.

And if the backup failed, then trying to download a non-existing backup should fail also.

So can you please elaborate, did the incremental backup succeeded after the snapshot removal in your case?

Comment 8 Yury.Panchenko 2021-05-07 15:14:45 UTC
Hello, Eyal.

> So can you please elaborate, did the incremental backup succeeded after the snapshot removal in your case?
We've done full backup and it was failed.

Comment 9 Eyal Shenitzky 2021-05-09 06:09:59 UTC
(In reply to Yury.Panchenko from comment #8)
> Hello, Eyal.
> 
> > So can you please elaborate, did the incremental backup succeeded after the snapshot removal in your case?
> We've done full backup and it was failed.

I'm sorry but I can't reproduce this bug with the given steps.
Please provide a clear way to reproduce it with the exact steps.

Comment 10 Yury.Panchenko 2021-05-11 13:22:10 UTC
Hello Eyal.
> I'm sorry but I can't reproduce this bug with the given steps.
> Please provide a clear way to reproduce it with the exact steps.
Full backup's created for that case.

Original problem required poweroff state of the vm.
Then after original steps, incremntal backup will created (and this is a bug)
but transfer's failed
2021-05-11 15:12:03,705+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.StartNbdServerVDSCommand] (default task-1463) [0cef2087-c946-431a-8fca-15df7a1a807b] Command 'StartNbdServerVDSCommand(HostName = pan-rhv44rel2, NbdServerVDSParameters:{hostId='9d30eb4c-c4d9-4eec-bc1f-d49cd225e200', serverId='8e6b8819-8554-4d9f-a0a8-87eab5e6cc0d', storageDomainId='1a0e9562-4acc-466c-8d2f-2fc17efb877e', imageId='8ebb90cb-c228-41df-8ba1-859dacebde2f', volumeId='52f18270-5070-4aa6-8305-8caac3248321', readonly='true', discard='true', backingChain='true', bitmap='4a9cf075-8a3a-4ed5-aceb-bb91b4601162'})' execution failed: VDSGenericException: VDSErrorException: Failed to StartNbdServerVDS, error = Unsupported operation: 'reason=Cannot export bitmap from RAW volume, context={}', code = 2009
2021-05-11 15:12:03,705+02 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.StartNbdServerVDSCommand] (default task-1463) [0cef2087-c946-431a-8fca-15df7a1a807b] FINISH, StartNbdServerVDSCommand, return: , log id: 52ca071
2021-05-11 15:12:03,707+02 ERROR [org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand] (default task-1463) [0cef2087-c946-431a-8fca-15df7a1a807b] Failed to prepare image for transfer session: EngineException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed to StartNbdServerVDS, error = Unsupported operation: 'reason=Cannot export bitmap from RAW volume, context={}', code = 2009 (Failed with error unexpected and code 16): org.ovirt.engine.core.common.errors.EngineException: EngineException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed to StartNbdServerVDS, error = Unsupported operation: 'reason=Cannot export bitmap from RAW volume, context={}', code = 2009 (Failed with error unexpected and code 16)
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.VdsHandler.handleVdsResult(VdsHandler.java:114)
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.VDSBrokerFrontendImpl.runVdsCommand(VDSBrokerFrontendImpl.java:33)

Comment 11 Eyal Shenitzky 2021-06-01 10:40:03 UTC
Yury, please provide full steps to reproduce this bug.

Comment 12 Avihai 2021-06-09 06:34:08 UTC
As this is a VEEAM/customer bug raising priority to high to get QE attention as well.

Comment 13 Yury.Panchenko 2021-06-14 14:51:04 UTC
Hello Eyal.
>Yury, please provide full steps to reproduce this bug.
It doesn't confirm on 4.4.6

Comment 14 Eyal Shenitzky 2021-06-15 07:02:58 UTC
Thanks Yury, closing.

Comment 15 Yury.Panchenko 2021-06-25 10:57:19 UTC
Hello Eyal,
Suddenly, it confirmed on 4.4.6
I have the vm with raw and qcow.
And i couldn't start backup for the raw disk
it always failed
Caused by: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed to StartNbdServerVDS, error = Unsupported operation: 'reason=Cannot export bitmap from RAW volume, context={}', code = 2009

Comment 16 Eyal Shenitzky 2021-06-27 07:16:29 UTC
Yury, can you please provide the steps to reproduce and the relevant logs?

Comment 17 Nir Soffer 2021-07-19 12:28:32 UTC
(In reply to Yury.Panchenko from comment #0)

> How reproducible:
> Create vm with QCOW and RAW disks

Did you start this VM?

> take a snapshot

This can be done when the VM is running or not...

> Enable incremental backup option on both disks
> Make full backup
> Make incremental backup
> Delete the snapshot

Same, the VM may be running on not...

> start incremental backup
> 
> Actual results:
> backup crushed

"Crashed" is not a good term. Please be more specific.

    I send request ... and it failed with error ...

> Expected results:
> RAW disk sucessfully backuped in full mode

Indeed, regardless of VM status (up, down), the next backup should report
that the disk can have full backup and backup should be successful.

> Additional info:
> 2021-04-23 15:07:09,806+02 ERROR
> [org.ovirt.engine.core.vdsbroker.vdsbroker.StartNbdServerVDSCommand]

Starting NBD server means that the VM was not running. Maybe it was running
when you started the backup and it was terminated for some reason, or this 
entire flow was done when the VM was down).

> (default task-276) [e1d58a42-e58b-4e65-8e56-4e385b45d404] Failed in
> 'StartNbdServerVDS' method
> 2021-04-23 15:07:09,806+02 WARN 
> [org.ovirt.engine.core.vdsbroker.vdsbroker.StartNbdServerVDSCommand]
> (default task-276) [e1d58a42-e58b-4e65-8e56-4e385b45d404] Unexpected return
> value: Status [code=2009, message=Unsupported operation: 'reason=Cannot
> export bitmap from RAW volume, context={}']
> 2021-04-23 15:07:09,815+02 ERROR
> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
> (default task-276) [e1d58a42-e58b-4e65-8e56-4e385b45d404] EVENT_ID:
> VDS_BROKER_COMMAND_FAILURE(10,802), VDSM pan-rhv44rel1 command
> StartNbdServerVDS failed: Unsupported operation: 'reason=Cannot export
> bitmap from RAW volume, context={}'

The issue seems to be that engine tries to export a bitmap after the snapshot was 
deleted, which convert the disk to RAW format, which does not support bitmaps.

> after unsucessfull backup engine trys to close the ticket, but can't do that
> 
> Failed to stop image transfer session. Ticket does not exist for image
> '4c23c18c-7804-4850-a5d1-41391fdc4482'

So the image transfer remain open or this is just a warning we get on the way
to cleaning up the image transfer? If the transfer does not finish, this is
another issue that should be handled in another bug.

Comment 18 Yury.Panchenko 2021-07-21 17:31:06 UTC
Hello, Nir and Eyal.
I couldn't confirm this bug again. It's some floating defect and i can't find proper way to confirm.
But after several manipulation in that vm with snapshots i got the new case.
When i try to privew the snapshot. It failed with the same error.
2021-07-21 17:30:32,652+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-76) [] Failed in 'HSMGetAllTasksStatusesVDS' method
2021-07-21 17:30:32,652+02 WARN  [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-76) [] Unexpected return value: TaskStatus [code=2009, message=value=Unsupported operation: "reason=Cannot add bitmaps from parent volume with raw format, context={'srcVolUUID': 'fcd40610-ed08-47ad-b542-a7df1a9174dd'}" abortedcode=2009]
-- Result: 'cleanSuccess'
-- Message: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = value=Unsupported operation: "reason=Cannot add bitmaps from parent volume with raw format, context={'srcVolUUID': 'fcd40610-ed08-47ad-b542-a7df1a9174dd'}" abortedcode=2009, code = 2009',
-- Exception: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = value=Unsupported operation: "reason=Cannot add bitmaps from parent volume with raw format, context={'srcVolUUID': 'fcd40610-ed08-47ad-b542-a7df1a9174dd'}" abortedcode=2009, code = 2009'

I'm trying to simplify the case for the new bug.

Comment 20 Eyal Shenitzky 2021-08-09 11:38:00 UTC
*** Bug 1990854 has been marked as a duplicate of this bug. ***

Comment 21 Eyal Shenitzky 2021-08-09 11:38:48 UTC
Steps to Reproduce from duplicate bug 1990854:
1. Create vm with qcow and raw disk and powered on it
2. Do full backup
3. Do incremental backup
4. Powered off VM
5. Do incremental backup

Comment 22 Eyal Shenitzky 2021-08-11 07:52:13 UTC
(In reply to Eyal Shenitzky from comment #21)
> Steps to Reproduce from duplicate bug 1990854:
> 1. Create vm with qcow and raw disk and powered on it
> 2. Do full backup
> 3. Do incremental backup
> 4. Powered off VM
> 5. Do incremental backup

easier steps to reproduce:

1. Create a VM with RAW and QCOW disk (can be done by creating the disks via the UI on NFS domain and setting 'Incremental backup enabled' for one disk and not for the other).
2. Create a full backup for the VM that includes all the VM's disks
3. Finalize the VM backup
4. Start an incremental backup that includes all the VM's disks and use the created checkpoint from step 2.
5. Download the RAW disk.

Downloading the RAW disk will fail with the mentioned error.

Expected results - 

Downloading the RAW disk in mixed backup should work without any issues.

Comment 23 Amit Sharir 2021-09-30 08:36:21 UTC
Version: 
vdsm-4.40.90.1-1.el8ev.x86_64
ovirt-engine-4.4.9-0.6.el8ev.noarch


Verification conclusions:
The expected output matched the actual output.
The total flow mentioned was done with no errors, I was able to download the RAW disk in mixed backup at the end of the flow without any issues.


Bug verified.

Comment 26 Sandro Bonazzola 2021-10-21 07:27:14 UTC
This bugzilla is included in oVirt 4.4.9 release, published on October 20th 2021.

Since the problem described in this bug report should be resolved in oVirt 4.4.9 release, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.