Bug 1905113 - OpenStack Block Storage Provider (Cinder) regression: oVirt 4.4 Disk resize broken
Summary: OpenStack Block Storage Provider (Cinder) regression: oVirt 4.4 Disk resize b...
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: ovirt-engine
Classification: oVirt
Component: BLL.Storage
Version: 4.4.3.12
Hardware: x86_64
OS: Linux
unspecified
medium
Target Milestone: ---
: ---
Assignee: Tal Nisan
QA Contact: Avihai
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-12-07 14:53 UTC by Konstantin Shalygin
Modified: 2021-02-12 13:45 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-12-21 15:28:21 UTC
oVirt Team: Storage
Embargoed:


Attachments (Terms of Use)
Proposal patch for this issue (origianl author is Ilya Krapchatov <i.krapchatov@e2e4online.ru>) (1.64 KB, patch)
2020-12-10 15:28 UTC, Konstantin Shalygin
no flags Details | Diff

Description Konstantin Shalygin 2020-12-07 14:53:33 UTC
Description of problem: After upgrade oVirt from 4.3 to 4.4 is impossible to resize Cinder Disk.


Version-Release number of selected component (if applicable): 4.4.3.12


How reproducible:


Steps to Reproduce:
1. Go to Storage -> Disks.
2. Select 'Disk Type': 'Cinder'.
3. Edit disk.
4. 'Extend size by (GiB)' to any int value.
5. Press 'OK' button.

Actual results:

Disk failed to resize.

Expected results:

Disk resized to new size.

Additional info:

Engine logs:

2020-12-07 15:30:54,970+07 INFO  [org.ovirt.engine.core.bll.storage.disk.UpdateDiskCommand] (default task-54) [914102cb-85f0-4a9e-a99e-626ffd1338e8] Lock Acquired to object 'EngineLock:{exclusiveLocks='[c3c1f95f-3139-4da2-8629-f566b5405cde=DISK]', sharedLocks='[cc0c3ca1-4801-448e-93cb-26d7075c2c58=VM]'}'          
2020-12-07 15:30:55,087+07 INFO  [org.ovirt.engine.core.bll.storage.disk.UpdateDiskCommand] (default task-54) [914102cb-85f0-4a9e-a99e-626ffd1338e8] Running command: UpdateDiskCommand internal: false. Entities affected :  ID: c3c1f95f-3139-4da2-8629-f566b5405cde Type: DiskAction group EDIT_DISK_PROPERTIES with role type USER
2020-12-07 15:30:55,104+07 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-54) [914102cb-85f0-4a9e-a99e-626ffd1338e8] EVENT_ID: USER_EXTENDED_DISK_SIZE(377), Extending disk 'ovirt_engine_4.4_Disk2' to <UNKNOWN> GB was initiated by k0ste@FreeIPA-authz.                     
2020-12-07 15:30:55,504+07 INFO  [org.ovirt.engine.core.bll.storage.disk.UpdateDiskCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-66) [914102cb-85f0-4a9e-a99e-626ffd1338e8] Starting to extend disk id 'c3c1f95f-3139-4da2-8629-f566b5405cde')                                             
2020-12-07 15:30:55,504+07 ERROR [org.ovirt.engine.core.bll.CommandsFactory] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-66) [914102cb-85f0-4a9e-a99e-626ffd1338e8] Can't find constructor for type org.ovirt.engine.core.bll.storage.disk.cinder.ExtendCinderDiskCommand with parameter types: [class org.ovirt.engine.core.common.action.VmDiskOperationParameterBase, class org.ovirt.engine.core.bll.context.CommandContext]
2020-12-07 15:30:55,504+07 ERROR [org.ovirt.engine.core.bll.CommandsFactory] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-66) [914102cb-85f0-4a9e-a99e-626ffd1338e8] An exception has occurred while trying to create a command object for command 'ExtendCinderDisk' with parameters 'VmDiskOperationParameterBase:{commandId='null', user='k0ste', commandType='Unknown', vmId='00000000-0000-0000-0000-000000000000'}': could not find matching constructor for Command class class org.ovirt.engine.core.bll.storage.disk.cinder.ExtendCinderDiskCommand                                                                   
2020-12-07 15:30:55,504+07 ERROR [org.ovirt.engine.core.bll.CommandsFactory] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-66) [914102cb-85f0-4a9e-a99e-626ffd1338e8] Exception: java.lang.RuntimeException: could not find matching constructor for Command class class org.ovirt.engine.core.bll.storage.disk.cinder.ExtendCinderDiskCommand
        at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandsFactory.findCommandConstructor(CommandsFactory.java:226)
        at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandsFactory.lambda$getCommandConstructor$0(CommandsFactory.java:201)
        at java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1705)
        at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandsFactory.getCommandConstructor(CommandsFactory.java:200)
        at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandsFactory.createCommand(CommandsFactory.java:92)
        at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.tasks.CommandCoordinatorImpl.executeAsyncCommand(CommandCoordinatorImpl.java:103)
        at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.tasks.CommandCoordinatorUtil.executeAsyncCommand(CommandCoordinatorUtil.java:400)
        at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.storage.disk.UpdateDiskCommand.extendCinderDiskSize(UpdateDiskCommand.java:698)
        at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.storage.disk.UpdateDiskCommand.executeDiskExtend(UpdateDiskCommand.java:684)
        at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.storage.disk.UpdateDiskCommand.performNextOperation(UpdateDiskCommand.java:1122)
        at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback.childCommandsExecutionEnded(SerialChildCommandsExecutionCallback.java:32)
        at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.ChildCommandsCallbackBase.doPolling(ChildCommandsCallbackBase.java:80)
        at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethodsImpl(CommandCallbacksPoller.java:175)
        at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethods(CommandCallbacksPoller.java:109)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
        at org.glassfish.javax.enterprise.concurrent//org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.access$201(ManagedScheduledThreadPoolExecutor.java:360)
        at org.glassfish.javax.enterprise.concurrent//org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.run(ManagedScheduledThreadPoolExecutor.java:511)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)
        at org.glassfish.javax.enterprise.concurrent//org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:227)

2020-12-07 15:30:55,504+07 ERROR [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-66) [914102cb-85f0-4a9e-a99e-626ffd1338e8] Command 'UpdateDisk' id: '34c7f960-8dff-49b8-8718-9f86910dea92' with children [] failed when attempting to perform the next operation, marking as 'ACTIVE'
2020-12-07 15:30:55,504+07 ERROR [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-66) [914102cb-85f0-4a9e-a99e-626ffd1338e8] null: java.lang.NullPointerException
        at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.tasks.CommandCoordinatorImpl.executeAsyncCommand(CommandCoordinatorImpl.java:104)
        at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.tasks.CommandCoordinatorUtil.executeAsyncCommand(CommandCoordinatorUtil.java:400)
        at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.storage.disk.UpdateDiskCommand.extendCinderDiskSize(UpdateDiskCommand.java:698)
        at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.storage.disk.UpdateDiskCommand.executeDiskExtend(UpdateDiskCommand.java:684)
        at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.storage.disk.UpdateDiskCommand.performNextOperation(UpdateDiskCommand.java:1122)
        at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback.childCommandsExecutionEnded(SerialChildCommandsExecutionCallback.java:32)
        at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.ChildCommandsCallbackBase.doPolling(ChildCommandsCallbackBase.java:80)
        at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethodsImpl(CommandCallbacksPoller.java:175)
        at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethods(CommandCallbacksPoller.java:109)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
        at org.glassfish.javax.enterprise.concurrent//org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.access$201(ManagedScheduledThreadPoolExecutor.java:360)
        at org.glassfish.javax.enterprise.concurrent//org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.run(ManagedScheduledThreadPoolExecutor.java:511)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)
        at org.glassfish.javax.enterprise.concurrent//org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:227)

2020-12-07 15:30:55,504+07 INFO  [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-66) [914102cb-85f0-4a9e-a99e-626ffd1338e8] Command 'UpdateDisk' id: '34c7f960-8dff-49b8-8718-9f86910dea92' child commands '[]' executions were completed, status 'FAILED'
2020-12-07 15:30:56,561+07 ERROR [org.ovirt.engine.core.bll.storage.disk.UpdateDiskCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-30) [914102cb-85f0-4a9e-a99e-626ffd1338e8] Ending command 'org.ovirt.engine.core.bll.storage.disk.UpdateDiskCommand' with failure.                        
2020-12-07 15:30:56,566+07 INFO  [org.ovirt.engine.core.bll.storage.disk.UpdateDiskCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-30) [914102cb-85f0-4a9e-a99e-626ffd1338e8] Lock freed to object 'EngineLock:{exclusiveLocks='[c3c1f95f-3139-4da2-8629-f566b5405cde=DISK]', sharedLocks='[cc0c3ca1-4801-448e-93cb-26d7075c2c58=VM]'}'
2020-12-07 15:30:56,575+07 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-30) [914102cb-85f0-4a9e-a99e-626ffd1338e8] EVENT_ID: USER_EXTENDED_DISK_SIZE(377), Extending disk 'ovirt_engine_4.4_Disk2' to <UNKNOWN> GB was
initiated by k0ste@FreeIPA-authz.

Comment 1 Konstantin Shalygin 2020-12-10 15:25:06 UTC
We think when [2] implemented, changes just missed for buildExtendCinderDiskParameters...
I attach patch for this, engine starts to see size:

```
EVENT_ID: USER_EXTEND_DISK_SIZE_FAILURE(370), Failed to extend size of the disk '222cinder' to 20 GB, User: k0ste@FreeIPA-authz.
```

But then fail due OpenStack issues, I think it's related to BZ1904669...

```

2020-12-10 21:20:19,755+07 INFO  [org.ovirt.engine.core.bll.storage.disk.UpdateDiskCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-6) [6c6ff7c0-a78b-4c0d-b0ea-abcd125ecee5] Starting to extend disk id '336069bf-5c00-4280-a715-b5dacde53f9d')
2020-12-10 21:20:19,763+07 INFO  [org.ovirt.engine.core.bll.storage.disk.cinder.ExtendCinderDiskCommand] (EE-ManagedExecutorService-commandCoordinator-Thread-10) [4b7c14e8] Lock Acquired to object 'EngineLock:{exclusiveLocks='[]', sharedLocks='[]'}'
2020-12-10 21:20:19,770+07 INFO  [org.ovirt.engine.core.bll.storage.disk.cinder.ExtendCinderDiskCommand] (EE-ManagedExecutorService-commandCoordinator-Thread-10) [4b7c14e8] Running command: ExtendCinderDiskCommand internal: true. Entities affected :  ID: 336069bf-5c00-4280-a715-b5dacde53f9d Type: DiskAction group EDIT_DISK_PROPERTIES with role type USER
2020-12-10 21:20:19,903+07 WARN  [org.ovirt.engine.core.bll.provider.network.openstack.CustomizedRESTEasyConnector] (EE-ManagedExecutorService-commandCoordinator-Thread-10) [4b7c14e8] Cannot register external providers trust store: java.io.IOException: Keystore was tampered with, or password was incorrect
2020-12-10 21:20:20,039+07 ERROR [org.ovirt.engine.core.bll.storage.disk.cinder.ExtendCinderDiskCommand] (EE-ManagedExecutorService-commandCoordinator-Thread-10) [4b7c14e8] Command 'org.ovirt.engine.core.bll.storage.disk.cinder.ExtendCinderDiskCommand' failed: Bad Request
2020-12-10 21:20:20,039+07 ERROR [org.ovirt.engine.core.bll.storage.disk.cinder.ExtendCinderDiskCommand] (EE-ManagedExecutorService-commandCoordinator-Thread-10) [4b7c14e8] Exception: com.woorea.openstack.base.client.OpenStackResponseException: Bad Request
	at com.woorea.openstack.sdk//com.woorea.openstack.connector.RESTEasyConnector.request(RESTEasyConnector.java:120)
	at com.woorea.openstack.sdk//com.woorea.openstack.base.client.OpenStackClient.request(OpenStackClient.java:51)
	at com.woorea.openstack.sdk//com.woorea.openstack.base.client.OpenStackClient.execute(OpenStackClient.java:66)
	at com.woorea.openstack.sdk//com.woorea.openstack.base.client.OpenStackRequest.execute(OpenStackRequest.java:102)
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.provider.storage.OpenStackVolumeProviderProxy.extendVolume(OpenStackVolumeProviderProxy.java:103)
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.storage.disk.cinder.CinderBroker.lambda$extendDisk$4(CinderBroker.java:149)
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.storage.disk.cinder.CinderBroker.execute(CinderBroker.java:53)
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.storage.disk.cinder.CinderBroker.extendDisk(CinderBroker.java:148)
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.storage.disk.cinder.ExtendCinderDiskCommand.executeCommand(ExtendCinderDiskCommand.java:39)
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1169)
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1327)
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:2003)
	at org.ovirt.engine.core.utils//org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInNewTransaction(TransactionSupport.java:181)
	at org.ovirt.engine.core.utils//org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInRequired(TransactionSupport.java:113)
	at org.ovirt.engine.core.utils//org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:81)
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1387)
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:419)
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.executor.DefaultBackendActionExecutor.execute(DefaultBackendActionExecutor.java:13)
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.Backend.runAction(Backend.java:442)
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.Backend.runAction(Backend.java:688)
	at jdk.internal.reflect.GeneratedMethodAccessor440.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.jboss.as.ee.0.Final//org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52)
	at org.jboss.invocation.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
	at org.jboss.invocation.2.Final//org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509)
	at org.jboss.as.weld.common.0.Final//org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:79)
	at org.jboss.as.weld.common.0.Final//org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:89)
	at org.jboss.as.weld.common.0.Final//org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:102)
	at org.jboss.as.ee.0.Final//org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63)
	at org.jboss.invocation.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
	at org.jboss.invocation.2.Final//org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509)
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.interceptors.CorrelationIdTrackerInterceptor.aroundInvoke(CorrelationIdTrackerInterceptor.java:13)
	at jdk.internal.reflect.GeneratedMethodAccessor107.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.jboss.as.ee.0.Final//org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptor.java:89)
	at org.jboss.invocation.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
	at org.jboss.as.ejb3.0.Final//org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43)
	at org.jboss.invocation.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
	at org.jboss.as.ee.0.Final//org.jboss.as.ee.concurrent.ConcurrentContextInterceptor.processInvocation(ConcurrentContextInterceptor.java:45)
	at org.jboss.invocation.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
	at org.jboss.invocation.2.Final//org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:40)
	at org.jboss.invocation.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
	at org.jboss.invocation.2.Final//org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:53)
	at org.jboss.as.ee.0.Final//org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:52)
	at org.jboss.invocation.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
	at org.jboss.as.ejb3.0.Final//org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53)
	at org.jboss.invocation.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
	at org.jboss.as.ejb3.0.Final//org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:228)
	at org.jboss.as.ejb3.0.Final//org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:430)
	at org.jboss.as.ejb3.0.Final//org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:160)
	at org.jboss.invocation.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
	at org.jboss.invocation.2.Final//org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509)
	at org.jboss.weld.core.3.Final//org.jboss.weld.module.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:81)
	at org.jboss.as.weld.common.0.Final//org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:89)
	at org.jboss.invocation.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
	at org.jboss.as.ejb3.0.Final//org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41)
	at org.jboss.invocation.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
	at org.jboss.as.ejb3.0.Final//org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47)
	at org.jboss.invocation.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
	at org.jboss.as.ejb3.0.Final//org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100)
	at org.jboss.invocation.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
	at org.jboss.as.ejb3.0.Final//org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22)
	at org.jboss.invocation.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
	at org.jboss.as.ejb3.0.Final//org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64)
	at org.jboss.invocation.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
	at org.jboss.as.ejb3.0.Final//org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67)
	at org.jboss.invocation.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
	at org.jboss.as.ee.0.Final//org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50)
	at org.jboss.invocation.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
	at org.jboss.invocation.2.Final//org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:60)
	at org.jboss.invocation.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
	at org.jboss.invocation.2.Final//org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:438)
	at org.wildfly.security.elytron-private.4.Final//org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:627)
	at org.jboss.invocation.2.Final//org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:57)
	at org.jboss.invocation.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
	at org.jboss.invocation.2.Final//org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:53)
	at org.jboss.as.ee.0.Final//org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198)
	at org.jboss.as.ee.0.Final//org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185)
	at org.jboss.as.ee.0.Final//org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:81)
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.interfaces.BackendCommandObjectsHandler$$$view3.runAction(Unknown Source)
	at jdk.internal.reflect.GeneratedMethodAccessor439.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.jboss.weld.core.3.Final//org.jboss.weld.util.reflection.Reflections.invokeAndUnwrap(Reflections.java:410)
	at org.jboss.weld.core.3.Final//org.jboss.weld.module.ejb.EnterpriseBeanProxyMethodHandler.invoke(EnterpriseBeanProxyMethodHandler.java:134)
	at org.jboss.weld.core.3.Final//org.jboss.weld.bean.proxy.EnterpriseTargetBeanInstance.invoke(EnterpriseTargetBeanInstance.java:56)
	at org.jboss.weld.core.3.Final//org.jboss.weld.module.ejb.InjectionPointPropagatingEnterpriseTargetBeanInstance.invoke(InjectionPointPropagatingEnterpriseTargetBeanInstance.java:68)
	at org.jboss.weld.core.3.Final//org.jboss.weld.bean.proxy.ProxyMethodHandler.invoke(ProxyMethodHandler.java:106)
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.BackendCommandObjectsHandler$BackendInternal$BackendLocal$2049259618$Proxy$_$$_Weld$EnterpriseProxy$.runAction(Unknown Source)
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.tasks.CommandExecutor.executeCommand(CommandExecutor.java:60)
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.tasks.CommandExecutor.lambda$executeAsyncCommand$0(CommandExecutor.java:49)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at org.glassfish.javax.enterprise.concurrent//org.glassfish.enterprise.concurrent.internal.ManagedFutureTask.run(ManagedFutureTask.java:117)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)
	at org.glassfish.javax.enterprise.concurrent//org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:227)

2020-12-10 21:20:20,044+07 ERROR [org.ovirt.engine.core.bll.storage.disk.cinder.ExtendCinderDiskCommand] (EE-ManagedExecutorService-commandCoordinator-Thread-10) [4b7c14e8] Transaction rolled-back for command 'org.ovirt.engine.core.bll.storage.disk.cinder.ExtendCinderDiskCommand'.
2020-12-10 21:20:20,046+07 INFO  [org.ovirt.engine.core.bll.storage.disk.cinder.ExtendCinderDiskCommand] (EE-ManagedExecutorService-commandCoordinator-Thread-10) [4b7c14e8] Lock freed to object 'EngineLock:{exclusiveLocks='[]', sharedLocks='[]'}'
2020-12-10 21:20:21,181+07 WARN  [org.ovirt.engine.core.bll.provider.network.openstack.CustomizedRESTEasyConnector] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-30) [4b7c14e8] Cannot register external providers trust store: java.io.IOException: Keystore was tampered with, or password was incorrect
2020-12-10 21:20:21,463+07 INFO  [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-30) [4b7c14e8] Exception in invoking callback of command ExtendCinderDisk (8e239438-5f61-46e5-8bea-2b12c8736591): MismatchedInputException: Root name 'volumes' does not match expected ('volume') for type [simple type, class com.woorea.openstack.cinder.model.Volume]
 at [Source: (org.jboss.resteasy.client.core.BaseClientResponse$InputStreamWrapper); line: 1, column: 2] (through reference chain: com.woorea.openstack.cinder.model.Volume["volumes"])
2020-12-10 21:20:21,463+07 ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-30) [4b7c14e8] Error invoking callback method 'onFailed' for 'EXECUTION_FAILED' command '8e239438-5f61-46e5-8bea-2b12c8736591'
2020-12-10 21:20:21,463+07 ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-30) [4b7c14e8] Exception: java.lang.RuntimeException: org.jboss.resteasy.spi.ReaderException: com.fasterxml.jackson.databind.exc.MismatchedInputException: Root name 'volumes' does not match expected ('volume') for type [simple type, class com.woorea.openstack.cinder.model.Volume]
 at [Source: (org.jboss.resteasy.client.core.BaseClientResponse$InputStreamWrapper); line: 1, column: 2] (through reference chain: com.woorea.openstack.cinder.model.Volume["volumes"])
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.storage.disk.cinder.CinderBroker.execute(CinderBroker.java:59)
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.storage.disk.cinder.CinderBroker.getDiskStatus(CinderBroker.java:257)
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.storage.disk.cinder.ExtendCinderDiskCommandCallback.childCommandsExecutionEnded(ExtendCinderDiskCommandCallback.java:46)
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.ChildCommandsCallbackBase.doPolling(ChildCommandsCallbackBase.java:80)
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethodsImpl(CommandCallbacksPoller.java:181)
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethods(CommandCallbacksPoller.java:109)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
	at org.glassfish.javax.enterprise.concurrent//org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.access$201(ManagedScheduledThreadPoolExecutor.java:360)
	at org.glassfish.javax.enterprise.concurrent//org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.run(ManagedScheduledThreadPoolExecutor.java:511)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)
	at org.glassfish.javax.enterprise.concurrent//org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:227)
Caused by: org.jboss.resteasy.spi.ReaderException: com.fasterxml.jackson.databind.exc.MismatchedInputException: Root name 'volumes' does not match expected ('volume') for type [simple type, class com.woorea.openstack.cinder.model.Volume]
 at [Source: (org.jboss.resteasy.client.core.BaseClientResponse$InputStreamWrapper); line: 1, column: 2] (through reference chain: com.woorea.openstack.cinder.model.Volume["volumes"])
	at org.jboss.resteasy.resteasy-jaxrs.2.Final//org.jboss.resteasy.client.core.BaseClientResponse.readFrom(BaseClientResponse.java:481)
	at org.jboss.resteasy.resteasy-jaxrs.2.Final//org.jboss.resteasy.client.core.BaseClientResponse.getEntity(BaseClientResponse.java:394)
	at org.jboss.resteasy.resteasy-jaxrs.2.Final//org.jboss.resteasy.client.core.BaseClientResponse.getEntity(BaseClientResponse.java:365)
	at org.jboss.resteasy.resteasy-jaxrs.2.Final//org.jboss.resteasy.client.core.BaseClientResponse.getEntity(BaseClientResponse.java:359)
	at com.woorea.openstack.sdk//com.woorea.openstack.connector.RESTEasyResponse.getEntity(RESTEasyResponse.java:25)
	at com.woorea.openstack.sdk//com.woorea.openstack.base.client.OpenStackClient.execute(OpenStackClient.java:67)
	at com.woorea.openstack.sdk//com.woorea.openstack.base.client.OpenStackRequest.execute(OpenStackRequest.java:102)
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.provider.storage.OpenStackVolumeProviderProxy.getVolumeById(OpenStackVolumeProviderProxy.java:115)
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.storage.disk.cinder.CinderBroker.lambda$getDiskStatus$12(CinderBroker.java:258)
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.storage.disk.cinder.CinderBroker.execute(CinderBroker.java:53)
	... 13 more
Caused by: com.fasterxml.jackson.databind.exc.MismatchedInputException: Root name 'volumes' does not match expected ('volume') for type [simple type, class com.woorea.openstack.cinder.model.Volume]
 at [Source: (org.jboss.resteasy.client.core.BaseClientResponse$InputStreamWrapper); line: 1, column: 2] (through reference chain: com.woorea.openstack.cinder.model.Volume["volumes"])
	at com.fasterxml.jackson.core.jackson-databind.1//com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:63)
	at com.fasterxml.jackson.core.jackson-databind.1//com.fasterxml.jackson.databind.DeserializationContext.reportPropertyInputMismatch(DeserializationContext.java:1477)
	at com.fasterxml.jackson.core.jackson-databind.1//com.fasterxml.jackson.databind.DeserializationContext.reportPropertyInputMismatch(DeserializationContext.java:1493)
	at com.fasterxml.jackson.core.jackson-databind.1//com.fasterxml.jackson.databind.ObjectReader._unwrapAndDeserialize(ObjectReader.java:1850)
	at com.fasterxml.jackson.core.jackson-databind.1//com.fasterxml.jackson.databind.ObjectReader._bind(ObjectReader.java:1679)
	at com.fasterxml.jackson.core.jackson-databind.1//com.fasterxml.jackson.databind.ObjectReader.readValue(ObjectReader.java:977)
	at com.fasterxml.jackson.jaxrs.jackson-jaxrs-json-provider.1//com.fasterxml.jackson.jaxrs.base.ProviderBase.readFrom(ProviderBase.java:814)
	at org.jboss.resteasy.resteasy-jaxrs.2.Final//org.jboss.resteasy.core.interception.AbstractReaderInterceptorContext.readFrom(AbstractReaderInterceptorContext.java:66)
	at org.jboss.resteasy.resteasy-jaxrs.2.Final//org.jboss.resteasy.core.interception.AbstractReaderInterceptorContext.proceed(AbstractReaderInterceptorContext.java:56)
	at org.jboss.resteasy.resteasy-jaxrs.2.Final//org.jboss.resteasy.client.core.BaseClientResponse.readFrom(BaseClientResponse.java:446)
	... 22 more

2020-12-10 21:20:22,470+07 ERROR [org.ovirt.engine.core.bll.storage.disk.cinder.ExtendCinderDiskCommandCallback] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-81) [4b7c14e8] Failed extending disk. ID: 336069bf-5c00-4280-a715-b5dacde53f9d
2020-12-10 21:20:22,474+07 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-81) [4b7c14e8] EVENT_ID: USER_EXTEND_DISK_SIZE_FAILURE(370), Failed to extend size of the disk '222cinder' to 20 GB, User: k0ste@FreeIPA-authz.
2020-12-10 21:20:22,477+07 ERROR [org.ovirt.engine.core.bll.storage.disk.cinder.ExtendCinderDiskCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-81) [4b7c14e8] Ending command 'org.ovirt.engine.core.bll.storage.disk.cinder.ExtendCinderDiskCommand' with failure.
2020-12-10 21:20:22,477+07 INFO  [org.ovirt.engine.core.bll.storage.disk.cinder.ExtendCinderDiskCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-81) [4b7c14e8] Lock freed to object 'EngineLock:{exclusiveLocks='[]', sharedLocks='[]'}'
```


[1] https://gerrit.ovirt.org/c/ovirt-engine/+/103448

Comment 2 Konstantin Shalygin 2020-12-10 15:28:56 UTC
Created attachment 1738230 [details]
Proposal patch for this issue (origianl author is Ilya Krapchatov <i.krapchatov>)

Comment 3 shani 2020-12-13 10:17:00 UTC
Thanks for your awareness, Konstantin.
I think the issue related patch was https://gerrit.ovirt.org/#/c/ovirt-engine/+/104301/.

From a brief look, I didn't see the referral to the parameters as you've mentioned, although the command's declaration was changed.

Comment 4 RHEL Program Management 2020-12-13 10:17:08 UTC
The documentation text flag should only be set after 'doc text' field is provided. Please provide the documentation text and set the flag to '?' again.

Comment 5 Konstantin Shalygin 2020-12-14 07:37:38 UTC
@shani, this just proposal, for first we need patch for BZ1904669, for API working again.

Comment 6 Eyal Shenitzky 2020-12-21 15:28:21 UTC
The "old" way of using cinder as an external provider is deprecated.

Please see the new cinder integration using cinderlib AKA as Managed Block Storage.

For more info, you can see - https://www.ovirt.org/develop/release-management/features/storage/cinderlib-integration.html

Comment 7 Sandro Bonazzola 2020-12-22 08:56:58 UTC
(In reply to Eyal Shenitzky from comment #6)
> The "old" way of using cinder as an external provider is deprecated.

Please open a proper deprecation bug and get it documented as deprecated feature.


> Please see the new cinder integration using cinderlib AKA as Managed Block
> Storage.
> 
> For more info, you can see -
> https://www.ovirt.org/develop/release-management/features/storage/cinderlib-
> integration.html

This feature page is obsolete and not a valid user documentation.
I'm working with doc team to get it properly added to oVirt Installation Guide.

Comment 8 Eyal Shenitzky 2020-12-22 09:21:26 UTC
(In reply to Sandro Bonazzola from comment #7)
> (In reply to Eyal Shenitzky from comment #6)
> > The "old" way of using cinder as an external provider is deprecated.
> 
> Please open a proper deprecation bug and get it documented as deprecated
> feature.

Already opened by Tal - bug 1899453.


Note You need to log in before you can comment on or make changes to this bug.