Description of problem: ----------------------- While checking the engine.log post RHHI-V deployment, there are few errors related to GlusterHooks operations. This is not causing any functional issues. Version-Release number of selected component (if applicable): ------------------------------------------------------------- RHV Manager version - 4.4.1-11 RHHI-V 1.8 How reproducible: -------------------- Always Steps to Reproduce: -------------------- 1. Complete hyperconverged deployment 2. Check engine.log for errors Actual results: ---------------- 1. Some repetitive errors related to 'GetGlusterHookContentVDSCommand' are seen 2. Exception related to reading glusterhook.read is also seen Expected results: ----------------- No errors should be seen in engine.log related to gluster hooks
Some of the errors looks like Python porting problem <snip> 2020-07-16 13:39:56,697Z ERROR [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterHookContentVDSCommand] (EE-ManagedThreadFactory-engine-Thread-2845) [] Failed in 'GetGlusterHookContentVDS' method 2020-07-16 13:39:56,697Z ERROR [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterHookContentVDSCommand] (EE-ManagedThreadFactory-engine-Thread-2842) [] Failed in 'GetGlusterHookContentVDS' method 2020-07-16 13:39:56,697Z WARN [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterHookContentVDSCommand] (EE-ManagedThreadFactory-engine-Thread-2842) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "Attempt to call function: <bound method GlusterHook.read of <vdsm.gluster.apiwrapper.GlusterHook object at 0x7fa9a0699128>> with arguments: ('add-brick', 'PRE', '28Quota-enable-root-xattr-heal.sh') error: a bytes-like object is required, not 'str'"}] 2020-07-16 13:39:56,697Z WARN [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterHookContentVDSCommand] (EE-ManagedThreadFactory-engine-Thread-2845) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "Attempt to call function: <bound method GlusterHook.read of <vdsm.gluster.apiwrapper.GlusterHook object at 0x7fa9a05e46d8>> with arguments: ('gsync-create', 'POST', '56glusterd-geo-rep-create-post.sh') error: a bytes-like object is required, not 'str'"}] 2020-07-16 13:39:56,699Z ERROR [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterHookContentVDSCommand] (EE-ManagedThreadFactory-engine-Thread-2845) [] Command 'GetGlusterHookContentVDSCommand(HostName = rhsqa-grafton8-nic2.lab.eng.blr.redhat.com, GlusterHookVDSParameters:{hostId='577aa8ad-a5bb-47db-b664-caf8738efed2'})' execution failed: VDSGenericException: VDSErrorException: Failed to GetGlusterHookContentVDS, error = Internal JSON-RPC error: {'reason': "Attempt to call function: <bound method GlusterHook.read of <vdsm.gluster.apiwrapper.GlusterHook object at 0x7fa9a05e46d8>> with arguments: ('gsync-create', 'POST', '56glusterd-geo-rep-create-post.sh') error: a bytes-like object is required, not 'str'"}, code = -32603 2020-07-16 13:39:56,699Z INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterHookContentVDSCommand] (EE-ManagedThreadFactory-engine-Thread-2845) [] FINISH, GetGlusterHookContentVDSCommand, return: , log id: 47f11e70 2020-07-16 13:39:56,702Z ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-2842) [] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM rhsqa-grafton8-nic2.lab.eng.blr.redhat.com command GetGlusterHookContentVDS failed: Internal JSON-RPC error: {'reason': "Attempt to call function: <bound method GlusterHook.read of <vdsm.gluster.apiwrapper.GlusterHook object at 0x7fa9a0699128>> with arguments: ('add-brick', 'PRE', '28Quota-enable-root-xattr-heal.sh') error: a bytes-like object is required, not 'str'"} 2020-07-16 13:39:56,702Z ERROR [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterHookContentVDSCommand] (EE-ManagedThreadFactory-engine-Thread-2842) [] Command 'GetGlusterHookContentVDSCommand(HostName = rhsqa-grafton8-nic2.lab.eng.blr.redhat.com, GlusterHookVDSParameters:{hostId='577aa8ad-a5bb-47db-b664-caf8738efed2'})' execution failed: VDSGenericException: VDSErrorException: Failed to GetGlusterHookContentVDS, error = Internal JSON-RPC error: {'reason': "Attempt to call function: <bound method GlusterHook.read of <vdsm.gluster.apiwrapper.GlusterHook object at 0x7fa9a0699128>> with arguments: ('add-brick', 'PRE', '28Quota-enable-root-xattr-heal.sh') error: a bytes-like object is required, not 'str'"}, code = -32603 2020-07-16 13:39:56,702Z INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterHookContentVDSCommand] (EE-ManagedThreadFactory-engine-Thread-2842) [] FINISH, GetGlusterHookContentVDSCommand, return: , log id: 105fbd65 2020-07-16 13:39:57,698Z WARN [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterHookContentVDSCommand] (EE-ManagedThreadFactory-engine-Thread-2844) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "Attempt to call function: <bound method GlusterHook.read of <vdsm.gluster.apiwrapper.GlusterHook object at 0x7fa9a0699438>> with arguments: ('delete', 'POST', '57glusterfind-delete-post') error: a bytes-like object is required, not 'str'"}] </snip> Some are run time exceptions <snip> 2020-07-16 13:39:59,685Z ERROR [org.ovirt.engine.core.bll.gluster.GlusterHookSyncJob] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-28) [] Exception in sync: java.lang.RuntimeException: java.util.concurrent.ExecutionException: org.ovirt.engine.core.common.errors.EngineException: EngineExcepti on: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed to GetGlusterHookContentVDS, error = Internal JSON-RPC error: {'reason': "Attempt to call function: <bound method GlusterHook.read of <vdsm.gluster.apiwrapper.GlusterHook object at 0x7fa9a0699128>> with a rguments: ('add-brick', 'PRE', '28Quota-enable-root-xattr-heal.sh') error: a bytes-like object is required, not 'str'"}, code = -32603 (Failed with error unex pected and code 16) at org.ovirt.engine.core.utils//org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil.invokeAll(ThreadPoolUtil.java:196) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.gluster.GlusterHookSyncJob.saveHookContent(GlusterHookSyncJob.java:234) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.gluster.GlusterHookSyncJob.saveNewHooks(GlusterHookSyncJob.java:226) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.gluster.GlusterHookSyncJob.addOrUpdateHooks(GlusterHookSyncJob.java:189) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.gluster.GlusterHookSyncJob.refreshHooksInCluster(GlusterHookSyncJob.java:85) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.gluster.GlusterHookSyncJob.refreshHooks(GlusterHookSyncJob.java:54) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.scheduling.OvirtGlusterSchedulingService$1.run(OvirtGlusterSchedulingService.java:221) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) at org.glassfish.javax.enterprise.concurrent.0.redhat-1//org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.access$201(ManagedScheduledThreadPoolExecutor.java:383) at org.glassfish.javax.enterprise.concurrent.0.redhat-1//org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.run(ManagedScheduledThreadPoolExecutor.java:534) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:834) at org.glassfish.javax.enterprise.concurrent.0.redhat-1//org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:250) </snip>
Created attachment 1701529 [details] engine.log
Verified with RHV 4.4.3.8-0.1.el8ev No such errors or tracebacks seen in engine.log post RHHI-V deployment