Bug 1420702

Summary: Error calculating MD5 is thrown sometimes when creating bigger deployment
Product: [JBoss] JBoss Operations Network Reporter: Filip Brychta <fbrychta>
Component: ContentAssignee: Michael Burman <miburman>
Status: CLOSED WONTFIX QA Contact: Mike Foley <mfoley>
Severity: low Docs Contact:
Priority: medium    
Version: JON 3.3.7, JON 3.3.8CC: fbrychta, miburman, rvargasp, spinder
Target Milestone: ER01Keywords: Triaged
Target Release: JON 3.3.10   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-01-25 15:35:09 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
rhq_package_version table
none
log and script
none
new repro script none

Description Filip Brychta 2017-02-09 10:44:27 UTC
Created attachment 1248778 [details]
rhq_package_version table

Description of problem:
Following warning msg is thrown to agent log when creating new or updating bigger eap6 deployment:
Error calculating MD5 [10017][PackageDetails[Key=PackageDetailsKey[Name=hello-eap6-4.war, Version=[sha256=e38f71a6582f72cac7e65b092ce1a2907128c2ce20d803e5345bdf652260fc05] Arch=noarch Type=file]]]
org.rhq.enterprise.communications.command.client.RemoteIOException: A remote input stream with an ID of [6] and server endpoint of [socket://10.16.23.175:16163/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&generalizeSocketException=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] has not yet been assigned a sender object - cannot access the stream
	at org.rhq.enterprise.communications.command.client.RemoteInputStream.sendRequest(RemoteInputStream.java:266)
	at org.rhq.enterprise.communications.command.client.RemoteInputStream.close(RemoteInputStream.java:186)
	at org.rhq.core.pc.content.RetrieveContentBitsRunner.run(RetrieveContentBitsRunner.java:112)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


Version-Release number of selected component (if applicable):
JON 3.3.7 but probably in earlier versions and 3.3.8 as well

How reproducible:
Sometimes

Steps to Reproduce:
1. prepare cli script which is deploying many war deployments to eap 6 - each deployment should be e.g. 10MB   
2. run the script and watch agent.log on eap6 host

Actual results:
There is sometimes following warn msgs:
Error calculating MD5 [10017][PackageDetails[Key=PackageDetailsKey[Name=hello-eap6-4.war, Version=[sha256=e38f71a6582f72cac7e65b092ce1a2907128c2ce20d803e5345bdf652260fc05] Arch=noarch Type=file]]]
org.rhq.enterprise.communications.command.client.RemoteIOException: A remote input stream with an ID of [6] and server endpoint of [socket://10.16.23.175:16163/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&generalizeSocketException=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] has not yet been assigned a sender object - cannot access the stream
	at org.rhq.enterprise.communications.command.client.RemoteInputStream.sendRequest(RemoteInputStream.java:266)
	at org.rhq.enterprise.communications.command.client.RemoteInputStream.close(RemoteInputStream.java:186)
	at org.rhq.core.pc.content.RetrieveContentBitsRunner.run(RetrieveContentBitsRunner.java:112)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2017-02-09 05:21:24,825 WARN  [WorkerThread#0[10.16.23.167:40238]] (command.impl.stream.server.RemoteInputStreamCommandService)- {RemoteInputStreamCommandService.invoke-stream-failure}Input stream being accessed from a remote client encountered an exception - the method invoked was [close]. The command was: [Command: type=[remotestream]; cmd-in-response=[true]; config=[{}]; params=[{streamId=5, invocation=NameBasedInvocation[close]}]]. Cause: java.lang.IllegalArgumentException: A command asked to access an unknown input stream with ID of [5].  The command was: [Command: type=[remotestream]; cmd-in-response=[true]; config=[{}]; params=[{streamId=5, invocation=NameBasedInvocation[close]}]]

Expected results:
No warnings no errors

Additional info:
See attached rhq_package_version table. Deployment with name hello-eap6-4.war - file_md5 column is empty but all file_sha256 are computed. Not sure if it's related to previous warning though.

Comment 1 Michael Burman 2017-02-09 14:20:53 UTC
Comment on this, it's not the MD5 calculation that fails, but closing the inputstream.

Comment 7 Filip Brychta 2017-11-07 11:41:52 UTC
Created attachment 1348908 [details]
log and script

Comment 10 Filip Brychta 2017-11-20 14:22:17 UTC
Created attachment 1355876 [details]
new repro script

Comment 11 Simeon Pinder 2018-01-09 21:42:45 UTC
@Micke, does the reproducer help clarify the issue and what still needs to be done to fix for ER01?

Comment 12 Michael Burman 2018-01-09 22:41:05 UTC
Appears I forgot to update our discussion with Filip here. Even the new reproducer does not reproduce it on my machine. So I could never test my fix as the code path is never taken on my machine.

And since there's no customer case attached, we agreed to drop spending more time on it until a customer hits it.

Comment 13 Filip Brychta 2018-01-25 15:35:09 UTC
Triage: Larry, Simeon, Filip - based on previous comment closing as won't fix