This service will be undergoing maintenance at 00:00 UTC, 2017-10-23 It is expected to last about 30 minutes
Bug 1013777 - [Eng] (6.4.0) LargeMessages eventually left unattended after failures, making org.hornetq.tests.integration.cluster.failover.BackupSyncLargeMessageTest::testDeleteLargeMessages to intermittently fail [NEEDINFO]
[Eng] (6.4.0) LargeMessages eventually left unattended after failures, making...
Status: VERIFIED
Product: JBoss Enterprise Application Platform 6
Classification: JBoss
Component: HornetQ (Show other bugs)
6.2.0
Unspecified Unspecified
unspecified Severity medium
: DR1
: EAP 6.4.0
Assigned To: Clebert Suconic
Martin Svehla
:
Depends On: 1016141
Blocks: 1132168 1132185
  Show dependency treegraph
 
Reported: 2013-09-30 14:25 EDT by Clebert Suconic
Modified: 2017-10-09 20:22 EDT (History)
9 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: The delete for a large message is done asynchronously after the journal record has been deleted. Consequence: Eventually after a failure files could be left unattended requiring manual intervention to delete them. Fix: When deleting a large message, we now add a temporary record on the journal and verify its existent on startup. Result: A few files on the large message on the folder after crashes. You would need to crash the server after the issue was deleted and before the executor with a file.delete was finished.
Story Points: ---
Clone Of:
: 1132185 (view as bug list)
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
dmichael: needinfo? (csuconic)


Attachments (Terms of Use)

  None (edit)
Description Clebert Suconic 2013-09-30 14:25:36 EDT
Description of problem:

Large messages not being deleted after failover on BackupSyncLargeMessageTest

In a race condition, large message's journal record could be deleted before the file itself. You could eventually miss the delete command on replication and you won't have how to remove the files unless done manually.

The cost / risk for this is small as the only thing happening is a file that's not deleted after failover on replicated state.

The file can be removed manually after some time easily.

Version-Release number of selected component (if applicable):


How reproducible:

On in 100

Steps to Reproduce:
1. Add a loop method on BackupSyncLargeMessageTest
   @Test
   public void testLoop() throws Exception
   {
      for (int i = 0; i < 1000; i++)
      {
         System.out.println("#test " + i);
         testDeleteLargeMessages();
         tearDown();
         setUp();
      }
   }


2. Run the testLoop and watch it fail.


Actual results:
The test is failing.

Expected results:
The test shouldn't fail even after 1000 iterations


Additional info:
Comment 1 Martin Svehla 2013-10-17 09:13:53 EDT
I added suggested test to BackupSyncLargeMessageTest, but it still fails after some time in 2.3.9 with

java.lang.AssertionError: we really ought to delete these after delivery expected:<10> but was:<11>
Comment 3 Clebert Suconic 2013-10-17 09:20:24 EDT
We have seen failures on this test on our runs... We could keep it to next version.. I investigated and the only scenario we could get was a test issue...


Lets postpone this to next release?
Comment 4 Martin Svehla 2013-10-17 09:35:49 EDT
Agreed, this is not functional issue, so we can postpone.

(Setting qa nack for now to indicate we're ok with not having this in 6.2.0, I'll ack it later when we have some timeline for fix.)
Comment 6 Andy Taylor 2014-07-23 08:57:20 EDT
im working on this
Comment 7 Clebert Suconic 2014-08-04 15:37:19 EDT
https://github.com/hornetq/hornetq/pull/1755
Comment 8 Kabir Khan 2014-08-24 06:59:30 EDT
Appears to be fixed by HQ upgrade to 2.3.21 https://bugzilla.redhat.com/show_bug.cgi?id=1132168. Setting to MODIFIED.

Note You need to log in before you can comment on or make changes to this bug.