Bug 1067429

Summary: [LinearStore] last file from deleted queue is not moved to EFP
Product: Red Hat Enterprise MRG Reporter: Zdenek Kraus <zkraus>
Component: qpid-cppAssignee: Kim van der Riet <kim.vdriet>
Status: POST --- QA Contact: Messaging QE <messaging-qe-bugs>
Severity: low Docs Contact:
Priority: low    
Version: 3.0CC: jross, pmoravec
Target Milestone: 3.3   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1443470 (view as bug list) Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1113452    
Attachments:
Description Flags
Proposed patch none

Description Zdenek Kraus 2014-02-20 12:58:22 UTC
Description of problem:
When deleting an empty durable queue, the last empty page that it held is not moved to EFP but dropped.

Version-Release number of selected component (if applicable):
0.22-35

How reproducible:
100%

Steps to Reproduce:
1. ./drain "q;{create:always, node: {durable: true}}"
2. ls /var/lib/qpidd/qls/p001/efp/2048k /var/lib/qpidd/qls/jrnl/q

3. ./drain "q;{delete:always}"
4. ls qls/p001/efp/2048k qls/jrnl/q

Actual results:
[2]
qls/jrnl/q:
68d88901-9310-4825-8c57-6c6c7f96c3ef.jrnl

qls/p001/efp/2048k:

[4]
ls: cannot access qls/jrnl/q: No such file or directory
qls/p001/efp/2048k:
Expected results:
[2]
qls/jrnl/q:
68d88901-9310-4825-8c57-6c6c7f96c3ef.jrnl

qls/p001/efp/2048k:

[4]
ls: cannot access qls/jrnl/q: No such file or directory
qls/p001/efp/2048k:
68d88901-9310-4825-8c57-6c6c7f96c3ef.jrnl

Additional info:

Comment 1 Kim van der Riet 2014-06-18 19:19:11 UTC
This is a known problem, which needs a fix at some point. It is  not very serious unless there is a large turnover of queues.

Comment 2 Kim van der Riet 2014-10-13 21:35:49 UTC
*** Bug 1150430 has been marked as a duplicate of this bug. ***

Comment 3 Pavel Moravec 2017-04-19 09:50:46 UTC
I think this can be a problem for Satellite where durable queues (named pulp.agent.*) are used for buffering & dispatching tasks to katello-agent running on client systems.

Roughly speaking, every client system managed by Sat6 has such a queue. The queue lifecycle corresponds to lifecycle of the system. So if somebody creates, uses and discards VMs managed by Satellite often, /var/lib/qpidd will get an orphaned journal file often.

So this should be fixed for Satellite6 with bigger priority than for MRG - I will clone it for Sat6.

Comment 4 Kim van der Riet 2017-05-17 17:04:21 UTC
Looking at the source, this bug could cause more than one journal file to be orphaned if the persistent queue is not empty and the queue content spans more than the last file.

At fault is the use of the LinearFileControlle::purgeEmptyFilesToEfp() method for destroying a queue. This function only removes empty files (which contain no message content) and is intended for harvesting empty journal files during the normal course of operations only.

A new function should be added which is intended to remove all journal files without checking for content, as it is assumed that when a queue is destroyed, the message content is no longer relevant.

Comment 5 Kim van der Riet 2017-05-17 17:15:56 UTC
Created attachment 1279762 [details]
Proposed patch

Needs testing