Bug 1294458 - [linearstore] enqueueing standalone small message takes 4kB, same as 10 messages in row
Summary: [linearstore] enqueueing standalone small message takes 4kB, same as 10 messa...
Keywords:
Status: NEW
Alias: None
Product: Red Hat Enterprise MRG
Classification: Red Hat
Component: qpid-cpp
Version: 3.2
Hardware: x86_64
OS: Linux
medium
high
Target Milestone: ---
: ---
Assignee: messaging-bugs
QA Contact: Messaging QE
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-12-28 10:03 UTC by Pavel Moravec
Modified: 2020-11-04 22:30 UTC (History)
1 user (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed:
Target Upstream Version:


Attachments (Terms of Use)

Description Pavel Moravec 2015-12-28 10:03:07 UTC
Description of problem:
Enqueueing a standalone durable message to a durable queue, the enqueue record takes 4kB of journal file. A *lot* of empty space filled by "QLSx" records is occupied that way. That is really inefficient usage of linearstore journals.

There is no 4kB boundary hardcoded for a single enqueue record, since sending 10 messages in one qpid-send, all fit into the same single 4kB "slot" inside the journal file.

Enqueueing a single message should consume just necessary disk space but not QLSx records.


Version-Release number of selected component (if applicable):
qpid-cpp-server-linearstore-0.34-5.el6.x86_64


How reproducible:
100%


Steps to Reproduce:
1. Enqueue 10 tiny durable messages to durable queue q1 and count it's journal file footprint:

qpid-config add queue q1 --durable
/usr/bin/qpid-send -a q1 --content-size=0 --sequence=no --timestamp=no --durable=yes -m10
hexdump -C /var/lib/qpidd/qls/jrnl2/q1/*

2. Enqueue the messages in 10 individual calls of qpid-send (to queue q2):
qpid-config add queue q2 --durable
for i in $(seq 1 10); do
    /usr/bin/qpid-send -a q2 --content-size=0 --sequence=no --timestamp=no --durable=yes -m1
done
hexdump -C /var/lib/qpidd/qls/jrnl2/q2/*


Actual results:
1. ends at:

*
00001f80  51 4c 53 78 ff ff ff ff  ff ff ff ff ff ff ff ff  |QLSx............|
00001f90  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff  |................|
*
00002000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00201000

2. ends at:
*
0000af80  51 4c 53 78 ff ff ff ff  ff ff ff ff ff ff ff ff  |QLSx............|
0000af90  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff  |................|
*
0000b000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00201000

See every tiny message consumed 4kB in the journal file.


Expected results:
Both 1. and 2. ends at 00002000.


Additional info:

Comment 1 Pavel Moravec 2015-12-28 10:26:17 UTC
The same applies to message _dequeue_ as well (not everytime).

I think the cause is flush interval that writes asynchronously to the disk data every 0.5s or so, in blocks of 4kB. So for frequent enqueues/dequeues, the bug *should* not be observed (but the reproducer triggers quite frequent operations, no?).


Note You need to log in before you can comment on or make changes to this bug.