Bug 975422 - [AMQP 1.0] delivery count set incorrectly
Summary: [AMQP 1.0] delivery count set incorrectly
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Enterprise MRG
Classification: Red Hat
Component: qpid-cpp
Version: Development
Hardware: Unspecified
OS: Unspecified
high
medium
Target Milestone: 3.0
: ---
Assignee: Gordon Sim
QA Contact: Petr Matousek
URL:
Whiteboard:
Depends On:
Blocks: 974061 1010399
TreeView+ depends on / blocked
 
Reported: 2013-06-18 12:45 UTC by Gordon Sim
Modified: 2015-01-21 12:56 UTC (History)
5 users (show)

Fixed In Version: qpid-cpp-0.22-5.el6, qpid-cpp-0.22-5.el5
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-01-21 12:56:34 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Apache JIRA QPID-4668 None None None Never

Description Gordon Sim 2013-06-18 12:45:30 UTC
Description of problem:

The broker sends the delivery count as 1 for the first delivery whereas the 1.0 specification requires that it be sent as 0. (Note that originally this bug manifested itself as all messages being marked redelivered by the client, but I mistakenly 'fixed' this on the client such that the clients expectations were met but the solution did not honour the specification. 

Version-Release number of selected component (if applicable):

Development

How reproducible:

100%

Steps to Reproduce:
1. pull a message off a queue (for the first time)
2. test the x-amqp-delivery-count property

Actual results:

x-amqp-delivery-count property is 1

Expected results:

x-amqp-delivery-count property is 0


Additional info:

Comment 1 Petr Matousek 2013-12-19 13:41:06 UTC
Hi Gordon, I see three issues here:

1. when the message is pulled for first time, the x-amqp-delivery-count property is not listed (ie. if x-amqp-delivery-count == 0, it is not listed in the received message properties map), is that expected?

# $cppapi/spout --connection-options {protocol:amqp1.0} --count 2 "q;{create:sender}"
# /var/dtests/node_data/clients/qc2_drain --log-msg-dict  --connection-options {protocol:amqp1.0} --count 1 q
{'redelivered': False, 'reply-to': None, 'subject': None, 'content_type': None, 'id': None, 'user_id': None, 'correlation_id': None, 'priority': None, 'durable': False, 'ttl': 0, 'properties': {'spout-id': 'f0e19b71-d03f-4209-b707-462ca625a3bc:0'}, 'content': None}
# qpid-receive --connection-options {protocol:amqp1.0} --print-headers true --address q
Properties: {spout-id:01e90b3e-96ba-437d-bdf2-d4d30781dc1f:1}

2. The behavior is as expected when using spout/drain client, ie:

# $cppapi/spout --connection-options {protocol:amqp1.0} --count 2 "q;{create:sender}"
# /var/dtests/node_data/clients/qc2_drain --log-msg-dict  --connection-options {protocol:amqp1.0} --count 1 q
{'redelivered': False, 'reply-to': None, 'subject': None, 'content_type': None, 'id': None, 'user_id': None, 'correlation_id': None, 'priority': None, 'durable': False, 'ttl': 0, 'properties': {'spout-id': '1b1fa22d-6c3c-471e-9e25-08b38c2a6778:0'}, 'content': None}
/* OK - x-amqp-delivery-count not listed (==0), redelivered flag is false (expected) */
/* next message will be released */
# /var/dtests/node_data/clients/qc2_drain --log-msg-dict  --connection-options {protocol:amqp1.0} --count 1 --accept release q
{'redelivered': False, 'reply-to': None, 'subject': None, 'content_type': None, 'id': None, 'user_id': None, 'correlation_id': None, 'priority': None, 'durable': False, 'ttl': 0, 'properties': {'spout-id': '65b9cb7e-1425-4304-9976-3a2837520492:1'}, 'content': None}
// OK - x-amqp-delivery-count not listed (==0), redelivered flag is false (expected)
# /var/dtests/node_data/clients/qc2_drain --log-msg-dict  --connection-options {protocol:amqp1.0} --count 1 q
{'redelivered': True, 'reply-to': None, 'subject': None, 'content_type': None, 'id': None, 'user_id': None, 'correlation_id': None, 'priority': None, 'durable': False, 'ttl': 0, 'properties': {'spout-id': '65b9cb7e-1425-4304-9976-3a2837520492:1', 'x-amqp-delivery-count': 1}, 'content': None}
// OK - redelivered flag set to true, x-amqp-delivery-count=1 (expected)

but I see wrong behavior when using qpid-send/qpid-receive clients - x-amqp-delivery-count increases while fetching next message:

# qpid-send --messages 3 --address "q;{create:sender}"
# qpid-receive --connection-options {protocol:amqp1.0} --print-headers true --messages 1 --address q
Properties: {sn:1, ts:1387456209158654200}
# qpid-receive --connection-options {protocol:amqp1.0} --print-headers true --messages 1 --address q
Redelivered: true
Properties: {sn:2, ts:1387456209158736790, x-amqp-delivery-count:1}
// False - x-amqp-delivery-count has increased when consuming next message 
# qpid-receive --connection-options {protocol:amqp1.0} --print-headers true --messages 1 --address q
Redelivered: true
Properties: {sn:3, ts:1387456209158753421, x-amqp-delivery-count:2}
// False - x-amqp-delivery-count has increased when consuming next message

Note this is not valid when all the messages are consumed at once
# qpid-send --messages 3 --address "q;{create:sender}"
# qpid-receive --connection-options {protocol:amqp1.0} --print-headers true --address q
Properties: {sn:1, ts:1387456129750236949}
Properties: {sn:2, ts:1387456129750319161}
Properties: {sn:3, ts:1387456129750335625}

3.) The following "pseudo headers/properties" shall be supported via amqp1.0:

x-amqp-first-acquirer, x-amqp-delivery-count, x-amqp-group-id, x-amqp-group-sequence, x-amqp-to, x-amqp-reply-to-group-id, x-amqp-absolute-expiry-time, x-amqp-creation-time

The retrieved message do not list all the properties set by the producer, ie.:

# $cppapi/spout -P "x-amqp-first-acquirer=True" -P "x-amqp-delivery-count=10" -P "x-amqp-to=q" -P "x-amqp-group-id=theGroup" -P "x-amqp-group-sequence=10" -P "x-amqp-reply-to-group-id=theGroup" -P "x-amqp-absolute-expiry-time=1387289524" -P "x-amqp-creation-time=1387289524" "q;{create:sender}" --count 2 --connection-options "{protocol:'amqp1.0'}"
# /var/dtests/node_data/clients/qc2_drain --log-msg-dict  --connection-options {protocol:amqp1.0} --count 1 q{'redelivered': False, 'reply-to': None, 'subject': None, 'content_type': None, 'id': None, 'user_id': None, 'correlation_id': None, 'priority': None, 'durable': False, 'ttl': 0, 'properties': {'spout-id': 'c757ba29-0765-4746-9185-4280d1b1c84f:0', 'x-amqp-group-id': 'theGroup', 'x-amqp-qroup-sequence': 10, 'x-amqp-reply-to-group-id': 'theGroup', 'x-amqp-to': 'q'}, 'content': None}
# qpid-receive --connection-options {protocol:amqp1.0} --print-headers true --messages 1 --address qProperties: {spout-id:58d36995-51f5-482f-be46-ac1381c58e43:1, x-amqp-group-id:theGroup, x-amqp-qroup-sequence:10, x-amqp-reply-to-group-id:theGroup, x-amqp-to:q}

Following properties are listed on received message:
x-amqp-group-id, x-amqp-qroup-sequence, x-amqp-reply-to-group-id, x-amqp-to

While following properties are missing:
x-amqp-first-acquirer, x-amqp-delivery-count (clear, because == 0), x-amqp-absolute-expiry-time, x-amqp-creation-time

is that expected? is there a way how to list these properties using messaging api? I believe there shall be possibility to list them if there is possibility to set them.

Thanks in advance for your feedback.

Comment 2 Gordon Sim 2013-12-19 14:00:01 UTC
(In reply to Petr Matousek from comment #1)
> Hi Gordon, I see three issues here:
> 
> 1. when the message is pulled for first time, the x-amqp-delivery-count
> property is not listed (ie. if x-amqp-delivery-count == 0, it is not listed
> in the received message properties map), is that expected?

Yes, the default is 0, so its only present if different from that.

> 2. The behavior is as expected when using spout/drain client, ie:

> but I see wrong behavior when using qpid-send/qpid-receive clients -
> x-amqp-delivery-count increases while fetching next message:

I believe that is just because of the prefetch. If you set --capacity 0, it should be the same as for drain.

There is a reasonable question about whether to increment the count for prefetched messages that have not been seen by the application. On the one hand they haven't been processed; on the other, they could be out of order anyway. Then again, the first-acquirer could be set to false perhaps...

I'm not entirely sure what I think is best there yet, would need to think about it a bit. In any case I think it would be a separate issue from this one. I.e. "don't increment the delivery count for prefetched message not delivered to application"

> 3.) The following "pseudo headers/properties" shall be supported via amqp1.0:
> 
> x-amqp-first-acquirer, x-amqp-delivery-count, x-amqp-group-id,
> x-amqp-group-sequence, x-amqp-to, x-amqp-reply-to-group-id,
> x-amqp-absolute-expiry-time, x-amqp-creation-time
> 
> The retrieved message do not list all the properties set by the producer,
[...]
> Following properties are listed on received message:
> x-amqp-group-id, x-amqp-qroup-sequence, x-amqp-reply-to-group-id, x-amqp-to
> 
> While following properties are missing:
> x-amqp-first-acquirer, x-amqp-delivery-count (clear, because == 0),
> x-amqp-absolute-expiry-time, x-amqp-creation-time
> 
> is that expected? is there a way how to list these properties using
> messaging api? I believe there shall be possibility to list them if there is
> possibility to set them.

creation-time and absolute-expiry-time should certainly be settable by sender and transmitted as set through to receiver. Again. I think that would merit a separate BZ.

first-acquirer and delivery-count are somewhat different. you could certainly argue that they should be settable (and they may not be). However the value of those fields on a received message need not - indeed probably should not- be the same as the value on the message published to the broker.

Does that make sense

Comment 3 Petr Matousek 2013-12-19 15:13:40 UTC
(In reply to Gordon Sim from comment #2)
> (In reply to Petr Matousek from comment #1)
> > 1.
[...]
> Yes, the default is 0, so its only present if different from that.

agreed
> > 2. The behavior is as expected when using spout/drain client, ie:
> I believe that is just because of the prefetch. If you set --capacity 0, it
> should be the same as for drain.
[...]

Confirmed neither delivery count is increased nor redelivered flag set when using capacity 0, created bug 1045092 to track this issue, feel free to change to doc bug if the behavior is identified to be correct.

> > 3.) The following "pseudo headers/properties" shall be supported via amqp1.0:
[..]
created bug 1045091 in order to track this issue.

Thanks Gordon.

Comment 4 Petr Matousek 2014-01-27 12:25:15 UTC
This This issue has been fixed. Verified on rhel6.5 (x86_64, i386).

packages used for test:
qpid-cpp-*-0.22-33

-> VERIFIED


Note You need to log in before you can comment on or make changes to this bug.