Bug 975422

Hi Gordon, I see three issues here:

1. when the message is pulled for first time, the x-amqp-delivery-count property is not listed (ie. if x-amqp-delivery-count == 0, it is not listed in the received message properties map), is that expected?

# $cppapi/spout --connection-options {protocol:amqp1.0} --count 2 "q;{create:sender}"
# /var/dtests/node_data/clients/qc2_drain --log-msg-dict  --connection-options {protocol:amqp1.0} --count 1 q
{'redelivered': False, 'reply-to': None, 'subject': None, 'content_type': None, 'id': None, 'user_id': None, 'correlation_id': None, 'priority': None, 'durable': False, 'ttl': 0, 'properties': {'spout-id': 'f0e19b71-d03f-4209-b707-462ca625a3bc:0'}, 'content': None}
# qpid-receive --connection-options {protocol:amqp1.0} --print-headers true --address q
Properties: {spout-id:01e90b3e-96ba-437d-bdf2-d4d30781dc1f:1}

2. The behavior is as expected when using spout/drain client, ie:

# $cppapi/spout --connection-options {protocol:amqp1.0} --count 2 "q;{create:sender}"
# /var/dtests/node_data/clients/qc2_drain --log-msg-dict  --connection-options {protocol:amqp1.0} --count 1 q
{'redelivered': False, 'reply-to': None, 'subject': None, 'content_type': None, 'id': None, 'user_id': None, 'correlation_id': None, 'priority': None, 'durable': False, 'ttl': 0, 'properties': {'spout-id': '1b1fa22d-6c3c-471e-9e25-08b38c2a6778:0'}, 'content': None}
/* OK - x-amqp-delivery-count not listed (==0), redelivered flag is false (expected) */
/* next message will be released */
# /var/dtests/node_data/clients/qc2_drain --log-msg-dict  --connection-options {protocol:amqp1.0} --count 1 --accept release q
{'redelivered': False, 'reply-to': None, 'subject': None, 'content_type': None, 'id': None, 'user_id': None, 'correlation_id': None, 'priority': None, 'durable': False, 'ttl': 0, 'properties': {'spout-id': '65b9cb7e-1425-4304-9976-3a2837520492:1'}, 'content': None}
// OK - x-amqp-delivery-count not listed (==0), redelivered flag is false (expected)
# /var/dtests/node_data/clients/qc2_drain --log-msg-dict  --connection-options {protocol:amqp1.0} --count 1 q
{'redelivered': True, 'reply-to': None, 'subject': None, 'content_type': None, 'id': None, 'user_id': None, 'correlation_id': None, 'priority': None, 'durable': False, 'ttl': 0, 'properties': {'spout-id': '65b9cb7e-1425-4304-9976-3a2837520492:1', 'x-amqp-delivery-count': 1}, 'content': None}
// OK - redelivered flag set to true, x-amqp-delivery-count=1 (expected)

but I see wrong behavior when using qpid-send/qpid-receive clients - x-amqp-delivery-count increases while fetching next message:

# qpid-send --messages 3 --address "q;{create:sender}"
# qpid-receive --connection-options {protocol:amqp1.0} --print-headers true --messages 1 --address q
Properties: {sn:1, ts:1387456209158654200}
# qpid-receive --connection-options {protocol:amqp1.0} --print-headers true --messages 1 --address q
Redelivered: true
Properties: {sn:2, ts:1387456209158736790, x-amqp-delivery-count:1}
// False - x-amqp-delivery-count has increased when consuming next message 
# qpid-receive --connection-options {protocol:amqp1.0} --print-headers true --messages 1 --address q
Redelivered: true
Properties: {sn:3, ts:1387456209158753421, x-amqp-delivery-count:2}
// False - x-amqp-delivery-count has increased when consuming next message

Note this is not valid when all the messages are consumed at once
# qpid-send --messages 3 --address "q;{create:sender}"
# qpid-receive --connection-options {protocol:amqp1.0} --print-headers true --address q
Properties: {sn:1, ts:1387456129750236949}
Properties: {sn:2, ts:1387456129750319161}
Properties: {sn:3, ts:1387456129750335625}

3.) The following "pseudo headers/properties" shall be supported via amqp1.0:

x-amqp-first-acquirer, x-amqp-delivery-count, x-amqp-group-id, x-amqp-group-sequence, x-amqp-to, x-amqp-reply-to-group-id, x-amqp-absolute-expiry-time, x-amqp-creation-time

The retrieved message do not list all the properties set by the producer, ie.:

# $cppapi/spout -P "x-amqp-first-acquirer=True" -P "x-amqp-delivery-count=10" -P "x-amqp-to=q" -P "x-amqp-group-id=theGroup" -P "x-amqp-group-sequence=10" -P "x-amqp-reply-to-group-id=theGroup" -P "x-amqp-absolute-expiry-time=1387289524" -P "x-amqp-creation-time=1387289524" "q;{create:sender}" --count 2 --connection-options "{protocol:'amqp1.0'}"
# /var/dtests/node_data/clients/qc2_drain --log-msg-dict  --connection-options {protocol:amqp1.0} --count 1 q{'redelivered': False, 'reply-to': None, 'subject': None, 'content_type': None, 'id': None, 'user_id': None, 'correlation_id': None, 'priority': None, 'durable': False, 'ttl': 0, 'properties': {'spout-id': 'c757ba29-0765-4746-9185-4280d1b1c84f:0', 'x-amqp-group-id': 'theGroup', 'x-amqp-qroup-sequence': 10, 'x-amqp-reply-to-group-id': 'theGroup', 'x-amqp-to': 'q'}, 'content': None}
# qpid-receive --connection-options {protocol:amqp1.0} --print-headers true --messages 1 --address qProperties: {spout-id:58d36995-51f5-482f-be46-ac1381c58e43:1, x-amqp-group-id:theGroup, x-amqp-qroup-sequence:10, x-amqp-reply-to-group-id:theGroup, x-amqp-to:q}

Following properties are listed on received message:
x-amqp-group-id, x-amqp-qroup-sequence, x-amqp-reply-to-group-id, x-amqp-to

While following properties are missing:
x-amqp-first-acquirer, x-amqp-delivery-count (clear, because == 0), x-amqp-absolute-expiry-time, x-amqp-creation-time

is that expected? is there a way how to list these properties using messaging api? I believe there shall be possibility to list them if there is possibility to set them.

Thanks in advance for your feedback.

(In reply to Petr Matousek from comment #1)
> Hi Gordon, I see three issues here:
> 
> 1. when the message is pulled for first time, the x-amqp-delivery-count
> property is not listed (ie. if x-amqp-delivery-count == 0, it is not listed
> in the received message properties map), is that expected?

Yes, the default is 0, so its only present if different from that.

> 2. The behavior is as expected when using spout/drain client, ie:

> but I see wrong behavior when using qpid-send/qpid-receive clients -
> x-amqp-delivery-count increases while fetching next message:

I believe that is just because of the prefetch. If you set --capacity 0, it should be the same as for drain.

There is a reasonable question about whether to increment the count for prefetched messages that have not been seen by the application. On the one hand they haven't been processed; on the other, they could be out of order anyway. Then again, the first-acquirer could be set to false perhaps...

I'm not entirely sure what I think is best there yet, would need to think about it a bit. In any case I think it would be a separate issue from this one. I.e. "don't increment the delivery count for prefetched message not delivered to application"

> 3.) The following "pseudo headers/properties" shall be supported via amqp1.0:
> 
> x-amqp-first-acquirer, x-amqp-delivery-count, x-amqp-group-id,
> x-amqp-group-sequence, x-amqp-to, x-amqp-reply-to-group-id,
> x-amqp-absolute-expiry-time, x-amqp-creation-time
> 
> The retrieved message do not list all the properties set by the producer,
[...]
> Following properties are listed on received message:
> x-amqp-group-id, x-amqp-qroup-sequence, x-amqp-reply-to-group-id, x-amqp-to
> 
> While following properties are missing:
> x-amqp-first-acquirer, x-amqp-delivery-count (clear, because == 0),
> x-amqp-absolute-expiry-time, x-amqp-creation-time
> 
> is that expected? is there a way how to list these properties using
> messaging api? I believe there shall be possibility to list them if there is
> possibility to set them.

creation-time and absolute-expiry-time should certainly be settable by sender and transmitted as set through to receiver. Again. I think that would merit a separate BZ.

first-acquirer and delivery-count are somewhat different. you could certainly argue that they should be settable (and they may not be). However the value of those fields on a received message need not - indeed probably should not- be the same as the value on the message published to the broker.

Does that make sense

(In reply to Gordon Sim from comment #2)
> (In reply to Petr Matousek from comment #1)
> > 1.
[...]
> Yes, the default is 0, so its only present if different from that.

agreed
> > 2. The behavior is as expected when using spout/drain client, ie:
> I believe that is just because of the prefetch. If you set --capacity 0, it
> should be the same as for drain.
[...]

Confirmed neither delivery count is increased nor redelivered flag set when using capacity 0, created bug 1045092 to track this issue, feel free to change to doc bug if the behavior is identified to be correct.

> > 3.) The following "pseudo headers/properties" shall be supported via amqp1.0:
[..]
created bug 1045091 in order to track this issue.

Thanks Gordon.

This This issue has been fixed. Verified on rhel6.5 (x86_64, i386).

packages used for test:
qpid-cpp-*-0.22-33

-> VERIFIED