The broker currently accepts (but does not advertise support for) selector filter values using the "apache.org:selector-filter:string" filter. This filter describes a mapping for JMS Header names in to an AMQP equivalent, so that non-JMS clients do not need to refer to the JMS header names. The old JMS client, and many if not all the other brokers using this filter do not perform this mapping, and send/recieve the JMS header names unchanged, and the new JMS client does the same for compatibility. As a result, certain operations do not work, such as attempting to select based on JMSCorrelationID. There are other issues with the filter as defined that are likely lead to a new one being defined for the long term, so after discussion it seems the easiest and cleanest way to resolve the current client interop issue with qpidd would be for the broker to accept the JMS header names in addition to the mapped AMQP equivalents. As part of this the handling for JMSType should be updated to actually align with what clients do. The filter defined that the value would be sent as the message annotation 'jms_type', but neither the old client or the new client do this, probably in part because the AMQP spec states that annotations not beginning "x-" are reserved and those not beginning "x-opt-" that are not understood MUST be met with link closure. The new JMS client sends the JMSType header value in the 'subject' field of the message Properties section. The old client sends it using the "x-opt-jms-type" symbol message annotation.
Fixed upstream: https://svn.apache.org/r1698426
It is not very clear to us on what should we focus while testing this bug. Could you please be more detailed about what was fixed/updated?
(In reply to Michal Toth from comment #3) > It is not very clear to us on what should we focus while testing this bug. > Could you please be more detailed about what was fixed/updated? Prior to the fix, the broker would not recognise names such as JMSType, JMSMessageID, JMSCorrelationID, JMSPriority in selectors as mapping onto specific fields in the AMQP 1.0 defined message format. Now it does. So e.g. a selector of the form "JMSCorrelationID = 'my-message-1'" should match messages where the correlation-id of the AMQP 1.0 message was set to the string value 'my-message-1'. "JMSType = 'abc'" would match a message where the subject field as defined in AMQP 1.0 is set to the string 'abc'.
I have had a look at this and I think there is (at least) one error, while using qpid-jms AMQP 1.0 client selectors with topics. With queues all works as expected. The issue is, that when I declare JMSType to any value for message and send this message to topic, subscribed clients either with or without message selector do not receive this message at all. Steps to reproduce: 1) Subscribe to amq.direct with client receiving all messages ./aac1_receiver.java.sh --log-msgs dict -a topic://amq.direct -t -1 2) Subscribe to amq.direct with client having message selector with "JMSType='<value>'" ./aac1_receiver.java.sh --log-msgs dict -a topic://amq.direct -t -1 --msg-selector "JMSType='test_message'" 3) Send a message to amq.direct with declared JMSType specific <value> matching the selector ./aac1_sender.java.sh --log-msgs dict -a topic://amq.direct -c 1 --msg-content-type "test_message" Actual behavior: Message is not received by any client. Check "qpid-stat -e" and observe that message was not send out to clients. Expected behavior: Message is received by both subscribed clients (in case <value> is matched by selector of course). Note: I think this test is also in TCK, which is failing. The name is com/sun/ts/tests/jms/core/topictests/TopicTests.java#msgSelectorMsgHeaderTopicTest_from_standalone I am not sure whether bug is on broker or client side, (imo on broker side). That's why I am setting needinfo on Robbie and failing QA as well to get information from Gordon. qpid-cpp-server-0.34-4.el6.x86_64 qpid-jms-client-0.5.0-2.el6.noarch
(In reply to Michal Toth from comment #5) > I have had a look at this and I think there is (at least) one error, > while using qpid-jms AMQP 1.0 client selectors with topics. > With queues all works as expected. > > The issue is, that when I declare JMSType to any value for message and send > this message to topic, subscribed clients > either with or without message selector do not receive this message at all. This is not related to selectors at all. It is to do with the way the pre1.0 exchanges (and biding to them) is exposed over AMQP 1.0. If you create a receiving link from amq.direct without specifying an 'legacy-amqp-direct-binding' filter, the broker will bind a subscription key with the empty string as the binding key. This means it will only match messages published with no subject. However as JMSType is mapped to the subject, setting that will cause the message not to match the binding. Replacing amq.direct with amq.fanout should work.
Gordon just beat me to answering a question I hand't asked yet, I was wondering whether something like that was the issue. The mentioned test passes for me, with my topics/exchanges having been creating using the 'topic' type. Altering my setup to use direct exchanges instead, it did then fail.
(In reply to Robbie Gemmell from comment #7) > The mentioned test passes for me, with my topics/exchanges having been > creating using the 'topic' type. Altering my setup to use direct exchanges > instead, it did then fail. Yes, as the topic exchange supports wildcards, the default binding used where none is explicitly specified is #, meaning that it will match any message sent to the exchange. With the fanout exchange type there is no key associated with the binding so again all messages match. The problem is with the direct exchange, whose behaviour does not support a means of binding a queue to receive all messages.
Thank you guys for your quick responses and great answer. This "news" could help our TCK tests as well :) Moving back to ON_QA status.
When sending a message using qpid-jms-client to MRGM broker and receiving this message with any but qpid-jms-client, the message id is not propagated. On the other hand, when receiving the very same message using qpid-jms-client the correct message id is there. Steps to reproduce: 1) Send a message with qpid-jms-client to the broker and note the message id /var/dtests/node_data/clients/java/aac/aac1_sender.java.sh --log-msgs dict -a "myqueue1" {'redelivered': False, 'reply_to': None, 'id': 'dhcp-75-171.lab.eng.brq.redhat.com-49424-1443102234716-0:1:1:1-1', 'user_id':None, 'correlation_id': None, 'priority': 4, 'durable': True, 'ttl': 0, 'type': None, 'expiration': 0, 'timestamp': 1443102235653, 'destination': 'myqueue1', 'properties': {'JMSXDeliveryCount': 1}, 'content': None} 2) Receive the message (use browsing mode) with qpid-jms-client /var/dtests/node_data/clients/java/aac/aac1_receiver.java.sh --log-msgs dict --recv-browse true -a "myqueue1" {'redelivered': False, 'reply_to': None, 'id': 'dhcp-75-171.lab.eng.brq.redhat.com-49424-1443102234716-0:1:1:1-1', 'correlation_id': None, 'priority': 4, 'durable': True, 'type': None, 'expiration': 0, 'timestamp': 1443102235653, 'properties': {'JMSXDeliveryCount': 1}, 'content': } 3) Receive the message (using browse mode) with non qpid-jms-client. Message id is not there. /var/dtests/node_data/clients/java/qpid/qc2_drain.java.sh -b localhost:5672 --log-msgs dict --connection-options "{sasl_mechanisms:ANONYMOUS }" "myqueue1;{mode:browse}" {'redelivered': False, 'reply_to': None, 'subject': None, 'id': None, 'correlation_id': None, 'priority': 4, 'durable': True, 'type': None, 'expiration': 0, 'timestamp': 0, 'properties': {}, 'content': } /var/dtests/node_data/clients/qc2_drain --log-msgs dict "myqueue1;{mode:browse}" {'redelivered': False, 'reply_to': None, 'subject': None, 'content_type': None, 'id': None, 'user_id': None, 'correlation_id': None, 'priority': 4, 'durable': True, 'ttl': 0.000000e+00, 'size': None, 'properties': {}, 'content': None} So this seems to me like a broker issue. Could you please have a look at it Gordon? Thank you.
The same is going on with timestamp. Example is show in previous comment.
(In reply to Michal Toth from comment #10) > When sending a message using qpid-jms-client to MRGM broker and receiving > this message with any but qpid-jms-client, > the message id is not propagated. > On the other hand, when receiving the very same message using > qpid-jms-client the correct message id is there. [...] > So this seems to me like a broker issue. Can you attach a broker protocol trace? The property is clearly not 'lost' as the JMS client is able to see it. The broker doesn't actually alter the bare message at all, so my initial suspicion is that it is a client side or interop issue. I'd also suggest this may be worth creating a separate BZ for, as it seems quite distinct from the question of selectors.
Robbie suggests that the other clients may be using AMQP 0-10? If that was the case then it is expected that the message id would be null, as this is constrained to be a UUID in 0-10 and a random string can't be converted into a uuid, so the conversion to an 0-10 message would just drop that property.
I think there is an issue with JMSTimestamp, where we want to check for message sent exactly at some time. For example "JMSTimestamp = 1443779787927" For information please have a look at the attached file "selector_jmstimestamp". Broker trace+ Selector logs are there as well.
Created attachment 1079405 [details] Selector JMSTimestamp broker log and used commands
I have created a new BZ for this. Please see Bug 1268257.
I would like to add, that expiration time is affected as well. (Seems like a different issue to Bug 1268257) Send a message org.apache.qpid.example.qc3_spout --log-msgs dict --broker guest/guest.75.219:5672 --connection-options "{ sasl_mechanisms : 'PLAIN', protocol : 'amqp1.0' }" --count 1 --content "KEKcL0zHkG" --ttl 100000 "test_jms_header_jmsexpiration" {'redelivered': False, 'reply_to': None, 'id': None, 'correlation_id': None, 'priority': 4, 'durable': True, 'destination': 'test_jms_header_jmsexpiration', 'type': None, 'expiration': 1444027015850, 'timestamp': 1444026915850, 'properties': {'JMSXDeliveryCount': 1, 'spout_id': 'e26a424b-f07b-4487-a5a7-754297451159:0'}, 'content': 'KEKcL0zHkG'} Browse the queue (8:40:32 in logs) $ org.apache.qpid.example.qc3_drain --timeout 2 --log-msgs dict --broker guest/guest.75.219:5672 --connection-options "{ sasl_mechanisms : 'PLAIN', protocol : 'amqp1.0' }" --msg-selector "JMSExpiration>1444026908172" --recv-browse true 'test_jms_header_jmsexpiration' {'redelivered': False, 'reply_to': None, 'id': None, 'correlation_id': None, 'priority': 4, 'durable': True, 'destination': 'test_jms_header_jmsexpiration', 'type': None, 'expiration': 1444027015850, 'timestamp': 1444026915850, 'properties': {'JMSXDeliveryCount': 1, 'spout_id': 'e26a424b-f07b-4487-a5a7-754297451159:0'}, 'content': 'KEKcL0zHkG'} Obtain the message (did not work) $ org.apache.qpid.example.qc3_drain --timeout 2 --log-msgs dict --broker guest/guest.75.219:5672 --connection-options "{ sasl_mechanisms : 'PLAIN', protocol : 'amqp1.0' }" --msg-selector "JMSExpiration=1444027015850" --recv-browse true 'test_jms_header_jmsexpiration' {} Here are the selector logs. 2015-10-05 08:40:32 [Broker] debug Selector parsed[JMSExpiration>1444026908172] into: (I:JMSExpiration>EXACT:1444026908172) 2015-10-05 08:40:32 [Broker] debug Selector lookup JMS identifier: JMSExpiration treated as alias for absolute_expiry_time 2015-10-05 08:40:32 [Broker] debug Selector identifier: JMSExpiration->EXACT:1444027316283 2015-10-05 08:40:50 [Broker] debug Selector parsed[JMSExpiration=1444027015850] into: (I:JMSExpiration=EXACT:1444027015850) 2015-10-05 08:40:50 [Broker] debug Selector lookup JMS identifier: JMSExpiration treated as alias for absolute_expiry_time 2015-10-05 08:40:50 [Broker] debug Selector identifier: JMSExpiration->EXACT:1444027316283 Also comparing on <,> worked (but it seems like the ">" is treated like ">=". {'redelivered': False, 'reply_to': None, 'id': None, 'correlation_id': None, 'priority': 4, 'durable': True, 'destination': 'test_jms_header_jmsexpiration', 'type': None, 'expiration': 1444027539151, 'timestamp': 1444027439151, 'properties': {'JMSXDeliveryCount': 1, 'spout_id': '8cc57602-20c6-4c74-a695-d72873ee5ce3:0'}, 'content': 'older msg'} {'redelivered': False, 'reply_to': None, 'id': None, 'correlation_id': None, 'priority': 4, 'durable': True, 'destination': 'test_jms_header_jmsexpiration', 'type': None, 'expiration': 1444027963737, 'timestamp': 1444027463737, 'properties': {'JMSXDeliveryCount': 1, 'spout_id': '6da2254e-5fa9-489d-a190-05f0d5df8805:0'}, 'content': 'shorter ttl'} {'redelivered': False, 'reply_to': None, 'id': None, 'correlation_id': None, 'priority': 4, 'durable': True, 'destination': 'test_jms_header_jmsexpiration', 'type': None, 'expiration': 1444029452765, 'timestamp': 1444027452765, 'properties': {'JMSXDeliveryCount': 1, 'spout_id': '69acbdb3-37a5-4722-ae29-f98c975366e8:0'}, 'content': 'longer ttl'} org.apache.qpid.example.qc3_drain --timeout 2 --log-msgs dict --broker guest/guest.75.219:5672 --connection-options "{ sasl_mechanisms : 'PLAIN', protocol : 'amqp1.0' }" --msg-selector "JMSExpiration > 1444029452765" --recv-browse true 'test_jms_header_jmsexpiration' {'redelivered': False, 'reply_to': None, 'id': None, 'correlation_id': None, 'priority': 4, 'durable': True, 'destination': 'test_jms_header_jmsexpiration', 'type': None, 'expiration': 1444029452765, 'timestamp': 1444027452765, 'properties': {'JMSXDeliveryCount': 1, 'spout_id': '69acbdb3-37a5-4722-ae29-f98c975366e8:0'}, 'content': 'longer ttl'} Not that it is a big issue, as JMSExpiration works, but it worths to be mentioned. I would give it same priority as Bug 1268257.
Above issues were decoupled into following bugs: Bug 1268257 - [Selectors] JMSTimestamp seems to be compared on seconds not milliseconds when messages used AMQP 1.0 protocol Bug 1268872 - [Selectors] JMSExpiration can't be queried on exact values properly Bug 1267268 - Qpid-jms / qpid-java interoperability issue with JMSTimestamp, JMSExpiration headers when receiving messages Bug 1268878 - [Selectors] JMSMessageID interoperability issue between AMQP 1.0 and 0.10 protocol As we track the defects, I propose to move this BZ to verified if all parties agree.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHEA-2015-1879.html