Bug 743107

Summary: msgGroups: filter can not be supplied to purge/reroute/queueMoveMessages methods
Product: Red Hat Enterprise MRG Reporter: Petr Matousek <pematous>
Component: qpid-cppAssignee: Ken Giusti <kgiusti>
Status: CLOSED CURRENTRELEASE QA Contact: MRG Quality Engineering <mrgqe-bugs>
Severity: unspecified Docs Contact:
Priority: medium    
Version: DevelopmentCC: jross
Target Milestone: 2.1.2   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
The queue "purge", "move", and "reroute" commands now take a filter argument. This filter allows the selected operation (purge/move/reroute) to be selectively applied based on the value of a given header field in the message. The filter is expressed as a 'name=value' map, with the following fields: { 'filter_type' : 'header_match_str', 'filter_params' : { 'header_key' : "<header name>", 'header_value' : "<value to match>" } } Where <header name> is the string value used as the header's key. And <value to match> is the string value of the header which, when equal to the header's value, will apply the given operation (move/reroute/purge, etc).
Story Points: ---
Clone Of: Environment:
Last Closed: Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Petr Matousek 2011-10-03 20:35:14 UTC
Description of problem:

According to https://issues.apache.org/jira/secure/attachment/12489760/UI.txt,
group filter can be supplied using qpid.group_id key in the given map to the following methods: purge, reroute, queueMoveMessages

Call of these methods with defined filter causes "Ignoring unrecognized message filter error".

Moreover the error is displayed even if empty map is supplied {} 

The operation is performed and the filter is ignored.

Version-Release number of selected component (if applicable):
qpidd (qpidc) version 0.13
snv branch QPID-3346 (revision 1177617)

How reproducible:
100%

Steps to Reproduce:
1. define queue with message group functionality enabled
2. queue some messages to that queue
3. qpid-tool: call <queue_id> purge 1 {"qpid.group_id":"A"}
2011-10-03 20:50:50 error Ignoring unrecognized message filter: '{qpid.group_id:A}'
qpid: OK (0) - {}
3. qpid-tool: call <queue_id> purge 1 {}
2011-10-03 20:50:50 error Ignoring unrecognized message filter: '{}'
qpid: OK (0) - {}

  
Actual results:
given filter is ignored

Expected results:
filter is correctly applied

Additional info:

Comment 1 Ken Giusti 2011-11-03 18:21:47 UTC
Hi Petr,

I apologize - we decided that the filter syntax as proposed in the UI.txt was too specific to the message group functionality.  A better approach would be to define a more general purpose syntax.  I neglected to update the documentation with the new syntax.

The filter syntax that was implemented uses the format:

{ 'filter_type' : 'header_match_str',
  'filter_params' : { 'header_key' : "<header name>",
                      'header_value' : "<value to match>" 
                    }
}

This allows the purge/reroute/etc methods to filter based on any message header, not just group id.  And it should be easier to extend this syntax (inverted match, regex, etc).

When you supply the filter via qpid-tool, you have to leave out spaces and quote all strings.  Here is an example showing how to purge messages based on group id:

qpid: call 116 query queue KENQ
qpid: OK (0) - {u'results': {u'qpid.message_group_queue': {u'group_state': [{u'timestamp': 0, u'msg_count': 3, u'group_id': 'P_0:0', u'consumer': ''}, {u'timestamp': 0, u'msg_count': 1, u'group_id': 'P_0:3', u'consumer': ''}, {u'timestamp': 0, u'msg_count': 3, u'group_id': 'P_0:1', u'consumer': ''}, {u'timestamp': 0, u'msg_count': 3, u'group_id': 'P_0:2', u'consumer': ''}], u'group_header_key': 'GH'}}}

# purge 1 message from group 'P_0:2'
qpid: call 126 purge 1 {"filter_type":"header_match_str","filter_params":{"header_key":"GH","header_value":"P_0:2"}}
Raw input=[['1', '{"filter_type":"header_match_str","filter_params":{"header_key":"GH","header_value":"P_0:2"}}']]
qpid: OK (0) - {}

qpid: call 116 query queue KENQ
qpid: OK (0) - {u'results': {u'qpid.message_group_queue': {u'group_state': [{u'timestamp': 0, u'msg_count': 3, u'group_id': 'P_0:0', u'consumer': ''}, {u'timestamp': 0, u'msg_count': 1, u'group_id': 'P_0:3', u'consumer': ''}, {u'timestamp': 0, u'msg_count': 3, u'group_id': 'P_0:1', u'consumer': ''}, {u'timestamp': 0, u'msg_count': 2, u'group_id': 'P_0:2', u'consumer': ''}], u'group_header_key': 'GH'}}}

# purge the remaining P_0:2 messages
qpid: call 126 purge 3 {"filter_type":"header_match_str","filter_params":{"header_key":"GH","header_value":"P_0:2"}}
qpid: OK (0) - {}

call 116 query queue KENQ
qpid: OK (0) - {u'results': {u'qpid.message_group_queue': {u'group_state': [{u'timestamp': 0, u'msg_count': 3, u'group_id': 'P_0:0', u'consumer': ''}, {u'timestamp': 0, u'msg_count': 1, u'group_id': 'P_0:3', u'consumer': ''}, {u'timestamp': 0, u'msg_count': 3, u'group_id': 'P_0:1', u'consumer': ''}], u'group_header_key': 'GH'}}}

I'll add a description of the filter syntax to the upstream documentation.

Comment 2 Petr Matousek 2012-01-17 14:39:38 UTC
This issue has been fixed.

Verified on RHEL5.7 and RHEL6.2, architectures: x86_64, i686 

packages installed:
qpid-cpp-mrg-0.14-3.el5
qpid-qmf-0.14-2.el5
qpid-cpp-0.14-1.el6
qpid-qmf-0.14-3.el6

-> VERIFIED

Comment 3 Ken Giusti 2012-03-13 17:00:41 UTC
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
The queue "purge", "move", and "reroute" commands now take a filter argument.  This filter allows the selected operation (purge/move/reroute) to be selectively applied based on the value of a given header field in the message.

The filter is expressed as a 'name=value' map, with the following fields:

{ 'filter_type' : 'header_match_str',
  'filter_params' : { 'header_key' : "<header name>",
                      'header_value' : "<value to match>" 
                    }
}

Where <header name> is the string value used as the header's key.  And <value to match> is the string value of the header which, when equal to the header's value, will apply the given operation (move/reroute/purge, etc).