Bug 800147

Summary:	Documentation: Need to indicate that search uses lucene syntax
Product:	Red Hat Satellite	Reporter:	Corey Welton <cwelton>
Component:	Docs User Guide	Assignee:	Shikha <snansi>
Status:	CLOSED CURRENTRELEASE	QA Contact:	ecs-bugs
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	6.0.0	CC:	achan, alyoung, bbuckingham, belong, cpelland, jskeoch, lbrindle, mmccune
Target Milestone:	Unspecified	Keywords:	Triaged
Target Release:	Unused
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:	CloudForms System Engine uses Apache Lucene as the default query syntax. This causes unexpected results if the user is not aware of Lucene syntax. Refer to the following page for a full specification on Apache Lucene Query Parser Syntax: http://lucene.apache.org/core/old_versioned_docs/versions/3_3_0/queryparsersyntax.html Use of the proper Lucene syntax helps a CloudForms System Engine perform accurate searches.	Story Points:	---
Clone Of:		Environment:
Last Closed:	2012-12-10 22:03:48 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:

Description Corey Welton 2012-03-05 20:34:08 UTC

Description of problem:
We're currently implementing elasticsearch, using lucene as the default syntax. This can lead to some unpredictable behavior if user is not aware of the proper search syntax. We should document this somewhere.

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.  Sync and promote RH Content
2.  Content Mgmt >  Promotions > Products > $rh > Packages 
3.  enter 'abrt' in the search field.
4.  modify search to read 'abrt*'  
Actual results:

* With 'abrt', some things are filtered but user still gets a boatload of apparently inaccurate results.

* With 'abrt*' -- which is the correct lucene syntax -- more appropriate results are returned.


Expected results:

Given that users might not natively use lucene syntax in their search, we should indicate to them somewhere in the user guide as to the proper use. Maybe something like


<!>Notice: 
The search mechanism used throughout CloudForms System Engine uses the lucene search syntax. Attempting to search using other methods may return unexpected results.  Please consult the following URL for more information.

http://lucene.apache.org/core/old_versioned_docs/versions/3_3_0/queryparsersyntax.html



Additional info:

Comment 1 Corey Welton 2012-03-05 20:35:58 UTC

See also: https://bugzilla.redhat.com/show_bug.cgi?id=789533#c5

Comment 2 Brad Buckingham 2012-03-05 20:42:09 UTC

One deviation that we have from Lucene syntax is that we are disabling the behavior of the '-' operator.  In Lucene, this boolean operator is used to exclude documents that contain the term after the -.  Since - is commonly used in CFSE as a separator in names (e.g. repository, package...etc), we have chosen to disable it within CFSE.  This basically means that we 'escape' it, treating it as a normal character.

Comment 3 Dan Macpherson 2012-03-12 00:49:00 UTC

    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
CloudForms System Engine uses Apache Lucene as the default query syntax. This causes unexpected results if the user is not aware of Lucene sytanx. Refer to the following page for a full specification on Apache Lucene Query Parser Syntax:

http://lucene.apache.org/core/old_versioned_docs/versions/3_3_0/queryparsersyntax.html

This does not include the use of the "-" character, which CloudForms System Engine treats as an escape character.

Use of the proper Lucene syntax helps a CloudForms System Engine perform accurate searches.

Comment 4 Dan Macpherson 2012-03-12 01:09:17 UTC

    Technical note updated. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    Diffed Contents:
@@ -2,6 +2,6 @@
 
 http://lucene.apache.org/core/old_versioned_docs/versions/3_3_0/queryparsersyntax.html
 
-This does not include the use of the "-" character, which CloudForms System Engine treats as an escape character.
+This does not include the use of the "-" operator, which CloudForms System Engine treats as a normal character.
 
 Use of the proper Lucene syntax helps a CloudForms System Engine perform accurate searches.

Comment 5 Lana Brindley 2012-03-27 23:13:30 UTC

Doc update will occur for 1.1. Release note to occur for 1.0.

LKB

Comment 6 Brad Buckingham 2012-04-09 16:48:09 UTC

(In reply to comment #2)
> One deviation that we have from Lucene syntax is that we are disabling the
> behavior of the '-' operator.  In Lucene, this boolean operator is used to
> exclude documents that contain the term after the -.  Since - is commonly used
> in CFSE as a separator in names (e.g. repository, package...etc), we have
> chosen to disable it within CFSE.  This basically means that we 'escape' it,
> treating it as a normal character.

While making updates to Katello to support consistency across search, we were able to come up with a solution that enables us to support the Lucene '-' operator for most scenarios.  As a result, the above caveat is no longer entirely true.  

The exceptions where the '-' operator is not supported is when performing a search from the System Templates UI.  In that UI, if the user has a template open, navigates to Packages, Repositories...etc and enters text, the user is essentially entering the 'name' of the entity (e.g. package) they are looking for.  Since that search supports auto-completing the name, the - is not treated as an operator in that context.

Updated the "Technical Notes" to remove the caveat on - operator.

Comment 7 Brad Buckingham 2012-04-09 16:48:10 UTC

    Technical note updated. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    Diffed Contents:
@@ -2,6 +2,4 @@
 
 http://lucene.apache.org/core/old_versioned_docs/versions/3_3_0/queryparsersyntax.html
 
-This does not include the use of the "-" operator, which CloudForms System Engine treats as a normal character.
-
 Use of the proper Lucene syntax helps a CloudForms System Engine perform accurate searches.

Comment 8 Corey Welton 2012-04-12 15:17:46 UTC

    Technical note updated. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    Diffed Contents:
@@ -1,4 +1,4 @@
-CloudForms System Engine uses Apache Lucene as the default query syntax. This causes unexpected results if the user is not aware of Lucene sytanx. Refer to the following page for a full specification on Apache Lucene Query Parser Syntax:
+CloudForms System Engine uses Apache Lucene as the default query syntax. This causes unexpected results if the user is not aware of Lucene syntax. Refer to the following page for a full specification on Apache Lucene Query Parser Syntax:
 
 http://lucene.apache.org/core/old_versioned_docs/versions/3_3_0/queryparsersyntax.html

Comment 9 Shikha 2012-05-15 07:05:06 UTC

Release note added. Link:

http://documentation-stage.bne.redhat.com/docs/en-US/CloudForms/1.0/html/Release_Notes/chap-Release_Notes-System_Engine.html#sect-Release_Notes-System_Engine-System_Engine_Considerations-known_issues_04

Regards,
Shikha

Comment 10 Lana Brindley 2012-06-12 03:20:05 UTC

Back to assigned for doc update.

LKB

Comment 11 Shikha 2012-09-19 07:30:01 UTC

Hi All,
This information has been added to the System Engine User guide as an admonition. Please review this section:
http://documentation-stage-02.lab.eng.bne.redhat.com/docs/en-US/CloudForms/1.1/html/System_Engine_User_Guide/chap-Content_Management.html#About_Content_Providers
Regards,
Shikha

Comment 12 Corey Welton 2012-10-17 08:48:08 UTC

mass-move of docs-related bugs to ecs-bugs.

If this appears to be incorrect, feel free to bounce back to katello-qa-list and/or request clarification.

Comment 15 Shikha 2012-10-31 07:55:32 UTC

This information is not suitable to be put in admonition in any one chapter. since the query is used across the book. I have added a small introductory chapter for now that tells the users about Lucene query. We will review this section in the next release and expand as required.

Link:
http://documentation-stage-02.lab.eng.bne.redhat.com/docs/en-US/CloudForms/1.1/html/System_Engine_User_Guide/chap-About_Apache_Lucene.html

Regards,
Shikha

Comment 17 Shikha 2012-11-05 05:31:56 UTC

Fixed the typo. Link:
http://documentation-stage-02.lab.eng.bne.redhat.com/docs/en-US/CloudForms/1.1/html/System_Engine_User_Guide/chap-About_Apache_Lucene.html

Regards,
Shikha

Comment 19 Lana Brindley 2012-11-19 02:31:09 UTC

This documentation has now been dropped to translation ahead of publication. For any further issues, please open a new a bug.

LKB

Comment 20 Lana Brindley 2012-12-10 22:03:48 UTC

This document is now publicly available on access.redhat.com. For any further issues, please raise a new bug.

LKB