Bug 503805

Summary: Issues with multibyte support and the -i argument to grep
Product: Red Hat Enterprise Linux 6 Reporter: Stew Benedict <stewb>
Component: grepAssignee: Jaroslav Škarvada <jskarvad>
Status: CLOSED CURRENTRELEASE QA Contact: Michal Nowak <mnowak>
Severity: medium Docs Contact:
Priority: low    
Version: 6.0CC: jskarvad, kasal, lkundrak, mnowak, ohudlick, pbonzini
Target Milestone: rcKeywords: Reopened
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: grep-2.6.3-1.el6 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2010-11-10 20:35:10 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Stew Benedict 2009-06-02 19:39:43 UTC
Description of problem:
We're seeing LSB tests for i18n support fail on a few distributions shipping latest versions of grep. What some distributions have found is that a patch that previously was added to fix an issue needs to be backed off.

Version-Release number of selected component (if applicable):
grep-2.5.3-4.fc11.i586

How reproducible:
Very

Steps to Reproduce:
1. install lsb-test-core
2. run the tests
3.
  
Actual results:
FAILS:
/tset/LI18NUX2K.L1/utils/egrep-tp/T.egrep-tp 5 FAIL
  * Verify this utility can handle the patterns which contain multibyte
characters.

* Verify this utility can handle the files which contain multibyte patterns.

* Verify this utility can match the strings which contain multibyte characters.

* Verify this utility can match the files which contain multibyte characters.

* Verify this utility can perform pattern matching in searches without regard
to case if -i option is specified.

Can't handle a pattern "Aï½ï½ï¼¤" with -i.

/tset/LI18NUX2K.L1/utils/grep-tp/T.grep-tp 5 FAIL
  * Verify this utility can handle the patterns which contain multibyte
characters.

* Verify this utility can handle the files which contain multibyte patterns.

* Verify this utility can match the strings which contain multibyte characters.

* Verify this utility can match the files which contain multibyte characters.

* Verify this utility can perform pattern matching in searches without regard
to case if -i option is specified.

Can't handle a pattern "Aï½ï½ï¼¤" with -i.


Expected results:
tests should pass

Additional info:
test source in bzr:
http://bzr.linux-foundation.org/lsb/devel/runtime-test?cmd=content;rev=ajosey-20030319171249-0b7508fbb7dab02c;pathrevid=stewb%40linux-foundation.org-20090527132832-ut4f6nm8knk5pz69;path=modules/li18nux2k.l1/tset/LI18NUX2K.L1/utils/grep-tp/grep-tp.sh

Comment 1 Paolo Bonzini 2010-03-31 10:08:12 UTC
Standalone testcase, not requiring even grep:

echo 'abCD' | LC_ALL=LTP_1.UTF-8 sed -ne 's/AbcD//ip' 

It works with LC_ALL set to something saner, so this happens because the locale is invalid.  I don't know if there is a need for some special set up, but I'm inclined to close this bug as invalid.

Comment 2 Stew Benedict 2010-03-31 10:38:27 UTC
Yes, the test suite installs and sets up the test locale. I'm pretty sure the problem resides in the RH grep build. Several other distributions also had this issue after a somewhat recent grep uplift (6 months ago when the bug was opened), and corrected it by reworking or dropping patches. It consistently fails in my testing every week on rawhide, and not on any other distribution.

Test package:

http://ftp.linuxfoundation.org/pub/lsb/snapshots/runtime-test/lsb-test-core-4.0.102-1.i486.rpm

Comment 3 Paolo Bonzini 2010-03-31 10:46:13 UTC
Please try the latest grep-2.6.x packages from koji.

Comment 4 Stew Benedict 2010-04-01 10:18:19 UTC
Test does seem to be passing with grep-2.6.1-1.fc14

Comment 5 Stew Benedict 2010-04-26 13:54:09 UTC
Re-open against the RHEL6 beta This version has grep-2.5.3-6.1.el6.i686 and fails the test.

Comment 7 RHEL Program Management 2010-04-26 15:14:49 UTC
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux major release.  Product Management has requested further
review of this request by Red Hat Engineering, for potential inclusion in a Red
Hat Enterprise Linux Major release.  This request is not yet committed for
inclusion.

Comment 8 Jaroslav Škarvada 2010-04-26 15:19:48 UTC
Fixed in grep-2.6.3-1.el6 that is heading RHEL-6.

Comment 11 Stew Benedict 2010-09-02 11:40:27 UTC
Appears to be corrected in RHEL6-beta2

Comment 13 releng-rhel@redhat.com 2010-11-10 20:35:10 UTC
Red Hat Enterprise Linux 6.0 is now available and should resolve
the problem described in this bug report. This report is therefore being closed
with a resolution of CURRENTRELEASE. You may reopen this bug report if the
solution does not work for you.