Bug 504789 - mbrtowc does not always reset its state
mbrtowc does not always reset its state
Status: CLOSED NOTABUG
Product: Fedora
Classification: Fedora
Component: glibc (Show other bugs)
14
All Linux
medium Severity medium
: ---
: ---
Assigned To: Jeff Law
Fedora Extras Quality Assurance
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2009-06-09 09:27 EDT by Göran Uddeborg
Modified: 2016-11-24 10:54 EST (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2012-01-26 11:37:52 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Test program triggering the problem. (671 bytes, text/plain)
2009-06-09 09:27 EDT, Göran Uddeborg
no flags Details

  None (edit)
Description Göran Uddeborg 2009-06-09 09:27:57 EDT
Created attachment 347022 [details]
Test program triggering the problem.

Description of problem:
The way to reset the internal state of mbrtowc is to convert a null character.  (At least, that is how I read the manual.)  But that does not work in all cases.  I attach a test case illustrating one case where it fails.

Version-Release number of selected component (if applicable):
glibc-2.9-3.x86_64

How reproducible:
Every time

Steps to Reproduce:
1. cc -Wall c.c -o c
2. ./c

Actual results:
Expected  1, got  1
Expected -2, got -2
Expected -1, got -1
Expected  0, got -1
Expected  1, got -1

Expected results:
Expected  1, got  1
Expected -2, got -2
Expected -1, got -1
Expected  0, got  0
Expected  1, got  1

Additional information:
In some other cases it does work as expected.  If, for example, one removes the call to mbrtowc with an incomplete byte seqence, the one that returns -2, then the reset works as expected.
Comment 1 Ulrich Drepper 2009-06-10 15:12:18 EDT
(In reply to comment #0)
> Description of problem:
> The way to reset the internal state of mbrtowc is to convert a null character. 
> (At least, that is how I read the manual.)

No, that's not correct.  The state is reset iff the returned wide character is L'\0'.  There is no input you can use to force mbrtowc() to move the internal state into the initial state.

I haven't looked yet in detail what is going on here but a strict reading of the standard allows the current implementation.  You shouldn't rely on your assumption.  Don't ever use the internal state.  Use a state object and then memset() it to zero in case you want to return the state to the initial state.

I will look at what's going on later.
Comment 2 Göran Uddeborg 2009-06-10 17:42:54 EDT
(In reply to comment #1)
> No, that's not correct.  The state is reset iff the returned wide character is
> L'\0'.

That is what I thought

  mbrtowc(0, "", 1, 0)

would do.  A null byte in a UTF-8 stream would become a null wide character, I thought.

> There is no input you can use to force mbrtowc() to move the internal
> state into the initial state.

> Don't ever use the internal state.  Use a state object and then
> memset() it to zero in case you want to return the state to the initial state.

Ok, if you say so.

(As you may have guessed, what I wanted to do in the real program was to skip over the illegal UTF-8 sequence and continue reading input after it.)
Comment 3 Ulrich Drepper 2009-06-10 21:55:00 EDT
(In reply to comment #1)
> No, that's not correct.  The state is reset iff the returned wide character is
> L'\0'.  There is no input you can use to force mbrtowc() to move the internal
> state into the initial state.

I have to qualify this.  In *general* it's not true.  For glibc it is true since all valid encodings for the locales must map all ASCII values to the wide chars with the same value.
Comment 4 Bug Zapper 2009-11-18 04:14:02 EST
This message is a reminder that Fedora 10 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 10.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '10'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 10's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 10 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Comment 5 Göran Uddeborg 2009-11-21 12:11:38 EST
The behaviour is not changed in Fedora 12.

It may still be invalid.  Comment 1 indicates that what I thought the documentation said might be wrong.  So it may be that this bug should be closed.  But not just because Fedora 10 is at its end of life.
Comment 6 Bug Zapper 2010-11-04 07:09:53 EDT
This message is a reminder that Fedora 12 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 12.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '12'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 12's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 12 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Comment 7 Göran Uddeborg 2010-11-04 18:15:03 EDT
The behaviour is not changed in Fedora 14.

It may still be invalid.  Comment 1 indicates that what I thought the documentation said might be wrong.  So it may be that this bug should be closed.  But not just because Fedora 12 is at its end of life.

(The bug zapper repeats itself, and so do I.:-)
Comment 8 Fedora Admin XMLRPC Client 2011-11-14 14:43:51 EST
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.
Comment 9 Jeff Law 2012-01-26 11:37:52 EST
As Uli mentions in c#1 you shouldn't be relying upon this behaviour as glibc's implementation is allowed per the spec.

Note You need to log in before you can comment on or make changes to this bug.