Description of problem: optparse doesn't handle Unicode help text Version-Release number of selected component (if applicable): python-2.4.3-21.el5 How reproducible: Every time. Steps to Reproduce: 1. Create a program that uses Unicode text for the help text in OptionParser.add_option(). 2. Run the program created in step 1. 3. Boom! or 1. Unpack attached tarball with test program. 2. run 'LANG=ja_JP.UTF-8 ./test3.py --help' 3. Boom! Actual results: $ LANG=ja_JP.UTF-8 ./test3.py --help Traceback (most recent call last): File "./test3.py", line 21, in ? (options,args) = parser.parse_args() File "/usr/lib64/python2.4/optparse.py", line 1275, in parse_args stop = self._process_args(largs, rargs, values) File "/usr/lib64/python2.4/optparse.py", line 1315, in _process_args self._process_long_opt(rargs, values) File "/usr/lib64/python2.4/optparse.py", line 1390, in _process_long_opt option.process(opt, value, values, self) File "/usr/lib64/python2.4/optparse.py", line 707, in process return self.take_action( File "/usr/lib64/python2.4/optparse.py", line 728, in take_action parser.print_help() File "/usr/lib64/python2.4/optparse.py", line 1534, in print_help file.write(self.format_help()) UnicodeEncodeError: 'ascii' codec can't encode characters in position 114-168: ordinal not in range(128) Expected results: No errors. Program runs normally. Additional info: See http://bugs.python.org/issue1498146
Created attachment 308878 [details] Test case
I should have a proposed patch that fixes this shortly.
Created attachment 309010 [details] Proposed patch Make optparse handle Unicode correctly. Adapted from upstream patches to optparse.py here: http://svn.python.org/view/python/trunk/Lib/optparse.py?rev=46861&r1=46507&r2=46861 and here: http://svn.python.org/view/python/trunk/Lib/optparse.py?rev=50791&r1=46863&r2=50791
Was this intended to be filed against python-virtinst? It seems like the patch you are presenting is against python itself.
D'oh! I was working on two bugs at once and got confused. Yes, this should be against python, not python-virtinst. Sorry.
In general python just doesn't play well with unicode, IMO. The whole API pretty much guarantees tracebacks. python 2.5.1 does the same thing, we we worked around it in yum by doing: - self.optparser.print_help() + sys.stdout.write(self.optparser.format_help()) - self.optparser.print_usage() + sys.stdout.write(self.optparser.format_usage()) ...see upstream commit: a3a53f16b45e06aeaa3666b47705dc879b182724 I'm loath to change anything in python/optparse to try and fix/workaround this, because as I said, it's almost impossible to end up with something you _know_ won't traceback under all sets of input.