Description of problem: Any who set web browser's lang to any other than english might not be able to access luci web interface at all because of the internal error (the error message on web page is "'charmap' codec can't encode characters ...."). Version-Release number of selected component (if applicable): luci-0.12.0-7.el5 How reproducible: always Steps to Reproduce: 1. set browser lang to any non english 2. init luci 3. open luci web page with the browser Actual results: internal error Expected results: the web page is shown correctly Additional info: I think that this is because the default encoding specified in /var/lib/luci/etc/zope.conf is not appropriate at all especially for non-english languages, e.g. Japanese. Here is a configuration change for this issue which works well as far as I tested and think better than current default even for English: --- /var/lib/luci/etc/zope.conf.org 2008-08-13 01:20:36.000000000 +0900 +++ /var/lib/luci/etc/zope.conf 2008-08-13 00:40:28.000000000 +0900 @@ -871,6 +871,7 @@ # Example: # # default_zpublisher_encoding utf-8 +default-zpublisher-encoding utf-8 # Directives: servers #
Here is the error message shown in the web. ----------------------------------------------------------------------------- Site Error An error was encountered while publishing this resource. UnicodeEncodeError 'charmap' codec can't encode characters in position 93-100: character maps to (Also, the following error occurred while attempting to render the standard error message, please see the event log for full details: 'charmap' codec can't encode characters in position 69-75: character maps to <undefined>)Traceback (innermost last): Module ZPublisher.Publish, line 196, in publish_module_standard Module Products.PlacelessTranslationService.PatchStringIO, line 34, in new_publish Module ZPublisher.Publish, line 146, in publish Module Zope2.App.startup, line 222, in zpublisher_exception_hook Module ZPublisher.Publish, line 115, in publish Module ZPublisher.mapply, line 88, in mapply Module ZPublisher.Publish, line 41, in call_object Module Products.CMFCore.FSPythonScript, line 108, in __call__ Module Shared.DC.Scripts.Bindings, line 311, in __call__ Module Shared.DC.Scripts.Bindings, line 348, in _bindAndExec Module Products.CMFCore.FSPythonScript, line 164, in _exec Module None, line 14, in require_login - - Line 14 Module Products.CMFFormController.ControllerPageTemplate, line 74, in __call__ Module Products.CMFFormController.BaseControllerPageTemplate, line 31, in _call Module Shared.DC.Scripts.Bindings, line 311, in __call__ Module Shared.DC.Scripts.Bindings, line 348, in _bindAndExec Module Products.PageTemplates.ZopePageTemplate, line 255, in _exec Module Products.PageTemplates.PageTemplate, line 104, in pt_render - Module TAL.TALInterpreter, line 238, in __call__ Module TAL.TALInterpreter, line 281, in interpret Module TAL.TALInterpreter, line 749, in do_useMacro Module TAL.TALInterpreter, line 281, in interpret Module TAL.TALInterpreter, line 457, in do_optTag_tal Module TAL.TALInterpreter, line 442, in do_optTag Module TAL.TALInterpreter, line 437, in no_tag Module TAL.TALInterpreter, line 281, in interpret Module TAL.TALInterpreter, line 749, in do_useMacro Module TAL.TALInterpreter, line 281, in interpret Module TAL.TALInterpreter, line 310, in do_startEndTag Module TAL.TALInterpreter, line 356, in do_startTag Module Products.PlacelessTranslationService.FasterStringIO, line 119, in write Module ZPublisher.HTTPResponse, line 454, in _encode_unicode Module encodings.iso8859_15, line 18, in encode UnicodeEncodeError: (see above) Troubleshooting Suggestions * The URL may be incorrect. * The parameters passed to this resource may be incorrect. * A resource that this resource relies on may be encountering an error. For more detailed information about the error, please refer to the error log. If the error persists please contact the site maintainer. Thank you for your patience. -------------------------------------------------------------------------
Here is the Zope log (/var/lib/luci/log/event.log): 2008-08-22T03:03:25 ERROR root Exception while rendering an error message Traceback (most recent call last): File "/usr/lib/luci/zope/lib/python/OFS/SimpleItem.py", line 223, in raise_standardErrorMessage v = s(**kwargs) File "/usr/lib/luci/zope/lib/python/Products/CMFCore/FSPythonScript.py", line 108, in __call__ return Script.__call__(self, *args, **kw) File "/usr/lib/luci/zope/lib/python/Shared/DC/Scripts/Bindings.py", line 311, in __call__ return self._bindAndExec(args, kw, None) File "/usr/lib/luci/zope/lib/python/Shared/DC/Scripts/Bindings.py", line 348, in _bindAndExec return self._exec(bound_data, args, kw) File "/usr/lib/luci/zope/lib/python/Products/CMFCore/FSPythonScript.py", line 164, in _exec result = f(*args, **kw) File "Script (Python)", line 25, in standard_error_message File "/usr/lib/luci/zope/lib/python/Shared/DC/Scripts/Bindings.py", line 311, in __call__ return self._bindAndExec(args, kw, None) File "/usr/lib/luci/zope/lib/python/Shared/DC/Scripts/Bindings.py", line 348, in _bindAndExec return self._exec(bound_data, args, kw) File "/usr/lib/luci/zope/lib/python/Products/CMFCore/FSPageTemplate.py", line 195, in _exec result = self.pt_render(extra_context=bound_names) File "/usr/lib/luci/zope/lib/python/Products/CMFCore/FSPageTemplate.py", line 134, in pt_render result = FSPageTemplate.inheritedAttribute('pt_render')( File "/usr/lib/luci/zope/lib/python/Products/PageTemplates/PageTemplate.py", line 104, in pt_render tal=not source, strictinsert=0)() File "/usr/lib/luci/zope/lib/python/TAL/TALInterpreter.py", line 238, in __call__ self.interpret(self.program) File "/usr/lib/luci/zope/lib/python/TAL/TALInterpreter.py", line 281, in interpret handlers[opcode](self, args) File "/usr/lib/luci/zope/lib/python/TAL/TALInterpreter.py", line 749, in do_useMacro self.interpret(macro) File "/usr/lib/luci/zope/lib/python/TAL/TALInterpreter.py", line 281, in interpret handlers[opcode](self, args) File "/usr/lib/luci/zope/lib/python/TAL/TALInterpreter.py", line 457, in do_optTag_tal self.do_optTag(stuff) File "/usr/lib/luci/zope/lib/python/TAL/TALInterpreter.py", line 442, in do_optTag return self.no_tag(start, program) File "/usr/lib/luci/zope/lib/python/TAL/TALInterpreter.py", line 437, in no_tag self.interpret(program) File "/usr/lib/luci/zope/lib/python/TAL/TALInterpreter.py", line 281, in interpret handlers[opcode](self, args) File "/usr/lib/luci/zope/lib/python/TAL/TALInterpreter.py", line 749, in do_useMacro self.interpret(macro) File "/usr/lib/luci/zope/lib/python/TAL/TALInterpreter.py", line 281, in interpret handlers[opcode](self, args) File "/usr/lib/luci/zope/lib/python/TAL/TALInterpreter.py", line 310, in do_startEndTag self.do_startTag(stuff, self.endsep, self.endlen) File "/usr/lib/luci/zope/lib/python/TAL/TALInterpreter.py", line 356, in do_startTag self._stream_write(_nulljoin(L)) File "/usr/lib/luci/zope/lib/python/Products/PlacelessTranslationService/FasterStringIO.py", line 119, in write s = response._encode_unicode(s) File "/usr/lib/luci/zope/lib/python/ZPublisher/HTTPResponse.py", line 454, in _encode_unicode return body.encode(encoding) File "/usr/lib/python2.4/encodings/iso8859_15.py", line 18, in encode return codecs.charmap_encode(input,errors,encoding_map) UnicodeEncodeError: 'charmap' codec can't encode characters in position 91-97: character maps to <undefined> ------
I found easier way to reproduce this problem. install w3m and run "LANG=ja_JP.UTF-8 w3m https://cluster-1:8084" ('cluster-1' must be replaced with a server name which runs luci). I'm trying to reproduce with wget also but haven't succeeded yet.
Here is a response header log of another target server (rhel 5.2 ga) taken with web developer extension for firefox 3. Response Headers - https://luci-server:8084/luci/acl_users/credentials_cookie_auth/require_login?came_from=https%3A//luci-server%3A8084/luci/homebase/index_html Server: Zope/(Zope 2.9.8-final, python 2.4.3, linux2) ZServer/1.1 Plone/2.5.5 Date: Sun, 17 Aug 2008 08:14:48 GMT Bobo-Exception-Line: 18 Content-Length: 3136 Bobo-Exception-Value: See the server error log for details Bobo-Exception-File: iso8859_15.py Bobo-Exception-Type: UnicodeEncodeError Content-Type: text/html; charset=iso-8859-15 500 Internal Server Error
Thanks for all the info and the patch. I'll apply it to the source tree.
thanks! Please let me know if you need further info.
No pressure, but it would be nice to have it soon because this bug may make luci unusable for anyone use languages which characters cannot be encoded in iso8859-15. Please let me know if you need further analysis or something.
verified, didn't see error message listed above
An advisory has been issued which should help the problem described in this bug report. This report is therefore being closed with a resolution of ERRATA. For more information on therefore solution and/or where to find the updated files, please follow the link below. You may reopen this bug report if the solution does not work for you. http://rhn.redhat.com/errata/RHBA-2009-0196.html