Bug 1635249

Summary: Outputting UTF8 characters during undercloud install/upgrade fails
Product: Red Hat OpenStack Reporter: Ganesh Kadam <gkadam>
Component: puppet-openstacklibAssignee: Sofer Athlan-Guyot <sathlang>
Status: CLOSED ERRATA QA Contact: Arik Chernetsky <achernet>
Severity: high Docs Contact:
Priority: high    
Version: 13.0 (Queens)CC: achernet, akaris, apevec, ccamacho, dpeacock, dwojewod, ebarrera, gkadam, jdennis, jjoyce, jpichon, jpretori, jruzicka, jschluet, jslagle, justinas.balciunas, lbezdick, lhh, mbollo, mbultel, mburns, pablo.iranzo, rcernin, rhel-osp-director-maint, sathlang, slinaber, smykhail, srevivo, tvignaud
Target Milestone: zstreamKeywords: Triaged, ZStream
Target Release: 10.0 (Newton)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: puppet-openstacklib-9.5.0-4.el7ost Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1547477
: 1691289 1692257 1703078 1703097 (view as bug list) Environment:
Last Closed: 2019-04-30 16:58:51 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1547477    
Bug Blocks: 1655473, 1691289, 1692257, 1703078, 1703097    

Comment 7 mathieu bultel 2019-02-19 11:01:12 UTC
So after some discussions with Darius via email.
I had a chance to reproduce the issue.
This is present in all releases of TripleO. I reproduced it even on master.

I found several attempts in instack-undercloud or other projects to fix this kind of race.

When I was debugging it appear that the execution of the openstack command via Puppet doesn't take in account the Python io encoding environment variable.
Even if this env variable has been set on the system or in the puppet manifest (puppet-stack-config.pp for instance).

So every openstack command thrown by the puppet-openstacklib provider needs to have the python io encoding forced to utf-8.
If not, the python code return an exception due to non-ascii characters. ('ascii' codec can't encode character u'\xe9' in position 1: ordinal not in range(128))

Comment 8 John Dennis 2019-02-19 14:29:14 UTC
I don't think you want to be trying to set the Python io encoding environment variable. Python's encoding should be correctly set when the Python interpreter initializes based on the locale.
I pointed this out in the original bug in this comment https://bugzilla.redhat.com/show_bug.cgi?id=1547477#c4 and Julie's follow-up comment seems to suggest the locale is not being initialized correctly somewhere. The locale is a shell property, it is not directly tied to Python (other than the fact Python queries the execution environment to determine what it is).

I suspect what is going on is that when Puppet or any other configuration tool executes a shell on the target it is not setting the locale correctly. In fact a very common mistake is to the the locale to C which will produce these types of problems. My suggestion would be focus on the locale setting rather than on trying to fudge things with Python specific environment variables so you get to the actual root of the problem.

Comment 11 Sofer Athlan-Guyot 2019-03-05 12:40:47 UTC
Hi,

so the bottom of this problem is that, by default, puppet reset the
locale to C when executing commands[1].

We have to pass override_locale false in order to prevent that behavior.

The review there[2] does it.  I'm waiting for a little bit of feedback
before updating the attached review on the bz, but I think this is the
proper solution.

To ease debugging/validation here's how to test it:

ssh undercloud

. stackrc

openstack project create --description "A tést" tést --domain default 

cat > test.pp<<EOP
  keystone_tenant { 'test1':
  ensure => present
} 
EOP

sudo puppet apply --noop --debug test.pp

[1]  https://github.com/puppetlabs/puppet/blob/master/lib/puppet/util/execution.rb#L349-L357
[2] https://review.openstack.org/#/c/640997/

Comment 18 mbollo 2019-04-04 15:55:09 UTC
The package is ready.

Comment 19 Lukas Bezdicka 2019-04-08 11:33:17 UTC
*** Bug 1547477 has been marked as a duplicate of this bug. ***

Comment 26 errata-xmlrpc 2019-04-30 16:58:51 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:0921

Comment 27 Lukas Bezdicka 2019-05-03 09:15:45 UTC
*** Bug 1672300 has been marked as a duplicate of this bug. ***