Bug 1032436 - Unable to list and add env variable any more after user try to add env variable with an invalid UTF-8 value by RESTAPI for app
Unable to list and add env variable any more after user try to add env variab...
Status: CLOSED CURRENTRELEASE
Product: OpenShift Online
Classification: Red Hat
Component: Master (Show other bugs)
2.x
Unspecified Unspecified
medium Severity medium
: ---
: ---
Assigned To: Lili Nader
libra bugs
: UpcomingRelease
Depends On:
Blocks: 1034647
  Show dependency treegraph
 
Reported: 2013-11-20 03:21 EST by Nan Wei
Modified: 2015-05-14 20:56 EDT (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1034647 (view as bug list)
Environment:
Last Closed: 2014-01-23 22:33:51 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Nan Wei 2013-11-20 03:21:33 EST
Description of problem:
User try to add env variable with an invalid UTF-8 value like "bar%B3" by RESTAPI, restapi retrun success, then try to list and add env var, it throws some error messages as follows: "Unable to complete the requested operation due to: invalid byte sequence in UTF-8". 
Version-Release number of selected component (if applicable):
devenv_4051
How reproducible:
100%
Steps to Reproduce:
1. The second time to add new env variable and its value is special characters by rest api
curl -k -H 'Accept: application/xml' --user nwei@redhat.com https://ec2-54-227-44-71.compute-1.amazonaws.com/broker/rest/domain/nweid/application/ptest/environment-variables -X POST -d name=foo1 -d value='bar%B3'
Enter host password for user 'nwei@redhat.com':
<message>
      <severity>info</severity>
      <text>Added environment variable 'foo1' to application ptest</text>
      <exit-code>0</exit-code>
      <field nil="true"></field>
      <index nil="true"></index>
    </message>
2. Check env variable output
[weinan@dhcp-65-25 test]$ rhc env list -a ptest
Unable to complete the requested operation due to: invalid byte sequence in
UTF-8.
Reference ID: 5cd86030999c4664a7b288302f65ad18
3.  Add one valid env variable by rhc
[weinan@dhcp-65-25 test]$ rhc env set name=value -a ptest 
Setting environment variable(s) ... 
Unable to complete the requested operation due to: invalid byte sequence in
UTF-8.
Reference ID: 8cc1d531ed9f6459fc3df38264243437
Actual results:
It throws some error message: "Unable to complete the requested operation due to: invalid byte sequence in UTF-8.
Reference ID: 2183148205576bb4af231690e8909bed" all the time.
Expected results:
User should not be allowed to add an invalid UTF value for a user env var, and it should not break the following list and add env var operation.
Additional info:
Comment 1 Lili Nader 2013-11-21 19:55:39 EST
Here's the findings from the debugging

1. curl encodes the data (-d or --data) as ascii.  The ascii encoded version of "bar%B3" is "bar\xB3". FYI, it works fine if  --data-urlencode is used instead.

2. rails expects everything to be encoded in UTF-8 and "bar\xB3" contains an invalid UTF-8 byte sequence.

3. The value is passed on to node without ever calling any methods on it so rails/ruby does not barf until user tries to read back the value from node at which point "invalid byte sequence in UTF-8" is raised.

4. This issue it not unique to env vars.  All other APIs have the same issue although the exception is raised earlier it the process.

Finally the fix.  Check all inputs for valid UTF-8 encoding.  See commit 

https://github.com/lnader/origin-server/commit/845ce9eef98d7313602a50e1a40ecbd14a86906b

in pull request

https://github.com/openshift/origin-server/pull/4213
Comment 2 Lili Nader 2013-11-26 17:46:43 EST
new pull request with additional checking for ruby 1.8.7 

https://github.com/openshift/origin-server/pull/4239
Comment 4 Nan Wei 2013-11-28 00:33:11 EST
Version-Release number of selected component (if applicable):
devenv_4080

[weinan@dhcp-65-25 Downloads]$ curl -k -H 'Accept: application/xml' --user "nwei@redhat.com:redhat" https://ec2-54-211-32-152.compute-1.amazonaws.com/broker/rest/domain/nweid/application/ptest/environment-variables -X POST -d name=foo1 -d value='bar%BBB3'
<message>
      <severity>error</severity>
      <text>Only valid UTF-8 encoded inputs are accepted</text>
      <exit-code nil="true"></exit-code>
      <field nil="true"></field>
      <index nil="true"></index>
</message>
[weinan@dhcp-65-25 Downloads]$ rhc env list ptest
[weinan@dhcp-65-25 Downloads]$ curl -k -H 'Accept: application/xml' --user nwei@redhat.com https://ec2-54-211-32-152.compute-1.amazonaws.com/broker/rest/domain/nweid/application/ptest/environment-variables -X POST -d name=foo1 -d value='bar%3bbb'
<message>
      <severity>info</severity>
      <text>Added environment variable 'foo1' to application ptest</text>
      <exit-code>0</exit-code>
      <field nil="true"></field>
      <index nil="true"></index>
</message>

Note You need to log in before you can comment on or make changes to this bug.