Bug 644700

Summary: API -> configchannel.lookupFileInfo fails for files with escape characters
Product: Red Hat Satellite 5 Reporter: Marcelo Moreira de Mello <mmello>
Component: APIAssignee: Tomas Lestach <tlestach>
Status: CLOSED ERRATA QA Contact: Šimon Lukašík <slukasik>
Severity: medium Docs Contact:
Priority: medium    
Version: 530CC: clasohm, cperry, jhutar, mmello, mzazrivec, stephan.duehr, xdmoon
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: spacewalk-java-1.2.39-56 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-06-17 02:34:22 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 677501    
Attachments:
Description Flags
API Python Program
none
Test File with escape characters none

Description Marcelo Moreira de Mello 2010-10-20 00:55:14 UTC
Description of problem:
In RHN Satellite v 5.3 if we upload the configuration file containing special characters like "Esc" then "configchannel.lookupFileInfo" API fails with following exception

Traceback (most recent call last):
File "/tmp/api-test.py", line 8, in ?
result = sc.configchannel.lookupFileInfo(sk,'cfg-stage-test',['/tmp/escape.txt'])
File "/usr/lib/python2.4/xmlrpclib.py", line 1096, in __call__
return self.__send(self.__name, args)
File "/usr/lib/python2.4/xmlrpclib.py", line 1383, in __request
verbose=self.__verbose
File "/usr/lib/python2.4/xmlrpclib.py", line 1147, in request
return self._parse_response(h.getfile(), sock)
File "/usr/lib/python2.4/xmlrpclib.py", line 1281, in _parse_response
p.feed(response)
File "/usr/lib/python2.4/xmlrpclib.py", line 527, in feed
self._parser.Parse(data, 0)
xml.parsers.expat.ExpatError: not well-formed (invalid token): line 11, column 53 

Version-Release number of selected component (if applicable):
Satellite 5.3

How reproducible:
100%

Steps to Reproduce:
1. Login to satellite WebUI
2. Go to "Configuration"
3. Click on "Configuration Channels"
4. Select any configuration channel -> Click on "Add files" and upload the provided escape.txt file. (mark the type as text file)
5. Run the provided API script
  
Actual results:
python lookupFileInfo.py 
Files in the channel
--------------------
/bin/ls
Informations about /bin/ls
Type:   file 
Channel: channel-escape 
Contents: None 
Revision: 1 
Owner: root 
Perms: 644 
Binary: True 
MD5: e0070c61b4567913010f8ba10f2b75f8 


/root/binary-file
Informations about /root/binary-file
Type:   file 
Channel: channel-escape 
Contents: None 
Revision: 1 
Owner: root 
Perms: 644 
Binary: True 
MD5: 39055b44bd8597d4b3d452ab9a831ffe 


/root/escape.txt
Traceback (most recent call last):
  File "lookupFileInfo.py", line 21, in <module>
    detail_files = client.configchannel.lookupFileInfo(key,'channel-escape',[file_path])
  File "/usr/lib64/python2.7/xmlrpclib.py", line 1224, in __call__
    return self.__send(self.__name, args)
  File "/usr/lib64/python2.7/xmlrpclib.py", line 1570, in __request
    verbose=self.__verbose
  File "/usr/lib64/python2.7/xmlrpclib.py", line 1264, in request
    return self.single_request(host, handler, request_body, verbose)
  File "/usr/lib64/python2.7/xmlrpclib.py", line 1297, in single_request
    return self.parse_response(response)
  File "/usr/lib64/python2.7/xmlrpclib.py", line 1462, in parse_response
    p.feed(data)
  File "/usr/lib64/python2.7/xmlrpclib.py", line 557, in feed
    self._parser.Parse(data, 0)
xml.parsers.expat.ExpatError: not well-formed (invalid token): line 11, column 53



Expected results:
API works normally. 



Additional info:

If we set the file /root/escape.txt as binary file, the API works perfectly. 

As we can see in the RHN API Description, the API config.lookupFileInfo returns a struct that has a several informations, inclusive the content from the file. 

<snip>
Method: lookupFileInfo
Returns:

    * array:
          o struct - Configuration Revision information
                + string "type"
                      # file
                      # directory
                + string "path" - File Path
                + string "channel" - Channel Name
                + string "contents" - File contents for text files only.
                + int "revision" - File Revision
                + dateTime.iso8601 "creation" - Creation Date
                + dateTime.iso8601 "modified" - Last Modified Date
                + string "owner" - File Owner
                + string "group" - File Group
                + int "permissions" - File Permissions (Deprecated)
                + string "permissions_mode" - File Permissions
                + boolean "binary" - true/false , Present for files only.
                + string "md5" - File's md5 signature. Present for files only.
                + string "macro-start-delimiter" - Macro start delimiter for a config file. Present for files only.
                + string "macro-end-delimiter" - Macro end delimiter for a config file. Present for files only.
<snip>

IMHO, when the API is used and the file contains escape sequences, the API tries to "recognize" the file, and the API fails. 

Per example: 
          $ file escape.txt 
          escape.txt: ASCII text, with escape sequences
          $ cat escape.txt 
          set t_Sb=[4%dm
          $ less escape.txt
          set t_Sb=ESC[4%dm
          escape.txt (END) 

As we can see above, vim or less understand the ANSI Escape Characters, but the system tries to "execute" them. 

At the same time, the API works perfectly when the file is set as Binary file, but the user will not be able to edit the file using the web UI, as it used to be. 

IMHO, we have two choices: 

a) consider that, if the file has a special characters, in the satellite web UI, the file MUST be set as Binary File

b) in the Satellite API, make a test if the file as some escape characters (like file) to dont get more errors. 

Best Regards, 
Marcelo Moreira de Mello

Comment 1 Marcelo Moreira de Mello 2010-10-20 00:58:15 UTC
Created attachment 454483 [details]
API Python Program

Comment 2 Marcelo Moreira de Mello 2010-10-20 00:58:47 UTC
Created attachment 454484 [details]
Test File with escape characters

Comment 6 Tomas Lestach 2011-04-13 16:29:07 UTC
The problem is, that the escape.txt contains an invalid xml character.

Fix: A file content doesn't get returned via API, if iy contains invalid xml characters.

spacewalk.git: b84a4b59f6df2aad23f7f4daf4a3e7c6677fb9a1

Comment 9 Šimon Lukašík 2011-04-29 14:13:48 UTC
Changing to Verified:

Testing procedure:
 - automated test
   (upload and download of attached escape file through API)

Verified against:
spacewalk-java-1.2.39-61

Note:
I thought, that instead of do-not-send, it might be nicer to send
the content encoded in base54. After the offline discussion I understand
that we do not send it, because of customer automations might expect
plain-text.

Comment 10 Milan Zázrivec 2011-06-03 15:05:05 UTC
Verified in stage w/ spacewalk-java-1.2.39-82 -> release pending.

Comment 11 Clifford Perry 2011-06-17 02:34:22 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

https://rhn.redhat.com/errata/RHSA-2011-0879.html