Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1159252

Summary: certify-network fails with `is not JSON serializable`
Product: [Retired] Red Hat OpenStack Certification Program Reporter: Lee Yarwood <lyarwood>
Component: rhosctsAssignee: Neeti Dahiya <rhoscert-dev>
Status: CLOSED EOL QA Contact:
Severity: high Docs Contact:
Priority: high    
Version: 5.0CC: jtrowbri, lyarwood, maniram.sekaran, ndahiya, sankarshan, sguha, skulkarn
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-07-15 12:09:52 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
CTS logs
none
service logs
none
content of /etc/rhoscts
none
logs for rhoscts --sysinfo
none
logs for rhoscts --certify-network none

Description Lee Yarwood 2014-10-31 10:00:38 UTC
Description of problem:

# rhoscts --certify-network
Please enter the partner name: xxxxx
Please enter the driver name: xxxx_xxxx    
Creating Network
Updating Network
Updating Network
Updating Network
Listing networks
Creating Network
Updating Network
Updating Network
Listing networks
Creating bulk networks
Creating bulk networks
Creating Subnet
Creating Subnet
Creating Subnet
Creating Subnet
Creating bulk subnets
Creating bulk subnets
Creating bulk subnets
Creating bulk subnets
Creating subnet with allocation pools
Creating Subnet
Creating Subnet
Creating subnet with allocation pools
Creating subnet with allocation pools
Creating Subnet
Creating Subnet
Creating subnet with allocation pools
Creating Port
Creating Port
Creating port
Creating Port
Creating Port
Creating port
Creating bulk ports
Creating bulk ports
Listing ports
Listing ports
Updating port
Updating port
Creating security group
Listing security groups
Deleting security group
Creating security group
Creating security group rule
Listing Security group rules
Deleting security group rule
Deleting security group
Creating local provider network
Creating flat provider network
Creating vlan provider network
Creating provider network
Creating gre provider network
Traceback (most recent call last):
  File "/usr/bin/rhoscts", line 33, in <module>
    utils.main('network')
  File "/usr/lib/python2.7/site-packages/librhos/utils.py", line 275, in main
    ctslogger.json_dump()
  File "/usr/lib/python2.7/site-packages/ctslogging/ctslogger.py", line 54, in json_dump
    json.dump(JSON_DATA, f_handle)
  File "/usr/lib64/python2.7/json/__init__.py", line 189, in dump
    for chunk in iterable:
  File "/usr/lib64/python2.7/json/encoder.py", line 434, in _iterencode
    for chunk in _iterencode_dict(o, _current_indent_level):
  File "/usr/lib64/python2.7/json/encoder.py", line 408, in _iterencode_dict
    for chunk in chunks:
  File "/usr/lib64/python2.7/json/encoder.py", line 332, in _iterencode_list
    for chunk in chunks:
  File "/usr/lib64/python2.7/json/encoder.py", line 408, in _iterencode_dict
    for chunk in chunks:
  File "/usr/lib64/python2.7/json/encoder.py", line 408, in _iterencode_dict
    for chunk in chunks:
  File "/usr/lib64/python2.7/json/encoder.py", line 442, in _iterencode
    o = _default(o)
  File "/usr/lib64/python2.7/json/encoder.py", line 184, in default
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: <Response [201]> is not JSON serializable

Version-Release number of selected component (if applicable):


How reproducible:
Always.

Steps to Reproduce:
1. rhoscts --certify-network

Actual results:
Failure.

Expected results:
Success.

Comment 3 Neeti Dahiya 2014-10-31 12:21:33 UTC
Hi Lee,

Could you please ask CISCO to send the service logs and the CTS logs generated. That might help us in understanding the issue better.

Service logs could be found at /var/log/neutron/
CTS logs could be found at /tmp/rhoscts-logs/

Neeti

Comment 4 John Trowbridge 2014-10-31 20:09:03 UTC
Created attachment 952580 [details]
CTS logs

Comment 5 John Trowbridge 2014-10-31 20:09:52 UTC
Created attachment 952581 [details]
service logs

Comment 6 Neeti Dahiya 2014-11-06 07:08:40 UTC
(In reply to John Trowbridge from comment #4)
> Created attachment 952580 [details]
> CTS logs

The CTS log tar is not complete. Few of the configuration files and the system information which is captured by CTS are missing.
I could only find the following files in CTS log tar: 
api.conf, network.json, the driver info file and the .log file of the API request and responses.

Absence of system information makes it difficult to check which version of CTS is being used. Could you please ask CISCO to re-run the command rhoscts --certify-network and in the /tmp/rhoscts-logs/ directory do a ls and check if the following files/directories are present or not:
api.conf
driverinfo.txt
hypervisor.json
network.ini
network.json
network.log
packager.json
platform.json
report.html
rpm-Va.log
selinux.json
neutron/
yum.repos.d/

Comment 7 Maniram Maplelabs 2014-11-12 16:59:51 UTC
With this issue none of the mentioned files are getting created.

Able to progress with this temporary fix we have created. Do use the fix as appropriate

in /usr/lib/python2.7/site-packages/ctslogging/ctslogger.py

#data_dump = {'API_CALL': api_call,
    #             'REQ_STRING': req_string,
    #             'RESP': {'HEADERS': resp_header,
    #                      'BODY': resp_body}
    #             }

    data_dump = {'API_CALL': api_call,
                 'REQ_STRING': req_string,
                 'RESP': {'HEADERS': {'status': re.search('[0-9]+', str(resp_header)).group()},
                          'BODY': resp_body}
                 }

This issue is straight forward reproducible.

Comment 8 Maniram Maplelabs 2014-11-12 19:47:45 UTC
To add in above comment, 

Only these files are create while running tests (before applying fix)
api.conf, 
network.json, 
driverinfo.txt
neutron_<>.log

but when Fix is added and test is run

all the files are getting created.

api.conf
driverinfo.txt
hypervisor.json
network.ini
network.json
network_<>.log
packager.json
platform.json
report.html
rpm-Va.log
selinux.json
neutron/
yum.repos.d/

Comment 9 Neeti Dahiya 2014-11-13 09:43:24 UTC
Hi, 

We need to reproduce the error on our system.
Could you please run the command "rhoscts --sysinfo" on the system and send the tar.gz generated in /tmp/rhoscts-logs/

Also, in the /etc/rhoscts/ directory there are configuration files (network.ini, api.conf etc) . Send those files with the configurations that were used to run the command "rhoscts--certify-network" (without the fix mentioned in comment #7).

Comment 10 Maniram Maplelabs 2014-11-14 09:39:07 UTC
Created attachment 957461 [details]
content of /etc/rhoscts

Comment 11 Maniram Maplelabs 2014-11-14 09:40:43 UTC
Created attachment 957462 [details]
logs for rhoscts --sysinfo

Comment 12 Maniram Maplelabs 2014-11-14 09:41:28 UTC
Created attachment 957463 [details]
logs for rhoscts --certify-network

Comment 13 Maniram Maplelabs 2015-05-07 15:15:39 UTC
Recently there were folks from Infoblox while doing L3 service plugin certification hit upon the same issue and contacted me for the fix. It got resolved after changed the code as said in this bug

It will be good if RedHat can fix this bug so that RHOS5.0 folks can use accordingly