Bug 1476385

Problem originally reported by mmirecki.

There was a typo in reproduction step 3, should be:

   Execute "PYTHONPATH=$(pwd)/../python python2 pyssh"

Updated reproducer (works with python3):

#coding: utf-8
import time, six, sys
import ovs.db.idl

def get_switches(idl):
    rows = idl.tables['Logical_Switch'].rows
    for row in six.itervalues(rows):
        print(u'ROW: uuid:' + six.text_type(row.uuid) + u'  name:' + row.name)


schema_helper = ovs.db.idl.SchemaHelper('../ovn/ovn-nb.ovsschema')
schema_helper.register_columns('Logical_Switch', ['name', 'ports', 'other_config', 'external_ids'])

ovs.stream.Stream.ssl_set_private_key_file('sandbox/ovnnb-privkey.pem')
ovs.stream.Stream.ssl_set_certificate_file('sandbox/ovnnb-cert.pem')
ovs.stream.Stream.ssl_set_ca_cert_file('sandbox/pki/switchca/cacert.pem')

idl = ovs.db.idl.Idl('ssl:127.0.0.1:6641', schema_helper)

while True:
    idl.run()
    if idl.has_ever_connected():
        print('Connected!')
        break

txn = ovs.db.idl.Transaction(idl)

row = txn.insert(idl.tables['Logical_Switch'])
#switchname = 'שלום'
switchname = 'רשת חיצונית'
row.__setattr__('name', switchname + "foo")
status = txn.commit_block()

get_switches(idl)

Upstream patch posted:

https://mail.openvswitch.org/pipermail/ovs-dev/2017-August/337039.html

I see that commit e7164d96bcbcf790 is merged upstream. Lance, can you now clone this bug to 7.4.z and backport it to ovs-2.7 ?

(In reply to Dan Kenigsberg from comment #5)
> I see that commit e7164d96bcbcf790 is merged upstream. Lance, can you now
> clone this bug to 7.4.z and backport it to ovs-2.7 ?

This would be delivered via the fast datapath stream, not the RHEL7 z-stream.
We should be able to get it in the next fdprod (2.7) release.

The port migration feature has also been committed upstream, in master
and 2.8 branches, should be able to backport that as well. We'd like to
deliver what you need in as few fdprod releases as possible, perhaps it
would be good if you could make sure Anita has the list for planning (we
are discussing a 2.8 fdprod package as well).

Anita, can we have this in 2.7 fdprod ?

The patch for this issue is in upstream 2.7 master, will be included in 2.7.3
when it is tagged and released.

commit 91a322a3ac91c4e47afd6f9a92eedeeb1a61d205
Author: Lance Richardson <lrichard>
Date:   Wed Aug 9 15:38:43 2017 -0400

    python: make python idl unicode-tolerant
    
    Ensure that JSON is utf-8 encoded and that bytes sent/received on
    the stream sockets are in utf-8 form. Add a test case to verify
    that unicode data can be sent/received successfully using Python
    IDL module.
    
    Co-authored-by: Terry Wilson <twilson>
    Signed-off-by: Terry Wilson <twilson>
    Signed-off-by: Lance Richardson <lrichard>
    Signed-off-by: Russell Bryant <russell>

I believe Meny's team can help testing this bug - in the RHV use case - but I would let him answer.

Mark,

Could you please provide a small reproducer for QE to validate our package?
Thanks,
fbl

Sure thing, I'll get it up here as soon as I can.

Flavio et al,

Lance already attached a reproducer script on this issue. Is there something wrong with it?

https://bugzilla.redhat.com/attachment.cgi?id=1306009

(In reply to Mark Michelson from comment #18)
> Flavio et al,
> 
> Lance already attached a reproducer script on this issue. Is there something
> wrong with it?
> 
> https://bugzilla.redhat.com/attachment.cgi?id=1306009

Hi Mark,

The reproducer is based on upstream git repository and the script https://bugzilla.redhat.com/attachment.cgi?id=1306009 is based on sandbox. We're now verify the bug is fixed in redhah rpm package after openvswitch-2.7.2-9.git20170914. So can you please give a procedure based on a machine with redhat rpm installed?

Thanks
Qijun

Thanks for the explanation. I will attempt to adapt Lance's reproducer patch to work in the scenario you mentioned.

Created attachment 1330717 [details]
Reproducer python script

I am uploading reproducer.py, which will reproduce the issue.

Steps to reproduce:
1) Install OVN RPMs
2) Start the OVN NorthD service with systemd

# sudo systemctl enable ovn-northd
# sudo systemctl start ovn-northd

3) Initialize the OVN northbound database

# sudo ovn-nbctl init

4) Run the reproducer.py script

# sudo reproducer.py


On a system that has the bug, you should see something like:
Connected!
Traceback (most recent call last):
  File "reproducer.py", line 29, in <module>
    status = txn.commit_block()
  File "/usr/lib/python2.7/site-packages/ovs/db/idl.py", line 1396, in commit_block
    status = self.commit()
  File "/usr/lib/python2.7/site-packages/ovs/db/idl.py", line 1379, in commit
    if not self.idl._session.send(msg):
  File "/usr/lib/python2.7/site-packages/ovs/jsonrpc.py", line 533, in send
    return self.rpc.send(msg)
  File "/usr/lib/python2.7/site-packages/ovs/jsonrpc.py", line 242, in send
    self.output += ovs.json.to_string(msg.to_json())
  File "/usr/lib/python2.7/site-packages/ovs/json.py", line 47, in to_string
    sort_keys=sort_keys)
  File "/usr/lib64/python2.7/json/__init__.py", line 250, in dumps
    sort_keys=sort_keys, **kw).encode(obj)
  File "/usr/lib64/python2.7/json/encoder.py", line 210, in encode
    return ''.join(chunks)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xd7 in position 1: ordinal not in range(128)


On a build that does not have the issue, you should see something like:
Connected!
ROW: uuid:a725f492-a5a4-46d5-89ac-1ed61160915e  name:שלוםfoo

Run with steps in comment#21
Reproduced with python-openvswitch-2.7.2-8.git20170719.el7fdp.noarch
Verified with python-openvswitch-2.7.2-10.git20170914.el7fdp.noarch
network-qe has no regression test cases for python-openvswitch.

Closing resolved bugs.