Created attachment 1475020 [details] /var/tmp/packstack/20180810-085725-cBTrNd/manifests/172.16.1.19_network.pp.log Description of problem: Packstack deployment fails with error: ... Preparing Puppet manifests [ DONE ] Copying Puppet modules and manifests [ DONE ] Applying 172.16.1.19_controller.pp 172.16.1.19_controller.pp: [ DONE ] Applying 172.16.1.19_network.pp 172.16.1.19_network.pp: [ ERROR ] Applying Puppet manifests [ ERROR ] ERROR : Error appeared during Puppet run: 172.16.1.19_network.pp Error: Systemd start for neutron-destroy-patch-ports failed! You will find full trace in log /var/tmp/packstack/20180810-085725-cBTrNd/manifests/172.16.1.19_network.pp.log Please check log file /var/tmp/packstack/20180810-085725-cBTrNd/openstack-setup.log for more information Additional information: * Time synchronization installation was skipped. Please note that unsynchronized time on server instances might be problem for some OpenStack components. * Warning: NetworkManager is active on 172.16.1.19. OpenStack networking currently does not work on systems that have the Network Manager service enabled. * File /root/keystonerc_admin has been created on OpenStack client host 172.16.1.19. To use the command line tools you need to source the file. * To access the OpenStack Dashboard browse to http://172.16.1.19/dashboard . Please, find your login credentials stored in the keystonerc_admin in your home directory. * Note temporary directory /var/tmp/packstack/0ebbed1dbe2e4207991645dacbcf5238 on host 172.16.1.19 was not deleted for debugging purposes. MSG: non-zero return code Error(s) in /var/tmp/packstack/20180810-085725-cBTrNd/manifests/172.16.1.19_network.pp.log: ... debug: Runing journalctl command to get logs for systemd start failure: journalctl -n 50 --since '5 minutes ago' -u neutron-destroy-patch-ports --no-pager Debug: Executing: 'journalctl -n 50 --since '5 minutes ago' -u neutron-destroy-patch-ports --no-pager' Error: Systemd start for neutron-destroy-patch-ports failed! journalctl log for neutron-destroy-patch-ports: -- Logs begin at Fri 2018-08-10 08:51:57 EDT, end at Fri 2018-08-10 09:14:56 EDT. -- systemd[1]: Starting OpenStack Neutron Destroy Patch Ports... python[5110]: /usr/lib/python2.7/site-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.21.1) or chardet (2.2.1) doesn't match a supported version! python[5110]: RequestsDependencyWarning) python[5110]: PMD: net_mlx5: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory python[5110]: PMD: net_mlx5: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx5) python[5110]: PMD: net_mlx4: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory python[5110]: PMD: net_mlx4: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx4) python[5110]: Traceback (most recent call last): python[5110]: File "/usr/lib64/python2.7/runpy.py", line 162, in _run_module_as_main python[5110]: "__main__", fname, loader, pkg_name) python[5110]: File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code python[5110]: exec code in run_globals python[5110]: File "/usr/lib/python2.7/site-packages/neutron/cmd/destroy_patch_ports.py", line 83, in <module> python[5110]: main() python[5110]: File "/usr/lib/python2.7/site-packages/neutron/cmd/destroy_patch_ports.py", line 78, in main python[5110]: port_cleaner = PatchPortCleaner(cfg.CONF) python[5110]: File "/usr/lib/python2.7/site-packages/neutron/cmd/destroy_patch_ports.py", line 44, in __init__ python[5110]: for bridge in mappings.values()] python[5110]: File "/usr/lib/python2.7/site-packages/neutron/agent/common/ovs_lib.py", line 218, in __init__ python[5110]: super(OVSBridge, self).__init__() python[5110]: File "/usr/lib/python2.7/site-packages/neutron/agent/common/ovs_lib.py", line 117, in __init__ python[5110]: self.ovsdb = ovsdb_api.from_config(self) python[5110]: File "/usr/lib/python2.7/site-packages/neutron/agent/ovsdb/api.py", line 30, in from_config python[5110]: ovsdb_api.interface_map[iface_name or cfg.CONF.OVS.ovsdb_interface]) python[5110]: File "/usr/lib/python2.7/site-packages/oslo_utils/importutils.py", line 73, in import_module python[5110]: __import__(import_str) python[5110]: File "/usr/lib/python2.7/site-packages/neutron/agent/ovsdb/impl_idl.py", line 17, in <module> python[5110]: from ovsdbapp.backend.ovs_idl import command python[5110]: File "/usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 16, in <module> python[5110]: from ovsdbapp.backend.ovs_idl import command as cmd python[5110]: File "/usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 21, in <module> python[5110]: from ovsdbapp.backend.ovs_idl import idlutils python[5110]: File "/usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 21, in <module> python[5110]: from ovs.db import idl python[5110]: File "/usr/lib64/python2.7/site-packages/ovs/db/idl.py", line 25, in <module> python[5110]: from ovs.db import custom_index python[5110]: File "/usr/lib64/python2.7/site-packages/ovs/db/custom_index.py", line 12, in <module> python[5110]: from ovs.compat import sortedcontainers python[5110]: ImportError: No module named compat systemd[1]: neutron-destroy-patch-ports.service: main process exited, code=exited, status=1/FAILURE systemd[1]: Failed to start OpenStack Neutron Destroy Patch Ports. systemd[1]: Unit neutron-destroy-patch-ports.service entered failed state. systemd[1]: neutron-destroy-patch-ports.service failed. ... Version-Release number of selected component (if applicable): 2018-08-10.2 How reproducible: 100% Steps to Reproduce: 1. Deploy OSP14 using packstack, InfraRed, topology AIO 2. Packstack deployments fails Additional info: openstack-packstack-13.0.0-0.20180731101322.73e0b07.el7ost.noarch openstack-packstack-puppet-13.0.0-0.20180731101322.73e0b07.el7ost.noarch python-neutron-13.0.0-0.20180803061300.765fd4d.el7ost.noarch openstack-neutron-openvswitch-13.0.0-0.20180803061300.765fd4d.el7ost.noarch python2-neutron-lib-1.18.0-0.20180725152803.67865c7.el7ost.noarch puppet-neutron-13.2.0-0.20180805213727.2a42bb2.el7ost.noarch openstack-neutron-common-13.0.0-0.20180803061300.765fd4d.el7ost.noarch openstack-neutron-13.0.0-0.20180803061300.765fd4d.el7ost.noarch python2-neutronclient-6.9.0-0.20180803115753.d090ea2.el7ost.noarch
Created attachment 1475021 [details] /var/tmp/packstack/20180810-085725-cBTrNd/openstack-setup.log
Created attachment 1475022 [details] answer file
Created attachment 1475105 [details] Add compat module to ovs package in setup.py This is happening because the python/setup.py forgot to include the compat/compat.sortedcontainers directory. I tested a quick build with the patch applied and it added the appropriate directories to the RPM. I'll go ahead and propose the patch upstream as well.
Patch proposed upstream at https://mail.openvswitch.org/pipermail/ovs-dev/2018-August/350966.html
The patch has been merged upstream in both master and branch-2.10.
openvswitch2.0 package including the package is available: https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=17734563 Please verify. fbl
After installing the fixed rpm from, doing from ovs.compat import sortedcontainers succeeds.
I can not confirm that this issue is fixed by openvswitch2.10-2.10.0-0.20180810git58a7ce6.el7fdp (it can be kept in version openvswitch2.10-2.10.0-0.20180808gitfaf64fb.el7fdp) but it is fixed by update of python-openvswitch2.10 to version python-openvswitch2.10-2.10.0-0.20180810git58a7ce6.el7fdp. I adjusted "Fixed in Version" field.
*** Bug 1615892 has been marked as a duplicate of this bug. ***
I can verify this is not issue since puddle 2018-08-17.2 where fixed package is included: packtack-aio $ rpm -qa | grep openvswitch2.10-2.10.0-0.20180810git58a7ce6.el7fd python-openvswitch2.10-2.10.0-0.20180810git58a7ce6.el7fdp.x86_64 openvswitch2.10-2.10.0-0.20180810git58a7ce6.el7fdp.x86_64
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2019:0045