Created attachment 1138919 [details] ovs firewall Description of problem: Install RDO-Mitaka (milestone 3) with packstack on CentOS. After installation changed OVS rpm to version 2.5 . after restart the services we can see that neutron-openvswitch-agent is in failed status. In log file : 2016-03-21 13:59:03.517 4548 ERROR neutron Traceback (most recent call last): 2016-03-21 13:59:03.517 4548 ERROR neutron File "/usr/bin/neutron-rootwrap-daemon", line 10, in <module> 2016-03-21 13:59:03.517 4548 ERROR neutron sys.exit(daemon()) 2016-03-21 13:59:03.517 4548 ERROR neutron File "/usr/lib/python2.7/site-packages/oslo_rootwrap/cmd.py", line 57, in daemon 2016-03-21 13:59:03.517 4548 ERROR neutron return main(run_daemon=True) 2016-03-21 13:59:03.517 4548 ERROR neutron File "/usr/lib/python2.7/site-packages/oslo_rootwrap/cmd.py", line 98, in main 2016-03-21 13:59:03.517 4548 ERROR neutron daemon_mod.daemon_start(config, filters) 2016-03-21 13:59:03.517 4548 ERROR neutron File "/usr/lib/python2.7/site-packages/oslo_rootwrap/daemon.py", line 98, in daemon_start 2016-03-21 13:59:03.517 4548 ERROR neutron server = manager.get_server() 2016-03-21 13:59:03.517 4548 ERROR neutron File "/usr/lib64/python2.7/multiprocessing/managers.py", line 493, in get_server 2016-03-21 13:59:03.517 4548 ERROR neutron self._authkey, self._serializer) 2016-03-21 13:59:03.517 4548 ERROR neutron File "/usr/lib64/python2.7/multiprocessing/managers.py", line 162, in __init__ 2016-03-21 13:59:03.517 4548 ERROR neutron self.listener = Listener(address=address, backlog=16) 2016-03-21 13:59:03.517 4548 ERROR neutron File "/usr/lib/python2.7/site-packages/oslo_rootwrap/jsonrpc.py", line 66, in __init__ 2016-03-21 13:59:03.517 4548 ERROR neutron self._socket.bind(address) 2016-03-21 13:59:03.517 4548 ERROR neutron File "/usr/lib64/python2.7/socket.py", line 224, in meth 2016-03-21 13:59:03.517 4548 ERROR neutron return getattr(self._sock,name)(*args) 2016-03-21 13:59:03.517 4548 ERROR neutron socket.error: [Errno 13] Permission denied Version-Release number of selected component (if applicable): [root@puma15 ~(keystone_admin)]# uname -a Linux puma15.scl.lab.tlv.redhat.com 4.5.0-1.el7.elrepo.x86_64 #1 SMP Mon Mar 14 10:24:58 EDT 2016 x86_64 x86_64 x86_64 GNU/Linux [root@puma15 ~(keystone_admin)]# rpm -qa |grep openvswitch python-openvswitch-2.5.0-2.el7.noarch openstack-neutron-openvswitch-8.0.0.0b4-0.20160304174813.0ae20a3.el7.centos.noarch openvswitch-2.5.0-2.el7.x86_64 [root@puma15 ~(keystone_admin)]# rpm -qa |grep neutron openstack-neutron-8.0.0.0b4-0.20160304174813.0ae20a3.el7.centos.noarch openstack-neutron-ml2-8.0.0.0b4-0.20160304174813.0ae20a3.el7.centos.noarch python-neutron-8.0.0.0b4-0.20160304174813.0ae20a3.el7.centos.noarch openstack-neutron-openvswitch-8.0.0.0b4-0.20160304174813.0ae20a3.el7.centos.noarch openstack-neutron-common-8.0.0.0b4-0.20160304174813.0ae20a3.el7.centos.noarch python-neutronclient-4.1.2-0.20160304195803.5d28651.el7.centos.noarch python-neutron-lib-0.0.3-0.20160227020344.999828a.el7.centos.noarch openstack-neutron-metering-agent-8.0.0.0b4-0.20160304174813.0ae20a3.el7.centos.noarch How reproducible: always Steps to Reproduce: 1.Install RDO-Mitaka with packstack in CentOS 2. rpm -e --nodeps openvswitch-2.4.0-1.el7.x86_64 3. rpm -e --nodeps python-openvswitch-2.4.0-1.el7.noarch 4. rpm -ivh http://cbs.centos.org/kojifiles/packages/openvswitch/2.5.0/2.el7/noarch/python-openvswitch-2.5.0-2.el7.noarch.rpm 5. rpm -ivh http://cbs.centos.org/kojifiles/packages/openvswitch/2.5.0/2.el7/x86_64/openvswitch-2.5.0-2.el7.x86_64.rpm 6. reboot host 7. openstack-status --- neutron ovs agent is down Actual results: neutron ovs agent is down Expected results: all services in active no errors Additional info: change Selinux to disable fixed the issue
Can you please switch to permissive mode, kill rootwrap daemon and start ovs agent again? This way we'll collect all denials that rootwrap daemon produces.
I get some more avc in audit log : type=AVC msg=audit(1458637620.131:3164): avc: denied { create } for pid=6988 comm="neutron-rootwra" name="rootwrap.sock" scontext=system_u:system_r:neutr85 comm="neutron-rootwra" name="rootwrap.sock" scontext=system_u:system_r:neutron_t:s0 tcontext=system_u:object_r:neutron_tmp_t:s0 tclass=sock_file permissive=0 type=AVC msg=audit(1458637776.467:3218): avc: denied { create } for pid=7r:neutron_t:s0 tcontext=system_u:object_r:neutron_tmp_t:s0 tclass=sock_file permissive=0 type=AVC msg=audit(1458637856.473:3249): avc: denied { create } for pid=7149 comm="neutron-rootwra" name="rootwrap.sock" scontext=system_u:systemile permissive=0 type=AVC msg=audit(1458637923.473:3273): avc: denied { create } for pid=7194 comm="neutron-rootwra" name="rootwrap.sock" scontext=system_u:system_r:neutron_t:s0 tcontext=system_u:object_r:neutron_tmp_t:s0 tclass=sock_c: denied { create } for pid=7246 comm="neutron-rootwra" name="rootwrap.sock" scontext=system_u:system_r:neutron_t:s0 tcontext=system_u:object_r:neutron_tmp_t:s0 tclass=sock_file permissive=0 type=AVC msg=audit(1458638014.464:3303): arootwrap.sock" scontext=system_u:system_r:neutron_t:s0 tcontext=system_u:object_r:neutron_tmp_t:s0 tclass=sock_file permissive=0 type=AVC msg=audit(1458638075.119:3327): avc: denied { create } for pid=7282 comm="neutron-rootwra" name=s=sock_file permissive=0 type=AVC msg=audit(1458638136.491:3351): avc: denied { create } for pid=7316 comm="neutron-rootwra" name="rootwrap.sock" scontext=system_u:system_r:neutron_t:s0 tcontext=system_u:object_r:neutron_tmp_t:s0 tcla8638216.498:3375): avc: denied { create } for pid=7371 comm="neutron-rootwra" name="rootwrap.sock" scontext=system_u:system_r:neutron_t:s0 tcontext=system_u:object_r:neutron_tmp_t:s0 tclass=sock_file permissive=0 type=AVC msg=audit(1426 comm="neutron-rootwra" name="rootwrap.sock" scontext=system_u:system_r:neutron_t:s0 tcontext=system_u:object_r:neutron_tmp_t:s0 tclass=sock_file permissive=0 type=AVC msg=audit(1458638296.507:3405): avc: denied { create } for pid=7n_t:s0 tcontext=system_u:object_r:neutron_tmp_t:s0 tclass=sock_file permissive=0 type=AVC msg=audit(1458638376.513:3429): avc: denied { create } for pid=7462 comm="neutron-rootwra" name="rootwrap.sock" scontext=system_u:system_r:neutr-rootwra" name="rootwrap.sock" scontext=system_u:system_r:neutron_t:s0 tcontext=system_u:object_r:neutron_tmp_t:s0 tclass=sock_file permissive=0 type=AVC msg=audit(1458638530.168:3495): avc: denied { create } for pid=7576 comm="neutrocontext=system_u:object_r:neutron_tmp_t:s0 tclass=sock_file permissive=0 type=AVC msg=audit(1458638590.825:3519): avc: denied { create } for pid=7628 comm="neutron-rootwra" name="rootwrap.sock" scontext=system_u:system_r:neutron_t:s0 type=AVC msg=audit(1458638656.533:3543): avc: denied { create } for pid=7658 comm="neutron-rootwra" name="rootwrap.sock" scontext=system_u:system_r:neutron_t:s0 tcontext=system_u:object_r:neutron_tmp_t:s0 tclass=sock_file permissive= { create } for pid=7696 comm="neutron-rootwra" name="rootwrap.sock" scontext=system_u:system_r:neutron_t:s0 tcontext=system_u:object_r:neutron_tmp_t:s0 tclass=sock_file permissive=0 type=AVC msg=audit(1458638742.904:3573): avc: deniecontext=system_u:system_r:neutron_t:s0 tcontext=system_u:object_r:neutron_tmp_t:s0 tclass=sock_file permissive=0 type=AVC msg=audit(1458638816.547:3597): avc: denied { create } for pid=7753 comm="neutron-rootwra" name="rootwrap.sock" type=AVC msg=audit(1458638894.855:3621): avc: denied { create } for pid=7809 comm="neutron-rootwra" name="rootwrap.sock" scontext=system_u:system_r:neutron_t:s0 tcontext=system_u:object_r:neutron_tmp_t:s0 tclass=sock_file permissive=31mavc: denied { create } for pid=7837 comm="neutron-rootwra" name="rootwrap.sock" scontext=system_u:system_r:neutron_t:s0 tcontext=system_u:object_r:neutron_tmp_t:s0 tclass=sock_file permissive=0 type=AVC msg=audit(1458638976.561:3651): [01.sock" scontext=system_u:system_r:neutron_t:s0 tcontext=system_u:object_r:neutron_tmp_t:s0 tclass=sock_file permissive=0 type=AVC msg=audit(1458639046.524:3682): avc: denied { create } for pid=7894 comm="neutron-rootwra" name="rootwra:object_r:neutron_tmp_t:s0 tclass=sock_file permissive=0 type=AVC msg=audit(1458639107.182:3706): avc: denied { create } for pid=7943 comm="neutron-rootwra" name="rootwrap.sock" scontext=system_u:system_r:neutron_t:s0 tcontext=system_8639176.577:3730): avc: denied { create } for pid=8001 comm="neutron-rootwra" name="rootwrap.sock" scontext=system_u:system_r:neutron_t:s0 tcontext=system_u:object_r:neutron_tmp_t:s0 tclass=sock_file permissive=0 type=AVC msg=audit(14rootwrap.sock" scontext=system_u:system_r:neutron_t:s0 tcontext=system_u:object_r:neutron_tmp_t:s0 tclass=sock_file permissive=0 type=AVC msg=audit(1458639258.806:3804): avc: denied { create } for pid=8116 comm="neutron-rootwra" name=context=system_u:object_r:neutron_tmp_t:s0 tclass=sock_file permissive=0 type=AVC msg=audit(1458639336.590:3828): avc: denied { create } for pid=8155 comm="neutron-rootwra" name="rootwrap.sock" scontext=system_u:system_r:neutron_t:s0 type=AVC msg=audit(1458639410.436:3852): avc: denied { create } for pid=8205 comm="neutron-rootwra" name="rootwrap.sock" scontext=system_u:system_r:neutron_t:s0 tcontext=system_u:object_r:neutron_tmp_t:s0 tclass=sock_file permissive= denied { create } for pid=8260 comm="neutron-rootwra" name="rootwrap.sock" scontext=system_u:system_r:neutron_t:s0 tcontext=system_u:object_r:neutron_tmp_t:s0 tclass=sock_file permissive=0 type=AVC msg=audit(1458639496.598:3882): avc.sock" scontext=system_u:system_r:neutron_t:s0 tcontext=system_u:object_r:neutron_tmp_t:s0 tclass=sock_file permissive=0 type=AVC msg=audit(1458639562.086:3906): avc: denied { create } for pid=8296 comm="neutron-rootwra" name="rootwras0 tclass=sock_file permissive=0 type=AVC msg=audit(1458639622.736:3937): avc: denied { create } for pid=8337 comm="neutron-rootwra" name="rootwrap.sock" scontext=system_u:system_r:neutron_t:s0 tcontext=system_u:object_r:neutron_tmp_t606:3961): avc: denied { create } for pid=8399 comm="neutron-rootwra" name="rootwrap.sock" scontext=system_u:system_r:neutron_t:s0 tcontext=system_u:object_r:neutron_tmp_t:s0 tclass=sock_file permissive=0 type=AVC msg=audit(1458639713"neutron-rootwra" name="rootwrap.sock" scontext=system_u:system_r:neutron_t:s0 tcontext=system_u:object_r:neutron_tmp_t:s0 tclass=sock_file permissive=0 type=AVC msg=audit(1458639776.614:3991): avc: denied { create } for pid=8474 commcontext=system_u:object_r:neutron_tmp_t:s0 tclass=sock_file permissive=0 type=AVC msg=audit(1458639856.620:4015): avc: denied { create } for pid=8506 comm="neutron-rootwra" name="rootwrap.sock" scontext=system_u:system_r:neutron_t:s0 udit(1458639926.203:4039): avc: denied { create } for pid=8542 comm="neutron-rootwra" name="rootwrap.sock" scontext=system_u:system_r:neutron_t:s0 tcontext=system_u:object_r:neutron_tmp_t:s0 tclass=sock_file permissive=0 type=AVC msg=82 comm="neutron-rootwra" name="rootwrap.sock" scontext=system_u:system_r:neutron_t:s0 tcontext=system_u:object_r:neutron_tmp_t:s0 tclass=sock_file permissive=0 type=AVC msg=audit(1458640016.627:4069): avc: denied { create } for pid=8r:neutron_t:s0 tcontext=system_u:object_r:neutron_tmp_t:s0 tclass=sock_file permissive=0 type=AVC msg=audit(1458640078.053:4093): avc: denied { create } for pid=8654 comm="neutron-rootwra" name="rootwrap.sock" scontext=system_u:systems=sock_file permissive=0 type=AVC msg=audit(1458640138.903:4117): avc: denied { create } for pid=8682 comm="neutron-rootwra" name="rootwrap.sock" scontext=system_u:system_r:neutron_t:s0 tcontext=system_u:object_r:neutron_tmp_t:s0 tcla31mavc: denied { create } for pid=8727 comm="neutron-rootwra" name="rootwrap.sock" scontext=system_u:system_r:neutron_t:s0 tcontext=system_u:object_r:neutron_tmp_t:s0 tclass=sock_file permissive=0 type=AVC msg=audit(1458640230.068:4154): [01" name="rootwrap.sock" scontext=system_u:system_r:neutron_t:s0 tcontext=system_u:object_r:neutron_tmp_t:s0 tclass=sock_file permissive=0 type=AVC msg=audit(1458640296.654:4186): avc: denied { create } for pid=8787 comm="neutron-rootwr:object_r:neutron_tmp_t:s0 tclass=sock_file permissive=0 type=AVC msg=audit(1458640376.660:4210): avc: denied { create } for pid=8854 comm="neutron-rootwra" name="rootwrap.sock" scontext=system_u:system_r:neutron_t:s0 tcontext=system_udit(1458640442.564:4234): avc: denied { create } for pid=8882 comm="neutron-rootwra" name="rootwrap.sock" scontext=system_u:system_r:neutron_t:s0 tcontext=system_u:object_r:neutron_tmp_t:s0 tclass=sock_file permissive=0 type=AVC msg= pid=8910 comm="neutron-rootwra" name="rootwrap.sock" scontext=system_u:system_r:neutron_t:s0 tcontext=system_u:object_r:neutron_tmp_t:s0 tclass=sock_file permissive=0 type=AVC msg=audit(1458640533.554:4264): avc: denied { create } for:neutron_t:s0 tcontext=system_u:object_r:neutron_tmp_t:s0 tclass=sock_file permissive=0 type=AVC msg=audit(1458640594.210:4288): avc: denied { create } for pid=8966 comm="neutron-rootwra" name="rootwrap.sock" scontext=system_u:systemissive=0 type=AVC msg=audit(1458640656.686:4312): avc: denied { create } for pid=9006 comm="neutron-rootwra" name="rootwrap.sock" scontext=system_u:system_r:neutron_t:s0 tcontext=system_u:object_r:neutron_tmp_t:s0 tclass=sock_file per denied { create } for pid=9110 comm="neutron-rootwra" name="rootwrap.sock" scontext=system_u:system_r:neutron_t:s0 tcontext=system_u:object_r:neutron_tmp_t:s0 tclass=sock_file permissive=1 type=AVC msg=audit(1458640736.695:4350): avc: path="/run/netns/qrouter-63deb918-e054-47fd-a2f8-dac1548acf18" dev="nsfs" ino=4026532504 scontext=system_u:system_r:neutron_t:s0 tcontext=system_u:object_r:unlabeled_t:s0 tclass=file permissive=1 type=AVC msg=audit(1458640737.582:4355): avc[0 msg='avc: received setenforce notice (enforcing=0) exe="/usr/lib/systemd/systemd" sauid=0 hostname=? addr=? terminal=?' type=AVC msg=audit(1458640835.553:4395): avc: denied { create } for pid=9514 comm="neutron-ro
Yariv, Is this one still valid? Thanks, Nir
We are verifying the following bug: https://bugzilla.redhat.com/show_bug.cgi?id=1278495 It seems related, we will update once it is verified
*** This bug has been marked as a duplicate of bug 1278495 ***