Bug 1291153

Summary: After OVS-DPDK package installation host responds very slowly
Product: Red Hat OpenStack Reporter: Eran Kuris <ekuris>
Component: openvswitch-dpdkAssignee: Open vSwitch development team <ovs-team>
Status: CLOSED NOTABUG QA Contact:
Severity: medium Docs Contact:
Priority: medium    
Version: 8.0 (Liberty)CC: aloughla, amuller, chrisw, ekuris, fleitner, jhsiao, nyechiel, rkhan, yeylon
Target Milestone: ---   
Target Release: 8.0 (Liberty)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-12-24 09:16:06 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
dpdk none

Description Eran Kuris 2015-12-14 07:14:26 UTC
Description of problem:
Install OVS-DPDK packages 
dpdk-tools-2.1.0-5.el7.x86_64
openvswitch-dpdk-2.4.0-0.10346.git97bab959.2.el7.x86_64
dpdk-2.1.0-5.el7.x86_64

After that every command or action taking too long. THe host response very slowly.

Version-Release number of selected component (if applicable):
dpdk-tools-2.1.0-5.el7.x86_64
openvswitch-dpdk-2.4.0-0.10346.git97bab959.2.el7.x86_64
dpdk-2.1.0-5.el7.x86_64
[root@puma06 ~]# rpm -qa |grep neutron
python-neutronclient-3.1.0-1.el7ost.noarch
python-neutron-7.0.0-5.el7ost.noarch
openstack-neutron-common-7.0.0-5.el7ost.noarch
openstack-neutron-7.0.0-5.el7ost.noarch
openstack-neutron-ml2-7.0.0-5.el7ost.noarch
openstack-neutron-openvswitch-7.0.0-5.el7ost.noarch


How reproducible:
always 
Steps to Reproduce:
1.Install OSP-8 ENV with packstack {AIO ENV}
2.install ovs-DPDK packages 
3.your user experience now is very slow 

Actual results:


Expected results:


Additional info:
add logs .

Comment 1 Eran Kuris 2015-12-14 07:17:01 UTC
Created attachment 1105503 [details]
dpdk

Comment 5 Jean-Tsung Hsiao 2015-12-16 15:19:10 UTC
(In reply to Eran Kuris from comment #0)
> Description of problem:
> Install OVS-DPDK packages 
> dpdk-tools-2.1.0-5.el7.x86_64
> openvswitch-dpdk-2.4.0-0.10346.git97bab959.2.el7.x86_64
> dpdk-2.1.0-5.el7.x86_64
> 
> After that every command or action taking too long. THe host response very
> slowly.
> 
> Version-Release number of selected component (if applicable):
> dpdk-tools-2.1.0-5.el7.x86_64
> openvswitch-dpdk-2.4.0-0.10346.git97bab959.2.el7.x86_64
> dpdk-2.1.0-5.el7.x86_64
> [root@puma06 ~]# rpm -qa |grep neutron
> python-neutronclient-3.1.0-1.el7ost.noarch
> python-neutron-7.0.0-5.el7ost.noarch
> openstack-neutron-common-7.0.0-5.el7ost.noarch
> openstack-neutron-7.0.0-5.el7ost.noarch
> openstack-neutron-ml2-7.0.0-5.el7ost.noarch
> openstack-neutron-openvswitch-7.0.0-5.el7ost.noarch
> 
> 
> How reproducible:
> always 
> Steps to Reproduce:
> 1.Install OSP-8 ENV with packstack {AIO ENV}
> 2.install ovs-DPDK packages 
> 3.your user experience now is very slow 
> 
> Actual results:
> 
> 
> Expected results:
> 
> 
> Additional info:
> add logs .

I think the issue is that the system under test is not powerful enough to drive the OSP env:

memory: 32 GB
CPU model: Intel(R) Xeon(R) CPU           E5645 @ 2.40GHz.

NOTE: With only 32 GB total you probably can only allocate 8 hugepages.

Comment 6 Jean-Tsung Hsiao 2015-12-16 16:46:25 UTC
After editing GRUB_CMDLINE_LINUX to have only 8 1GB hugepages allocated, grub2, and reboot, system has come up with openvswitch running.

AnonHugePages:    585728 kB
HugePages_Total:       8
HugePages_Free:        7
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:    1048576 kB
DirectMap4k:      107968 kB
DirectMap2M:     6174720 kB
DirectMap1G:    29360128 kB

[root@puma06 jhsiao]# !ps
ps -elf | grep ovs-vs
1 S root      7214     1  0  70 -10 - 295366 wait  18:31 ?        00:00:00 ovs-vswit... --dpdk -d /usr/lib64/librte_pmd_ixgbe.so -d /usr/lib64/librte_pmd_i40e -l 1,2 -n 1 --socket-mem 1024 0 -- unix:/var/run/openvswitch/db.sock -vconsole:emer -vsyslog:err -vfile:info --mlockall --no-chdir --log-file=/var/log/openvswitch/ovs-vswitchd.log --pidfile=/var/run/openvswitch/ovs-vswitchd.pid --detach --monitor
5 S root      7215  7214  0  70 -10 - 441050 poll_s 18:31 ?       00:00:00 ovs-vswitchd --dpdk -d /usr/lib64/librte_pmd_ixgbe.so -d /usr/lib64/librte_pmd_i40e -l 1,2 -n 1 --socket-mem 1024 0 -- unix:/var/run/openvswitch/db.sock -vconsole:emer -vsyslog:err -vfile:info --mlockall --no-chdir --log-file=/var/log/openvswitch/ovs-vswitchd.log --pidfile=/var/run/openvswitch/ovs-vswitchd.pid --detach --monitor

NOTE: The system has neither ixgbe nor i40e NICs present.

Comment 7 Flavio Leitner 2015-12-23 20:52:33 UTC
Please attach an sosreport from the system reproducing the issue.

Comment 8 Eran Kuris 2015-12-24 07:46:16 UTC
After Jean edit the Hugepagesize the system start to response as  I expected . 
I think to move the bug to "not a bug " status

Comment 9 Nir Yechiel 2015-12-24 09:16:06 UTC
(In reply to Eran Kuris from comment #8)
> After Jean edit the Hugepagesize the system start to response as  I expected
> . 
> I think to move the bug to "not a bug " status

Let me close this for now. Please repoen if needed.