Bug 1141518

Summary: NOVA API will not update nova compute with vif-plugged events when booting many VMs
Product: Red Hat OpenStack Reporter: Ofer Blaut <oblaut>
Component: openstack-novaAssignee: Russell Bryant <rbryant>
Status: CLOSED ERRATA QA Contact: nlevinki <nlevinki>
Severity: high Docs Contact:
Priority: medium    
Version: 5.0 (RHEL 7)CC: eglynn, ndipanov, nlevinki, oblaut, sclewis, sgordon, slong, yeylon
Target Milestone: z4Keywords: ZStream
Target Release: 5.0 (RHEL 7)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: openstack-nova-2014.1.4-1.el7ost Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-04-16 14:34:40 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Logs none

Description Ofer Blaut 2014-09-14 10:24:07 UTC
Created attachment 937311 [details]
Logs

Description of problem:

When booting many VMs (100 - 160 vms) nova boot VM-cirr --image  7f5e271d-8c4e-45d0-8f1b-4edfe5babef7 --flavor 10 --key_name oskey1 --num-instances 160

Many VMs will fail on VIF plugged timeout .

When Nova notify is OFF , i can reach 250 ACTIVE VMs on the same setup

Nova ACK neutron about the plugged event , but doesn't update nova compute which fails the VMs.
 

There are two bugs on upstream which we should track as well
https://bugs.launchpad.net/nova/+bug/1337860
https://bugs.launchpad.net/nova/+bug/1338679


Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.disable L2 pop
2.create setup with controller/networker and  two computes
3.launch 100/160 Nano VMs ( cirros based  )
 

Actual results:


Expected results:


Additional info:

Comment 1 Stephen Gordon 2014-10-08 01:44:32 UTC
Hi Russell,

Can I please get somebody to check this? It looks like Vladik fixed this in LP # 1337860 and it already hit Icehouse stable branch so we may be getting this via rebase "for free".

Thanks,

Steve

Comment 2 Russell Bryant 2014-10-08 12:45:24 UTC
(In reply to Stephen Gordon from comment #1)
> Hi Russell,
> 
> Can I please get somebody to check this? It looks like Vladik fixed this in
> LP # 1337860 and it already hit Icehouse stable branch so we may be getting
> this via rebase "for free".

Yes, I think you're right.  We should re-test after the rebase.

Comment 6 Eoghan Glynn 2015-03-26 10:49:40 UTC
Fix came in via rebase:

  https://git.openstack.org/cgit/openstack/nova/commit/?id=94472033

Comment 9 nlevinki 2015-04-06 07:28:52 UTC
rpm -qa| grep openstack-nova
openstack-nova-conductor-2014.1.4-3.el7ost.noarch
openstack-nova-api-2014.1.4-3.el7ost.noarch
openstack-nova-compute-2014.1.4-3.el7ost.noarch
openstack-nova-novncproxy-2014.1.4-3.el7ost.noarch
openstack-nova-console-2014.1.4-3.el7ost.noarch
openstack-nova-scheduler-2014.1.4-3.el7ost.noarch
openstack-nova-common-2014.1.4-3.el7ost.noarch
openstack-nova-cert-2014.1.4-3.el7ost.noarch

created 150 vms, after tweaking Neutron.conf it worked

Comment 10 Summer Long 2015-04-14 06:20:34 UTC
NEEDINFO:   Eoghan Glynn

This bug is marked for inclusion in RHSA-2015:20137 [1] but does not currently contain draft documentation text. To ensure the timely release of this advisory, please provide draft documentation text for this bug as soon as possible.

If you do not think this bug requires errata documentation, set the requires_doc_text flag to "-". 

To add draft documentation text:
* Select the documentation type from the "Doc Type" drop down field.
* A template will be provided in the "Doc Text" field based on the "Doc Type" value selected. Enter draft text in the "Doc Text" field.

For further information on this process, refer to "Describing Errata Release and Technical Notes for Engineers" [2]. Thank you for your assistance!

[1] https://errata.devel.redhat.com/advisory/20137
[2] https://engineering.redhat.com/docs/en-US/Policy/70.ecs/html/Describing_Errata_Release_and_Technical_Notes_for_Engineers/index.html

Comment 12 errata-xmlrpc 2015-04-16 14:34:40 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-0843.html

Comment 14 Red Hat Bugzilla 2023-09-18 00:10:50 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days