Bug 1370047
| Summary: | Cant revert from neutron-port PF to VF because sriov_numvfs parameter get "0" value | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Eran Kuris <ekuris> | ||||
| Component: | openstack-tripleo | Assignee: | James Slagle <jslagle> | ||||
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Arik Chernetsky <achernet> | ||||
| Severity: | high | Docs Contact: | |||||
| Priority: | high | ||||||
| Version: | 10.0 (Newton) | CC: | atelang, beagles, berrange, dasmith, eglynn, ekuris, fbaudin, jdonohue, jschluet, kchamart, ksundara, mburns, nyechiel, oblaut, rhel-osp-director-maint, rnoriega, sbauza, sferdjao, sgordon, skramaja, srevivo, vchundur, vromanso, yrachman | ||||
| Target Milestone: | --- | Keywords: | Reopened, ZStream | ||||
| Target Release: | 11.0 (Ocata) | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2017-07-14 15:23:32 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Bug Depends On: | |||||||
| Bug Blocks: | 1233921, 1401639, 1401640, 1438828 | ||||||
| Attachments: |
|
||||||
|
Description
Eran Kuris
2016-08-25 07:23:12 UTC
Eran was this on a system where you were modifying the VF count to 0, or is it VF >0 allocate PF deallocate PF VF == 0 ? Brent I did not change the VF count to 0 manually . The scenario is : VF >0 allocate PF deallocate PF VF == 0 According to this bug my opinion is that RFE : https://bugzilla.redhat.com/show_bug.cgi?id=1233921 is block because its not looks like there is dynamic change between PF & VF When the tripleo SR-IOV support is completed, this should be taken care of because there will be a script that runs when the interface is brought back up the VFs will get reset to the expected configured value. Of course this is contingent on there being and ifup on that PF. @Eran, can you check the up/down status of the PF once it's been "released" if you get the chance? (In reply to Brent Eagles from comment #5) > When the tripleo SR-IOV support is completed, this should be taken care of > because there will be a script that runs when the interface is brought back > up the VFs will get reset to the expected configured value. Of course this > is contingent on there being and ifup on that PF. > > @Eran, can you check the up/down status of the PF once it's been "released" > if you get the chance? Yes Brent it's been released after I delete the VM that associate to PF Actually, that's not what I meant to ask. I was referring to whether the interface was up or down. If it is not set to "up" then we cannot rely on the ifup-local hook that we install to resolve the VF count issue. If it is down, can you try bringing it up and seeing if the VFs come back or not. (In reply to Brent Eagles from comment #7) > Actually, that's not what I meant to ask. I was referring to whether the > interface was up or down. If it is not set to "up" then we cannot rely on > the ifup-local hook that we install to resolve the VF count issue. If it is > down, can you try bringing it up and seeing if the VFs come back or not. It set to up after I release the PF . Okay thanks. So this means that the persistent VF thing added by tripleo isn't going to help. Vladik, is this something that nova can do when the pci device is released? Alternatively, we'll have to get the SR-IOV agent involved on the compute node. Created attachment 1197223 [details]
pf_test from vladikr env.
Could we please reproduce the bug, but this time please enable the VFs using the max_vfs parameter and not via sysfs tunable. Unload the ixgbe (or other driver of the card ) driver and load it again modprobe ixgbe max_vfs=X I can't reproduce this problem with my card. I've added an output from my server. Regardless, it looks like this is a VFIO or a libvirt issue (hostdev managed=True), rather than nova. In Nova, we are relying on this behaviour as well. I think we should try reproducing with max_vfs and if it doesn't work we should try consulting Alex Williamson from the kvm team. Vladik This is my take away from the testing environment for Telefonica. - sysfs: Using a command like this "echo 4 > /sys/class/net/em1/device/sriov_numvfs" does not persist the number of VFs per network adapter. So after allocating/deallocating a PF, the VFs configured previously are gone. - ixgbe max_vfs parameter works well. Doing the same kind of test as before, the VFs are back, however, the parent interface has got its link state as DOWN what makes its children (VFs) not to be available for allocation. Ricky Latest news! As per Vladik instructions, with the NetworkManager enabled, the parent interface comes back in UP state. So combination of ixgbe max_vfs parameter + NetworkManager service makes the job. Ricky it still exist from my last check (In reply to Eran Kuris from comment #22) > it still exist from my last check I think we were really expecting you to be a little more expansive here as to the ask, as the upstream comment Nir referred to above was yes that's right and this is expected behavior: https://bugs.launchpad.net/nova/+bug/1616769/comments/1 This was fixed via changes to tripleo/director. The user needs to enable network management using "nm_controlled: true" on RHEL or "hotplug: true" on CentOS on the relevant interfaces in the network environment files. Please see: https://bugzilla.redhat.com/show_bug.cgi?id=1392584 https://bugzilla.redhat.com/show_bug.cgi?id=1392585 |