This bug is to track libvirt support for NPIV which is imminently to be completed upstream...
Final patch ready for use http://www.redhat.com/archives/libvir-list/2009-June/msg00005.html
Patch commited upstream, Daniel
QLogic/Emulex/EMC - Can we count on you to test this new functionality?
libvirt-0.6.3-4.el5 has been built in dist-5E-qu-candidate with the patch, Daniel
Created attachment 347247 [details] Brief instructions on how to test NPIV I've attached a short document that describes how to test the NPIV create/destroy operations using virsh.
*** URGENT *** This is a call to all partners on this bugzilla - Dave has submitted instructions on how to test the forthcoming functionality of NPIV in Comment #8 above. When the partner Alpha is released on Friday, please use these bits with the instructions above, and post feedback in this Bugzilla. If you have any questions, please don't hesitate to ask here.
~~ Attention Partners RHEL 5.4 Partner Alpha Released! ~~ RHEL 5.4 Partner Alpha has been released on partners.redhat.com. There should be a fix present that addresses this particular request. Please test and report back your results here, at your earliest convenience. Our Public Beta release is just around the corner! If you encounter any issues, please set the bug back to the ASSIGNED state and describe the issues you encountered. If you have verified the request functions as expected, please set your Partner ID in the Partner field above to indicate successful test results. Do not flip the bug status to VERIFIED. Further questions can be directed to your Red Hat Partner Manager. Thanks!
(In reply to comment #8) > Created an attachment (id=347247) [details] > Brief instructions on how to test NPIV > > I've attached a short document that describes how to test the NPIV > create/destroy operations using virsh. We tried these instructions on a RHEL 5.4 alpha system here, but ran into a couple of difficulties. virsh doesn't show the correct WWPN/WWNNs for the FC adapters, instead it shows all 'f's for all adapter WWNs: # virsh nodedev-dumpxml pci_10df_fd00_scsi_host <device> <name>pci_10df_fd00_scsi_host</name> <parent>pci_10df_fd00</parent> <capability type='scsi_host'> <host>7</host> <capability type='fc_host'> <wwnn>ffffffffffffffff</wwnn> <wwpn>ffffffffffffffff</wwpn> </capability> </capability> </device> Also, virsh does not show <capability type='vport_ops' /> for any of the NPIV-capable adapters in this test system. # virsh nodedev-create new.xml generates a Segmentation fault. The new.xml file I'm using is: <device> <parent>pci_10df_fd00_0_scsi_host</parent> <capability type='scsi_host'> <capability type='fc_host'> <wwpn>2000000033334444</wwpn> <wwnn>1000000077778888</wwnn> </capability> </capability> </device> I think these items are probably related to differences between RHEL 5.4 and the current upstream layout of virtual port information in sysfs (scsi_host vs. fc_host). With respect to the instructions themselves, this example command: echo '1111222233334444:5555666677778888' > /sys/class/fc_host/host5/vport_create has 2 issues. First the vport_create sysfs entry is under the scsi_host, not the fc_host, e.g. /sys/class/scsi_host/host5/vport_create (this is an upstream vs RHEL 5.4 kernel difference). Second, the WWN format given in this example is invalid and gets rejected by the lpfc driver. This should work though: echo '2000000033334444:1000000077778888' > /sys/class/scsi_host/host5/vport_create
I believe you are correct that the problems are due to differing locations of information in sysfs. I'm fixing that, and the segfault now. I didn't specify a valid WWN in the example because I want people to use WWNs that are valid for their environments. I'll update the doc to note that the example WWN is bogus. Please hold off further testing until I have a patch ready.
Jamie, The lack of vport capability on your system makes sense to me, as the vport_create and vport_delete moved from /sys/class/scsi_host/hostN to /sys/class/fc_host/hostN, and I'm just about finished with the fix for that. I'm not sure, though, why the WWNs aren't being discovered correctly. Do /sys/class/fc_host/host7/node_name and port_name exist on your system? Those two files are in the same places on my RHEL 5.4 and Fedora 10 systems. Dave
Dave, virsh shows the correct WWNs if lpfc is loaded with lpfc_enable_npiv=0 (the default). virsh shows all f's for all WWNNs/WWPNs when lpfc is loaded with lpfc_enable_npiv=1. The /sys/class/fc_host/hostN/node_name and /sys/.../port_name files exist and contain the correct numbers in both cases (with and without npiv enabled). Jamie
libvirt-0.6.3-11.el5 has been built in dist-5E-qu-candidate with the fix for the 2 different /sys paths. Daniel
(In reply to comment #21) > libvirt-0.6.3-11.el5 has been built in dist-5E-qu-candidate with the fix > for the 2 different /sys paths. > > Daniel Does this build fix the all f's WWN display?
Yes, however, I was not seeing that particular problem (or the segfault) on my test system, so I am very interested in your experience.
To speed up testing I have uploaded libvirt-0.6.3-11.el5 I have put the built packages in http://veillard.com/libvirt/5.4/ repo, Daniel
I just installed that package and was able to create and destroy an adapter.
Created attachment 348700 [details] Revised NPIV test instructions Here's a revised test instructions document with updates as suggested by Jamie.
With libvirt-0.6.3-11.el5, the vport_ops capability shows up correctly in the nodedev-dumpxml output and I can create and destroy vports with virsh. The WWNs show up correctly when the system first boots, but if I rmmod and insmod lpfc (to change lpfc_enable_npiv, for instance), then "virsh nodedev-dumpxml" shows all f's for the WWNs. service libvirtd restart fixes that until the next time I reload lpfc. I imagine this is the same behavior as the old package - I just didn't figure out the causal relationship.
Ugh. I've reproduced that on my system as well. I'm looking into it now.
The problem isn't limited to RHEL5.4, it also exists in the head on Fedora 10.
I have found out what's going on, although I don't have a fix yet. The WWN of ffffffffffffffff is actually being reported by the module itself in sysfs as it's loading. Once the module has initialized completely, it reports the correct WWN in sysfs, but by then libvirt has already read the wrong names. I am thinking that the code should just pause for a while if it sees a WWN of ffffffffffffffff and retry. The wrong WWN does not affect the creation/deletion of virtual adapters on my system.
The driver might be busy talking to storage for awhile after loading. The amount of storage on a SAN is somewhat open ended, and the storage units might also be doing other things. So the time here might be hard to define...
Perhaps it'll be beneficial to refresh the WWN at the time of the libvirt virNodeDeviceDumpXML call, so we always have the most up2date info. This would avoid having to sleep/retry when the device initially appears.
Refreshing the WWN at dumpxml time is a reasonable solution. It's certainly more deterministic than trying to figure out when the module has completed initializing.
I have a fix for the WWN problem. Would it be acceptable to open a separate bug against that problem as it does not affect create/destroy operations?
A separate bug for the WWN problem is find with me - I'm not sure you're asking me, though :)
libvirt-0.6.3-12.el5 has been built in dist-5E-qu-candidate with the fix for the bogus WWN on reload. To fasten the tests on this bug fix I have updated the repo at http://veillard.com/libvirt/5.4/ to include the new build, please give it a try and report, then if positive I will update that build as a candidate for release and internal QA, thanks in advance, Daniel
~~ Attention - RHEL 5.4 Beta Released! ~~ RHEL 5.4 Beta has been released! There should be a fix present in the Beta release that addresses this particular request. Please test and report back results here, at your earliest convenience. RHEL 5.4 General Availability release is just around the corner! If you encounter any issues while testing Beta, please describe the issues you have encountered and set the bug into NEED_INFO. If you encounter new issues, please clone this bug to open a new issue and request it be reviewed for inclusion in RHEL 5.4 or a later update, if it is not of urgent severity. Please do not flip the bug status to VERIFIED. Only post your verification results, and if available, update Verified field with the appropriate value. Questions can be posted to this bug or your customer or partner representative.
~~ Attention Partners - RHEL 5.4 Snapshot 1 Released! ~~ RHEL 5.4 Snapshot 1 has been released on partners.redhat.com. If you have already reported your test results, you can safely ignore this request. Otherwise, please notice that there should be a fix available now that addresses this particular request. Please test and report back your results here, at your earliest convenience. The RHEL 5.4 exception freeze is quickly approaching. If you encounter any issues while testing Beta, please describe the issues you have encountered and set the bug into NEED_INFO. If you encounter new issues, please clone this bug to open a new issue and request it be reviewed for inclusion in RHEL 5.4 or a later update, if it is not of urgent severity. Do not flip the bug status to VERIFIED. Instead, please set your Partner ID in the Verified field above if you have successfully verified the resolution of this issue. Further questions can be directed to your Red Hat Partner Manager or other appropriate customer representative.
Moved to VERIFIED based on #comment 27, QE please create a new bug for the WWN issue and leave a comment with the bug number in this bug once its done. Thanks.
Bug 514408 is created for the WWN issue.
I'm assuming based on all feedback that all partners have completed NPIV testing on 5.4. Let me know if this isn't true.
An advisory has been issued which should help the problem described in this bug report. This report is therefore being closed with a resolution of ERRATA. For more information on therefore solution and/or where to find the updated files, please follow the link below. You may reopen this bug report if the solution does not work for you. http://rhn.redhat.com/errata/RHEA-2009-1269.html