Bug 1141943

Hi John,

I verify the bug by using only one vHBA. In other words, I can define a scsi pool defined adapter as 'scsi_host' type with a vHBA, and then define another scsi pool defined adapter as 'fc_host' type with the SAME vHBA. Does it make sense ?

I met some issue when I verified the bug with libvirt-1.2.8-3.el7.x86_64.
Could you please help check them? Thanks in advance!

Prepare a vhba
1. create vhba
# cat vHBA.xml 
    <device>
    <parent>scsi_host5</parent>
    <capability type='scsi_host'>
    <capability type='fc_host'>
    <wwnn>2101001b32a90002</wwnn>
    <wwpn>2101001b32a90003</wwpn>
    <capability type='vport_ops'/>
    </capability>
    </capability>
    </device>

# virsh nodedev-create vHBA.xml 
Node device scsi_host8 created from vHBA.xml

# virsh nodedev-list scsi_host
scsi_host0
scsi_host1
scsi_host2
scsi_host3
scsi_host4
scsi_host5
scsi_host8

# lspci | grep -i fibre
04:00.0 Fibre Channel: QLogic Corp. ISP2432-based 4Gb Fibre Channel to PCI Express HBA (rev 03)
04:00.1 Fibre Channel: QLogic Corp. ISP2432-based 4Gb Fibre Channel to PCI Express HBA (rev 03)

scenario 1: define an scsi pool with scsi_host type adapter first, 
            then define another scsi pool with fc_host type adapter         
1. define/start a scsi pool with scsi_host adapter
# cat scsi-pool.xml 
<pool type='scsi'>
<name>scsi-pool</name>
<source>
<adapter type='scsi_host' name='scsi_host8'/>
</source>
<target>
<path>/dev/disk/by-path</path>
<permissions>
<mode>0700</mode>
<owner>0</owner>
<group>0</group>
</permissions>
</target>
</pool>

[root@dell-pet105-04 yy]# virsh pool-define scsi-pool.xml 
Pool scsi-pool defined from scsi-pool.xml

[root@dell-pet105-04 yy]# virsh pool-start scsi-pool
Pool scsi-pool started

2. define a fc pool without parent name
# cat fc-pool-1.xml 
<pool type='scsi'>
<name>fc-pool-1</name>
<source>
<adapter type='fc_host' wwnn='2101001b32a90002' wwpn='2101001b32a90003'/>
</source>
<target>
<path>/dev/disk/by-path</path>
<permissions>
<mode>0700</mode>
<owner>0</owner>
<group>0</group>
</permissions>
</target>
</pool>

[root@dell-pet105-04 yy]# virsh pool-define fc-pool-1.xml 
Pool fc-pool-1 defined from fc-pool-1.xml

[root@dell-pet105-04 yy]# virsh pool-start fc-pool-1
Pool fc-pool-1 started

# service libvirtd status
   Active: active (running) 

3. define a fc pool with parent name
# cat fc-pool.xml 
<pool type='scsi'>
<name>fc-pool</name>
<source>
<adapter type='fc_host' parent='scsi_host5' wwnn='2101001b32a90002' wwpn='2101001b32a90003'/>
</source>
<target>
<path>/dev/disk/by-path</path>
<permissions>
<mode>0700</mode>
<owner>0</owner>
<group>0</group>
</permissions>
</target>
</pool>

[root@dell-pet105-04 yy]# virsh pool-define fc-pool.xml 
error: Failed to define pool from fc-pool.xml
error: operation failed: Storage source conflict with pool: 'fc-pool-1'

[root@dell-pet105-04 yy]# virsh pool-destroy fc-pool-1
Pool fc-pool-1 destroyed
[root@dell-pet105-04 yy]# virsh pool-undefine fc-pool-1
Pool fc-pool-1 has been undefined


[root@dell-pet105-04 yy]# virsh pool-define fc-pool.xml 
Pool fc-pool defined from fc-pool.xml

[root@dell-pet105-04 yy]# virsh pool-start fc-pool
Pool fc-pool started

Hi John,
I destroyed/undefined fc-pool and scsi-pool, then tried to define/start an scsi pool with fc-pool-1.xml, however the vhba's name is changed

# virsh nodedev-list scsi_host
scsi_host0
scsi_host1
scsi_host2
scsi_host3
scsi_host4
scsi_host5
scsi_host9  --------> it is changed from scsi_host8 to scsi_host9
                      Is it expected ?
                      

senario 2: define an scsi pool with 'fc_host' adapter first
           then define another scsi pool with 'scsi_host' adapter 

1. define/start a scsi pool with adapter type as
    'fc_host' and parent attribute missing

[root@dell-pet105-04 yy]# virsh pool-define fc-pool-1.xml 
Pool fc-pool-1 defined from fc-pool-1.xml

[root@dell-pet105-04 yy]# virsh pool-start fc-pool-1
Pool fc-pool-1 started

[root@dell-pet105-04 yy]# virsh vol-list fc-pool-1
 Name                 Path                                    
------------------------------------------------------------------------------
 unit:0:1:0           /dev/disk/by-path/pci-0000:04:00.1-fc-0x203600a0b85b5dd4-lun-0

2. define/start another scsi pool with 'scsi_host' adapter
[root@dell-pet105-04 yy]# virsh pool-define scsi-pool.xml 
Pool scsi-pool defined from scsi-pool.xml

[root@dell-pet105-04 yy]# virsh pool-start scsi-pool
Pool scsi-pool started

[root@dell-pet105-04 yy]# virsh vol-list scsi-pool
 Name                 Path                                    
------------------------------------------------------------------------------
 unit:0:1:0           /dev/disk/by-path/pci-0000:04:00.1-fc-0x203600a0b85b5dd4-lun-0


3. clean env
[root@dell-pet105-04 yy]# virsh pool-destroy scsi-pool
Pool scsi-pool destroyed

[root@dell-pet105-04 yy]# virsh pool-undefine scsi-pool
Pool scsi-pool has been undefined

[root@dell-pet105-04 yy]# virsh pool-destroy fc-pool-1
Pool fc-pool-1 destroyed

[root@dell-pet105-04 yy]# virsh pool-undefine fc-pool-1
Pool fc-pool-1 has been undefined

4. define/start a scsi pool with 'fc_host' type adapter 
   and parent attribute defined

# virsh pool-define fc-pool.xml 
Pool fc-pool defined from fc-pool.xml

# virsh pool-start fc-pool
Pool fc-pool started

# virsh vol-list fc-pool
 Name                 Path                                    
------------------------------------------------------------------------------
 unit:0:1:0           /dev/disk/by-path/pci-0000:04:00.1-fc-0x203600a0b85b5dd4-lun-0

5. define/start another scsi pool with 'scsi_host' type adapter
# virsh pool-define scsi-pool.xml 
Pool scsi-pool defined from scsi-pool.xml

# virsh pool-start scsi-pool
Pool scsi-pool started

# virsh vol-list scsi-pool
 Name                 Path                                    
------------------------------------------------------------------------------
 unit:0:1:0           /dev/disk/by-path/pci-0000:04:00.1-fc-0x203600a0b85b5dd4-lun-0

Hi John,
After I destroyed/undefined scsi-pool and fc-pool to clean env, the vhba 'scsi_host9' disappared. Is it expected?
# virsh nodedev-list scsi_host
scsi_host0
scsi_host1
scsi_host2
scsi_host3
scsi_host4
scsi_host5


scenario 3: define a scsi pool with 'scsi_host' type adapter name 
            attribute defined first then define a scsi pool with
            'scsi_host' type adapter and parrentaddr attribute defined. 
            The 2nd scsi pool failed to define and caused that libvirtd        
            restarted

1. define/start a scsi pool with scsi_host type adpater 
    and name attribute defined
# virsh nodedev-list scsi_host
scsi_host0
scsi_host1
scsi_host10 -------vHBA         
scsi_host2
scsi_host3
scsi_host4
scsi_host5

# cat scsi-pool.xml 
<pool type='scsi'>
<name>scsi-pool</name>
<source>
<adapter type='scsi_host' name='scsi_host10'/>
</source>
<target>
<path>/dev/disk/by-path</path>
<permissions>
<mode>0700</mode>
<owner>0</owner>
<group>0</group>
</permissions>
</target>
</pool>

# virsh pool-define scsi-pool.xml 
Pool scsi-pool defined from scsi-pool.xml

# virsh pool-start scsi-pool
Pool scsi-pool started

# virsh vol-list scsi-pool
 Name                 Path                                    
------------------------------------------------------------------------------
 unit:0:2:0           /dev/disk/by-path/pci-0000:04:00.1-fc-0x203600a0b85b5dd4-lun-0

2. define/start a scsi pool with 'scsi_host' type adapter 
   and parrentaddr attribute defined

# cat scsi-pool-1.xml 
<pool type='scsi'>
<name>scsi-pool-1</name>
<source>
<adapter type='scsi_host'>
<parentaddr unique_id='10'>
<address domain='0' bus='0x04' slot='0' function='1'/>
</parentaddr>
</adapter>
</source>
<target>
<path>/dev/disk/by-path</path>
<permissions>
<mode>0700</mode>
<owner>0</owner>
<group>0</group>
</permissions>
</target>
</pool>

# ps -ef|grep libvirtd
root     20642     1  1 00:26 ?        00:00:00 /usr/sbin/libvirtd

# virsh pool-define scsi-pool-1.xml 
error: Failed to define pool from scsi-pool-1.xml
error: End of file while reading data: Input/output error
error: Failed to reconnect to the hypervisor

# ps -ef|grep libvirtd
root     20707     1 12 00:26 ?        00:00:00 /usr/sbin/libvirtd

Hi John,

The 2nd scsi pool failed
to define and caused that libvirtd restarted

However, if the scsi pool with 'scsi_host' type adapter
            and parrentaddr attribute was defined first, then
            defined the 2nd scsi pool with scsi_host type adpater 
    and name attribute defined. It PASSED.

Scenario 4: define a scsi pool with 'scsi_host' type adapter
            and parrentaddr attribute defined first, then define 
            the 2nd scsi pool with 'fc_host' type adapter

1. define/start a scsi pool with 'scsi_host' type adapter
   and parrentaddr attribute defined

# virsh pool-define scsi-pool-1.xml 
Pool scsi-pool-1 defined from scsi-pool-1.xml

# virsh pool-start scsi-pool-1
Pool scsi-pool-1 started

# virsh vol-list scsi-pool-1
 Name                 Path                                    
------------------------------------------------------------------------------
 unit:0:2:0           /dev/disk/by-path/pci-0000:04:00.1-fc-0x203600a0b85b5dd4-lun-0

2. define the 2nd scsi pool with 'fc_host' type adapter 
   and parent attribute missing

# virsh pool-define fc-pool-1.xml 
Pool fc-pool-1 defined from fc-pool-1.xml

# virsh pool-start fc-pool-1
Pool fc-pool-1 started

# virsh vol-list fc-pool-1
 Name                 Path                                    
------------------------------------------------------------------------------
 unit:0:2:0           /dev/disk/by-path/pci-0000:04:00.1-fc-0x203600a0b85b5dd4-lun-0

3. destroy/undefine fc-pool-1 and define/start a scsi pool 
  with 'fc_host' type adapter and parent attribute defined

# virsh pool-destroy fc-pool-1
Pool fc-pool-1 destroyed

# virsh pool-undefine fc-pool-1
Pool fc-pool-1 has been undefined

# virsh pool-define fc-pool.xml 
Pool fc-pool defined from fc-pool.xml

# virsh pool-start fc-pool
Pool fc-pool started

# virsh vol-list fc-pool
 Name                 Path                                    
------------------------------------------------------------------------------
 unit:0:2:0           /dev/disk/by-path/pci-0000:04:00.1-fc-0x203600a0b85b5dd4-lun-0

Hi John,
 After I destroyed/undefined fc-pool and scsi-pool-1 to clean env, the vHBA disappared
# virsh nodedev-list scsi_host
scsi_host0
scsi_host1
scsi_host2
scsi_host3
scsi_host4
scsi_host5

I have no idea what you're trying to test. I thought the context of the bug report was fairly clear and it seems you were able to verify that.

Beyond that you have multiple scenarios and questions that I'd need to dig into more and understand.  If you think each/any of these scenarios are bugs, then please write separate bug reports for each so that they each can be worked on.

Using this bug report to obtain feedback about specific environments within the realm of the code, but outside the scope of the base problem isn't good.

I'm really not quite sure what you're attempting to accomplish with each of these scenarios and I haven't dug into that code recently so it's not fresh in my mind with respect to what is allowed and/or expected. I also assume you have/use http://wiki.libvirt.org/page/NPIV_in_libvirt - it has some verbage about the transient nature of vHBA's across host reboots, but if you consider it's persistent nature is tied to the storage pool definition and by undefining a storage pool you are removing the reference to the vHBA, perhaps what you're seeing and the glimpses in time that you've shown me are working as expected...

Couple of notes...

For scenario1: 
 - After each step - do the "virsh nodedev-list scsi_host" as I have a feeling that something within steps 2 & 3 may have generated scsi_host9. After each step looking at a dumpxml will will probably help decipher whether you got the expected results. Also, I do seem to remember some code that would determine there was no need to keep the libvirt generated scsi_host# once it was destroyed and undefined, but it's been a while since I looked at the code. It may well be it's expected.  I do know it's an area not well documented (something I noted when the code went in).
 - A nodedev-dumpxml scsi_host5 may be beneficial...

For scenario2:
 - When/if you create a new bug report - be sure to place the various *.xml that you're using in the report.  Makes it easier to not have scroll up and down within the page to see what it is and of course to have to find perhaps another bug that has the definition.
 - I'd need to see similar data/output from scenario1 each step of the way and then what your expectations would be
 - Your service libvirtd status shows, yes that it's running, but doesn't indicate if it's been restarted along the way... So prior to doing anything a 'ps -ef | grep libvirtd' and then after doing everything a similar command will show that libvirtd hasn't restarted (if in fact that's what you're trying to show).

For scenario3:
 - Looks like a real bug that certain deserves its own BZ since it's causing a libvirtd crash with recently added functionality, but you don't describe your step1 with enough detail or do it out of order. In particular, you show a nodedev-list with scsi_host10 defined already, then go and try to define it.
 - Seems like you're pointing out that step2 has a bug where a similarly defined adapter would cause libvirtd to crash because something isn't checked when unique_id is passed.  I know that's a recent change (OK sometime in the last 5-7 months), but I need a separate bug report.
 - Remember when providing the bug report - don't refer me to this bug, but provide the xml in the bug and go step by step.  It's far easier to handle then.
 - It might be beneficial to show the option that did work. 
 - Again the *-dumpxml's really help each step of the way.

For scenario4:
 - I think if I see this one with the details in each order it'll make more sense, but I don' have memorized what each of those .xml files has in it and the naming scheme doesn't help. Again, like with 1 & 2 - you are defining/undefining a storage pool along the way which may cause the transient vHBA to be removed, but I cannot *see* that without showing me your steps starting with nothing and then going through each step and understanding what I would expect from looking at the code (when I have the code paged into my short term memory).

Thanks John.

I'd like to verify the bug with the following steps

1. create a vHBA
# virsh nodedev-create vHBA.xml 
Node device scsi_host6 created from vHBA.xml

# virsh nodedev-dumpxml scsi_host6
<device>
  <name>scsi_host6</name>
  <path>/sys/devices/pci0000:00/0000:00:0d.0/0000:04:00.1/host5/vport-5:0-0/host6</path>
  <parent>scsi_host5</parent>
  <capability type='scsi_host'>
    <host>6</host>
    <unique_id>6</unique_id>
    <capability type='fc_host'>
      <wwnn>2101001b32a90002</wwnn>
      <wwpn>2101001b32a90003</wwpn>
      <fabric_wwn>2001000dec9877c1</fabric_wwn>
    </capability>
  </capability>
</device>

# virsh nodedev-list scsi_host
scsi_host0
scsi_host1
scsi_host2
scsi_host3
scsi_host4
scsi_host5
scsi_host6

scenario 1: define an scsi pool with scsi_host type adapter first,
then define another scsi pool with fc_host type adapter 

1. define an scsi pool with scsi_host type adapter
#cat scsi-pool.xml
<pool type='scsi'>
<name>scsi-pool</name>
<source>
<adapter type='scsi_host' name='scsi_host6'/>
</source>
<target>
<path>/dev/disk/by-path</path>
<permissions>
<mode>0700</mode>
<owner>0</owner>
<group>0</group>
</permissions>
</target>
</pool>

# virsh pool-define scsi-pool.xml 
Pool scsi-pool defined from scsi-pool.xml

[root@dell-pet105-04 yy]# virsh pool-start scsi-pool
Pool scsi-pool started

2. define another scsi pool with fc_host type adapter 
   missing parent attribute

cat fc-pool-1.xml
<pool type='scsi'>
<name>fc-pool-1</name>
<source>
<adapter type='fc_host' wwnn='2101001b32a90002' wwpn='2101001b32a90003'/>
</source>
<target>
<path>/dev/disk/by-path</path>
<permissions>
<mode>0700</mode>
<owner>0</owner>
<group>0</group>
</permissions>
</target>
</pool>

[root@dell-pet105-04 yy]# virsh pool-define fc-pool-1.xml 
Pool fc-pool-1 defined from fc-pool-1.xml

[root@dell-pet105-04 yy]# virsh pool-start fc-pool-1
Pool fc-pool-1 started

# service libvirtd status
Active: active (running)

3. define a fc pool with parent name
destroy/undefine fc-pool-1 first
# virsh pool-destroy fc-pool-1
Pool fc-pool-1 destroyed
# virsh pool-undefine fc-pool-1
Pool fc-pool-1 has been undefined

# cat fc-pool.xml 
<pool type='scsi'>
<name>fc-pool</name>
<source>
<adapter type='fc_host' parent='scsi_host5' wwnn='2101001b32a90002' wwpn='2101001b32a90003'/>
</source>
<target>
<path>/dev/disk/by-path</path>
<permissions>
<mode>0700</mode>
<owner>0</owner>
<group>0</group>
</permissions>
</target>
</pool>

# virsh pool-define fc-pool.xml 
Pool fc-pool defined from fc-pool.xml
# virsh pool-start fc-pool
Pool fc-pool started

# service libvirtd status
Active: active (running)

clean env

scenario 2: define an scsi pool with 'fc_host' adapter first
then define another scsi pool with 'scsi_host' adapter

1. create a vHBA

# virsh nodedev-create vHBA.xml 
Node device scsi_host7 created from vHBA.xml

# virsh nodedev-dumpxml scsi_host7
<device>
  <name>scsi_host7</name>
  <path>/sys/devices/pci0000:00/0000:00:0d.0/0000:04:00.1/host5/vport-5:0-1/host7</path>
  <parent>scsi_host5</parent>
  <capability type='scsi_host'>
    <host>7</host>
    <unique_id>7</unique_id>
    <capability type='fc_host'>
      <wwnn>2101001b32a90002</wwnn>
      <wwpn>2101001b32a90003</wwpn>
      <fabric_wwn>2001000dec9877c1</fabric_wwn>
    </capability>
  </capability>
</device>


[root@dell-pet105-04 yy]# virsh nodedev-list scsi_host
scsi_host0
scsi_host1
scsi_host2
scsi_host3
scsi_host4
scsi_host5
scsi_host7

2. define/start a scsi pool with adapter type as
'fc_host' and parent attribute missing

# cat fc-pool-1.xml 
<pool type='scsi'>
<name>fc-pool-1</name>
<source>
<adapter type='fc_host' wwnn='2101001b32a90002' wwpn='2101001b32a90003'/>
</source>
<target>
<path>/dev/disk/by-path</path>
<permissions>
<mode>0700</mode>
<owner>0</owner>
<group>0</group>
</permissions>
</target>
</pool>

# virsh pool-define fc-pool-1.xml 
Pool fc-pool-1 defined from fc-pool-1.xml

[root@dell-pet105-04 yy]# virsh pool-start fc-pool-1
Pool fc-pool-1 started

3.  define/start another scsi pool with 'scsi_host' adapter

# cat scsi-pool.xml 
<pool type='scsi'>
<name>scsi-pool</name>
<source>
<adapter type='scsi_host' name='scsi_host7'/>
</source>
<target>
<path>/dev/disk/by-path</path>
<permissions>
<mode>0700</mode>
<owner>0</owner>
<group>0</group>
</permissions>
</target>
</pool>

[root@dell-pet105-04 yy]# virsh pool-define scsi-pool.xml 
Pool scsi-pool defined from scsi-pool.xml

[root@dell-pet105-04 yy]# virsh pool-start scsi-pool
Pool scsi-pool started

# service libvirtd status
 Active: active (running)

After above steps, libvirtd works well. So set it verified.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-0323.html