Bug 1532683

Summary: Virtual Device Role Tagging does not work with interfaces of vnic-type: direct-physical
Product: Red Hat OpenStack Reporter: Ricardo Noriega <rnoriega>
Component: openstack-novaAssignee: Artom Lifshitz <alifshit>
Status: CLOSED ERRATA QA Contact: Joe H. Rahme <jhakimra>
Severity: medium Docs Contact:
Priority: medium    
Version: 10.0 (Newton)CC: alifshit, berrange, dasmith, eglynn, kchamart, lyarwood, sbauza, sgordon, srevivo, vromanso
Target Milestone: z10Keywords: Triaged, ZStream
Target Release: 10.0 (Newton)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: openstack-nova-14.1.0-35.el7ost Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-01-16 17:09:01 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1235009    

Description Ricardo Noriega 2018-01-09 15:18:00 UTC
Description of problem:

After tagging a PF (Physical Function), metadata service does not show info for that interface.

[fedora@test-sriov-vf-2 ~]$ curl http://169.254.169.254/openstack/latest/meta_data.json
{"random_seed": "mtvyCsG177Q0YnAEIRRVbphecF9sAAx69U0qr+gPij70FYNtWEGuuKeW3zM/9abrfa+PbDKnxXMUY4iG7sX9AkbcaZqWA8tZq/YganebTPpdEYoFxM06rbMypldYaw/zHgqRE7YLubCUcORAPRYo7xCZG4OoUQh/QhUNURbc8LkFYh76TjAd/V0KWvmf8PRPyTlFgydTKd0X0CWr4n1aql3bhHsuhSpSHIKIDcK1aY+57K8woh9yATJwc/QdPH7oEc9nyxyPPeNBxVEJ+nwD+BF9CDKt41xsKTewYJQyibLBh5X8/5JCOBKD6Koeoaw9EGSeNVstiLrW2NlHFPXcfSHFwiiRVxyniCGVo/85KB+KfeMnEVPvPzwajPd1Z/wGJD0LAXpSYsMEIyFc9/irsDKycwtIjciJ5pld4v1E18i3CGeJ1JJetsEQXra1WPkf8sprTW0vmKGxCe5SxJZrB9E37MI3TAOBFMAWpL7o6DsndLazgh3jlgh8eavdQKZ6iqllJqq3A75tufu5Duf0RApZBSQeCj8B2BegD/8g/Gm8KlAzRPzY9NNhQapalN0B5Gk5Q00M0M+hIQUdwcgJdIy7arbaKVXjgeB6EO+xiHUX1Kq+BYHQgcJTPGtSeXUyANTpUodUBF75QJsdfc7iPnVHlqzh+ybP6tlZ+AQ+pZ0=", "uuid": "0649f68a-33ea-407d-a699-13bc9975c66f", "availability_zone": "sriov", "keys": [{"data": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCWaUoyNJBibUkeDkdo2S6jVus38BaSShkMUfMkerokoG0Py2bTzR4C1N+8vTvwdYmu/e9l97muezNhp7hf0jpvMZv+0sGuQu4f0CZ0EhsjcW+KsfP8I65DH5S1syEKWV0vz41L/Kj+BnzQoEP4E1GH+F2vRsvvxrfQaXX7scFlzn87GgEhinagFnZ+Bej2FfL9l4uHDGujSaGRJBV0oTLvgY/+zcl0u39odOXaO+SLcysewejZC8jU+5YKcazRGC0l6zXLhdPrMbDo1YCCs6xwv/cb1pqMHfTIzWQr01IZdVgr1FB3mIKw9xR0g7zen8v4W1C4q2WtIfYOl39njJFp\n", "type": "ssh", "name": "undercloud-stack"}], "hostname": "test-sriov-vf-2", "launch_index": 0, "devices": [{"bus": "pci", "mac": "fa:16:3e:12:f2:16", "tags": ["ricky"], "type": "nic", "address": "0000:00:03.0"}], "public_keys": {"undercloud-stack": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCWaUoyNJBibUkeDkdo2S6jVus38BaSShkMUfMkerokoG0Py2bTzR4C1N+8vTvwdYmu/e9l97muezNhp7hf0jpvMZv+0sGuQu4f0CZ0EhsjcW+KsfP8I65DH5S1syEKWV0vz41L/Kj+BnzQoEP4E1GH+F2vRsvvxrfQaXX7scFlzn87GgEhinagFnZ+Bej2FfL9l4uHDGujSaGRJBV0oTLvgY/+zcl0u39odOXaO+SLcysewejZC8jU+5YKcazRGC0l6zXLhdPrMbDo1YCCs6xwv/cb1pqMHfTIzWQr01IZdVgr1FB3mIKw9xR0g7zen8v4W1C4q2WtIfYOl39njJFp\n"}, "project_id": "c3cb3b4993b54c28b7d68dea6a9b5887", "name": "test-sriov_vf_2"}[

How reproducible: Always


Steps to Reproduce:

neutron port-create sriov --binding:vnic-type direct-physical --name pf
nova --os-compute-api-version 2.32 boot test-sriov_vf_2 --availability-zone sriov --security-groups all-access --flavor m1.small --key-name undercloud-stack --image fedora   --nic net-name=default,tag="ricky" --nic port-id=88691de9-1a20-4f86-9931-e694786b5bdf,tag=ticky

Within the VM:
[fedora@test-sriov-vf-2 ~]$ curl http://169.254.169.254/openstack/latest/meta_data.json                                                                                    
{"random_seed": "mtvyCsG177Q0YnAEIRRVbphecF9sAAx69U0qr+gPij70FYNtWEGuuKeW3zM/9abrfa+PbDKnxXMUY4iG7sX9AkbcaZqWA8tZq/YganebTPpdEYoFxM06rbMypldYaw/zHgqRE7YLubCUcORAPRYo7xCZG4OoUQh/QhUNURbc8LkFYh76TjAd/V0KWvmf8PRPyTlFgydTKd0X0CWr4n1aql3bhHsuhSpSHIKIDcK1aY+57K8woh9yATJwc/QdPH7oEc9nyxyPPeNBxVEJ+nwD+BF9CDKt41xsKTewYJQyibLBh5X8/5JCOBKD6Koeoaw9EGSeNVstiLrW2NlHFPXcfSHFwiiRVxyniCGVo/85KB+KfeMnEVPvPzwajPd1Z/wGJD0LAXpSYsMEIyFc9/irsDKycwtIjciJ5pld4v1E18i3CGeJ1JJetsEQXra1WPkf8sprTW0vmKGxCe5SxJZrB9E37MI3TAOBFMAWpL7o6DsndLazgh3jlgh8eavdQKZ6iqllJqq3A75tufu5Duf0RApZBSQeCj8B2BegD/8g/Gm8KlAzRPzY9NNhQapalN0B5Gk5Q00M0M+hIQUdwcgJdIy7arbaKVXjgeB6EO+xiHUX1Kq+BYHQgcJTPGtSeXUyANTpUodUBF75QJsdfc7iPnVHlqzh+ybP6tlZ+AQ+pZ0=", "uuid": "0649f68a-33ea-407d-a699-13bc9975c66f", "availability_zone": "sriov", "keys": [{"data": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCWaUoyNJBibUkeDkdo2S6jVus38BaSShkMUfMkerokoG0Py2bTzR4C1N+8vTvwdYmu/e9l97muezNhp7hf0jpvMZv+0sGuQu4f0CZ0EhsjcW+KsfP8I65DH5S1syEKWV0vz41L/Kj+BnzQoEP4E1GH+F2vRsvvxrfQaXX7scFlzn87GgEhinagFnZ+Bej2FfL9l4uHDGujSaGRJBV0oTLvgY/+zcl0u39odOXaO+SLcysewejZC8jU+5YKcazRGC0l6zXLhdPrMbDo1YCCs6xwv/cb1pqMHfTIzWQr01IZdVgr1FB3mIKw9xR0g7zen8v4W1C4q2WtIfYOl39njJFp\n", "type": "ssh", "name": "undercloud-stack"}], "hostname": "test-sriov-vf-2", "launch_index": 0, "devices": [{"bus": "pci", "mac": "fa:16:3e:12:f2:16", "tags": ["ricky"], "type": "nic", "address": "0000:00:03.0"}], "public_keys": {"undercloud-stack": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCWaUoyNJBibUkeDkdo2S6jVus38BaSShkMUfMkerokoG0Py2bTzR4C1N+8vTvwdYmu/e9l97muezNhp7hf0jpvMZv+0sGuQu4f0CZ0EhsjcW+KsfP8I65DH5S1syEKWV0vz41L/Kj+BnzQoEP4E1GH+F2vRsvvxrfQaXX7scFlzn87GgEhinagFnZ+Bej2FfL9l4uHDGujSaGRJBV0oTLvgY/+zcl0u39odOXaO+SLcysewejZC8jU+5YKcazRGC0l6zXLhdPrMbDo1YCCs6xwv/cb1pqMHfTIzWQr01IZdVgr1FB3mIKw9xR0g7zen8v4W1C4q2WtIfYOl39njJFp\n"}, "project_id": "c3cb3b4993b54c28b7d68dea6a9b5887", "name": "test-sriov_vf_2"}[fedora@test-sriov-vf-2 ~]$                                

Ticky tag is missing!

With SRIOV VFs is working.


Actual results:

No tag "ticky" is present via metadata service.

Expected results:

Tag "ticky" should be available.

Comment 1 Ricardo Noriega 2018-01-09 18:52:10 UTC
Sosreports are stored here:

https://drive.google.com/drive/folders/1Kmeh0v--UIbSHQwurUMhfk0Kftf3wChZ?usp=sharing

Comment 3 Artom Lifshitz 2018-01-11 22:03:01 UTC
Haven't reproduced this at all, but based on just thinking about this and looking at the sosreports, I think this is a real bug.

I believe when we generate the metadata, we match the libvirt NIC (which has information like the MAC and PCI addresses) with its corresponding VirtualInterface nova object (which has information like the device tag) on MAC address, which is the only piece of unique information that both data structures share.

However, PF devices will appear in the libvirt XML as <hostdev> and not <interface>, and more importantly they appear *without* their MAC address:

    <hostdev mode="subsystem" type="pci" managed="yes">
      <source>
        <address bus="0x06" domain="0x0000" function="0x1" slot="0x00"/>
      </source>
    </hostdev>

Compared to the VIF JSON, in which the MAC appears:

{
    "address": "a0:36:9f:ba:04:1e",
    [...]
    "profile": {
        "pci_slot": "0000:06:00.1",
        "pci_vendor_info": "8086:1528",
        "physical_network": "sriov"
    },
    [...]
}

I have no ideas as to the fix for now, nor whether it will be backportable to OSP10, given that Newton has been EOLed upstream.

Comment 4 Artom Lifshitz 2018-01-30 19:48:22 UTC
Hey Ricardo,

While my patch is waiting for upstream reviews, I've made an OSP10 scratch build [1]. If you want you can play around with it in a test env, to confirm whether it actually fixes the bug or not. I want to be absolutely clear that the build was done with code that is merged absolutely nowhere, and thus comes with no promises whatsoever, except that I did my best to fix the bug.

Cheers!

[1] https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=15149578

Comment 5 Artom Lifshitz 2018-01-30 19:49:33 UTC
Actually, use [2] to download the RPMs, to save you a click from my previous link.

[2] https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=15149579

Comment 15 errata-xmlrpc 2019-01-16 17:09:01 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:0074