Bug 1423006
| Summary: | Setting root device hints automatically with "configure boot --root-device smallest" doesn't work when wwn_vendor_extension is used | ||
|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Alexander Chuzhoy <sasha> |
| Component: | openstack-tripleo-common | Assignee: | Dmitry Tantsur <dtantsur> |
| Status: | CLOSED ERRATA | QA Contact: | mlammon |
| Severity: | medium | Docs Contact: | |
| Priority: | medium | ||
| Version: | 11.0 (Ocata) | CC: | athomas, bfournie, dsneddon, dtantsur, hbrock, jslagle, mburns, mlammon, racedoro, rhel-osp-director-maint, sasha, sclewis, slinaber, srevivo |
| Target Milestone: | z4 | Keywords: | Triaged, ZStream |
| Target Release: | 11.0 (Ocata) | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | openstack-tripleo-common-6.1.4-1.el7ost | Doc Type: | Bug Fix |
| Doc Text: |
Certain hardware contains hard drives that use the same WWN and differ only in a vendor extension. Consequently, the root device hints using WWN values would not select the correct disk. With this update, the WWN plus the vendor extension is taken into account, and no longer only considers the WWN when applying root device hints. As a result, you can now set root device hints using the 'configure boot' command for such disks.
|
Story Points: | --- |
| Clone Of: | Environment: | ||
| Last Closed: | 2018-02-13 16:31:14 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Using the "serial" hint also doesn't work:
I executed:
ironic node-update `ironic node-list|awk '/power/ {print $2}'|tail -2|head -n1` add properties/root_device='{"serial": "644a84202a62db001d27e9240f81562c"}'
It looks good in the output from ironic node-show:
[stack@undercloud ~]$ ironic node-show 6f503d18-e9b0-41c1-8f01-f0791088d2e7
+------------------------+--------------------------------------------------------------------------+
| Property | Value |
+------------------------+--------------------------------------------------------------------------+
| chassis_uuid | None |
| clean_step | {} |
| console_enabled | False |
| created_at | 2017-02-17T01:14:18+00:00 |
| driver | pxe_ipmitool |
| driver_info | {u'deploy_kernel': u'd6a3e8fa-6bd5-4113-ac6f-37574f81eef7', |
| | u'ipmi_address': u'10.19.184.37', u'deploy_ramdisk': |
| | u'cad0f4a2-1446-40d7-a7b6-1189ef7ed524', u'ipmi_password': u'******', |
| | u'ipmi_username': u'root'} |
| driver_internal_info | {u'agent_url': u'http://192.168.24.11:9999', u'root_uuid_or_disk_id': |
| | u'2457f6a3-948a-4a21-acad-1be45a974d2f', u'is_whole_disk_image': False} |
| extra | {u'hardware_swift_object': u'extra_hardware- |
| | 6f503d18-e9b0-41c1-8f01-f0791088d2e7'} |
| inspection_finished_at | None |
| inspection_started_at | None |
| instance_info | {u'root_gb': u'40', u'display_name': u'overcloud-cephstorage-0', |
| | u'image_source': u'd1fdc382-026f-4be2-83d9-e5e2f00fc517', |
| | u'capabilities': u'{"profile": "ceph-storage", "boot_option": "local"}', |
| | u'memory_mb': u'4096', u'vcpus': u'1', u'local_gb': u'76', |
| | u'configdrive': u'******', u'swap_mb': u'0', u'nova_host_id': |
| | u'undercloud.localdomain'} |
| instance_uuid | 5edfd1fd-7328-4a8c-80ed-c716841c2aa0 |
| last_error | None |
| maintenance | False |
| maintenance_reason | None |
| name | None |
| network_interface | |
| power_state | power on |
| properties | {u'cpu_arch': u'x86_64', u'root_device': {u'serial': |
| | u'644a84202a62db001d27e9240f81562c'}, u'cpus': u'12', u'capabilities': |
| | u'profile:ceph-storage,cpu_hugepages:true,ceph-storage_profile:1,cpu_txt |
| | :true,cpu_vt:true,compute_profile:1,boot_option:local,cpu_aes:true,cpu_h |
| | ugepages_1g:true', u'memory_mb': u'49152', u'local_gb': u'76'} |
| provision_state | active |
| provision_updated_at | 2017-02-17T01:48:25+00:00 |
| raid_config | |
| reservation | None |
| resource_class | |
| target_power_state | None |
| target_provision_state | None |
| target_raid_config | |
| updated_at | 2017-02-17T01:48:25+00:00 |
| uuid | 6f503d18-e9b0-41c1-8f01-f0791088d2e7 |
+------------------------+--------------------------------------------------------------------------+
This is the last disk in the output below:
[stack@undercloud ~]$ openstack baremetal introspection data save 6f503d18-e9b0-41c1-8f01-f0791088d2e7 | jq '.inventory.disks'
[
{
"size": 1979120091136,
"serial": "644a84202a62db001d27e91a0eec3dd7",
"rotational": true,
"vendor": "DELL",
"name": "/dev/sda",
"wwn_vendor_extension": "0x1d27e91a0eec3dd7",
"hctl": "0:2:0:0",
"wwn_with_extension": "0x644a84202a62db001d27e91a0eec3dd7",
"model": "PERC H710",
"wwn": "0x644a84202a62db00"
},
{
"size": 1979120091136,
"serial": "644a84202a62db001d27e91d0f14b234",
"rotational": true,
"vendor": "DELL",
"name": "/dev/sdb",
"wwn_vendor_extension": "0x1d27e91d0f14b234",
"hctl": "0:2:1:0",
"wwn_with_extension": "0x644a84202a62db001d27e91d0f14b234",
"model": "PERC H710",
"wwn": "0x644a84202a62db00"
},
{
"size": 1979120091136,
"serial": "644a84202a62db001d27e91f0f38b81f",
"rotational": true,
"vendor": "DELL",
"name": "/dev/sdc",
"wwn_vendor_extension": "0x1d27e91f0f38b81f",
"hctl": "0:2:2:0",
"wwn_with_extension": "0x644a84202a62db001d27e91f0f38b81f",
"model": "PERC H710",
"wwn": "0x644a84202a62db00"
},
{
"size": 1979120091136,
"serial": "644a84202a62db001d27e9210f5cd6e1",
"rotational": true,
"vendor": "DELL",
"name": "/dev/sdd",
"wwn_vendor_extension": "0x1d27e9210f5cd6e1",
"hctl": "0:2:3:0",
"wwn_with_extension": "0x644a84202a62db001d27e9210f5cd6e1",
"model": "PERC H710",
"wwn": "0x644a84202a62db00"
},
{
"size": 82896224256,
"serial": "644a84202a62db001d27e9240f81562c",
"rotational": true,
"vendor": "DELL",
"name": "/dev/sde",
"wwn_vendor_extension": "0x1d27e9240f81562c",
"hctl": "0:2:4:0",
"wwn_with_extension": "0x644a84202a62db001d27e9240f81562c",
"model": "PERC H710",
"wwn": "0x644a84202a62db00"
}
]
Yet, on the deployed node - wrong disk was selected as root.
Environment: python-ironicclient-1.11.0-1.el7ost.noarch puppet-ironic-10.3.0-1.el7ost.noarch instack-undercloud-6.0.0-0.20170220135156.d695df8.el7ost.noarch python-ironic-lib-2.5.2-1.el7ost.noarch openstack-ironic-api-7.0.1-0.20170223011831.a13ea4f.el7ost.noarch openstack-ironic-conductor-7.0.1-0.20170223011831.a13ea4f.el7ost.noarch python-ironic-inspector-client-1.11.0-1.el7ost.noarch openstack-ironic-inspector-5.0.1-0.20170214181727.babc2b6.el7ost.noarch openstack-ironic-common-7.0.1-0.20170223011831.a13ea4f.el7ost.noarch Don't reproduce now the issue in comment #1. Sasha, based on your last comment, do you think we can move this BZ to VERIFIED then? No, the "--root-device smallest" didn't work and the fix needs to be tested explicitly. FailedQA:
Environment:
openstack-ironic-api-7.0.0-3.el7ost.noarch
openstack-tripleo-heat-templates-6.0.0-3.el7ost.noarch
python-ironic-lib-2.5.2-1.el7ost.noarch
python-ironicclient-1.11.1-1.el7ost.noarch
openstack-ironic-inspector-5.0.0-3.el7ost.noarch
instack-undercloud-6.0.0-5.el7ost.noarch
openstack-ironic-common-7.0.0-3.el7ost.noarch
puppet-ironic-10.4.0-3.el7ost.noarch
python-ironic-inspector-client-1.11.0-1.el7ost.noarch
openstack-ironic-conductor-7.0.0-3.el7ost.noarch
Ran before deploying:
openstack baremetal introspection bulk start
openstack baremetal configure boot --root-device smallest
Wasn't even able to pings nodes with multiple disks (because the booted from the wrong disk).
Below is an entry of a machine with several disks:
[stack@undercloud ~]$ ironic node-show 1a835055-7016-4dd0-92c5-63b653bd63a3
+------------------------+--------------------------------------------------------------------------+
| Property | Value |
+------------------------+--------------------------------------------------------------------------+
| chassis_uuid | None |
| clean_step | {} |
| console_enabled | False |
| created_at | 2017-04-10T19:54:08+00:00 |
| driver | pxe_ipmitool |
| driver_info | {u'deploy_kernel': u'137d5bb5-e8e0-4104-b708-c6412f3b44dc', |
| | u'ipmi_address': u'10.19.184.35', u'deploy_ramdisk': u'089137cb-afab- |
| | 4d2c-87cf-86204e061747', u'ipmi_password': u'******', u'ipmi_username': |
| | u'root'} |
| driver_internal_info | {u'agent_url': u'http://192.168.24.6:9999', u'root_uuid_or_disk_id': |
| | u'e9a6940a-3777-4be4-a74a-f539d4a9a138', u'is_whole_disk_image': False} |
| extra | {u'hardware_swift_object': u'extra_hardware- |
| | 1a835055-7016-4dd0-92c5-63b653bd63a3'} |
| inspection_finished_at | None |
| inspection_started_at | None |
| instance_info | {u'root_gb': u'40', u'display_name': u'overcloud-cephstorage-0', |
| | u'image_source': u'5344347d-f752-407b-9b8f-6781805469a0', |
| | u'capabilities': u'{"profile": "ceph-storage", "boot_option": "local"}', |
| | u'memory_mb': u'4096', u'vcpus': u'1', u'local_gb': u'76', |
| | u'configdrive': u'******', u'swap_mb': u'0', u'nova_host_id': |
| | u'undercloud.localdomain'} |
| instance_uuid | e0f9e0f7-cc90-42a1-8a97-91c56f3a5de9 |
| last_error | None |
| maintenance | False |
| maintenance_reason | None |
| name | None |
| network_interface | |
| power_state | power on |
| properties | {u'cpu_arch': u'x86_64', u'root_device': {u'wwn': |
| | u'0x644a84202a631d00'}, u'cpus': u'12', u'capabilities': u'profile:ceph- |
| | storage,cpu_aes:true,cpu_hugepages:true,ceph-storage_profile:1,cpu_txt:t |
| | rue,compute_profile:1,boot_option:local,cpu_vt:true,cpu_hugepages_1g:tru |
| | e', u'memory_mb': u'49152', u'local_gb': 76} |
| provision_state | active |
| provision_updated_at | 2017-04-10T20:29:05+00:00 |
| raid_config | |
| reservation | None |
| resource_class | |
| target_power_state | None |
| target_provision_state | None |
| target_raid_config | |
| updated_at | 2017-04-10T20:29:05+00:00 |
| uuid | 1a835055-7016-4dd0-92c5-63b653bd63a3 |
+------------------------+--------------------------------------------------------------------------+
[stack@undercloud ~]$ openstack baremetal introspection data save 1a835055-7016-4dd0-92c5-63b653bd63a3 |jq .inventory.disks
[
{
"size": 1979120091136,
"serial": "644a84202a631d001d27f3810f3b47a9",
"rotational": true,
"vendor": "DELL",
"name": "/dev/sda",
"wwn_vendor_extension": "0x1d27f3810f3b47a9",
"hctl": "0:2:0:0",
"wwn_with_extension": "0x644a84202a631d001d27f3810f3b47a9",
"model": "PERC H710",
"wwn": "0x644a84202a631d00"
},
{
"size": 1979120091136,
"serial": "644a84202a631d001d27f3840f640704",
"rotational": true,
"vendor": "DELL",
"name": "/dev/sdb",
"wwn_vendor_extension": "0x1d27f3840f640704",
"hctl": "0:2:1:0",
"wwn_with_extension": "0x644a84202a631d001d27f3840f640704",
"model": "PERC H710",
"wwn": "0x644a84202a631d00"
},
{
"size": 1979120091136,
"serial": "644a84202a631d001d27f3860f87f2d4",
"rotational": true,
"vendor": "DELL",
"name": "/dev/sdc",
"wwn_vendor_extension": "0x1d27f3860f87f2d4",
"hctl": "0:2:2:0",
"wwn_with_extension": "0x644a84202a631d001d27f3860f87f2d4",
"model": "PERC H710",
"wwn": "0x644a84202a631d00"
},
{
"size": 1979120091136,
"serial": "644a84202a631d001d27f3880fac2428",
"rotational": true,
"vendor": "DELL",
"name": "/dev/sdd",
"wwn_vendor_extension": "0x1d27f3880fac2428",
"hctl": "0:2:3:0",
"wwn_with_extension": "0x644a84202a631d001d27f3880fac2428",
"model": "PERC H710",
"wwn": "0x644a84202a631d00"
},
{
"size": 82896224256,
"serial": "644a84202a631d001d27f38b0fd083d3",
"rotational": true,
"vendor": "DELL",
"name": "/dev/sde",
"wwn_vendor_extension": "0x1d27f38b0fd083d3",
"hctl": "0:2:4:0",
"wwn_with_extension": "0x644a84202a631d001d27f38b0fd083d3",
"model": "PERC H710",
"wwn": "0x644a84202a631d00"
}
]
Are you sure you remove the current content of properties/root_disk before trying this command? Mind pasting node properties before and after running it? This is a scripted installation:
1)
source stackrc && openstack baremetal import --json instackenv.json
2)
openstack baremetal introspection rule import ./rules.json
3)
openstack baremetal introspection bulk start
4)
openstack baremetal configure boot --root-device smallest
Typical node entry in the instackenv.json:
{
"pm_type":"pxe_ipmitool",
"mac":[
"00:0a:f7:7f:24:5e"
],
"cpu":"12",
"memory":"48091",
"disk":"1979",
"arch":"x86_64",
"pm_user":"USER",
"pm_password":"PASS",
"pm_addr":"X.X.X.X"
}
I see the problem now: all WWN are the same, we have to use WWN_WITH_EXTENSION instead. Dmitry, is there any patch upstream published that we can link to? No, sorry. I did not have time for it, and won't have it any soon. Root device hint issues will be investigated in OSP-13, adding this to stabilization list via https://bugzilla.redhat.com/show_bug.cgi?id=1473267 Ouch, sorry, this was reported against Ocata and has to be fixed there. The master and Pike patches were merged, an Ocata backport is proposed. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2018:0310 |
openstack-ironic: Setting root device hints automatically with "openstack baremetal configure boot --root-device smallest" doesn't work Environment: openstack-ironic-conductor-6.2.1-0.20170130230022.7dff244.el7ost.noarch openstack-ironic-inspector-4.2.1-0.20170131024823.51ff7de.el7ost.noarch puppet-ironic-10.2.0-0.20170127174021.d991f28.el7ost.noarch python-ironic-inspector-client-1.10.0-0.20170121033230.0eae82e.el7ost.noarch instack-undercloud-6.0.0-0.20170130174946.5388cd1.el7ost.noarch openstack-ironic-common-6.2.1-0.20170130230022.7dff244.el7ost.noarch python-ironic-lib-2.5.2-0.20170123115307.ace87b6.el7ost.noarch openstack-ironic-api-6.2.1-0.20170130230022.7dff244.el7ost.noarch python-ironicclient-1.11.0-0.20170127204609.f1f10cb.el7ost.noarch Steps to reproduce: 1) added 4 machines with 5 disks (4 * 2TB and 1 * 80G) 2) Run introspection on all 3) Checking the root disk on one node: [stack@undercloud ~]$ ironic node-show 9fcbd0b0-8215-463f-b12b-e1be8f61c8e8 |grep -A1 root | | u'ipmi_username': u'root'} | | driver_internal_info | {} | -- | properties | {u'cpu_arch': u'x86_64', u'root_device': {u'wwn': | | | u'0x644a84202a62da00'}, u'cpus': u'12', u'capabilities': | 4) See the collected data about disks for the same node: openstack baremetal introspection data save 9fcbd0b0-8215-463f-b12b-e1be8f61c8e8 | jq '.inventory.disks' [ { "size": 1979120091136, "serial": "644a84202a62da001d27ec020f9dd036", "rotational": true, "vendor": "DELL", "name": "/dev/sda", "wwn_vendor_extension": "0x1d27ec020f9dd036", "hctl": "0:2:0:0", "wwn_with_extension": "0x644a84202a62da001d27ec020f9dd036", "model": "PERC H710", "wwn": "0x644a84202a62da00" }, { "size": 1979120091136, "serial": "644a84202a62da001d27ec040fc6a34a", "rotational": true, "vendor": "DELL", "name": "/dev/sdb", "wwn_vendor_extension": "0x1d27ec040fc6a34a", "hctl": "0:2:1:0", "wwn_with_extension": "0x644a84202a62da001d27ec040fc6a34a", "model": "PERC H710", "wwn": "0x644a84202a62da00" }, { "size": 1979120091136, "serial": "644a84202a62da001d27ec070fea9fe1", "rotational": true, "vendor": "DELL", "name": "/dev/sdc", "wwn_vendor_extension": "0x1d27ec070fea9fe1", "hctl": "0:2:2:0", "wwn_with_extension": "0x644a84202a62da001d27ec070fea9fe1", "model": "PERC H710", "wwn": "0x644a84202a62da00" }, { "size": 1979120091136, "serial": "644a84202a62da001d27ec09100eea52", "rotational": true, "vendor": "DELL", "name": "/dev/sdd", "wwn_vendor_extension": "0x1d27ec09100eea52", "hctl": "0:2:3:0", "wwn_with_extension": "0x644a84202a62da001d27ec09100eea52", "model": "PERC H710", "wwn": "0x644a84202a62da00" }, { "size": 82896224256, "serial": "644a84202a62da001d27ec0b1033729c", "rotational": true, "vendor": "DELL", "name": "/dev/sde", "wwn_vendor_extension": "0x1d27ec0b1033729c", "hctl": "0:2:4:0", "wwn_with_extension": "0x644a84202a62da001d27ec0b1033729c", "model": "PERC H710", "wwn": "0x644a84202a62da00" } ] #note that the wwn is the same for all: openstack baremetal introspection data save 9fcbd0b0-8215-463f-b12b-e1be8f61c8e8 | jq '.inventory.disks' |grep 0x644a84202a62da00 "wwn_with_extension": "0x644a84202a62da001d27ec020f9dd036", "wwn": "0x644a84202a62da00" "wwn_with_extension": "0x644a84202a62da001d27ec040fc6a34a", "wwn": "0x644a84202a62da00" "wwn_with_extension": "0x644a84202a62da001d27ec070fea9fe1", "wwn": "0x644a84202a62da00" "wwn_with_extension": "0x644a84202a62da001d27ec09100eea52", "wwn": "0x644a84202a62da00" "wwn_with_extension": "0x644a84202a62da001d27ec0b1033729c", "wwn": "0x644a84202a62da00" On the overcloud deployment wrong disk was selected as the root disk.