Bug 1849469
| Summary: | [OSP16.1]The emulator thread processes are not placed on the cores from NovaComputeCpuSharedSet during real time deployment | ||
|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Pei Zhang <pezhang> |
| Component: | openstack-nova | Assignee: | OSP DFG:Compute <osp-dfg-compute> |
| Status: | CLOSED NOTABUG | QA Contact: | OSP DFG:Compute <osp-dfg-compute> |
| Severity: | unspecified | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 16.1 (Train) | CC: | chayang, dasmith, eglynn, jhakimra, jinzhao, juzhang, kchamart, lcapitulino, sbauza, sgordon, stephenfin, vromanso |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2020-06-22 10:47:23 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
(In reply to Pei Zhang from comment #0) .. > > Actual results: > emulatorpin thread is not located on an isolated core. ^^^ Sorry typo here: remove "not". > $ openstack flavor show rt_1vcpu.flavor
> +----------------------------+-----------------------------------------------
> -----------------------------------------------------------------------------
> -----------------------------------------------------+
> | Field | Value
> |
> +----------------------------+-----------------------------------------------
> -----------------------------------------------------------------------------
> -----------------------------------------------------+
> | OS-FLV-DISABLED:disabled | False
> |
> | OS-FLV-EXT-DATA:ephemeral | 0
> |
> | access_project_ids | None
> |
> | description | None
> |
> | disk | 42
> |
> | extra_specs | {'hw:cpu_policy': 'dedicated',
> 'hw:cpu_realtime': 'yes', 'hw:cpu_realtime_mask': '^0',
> 'hw:emulator_threads_policy': 'isolate', 'hw:mem_page_size': 'large',
> 'hw:pmu': 'False'} |
> | id | f208002f-79ca-40b8-af58-9a2d4b203424
> |
> | name | rt_1vcpu.flavor
> |
> | os-flavor-access:is_public | True
> |
> | properties | hw:cpu_policy='dedicated',
> hw:cpu_realtime='yes', hw:cpu_realtime_mask='^0',
> hw:emulator_threads_policy='isolate', hw:mem_page_size='large',
> hw:pmu='False' |
> | ram | 8192
> |
> | rxtx_factor | 1.0
> |
> | swap | 0
> |
> | vcpus | 2
> |
> +----------------------------+-----------------------------------------------
> -----------------------------------------------------------------------------
> -----------------------------------------------------+
This flavor is incorrect. You need to set 'hw:emulator_threads_policy' to 'share', not 'isolate'.
(In reply to Stephen Finucane from comment #2) > > $ openstack flavor show rt_1vcpu.flavor > > +----------------------------+----------------------------------------------- > > ----------------------------------------------------------------------------- > > -----------------------------------------------------+ > > | Field | Value > > | > > +----------------------------+----------------------------------------------- > > ----------------------------------------------------------------------------- > > -----------------------------------------------------+ > > | OS-FLV-DISABLED:disabled | False > > | > > | OS-FLV-EXT-DATA:ephemeral | 0 > > | > > | access_project_ids | None > > | > > | description | None > > | > > | disk | 42 > > | > > | extra_specs | {'hw:cpu_policy': 'dedicated', > > 'hw:cpu_realtime': 'yes', 'hw:cpu_realtime_mask': '^0', > > 'hw:emulator_threads_policy': 'isolate', 'hw:mem_page_size': 'large', > > 'hw:pmu': 'False'} | > > | id | f208002f-79ca-40b8-af58-9a2d4b203424 > > | > > | name | rt_1vcpu.flavor > > | > > | os-flavor-access:is_public | True > > | > > | properties | hw:cpu_policy='dedicated', > > hw:cpu_realtime='yes', hw:cpu_realtime_mask='^0', > > hw:emulator_threads_policy='isolate', hw:mem_page_size='large', > > hw:pmu='False' | > > | ram | 8192 > > | > > | rxtx_factor | 1.0 > > | > > | swap | 0 > > | > > | vcpus | 2 > > | > > +----------------------------+----------------------------------------------- > > ----------------------------------------------------------------------------- > > -----------------------------------------------------+ > > This flavor is incorrect. You need to set 'hw:emulator_threads_policy' to > 'share', not 'isolate'. It works as expected after setting with "share". Thanks Stephen. Best regards, Pei |
Description of problem: Deploy OpenStack 16.1 with real time compute node. After boot VM, the "emulatorpin cpuset=" is not placed on the cores from "NovaComputeCpuSharedSet:", but it's placed on an isolated core. Version-Release number of selected component (if applicable): $ cat containers-prepare-parameter.yaml parameter_defaults: NeutronMechanismDrivers: ovn ContainerImagePrepare: - push_destination: 192.168.24.1:8787 set: namespace: registry-proxy.engineering.redhat.com/rh-osbs name_prefix: rhosp16-openstack- name_sufix: '' tag: 16.1_20200611.1 rhel_version: 8.2 ... $ rpm -qa | grep openstack openstack-tripleo-puppet-elements-11.2.2-0.20200527003426.226ce95.el8ost.noarch openstack-heat-common-13.0.2-0.20200529053437.33972cc.el8ost.noarch openstack-heat-monolith-13.0.2-0.20200529053437.33972cc.el8ost.noarch python-openstackclient-lang-4.0.0-0.20200310193636.aa64eb6.el8ost.noarch puppet-openstack_extras-15.4.1-0.20200528113453.371931c.el8ost.noarch ansible-role-openstack-operations-0.0.1-0.20200311080930.274739e.el8ost.noarch python3-openstacksdk-0.36.3-0.20200424135113.c07350e.el8ost.noarch openstack-ironic-python-agent-builder-2.0.1-0.20200608173428.cb415ef.el8ost.noarch openstack-tripleo-heat-templates-11.3.2-0.20200616081525.396affd.el8ost.noarch puppet-openstacklib-15.4.1-0.20200403203429.5fdf43c.el8ost.noarch openstack-tripleo-common-11.3.3-0.20200611110653.f7715be.el8ost.noarch openstack-tripleo-common-containers-11.3.3-0.20200611110653.f7715be.el8ost.noarch openstack-heat-agents-1.10.1-0.20200311091123.96b819c.el8ost.noarch python3-openstackclient-4.0.0-0.20200310193636.aa64eb6.el8ost.noarch openstack-heat-api-13.0.2-0.20200529053437.33972cc.el8ost.noarch openstack-tripleo-image-elements-10.6.2-0.20200528043425.7dc0fa1.el8ost.noarch openstack-selinux-0.8.20-0.20200428133425.3300746.el8ost.noarch openstack-heat-engine-13.0.2-0.20200529053437.33972cc.el8ost.noarch openstack-tripleo-validations-11.3.2-0.20200611115251.08f469d.el8ost.noarch repo I was using: http://download.eng.bos.redhat.com/rcm-guest/puddles/OpenStack/16.1-RHEL-8/RHOS-16.1-RHEL-8-20200616.n.0/compose/ real time compute node version: 4.18.0-193.9.1.rt13.60.el8_2.x86_64 libvirt-client-6.0.0-24.module+el8.2.1+6997+c666f621.x86_64 qemu-kvm-4.2.0-27.module+el8.2.1+7092+9d345e72.x86_64 How reproducible: 100% Steps to Reproduce: 1. Deploy with real time compute node. Set NovaComputeCpuSharedSet: "2,4,6,8" as below: $ cat /home/stack/templates/compute-real-time.yaml parameter_defaults: ComputeRealTimeParameters: IsolCpusList: "1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31,30,28,26,24,22,20,18,16,14,12" NovaComputeCpuDedicatedSet: "1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31,30,28,26,24,22,20,18,16,14,12" NovaComputeCpuSharedSet: "2,4,6,8" KernelArgs: "default_hugepagesz=1G hugepagesz=1G hugepages=32" $ cat /home/stack/templates/rt_roles_data.yaml ... ############################################################################### # Role: ComputeRealTime # ############################################################################### - name: ComputeRealTime description: | Compute role that is optimized for real-time behaviour. When using this role it is mandatory that an overcloud-realtime-compute image is available and the role specific parameters IsolCpusList, NovaComputeCpuDedicatedSet, and NovaComputeCpuSharedSet are set according to the hardware of the real-time compute nodes. CountDefault: 1 networks: InternalApi: subnet: internal_api_subnet Tenant: subnet: tenant_subnet Storage: subnet: storage_subnet HostnameFormatDefault: '%stackname%-computerealtime-%index%' ImageDefault: overcloud-realtime-compute RoleParametersDefault: TunedProfileName: "realtime-virtual-host" KernelArgs: "default_hugepagesz=1G hugepagesz=1G hugepages=32" IsolCpusList: "1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31,30,28,26,24,22,20,18,16,14,12" NovaComputeCpuDedicatedSet: "1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31,30,28,26,24,22,20,18,16,14,12" NovaComputeCpuSharedSet: "2,4,6,8" # NovaLibvirtMemStatsPeriodSeconds: 0 $ openstack overcloud deploy --templates -r /home/stack/templates/rt_roles_data.yaml -e /home/stack/containers-prepare-parameter.yaml -e /home/stack/templates/compute-real-time.yaml -e /home/stack/templates/node-info.yaml --ntp-server 192.168.24.1 --log-file overcloud_deployment.log 2. Check NovaComputeCpuSharedSet is applied in the nova.conf well. [heat-admin@overcloud-computerealtime-0 ~]$ sudo grep ^cpu_shared_set /var/lib/config-data/puppet-generated/nova_libvirt/etc/nova/nova.conf cpu_shared_set=2,4,6,8 3. Create a falvor for real time named with "rt_1vcpu.flavor" $ openstack flavor show rt_1vcpu.flavor +----------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value | +----------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | OS-FLV-DISABLED:disabled | False | | OS-FLV-EXT-DATA:ephemeral | 0 | | access_project_ids | None | | description | None | | disk | 42 | | extra_specs | {'hw:cpu_policy': 'dedicated', 'hw:cpu_realtime': 'yes', 'hw:cpu_realtime_mask': '^0', 'hw:emulator_threads_policy': 'isolate', 'hw:mem_page_size': 'large', 'hw:pmu': 'False'} | | id | f208002f-79ca-40b8-af58-9a2d4b203424 | | name | rt_1vcpu.flavor | | os-flavor-access:is_public | True | | properties | hw:cpu_policy='dedicated', hw:cpu_realtime='yes', hw:cpu_realtime_mask='^0', hw:emulator_threads_policy='isolate', hw:mem_page_size='large', hw:pmu='False' | | ram | 8192 | | rxtx_factor | 1.0 | | swap | 0 | | vcpus | 2 | +----------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 4. Boot VM with rt_1vcpu.flavor. VM can boot well. $ openstack server create --flavor rt_1vcpu.flavor --image rhel8.2.0.z_rt_1vcpu --nic net-id=e9f426f8-1eeb-4703-8247-5244bc9ea8bc rhel8.2.0.z_rt_single_vm_1vcpu_test2 5. Check VM XML, the emulatorpin is placed on an isolated core (eg, it's 28), but not NovaComputeCpuSharedSet (eg, 2,4,6,8). This is not expected. [heat-admin@overcloud-computerealtime-0 ~]$ sudo virsh dumpxml instance-00000001 <cputune> <shares>2048</shares> <vcpupin vcpu='0' cpuset='26'/> <vcpupin vcpu='1' cpuset='22'/> <emulatorpin cpuset='28'/> <vcpusched vcpus='1' scheduler='fifo' priority='1'/> </cputune> Actual results: emulatorpin thread is not located on an isolated core. Expected results: emulatorpin thread should be located on cores from NovaComputeCpuSharedSet. Additional info: