Bug 1415847 - [Docs][NFV] Fix documentation bugs in OVS DPDK documentation
Summary: [Docs][NFV] Fix documentation bugs in OVS DPDK documentation
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: documentation
Version: 10.0 (Newton)
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: async
: 10.0 (Newton)
Assignee: RHOS Documentation Team
QA Contact: Sandra McCann
URL:
Whiteboard:
Depends On:
Blocks: 1438643
TreeView+ depends on / blocked
 
Reported: 2017-01-23 23:22 UTC by Andreas Karis
Modified: 2020-05-14 15:34 UTC (History)
13 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-04-04 02:23:01 UTC
Target Upstream Version:


Attachments (Terms of Use)

Description Andreas Karis 2017-01-23 23:22:11 UTC
Description of problem:
Please fix documentation bugs in OVS DPDK documentation
https://access.redhat.com/documentation/en/red-hat-openstack-platform/10/single/network-functions-virtualization-configuration-guide/#configure_ovs_dpdk_in_red_hat_openstack_platform

Additional info:

i) Modify the network-environment.yaml file:

    Add the first-boot.yaml to set the kernel parameters:

    OS::TripleO::NodeUserData: /home/stack/templates/first-boot.yaml

===> this should better be (otherwise it really isn't very clear):

 Modify the network-environment.yaml file:

    Add the first-boot.yaml to set the kernel parameters. Add the following line under `resource_registry`:

~~~
  resource_registry:
    OS::TripleO::NodeUserData: /home/stack/templates/first-boot.yaml
~~~

ii)  Add the ComputeKernelArgs parameters to the default grub file:

ComputeKernelArgs: "intel_iommu=on default_hugepagesz=1GB hugepagesz=1G hugepages=12

=====> this should rather be (otherwise it isn't clear):

Add the ComputeKernelArgs parameters to the default grub file. Add this to the `default_parameters` section:
~~~
default_parameters:
  ComputeKernelArgs: "intel_iommu=on default_hugepagesz=1GB hugepagesz=1G hugepages=12
~~~

iii)   Add the post-install.yaml file to set the DPDK arguments:

OS::TripleO::NodeUserData: <relative directory>/post-install.yaml

====> this is a documentation bug. this should be:

OS::TripleO::NodeExtraConfigPost:

iv)  Add a list or range of physical CPU cores to be reserved for virtual machine processes:

NovaVcpuPinSet: ['4-12','^8'] will reserve cores from 4-12 excluding 8

===> this lacks a comment after the ']'


++++++++++++++++++++++++++++++++++++++++++++++++

Please note: I stopped reading the doc after this, it may be worth going through the whole doc once more, given the number of issues with it.

Thanks,

Andreas

Comment 1 Lucy Bopf 2017-01-24 04:21:36 UTC
Assigning to Deepti for review.

Comment 2 Andreas Karis 2017-01-26 18:19:16 UTC
There is a whole lot of other documentation issues:

*) https://access.redhat.com/documentation/en/red-hat-openstack-platform/10/single/network-functions-virtualization-planning-guide/ 
section 5 is a prerequisite for the other guide, but there's no clear connection between the 2. Users will simply start with the second guide, and will ignore the first.

*) https://access.redhat.com/documentation/en/red-hat-openstack-platform/10/single/network-functions-virtualization-planning-guide/ 

"Role Definition" -> file roles_data.yaml needs to be copied from the originial templates into the template folder. The guide does not mention this, nor does it mention where to find the roles_data.yaml file

*) https://access.redhat.com/documentation/en/red-hat-openstack-platform/10/single/network-functions-virtualization-planning-guide/ 

Environment Template -> 
does not mention where to find neutron-ovs-dpdk-agent.yaml and that it needs to be copied
does not explain how to create the computeovsdpdk.yaml file as a copy of compute.yaml

does not explain the port mappings, e.g.:
~~~
 # Port assignments for the compute role
  OS::TripleO::PinningCompute::Ports::ExternalPort: /usr/share/openstack-tripleo-heat-templates/network/ports/noop.yaml
  OS::TripleO::PinningCompute::Ports::InternalApiPort: /usr/share/openstack-tripleo-heat-templates/network/ports/internal_api.yaml
  OS::TripleO::PinningCompute::Ports::StoragePort: /usr/share/openstack-tripleo-heat-templates/network/ports/storage.yaml
  OS::TripleO::PinningCompute::Ports::StorageMgmtPort: /usr/share/openstack-tripleo-heat-templates/network/ports/noop.yaml
  OS::TripleO::PinningCompute::Ports::TenantPort: /usr/share/openstack-tripleo-heat-templates/network/ports/tenant.yaml
  #OS::TripleO::PinningCompute::Ports::ManagementPort: /usr/share/openstack-tripleo-heat-templates/network/ports/management.yaml
  OS::TripleO::PinningCompute::Net::SoftwareConfig: nic-configs/pinningcompute.yaml
~~~

Comment 3 Andreas Karis 2017-01-26 18:27:28 UTC
flavor definition and counts are missing from the parameter_defaults .. while the count is not bad, the flavor needs to be set

parameter_defaults:
  OvercloudPinningComputeFlavor: pinningcompute
  PinningComputeCount: 1

Comment 4 Andreas Karis 2017-01-26 21:43:24 UTC
This is minor, but it's still not pretty:
~~~
(...)
 boot_config:
    type: OS::Heat::CloudConfig
    properties:
      cloud_config:
        yum_repos:
          # Overcloud images deployed without any repos.
          # In order to install required tuned profile an activate it, we should create FDP repo.
          rhelosp-rhel-7-fast-datapth:
            name: Fast Datapath packages
            baseurl: http://pulp.dist.prod.ext.phx2.redhat.com/content/dist/rhel/server/7/7Server/x86_64/fast-datapath/os/
            enabled: true
            gpgcheck: false
          rhel-7-common:
            name: RHEL 7 Common
            baseurl: http://pulp.dist.prod.ext.phx2.redhat.com
(...)
~~~

Should be called:
~~~
rhelosp-rhel-7-fast-datapth:
~~~

Comment 5 Andreas Karis 2017-01-26 21:43:48 UTC
Should be called:
~~~
rhelosp-rhel-7-fast-datapath:
~~~

Comment 6 Andreas Karis 2017-01-26 21:53:20 UTC
Also, we are enabling these repos:
~~~
      cloud_config:
        yum_repos:
          # Overcloud images deployed without any repos.
          # In order to install required tuned profile an activate it, we should create FDP repo.
          rhelosp-rhel-7-fast-datapth:
            name: Fast Datapath packages
            baseurl: http://pulp.dist.prod.ext.phx2.redhat.com/content/dist/rhel/server/7/7Server/x86_64/fast-datapath/os/
            enabled: true
            gpgcheck: false
          rhel-7-common:
            name: RHEL 7 Common
            baseurl: http://pulp.dist.prod.ext.phx2.redhat.com/content/dist/rhel/server/7/7Server/$basearch/rh-common/os/
            enabled: 1
            gpgcheck: 0
          red-hat-enterprise-linux:
            name: Red Hat Enterprise Linux $releasever - $basearch - Server
            baseurl: http://pulp.dist.prod.ext.phx2.redhat.com/content/dist/rhel/server/7/7Server/x86_64/os/
            enabled: 1
            gpgcheck: 0
~~~

So we enable the fast-datapath repo:
~~~
 yum install -y tuned-profiles-cpu-partitioning
~~~

Instead, we should be enabling:
~~~
rhel-7-server-openstack-10-rpms 
~~~
because all customers with an OpenStack license have access to the above.



Also, what are internal URLs doing in a customer facing documentation?
http://pulp.dist.prod.ext.phx2.redhat.com/content/dist/rhel/server/7/7Server/x86_64/fast-datapath/os/

Comment 7 Deepti Navale 2017-01-31 03:41:03 UTC
Emailed Yariv with a couple of queries re. some of the questions. Waiting for a response.

Comment 11 Deepti Navale 2017-02-14 02:05:05 UTC
Hey Andreas, 

I've updated the guides based on your feedback. Comment 8 and comment 10 have the updated drafts. Please verify and let me know if you have more feedback. 

Thanks, 
Deepti

Comment 12 Andreas Karis 2017-02-14 21:43:52 UTC
Hello,

I am forwarding the new documentation to one of our customers for validation.

However, I just read over it quickly, and there still is no obvious link between the Config guide and the Planning guide.

The planning guide defines a role for DPDK:
"The following section lists to steps to create, add and modify a OVS-DPDK customized role"

It should be made clear that customers should create this custom role before creating the rest of the configuration, and that the additional parameters in section "3.1. Configure OVS-DPDK in Red Hat OpenStack Platform", "1. Modify the network-environment.yaml file: ", "d. Provide a list of cores that can be used as DPDK PMDs in the format - [allowed_pattern: "'[0-9,-]+'"]: " will only work if a role is created that contains the ComputeNeutronOvsDpdkAgent service. If a customer does not either modify the actual Compute role, or create the DPDK compute role, these parameters will have no effect.

Therefore, I personally think that the role creation should be moved to the configuration guide, and that the parameters in section "Add the post-install.yaml file to set the DPDK arguments: " should go into an environment file which is dedicated for the new role (however, this is optional, and they can be put into the network-environment.yaml).

2 other minor things:
"Add the post-install.yaml file to set the DPDK arguments: " does not say that it should go into the resource_registry: section
"Provide a list of cores that can be used as DPDK PMDs in the format - [allowed_pattern: "'[0-9,-]+'"]: " does not say that it should go under the parameter_defaults: section

- Andreas

Comment 13 Andreas Karis 2017-02-15 16:25:25 UTC
Hi,

I created an knowledge base article based upon the 2 guides and instructions which I gave to a customer who successfully set this up in his environment. I don't have an environment to test this article, so I hope that this is correct:
https://access.redhat.com/solutions/2930291

Note that I take a different, step by step approach that contrasts the structure of the current guide, where users have to read back and forth to understand what they need to do. The knowledge base article is meant to be read and followed top to bottom.

Do you have a test environment where I could test/verify my suggestions?

Regards,

Andreas

Comment 16 Andreas Karis 2017-02-17 16:09:10 UTC
2 new things about https://access.redhat.com/documentation/en/red-hat-openstack-platform/10/single/network-functions-virtualization-configuration-guide/

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

less critical:

A.2. Sample OVS-DPDK YAML Files
A.2.1. network-environment.yaml

contains a duplication of the last line:
~~~
  # Ex. HostCpusList: '4-12' will tune cores from 4-12
  HostCpusList: "2,4,6,8,10,12,14,18,20,22,24,26,28,30"
  # Ex. HostCpusList: '4-12' will tune cores from 4-12
  HostCpusList: "2,4,6,8,10,12,14,18,20,22,24,26,28,30"
~~~

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
critical:

And note that the only place where the following, critical parameters NeutronDatapathType and NeutronVhostuserSocketDir ever appear are in the example file in section:
A.2. Sample OVS-DPDK YAML Files
A.2.1. network-environment.yaml

They don't appear anywhere else, just search for NeutronDatapathType and NeutronVhostuserSocketDir in the single page version: https://access.redhat.com/documentation/en/red-hat-openstack-platform/10/single/network-functions-virtualization-configuration-guide/


Note that not setting the above variables will lead to:
~~~
Error: Datapath type for ovs agent must be set when DPDK is enabled at /etc/puppet/modules/neutron/manifests/agents/ml2/ovs.pp:193 on node overcloud-compute-dpdk-1.localdomain\u001b[0m\n\u001b[1;31mError: Datapath type for ovs agent must be set when DPDK is enabled at /etc/puppet/modules/neutron/manifests/agents/ml2/ovs.pp:193 on node overcloud-compute-dpdk-1.localdomain\u001b[0m\n", 
~~~

The error is coming from /etc/puppet/modules/neutron/manifests/agents/ml2/ovs.pp:
~~~
    192   if $enable_dpdk and is_service_default($datapath_type) {
    193     fail('Datapath type for ovs agent must be set when DPDK is enabled')
    194   }
~~~
Where on of the parameters to the class is:
~~~
    153 class neutron::agents::ml2::ovs (
(...)
    174   $datapath_type              = $::os_service_default,
(...)
~~~

Comment 17 Andreas Karis 2017-02-17 16:09:32 UTC
moving this back to assigned state

Comment 18 Franck Baudin 2017-02-24 11:05:27 UTC
These values are positioned in /environments/neutron-ovs-dpdk.yaml:
.../...
  NeutronDatapathType: "netdev"
  NeutronVhostuserSocketDir: "/var/run/openvswitch"
.../...

As long as this file is included in the top environment YAML file, you should be good.

Comment 19 Andreas Karis 2017-02-24 15:20:54 UTC
Hi, 

Totally my fault. I did not see that /usr/share/openstack-tripleo-heat-templates/environments/neutron-ovs-dpdk.yaml  was included. Thanks :-)

Oh, and it also contains the service overwrite, which is perfect :-) So these 2 complaints were invalid, sorry about that!

resource_registry:
  OS::TripleO::Services::ComputeNeutronOvsAgent: ../puppet/services/neutron-ovs-dpdk-agent.yaml

- Andreas

Comment 20 Andreas Karis 2017-02-24 20:18:20 UTC
Next thing ....

Please never overwrite unit files in /usr/lib/systemd directly ... either, copy the file to /etc/systemd and modify it there, or even better, use drop-in files ...
~~~
if [[ $(hostname) == *$FORMAT* ]] ; then
              if [ -f /usr/lib/systemd/system/openvswitch-nonetwork.service ]; then
                ovs_service_path="/usr/lib/systemd/system/openvswitch-nonetwork.service"
              elif [ -f /usr/lib/systemd/system/ovs-vswitchd.service ]; then
                ovs_service_path="/usr/lib/systemd/system/ovs-vswitchd.service"
              fi
              grep -q "RuntimeDirectoryMode=.*" $ovs_service_path
              if [ "$?" -eq 0 ]; then
                sed -i 's/RuntimeDirectoryMode=.*/RuntimeDirectoryMode=0775/' $ovs_service_path
              else
                echo "RuntimeDirectoryMode=0775" >> $ovs_service_path
              fi
              grep -Fxq "Group=qemu" $ovs_service_path
              if [ ! "$?" -eq 0 ]; then
                echo "Group=qemu" >> $ovs_service_path
              fi
              grep -Fxq "UMask=0002" $ovs_service_path
              if [ ! "$?" -eq 0 ]; then
                echo "UMask=0002" >> $ovs_service_path
              fi
              ovs_ctl_path='/usr/share/openvswitch/scripts/ovs-ctl'
              grep -q "umask 0002 \&\& start_daemon \"\$OVS_VSWITCHD_PRIORITY\"" $ovs_ctl_path
              if [ ! "$?" -eq 0 ]; then
                sed -i 's/start_daemon \"\$OVS_VSWITCHD_PRIORITY.*/umask 0002 \&\& start_daemon \"$OVS_VSWITCHD_PRIORITY\" \"$OVS_VSWITCHD_WRAPPER\" \"$@\"/' $ovs_ctl_path
              fi

              tuned_service=/usr/lib/systemd/system/tuned.service
              grep -q "network.target" $tuned_service
              if [ "$?" -eq 0 ]; then
                sed -i '/After=.*/s/network.target//g' $tuned_service
              fi
              grep -q "Before=.*network.target" $tuned_service
              if [ ! "$?" -eq 0 ]; then
                grep -q "Before=.*" $tuned_service
                if [ "$?" -eq 0 ]; then
                  sed -i 's/^\(Before=.*\)/\1 network.target openvswitch.service/g' $tuned_service
                else
                  sed -i '/After/i Before=network.target openvswitch.service' $tuned_service
                fi
              fi
~~~



https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/System_Administrators_Guide/sect-Managing_Services_with_systemd-Unit_Files.html


 Services installed on the system come with default unit files that are stored in the /usr/lib/systemd/system/ directory. System Administrators should not modify these files directly, therefore any customization must be confined to configuration files in the /etc/systemd/system/ directory. Depending on the extent of the required changes, pick one of the following approaches:

    Create a directory for supplementary configuration files at /etc/systemd/system/unit.d/. This method is recommended for most use cases. It enables extending the default configuration with additional functionality, while still referring to the original unit file. Changes to the default unit introduced with a package upgrade are therefore applied automatically. See Section 9.6.4, “Extending the Default Unit Configuration” for more information.
    Create a copy of the original unit file /usr/lib/systemd/system/ in /etc/systemd/system/ and make changes there. The copy overrides the original file, therefore changes introduced with the package update are not applied. This method is useful for making significant unit changes that should persist regardless of package updates. See Section 9.6.4, “Overriding the Default Unit Configuration” for details.

Comment 21 Deepti Navale 2017-02-28 05:43:59 UTC
NFV Configuration guide - Includes the following updates:

* new sections to create flavor and deploy instance post configuration for SRIOV and OVS-DPDK and a known limitations section for OVS-DPDK
* a note to address that for OVS-DPDK the bridges should be of type ovs_user_bridge for BZ1390065 in the config procedure
* updates to the YAML files to include latest changes
* updates to the overview to include tuned profile

CP link: https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/10/html-single/network_functions_virtualization_configuration_guide/

NFV Planning guide - Includes the following updates:

* new Hardware Capacities, Hardware Topology sections
* updates to the Managing Deployments section based on BZ#1415847
* updates to the Networking Considerations chapter
* new images for Hardware Topology for SRIOV, SRIOV with HCI and OVS-DPDK deployments

CP link: https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/10/html-single/network_functions_virtualization_planning_guide/

Comment 22 Deepti Navale 2017-02-28 05:47:56 UTC
All the smaller issues listed in the bug comments have been addressed. 

Changing assignee to rhos-docs for now. And adding a NEEDINFO for Derek until there is a confirmation on the next steps for the restructuring issue.

Comment 23 Derek 2017-04-03 14:06:36 UTC
Met with Andreas to review his input.  We'll document that and review with the NFV team.  Given the feature workload in 10.z and 11, I expect that any structural changes will be targeted for OSP 12.

Comment 24 Lucy Bopf 2017-04-04 02:23:01 UTC
Deepti did make some significant changes for the RHOSP 10 book as part of this bug, so I'd prefer to close it now and use a new tracker for the structural changes.

I've raised bug 1438643 for the structural changes, and set the target release for 12.

Moving to CLOSED.


Note You need to log in before you can comment on or make changes to this bug.