Bug 1488369 - Manual reboot needed to apply tuned kernel arguments
Summary: Manual reboot needed to apply tuned kernel arguments
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates
Version: 11.0 (Ocata)
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: z5
: 11.0 (Ocata)
Assignee: Saravanan KR
QA Contact: Yariv
URL:
Whiteboard:
Depends On: 1475571
Blocks: 1488517
TreeView+ depends on / blocked
 
Reported: 2017-09-05 08:37 UTC by Eyal Dannon
Modified: 2018-07-18 13:59 UTC (History)
16 users (show)

Fixed In Version:
Doc Type: Known Issue
Doc Text:
Clone Of:
: 1488517 (view as bug list)
Environment:
Last Closed: 2018-07-18 13:59:48 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Eyal Dannon 2017-09-05 08:37:20 UTC
Description of problem:

After deployment of DPDK and SR-IOV environment, manual reboot is needed to apply tuned kernel arguments,

The proper profile is active. To me the reboot has been done before activating tuned’s profile:

[root@overcloud-compute-0 ~]# tuned-adm active
Current active profile: cpu-partitioning
[root@overcloud-compute-0 ~]# cat /proc/cmdline 
BOOT_IMAGE=/boot/vmlinuz-3.10.0-693.el7.x86_64 root=UUID=8a1ee696-e3f1-416c-a126-e1e3350310e7 ro console=tty0 console=ttyS0,115200n8 crashkernel=auto rhgb quiet intel_iommu=on default_hugepagesz=1GB hugepagesz=1G hugepages=32 iommu=pt isolcpus=1-21,23-43,45-65,67-87

After Manual reboot:

[root@overcloud-compute-0 ~]# cat /proc/cmdline 
BOOT_IMAGE=/boot/vmlinuz-3.10.0-693.el7.x86_64 root=UUID=8a1ee696-e3f1-416c-a126-e1e3350310e7 ro console=tty0 console=ttyS0,115200n8 crashkernel=auto rhgb quiet intel_iommu=on default_hugepagesz=1GB hugepagesz=1G hugepages=32 iommu=pt isolcpus=1-21,23-43,45-65,67-87 nohz=on nohz_full=1-21,23-43,45-65,67-87 rcu_nocbs=1-21,23-43,45-65,67-87 tuned.non_isolcpus=00000004,00001000,00400001 intel_pstate=disable nosoftlockup

Using the following templates as reference:
https://code.engineering.redhat.com/gerrit/gitweb?p=nfv-qe.git;a=tree;f=ospd-11-vlan-sriov-single-port-composable-roles;hb=docs


Version-Release number of selected component (if applicable):
OSPD 11
tuned 2.8

How reproducible:
Always

Steps to Reproduce:
1. deploy the environemt
2. Check /proc/cmdline
3. reboot the system
4. Check /proc/cmdline

Actual results:
Kernel params are missing

Expected results:
Should exists

Additional info:

Comment 1 Saravanan KR 2017-09-05 10:24:38 UTC
/var/log/tuned/tuned.log (with manual reboot - 2 reboots, first on cloud-init, second is manual)
http://chunk.io/krsacme/7e7eeed6c1574f3488bbe04547a24994

First reboot (cloud-init) is having below log
2017-09-05 05:14:48,657 INFO     tuned.daemon.daemon: terminating Tuned, rolling back all changes

When the reboot is triggered from first-boot script (which is run by cloud-init), the command "systemctl is-system-running" is returned as "starting", which is initiating the rollback of all changes.

Related change:
https://github.com/redhat-performance/tuned/commit/df9aa2f5c46e4db08a077081ca15b6da541b4514?diff=split#diff-ea63e0c5d4daa711fe01bc50e4db0145R151

Adding tuned team to comment on it.

Comment 2 Eyal Dannon 2017-09-05 13:53:18 UTC
I've tested it on OSP10.
- latest OSPd10 provides tuned2.8 as OSPd11, gives the same result
- OSPd10 z3 provides:
[root@compute-0 ~]# rpm -qa | grep tuned
tuned-2.7.1-3.el7_3.2.noarch
tuned-profiles-cpu-partitioning-2.7.1-5.el7fdp.noarch

Gives us which result:
[root@compute-0 ~]# cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-3.10.0-514.21.1.el7.x86_64 root=UUID=fa9e939e-9e3c-4f1c-a07c-3f506756ad7b ro console=tty0 console=ttyS0,115200n8 crashkernel=auto rhgb quiet default_hugepagesz=1GB hugepagesz=1G hugepages=32 iommu=pt intel_iommu=on isolcpus=1,2,3,4,5,16,17,18,19,20,21 nohz=on nohz_full=1,2,3,4,5,16,17,18,19,20,21 rcu_nocbs=1,2,3,4,5,16,17,18,19,20,21 intel_pstate=disable nosoftlockup

I suppose it's related to the tuned package.

Comment 3 Yariv 2017-09-05 18:52:37 UTC
Removing blocker flag added requires_doc_text ? as a known issue with w/a

Comment 4 Ondřej Lysoněk 2017-09-08 14:27:40 UTC
This is indeed a regression in Tuned, caused by the commit listed in comment#1. The fix is available here:
https://github.com/redhat-performance/tuned/pull/66

Comment 5 atelang 2017-09-21 03:35:24 UTC
Vijay, 
Please ack this for 11Z.

Thanks,
Anjali

Comment 6 Yariv 2017-10-16 07:40:25 UTC
Automation is Ready
https://github.com/redhat-openstack/tempest-nfv-plugin/blob/devel/tempest_plugin/tests/scenario/test_nfv_epa.py#L205

Vijay
Please devel_ack
Anjali
Please pm_ack

Comment 9 Sanjay Upadhyay 2018-01-29 09:54:59 UTC
tracking bug only, the tuned params have been verified as per comment - https://bugzilla.redhat.com/show_bug.cgi?id=1475571#c27

Moving this to QA for verifying from DFG:NFV perspective


Note You need to log in before you can comment on or make changes to this bug.