RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1708418 - tuned-profiles-sap: result of integer comparison in script.sh incorrect if one value is greater than 2**63-1
Summary: tuned-profiles-sap: result of integer comparison in script.sh incorrect if on...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: tuned
Version: 8.0
Hardware: Unspecified
OS: Unspecified
high
medium
Target Milestone: rc
: 8.1
Assignee: Jaroslav Škarvada
QA Contact: Robin Hack
URL:
Whiteboard:
Depends On:
Blocks: 1654309 1710589
TreeView+ depends on / blocked
 
Reported: 2019-05-09 20:28 UTC by Bernd Finger
Modified: 2020-11-14 11:32 UTC (History)
5 users (show)

Fixed In Version: tuned-2.12.0-0.1.rc1.el8
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-11-05 22:31:10 UTC
Type: Bug
Target Upstream Version:
Embargoed:
pm-rhel: mirror+


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github redhat-performance tuned pull 190 0 None closed Drop sap-hana-vmware profile, fix other SAP profiles 2021-02-05 05:30:17 UTC
Red Hat Product Errata RHBA-2019:3633 0 None None None 2019-11-05 22:31:29 UTC

Description Bernd Finger 2019-05-09 20:28:02 UTC
Description of problem:
Running the script /usr/lib/tuned/sap-netweaver/script.sh will set the values of kernel.shmall and kernel.shmmax lower than the RHEL 8 defaults of 18446744073692774399.

Version-Release number of selected component (if applicable):
tuned-profiles-sap-2.10.0-15.el8.noarch

How reproducible:
Run:
Install tuned-profiles-sap-2.10.0-15.el8.noarch.rpm, start tuned, and enable tuned profile sap-netweaver. Or install the package and run "/usr/lib/tuned/sap-netweaver/script.sh start" and check the values of kernel.shmall and kernel.shmmax.

Steps to Reproduce:
1) Display the current RHEL 8 defaults of kernel.shmall and kernel.shmmax:
# sysctl -n kernel.shmall
# sysctl -n kernel.shmmax

2) Install the tuned-profiles-sap package:
# yum install tuned-profiles-sap

3) Start and enable tuned and switch on profile "sap-netweaver":
# systemctl start tuned
# systemctl enable tuned
# tuned-adm profile sap-netweaver

4) Display the values of kernel.shmall and kernel.shmmax:
# sysctl -n kernel.shmall
# sysctl -n kernel.shmmax

Notice the differences to the outputs of step 1

You can run:
# tuned-adm off
to set the previous (default) values again.


Actual results (depends on the size of virtual memory):
# sysctl -n kernel.shmall
24379392
# sysctl -n kernel.shmmax
99857989632


Expected results:
# sysctl -n kernel.shmall
18446744073692774399
# sysctl -n kernel.shmmax
18446744073692774399


Additional info:
The issue is caused by bash not being able to handle integer values larger than 2**63-1.
See also:
https://unix.stackexchange.com/questions/117280/what-is-the-rationale-for-the-bash-shell-not-warning-you-of-arithmetic-overflow
https://stackoverflow.com/questions/51884432/how-to-handle-integers-in-bash-with-values-larger-than-263

Proposed fix:
1) Do no longer deliver file /usr/lib/tuned/sap-netweaver/script.sh and remove the last 3 lines of file /usr/lib/tuned/sap-netweaver/tuned.conf, as there is no need to lower the defaults of kernel.shmall and kernel.shmmax. SAP note 941735 (Version 11 from 04.05.2018) recommends these defaults as well.
2) Or, in case you want to keep the script, use bc to compare these values, as follows:
Replace lines 25 and 26 by the following lines:
        (( $(bc <<< "${SAP_SHMALL} > ${CURR_SHMALL}") )) && sysctl -w kernel.shmall="$SAP_SHMALL"
        (( $(bc <<< "${SAP_SHMMAX} > ${CURR_SHMMAX}") )) && sysctl -w kernel.shmmax="$SAP_SHMMAX"

# awk 'NR==25||NR==26{print}' /usr/lib/tuned/sap-netweaver/script.sh
        (( $(bc <<< "${SAP_SHMALL} > ${CURR_SHMALL}") )) && sysctl -w kernel.shmall="$SAP_SHMALL"
        (( $(bc <<< "${SAP_SHMMAX} > ${CURR_SHMMAX}") )) && sysctl -w kernel.shmmax="$SAP_SHMMAX"
# tuned-adm profile sap-netweaver
# sysctl -n kernel.shmall
18446744073692774399
# sysctl -n kernel.shmmax
18446744073692774399
# tuned-adm off
# sysctl -n kernel.shmall
18446744073692774399
# sysctl -n kernel.shmmax
18446744073692774399

Comment 1 Bernd Finger 2019-05-09 21:03:39 UTC
FYI - The issue is also present in Bash release 5 (5.0.0(1)-release).

Comment 2 Bernd Finger 2019-05-10 12:25:47 UTC
In case the script /usr/lib/tuned/sap-netweaver/script.sh should remain in place and part of tuned-profiles-sap, the package will depend on the "bc" package, which is part of BaseOS. Because the bc package is not part of the Minimal Install environment group, a dependency to bc should be explicitly added.

Comment 3 Ondřej Lysoněk 2019-05-10 13:38:48 UTC
Thanks for the report and the analysis!

Removing the script is fine with me. But it's funny that the script is there in the first place, because the RHEL-7 defaults are exactly the same.

Comment 4 Bernd Finger 2019-05-10 13:49:44 UTC
I discussed the issue with my colleague Frank Danapfel. There might be good reasons to keep the script, in case a customer has lowered the defaults of kernel.shmmax and kernel.shmall (for example due to a misconfiguration).

Also in BZ 1701394 which is about a similar issue regarding kernel.sem, the last entry was:
Thanks for info, we will probably add conditional rule to the Tuned profile, i.e. it will increase the value only if it is lower.

So maybe the best approach would be to keep the script and fix the error, and decide later if we want to remove it in future RHEL releases, for example based on experiences with customer cases.

Comment 5 Ondřej Lysoněk 2019-05-10 14:13:18 UTC
OK, I see two options then:
1. use bc in the existing script
2. using the Tuned's 'sysctl' plugin and a conditional rule instead of having a script do the work. However, we'll need a way to calculate the values of the parameters. To do that, we could either:
   a) Add support for arithmetics in tuned profiles. I think it would be best to create built-in functions for addition, multiplication, etc., such as ${add:value1:value2} and ${multiply:value1:value2}.
   b) Add specialized built-in functions to calculate the shmall/shmmax values.

I think I'd prefer option 2b. It would save us from adding the dependency on 'bc' and from having to run a bash script (which would speed up applying the profile a bit). I also prefer it over 2a, as I think 2b would be more readable.

Comment 6 Bernd Finger 2019-05-20 12:40:55 UTC
Would be great if the fix for this issue could be in RHEL 8.1.

Comment 7 Ondřej Lysoněk 2019-05-21 07:47:12 UTC
(In reply to Ondřej Lysoněk from comment #5)
> OK, I see two options then:
> 1. use bc in the existing script
> 2. using the Tuned's 'sysctl' plugin and a conditional rule instead of
> having a script do the work. However, we'll need a way to calculate the
> values of the parameters. To do that, we could either:
>    a) Add support for arithmetics in tuned profiles. I think it would be
> best to create built-in functions for addition, multiplication, etc., such
> as ${add:value1:value2} and ${multiply:value1:value2}.
>    b) Add specialized built-in functions to calculate the shmall/shmmax
> values.
> 
> I think I'd prefer option 2b. It would save us from adding the dependency on
> 'bc' and from having to run a bash script (which would speed up applying the
> profile a bit). I also prefer it over 2a, as I think 2b would be more
> readable.

Alternatively, we could just rewrite the script in python. That would be the simplest solution and we would not have to clutter up our function list with profile-specific functions.

Comment 8 Bernd Finger 2019-05-22 11:30:57 UTC
Rewriting the script in a language which supports larger integer values is certainly a good option and in this case should not take very long.

However, I suggest to no longer use the script but set shmmax and shmall to the RHEL 8 default of 18446744073692774399, for the following reasons:

The referenced SAP note 941735 "SAP memory management system for 64-bit Linux systems" says:

--->
kernel.shmmax
This kernel parameter specifies how large an individual SysV Shared Memory Segment is allowed to be. 
The value is specified in bytes, so "214748364800", for example, allows segments of 200 GB in size.
By default, the Linux distributions already set extremely large values (RHEL: 18446744073692774399, SLES: 18446744073709551615); we recommend that you retain these unchanged.

kernel.shmall
This kernel parameter specifies how much SysV Shared Memory is available in the Linux system as a whole. 
The value is specified in 4 KB blocks, so "52428800", for example, allows 200 GB of SysV shared memory in the entire Linux system.
By default, the Linux distributions already set extremely large values (RHEL: 18446744073692774399, SLES: 1152921504606846720); we recommend that you retain these unchanged.
<---

I confirmed with the author of this SAP note that this SAP note is valid for all 64-bit Linux distributions and for all SAP kernel versions from 6.40 to the most recent ones.

I also checked the change history of SAP note 941735 and found that the last version of that note which does not recommend to use the default very high values of 18446744073692774399 was version 8, dated 12-Nov-2009. The first version which recommends to use the default values was version 9, dated 09-Dec-2014. The current version is 11, dated 04-May-2018.

I am now waiting for a final confirmation that this recommendation is valid for all supported RHEL releases from my SAP contact and will then update this bug again.

Comment 9 Bernd Finger 2019-05-24 14:30:23 UTC
I just have been notified by our SAP contact in the SAP LinuxLab that the recommendation to use the default high value of 18446744073692774399 for shmmax and shmall is valid for all SAP NetWeaver installations on RHEL 7 and RHEL 8.

So from my point of view we can set kernel.shmmax and kernel.shmall to the value of 18446744073692774399 unconditionally
in file /usr/lib/tuned/sap-netweaver/tuned.conf for RHEL 7 and RHEL 8.

Regards,
Bernd

Comment 10 Ondřej Lysoněk 2019-05-24 14:52:24 UTC
Sounds good, thanks.

Comment 11 Ondřej Lysoněk 2019-05-30 15:20:13 UTC
Upstream pull request:
https://github.com/redhat-performance/tuned/pull/190

Comment 12 Robin Hack 2019-06-11 08:49:38 UTC
QA_ACK+ RHEL8.1

Comment 16 errata-xmlrpc 2019-11-05 22:31:10 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:3633


Note You need to log in before you can comment on or make changes to this bug.