Bug 1088046 - JON storage node script displays error ./cassandra: numactl: not found
Summary: JON storage node script displays error ./cassandra: numactl: not found
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: JBoss Operations Network
Classification: JBoss
Component: Launch Scripts, Storage Node
Version: JON 3.2
Hardware: Unspecified
OS: Unspecified
unspecified
low
Target Milestone: CR03
: JON 3.3.1
Assignee: Simeon Pinder
QA Contact: Sunil Kondkar
URL:
Whiteboard:
Depends On: 1193447
Blocks: 1206641
TreeView+ depends on / blocked
 
Reported: 2014-04-15 22:48 UTC by Larry O'Leary
Modified: 2018-12-05 18:11 UTC (History)
6 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2015-02-27 19:58:33 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 987120 0 unspecified CLOSED Storage node start script fails on Solaris 2021-02-22 00:41:40 UTC
Red Hat Knowledge Base (Solution) 791213 0 None None None Never

Internal Links: 987120

Description Larry O'Leary 2014-04-15 22:48:47 UTC
Description of problem:
When launching the storage node script using rhqctl the output on the console indicates that numactl is missing:

./cassandra: numactl: not found

This message should be suppressed.

Version-Release number of selected component (if applicable):
3.2.0.GA

How reproducible:
Always

Steps to Reproduce:
1. Launch rhqctl install or rhqctl start (assuming storage node is included) on a platform that numactl is missing and for which the "which" command returns 0 no matter what such as on Solaris.

Actual results:
...
17:15:34,079 INFO  [org.rhq.storage.installer.StorageInstaller] Installation of the storage node is complete
17:15:34,110 INFO  [org.rhq.server.control.command.Install] The storage node installer has finished with an exit value of 0
./cassandra: numactl: not found
 INFO 17:15:35,161 Logging initialized
 INFO 17:15:35,190 32bit JVM detected.  It is recommended to run Cassandra on a 64bit JVM for better performance.
...

Expected results:
No ./cassandra: numactl: not found message is displayed

Additional info:
This is because the cassandra script does the following:

    # If numactl is available, use it. For Cassandra, the priority is to
    # avoid disk I/O. Even for the purpose of CPU efficiency, we don't
    # really have CPU<->data affinity anyway. Also, empirically test that numactl
    # works before trying to use it (CASSANDRA-3245).
    NUMACTL_ARGS="--interleave=all"
    if which numactl >/dev/null 2>/dev/null && numactl $NUMACTL_ARGS ls / >/dev/null 2>/dev/null
    then
        NUMACTL="numactl $NUMACTL_ARGS"
    else
        NUMACTL=""
    fi


The expectation is that if numactl isn't available, the which numactl would return with exit code of 1 causing the evaluation in the *if* statement to jump to its *elxe* block. However, it appears that *which* may not always return a non-zero exit code in the event the requested command can not be found. Therefore, the script continues with the execution of numactl $NUMACTL_ARGS ls which causes the output of "./cassandra: numactl: not found" and returns an exit code of 1. Therefore, the script does jump to its *else* block as expected but essentially the *which* command has no affect and therefore does not do what was intended.

I am not certain if which is expected to return a non-zero or not. But in the case of Solaris 10 I can see that the type command does what we want. Therefore, the following seems to be a valid solution assuming type is portable: 


    # If numactl is available, use it. For Cassandra, the priority is to
    # avoid disk I/O. Even for the purpose of CPU efficiency, we don't
    # really have CPU<->data affinity anyway. Also, empirically test that numactl
    # works before trying to use it (CASSANDRA-3245).
    NUMACTL_ARGS="--interleave=all"
    if type numactl >/dev/null 2>/dev/null && numactl $NUMACTL_ARGS ls / >/dev/null 2>/dev/null
    then
        NUMACTL="numactl $NUMACTL_ARGS"
    else
        NUMACTL=""
    fi


This issue seems to have been captured in RHQ as bug 987120 but closed as verified. Doesn't seem anything was actually done though. To be clear, this issue is only cosmetic as the script does the right thing in the end even if it does display bad information on some platforms.

Comment 1 Simeon Pinder 2014-09-29 08:12:58 UTC
Moving into ER05 as didn't make the ER04 cut.

Comment 3 Libor Zoubek 2014-11-19 15:08:04 UTC
branch:  master
link:    https://github.com/rhq-project/rhq/commit/0651c1591
time:    2014-11-19 16:06:42 +0100
commit:  0651c15919b6d89fa81e1f2d1781eedd46277f50
author:  Libor Zoubek - lzoubek
message: Bug 1088046 - JON storage node script displays error ./cassandra:
         numactl: not found
         Changed cassandra start script to use type instead of which to
         detect numactl - 'type' behaves correctly on Linux,Mac and
         Solaris

Comment 4 Michael Burman 2015-01-15 08:33:33 UTC
Cherry-picked to release/jon3.3.x:

commit a028f1db3ce3f6f3ec1aec36c19a23e034ae610f
Author: Libor Zoubek <lzoubek>
Date:   Wed Nov 19 16:03:41 2014 +0100

    Bug 1088046 - JON storage node script displays error ./cassandra: numactl:
    not found
    
    Changed cassandra start script to use type instead of which to detect
    numactl - 'type' behaves correctly on Linux,Mac and Solaris
    
    (cherry picked from commit 0651c15919b6d89fa81e1f2d1781eedd46277f50)

Comment 5 Simeon Pinder 2015-01-26 08:15:15 UTC
Moving to ON_QA as available for test with the latest 3.3.1.ER01 bits from here:
http://download.devel.redhat.com/brewroot/packages/org.jboss.on-jboss-on-parent/3.3.0.GA/12/maven/org/jboss/on/jon-server-patch/3.3.0.GA/jon-server-patch-3.3.0.GA.zip

Comment 6 Sunil Kondkar 2015-01-30 10:56:43 UTC
Tested on 3.3.1.ER01 (JON 3.3.0.GA Update 01)
The error './cassandra: numactl: not found' is observed while starting storage node after applying JON 3.3.0.GA Update 01.

Performed below steps:
-------------------------------------------
- Installed JBoss ON 3.3 GA on solaris 10
- Inventoried resources
- ./rhqctl stop
- Applied JON 3.3.0.GA Update 01
- started storage node with ./rhqctl start --storage
- Observed following in the console-

# ./rhqctl start --storage
WARNING: The readlink command is not available on this platform.
         If this script was launched from a symbolic link, errors may occur.
         Consider installing readlink on this platform.
16:04:48,589 INFO  [org.jboss.modules] JBoss Modules version 1.3.3.Final-redhat-1
./cassandra: numactl: not found

------------------------------------------
Additional Info:

Storage node, server and agent started successfully after applying JON 3.3.0.GA Update 01, however observed that the file ../jon-server-3.3.0.GA/rhq-storage/bin/cassandra has below after applying JON 3.3.0.GA Update 01:

NUMACTL_ARGS="--interleave=all"
if which numactl >/dev/null 2>/dev/null && numactl $NUMACTL_ARGS ls / >/dev/null 2>/dev/null
then
    NUMACTL="numactl $NUMACTL_ARGS"
else
    NUMACTL=""
fi


Environment details:

# uname -a
SunOS solaris10 5.10 Generic_147148-26 i86pc i386 i86pc
# isainfo -kv
64-bit amd64 kernel modules
# which numactl; echo $?
no numactl in /export/home/java/jdk1.7.0_60/bin /usr/sbin /usr/bin
0

Comment 7 Sunil Kondkar 2015-01-30 16:05:12 UTC
After discussing with Libor, performed below steps on 3.3.0.GA Update 01 running instance on Solaris 10:

1) ./rhqctl stop
2) Deleted rhq-storage directory 
3) ./rhqctl install

The console displays:
21:17:14,480 INFO  [org.rhq.server.control.command.Install] The storage node installer has finished with an exit value of 0
./cassandra: numactl: not found

4) ./rhqctl start
The console displays:
21:19:24,551 INFO  [org.jboss.modules] JBoss Modules version 1.3.3.Final-redhat-1
./cassandra: numactl: not found

5) The file rhq-storage/bin/cassandra has below on line#97:
if which numactl >/dev/null 2>/dev/null && numactl $NUMACTL_ARGS ls / >/dev/null 2>/dev/null

Comment 11 Simeon Pinder 2015-02-16 04:49:37 UTC
Moving to ON_QA as available to test with latest CP build:
http://download.devel.redhat.com/brewroot/packages/org.jboss.on-jboss-on-parent/3.3.0.GA/16/maven/org/jboss/on/jon-server-patch/3.3.0.GA/jon-server-patch-3.3.0.GA.zip
*Note: jon-server-patch-3.3.0.GA.zip maps to CR01 build of jon-server-3.3.0.GA-update-01.zip.

Comment 12 Sunil Kondkar 2015-02-17 10:47:11 UTC
Tested on 3.3.1.CR01

When tried to apply the patch on JON 3.3.0.GA on Solaris 10, it displays a syntax error:

# ./apply-updates.sh /export/home/JON3.3GA/jon-server-3.3.0.GA
./apply-updates.sh: syntax error at line 122: `LINES_TO_REMOVE=$' unexpected

-------------------------------

The line#122 in ./apply-updates.sh script has below:

LINES_TO_REMOVE=$(wc -l < "$FILES_TO_MOVE");

Steps:

-Install JON 3.3.0.GA
-Apply the CR01 patch with ./apply-updates.sh

Comment 13 Sunil Kondkar 2015-02-17 11:07:35 UTC
Moving this bug back to ON_QA after discussing with Libor and filing new bug#1193447 for the syntax error on applying JON 3.3.1.CR01 patch.

Comment 15 Sunil Kondkar 2015-02-23 08:40:13 UTC
Verified on JON 3.3.1.CR03

The error './cassandra: numactl: not found' is no more visible while starting/stopping storage node after applying JON 3.3.0.GA Update 01.

Also verified that the file rhq-storage/bin/cassandra has below on line#97:

if type numactl >/dev/null 2>/dev/null && numactl $NUMACTL_ARGS ls / >/dev/null 2>/dev/null


Note You need to log in before you can comment on or make changes to this bug.