Bug 1226413

Summary: Bundle rhq:handover can hang when management interface is using SSL and jboss-cli does not yet trust certs
Product: [JBoss] JBoss Operations Network Reporter: dsteigne
Component: Plugin -- JBoss EAP 6Assignee: Thomas Heute <theute>
Status: CLOSED ERRATA QA Contact: Mike Foley <mfoley>
Severity: high Docs Contact:
Priority: high    
Version: JON 3.3.1, JON 3.3.2CC: fbrychta, hhovsepy, loleary, spinder, theute
Target Milestone: post-GAKeywords: Triaged
Target Release: One-off release   
Hardware: All   
OS: All   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-01-28 13:29:25 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1181217    

Description dsteigne 2015-05-29 17:23:14 UTC
Description of problem:
If using secure connection to the EAP6 server, when you use the rhq:handover to run a CLI at the EAP server, it hangs:

15623 pts/0    S      0:00 /bin/sh /home/dsteigne/NotBackedUp/jboss-eap-6.2/bin/jboss-cli.sh --connect --file=/NotBackedUp/dsteigne/jon3.3.0/rhq-agent/data/tmp/logging.cli202739193411657036.tmp --user=admin62 --password=62_admin --controller=127.0.0.1:9999

If running the actual command from commandline you'll see it prompt to accept cert.

[dsteigne@dsteigne bin]$ ./jboss-cli.sh --connect --file=/NotBackedUp/dsteigne/jon3.3.0/rhq-agent/data/tmp/logging.cli202739193411657036.tmp --user=admin62 --password=62_admin --controller=127.0.0.1:9999
Unable to connect due to unrecognised server certificate
Subject    - CN=appserver,OU=Sales,O=Systems Inc,L=Raleigh,ST=NC,C=US
Issuer     - CN=appserver, OU=Sales, O=Systems Inc, L=Raleigh, ST=NC, C=US
Valid From - Mon Apr 27 12:01:33 CDT 2015
Valid To   - Wed Apr 26 12:01:33 CDT 2017
MD5 : 89:6c:04:0e:15:10:41:65:06:ee:34:84:dd:23:3d:2f
SHA1 : c9:4c:fe:2c:f5:4e:63:3b:ff:69:18:db:1a:96:23:b3:e3:8b:71:ea

Accept certificate? [N]o, [T]emporarily, [P]ermenantly : P
[dsteigne@dsteigne bin]$

Can an option be added to pass without the user/password arguments so that it would use local auth? 

Version-Release number of selected component (if applicable):
3.3.1, 3.3.2

How reproducible:
Everytime

Steps to Reproduce:
1. Switch an EAP6 Server to run secure.
2. create a simple EAP cli script, and deploy as a bundle
<?xml version="1.0"?>
<project name="LoggingCLI" default="main"
    xmlns:rhq="antlib:org.rhq.bundle">

    <rhq:bundle name="LoggingCLI" version="1.0" description="Execute EAP handover script">
        <rhq:deployment-unit name="LoggingCLI" preinstallTarget="preinstall" postinstallTarget="postinstall" manageRootDir="false">
      <rhq:file name="logging.cli" replace="true">
  <rhq:handover action="execute-script" failonerror="false"/>
</rhq:file>
      </rhq:deployment-unit>
  </rhq:bundle>
        
    <target name="main" />

    <target name="preinstall">
        <echo>Deploying LoggingCLI...</echo>
        <property name="preinstallTargetExecuted" value="true"/>
    </target>

    <target name="postinstall">
        <echo>Done deploying LoggingCLI</echo>
        <property name="postinstallTargetExecuted" value="true"/>
        </target>
</project>
3. Deploy the bundle to that EAP server

Actual results:
Process hangs

Expected results:


Additional info:

Comment 1 dsteigne 2015-05-29 17:25:51 UTC
Customer logging this ticket has made code change suggestions:

this should be a new overloaded method:

public ProcessExecutionResults executeCliScript(File scriptFile, Boolean localAuth)

The existing executeCliScript(File scriptFile) method will need to remain untouched to avoid breaking existing functionality. 

I think the code change required would look a little something like this:


https://github.com/rhq-project/rhq/blob/c31f2fba8204ee73fe2065f7ef2193f87f56e14b/modules/plugins/jboss-as-7/src/main/java/org/rhq/modules/plugins/jbossas7/ServerControl.java

Line 249:

		public ProcessExecutionResults executeCliScript(File scriptFile, Boolean localAuth) {
            File homeDir = serverPluginConfig.getHomeDir();
            File script = scriptFile;
            if (!script.isAbsolute()) {
                script = new File(homeDir, scriptFile.getPath());
            }
            File executable = new File("bin", serverMode.getCliScriptFileName());
            String connect = disconnected ? null : "--connect";
            String file = "--file=" + script.getAbsolutePath();
            String user = disconnected ? null : "--user=" + serverPluginConfig.getUser();
            String password = disconnected ? null : "--password=" + serverPluginConfig.getPassword();
            String controller = disconnected ? null : "--controller=" + serverPluginConfig.getNativeHost() + ":"
                + serverPluginConfig.getNativePort();

            if (systemInfo.getOperatingSystemType() != OperatingSystemType.WINDOWS) {
                if (localAuth == true) {
					return execute(null, executable, connect, file, controller);
				}
				else {
					return execute(null, executable, connect, file, user, password, controller);
				}
            }
			WinCliHelper cliHelper = new WinCliHelper();
			if (localAuth == true) {
				cliHelper = new WinCliHelper(executable, connect, file, controller);
			}
			else {
				cliHelper = new WinCliHelper(executable, connect, file, user, password, controller);
			}
            return cliHelper.execute();
        }
		

		
https://github.com/rhq-project/rhq/blob/289920aeb8d88c8a2a7f57f9a5327e3e2f19df39/modules/plugins/jboss-as-7/src/main/java/org/rhq/modules/plugins/jbossas7/BaseServerComponent.java

Line 766:

 protected BundleHandoverResponse handleExecuteScript(BundleHandoverRequest handoverRequest) throws IOException {
        Map<String, String> params = handoverRequest.getParams();

        long waitTime;
        String waitTimeParam = params.get("waitTime");
        if (waitTimeParam != null) {
            try {
                waitTime = Long.parseLong(waitTimeParam);
                if (waitTime <= 0) {
                    return BundleHandoverResponse.failure(INVALID_PARAMETER, "waitTime must greater than 0");
                }
            } catch (NumberFormatException e) {
                return BundleHandoverResponse.failure(INVALID_PARAMETER, "waitTime is not a number");
            }
        } else {
            waitTime = HOURS.toMillis(1);
        }

		boolean localAuth = new Boolean(false);
        String localAuthParam = params.get("localAuth");
        if (localAuthParam != null) {
            try {
				localAuth = Boolean.parseBoolean(localAuthParam);
			}
			catch(BooleanFormatException e) {
				return BundleHandoverResponse.failure(INVALID_PARAMETER, "localAuth value could not be parsed");
			}
		}
		
        boolean killOnTimeout = Boolean.parseBoolean(params.get("killOnTimeout"));

        File scriptFile = null;
        try {
            scriptFile = File.createTempFile(handoverRequest.getFilename(), ".tmp", context.getTemporaryDirectory());
            FileUtil.writeFile(handoverRequest.getContent(), scriptFile);

            ProcessExecutionResults results = ServerControl //
                .onServer( //
                        getServerPluginConfiguration().getPluginConfig(), //
                        getMode(), //
                    context.getSystemInformation() //
                ) //
                .waitingFor(waitTime) //
                .killingOnTimeout(killOnTimeout) //
                .cli() //
                .executeCliScript(scriptFile.getAbsoluteFile(), localAuth);

            logExecutionResults(results);

            Throwable error = results.getError();
            if (error != null) {
                return BundleHandoverResponse.failure(EXECUTION, error.getMessage(), error);
            }
            Integer exitCode = results.getExitCode();
            if (exitCode == null) {
                return BundleHandoverResponse.failure(EXECUTION, "Timeout waiting for completion of the CLI process");
            }
            if (exitCode != 0) {
                return BundleHandoverResponse.failure(EXECUTION, "CLI process exit code is " + exitCode);
            }
            return BundleHandoverResponse.success();

        } finally {
            if (scriptFile != null) {
                scriptFile.delete();
            }
        }
    }

Comment 2 Libor Zoubek 2015-06-12 12:58:21 UTC
I think issue has been addressed within Bug 1227459. I also think, that bundle should not provide authorization method to be used by plugin. 

My suggestion is to close this bug as duplicate of Bug 1227459

Comment 8 Filip Brychta 2015-07-01 13:39:39 UTC
Error from comment 7 is caused by EAP bz1238263

Comment 15 Filip Brychta 2015-07-09 13:05:57 UTC
Verified that rhq:handover does not hang and error message contains "Unable to connect due to unrecognised server certificate. Server's certificate needs to be manually accepted by user." and no jboss cli processes are running.

Also verified that handover works once the certificate is permanently accepted.

On linux for EAP 6.2, 6.3, 6.4
On win for EAP 6.4

There is one limitation - see https://bugzilla.redhat.com/show_bug.cgi?id=1227459#c14