Bug 961425 - EAP 6.1 CLI script usage breaks on last new line
Summary: EAP 6.1 CLI script usage breaks on last new line
Alias: None
Product: JBoss Enterprise Application Platform 6
Classification: JBoss
Component: CLI
Version: 6.1.0
Hardware: Unspecified
OS: Unspecified
Target Milestone: ER5
: EAP 6.1.1
Assignee: Ståle W. Pedersen
QA Contact:
Depends On:
Blocks: 990126
TreeView+ depends on / blocked
Reported: 2013-05-09 15:14 UTC by Tom Fonteyne
Modified: 2018-12-06 15:04 UTC (History)
14 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
A bug in the Aesh (Another Extendable SHell) component was causing Aesh to not parse data chunks correctly. If the last byte of data was a new line (and the preceding data had contained newlines) it would be swallowed and not parsed. This problem was found to be causing users' custom CLI scripts to fail. The Aesh bug has been fixed and user scripts now function as they did with earlier versions of Red Hat JBoss Enterprise Application Platform 6.
Clone Of:
Last Closed: 2013-09-16 20:23:13 UTC
Type: Bug
cdewolf: needinfo+

Attachments (Terms of Use)
pkremens_reproducer (125 bytes, application/x-shellscript)
2013-07-31 12:08 UTC, Petr Kremensky
no flags Details
pkremens_reproducer2 (114 bytes, application/x-shellscript)
2013-07-31 12:10 UTC, Petr Kremensky
no flags Details
jboss-cli patch (1.03 KB, patch)
2013-08-12 12:31 UTC, Ståle W. Pedersen
no flags Details | Diff

System ID Priority Status Summary Last Updated
Red Hat Bugzilla 990636 None None None Never
Red Hat Bugzilla 996164 None None None Never
Red Hat One Jira Issue Tracker WFLY-1841 Critical Resolved Upgrade aesh to 0.33.7 2017-06-30 10:26:35 UTC

Internal Links: 990126 990636 996164

Description Tom Fonteyne 2013-05-09 15:14:06 UTC
Description of problem:

I have been using bash scripts for the majority of my setups; automating things. Here is an example:

$JBOSS_HOME/bin/jboss-cli.sh <<EOFADD
$PROFILE_PREFIX/subsystem=security/security-domain=$REALM/authentication=classic:add(login-modules=[{"code" => "org.picketlink.identity.federation.bindings.jboss.auth.SAML2LoginModule","flag" => "required" }])


The bash variables get replaced before the CLI gets to see the lines of course (in this example $PROFILE_PREFIX resolve to being empty btw)
This worked perfectly well in EAP 6.0.0 and 6.0.1, but fails in 6.1.0 (tested "beta" and "er6")

Example of output with 6.1:

You are disconnected at the moment. Type 'connect' to connect to the server or 'help' for the list of supported commands.
[disconnected /] connect orac.usersys.redhat.com
s.jboss.auth.SAML2LoginModule", "flag" => "required"}])assic:add(login-modules=[{"code" => "org.picketlink.identity.federation.binding

Look at how the second line got completely broken. The CLI hangs after this output btw, ctrl-c to recover.

I have several such scripts with various commands - they all break similarly 

Version-Release number of selected component (if applicable):

 6.1.0 (tested "beta" and "er6")

How reproducible:


Steps to Reproduce:
1. create and run a script as mentioned

Actual results:

s.jboss.auth.SAML2LoginModule", "flag" => "required"}])assic:add(login-modules=[{"code" => "org.picketlink.identity.federation.binding

Expected results:

/subsystem=security/security-domain=sp/authentication=classic:add(login-modules=[{"code" => "org.picketlink.identity.federation.bindings.jboss.auth.SAML2LoginModule","flag" => "required" }])

Additional info:

I can provide more example script at request. But they all fail in a similar manner.

Comment 1 Josef Cacek 2013-05-22 12:57:28 UTC
Requesting blocker flag for 6.2. 
Unfortunately this issue had neither the blocker flag requested nor Regression keyword filled, so it didn't take part in 6.1 triage.

Comment 2 Tom Fonteyne 2013-05-23 08:48:01 UTC
@Josef: that was intentional. We realized full well we could not stop the 6.1 release. The plan is to get a one-off ready in the next couple of days. Ståle has already fixed it upstream but is still tweaking the code a little.

Comment 3 baranowb 2013-06-03 12:48:42 UTC
1. Issue status should not be 'NEW'
2. if this has been fixed in upstream, where is ISSUE_#/PR ?

Comment 4 Tom Fonteyne 2013-06-03 12:54:49 UTC
resolved upstream and in

the eap6.1 branch for æsh can be found here:
i've also created a tag for the latest version called 0.33.5 

ready for inclusion in 6.1.1 and later 6.2.0
It got fixed by Ståle faster then we could get the BZ updated :)

Comment 5 Tom Fonteyne 2013-06-03 12:56:05 UTC

Can you please check and close the BZ ?

Kind regards

Comment 7 Alexey Loubyansky 2013-06-17 11:11:14 UTC
Stale is on PTO for two weeks. If it's urgent, please, let me know, I'll check it out.

Comment 9 Tom Fonteyne 2013-07-17 11:24:44 UTC
As already touched upon in email, there are further issues with carriage returns and/or line feeds.

if for example the management realm is setup to always require a user/password, then scripting like this:

$JBOSS_HOME/bin/jboss-cli.sh <<EOFDEL

results in:

You are disconnected at the moment. Type 'connect' to connect to the server or 'help' for the list of supported commands.
[disconnected /] connect
Username: myuserPassword: mypassword[domain@orac.usersys.redhat.com:9999 /] batchsomacommand

Similarly after "run-batch":

 #] run-batchThe batch executed successfully

Comment 10 Ståle W. Pedersen 2013-07-19 22:53:12 UTC
ive managed to reproduce this locally. working on a fix..

Comment 11 Ståle W. Pedersen 2013-07-22 13:06:13 UTC
ive pushed aesh 0.33.6 to nexus. the only other thing that needs to change is the version in the pom.
i can probably do that, but im not sure in which branch i should do this (and how its accepted etc)...

Comment 15 Petr Kremensky 2013-07-31 09:49:55 UTC
I've reproduced issues from customer cases 00904577, 00889193 and 00910827, but example with working required user/password still not working for me.

./jboss-cli.sh -c <<EOF

steps to reproduce:
(1) unzip a fresh EAP 6.x and remove the following two lines in standalone.xml: <local default-user="$local"/> <local default-user="$local" allowed-users="*"/> 
(2) add a management user ( ./bin/add-user.sh bob admin123$ )
(3) start the server in standalone mode (./bin/standalone.sh)
(4) run reproducer.sh (./bin/reproducer.sh)

result with EAP 6.0.1 -> server is successfully stopped

result with EAP 6.1.1 ER4 -> 
Authenticating against security realm: ManagementRealm
Username: bob
[standalone@localhost:9999 /] batch:shutdown
'batch:shutdown' is not a valid operation name.
[standalone@localhost:9999 /] run-batch
// cli hangs here, must use Ctrl+C to rescue

Comment 16 Petr Kremensky 2013-07-31 10:05:48 UTC
Or is there another open BZ for this remaining issue?

Comment 17 Tom Fonteyne 2013-07-31 11:20:18 UTC
My own results with 6.1.1er4

./jboss-cli.sh -c <<EOF



The empty lines are needed as is the exit

[tom@orac 611er4]$ ./r.sh 
Username: tom
[standalone@localhost:9999 /] batch
[standalone@localhost:9999 / #] /system-property=bar:add(value=foo)
#1 /system-property=bla:add(value=foo)
[standalone@localhost:9999 / #] run-batch
The batch executed successfully
[standalone@localhost:9999 /] exit

So I cannot reproduce the concatenation of the "batch" and the first actual command
The need of "exit" is legitimate, as you want to exit the cli before going back to bash mode.

The fact that you need empty lines both before and after exit seems to be a valid issue
@Stale ?

Comment 18 Ståle W. Pedersen 2013-07-31 11:32:11 UTC
hi, the empty line after exit is needed since the input stream is identical to when you just paste text into the terminal. so if there isnt any new line found after exit it wont run that command (its waiting for more input).

regarding a new line after run-batch seems a bit "strange", ill look into it when im back from pto on monday if thats ok (might have some time tonight, but cant promise anything).

Comment 20 Petr Kremensky 2013-07-31 12:08:04 UTC
Created attachment 781070 [details]

Comment 21 Petr Kremensky 2013-07-31 12:08:46 UTC
Hmm that is really strange, I have same problem (concatenation of the "batch" and the first command) even with you example. Can you please try to run rep.sh (attachment 781070 [details])?

I am getting:
$ ./rep.sh 
Authenticating against security realm: ManagementRealm
Username: bob
[standalone@localhost:9999 /] batch/system-property=bar:add(value=foo)
'batch/system-property=bar:add(value=foo)' is not a valid operation name.
[standalone@localhost:9999 /] run-batch
Without arguments the command can be executed only in the batch mode.
[standalone@localhost:9999 /] 
[standalone@localhost:9999 /] exit

I must put empty line also between 'batch' and first command to make things work.

Comment 22 Petr Kremensky 2013-07-31 12:10:34 UTC
Created attachment 781074 [details]

Comment 23 Petr Kremensky 2013-07-31 12:11:27 UTC
I uploaded working example by mistake, please try attachment 781074 [details]

Comment 24 Tom Fonteyne 2013-07-31 12:32:12 UTC
tested 781074 and works for me, I changed user/pass, nothing else:

[tom@orac 611er4]$ ./rep.sh 
Username: tom
[standalone@localhost:9999 /] batch
[standalone@localhost:9999 / #] /system-property=bar:add(value=foo)
#1 /system-property=bar:add(value=foo)
[standalone@localhost:9999 / #] run-batch
The batch executed successfully
[standalone@localhost:9999 /] exit

Just to confirm, this is were I downloaded ER4:


and my install is on Fedora 18 (latest patches as of last month)

Comment 25 Tom Fonteyne 2013-07-31 13:29:20 UTC
reproduce both ways now, once working and once not working.
I have the habit of using LDAP authentication for everything, so my own tests were done with the management realm pointing to LDAP.

With LDAP, it works:

[tom@orac bin]$ ./rep.sh 
Username: tom
[standalone@localhost:9999 /] batch
[standalone@localhost:9999 / #] /system-property=bar:add(value=foo)
#1 /system-property=bar:add(value=foo)
[standalone@localhost:9999 / #] run-batch
The batch executed successfully
[standalone@localhost:9999 /] exit

Now, with management realm using property files:

[tom@orac bin]$ ./rep.sh 
Authenticating against security realm: ManagementRealm
Username: bob
[standalone@localhost:9999 /] batch/system-property=bar:add(value=foo)
'batch/system-property=bar:add(value=foo)' is not a valid operation name.
[standalone@localhost:9999 /] run-batch
Without arguments the command can be executed only in the batch mode.
[standalone@localhost:9999 /] 
[standalone@localhost:9999 /] exit

note the extra line of output:
==>  Authenticating against security realm: ManagementRealm

which is the one that is eating the linefeed !

Comment 30 Ståle W. Pedersen 2013-08-05 13:34:19 UTC
hi, im back from pto today and look at it now

Comment 31 Ståle W. Pedersen 2013-08-05 15:50:18 UTC
it seems that its the $ sign in the password that causes this since its parsed by the shell as a variable afaik ( havent googled it yet).
so try to escape the $ sign or use a password that doesnt contain $.
let my know how that works.

Comment 32 Ståle W. Pedersen 2013-08-05 17:16:39 UTC
the $ sign was only the problem for that example, but ive find out what happens and will have a fix and a new release of aesh out in 1-2 hours.

Comment 33 Ståle W. Pedersen 2013-08-05 19:34:18 UTC
ive pushed out a new version of aesh (0.33.7) that should fix this. it should also be no need for an extra new line after the exit statement.

Comment 34 Scott Mumford 2013-08-06 00:48:31 UTC
Hi Ståle,

Is it possible to get some idea of what was breaking the scripts and how it was fixed for the Release Note?

Comment 35 Ståle W. Pedersen 2013-08-06 07:23:18 UTC
hi scott, the jira i created for this is here: https://issues.jboss.org/browse/AESH-105
its a fairly short description (albeit correct) description on what was wrong.

in more detail æsh have a separate thread that reads from the stream so it gets data in chunks. when it received a chunk of data that contained several new-lines and the last byte also was a new-line the last byte was swallowed and not parsed correctly... :/

Comment 37 Scott Mumford 2013-08-07 04:02:49 UTC
Thanks Ståle

I added a draft Release Note. Please comment on technical it's accuracy.

Comment 39 Tom Fonteyne 2013-08-12 10:06:17 UTC
assigning back to stalep

as found by pkremers:

There is still problem with blocked CLI after passing the command via stdin.

echo 'ls' > /tmp/commands

jboss-eap-6.0/bin]$ ./jboss-cli.sh -c < /tmp/commands
=> Runs ls, and returns (exit correcly)

jboss-eap-6.1/bin]$ ./jboss-cli.sh -c < /tmp/commands
=> Runs ls, input is locked, only CTRL-C return to shell

to 'fix' this,


needs to contain:


e.g. the "exit" *must* be present

which if you think about it makes sense..... stdin is handled by the redirect and the CLI is correctly waiting for more input but you cannot type as stdin in not yours

So theoretically we could document this and done (though I'm not in favour to do this as this worked without 'exit' in EAP 6.0 obviously)

Comment 41 Ståle W. Pedersen 2013-08-12 11:19:27 UTC
i have found out why this happens and ill do a new release now.
however this would also require a small change to jboss-cli as well.
- a null check to prevent a nullpointer exception.

Comment 42 Ståle W. Pedersen 2013-08-12 11:35:40 UTC
ive pushed out 0.33.8 of aesh now.
as stated above it also needs a change to jboss-cli. ill attach the diff here and contact alexey.
just to be clear that this issue is very different than the original description. imo it could/should be another issue.

Comment 43 Ståle W. Pedersen 2013-08-12 12:31:34 UTC
Created attachment 785671 [details]
jboss-cli patch

Comment 45 Carlo de Wolf 2013-08-12 14:38:39 UTC
Opened up https://bugzilla.redhat.com/show_bug.cgi?id=996164

Comment 46 Alexey Loubyansky 2013-08-12 14:56:30 UTC
About the patch Stale attached, sure. Feel free to submit a PR or I'll include it in the issues I am working on this week.

Comment 47 Petr Kremensky 2013-08-14 06:15:41 UTC
Verified on EAP 6.1.1. ER5

Comment 48 Scott Mumford 2013-08-26 00:52:35 UTC
Setting for inclusion in the 6.1.1 Release notes using the above doc text.

Note You need to log in before you can comment on or make changes to this bug.