In previous versions of Red Hat JBoss Enterprise Application Platform 6 the metadata for the +whoami+ operation was not marked as read-only. This meant that executing the operation would acquire a domain wide write lock. This could adversely affect the execution of other tasks and, in some cases, lead to an +Out of Memory+ error. This release of the product corrects this behavior by setting a 'read-only' flag on the +whoami+ metadata. This allows other operations to function as expected.
Created attachment 744648[details]
slowdeployweb.war
Description of problem:
'whoami' operation in domain mode seems to require the domain wide configuration lock. This is problematic when there is an editing operation of domain.xml like a long running undeployment because all 'whoami' requests are blocked by the existing task. The number of blocked thread increases indefinitely and it will cause an OutOfMemoryError.
It doesn't happen in standalone mode.
Steps to Reproduce:
1. Start EAP 6 in domain mode. The out-of-the-box domain.xml is sufficient.
2. Deploy attached 'slowdeployweb.war'.
3. Undeploy it from CLI. It will take about 1 hour.
4. In another terminal, execute several 'whoami' HTTP management requests like this:
--
curl --digest -L -D - http://admin:password@localhost:9990/management --header "Content-Type: application/json" -d '{"operation":"whoami","address":[],"json.pretty":1}' &
--
5. Take a thread dump of the host controller. You can see all 'HttpManagementService-threads' are stuck.
It reproduces in EAP 6.1.0.Beta1 (Wildfly 7.2.0.Final) too. I created a ticket for the upstream project.
https://issues.jboss.org/browse/WFLY-1329
Note that JON agent sends a lot of 'whoami' requests periodically. A long running undeployment and JON are a common environment and OOME in DC is actually happening at the customer.
Emanue, can I ask you to provide some details for the 6.1.1 release notes?
We need an outline of what was causing the problem to manifest, what change was implemented to fix it, and how the product behaves now to avoid the issue.
Thanks for any help you can offer.
Thanks for the content Emanual.
I've used it to craft a release note which I'm marking for inclusion in the document.
Please comment if you feel anything has been lost in translation.