Bug 853407

This is to track the resolution of the issue in JON 3.1.2

+++ This bug was initially created as a clone of Bug #836388 +++

Created attachment 595146 [details]
Excerpt from server log showing complete stack trace

Description of problem:
When creating/editing an alert definition on a compatible group which contains a Notification of type CLI Script an error is displayed in the UI and a very large stack trace is logged (error.txt)

Version-Release number of selected component (if applicable):
JON 3.0.1

How reproducible:
Always

Steps to Reproduce:
1.  Install and configure JON server and agent
2.  Start JON server and agent
3.  Add platform to inventory
4.  Create a compatible group of platforms
    1.  From **Inventory > Compatible Groups** select **New**

        *   **Name:** `Some Platforms`
        *   **Category:** **Platform**
        *   Add platform to groups **Assigned resources**
        *   Click **Finish**

5.  Add an alert definition to the compatible group with a CLI Notification
    1.  From **Inventory > Compatible Groups** select **Some Platforms**
    2.  Select the **Alerts** tab
    3.  Select the **Definitions** subtab
    4.  Click **New**

        *   **Name:** `Alert Definition 1`
        *   **Condition 1:**
                **Measurement Absolute Value Threshold**
                **Free Memory** 
                **< (Less than)** 
                `0`
        *   **Notification 1:**
                **Notification Sender:** **CLI Script**
                **User To Run The Script As:**   **Myself**
                **Repository:** *JBoss Patches**
                Upload new script showMetricsData.js or any CLI script

    5.  Click **Save**

  
Actual results:
    JON UI Message Center contains the following message: 
        Alert definition creation failed
        Error
        	java.lang.RuntimeException:[1340919066677] org.rhq.enterprise.server.alert.AlertDefinitionCreationException:Could not create alert definition child for Resources [10001] with group AlertDefinition[ id=10042, name=Alert Definition 1 ] -> javax.ejb.EJBTransactionRolledbackException:null -> java.lang.NullPointerException:null
            --- STACK TRACE FOLLOWS ---
            [1340919066677] org.rhq.enterprise.server.alert.AlertDefinitionCreationException:Could not create alert definition child for Resources [10001] with group AlertDefinition[ id=10042, name=Alert Definition 1 ] -> javax.ejb.EJBTransactionRolledbackException:null -> java.lang.NullPointerException:null
               at Unknown.java_lang_RuntimeException_$RuntimeException__Ljava_lang_RuntimeException_2Ljava_lang_RuntimeException_2(Unknown source:0)
               at Unknown.com_google_gwt_user_client_rpc_core_java_lang_RuntimeException_1FieldSerializer_instantiate__Lcom_google_gwt_user_client_rpc_SerializationStreamReader_2Ljava_lang_RuntimeException_2(Unknown source:0)
               at Unknown.com_google_gwt_user_client_rpc_impl_SerializerBase$MethodMap_$instantiate__Lcom_google_gwt_user_client_rpc_impl_SerializerBase$MethodMap_2Lcom_google_gwt_user_client_rpc_SerializationStreamReader_2Ljava_lang_String_2Ljava_lang_Object_2(Unknown source:0)
               at Unknown.com_google_gwt_user_client_rpc_impl_SerializerBase_$instantiate__Lcom_google_gwt_user_client_rpc_impl_SerializerBase_2Lcom_google_gwt_user_client_rpc_SerializationStreamReader_2Ljava_lang_String_2Ljava_lang_Object_2(Unknown source:0)
               at Unknown.com_google_gwt_user_client_rpc_impl_AbstractSerializationStreamReader_$readObject__Lcom_google_gwt_user_client_rpc_impl_AbstractSerializationStreamReader_2Ljava_lang_Object_2(Unknown source:0)
               at Unknown.com_google_gwt_user_client_rpc_impl_RequestCallbackAdapter_$onResponseReceived__Lcom_google_gwt_user_client_rpc_impl_RequestCallbackAdapter_2Lcom_google_gwt_http_client_Request_2Lcom_google_gwt_http_client_Response_2V(Unknown source:0)
               at Unknown.org_rhq_enterprise_gui_coregui_client_util_rpc_TrackingRequestCallback_onResponseReceived__Lcom_google_gwt_http_client_Request_2Lcom_google_gwt_http_client_Response_2V(Unknown source:0)
               at Unknown.com_google_gwt_http_client_Request_$fireOnResponseReceived__Lcom_google_gwt_http_client_Request_2Lcom_google_gwt_http_client_RequestCallback_2V(Unknown source:0)
               at Unknown.com_google_gwt_http_client_RequestBuilder$1_onReadyStateChange__Lcom_google_gwt_xhr_client_XMLHttpRequest_2V(Unknown source:0)
               at Unknown.anonymous(Unknown source:0)
               at Unknown.com_google_gwt_core_client_impl_Impl_entry0__Ljava_lang_Object_2Ljava_lang_Object_2Ljava_lang_Object_2Ljava_lang_Object_2(Unknown source:0)
               at Unknown.anonymous(Unknown source:0)
               at Unknown.anonymous(Unknown source:0)

    Server log contains:
        org.rhq.enterprise.server.alert.AlertDefinitionCreationException: Could not create alert definition child for Resources [10001] with group AlertDefinition[ id=10042, name=Alert Definition 1 ]
            at org.rhq.enterprise.server.alert.GroupAlertDefinitionManagerBean.createGroupAlertDefinitions(GroupAlertDefinitionManagerBean.java:177)
            at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
            ...
        Caused by: javax.ejb.EJBTransactionRolledbackException
            at org.jboss.ejb3.tx.Ejb3TxPolicy.handleInCallerTx(Ejb3TxPolicy.java:87)
            at org.jboss.aspects.tx.TxPolicy.invokeInCallerTx(TxPolicy.java:130)
            at org.jboss.aspects.tx.TxInterceptor$Required.invoke(TxInterceptor.java:195)
            ...
            at org.jboss.ejb3.stateless.StatelessLocalProxy.invoke(StatelessLocalProxy.java:84)
            at $Proxy218.checkAuthentication(Unknown Source)
            at org.rhq.enterprise.server.plugins.alertCli.CliSender.validateAndFinalizeConfiguration(CliSender.java:233)
            at org.rhq.enterprise.server.alert.AlertNotificationManagerBean.finalizeNotifications(AlertNotificationManagerBean.java:317)
            at sun.reflect.GeneratedMethodAccessor450.invoke(Unknown Source)
            ...
            at org.jboss.ejb3.stateless.StatelessLocalProxy.invoke(StatelessLocalProxy.java:84)
            at $Proxy464.finalizeNotifications(Unknown Source)
            at org.rhq.enterprise.server.alert.AlertDefinitionManagerBean.checkAlertDefinition(AlertDefinitionManagerBean.java:629)
            at org.rhq.enterprise.server.alert.AlertDefinitionManagerBean.createAlertDefinition(AlertDefinitionManagerBean.java:202)
            ...
            at $Proxy314.createAlertDefinition(Unknown Source)
            at org.rhq.enterprise.server.alert.GroupAlertDefinitionManagerBean.createGroupAlertDefinitions(GroupAlertDefinitionManagerBean.java:167)
            ... 125 more
        Caused by: java.lang.NullPointerException
            at org.rhq.enterprise.server.auth.SubjectManagerBean._checkAuthentication(SubjectManagerBean.java:372)
            at org.rhq.enterprise.server.auth.SubjectManagerBean.checkAuthentication(SubjectManagerBean.java:363)
            at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
            ...


Expected results:
No error and CLI Script should be executed if alert condition occurs.

Additional info:
This appears to be confined to an alert definition being defined on a group. When defining the exact same alert definition on an individual resource, all is well. Additionally, if changing from **Myself** to entering my own user name and password, all is fine on the group alert definition.

--- Additional comment from ccrouch on 2012-07-03 11:00:58 EDT ---

Dropping priority as per triage

--- Additional comment from lkrejci on 2012-08-08 07:48:56 EDT ---

As mentioned on the bug 846623 that tracks this issue in master, the reported error appeared not only when creating group alert definitions but also when creating a template alert definition of a resource type.

--- Additional comment from lkrejci on 2012-08-08 07:50:34 EDT ---

release/jon3.1.x http://git.fedorahosted.org/cgit/rhq/rhq.git/diff/?id=97f5db48e4570c3cf8ee516de5bda7aff31c59a9
Author: Lukas Krejci <lkrejci>
Date:   Wed Aug 8 13:41:26 2012 +0200

    [BZ 846623] - When creating the "child" alert definitions of group or template alert definitions, pass the real user that creates the alert def and circumvent authz.
    This behaves exactly the same as before but instead of bypassing the authz by passing the overlord when creating the child alert,
    a new local SLSB method is used that doesn't perform the authz checks and can therefore receive the original user that request the creation of the group/template
    alert def.
    
    This is good for the CLI alert sender that, when creating an alert script to be run as "myself", checks if the user creating the alert def is the same as the one set
    to run it.
    (cherry picked from commit 581324eeb3d32f4d8879d4749423fbde0f76de8c)

--- Additional comment from jsanda on 2012-08-13 22:16:36 EDT ---

Moving to ON_QA since JON 3.1.1 ER2 build is availble - https://brewweb.devel.redhat.com/buildinfo?buildID=228250

--- Additional comment from skondkar on 2012-08-27 07:27:29 EDT ---

Tested by following the steps on JON 3.1.1.ER3 build.

1. Creating alert alert definition with CLI script notification on a resource, compatible group and alert definition template on a resource works as expected without any error.
2. Editing and saving alert definition with CLI script notification on a resource works as expected without any error.

However, when I tried to edit the alert definition with CLI script notification on a compatible group 
and when editing a template alert definition of a resource type, it resulted displaying 'Alert definition update failed' in UI after clicking 'save' button. No errors in server/agent log.

Please find the details of error displayed in message center.

Build: Version: 3.1.1.ER3
Build Number: 37108ca:67c6de8
Database: PostgreSQL 8.4.2
Browser: Firefox 10.0.2
Java Version: jre1.7.0_05

Steps to reproduce:

1. Create alert definition with CLI script notification on a compatible group.
2. Click on the alert definition created above and navigate to 'Notifications' tab.
3. Click on 'Edit' button.
4. Modify anything like alert description or change the priority.
5. Click on 'Save' button.
6. It displays 'Alert definition update failed' in UI.

7. Create alert definition with CLI script notification using alert definition template on a resource.
8. Follow above steps 2 to 5

--- Additional comment from skondkar on 2012-08-27 07:28:23 EDT ---

Created attachment 607185 [details]
Error_details

--- Additional comment from lkrejci on 2012-08-29 08:33:55 EDT ---

Ok, I am very sorry, but I completely missed the update workflows in my original fix. Thanks Sunil for careful testing.

The problem now is that to fix all the possible update workflows is a much bigger task than the original fix because there are many more places throughout the codebase we need to visit.

--- Additional comment from lkrejci on 2012-08-29 09:42:28 EDT ---

Just a note that this is not a regression from previous JON versions.
I will reopen the associated master BZ (bug 846623) and will post further progress in there.

Once this is fixed in master, we can revisit this bug and decide whether to put it to JON 3.1.1 (or a later version) or not.

--- Additional comment from lkrejci on 2012-08-30 12:27:08 EDT ---

Further details:

1) The issue described in this bug is not a regression and has existed since the introduction of CLI Script alert notification.
2) The partial fix applied opens the door for this issue to crop up in several other scenarios described in https://bugzilla.redhat.com/show_bug.cgi?id=846623#c4

I see 2 ways for going forward in JON 311:

1) Revert the partial fix and leave this bug for after JON 311 due to the complexity of the fix.
2) Fix this fully - this is going to require updating the logic in several SLSBs and non-trivial testing effort - both for QE and for integration tests needed to test the listed issues programatically.

Given the time constraints on JON 311 I unfortunately slightly lean towards 1).

--- Additional comment from loleary on 2012-08-30 12:56:59 EDT ---

My vote is for 1) and to re-target this to 3.1.2 as high priority. Of course Charles has final say.

--- Additional comment from ccrouch on 2012-08-31 07:57:37 EDT ---

Lukas, please go ahead with option 1) the revert current fix.
Can you also please raise another BZ and target it at JON312 so that we can have an issue for fixing the entire problem in the next micro release.

--- Additional comment from lkrejci on 2012-08-31 08:38:22 EDT ---

release/jon3.1.x http://git.fedorahosted.org/cgit/rhq/rhq.git/diff/?id=e23f9cea52ff4576546db93c546942ea95043ec6
Author: Lukas Krejci <lkrejci>
Date:   Fri Aug 31 14:34:38 2012 +0200

    Revert "[BZ 846623] - When creating the "child" alert definitions of group or template alert definitions, pass the real user that creates the alert def and circumvent authz."
    
    This reverts commit 97f5db48e4570c3cf8ee516de5bda7aff31c59a9.