Bug 851480

Summary: ISE: A SCAP scan reporting two indentical numbers brings Satellite to its knees.
Product: Red Hat Satellite 5 Reporter: Šimon Lukašík <slukasik>
Component: WebUIAssignee: Tomas Lestach <tlestach>
Status: CLOSED CURRENTRELEASE QA Contact: Matej Kollar <mkollar>
Severity: high Docs Contact:
Priority: high    
Version: 550CC: cperry, mkollar, tlestach
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: spacewalk-java-1.7.54-99 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-09-21 09:20:41 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 818987    

Description Šimon Lukašík 2012-08-24 09:45:31 UTC
Description of problem:
If a SCAP result reports two identical counts of results, most
of the SCAP pages will fail with ISE. Throuwing:

java.lang.IndexOutOfBoundsException: Index: 2, Size: 2
        at java.util.ArrayList.get(ArrayList.java:352)
        at com.redhat.rhn.frontend.dto.XccdfTestResultCounts.getCountMap(XccdfTestResultCounts.java:194)
        at com.redhat.rhn.frontend.dto.XccdfTestResultCounts.getCountOf(XccdfTestResultCounts.java:185)
        at com.redhat.rhn.frontend.dto.XccdfTestResultCounts.getNotselected(XccdfTestResultCounts.java:129)

Technical Background:
There is a mechanism for elaborator to return multiple rows for
a single id. See occurencies of multiple="t" within SQL queries.
If the multiple="t" is used and the elaborator returns multiple
results per ID, each column is stored in its own java.util.List.

The problem is that since a0200a462868a25ea36790c7dd6725b8c047ac6c
the lists does not behave as lists but as sets.
(thus the very same number get stored only once).

Note: There are similar (no scap) DTOs affected by this misbehaviour.
(See UpgradablePackageListItem, ScheduledAction, ...) However,
for these the problem is not lethal.


What I think we need to do:
 1) find more appropriate fix to bug 453477
 2) go through the code base veryfying that there is *not*
    another similar place expecting the list to behave as a set.
 2) and revert a0200a462868a25ea36790c7dd6725b8c047ac6c

Version-Release number of selected component (if applicable):
Satellite 5.5.0

How reproducible:
deterministic

Steps to Reproduce:
1. Scan arbitrary machinee with arbitrary content.
   Make it happen that in statistics there will be two identical
   non-zero numbers. E.G.
   notselected=69, pass=69, fail=5
2. Visit /rhn/audit/ListXccdf.do
3.
  
Actual results:
ISE.

Expected results:
No ISE.

Comment 1 Šimon Lukašík 2012-08-27 08:47:19 UTC
(In reply to comment #0)
>  2) go through the code base veryfying that there is *not*
>     another similar place expecting the list to behave as a set.

I went through usage of multiple="true" in SQL queries and searched
for such usage where the elaborator column is expected to behave as
a set.

I didn't found a such. In fact, I have found a several queries which
expected the old list behaviour. These returns incorrect result
on certain (unlikely) data.

I have only read the application code and I didn't take csv exports
into account. (There might still be CSV exports other than bug 453477
which could be affected by list behaviour).

Following is a list of occurences:
# git grep multiple= code/src/ | grep -v test_queries.xml
code/src/com/redhat/rhn/common/db/datasource/xml/Action_queries.xml:<query name="action_overview_elab" params="user_id, org_id" multiple="t">
 - code assumes list behaviour
code/src/com/redhat/rhn/common/db/datasource/xml/Errata_queries.xml:<query name="related_channels_owned_by_org" multiple="t" params="org_id">
 - doesn't matter since there are no identities
code/src/com/redhat/rhn/common/db/datasource/xml/Package_queries.xml:     multiple="t"
 - doesn't matter since there are no identities
code/src/com/redhat/rhn/common/db/datasource/xml/Package_queries.xml:  <elaborator multiple="t" params="sid">
 - code assumes list behaviour
code/src/com/redhat/rhn/common/db/datasource/xml/Package_queries.xml:  <elaborator params="sid" multiple="t">
 - doesn't matter, a usage of multiple is a bit pointless here
code/src/com/redhat/rhn/common/db/datasource/xml/Package_queries.xml:  <elaborator params="sid" multiple="t">
 - doesn't matter since there are no identities
code/src/com/redhat/rhn/common/db/datasource/xml/Package_queries.xml:  <elaborator params="sid" multiple="t">
 - code assumes list behaviour, but due to other constrain the set should never be errorneous
code/src/com/redhat/rhn/common/db/datasource/xml/Package_queries.xml:  <elaborator params="sid" multiple="t">
 - code assumes list behaviour
code/src/com/redhat/rhn/common/db/datasource/xml/System_queries.xml:  <elaborator params="user_id, package_set_label" multiple="t">
 - doesn't matter since there are no identities
code/src/com/redhat/rhn/common/db/datasource/xml/System_queries.xml:  <elaborator params="formvar_uid" multiple="t">
 - doesn't matter since there are no identities
code/src/com/redhat/rhn/common/db/datasource/xml/System_queries.xml:  <elaborator params="eid" multiple="t">
 - code assumes list behaviour
code/src/com/redhat/rhn/common/db/datasource/xml/System_queries.xml:   <elaborator params="user_id, package_set_label" multiple="t">
 - doesn't matter since there are no identities
code/src/com/redhat/rhn/common/db/datasource/xml/System_queries.xml:   <elaborator params="user_id, package_set_label" multiple="t">
 - doesn't matter since there are no identities
code/src/com/redhat/rhn/common/db/datasource/xml/System_queries.xml:<query name="virtual_system_entitlements" params="" multiple="t">
 - doesn't matter since there should be no identities
code/src/com/redhat/rhn/common/db/datasource/xml/System_queries.xml:<query name="entitlements" params="" multiple="t">
 - doesn't matter since there are no identities
code/src/com/redhat/rhn/common/db/datasource/xml/package_search_elaborators.xml:  <elaborator params="" multiple="t">
 - doesn't matter since there are no identities
code/src/com/redhat/rhn/common/db/datasource/xml/package_search_elaborators.xml:  <elaborator params="" multiple="t">
 - doesn't matter since there are no identities
code/src/com/redhat/rhn/common/db/datasource/xml/scap_queries.xml:<query name="testresult_counts" multiple="t" column="xid">
 - this bug report

Comment 2 Šimon Lukašík 2012-08-30 08:16:53 UTC
spacewalk.git 1f887b97fd27f1ac770a11fea7e445f0a8755e77
spacewalk.git fb05a2cc003088aba405718f551915ea92ab6eef

It's fixed in spacewalk, thus I am reassigning to the Satellite developer.

Comment 3 Šimon Lukašík 2012-08-30 08:30:29 UTC
Test Plan: There is multiple things to test.

1) First of all it's the reproducer from comment 0. (I can help with
   setting up the client to return the same statistical numbers).
2) The reproducer of bug 453477.
3) Test CSV exports overall, since the gears underneath was changed.
    - Visit several pages of RHN Satellite where the CSV export is
      available. (Systems, Errata, Packages, scap, various searches)
    - Make sure the all the data gets exported (number of rows matches,
      columns are not empty, no duplicate items)

Thanks!

Comment 7 Matej Kollar 2012-09-07 08:14:10 UTC
I have verified that with

  spacewalk-java-1.7.54-{99,100}.el6sat.noarch.rpm
  spacewalk-java-config-1.7.54-{99,100}.el6sat.noarch.rpm
  spacewalk-java-lib-1.7.54-{99,100}.el6sat.noarch.rpm
  spacewalk-java-oracle-1.7.54-{99,100}.el6sat.noarch.rpm
  spacewalk-taskomatic-1.7.54-{99,100}.el6sat.noarch.rpm

everything works according to expectations, especially reproducer from Comment 0. Several CVS exports have been examined.

Comment 8 Clifford Perry 2012-09-21 09:20:41 UTC
This issue is resolved with the release of RHN Satellite 5.5. 

As of September 20th 2012, RHN Satellite 5.5 has been generally available. 

Release Notes and other 5.5 documentation can be found here:

 https://access.redhat.com/knowledge/docs/Red_Hat_Network_Satellite/

The associated Errata for the 5.5 release are:

5.5 Satellite GA Errata
 - http://rhn.redhat.com/errata/RHEA-2012-1296.html

5.5 Upgrade Errata
 - http://rhn.redhat.com/errata/RHEA-2012-1298.html

5.5 RHN Proxy GA Errata
 - http://rhn.redhat.com/errata/RHEA-2012-1297.html

5.5 RHN Tools GA Errata
 - http://rhn.redhat.com/errata/RHEA-2012-1299.html

Regards,
Clifford
- Engineering Manager, Satellite