Bug 571608 - ISE (Internal Server Error) and traceback email when trying to update packages in Kickstart -> Profile -> Software -> Package Groups for profiles created before RHBA-2009-1566
Summary: ISE (Internal Server Error) and traceback email when trying to update package...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Satellite 5
Classification: Red Hat
Component: Server
Version: 530
Hardware: All
OS: Linux
high
high
Target Milestone: ---
Assignee: Justin Sherrill
QA Contact: Dimitar Yordanov
URL:
Whiteboard:
Depends On:
Blocks: sat540-blockers
TreeView+ depends on / blocked
 
Reported: 2010-03-09 00:24 UTC by Xixi
Modified: 2018-10-30 20:04 UTC (History)
7 users (show)

Fixed In Version: spacewalk-schema-1.2.21-5
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2010-10-28 15:04:12 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
bug-571608-kickstart-workaround.TEST.571608.sql (400 bytes, text/x-sql)
2010-03-09 00:29 UTC, Xixi
no flags Details
rhnKickstartPackage.TEST.571608.1.patch (548 bytes, patch)
2010-03-09 00:36 UTC, Xixi
no flags Details | Diff

Description Xixi 2010-03-09 00:24:36 UTC
Description of problem:
Customer gets ISE and java traceback email with the following stacktrace when he tries to apply changes to the package/group list on existing kickstart profiles (both RHEL 4 and RHEL 5) in Satellite 5.3. This only happens for kickstart profile created before they applied RHBA-2009-1566 (spacewalk-java* and spacewalk-taskomatic later than 0.5.44-49), and they verified kickstart profiles created after that does not have this issue.

...
Exception:
javax.servlet.ServletException: Batch update returned unexpected row count from update [0]; actual row count: 2; expected: 1
  at org.apache.struts.action.RequestProcessor.processException(RequestProcessor.java:535)
...
Caused by: org.hibernate.jdbc.BatchedTooManyRowsAffectedException: Batch update returned unexpected row count from update [0]; actual row count: 2; expected: 1
  at org.hibernate.jdbc.Expectations$BasicExpectation.checkBatched(Expectations.java:71)
  at org.hibernate.jdbc.Expectations$BasicExpectation.verifyOutcome(Expectations.java:46)
  at org.hibernate.jdbc.NonBatchingBatcher.addToBatch(NonBatchingBatcher.java:24)
  at org.hibernate.persister.entity.AbstractEntityPersister.delete(AbstractEntityPersister.java:2524)
  at org.hibernate.persister.entity.AbstractEntityPersister.delete(AbstractEntityPersister.java:2701)
  at org.hibernate.action.EntityDeleteAction.execute(EntityDeleteAction.java:74)
  at org.hibernate.engine.ActionQueue.execute(ActionQueue.java:250)
  at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:234)
  at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:146)
  at org.hibernate.event.def.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:298)
  at org.hibernate.event.def.DefaultAutoFlushEventListener.onAutoFlush(DefaultAutoFlushEventListener.java:41)
  at org.hibernate.impl.SessionImpl.autoFlushIfRequired(SessionImpl.java:969)
  at org.hibernate.impl.SessionImpl.list(SessionImpl.java:1114)
  at org.hibernate.impl.QueryImpl.list(QueryImpl.java:79)
  at com.redhat.rhn.domain.kickstart.KickstartFactory.lookupKsPackageByKsDataAndPackageName(KickstartFactory.java:1070)
  at com.redhat.rhn.frontend.action.kickstart.EditPackagesAction.transferEdits(EditPackagesAction.java:158)
  at com.redhat.rhn.frontend.action.kickstart.EditPackagesAction.save(EditPackagesAction.java:90)
  at com.redhat.rhn.frontend.action.kickstart.EditPackagesAction.execute(EditPackagesAction.java:63)
  at org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:431)
  ... 40 more

Based on db queries (such as "select kp.kickstart_id, kp.package_name_id, pn.name, kp.position from rhnKickstartPackage kp, rhnKSData kd, rhnPackageName pn where kp.kickstart_id = kd.id and kp.package_name_id = pn.id and kp.kickstart_id in (ksid1, ksid2, ksid3) order by kp.kickstart_id, pn.name;"), looks like this is due to entries with the same kickstart_id and package_name_id but different position in rhnKickstartPackage table.  This causes KickstartPackage.findByKickstartDataAndPackageName to return more than one row, which violates the composite-id on ksData and packageName in the hibernate definition for rhnKickstartPackage.

Version-Release number of selected component (if applicable):
Red Hat Network (RHN) Satellite 5.3.0

System architecture(s):
RHEL 4U8 32-bit

How reproducible:
Always.

Steps to Reproduce:
1. Have a 5.3 satellite with a kickstart profile with some packages and/or package groups in the Software section.
2. Insert a row in rhnKickstartPackage for the above profile such that it has the same kickstart_id and package_name_id as one of the existing rows, but different position.
e.g., 
insert into rhnKickstartPackage values (205, 2077, '08-MAR-10', '08-MAR-10', 39);
3. Go to Kickstart -> Profile -> Software -> Package Groups and click on Update button.

Actual results:
Internal Server Error and above stacktrace in catalina.out and traceback email.

Expected results:
Kickstart profile is updated successfully, no ISE or traceback. 

Additional info:
Already discussed with Engineering (Justin), proposed fix and workaround to be attached.

Comment 1 Xixi 2010-03-09 00:29:22 UTC
Created attachment 398649 [details]
bug-571608-kickstart-workaround.TEST.571608.sql

Comment 2 Xixi 2010-03-09 00:36:20 UTC
Created attachment 398650 [details]
rhnKickstartPackage.TEST.571608.1.patch

Comment 3 Xixi 2010-03-09 00:41:01 UTC
Proposed workaround:

1) download SQL script bug-571608-kickstart-workaround.TEST.571608.sql attached to this BZ
2) get your satellite database connection string
3) sqlplus dbuser/dbpassword@dbsid @bug-571608-kickstart-workaround.TEST.571608.sql
4) move /var/lib/rhn/kickstarts/wizard/* to some place else
5) wait 10 minutes for the correct/new kickstart files to be regenerated   

Proposed fix attached: rhnKickstartPackage.TEST.571608.1.patch
This adds a uniqueness constraint on (kickstart_id, package_name_id) for rhnKickstartPackage table to be consistent with the hibernate definition. Of course, prior to adding the constraint probably will need to fix any existing bad data with something like bug-571608-kickstart-workaround.TEST.571608.sql in workaround above.

Since the fix requires a schema change, aligning/proposing for sat-600 instead of sat-530.

Comment 4 Xixi 2010-03-09 00:48:10 UTC
(In reply to comment #3)
Needless to say, make sure there's a working copy of the Satellite database before attempting the workaround in Comment #3.

Comment 6 Justin Sherrill 2010-08-10 20:51:20 UTC
fixed schema and added upgrade script.  Thanks for that xixi!

17ddce1bf66fe747366abbf7ee4aee0fec728f67

Comment 7 Milan Zázrivec 2010-10-11 14:47:42 UTC
We still need to remove the duplicate kickstart_id, package_name_id values
from the table before we can apply the "unique (kickstart_id, package_name_id)"
constraint otherwise the schema upgrade would fail.

Comment 8 Milan Zázrivec 2010-10-11 16:14:13 UTC
duplicate kickstart_id, package_name_id pairs removed during schema upgrade:

spacewalk.git master: e81db82da535a81c7ff65c925bbeb436bf368824
satellite.git master: 6861e0f64e2296239ba682c0cb75ed417cca428f

Comment 10 Milan Zázrivec 2010-10-13 14:47:39 UTC
I'm giving up the qa ownership of the bug, since I was doing modification
to it.

Comment 11 Dimitar Yordanov 2010-10-16 21:10:38 UTC
VERIFIED.
I tested the issue on Satellite-5.4.0-RHEL5-re20101013.0, spacewalk-schema-1.2.21-7.el5sat and I was not able to add package from the list in a different position.



SQL> select * from rhnKickstartPackage;

KICKSTART_ID PACKAGE_NAME_ID   POSITION CREATED   MODIFIED
------------ --------------- ---------- --------- ---------
	   1		2681	      0 16-OCT-10 16-OCT-10
	   1		2682	      1 16-OCT-10 16-OCT-10
	   1		1590	      2 16-OCT-10 16-OCT-10
	   1		1072	      3 16-OCT-10 16-OCT-10
	   1		 720	      4 16-OCT-10 16-OCT-10

SQL> insert into rhnKickstartPackage values (1, 720, 5,'08-MAR-10', '08-MAR-10');
insert into rhnKickstartPackage values (1, 720, 5,'08-MAR-10', '08-MAR-10')
*
ERROR at line 1:
ORA-00001: unique constraint (RHNSAT.RHN_KSPACKAGE_NAME_UQ) violated


Dimi

Comment 12 Michael Mráka 2010-10-22 12:23:45 UTC
Verified in stage.

Package can be only once in kickstart (there is unique constraint).

SQL> select * from rhnKickstartPackage where KICKSTART_ID = 163;

KICKSTART_ID PACKAGE_NAME_ID   POSITION CREATED   MODIFIED
------------ --------------- ---------- --------- ---------
         163            1052          1 22-OCT-10 22-OCT-10
         163            1512          2 22-OCT-10 22-OCT-10
         163            4212          0 22-OCT-10 22-OCT-10

SQL> insert into rhnKickstartPackage values (163, 1052, 3, sysdate, sysdate);
insert into rhnKickstartPackage values (163, 1052, 3, sysdate, sysdate)
*
ERROR at line 1:
ORA-00001: unique constraint (RHNSAT.RHN_KSPACKAGE_NAME_UQ) violated

Comment 13 Clifford Perry 2010-10-28 14:58:51 UTC
The 5.4.0 RHN Satellite and RHN Proxy release has occurred. This issue has been resolved with this release. 


RHEA-2010:0801 - RHN Satellite Server 5.4.0 Upgrade
https://rhn.redhat.com/rhn/errata/details/Details.do?eid=10332

RHEA-2010:0803 - RHN Tools enhancement update
https://rhn.redhat.com/rhn/errata/details/Details.do?eid=10333

RHEA-2010:0802 - RHN Proxy Server 5.4.0 bug fix update
https://rhn.redhat.com/rhn/errata/details/Details.do?eid=10334

RHEA-2010:0800 - RHN Satellite Server 5.4.0
https://rhn.redhat.com/rhn/errata/details/Details.do?eid=10335

Docs are available:

http://docs.redhat.com/docs/en-US/Red_Hat_Network_Satellite/index.html 

Regards,
Clifford


Note You need to log in before you can comment on or make changes to this bug.