Bug 484305 - Package deletion from GUI causes 500 error
Package deletion from GUI causes 500 error
Status: CLOSED CURRENTRELEASE
Product: Red Hat Satellite 5
Classification: Red Hat
Component: WebUI (Show other bugs)
530
All Linux
low Severity medium
: ---
: ---
Assigned To: Jay Dobies
Michael Mráka
:
: 465492 487412 (view as bug list)
Depends On:
Blocks: 456985
  Show dependency treegraph
 
Reported: 2009-02-05 18:10 EST by Mike McCune
Modified: 2009-10-28 15:23 EDT (History)
5 users (show)

See Also:
Fixed In Version: sat530
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 524625 (view as bug list)
Environment:
Last Closed: 2009-10-28 15:23:12 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Mike McCune 2009-02-05 18:10:03 EST
* Create custom base channel.
* Push F9 i386 packages into channel.  Should end up with about 2500 packages.
* Delete channel from GUI which should 'orphan' all 2500 F9 packages.
* Go to Manage Software Packages
* Try to delete all 2500 F9 packages and you get:


Caused by: com.redhat.rhn.common.hibernate.HibernateRuntimeException: HibernateException executing CachedStatement
	at com.redhat.rhn.common.db.datasource.CachedStatement.execute(CachedStatement.java:463)
	at com.redhat.rhn.common.db.datasource.CachedStatement.execute(CachedStatement.java:425)
	at com.redhat.rhn.common.db.datasource.CachedStatement.executeUpdate(CachedStatement.java:269)
	at com.redhat.rhn.common.db.datasource.WriteMode.executeUpdate(WriteMode.java:32)
	at com.redhat.rhn.manager.rhnpackage.PackageManager.clearNeededPackageCache(PackageManager.java:1004)
	at com.redhat.rhn.manager.rhnpackage.PackageManager.deletePackages(PackageManager.java:1020)
	at com.redhat.rhn.frontend.action.rhnpackage.CustomPackagesDeleteConfirmAction.deletePackages(CustomPackagesDeleteConfirmAction.java:103)
	at com.redhat.rhn.frontend.action.rhnpackage.CustomPackagesDeleteConfirmAction.execute(CustomPackagesDeleteConfirmAction.java:80)
	at org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:431)
	... 40 more
Caused by: org.hibernate.StaleStateException: Batch update returned unexpected row count from update [0]; actual row count: 0; expected: 1
	at org.hibernate.jdbc.Expectations$BasicExpectation.checkBatched(Expectations.java:61)
	at org.hibernate.jdbc.Expectations$BasicExpectation.verifyOutcome(Expectations.java:46)
	at org.hibernate.jdbc.NonBatchingBatcher.addToBatch(NonBatchingBatcher.java:24)
	at org.hibernate.persister.entity.AbstractEntityPersister.delete(AbstractEntityPersister.java:2520)
	at org.hibernate.persister.entity.AbstractEntityPersister.delete(AbstractEntityPersister.java:2697)
	at org.hibernate.action.EntityDeleteAction.execute(EntityDeleteAction.java:74)
	at org.hibernate.engine.ActionQueue.execute(ActionQueue.java:250)
	at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:234)
	at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:146)
	at org.hibernate.event.def.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:298)
	at org.hibernate.event.def.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:27)
	at org.hibernate.impl.SessionImpl.flush(SessionImpl.java:1000)
	at com.redhat.rhn.common.db.datasource.CachedStatement.stealConnection(CachedStatement.java:847)
	at com.redhat.rhn.common.db.datasource.CachedStatement.execute(CachedStatement.java:435)
	... 48 more
2009-02-05 17:28:57,719 [TP-Processor2] ERROR org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/rhn].[action] - Servlet.service() for servlet action threw exception
org.hibernate.StaleStateException: Batch update returned unexpected row count from update [0]; actual row count: 0; expected: 1
	at org.hibernate.jdbc.Expectations$BasicExpectation.checkBatched(Expectations.java:61)
	at org.hibernate.jdbc.Expectations$BasicExpectation.verifyOutcome(Expectations.java:46)
	at org.hibernate.jdbc.NonBatchingBatcher.addToBatch(NonBatchingBatcher.java:24)
	at org.hibernate.persister.entity.AbstractEntityPersister.delete(AbstractEntityPersister.java:2520)
	at org.hibernate.persister.entity.AbstractEntityPersister.delete(AbstractEntityPersister.java:2697)
	at org.hibernate.action.EntityDeleteAction.execute(EntityDeleteAction.java:74)
	at org.hibernate.engine.ActionQueue.execute(ActionQueue.java:250)
	at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:234)
	at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:146)
	at org.hibernate.event.def.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:298)
	at org.hibernate.event.def.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:27)
	at org.hibernate.impl.SessionImpl.flush(SessionImpl.java:1000)
	at com.redhat.rhn.common.db.datasource.CachedStatement.stealConnection(CachedStatement.java:847)
	at com.redhat.rhn.common.db.datasource.CachedStatement.execute(CachedStatement.java:435)
	at com.redhat.rhn.common.db.datasource.CachedStatement.execute(CachedStatement.java:425)
	at com.redhat.rhn.common.db.datasource.CachedStatement.executeUpdate(CachedStatement.java:269)
	at com.redhat.rhn.common.db.datasource.WriteMode.executeUpdate(WriteMode.java:32)
	at com.redhat.rhn.manager.rhnpackage.PackageManager.clearNeededPackageCache(PackageManager.java:1004)
	at com.redhat.rhn.manager.rhnpackage.PackageManager.deletePackages(PackageManager.java:1020)
	at com.redhat.rhn.frontend.action.rhnpackage.CustomPackagesDeleteConfirmAction.deletePackages(CustomPackagesDeleteConfirmAction.java:103)
	at com.redhat.rhn.frontend.action.rhnpackage.CustomPackagesDeleteConfirmAction.execute(CustomPackagesDeleteConfirmAction.java:80)
	at org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:431)
	at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:237)
	at com.redhat.rhn.frontend.struts.RhnRequestProcessor.process(RhnRequestProcessor.java:82)
	at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1196)
	at org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:432)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
	at com.redhat.rhn.frontend.servlets.AuthFilter.doFilter(AuthFilter.java:73)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
	at com.opensymphony.module.sitemesh.filter.PageFilter.parsePage(PageFilter.java:142)
	at com.opensymphony.module.sitemesh.filter.PageFilter.doFilter(PageFilter.java:58)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
	at com.redhat.rhn.frontend.servlets.LocalizedEnvironmentFilter.doFilter(LocalizedEnvironmentFilter.java:67)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
	at com.redhat.rhn.frontend.servlets.EnvironmentFilter.doFilter(EnvironmentFilter.java:108)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
	at com.redhat.rhn.frontend.servlets.SessionFilter.doFilter(SessionFilter.java:55)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
	at com.redhat.rhn.frontend.servlets.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:97)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:210)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:172)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:151)
	at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:200)
	at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:283)
	at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:773)
	at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:703)
	at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:895)
	at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:685)
	at java.lang.Thread.run(Thread.java:636)
[root@rlx-0-04 ~]#
Comment 1 Jay Dobies 2009-02-27 10:42:52 EST
Commit: efdafa2977d28ed0729f7d61b43d45c31ca92de3

484305 - The ChangeLogEntry id should have simply been the id value and not a composite key. The composite key can cause issues with cascading deletes (in this case from Package->ChangeLogEntry) manifesting themselves in StaleStateExceptions. The API indicates this may be an issue with unsaved-value, however since we have access to an ID value in the DB it's much easier to correct the relationship in the hibernate mapping.

http://www.hibernate.org/hib_docs/v3/api/org/hibernate/StaleStateException.html
Comment 2 Jay Dobies 2009-02-27 11:39:54 EST
Going to reopen this and address some performance issues in this BZ. Now that the hibernate issues are addressed and this can run, now at times, I run into "java.lang.OutOfMemoryError: GC overhead limit exceeded".
Comment 3 Jay Dobies 2009-03-02 14:50:06 EST
Commit: 45bdda0b44113957abb494b3e92cc5675e316675

Refactored to use sets to keep the heavy lifting for package deletion taking place in the database. Using this approach instead of hibernate fixes OutOfMemoryErrors when deleting large numbers of packages in addition to the sheer performance increase.
Comment 4 Pradeep Kilambi 2009-03-04 19:05:34 EST
*** Bug 487412 has been marked as a duplicate of this bug. ***
Comment 5 Brad Buckingham 2009-03-10 14:37:38 EDT
*** Bug 465492 has been marked as a duplicate of this bug. ***
Comment 6 Michael Mráka 2009-04-17 05:18:00 EDT
Verified. Satellite-5.3.0-RHEL4-re20090414.0 x86_64

No ISE, 2500+ packages deleted in 5 mins.
Comment 10 Jay Dobies 2009-09-21 10:26:59 EDT
What's happening here is that the connection between Tomcat and Apache is timing out. The performance fixes minimize this to a certain extent, but in very large environments it is still a possibility.

To be honest, I'm a bit surprised 4000 have this error. The idea was to queue up the deletions to be addressed asynchronously. However, that queuing does still take time. It sounds like this needs a second look. Personally, I'd close this bug out and file a new one as an issue deleting large amounts of packages as this bug is centered around issues in a much wider range of environments.
Comment 11 Petr Sklenar 2009-09-21 10:39:03 EDT
I deleted 2500 packages in a while without any problem.
tried Satellite-5.3.0-RHEL5-re20090820.1

but I could see timeout issue with more amount of packages
created :  Bug 524625 -  Package deletion from GUI causes 500 error

moving this bug with amount=2500 to release_pending

Note You need to log in before you can comment on or make changes to this bug.