Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 874604

Summary: The following exception occurred while executing this request: GET /rhn/channels/ChannelDetail.do
Product: [Community] Spacewalk Reporter: Joe Belliveau <joe.belliveau>
Component: ServerAssignee: Jan Pazdziora (Red Hat) <jpazdziora>
Status: CLOSED WORKSFORME QA Contact: Red Hat Satellite QA List <satqe-list>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: 1.9CC: joe.belliveau, jpazdziora, tlestach
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-11-13 09:27:41 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1484117    

Description Joe Belliveau 2012-11-08 14:21:05 UTC
Description of problem: Internal Server Error on webui and machines connected to spacewalk fail to update from the channel.


Version-Release number of selected component (if applicable): 1.9 nightly


How reproducible: Constantly reproduceable


Steps to Reproduce:
1. Upgrade 1.8 to 1.9 nightly 
2. Run schema update
3. Restart spacewalk
4. Login and try and view a channel
  
Actual results:
Internal Server Error


Expected results:
View Channel or Sub channel info

Additional info:

Traceback attached.


Exception:
javax.servlet.ServletException: com.redhat.rhn.common.hibernate.HibernateRuntimeException: Executing query DistChannelMap.findByChannel with params {channel=com.redhat.rhn.domain.channel.Channel@7e8138a6[id=144,label=centos6-x86_64]} failed
		 at org.apache.struts.action.RequestProcessor.processException(RequestProcessor.java:520)
		 at org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:427)
		 at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:228)
		 at com.redhat.rhn.frontend.struts.RhnRequestProcessor.process(RhnRequestProcessor.java:99)
		 at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1913)
		 at org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:449)
		 at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
		 at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
		 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
		 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
		 at com.redhat.rhn.frontend.servlets.AuthFilter.doFilter(AuthFilter.java:93)
		 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
		 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
		 at com.opensymphony.sitemesh.webapp.SiteMeshFilter.obtainContent(SiteMeshFilter.java:129)
		 at com.opensymphony.sitemesh.webapp.SiteMeshFilter.doFilter(SiteMeshFilter.java:77)
		 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
		 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
		 at com.redhat.rhn.frontend.servlets.LocalizedEnvironmentFilter.doFilter(LocalizedEnvironmentFilter.java:67)
		 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
		 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
		 at com.redhat.rhn.frontend.servlets.EnvironmentFilter.doFilter(EnvironmentFilter.java:108)
		 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
		 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
		 at com.redhat.rhn.frontend.servlets.SessionFilter.doFilter(SessionFilter.java:55)
		 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
		 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
		 at com.redhat.rhn.frontend.servlets.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:97)
		 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
		 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
		 at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
		 at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
		 at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
		 at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
		 at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
		 at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
		 at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:190)
		 at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:291)
		 at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:769)
		 at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:698)
		 at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:891)
		 at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:690)
		 at java.lang.Thread.run(Thread.java:679)
Caused by: com.redhat.rhn.common.hibernate.HibernateRuntimeException: Executing query DistChannelMap.findByChannel with params {channel=com.redhat.rhn.domain.channel.Channel@7e8138a6[id=144,label=centos6-x86_64]} failed
		 at com.redhat.rhn.common.hibernate.HibernateFactory.lookupObjectByNamedQuery(HibernateFactory.java:188)
		 at com.redhat.rhn.common.hibernate.HibernateFactory.lookupObjectByNamedQuery(HibernateFactory.java:158)
		 at com.redhat.rhn.domain.channel.ChannelFactory.lookupDistChannelMap(ChannelFactory.java:902)
		 at com.redhat.rhn.domain.channel.Channel.isChannelRepodataRequired(Channel.java:833)
		 at com.redhat.rhn.frontend.action.channel.ChannelDetailsAction.execute(ChannelDetailsAction.java:90)
		 at org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:425)
		 ... 40 more
Caused by: org.hibernate.exception.SQLGrammarException: could not execute query
		 at org.hibernate.exception.SQLStateConverter.convert(SQLStateConverter.java:67)
		 at org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:43)
		 at org.hibernate.loader.Loader.doList(Loader.java:2223)
		 at org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2104)
		 at org.hibernate.loader.Loader.list(Loader.java:2099)
		 at org.hibernate.loader.hql.QueryLoader.list(QueryLoader.java:378)
		 at org.hibernate.hql.ast.QueryTranslatorImpl.list(QueryTranslatorImpl.java:338)
		 at org.hibernate.engine.query.HQLQueryPlan.performList(HQLQueryPlan.java:172)
		 at org.hibernate.impl.SessionImpl.list(SessionImpl.java:1121)
		 at org.hibernate.impl.QueryImpl.list(QueryImpl.java:79)
		 at org.hibernate.impl.AbstractQueryImpl.uniqueResult(AbstractQueryImpl.java:811)
		 at com.redhat.rhn.common.hibernate.HibernateFactory.lookupObjectByNamedQuery(HibernateFactory.java:182)
		 ... 45 more
Caused by: org.postgresql.util.PSQLException: ERROR: column distchanne0_.id does not exist
  Position: 8
		 at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2062)
		 at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1795)
		 at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257)
		 at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:479)
		 at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:367)
		 at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271)
		 at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeQuery(NewProxyPreparedStatement.java:440)
		 at org.hibernate.jdbc.AbstractBatcher.getResultSet(AbstractBatcher.java:186)
		 at org.hibernate.loader.Loader.getResultSet(Loader.java:1787)
		 at org.hibernate.loader.Loader.doQuery(Loader.java:674)
		 at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:236)
		 at org.hibernate.loader.Loader.doList(Loader.java:2220)
		 ... 54 more

Comment 1 Tomas Lestach 2012-11-09 15:07:34 UTC
I am sorry, but upgrade from 1.8 to 1.9 nightly is not a valid use case.

Just curious, could you paste the output of 'SELECT * FROM rhnDistChannelMap;' query run in psql?

Comment 2 Joe Belliveau 2012-11-09 16:43:13 UTC
No problem, sorry new to bugzilla


spacewalk=> SELECT * FROM rhnDistChannelMap;
ERROR:  relation "rhndistchannelmap" does not exist
LINE 1: SELECT * FROM rhnDistChannelMap;

Comment 3 Joe Belliveau 2012-11-09 16:57:13 UTC
spacewalk=> \l
                                   List of databases
    Name     |  Owner   | Encoding |  Collation  |    Ctype    |   Access privileges   
-------------+----------+----------+-------------+-------------+-----------------------
 postgres    | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | 
 spaceschema | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | 
 spacewalk   | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =Tc/postgres
                                                               : postgres=CTc/postgres
                                                               : spaceman=CTc/postgres

Sorry i don't know the databsse layouts enough to be very helpful on this one

Comment 4 Jan Pazdziora (Red Hat) 2012-11-10 16:21:19 UTC
Could you please paste output of

select label, created, ( select evr from rhnpackageevr where evr_id = id ) as evr from rhnVersionInfo order by created ;

when run in your psql?

Also, you seem to have two databases that seem kinda related to spacewalk -- spaceschema and spacewalk. Which one is the correct one?

Can you describe what were the exact steps that you did? Was is a fresh 1.8 installation or upgrade from older version? How did you upgrade to nightly? Did the 1.8 version work?

Comment 5 Joe Belliveau 2012-11-12 19:08:39 UTC
spaceschema=> select label, created, ( select evr from rhnpackageevr where evr_id = id ) as evr from rhnVersionInfo order by created ;
ERROR:  relation "rhnversioninfo" does not exist
LINE 1: ...rom rhnpackageevr where evr_id = id ) as evr from rhnVersion...

spacewalk=> select label, created, ( select evr from rhnpackageevr where evr_id = id ) as evr from rhnVersionInfo order by created ;
ERROR:  relation "rhnversioninfo" does not exist
LINE 1: ...rom rhnpackageevr where evr_id = id ) as evr from rhnVersion...

Looks like neither work.

Comment 6 Joe Belliveau 2012-11-12 19:10:45 UTC
db_name = spaceschema
db_host = 127.0.0.1
db_port = 5432


Looks like spaceschema is the proper , from /etc/rhn/rhn.conf anyway.

Comment 7 Jan Pazdziora (Red Hat) 2012-11-13 07:39:02 UTC
(In reply to comment #5)
> spaceschema=> select label, created, ( select evr from rhnpackageevr where
> evr_id = id ) as evr from rhnVersionInfo order by created ;
> ERROR:  relation "rhnversioninfo" does not exist
> LINE 1: ...rom rhnpackageevr where evr_id = id ) as evr from rhnVersion...

What does

\d

print? Because it looks like your database is not populated at all.

We will also need to know the exact steps and history of the installation.

Comment 8 Joe Belliveau 2012-11-13 08:43:52 UTC
No relations on either db. So they are indeed empty.

Any logs that I can provide to prevent this from happening to others ?

Comment 9 Jan Pazdziora (Red Hat) 2012-11-13 09:10:25 UTC
(In reply to comment #8)
> No relations on either db. So they are indeed empty.

If you database is empty, it's really weird that you are able to start Spacewalk and log in to it.

> Any logs that I can provide to prevent this from happening to others ?

As I already wrote: we need to know how *exactly* you've installed the Spacewalk and how *exactly* you've upgraded it.

Comment 10 Joe Belliveau 2012-11-13 09:11:49 UTC
Rebuilt the db from scratch and lost all the channels and systems. Any logs I can provide that will show what commands where run against the db so we can prevent this in the future.

Thanks.

Comment 11 Joe Belliveau 2012-11-13 09:15:02 UTC
well as for the login I use pam/ad. 

The basically I followed the centos 6.2 write up except it is on a redhat host.

Upgrades and installs all came from rpm packages. One moment and I will find the link for the instal instructions.

Comment 12 Joe Belliveau 2012-11-13 09:18:53 UTC
http://computerglitch.net/bin/texts/CentOS6_Spacewalk_Server.php

Pretty much the steps.

Comment 13 Jan Pazdziora (Red Hat) 2012-11-13 09:27:41 UTC
(In reply to comment #12)
> http://computerglitch.net/bin/texts/CentOS6_Spacewalk_Server.php
> 
> Pretty much the steps.

This link describes installation of Spacewalk 1.5, not 1.8.

Comment 14 Joe Belliveau 2012-11-13 09:40:03 UTC
right the server started as a 1.5 then Upgraded to 1.7, then 1.8 then 1.9 nightly to solve some bugs I was running into.

Comment 15 Eric Herget 2017-09-28 17:57:02 UTC
This BZ closed some time during 2.5, 2.6 or 2.7.  Adding to 2.7 tracking bug.