Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1085033

Summary: rhnsearchd connecting to PostgreSQL over 127.0.0.1 regardless of whats configured in /etc/rhn.conf
Product: [Community] Spacewalk Reporter: Paul Robert Marino <prmarino1>
Component: ServerAssignee: Michael Mráka <mmraka>
Status: CLOSED CURRENTRELEASE QA Contact: Red Hat Satellite QA List <satqe-list>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 2.1CC: cperry, jesse, tlestach
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1146023 (view as bug list) Environment:
Last Closed: 2015-04-14 19:04:04 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1207293    
Attachments:
Description Flags
Patch for rhnsearchd on external PostgreSQL none

Description Paul Robert Marino 2014-04-07 15:05:43 UTC 
Description of problem:
I noticed that when rhnsearch is running it connects on the loopback to the database regardless of whats set in /etc/rhn.conf

Version-Release number of selected component (if applicable):
spacewalk-search-2.1.14-1.el6.noarch

How reproducible:
in my configuration its fairly easy I have two spacewalk servers running PostgreSQL 9.1 in hot spare syncronous replication mode. all of the processes except for taskomatic are running on both nodes.

Steps to Reproduce:
1.set db_host /etc/rhn/rhn.conf to an external host name
2. run /etc/init.d/rhn-search start
3. run ps ax |grep postgres|grep 127.0.0.1

you will see several connection to the loop back address

4. run /etc/init.d/rhn-search stop
5. run ps ax |grep postgres|grep 127.0.0.1

you will now see all of the PostgreSQL connections over the loop back have gone away


Actual results:

rhnsearchd is connecting to the database over the loopback


Expected results:

rhnsearchd should read the db_host variable in the configuration file and connect to the database host specified.


Additional info:

here is some of the log output from /var/log/rhn/search/rhn_search_daemon.log on the secondary host on which the database in in read only hot spare mode.

INFO   | jvm 1    | 2014/04/07 10:59:52 | 2014-04-07 10:59:52,915 [QuartzScheduler_Worker-3] ERROR org.quartz.core.ErrorLogger - Job (updateIndex.server threw an exception.
INFO   | jvm 1    | 2014/04/07 10:59:52 | org.quartz.SchedulerException: Job threw an unhandled exception. [See nested exception: org.apache.ibatis.exceptions.PersistenceException:
INFO   | jvm 1    | 2014/04/07 10:59:52 | ### Error updating database.  Cause: org.postgresql.util.PSQLException: ERROR: cannot execute UPDATE in a read-only transaction
INFO   | jvm 1    | 2014/04/07 10:59:52 | ### The error may involve server.updateLastServer-Inline
INFO   | jvm 1    | 2014/04/07 10:59:52 | ### The error occurred while setting parameters
INFO   | jvm 1    | 2014/04/07 10:59:52 | ### SQL: UPDATE rhnIndexerWork SET last_id = ?, last_modified = ? where object_type = 'server'
INFO   | jvm 1    | 2014/04/07 10:59:52 | ### Cause: org.postgresql.util.PSQLException: ERROR: cannot execute UPDATE in a read-only transaction]
INFO   | jvm 1    | 2014/04/07 10:59:52 |       at org.quartz.core.JobRunShell.run(JobRunShell.java:227)
INFO   | jvm 1    | 2014/04/07 10:59:52 |       at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:549)
INFO   | jvm 1    | 2014/04/07 10:59:52 | Caused by: org.apache.ibatis.exceptions.PersistenceException:
INFO   | jvm 1    | 2014/04/07 10:59:52 | ### Error updating database.  Cause: org.postgresql.util.PSQLException: ERROR: cannot execute UPDATE in a read-only transaction
INFO   | jvm 1    | 2014/04/07 10:59:52 | ### The error may involve server.updateLastServer-Inline
INFO   | jvm 1    | 2014/04/07 10:59:52 | ### The error occurred while setting parameters
INFO   | jvm 1    | 2014/04/07 10:59:52 | ### SQL: UPDATE rhnIndexerWork SET last_id = ?, last_modified = ? where object_type = 'server'
INFO   | jvm 1    | 2014/04/07 10:59:52 | ### Cause: org.postgresql.util.PSQLException: ERROR: cannot execute UPDATE in a read-only transaction
INFO   | jvm 1    | 2014/04/07 10:59:52 |       at org.apache.ibatis.exceptions.ExceptionFactory.wrapException(ExceptionFactory.java:23)
INFO   | jvm 1    | 2014/04/07 10:59:52 |       at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:150)
INFO   | jvm 1    | 2014/04/07 10:59:52 |       at com.redhat.satellite.search.db.WriteQuery.update(WriteQuery.java:57)
INFO   | jvm 1    | 2014/04/07 10:59:52 |       at com.redhat.satellite.search.scheduler.tasks.GenericIndexTask.updateLastRecord(GenericIndexTask.java:112)
INFO   | jvm 1    | 2014/04/07 10:59:52 |       at com.redhat.satellite.search.scheduler.tasks.GenericIndexTask.execute(GenericIndexTask.java:76)
INFO   | jvm 1    | 2014/04/07 10:59:52 |       at org.quartz.core.JobRunShell.run(JobRunShell.java:216)
INFO   | jvm 1    | 2014/04/07 10:59:52 |       ... 1 more
INFO   | jvm 1    | 2014/04/07 10:59:52 | Caused by: org.postgresql.util.PSQLException: ERROR: cannot execute UPDATE in a read-only transaction
INFO   | jvm 1    | 2014/04/07 10:59:52 |       at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2062)
INFO   | jvm 1    | 2014/04/07 10:59:52 |       at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1795)
INFO   | jvm 1    | 2014/04/07 10:59:52 |       at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257)
INFO   | jvm 1    | 2014/04/07 10:59:52 |       at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:479)
INFO   | jvm 1    | 2014/04/07 10:59:52 |       at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:367)
INFO   | jvm 1    | 2014/04/07 10:59:52 |       at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:360)
INFO   | jvm 1    | 2014/04/07 10:59:52 |       at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.execute(NewProxyPreparedStatement.java:362)
INFO   | jvm 1    | 2014/04/07 10:59:52 |       at org.apache.ibatis.executor.statement.PreparedStatementHandler.update(PreparedStatementHandler.java:41)
INFO   | jvm 1    | 2014/04/07 10:59:52 |       at org.apache.ibatis.executor.statement.RoutingStatementHandler.update(RoutingStatementHandler.java:66)
INFO   | jvm 1    | 2014/04/07 10:59:52 |       at org.apache.ibatis.executor.SimpleExecutor.doUpdate(SimpleExecutor.java:45)
INFO   | jvm 1    | 2014/04/07 10:59:52 |       at org.apache.ibatis.executor.BaseExecutor.update(BaseExecutor.java:100)
INFO   | jvm 1    | 2014/04/07 10:59:52 |       at org.apache.ibatis.executor.CachingExecutor.update(CachingExecutor.java:75)
INFO   | jvm 1    | 2014/04/07 10:59:52 |       at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:148)
INFO   | jvm 1    | 2014/04/07 10:59:52 |       ... 5 more
Comment 1 Jesse Weisner 2014-08-27 19:39:11 UTC 
Created attachment 931659 [details]
Patch for rhnsearchd on external PostgreSQL

This patch made rhnsearchd work with external PostgreSQL, but I haven't tested it with "embedded" PostgreSQL yet.

This patch is against the SRPM of the current stable version (2.2.8).
Comment 2 Tomas Lestach 2014-09-23 09:12:43 UTC 
Hello,

the patch does not seem to be correct.
The 'else' path at [1] constructs the 'db_name' according to your rhn.conf file.

What does
# egrep 'db_(host|name|port)' /etc/rhn/rhn.conf
say?

[1] https://github.com/spacewalkproject/spacewalk/blob/master/search-server/spacewalk-search/src/java/com/redhat/satellite/search/db/DatabaseManager.java#L76
Comment 3 Jesse Weisner 2014-09-23 21:54:01 UTC 
(In reply to Tomas Lestach from comment #2)
> Hello,
> 
> the patch does not seem to be correct.
> The 'else' path at [1] constructs the 'db_name' according to your rhn.conf
> file.
> 
> What does
> # egrep 'db_(host|name|port)' /etc/rhn/rhn.conf
> say?
> 
> [1]
> https://github.com/spacewalkproject/spacewalk/blob/master/search-server/
> spacewalk-search/src/java/com/redhat/satellite/search/db/DatabaseManager.
> java#L76

# egrep 'db_(host|name|port)' /etc/rhn/rhn.conf
db_name = spacewalk
db_host = pgsql.example.com
db_port = 5432
Comment 4 Jesse Weisner 2014-09-23 22:04:06 UTC 
Without adding db_host to the list of "options" (which is what I assume https://github.com/spacewalkproject/spacewalk/blob/master/search-server/spacewalk-search/src/java/com/redhat/satellite/search/db/DatabaseManager.java#L69 does), db_host is empty, so it can't get added to the jdbcUrl property at https://github.com/spacewalkproject/spacewalk/blob/master/search-server/spacewalk-search/src/config/com/redhat/satellite/search/db/config.xml#L19

This made rhn-search work for me, but I have no idea if it's the right solution 8-)
Comment 5 Tomas Lestach 2014-09-24 10:14:04 UTC 
That you for the provided information, now I see the issue.

spacewalk.git: 4073abaf7bc7f17b738bcc0ca64e657ab9d074bf
Comment 6 Grant Gainey 2015-03-23 16:59:34 UTC 
Moving bugs to ON_QA as we move to release Spacewalk 2.3
Comment 7 Grant Gainey 2015-04-14 19:04:04 UTC 
Spacewalk 2.3 has been released. See

https://fedorahosted.org/spacewalk/wiki/ReleaseNotes23