Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1342014

Summary: Upgrade: POST "/api/hosts/facts" raises PG::Error: ERROR: relation "katello_content_facets" and "katello_subscription_facets" does not exist
Product: Red Hat Satellite Reporter: Sachin Ghai <sghai>
Component: UpgradesAssignee: satellite6-bugs <satellite6-bugs>
Status: CLOSED ERRATA QA Contact: Sachin Ghai <sghai>
Severity: high Docs Contact:
Priority: unspecified    
Version: 6.2.0CC: bbuckingham, jsherril
Target Milestone: UnspecifiedKeywords: Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-07-27 11:09:10 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1335807    
Attachments:
Description Flags
complete execption related to ERROR: relation "katello_content_facets" does not exist
none
complete exception related to ERROR: relation "katello_subscription_facets" does not exist
none
foreman-debug none

Description Sachin Ghai 2016-06-02 09:15:18 UTC
Description of problem:
I got following exception in production.log while performing upgrade from sat6.1.9 => sat6.2 GA snap14 on rhel7

==> /var/log/foreman/production.log <==
2016-06-02 03:18:28 [app] [I] Started POST "/api/hosts/facts" for 10.8.119.1 at 2016-06-02 03:18:28 -0400
2016-06-02 03:18:28 [app] [I] Processing by Api::V2::HostsController#facts as JSON
2016-06-02 03:18:28 [app] [I]   Parameters: {"facts"=>"[FILTERED]", "name"=>"qe-sat6-upgrade-rhel7.satqe.lab.eng.rdu2.redhat.com", "certname"=>"qe-sat6-upgrade-rhel7.satqe.lab.eng.rdu2.redhat.com", "apiv"=>"v2", :host=>{"name"=>"qe-sat6-upgrade-rhel7.satqe.lab.eng.rdu2.redhat.com", "certname"=>"qe-sat6-upgrade-rhel7.satqe.lab.eng.rdu2.redhat.com"}}
2016-06-02 03:18:28 [app] [I] Import facts for 'qe-sat6-upgrade-rhel7.satqe.lab.eng.rdu2.redhat.com' completed. Added: 11, Updated: 14, Deleted 2 facts
2016-06-02 03:18:28 [app] [W] Action failed
 | PG::Error: ERROR:  relation "katello_subscription_facets" does not exist
 | LINE 5:                WHERE a.attrelid = '"katello_subscription_fac...
 |                                           ^
 | 
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/connection_adapters/postgresql_adapter.rb:822:in `exec'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/connection_adapters/postgresql_adapter.rb:822:in `block in exec_no_cache'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/connection_adapters/abstract_adapter.rb:373:in `block in log'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/connection_adapters/abstract_adapter.rb:367:in `log'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/connection_adapters/postgresql_adapter.rb:822:in `exec_no_cache'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/connection_adapters/postgresql/database_statements.rb:137:in `exec_query'


==> /var/log/foreman/production.log <==
2016-06-02 03:18:29 [app] [I] Started POST "/api/reports" for 10.8.119.1 at 2016-06-02 03:18:29 -0400
2016-06-02 03:18:29 [app] [I] Processing by Api::V2::ReportsController#create as JSON
2016-06-02 03:18:29 [app] [I]   Parameters: {"report"=>"[FILTERED]", "apiv"=>"v2"}
2016-06-02 03:18:29 [app] [I] processing report for qe-sat6-upgrade-rhel7.satqe.lab.eng.rdu2.redhat.com
2016-06-02 03:18:29 [app] [W] Action failed
 | PG::Error: ERROR:  relation "katello_subscription_facets" does not exist
 | LINE 5:                WHERE a.attrelid = '"katello_subscription_fac...
 |                                           ^
 | 
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/connection_adapters/postgresql_adapter.rb:822:in `exec'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/connection_adapters/postgresql_adapter.rb:822:in `block in exec_no_cache'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/connection_adapters/abstract_adapter.rb:373:in `block in log'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/connection_adapters/abstract_adapter.rb:367:in `log'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/connection_adapters/postgresql_adapter.rb:822:in `exec_no_cache'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/connection_adapters/postgresql/database_statements.rb:137:in `exec_query'


these execeptions were raised when installer failed at:

 Upgrade Step: Running installer...
 /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]: Failed to call refresh: /usr/sbin/foreman-rake db:migrate returned 1 instead of one of [0]
 /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]: /usr/sbin/foreman-rake db:migrate returned 1 instead of one of [0]
 /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]: Failed to call refresh: /usr/sbin/foreman-rake db:seed returned 1 instead of one of [0]
 /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]: /usr/sbin/foreman-rake db:seed returned 1 instead of one of [0]
Installing             Done                                               [100%] [..................................................................]
  Something went wrong! Check the log for ERROR-level output
  The full log is at /var/log/foreman-installer/satellite.log
Upgrade Step: restart_services...
Upgrade Step: db_seed...
Upgrade step db_seed failed. Check logs for more information.


Version-Release number of selected component (if applicable):
upgrade from sat6.1.9 => sat6.2 GA snap14 on rhel7

How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:
No exception should be raised in logs while performing upgrade.

Additional info:

Comment 1 Sachin Ghai 2016-06-02 09:17:00 UTC
Created attachment 1163990 [details]
complete execption related to ERROR:  relation "katello_content_facets" does not exist

Comment 2 Sachin Ghai 2016-06-02 09:18:24 UTC
Created attachment 1163991 [details]
complete exception related  to ERROR:  relation "katello_subscription_facets" does not exist

Comment 3 Sachin Ghai 2016-06-02 09:20:07 UTC
Created attachment 1164004 [details]
foreman-debug

Comment 5 Justin Sherrill 2016-06-02 12:29:43 UTC
The fact that this failed:

 /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]: Failed to call refresh: /usr/sbin/foreman-rake db:migrate returned 1 instead of one of [0]

indicates that the database was not fully migrated and would be missing a bunch of tables, columns, and what not.  I would expect a TON of different errors when in this state.  It seems this was the result of https://bugzilla.redhat.com/show_bug.cgi?id=1341974

I would suggest this is NOTABUG as it should go away when https://bugzilla.redhat.com/show_bug.cgi?id=1341974 is fixed.  We can leave it open until if you'd like.

Comment 6 Sachin Ghai 2016-06-02 12:42:16 UTC
Thanks Justin, Lets keep it and will close it with next snap, if can't reproduce it.

Comment 7 Sachin Ghai 2016-06-03 07:57:41 UTC
Verified with upgrade from sat6.1.9 -> sat6.2 GA snap14.2 on rhel7.

I was not able to reproduce this issue. As mentioned in comment5, it was appearing due to a different bz 1341974. Moving this to verified.

And this time upgrade was completed successfully.

Comment 8 Sachin Ghai 2016-06-03 08:30:39 UTC
Just to correct: Verification is done with upgrade from sat6.1.9 -> sat6.2 GA snap14.1 on rhel7. Rpm version is 14.2

satellite-6.2.0-14.2.el7sat.noarch

Comment 9 Bryan Kearney 2016-07-27 11:09:10 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:1501