Bug 1339793

Summary: Upgrade error (db_migrate.log): AddOstreeBranches: uninitialized constant Katello::ContentView::CONTENT_DIR
Product: Red Hat Satellite Reporter: sthirugn <sthirugn>
Component: UpgradesAssignee: Brad Buckingham <bbuckingham>
Status: CLOSED ERRATA QA Contact: Katello QA List <katello-qa-list>
Severity: high Docs Contact:
Priority: unspecified    
Version: 6.2.0CC: bbuckingham, bkearney, lpramuk, sghai, sthirugn
Target Milestone: UnspecifiedKeywords: Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-07-27 11:21:01 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1335807    

Description sthirugn@redhat.com 2016-05-25 22:03:05 UTC
Description of problem:
Upgrade from sat 6.1.9 to sat 6.2 failed

Version-Release number of selected component (if applicable):
sat-6.2.0-GA-Snap13

How reproducible:
Always

Steps to Reproduce:
1.
# satellite-installer --scenario satellite --upgrade
Upgrading...
Upgrade Step: stop_services...
Upgrade Step: start_databases...
Upgrade Step: update_http_conf...
Upgrade Step: migrate_pulp...
Upgrade Step: fix_pulp_httpd_conf...
Upgrade Step: start_httpd...
Upgrade Step: start_qpidd...
Upgrade Step: start_pulp...
Upgrade Step: migrate_candlepin...
Upgrade Step: start_tomcat...
Upgrade Step: fix_katello_settings_file...
Upgrade Step: migrate_foreman...
Upgrade Step: migrate_gutterball...
Upgrade Step: remove_nodes_distributors...
Upgrade Step: Running installer...
 /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]: Failed to call refresh: /usr/sbin/foreman-rake db:migrate returned 1 instead of one of [0]
 /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]: /usr/sbin/foreman-rake db:migrate returned 1 instead of one of [0]
 /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]: Failed to call refresh: /usr/sbin/foreman-rake db:seed returned 1 instead of one of [0]
 /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]: /usr/sbin/foreman-rake db:seed returned 1 instead of one of [0]
Installing             Done                                               [100%] [...............................................]
  Something went wrong! Check the log for ERROR-level output
  The full log is at /var/log/foreman-installer/satellite.log
Upgrade Step: restart_services...
Upgrade Step: db_seed...
Upgrade step db_seed failed. Check logs for more information.


Actual results:
Error as shown above

Expected results:
Successful upgrade

Additional info:

Comment 3 Lukas Pramuk 2016-05-26 08:59:45 UTC
Picking out interesting stuff - db_migrate.log:

== 20150513034751 AddOstreeBranches: migrating ================================
-- create_table(:katello_ostree_branches)rake aborted!
StandardError: An error has occurred, this and all later migrations canceled:

uninitialized constant Katello::ContentView::CONTENT_DIR/opt/theforeman/tfm/root/usr/share/gems/gems/foreman_hooks-0.3.9/lib/foreman_hooks.rb:83:in `load_missing_constant_with_hooks'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.35/app/models/katello/kt_environment.rb:54:in `<class:KTEnvironment>'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.35/app/models/katello/kt_environment.rb:2:in `<module:Katello>'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.35/app/models/katello/kt_environment.rb:1:in `<top (required)>'
/opt/rh/rh-ror41/root/usr/share/gems/gems/polyglot-0.3.4/lib/polyglot.rb:65:in `require'
...

== 20150826170004 AddContentFacet: migrating ==================================
-- create_table("katello_content_facets")
   -> 0.0422s
-- add_index(:katello_content_facets, [:host_id], {:unique=>true, :name=>:katello_content_facets_host_id})
   -> 0.0034s
-- add_foreign_key("katello_content_facets", "hosts", {:name=>"katello_content_facets_host_id", :column=>"host_id"})
   -> 0.0020s
-- add_foreign_key("katello_content_facets", "katello_content_views", {:name=>"katello_content_facets_content_view_id", :column=>"content_view_id"})
   -> 0.0017s
-- add_foreign_key("katello_content_facets", "katello_environments", {:name=>"katello_content_facets_life_environment_id", :column=>"lifecycle_environment_id"})
   -> 0.0015s
-- create_table("katello_installed_packages")
   -> 0.0060s
-- create_table("katello_host_installed_packages")
   -> 0.0266s
-- add_foreign_key("katello_host_installed_packages", "hosts", {:name=>"katello_host_installed_packages_host_id", :column=>"host_id"})rake aborted!
StandardError: An error has occurred, this and all later migrations canceled:

Some backend services are not running: {:status=>"FAIL", :services=>{:pulp=>{:status=>"FAIL", :message=>"404 Resource Not Found"}, :pulp_auth=>{:status=>"FAIL", :message=>"Skipped pulp_auth check after failed pulp check"}, :candlepin=>{:status=>"FAIL", :message=>"404 Resource Not Found"}, :candlepin_auth=>{:status=>"FAIL", :message=>"Katello::Resources::Candlepin::CandlepinPing: 404 Resource Not Found  (GET /candlepin/status)"}, :foreman_tasks=>{:status=>"ok", :duration_ms=>"4"}}}/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.35/db/migrate/20150930183738_migrate_content_hosts.rb:297:in `up'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/migration.rb:598:in `exec_migration'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/migration.rb:579:in `block (2 levels) in migrate'
...

== 20150930183738 MigrateContentHosts: migrating ==============================
rake aborted!
StandardError: An error has occurred, this and all later migrations canceled:

Some backend services are not running: {:status=>"FAIL", :services=>{:pulp=>{:status=>"FAIL", :message=>"404 Resource Not Found"}, :pulp_auth=>{:status=>"FAIL", :message=>"Skipped pulp_auth check after failed pulp check"}, :candlepin=>{:status=>"FAIL", :message=>"404 Resource Not Found"}, :candlepin_auth=>{:status=>"FAIL", :message=>"Katello::Resources::Candlepin::CandlepinPing: 404 Resource Not Found  (GET /candlepin/status)"}, :foreman_tasks=>{:status=>"ok", :duration_ms=>"6"}}}/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.35/db/migrate/20150930183738_migrate_content_hosts.rb:297:in `up'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/migration.rb:598:in `exec_migration'

Comment 4 Lukas Pramuk 2016-05-26 09:03:41 UTC
Some backend services are not running -> pulp
- I think we have this already reported, though not sure if also for pulp service

AddOstreeBranches: uninitialized constant Katello::ContentView::CONTENT_DIR
- this is unique for this bug report and quite interesting

Comment 6 Brad Buckingham 2016-05-31 14:55:38 UTC
Update regarding the errors noted with this bug:

ERROR 1: Some backend services are not running -> pulp

   -> UPDATE: This was fixed with bug 1336941

ERROR 2 (from satellite.log) Could not find the source association(s) :content_facets in model MigrateContentHosts::ContentFacetRepository. Try 'has_many :bound_repositories, :through => :content_facet_repositories, :source => <name>'. Is it one of content_facet or repository?/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/reflection.rb:698:in `check_validity!'

    -> UPDATE: This is fixed with bug 1339682

ERROR 3: AddOstreeBranches: uninitialized constant Katello::ContentView::CONTENT_DIR

    -> UPDATE: This is not captured by any other bugs.

I'd propose we close this bug and create a separate one to document ERROR 3.  Alternatively, we can re-purpose this bug to fix that error only.

Comment 7 Brad Buckingham 2016-05-31 15:06:43 UTC
Based on discussion in IRC, we'll repurpose this bug to address ERROR 3 referenced in comment 6.


ERROR 3 (from db_migrate.log): AddOstreeBranches: uninitialized constant Katello::ContentView::CONTENT_DIR

Comment 9 sthirugn@redhat.com 2016-06-07 20:57:53 UTC
Closing this bug in favor of other specific upgrade bugs.

Comment 10 Bryan Kearney 2016-07-27 11:21:01 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:1501