Bug 1253010 - Satellite 6.1 upgrade fails with "errata_import failed"
Satellite 6.1 upgrade fails with "errata_import failed"
Status: CLOSED ERRATA
Product: Red Hat Satellite 6
Classification: Red Hat
Component: Upgrades (Show other bugs)
6.1.0
Unspecified Unspecified
unspecified Severity high (vote)
: Unspecified
: --
Assigned To: satellite6-bugs
Katello QA List
http://projects.theforeman.org/issues...
:
: 1253006 (view as bug list)
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-08-12 13:36 EDT by Dylan Gross
Modified: 2017-02-23 14:48 EST (History)
13 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2015-08-13 11:09:33 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 1575043 None None None Never

  None (edit)
Description Dylan Gross 2015-08-12 13:36:52 EDT
Description of problem:

 During an upgrade from Satellite v6.0.8, the katello-installer fails with:

    Tasks: TOP => katello:upgrades:2.1:import_errata
    (See full trace by running task with --trace)
    Importing Errata


Version-Release number of selected component (if applicable):

  Red Hat Satellite v6.1

How reproducible:

  From list, it appears this can happen when a private repo exists in Pulp, but not katello:  https://github.com/Katello/katello/pull/5358

Steps to Reproduce:
1.   Unknown

Actual results:

  errata_import fails.

Expected results:

  errata_import succeeds during "katello-installer --upgrade"

Additional info:



# foreman-rake katello:upgrades:2.1:import_errata --trace
** Invoke katello:upgrades:2.1:import_errata (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute katello:upgrades:2.1:import_errata
Importing Errata
rake aborted!
undefined method `id' for nil:NilClass
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.65/app/models/katello/glue/pulp/pulp_content_unit.rb:80:in `block in sync_repository_associations'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.65/app/models/katello/glue/pulp/pulp_content_unit.rb:80:in `map'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.65/app/models/katello/glue/pulp/pulp_content_unit.rb:80:in `sync_repository_associations'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.65/app/models/katello/glue/pulp/pulp_content_unit.rb:113:in `block (2 levels) in update_repository_associations'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.65/app/models/katello/glue/pulp/pulp_content_unit.rb:112:in `each'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.65/app/models/katello/glue/pulp/pulp_content_unit.rb:112:in `block in update_repository_associations'
/opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/connection_adapters/abstract/database_statements.rb:192:in `transaction'
/opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/transactions.rb:208:in `transaction'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.65/app/models/katello/glue/pulp/pulp_content_unit.rb:103:in `update_repository_associations'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.65/app/models/katello/glue/pulp/pulp_content_unit.rb:60:in `import_all'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.65/lib/katello/tasks/upgrades/2.1/import_errata.rake:27:in `block (4 levels) in <top (required)>'
/opt/rh/ruby193/root/usr/share/ruby/rake/task.rb:205:in `call'
/opt/rh/ruby193/root/usr/share/ruby/rake/task.rb:205:in `block in execute'
/opt/rh/ruby193/root/usr/share/ruby/rake/task.rb:200:in `each'
/opt/rh/ruby193/root/usr/share/ruby/rake/task.rb:200:in `execute'
/opt/rh/ruby193/root/usr/share/ruby/rake/task.rb:158:in `block in invoke_with_call_chain'
/opt/rh/ruby193/root/usr/share/ruby/monitor.rb:211:in `mon_synchronize'
/opt/rh/ruby193/root/usr/share/ruby/rake/task.rb:151:in `invoke_with_call_chain'
/opt/rh/ruby193/root/usr/share/ruby/rake/task.rb:144:in `invoke'
/opt/rh/ruby193/root/usr/share/ruby/rake/application.rb:116:in `invoke_task'
/opt/rh/ruby193/root/usr/share/ruby/rake/application.rb:94:in `block (2 levels) in top_level'
/opt/rh/ruby193/root/usr/share/ruby/rake/application.rb:94:in `each'
/opt/rh/ruby193/root/usr/share/ruby/rake/application.rb:94:in `block in top_level'
/opt/rh/ruby193/root/usr/share/ruby/rake/application.rb:133:in `standard_exception_handling'
/opt/rh/ruby193/root/usr/share/ruby/rake/application.rb:88:in `top_level'
/opt/rh/ruby193/root/usr/share/ruby/rake/application.rb:66:in `block in run'
/opt/rh/ruby193/root/usr/share/ruby/rake/application.rb:133:in `standard_exception_handling'
/opt/rh/ruby193/root/usr/share/ruby/rake/application.rb:63:in `run'
/opt/rh/ruby193/root/usr/bin/rake:32:in `<main>'
Tasks: TOP => katello:upgrades:2.1:import_errata
Comment 1 Bryan Kearney 2015-08-12 13:41:26 EDT
Connecting redmine issue http://projects.theforeman.org/issues/11140 from this bug
Comment 3 Bryan Kearney 2015-08-12 14:29:53 EDT
*** Bug 1253006 has been marked as a duplicate of this bug. ***
Comment 5 Justin Sherrill 2015-08-12 15:04:13 EDT
Steps to reproduce:

1.  install a satellite 6.0.8 Satellite
2.  Create a custom yum repository in the UI and point it to some upstream url with errata, https://jlsherrill.fedorapeople.org/fake-repos/needed-errata/ is a good example.
3.  Sync the custom repo, verify that there is at least one errata
4.  delete the repo from the katello db using these commands:

# foreman-rake console

Katello::Repository.find(5).destroy

replacing '5' with the id of the repo.

5.  Upgrade to 6.1.1
Comment 8 Sachin Ghai 2015-08-13 03:07:30 EDT
On sat6.0.8, I tried steps mentioned in comment 5

[root@cloud-qe-6 ~]# hammer -u admin -p changeme repository list --organization-id 1
---|------|---------|--------------|--------------------------------------------------------------
ID | NAME | PRODUCT | CONTENT TYPE | URL                                                          
---|------|---------|--------------|--------------------------------------------------------------
1  | fake | bug     | yum          | https://jlsherrill.fedorapeople.org/fake-repos/needed-errata/
---|------|---------|--------------|--------------------------------------------------------------


[root@cloud-qe-6 ~]# foreman-rake console
Loading production environment (Rails 3.2.8)

irb(main):001:0> Katello::Repository.find(1)        
  Katello::Repository Load (0.7ms)  SELECT "katello_repositories".* FROM "katello_repositories" WHERE "katello_repositories"."id" = $1 ORDER BY katello_repositories.name ASC LIMIT 1  [["id", 1]]
=> #<Katello::Repository id: 1, name: "fake", pulp_id: "Default_Organization-bug-fake", created_at: "2015-08-13 06:59:45", updated_at: "2015-08-13 07:00:25", major: nil, minor: nil, gpg_key_id: nil, cp_label: nil, library_instance_id: nil, content_id: "1439449186169", arch: "noarch", label: "fake", content_view_version_id: 1, relative_path: "Default_Organization/Library/custom/bug/fake", url: "https://jlsherrill.fedorapeople.org/fake-repos/need...", unprotected: true, content_type: "yum", product_id: 1, environment_id: 1, checksum_type: "sha256">
Comment 9 Sachin Ghai 2015-08-13 03:09:24 EDT
however on "Katello::Repository.find(1).destroy", I got traceback, is it expected ?



irb(main):002:0> Katello::Repository.find(1).destroy
  Katello::Repository Load (0.4ms)  SELECT "katello_repositories".* FROM "katello_repositories" WHERE "katello_repositories"."id" = $1 ORDER BY katello_repositories.name ASC LIMIT 1  [["id", 1]]
   (0.1ms)  BEGIN
  Katello::KTEnvironment Load (0.4ms)  SELECT "katello_environments".* FROM "katello_environments" WHERE "katello_environments"."id" = 1 LIMIT 1
  Katello::ContentViewVersion Load (0.4ms)  SELECT "katello_content_view_versions".* FROM "katello_content_view_versions" WHERE "katello_content_view_versions"."id" = 1 LIMIT 1
  Katello::ContentView Load (0.5ms)  SELECT "katello_content_views".* FROM "katello_content_views" WHERE "katello_content_views"."id" = 1 LIMIT 1
  Organization Load (0.5ms)  SELECT "taxonomies".* FROM "taxonomies" WHERE "taxonomies"."type" IN ('Organization') AND "taxonomies"."id" = 1 ORDER BY title LIMIT 1
   (0.7ms)  SELECT COUNT(*) FROM "foreman_tasks_tasks" INNER JOIN "foreman_tasks_locks" ON "foreman_tasks_locks"."task_id" = "foreman_tasks_tasks"."id" WHERE "foreman_tasks_tasks"."type" IN ('ForemanTasks::Task::DynflowTask') AND "foreman_tasks_tasks"."label" = 'Actions::Katello::Organization::Destroy' AND "foreman_tasks_locks"."resource_id" = 1 AND "foreman_tasks_locks"."resource_type" = 'Organization' AND (state != 'stopped')
  Katello::Product Load (0.5ms)  SELECT "katello_products".* FROM "katello_products" WHERE "katello_products"."id" = 1 LIMIT 1
  Katello::Provider Load (0.4ms)  SELECT "katello_providers".* FROM "katello_providers" WHERE "katello_providers"."id" = 1 LIMIT 1
   (0.2ms)  ROLLBACK
NoMethodError: undefined method `can?' for nil:NilClass
	from /opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/models/katello/authorization.rb:22:in `authorized?'
	from /opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/models/katello/authorization/product.rb:30:in `editable?'
	from /opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/models/katello/authorization/repository.rb:29:in `deletable?'
	from /opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/models/katello/repository.rb:431:in `assert_deletable'
	from /opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:407:in `_run__3452354115620553830__destroy__4235214515640820274__callbacks'
	from /opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:405:in `__run_callback'
	from /opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:385:in `_run_destroy_callbacks'
	from /opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:81:in `run_callbacks'
	from /opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/callbacks.rb:254:in `destroy'
	from /opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/transactions.rb:236:in `block in destroy'
	from /opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/transactions.rb:295:in `block in with_transaction_returning_status'
	from /opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/connection_adapters/abstract/database_statements.rb:192:in `transaction'
	from /opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/transactions.rb:208:in `transaction'
	from /opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/transactions.rb:293:in `with_transaction_returning_status'
	from /opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/transactions.rb:236:in `destroy'
	from /opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/models/katello/glue.rb:94:in `destroy'
... 6 levels...
	from /opt/rh/ruby193/root/usr/share/ruby/rake/task.rb:200:in `each'
	from /opt/rh/ruby193/root/usr/share/ruby/rake/task.rb:200:in `execute'
	from /opt/rh/ruby193/root/usr/share/ruby/rake/task.rb:158:in `block in invoke_with_call_chain'
	from /opt/rh/ruby193/root/usr/share/ruby/monitor.rb:211:in `mon_synchronize'
	from /opt/rh/ruby193/root/usr/share/ruby/rake/task.rb:151:in `invoke_with_call_chain'
	from /opt/rh/ruby193/root/usr/share/ruby/rake/task.rb:144:in `invoke'
	from /opt/rh/ruby193/root/usr/share/ruby/rake/application.rb:116:in `invoke_task'
	from /opt/rh/ruby193/root/usr/share/ruby/rake/application.rb:94:in `block (2 levels) in top_level'
	from /opt/rh/ruby193/root/usr/share/ruby/rake/application.rb:94:in `each'
	from /opt/rh/ruby193/root/usr/share/ruby/rake/application.rb:94:in `block in top_level'
	from /opt/rh/ruby193/root/usr/share/ruby/rake/application.rb:133:in `standard_exception_handling'
	from /opt/rh/ruby193/root/usr/share/ruby/rake/application.rb:88:in `top_level'
	from /opt/rh/ruby193/root/usr/share/ruby/rake/application.rb:66:in `block in run'
	from /opt/rh/ruby193/root/usr/share/ruby/rake/application.rb:133:in `standard_exception_handling'
	from /opt/rh/ruby193/root/usr/share/ruby/rake/application.rb:63:in `run'
	from /opt/rh/ruby193/root/usr/bin/rake:32:in `<main>'irb(main):003:0>
Comment 10 Sachin Ghai 2015-08-13 03:13:03 EDT
And I can see same repository after step: "Katello::Repository.find(1).destroy"

 `<main>'irb(main):005:0> Katello::Repository.find(1)
  Katello::Repository Load (0.4ms)  SELECT "katello_repositories".* FROM "katello_repositories" WHERE "katello_repositories"."id" = $1 ORDER BY katello_repositories.name ASC LIMIT 1  [["id", 1]]
=> #<Katello::Repository id: 1, name: "fake", pulp_id: "Default_Organization-bug-fake", created_at: "2015-08-13 06:59:45", updated_at: "2015-08-13 07:00:25", major: nil, minor: nil, gpg_key_id: nil, cp_label: nil, library_instance_id: nil, content_id: "1439449186169", arch: "noarch", label: "fake", content_view_version_id: 1, relative_path: "Default_Organization/Library/custom/bug/fake", url: "https://jlsherrill.fedorapeople.org/fake-repos/need...", unprotected: true, content_type: "yum", product_id: 1, environment_id: 1, checksum_type: "sha256">
Comment 11 Sachin Ghai 2015-08-13 03:58:08 EDT
Later, after the above traceback, I went ahead and tried upgrade.

[root@cloud-qe-6 ~]# katello-installer --upgrade
Upgrading...
Upgrade Step: stop_services...
Upgrade Step: start_mongo...
Upgrade Step: migrate_pulp...
Upgrade Step: start_httpd...
Upgrade Step: migrate_candlepin...
Upgrade Step: migrate_foreman...
Upgrade Step: Running installer...
Installing             Done                                               [100%] [..................................................................]
  The full log is at /var/log/katello-installer/katello-installer.log
Upgrade Step: restart_services...
Upgrade Step: db_seed...
Upgrade Step: errata_import (this may take a while) ...
Upgrade Step: update_gpg_urls (this may take a while) ...
Upgrade Step: update_repository_metadata (this may take a while) ...
Katello upgrade completed!


it was completed successfully.
Comment 13 Sachin Ghai 2015-08-13 04:53:03 EDT
After above steps (comment 12): I performed upgrade with snap16 and upgrade was completed successfully on rhel71.


[root@cloud-qe-7 ~]# katello-installer --upgrade
Upgrading...
Upgrade Step: stop_services...
Upgrade Step: start_mongo...
Upgrade Step: migrate_pulp...
Upgrade Step: start_httpd...
Upgrade Step: migrate_candlepin...
Upgrade Step: migrate_foreman...
Upgrade Step: Running installer...
Installing             Done                                               [100%] [..................................................................]
  The full log is at /var/log/katello-installer/katello-installer.log
Upgrade Step: restart_services...
Upgrade Step: db_seed...
Upgrade Step: errata_import (this may take a while) ...
Upgrade Step: update_gpg_urls (this may take a while) ...
Upgrade Step: update_repository_metadata (this may take a while) ...
Katello upgrade completed!
[root@cloud-qe-7 ~]#
Comment 14 Sachin Ghai 2015-08-13 05:40:31 EDT
On rhel66 setup, I installed sat6.0.8 and upgraded the setup with GA RC to recreate the original issue. And I was able to re-create the issue.

Later, I upgraded the setup with sanp16. and found two packages were updated:

--
ruby193-rubygem-foreman-redhat_access                   noarch                   0.2.2-1.el6_6sat                       sat6                    10 M
 ruby193-rubygem-katello                                 noarch                   2.2.0.66-1.el6_6sat                    sat6                   4.0 M
--


After updaing above packages, I re-run the "katello-installer --upgrade" and this time upgrade completed sucessfully. thanks
Comment 17 Sachin Ghai 2015-08-13 07:53:30 EDT
Verified with sat6.1 snap16. that includes two updated packages :

--
ruby193-rubygem-foreman-redhat_access                   noarch                   0.2.2-1.el6_6sat                       sat6                    10 M
 ruby193-rubygem-katello                                 noarch                   2.2.0.66-1.el6_6sat                    sat6                   4.0 M
--

And as per comment 13 and comment 14, the reported is issue is fixed with new snap. thanks
Comment 19 errata-xmlrpc 2015-08-13 11:09:33 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2015:1620

Note You need to log in before you can comment on or make changes to this bug.