Satellite 6.1.11 (not .10 as in the BZ version, seems there is no .11?)
Description of problem:
While running "foreman-rake katello:upgrade_check" on a big customer database, rake would abort while scanning the systems:
$ foreman-rake katello:preupgrade_content_host_check --trace
** Invoke katello:preupgrade_content_host_check (first_time)
** Invoke environment (first_time)
** Execute environment
API controllers newer than Apipie cache! Run apipie:cache rake task to regenerate cache.
** Execute katello:preupgrade_content_host_check
Calculating Host changes on upgrade. This may take a few minutes.
Expect initializer to return hash if a group of attributes is defined by lazy_accessor
/opt/rh/ruby193/root/usr/share/gems/gems/katello-22.214.171.124/app/lib/katello/lazy_accessor.rb:74:in `block (2 levels) in lazy_accessor'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-126.96.36.199/lib/katello/tasks/preupgrade_content_host_check.rake:48:in `block in get_systems_with_facts'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-188.8.131.52/lib/katello/tasks/preupgrade_content_host_check.rake:103:in `block (2 levels) in <top (required)>'
/opt/rh/ruby193/root/usr/share/ruby/rake/task.rb:205:in `block in execute'
/opt/rh/ruby193/root/usr/share/ruby/rake/task.rb:158:in `block in invoke_with_call_chain'
/opt/rh/ruby193/root/usr/share/ruby/rake/application.rb:94:in `block (2 levels) in top_level'
/opt/rh/ruby193/root/usr/share/ruby/rake/application.rb:94:in `block in top_level'
/opt/rh/ruby193/root/usr/share/ruby/rake/application.rb:66:in `block in run'
Tasks: TOP => katello:preupgrade_content_host_check
The code in question seems to be:
systems.each do |system|
facts = system.facts
Line 48 is "facts = system.facts".
After adding a tactical "puts system.inspect" just before the system.facts line, we could identify the bad system:
#<Katello::System id: 6891, uuid: nil, name: "hostname", description: "Initial Registration Params", location: "None", environment_id: 4, created_at: "2016-11-15 12:58:01", updated_at: "2016-11-15 12:58:01", type: "Katello::System", content_view_id: 15, host_id: nil>
Looking into PostgreSQL revealed that we actually had two systems with that symptom:
foreman=# select * from katello_systems where uuid is null;
id | uuid | name | description | location | environment_id | created_at | updated_at | type | content_view_id | host_id
6891 | | hostname | Initial Registration Params | None | 4 | 2016-11-15 12:58:01.246012 | 2016-11-15 12:58:01.246012 | Katello::System | 15 |
6262 | | hostname2 | Initial Registration Params | None | 4 | 2016-09-26 09:06:38.945969 | 2016-09-26 09:06:38.945969 | Katello::System | 16 |
PostgreSQL would also tell us that there were another two systems with those hostnames, but now with proper UUIDs.
Seems the initial registration of those wen't badly and they were re-registered.
After erasing the two broken systems from the DB the upgrade_check would run fine.
I think the upgrade_check.rake needs a bit more of error handling, as I would expect it to catch this bad systems and tell me about them, not choke on them.
Version-Release number of selected component (if applicable):
Always, but no idea how the initial problematic host was created
Steps to Reproduce:
1. create a katello::system without a uuid
2. run foreman-rake katello:upgrade_check
system is said to be faulty
The reason for this I can imagine could be a failed orchestration of the host, so that was never finished (perhaps the task was force-unlocked).
I agree the upgrade check should handle such a situatino
Created redmine issue http://projects.theforeman.org/issues/17905 from this bug
It seems it's similar to https://bugzilla.redhat.com/show_bug.cgi?id=1329561
FWIW, those hosts are content-only, so there was no corresponding Foreman-Host at all (and that part is fine).
I do not forsee fixing this for 6.1. I am aligning this to 6.2.z only.
any reason you flagged this to 6.3 and not to 6.2 then?
It is linked to both 6.3 and 6.2.z so that if we fix it in one, we fix it in both.
This did not make it in time for 6.3, moving it to backlog.
Thank you for your interest in Satellite 6. We have evaluated this request, and we do not expect this to be implemented in the product in the foreseeable future. We are therefore closing this out as WONTFIX. If you have any concerns about this, please feel free to contact Rich Jerrido or Bryan Kearney. Thank you.