Hide Forgot
Description of problem: When sending a virt-who report to Satellite a short time after a previous one (from even different set of hypervisors), Satellite's task Hypervisors fail on undefined method `[]' for nil:NilClass error. I reproduced it on 6.3.4 snap 2, but older/other versions of 6.3 are expected to have the bug as well. Version-Release number of selected component (if applicable): Sat 6.3.4 (should be reproducible on older versions as well) - satellite-6.3.4-2.el7sat.noarch - tfm-rubygem-katello-3.4.5.86-1.el7sat.noarch - candlepin-2.1.24-1.el7.noarch How reproducible: 100% in several attempts Steps to Reproduce: 1. Send concurrently (within 10 seconds or so) several virt-who reports - ideally, generate the json file via virt-who -op for each hypervisor (to have multiple reports), and then run: for file in $(ls virt-who.report.*.json); do curl -ks -u admin:Password -X POST -H "Content-Type: text/plain" -d @${file} 'https://satellite.example.com/rhsm/hypervisors/ORGANIZATION' & done 2. Check status of Hypervisors tasks spawned Actual results: Some of the tasks fail with: 2018-10-08 14:34:49 [foreman-tasks/action] [E] undefined method `[]' for nil:NilClass (NoMethodError) | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5.86/app/lib/actions/katello/host/hypervisors.rb:23:in `block in parse_hypervisors' | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5.86/app/lib/actions/katello/host/hypervisors.rb:22:in `each' | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5.86/app/lib/actions/katello/host/hypervisors.rb:22:in `parse_hypervisors' | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5.86/app/lib/actions/candlepin/async_hypervisors.rb:14:in `poll_external_task' | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5.86/app/lib/actions/candlepin/async_hypervisors.rb:20:in `invoke_external_task' | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/action/polling.rb:83:in `initiate_external_action' | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/action/polling.rb:18:in `run' | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5.86/app/lib/actions/candlepin/abstract_async_task.rb:9:in `run' .. Expected results: No such nullpointer exception Additional info: The reason is, Candlepin responded with WAITING state of a task (not sure if it is a new task or an announcement candlepin is busy), what katello cant react on properly. That is evident when I put to /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5.86/app/lib/actions/candlepin/async_hypervisors.rb : def poll_external_task(task_id = external_task[:id]) task = ::Katello::Resources::Candlepin::Job.get(task_id, :result_data => true) unless ::Katello::Resources::Candlepin::Job.not_finished?(task) Rails.logger.info("PavelM: finished task=#{task.inspect}") ################### add this debug line output[:hypervisors] = ::Actions::Katello::Host::Hypervisors.parse_hypervisors(task.delete('resultData')) end task end that returned: 2018-10-08 15:39:26 4d9bdaeb [app] [I] PavelM: finished task={"id"=>"hypervisor_update_0376d314-c552-4920-b7f3-af2f1679bb8b", "state"=>"WAITING", "startTime"=>nil, "finishTime"=>nil, "result"=>nil, "principalName"=>"foreman_admin", "targetType"=>"owner", "targetId"=>"ORGANIZATION", "ownerId"=>"ORGANIZATION", "correlationId"=>"4d9bdaeb", "resultData"=>nil, "statusPath"=>"/jobs/hypervisor_update_0376d314-c552-4920-b7f3-af2f1679bb8b", "done"=>false, "group"=>"async group", "created"=>"2018-10-08T13:39:25+0000", "updated"=>"2018-10-08T13:39:25+0000"} instead of typical: 2018-10-08 15:39:28 c9174a73 [app] [I] PavelM: finished task={"id"=>"hypervisor_update_b5bc3425-39ab-4792-98e2-37fecf2918c1", "state"=>"FINISHED", "startTime"=>"2018-10-08T13:39:22+0000", "finishTime"=>"2018-10-08T13:39:27+0000", "result"=>"Created: 0, Updated: 47, Unchanged:123, ..
Brad, this looks like a Katello issue. Katello should handle a status of "WAITING " similar to "PENDING". Thoughts?
My change was not intended to remove the needinfo. Adding it back.
Connecting redmine issue https://projects.theforeman.org/issues/25643 from this bug
Moving this bug to POST for triage into Satellite 6 since the upstream issue https://projects.theforeman.org/issues/25643 has been resolved.
Verified fix in sat 6.5 snap 11. I ran : curl -k -X POST --user $SATUSER:$SATPASS https://$SATHOST/rhsm/hypervisors/Default_Organization -H "Content-Type: application/json" -d @"/root/foo.json" Multiple times in quick succession. foo.json contained a fairly large number of reports. All of the Hypervisors tasks were successful
*** Bug 1667545 has been marked as a duplicate of this bug. ***
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2019:1222