Bug 1689504

Summary: Syncing Red Hat Enterprise Linux Atomic Host Trees x86_64 from CDN gives undefined method `to_datetime' for nil:NilClass
Product: Red Hat Satellite Reporter: Nikhil Kathole <nkathole>
Component: AtomicAssignee: Samir Jha <sajha>
Status: CLOSED ERRATA QA Contact: Nikhil Kathole <nkathole>
Severity: high Docs Contact:
Priority: unspecified    
Version: 6.5.0CC: egolov, jcallaha, zhunting
Target Milestone: ReleasedKeywords: Regression, Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: tfm-rubygem-katello-3.10.0.37-1 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-05-14 19:57:53 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1514013, 1645017    

Description Nikhil Kathole 2019-03-16 09:09:18 UTC
Description of problem:

Syncing Red Hat Enterprise Linux Atomic Host Trees x86_64 from CDN gives undefined method `to_datetime' for nil:NilClass

Exception:

NoMethodError: undefined method `to_datetime' for nil:NilClass

Backtrace:

/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.29/app/models/katello/ostree_branch.rb:24:in `update_from_json'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.29/app/models/katello/concerns/pulp_database_unit.rb:61:in `block (2 levels) in import_all'
/opt/theforeman/tfm/root/usr/share/gems/gems/runcible-2.10.1/lib/runcible/response.rb:32:in `each'
/opt/theforeman/tfm/root/usr/share/gems/gems/runcible-2.10.1/lib/runcible/response.rb:32:in `method_missing'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.29/app/models/katello/concerns/pulp_database_unit.rb:56:in `block in import_all'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.29/app/services/katello/pulp/pulp_content_unit.rb:51:in `block in fetch_by_uuids'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.29/app/services/katello/pulp/pulp_content_unit.rb:48:in `each'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.29/app/services/katello/pulp/pulp_content_unit.rb:48:in `each_slice'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.29/app/services/katello/pulp/pulp_content_unit.rb:48:in `fetch_by_uuids'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.29/app/models/katello/concerns/pulp_database_unit.rb:71:in `import_all'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.29/app/models/katello/concerns/pulp_database_unit.rb:89:in `import_for_repository'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.29/app/models/katello/glue/pulp/repo.rb:456:in `index_content'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.29/app/lib/actions/katello/repository/index_content.rb:16:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/action.rb:539:in `block (3 levels) in execute_run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware.rb:18:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.29/app/lib/actions/middleware/execute_if_contents_changed.rb:5:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware.rb:18:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.14.4.4/app/lib/actions/middleware/rails_executor_wrap.rb:14:in `block in run'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/execution_wrapper.rb:87:in `wrap'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.14.4.4/app/lib/actions/middleware/rails_executor_wrap.rb:13:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware.rb:18:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/action/progress.rb:30:in `with_progress_calculation'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/action/progress.rb:16:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware.rb:18:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.29/app/lib/actions/middleware/keep_locale.rb:11:in `block in run'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.29/app/lib/actions/middleware/keep_locale.rb:22:in `with_locale'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.29/app/lib/actions/middleware/keep_locale.rb:11:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware.rb:18:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.14.4.4/app/lib/actions/middleware/keep_current_timezone.rb:15:in `block in run'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.14.4.4/app/lib/actions/middleware/keep_current_timezone.rb:44:in `restore_curent_timezone'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.14.4.4/app/lib/actions/middleware/keep_current_timezone.rb:15:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware.rb:18:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.14.4.4/app/lib/actions/middleware/keep_current_user.rb:15:in `block in run'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.14.4.4/app/lib/actions/middleware/keep_current_user.rb:43:in `restore_curent_user'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.14.4.4/app/lib/actions/middleware/keep_current_user.rb:15:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware.rb:18:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.14.4.4/app/lib/actions/middleware/keep_current_taxonomies.rb:15:in `block in run'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.14.4.4/app/lib/actions/middleware/keep_current_taxonomies.rb:44:in `restore_current_taxonomies'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.14.4.4/app/lib/actions/middleware/keep_current_taxonomies.rb:15:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware.rb:18:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware.rb:31:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware/world.rb:30:in `execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/action.rb:538:in `block (2 levels) in execute_run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/action.rb:537:in `catch'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/action.rb:537:in `block in execute_run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/action.rb:451:in `block in with_error_handling'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/action.rb:451:in `catch'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/action.rb:451:in `with_error_handling'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/action.rb:532:in `execute_run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/action.rb:278:in `execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:17:in `block (2 levels) in execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/execution_plan/steps/abstract.rb:162:in `with_meta_calculation'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:16:in `block in execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:30:in `open_action'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:15:in `execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/director.rb:43:in `execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/executors/parallel/worker.rb:14:in `block in on_message'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/executors.rb:12:in `run_user_code'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/executors/parallel/worker.rb:13:in `on_message'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/context.rb:46:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/executes_context.rb:7:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/actor.rb:26:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/awaits.rb:15:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/sets_results.rb:14:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/buffer.rb:38:in `process_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/buffer.rb:31:in `process_envelopes?'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/buffer.rb:20:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/termination.rb:55:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/removes_child.rb:10:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/sets_results.rb:14:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/core.rb:161:in `process_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/core.rb:95:in `block in on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/core.rb:118:in `block (2 levels) in schedule_execution'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/synchronization/mri_lockable_object.rb:38:in `block in synchronize'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/synchronization/mri_lockable_object.rb:38:in `synchronize'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/synchronization/mri_lockable_object.rb:38:in `synchronize'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/core.rb:115:in `block in schedule_execution'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/serialized_execution.rb:18:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/serialized_execution.rb:96:in `work'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/serialized_execution.rb:77:in `block in call_job'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:348:in `run_task'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:337:in `block (3 levels) in create_worker'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:320:in `loop'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:320:in `block (2 levels) in create_worker'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:319:in `catch'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:319:in `block in create_worker'
/opt/theforeman/tfm/root/usr/share/gems/gems/logging-2.2.2/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'


Version-Release number of selected component (if applicable):
Satellite 6.5 snap 20

# rpm -qa | grep ostree
python-pulp-ostree-common-1.4.0-2.el7sat.noarch
ostree-2017.1-2.atomic.el7.x86_64
pulp-ostree-plugins-1.4.0-2.el7sat.noarch


How reproducible:
always


Steps to Reproduce:
1. Enable Red Hat Enterprise Linux Atomic Host Trees x86_64 
rhel-atomic-host-ostree
2. try to sync

Actual results:

Sync failed.


Expected results:
Successful sync


Additional info:
This was working was snap 19.

Comment 4 Nikhil Kathole 2019-03-19 05:43:23 UTC
*** Bug 1690112 has been marked as a duplicate of this bug. ***

Comment 5 Samir Jha 2019-03-25 21:38:25 UTC
We had a change in pulp where the created date from ostree branches was dropped here -> https://github.com/pulp/pulp_ostree/pull/104/files#diff-b39f3b7e68d52d872bf0606a7113674b.
Katello still references this field causing the method call on nil object. The plan to fix this is to drop the version_date field on our ostree model.

Comment 6 Samir Jha 2019-03-25 21:41:12 UTC
Created redmine issue http://projects.theforeman.org/issues/26447 from this bug

Comment 7 Bryan Kearney 2019-03-26 22:00:54 UTC
Moving this bug to POST for triage into Satellite 6 since the upstream issue https://projects.theforeman.org/issues/26447 has been resolved.

Comment 9 Bryan Kearney 2019-03-27 02:00:54 UTC
Moving this bug to POST for triage into Satellite 6 since the upstream issue https://projects.theforeman.org/issues/26447 has been resolved.

Comment 11 Bryan Kearney 2019-03-28 00:00:57 UTC
Moving this bug to POST for triage into Satellite 6 since the upstream issue https://projects.theforeman.org/issues/26447 has been resolved.

Comment 13 Evgeni Golov 2019-04-04 18:40:49 UTC
*** Bug 1696350 has been marked as a duplicate of this bug. ***

Comment 14 Nikhil Kathole 2019-04-10 13:40:07 UTC
VERIFIED

Version tested:
Satellite 6.5 snap 23

Successful sync of Red Hat Enterprise Linux Atomic Host Trees x86_64 from CDN on fresh install but facing issue on upgraded box PLP0000: Too many branch HEADs.

Open issue for upgraded instance: https://bugzilla.redhat.com/show_bug.cgi?id=1696988

Comment 15 Bryan Kearney 2019-05-14 19:57:53 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2019:1222