1514009 – Syncing a atomic host trees repo using "all history" and "custom depth" sync policy raises [E] OST0006: Pulling local refs failed. Reason: GLib.Error

Bug 1514009 - Syncing a atomic host trees repo using "all history" and "custom depth" sync policy raises [E] OST0006: Pulling local refs failed. Reason: GLib.Error

Summary: Syncing a atomic host trees repo using "all history" and "custom depth" sync ...

Keywords:
Status:	CLOSED DUPLICATE of bug 1369892
Alias:	None
Product:	Red Hat Satellite
Classification:	Red Hat
Component:	Pulp
Sub Component:
Version:	6.3.0
Hardware:	Unspecified
OS:	Unspecified
Priority:	unspecified
Severity:	high
Target Milestone:	Unspecified
Assignee:	satellite6-bugs
QA Contact:	Katello QA List
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+	depends on / blocked

Reported:	2017-11-16 13:34 UTC by Sachin Ghai
Modified:	2021-12-10 15:24 UTC (History)
CC List:	7 users (show)
Fixed In Version:
Doc Type:	If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:	2018-01-03 17:12:11 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Description Sachin Ghai 2017-11-16 13:34:28 UTC

Description of problem:

On syncing a RHEL Atomic host Trees repo using following upstream policies raises error: 
 [foreman-tasks/action] [E] OST0006: Pulling local refs failed. Reason: GLib.Error('Importing 2eceacd73c4de494ff3f90448467934614cf9f72cb316693db2cad47bd79015e.commit: linkat: No such file or directory', 'g-io-error-quark', 1) (Katello::Errors::PulpError)

Select sync policy: All history, custom depth =-1



Version-Release number of selected component (if applicable):
sat 6.3 beta snap24

How reproducible:


Steps to Reproduce:
1. sync Atomic host trees repo w/ All history, custom depth =-1 sync policies
2.
3.

Actual results:
sync failed

Expected results:
Sync should succeed and I think all branches should be pulled instead of latest


Additional info:


Sync w/ default policy 'Latest only" works fine.


traceback w/ above policies:
===================================
2017-11-16 07:59:44 e768ffcb [app] [D] Body: [{"search_params":{"type":"task","task_id":"0138af89-e2c6-4c33-bf60-7e305cc2fb22","search_id":"2"},"results":[{"id":"0138af89-e2c6-4c33-bf60-7e305cc2fb22","label":"Actions::Katello::Repository::Sync","pending":true,"username":"admin","started_at":"2017-11-16T12:59:41.000Z","ended_at":null,"state":"running","result":"pending","progress":0.5263157894736842,"input":{"repository":{"id":35,"name":"Red Hat Enterprise Linux Atomic Host Trees","label":"Red_Hat_Enterprise_Linux_Atomic_Host_Trees"},"product":{"id":20,"name":"Red Hat Enterprise Linux Atomic Host","label":"Red_Hat_Enterprise_Linux_Atomic_Host","cp_id":"271"},"provider":{"id":2,"name":"Red Hat"},"organization":{"id":1,"name":"Default Organization","label":"Default_Organization"},"id":35,"sync_result":{"class":"Dynflow::ExecutionPlan::OutputReference","execution_plan_id":"34d80544-4dd4-4f51-b51a-6b95a8bc1211","step_id":3,"action_id":2,"subkeys":[]},"skip_metadata_check":false,"validate_contents":false,"contents_changed":{"class":"Dynflow::ExecutionPlan::OutputReference","execution_plan_id":"34d80544-4dd4-4f51-b51a-6b95a8bc1211","step_id":3,"action_id":2,"subkeys":["contents_changed"]},"current_user_id":3},"output":{},"humanized":{"action":"Synchronize","input":[["repository",{"text":"repository 'Red Hat Enterprise Linux Atomic Host Trees'","link":null}],["product",{"text":"product 'Red Hat Enterprise Linux Atomic Host'","link":"#/products/20/info"}],["organization",{"text":"organization 'Default Organization'","link":"/organizations/1/edit"}]],"output":"Branches updated","errors":[]},"cli_example":null}]}]
2017-11-16 07:59:44 e768ffcb [app] [I] Completed 200 OK in 45ms (Views: 1.4ms | ActiveRecord: 2.2ms)
2017-11-16 07:59:45  [foreman-tasks/action] [E] OST0006: Pulling local refs failed. Reason: GLib.Error('Importing 2eceacd73c4de494ff3f90448467934614cf9f72cb316693db2cad47bd79015e.commit: linkat: No such file or directory', 'g-io-error-quark', 1) (Katello::Errors::PulpError)
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5.22/app/lib/actions/pulp/abstract_async_task.rb:121:in `block in external_task='
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5.22/app/lib/actions/pulp/abstract_async_task.rb:119:in `each'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5.22/app/lib/actions/pulp/abstract_async_task.rb:119:in `external_task='
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5.22/app/lib/actions/pulp/repository/sync.rb:48:in `external_task='
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.30/lib/dynflow/action/polling.rb:98:in `poll_external_task_with_rescue'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.30/lib/dynflow/action/polling.rb:21:in `run'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.30/lib/dynflow/action/cancellable.rb:13:in `run'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5.22/app/lib/actions/pulp/abstract_async_task.rb:45:in `run'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.30/lib/dynflow/action.rb:513:in `block (3 levels) in execute_run'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.30/lib/dynflow/middleware/stack.rb:26:in `pass'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.30/lib/dynflow/middleware.rb:17:in `pass'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.30/lib/dynflow/middleware.rb:30:in `run'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.30/lib/dynflow/middleware/stack.rb:22:in `call'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.30/lib/dynflow/middleware/stack.rb:26:in `pass'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.30/lib/dynflow/middleware.rb:17:in `pass'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5.22/app/lib/actions/middleware/remote_action.rb:16:in `block in run'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5.22/app/lib/actions/middleware/remote_action.rb:40:in `block in as_remote_user'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5.22/app/models/katello/concerns/user_extensions.rb:21:in `cp_config'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5.22/app/lib/actions/middleware/remote_action.rb:27:in `as_cp_user'

Comment 1 Sachin Ghai 2017-11-16 13:35:25 UTC

/var/log/messages:
=====================

Nov 16 08:32:33 SAT_SERVER pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[a4d6932f-8ded-42df-9c1f-2d62c619526e] succeeded in 0.0393983109389s: None
Nov 16 08:32:33 SAT_SERVER pulp: py.warnings:WARNING: [f2b3df4f] (17919-92800) /usr/lib/python2.7/site-packages/pulp_ostree/plugins/lib.py:49: PyGIWarning: OSTree was imported without specifying a version first. Use gi.require_version('OSTree', '1.0') before import to ensure that the right version gets loaded.
Nov 16 08:32:33 SAT_SERVER pulp: py.warnings:WARNING: [f2b3df4f] (17919-92800)   lib = getattr(__import__('gi.repository', fromlist=[name]), name)
Nov 16 08:32:33 SAT_SERVER pulp: py.warnings:WARNING: [f2b3df4f] (17919-92800)
Nov 16 08:32:33 SAT_SERVER pulp: py.warnings:WARNING: [f2b3df4f] (17919-92800) /usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py:1271: DeprecationWarning: save is deprecated. Use insert_one or replace_one instead
Nov 16 08:32:33 SAT_SERVER pulp: py.warnings:WARNING: [f2b3df4f] (17919-92800)   publish_result_coll.save(result)
Nov 16 08:32:33 SAT_SERVER pulp: py.warnings:WARNING: [f2b3df4f] (17919-92800)
Nov 16 08:32:33 SAT_SERVER pulp: pulp.server.controllers.repository:ERROR: [f2b3df4f] (17919-92800) Exception caught from plugin during publish for repo [3ec7560c-159b-47e6-9cff-c4034220ab34]
Nov 16 08:32:33 SAT_SERVER pulp: pulp.server.controllers.repository:ERROR: [f2b3df4f] (17919-92800) Traceback (most recent call last):
Nov 16 08:32:33 SAT_SERVER pulp: pulp.server.controllers.repository:ERROR: [f2b3df4f] (17919-92800)   File "/usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py", line 1252, in _do_publish
Nov 16 08:32:33 SAT_SERVER pulp: pulp.server.controllers.repository:ERROR: [f2b3df4f] (17919-92800)     publish_report = publish_repo(transfer_repo, conduit, call_config)
Nov 16 08:32:33 SAT_SERVER pulp: pulp.server.controllers.repository:ERROR: [f2b3df4f] (17919-92800)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 730, in wrap_f
Nov 16 08:32:33 SAT_SERVER pulp: pulp.server.controllers.repository:ERROR: [f2b3df4f] (17919-92800)     return f(*args, **kwargs)
Nov 16 08:32:33 SAT_SERVER pulp: pulp.server.controllers.repository:ERROR: [f2b3df4f] (17919-92800)   File "/usr/lib/python2.7/site-packages/pulp_ostree/plugins/distributors/web.py", line 87, in publish_repo
Nov 16 08:32:33 SAT_SERVER pulp: pulp.server.controllers.repository:ERROR: [f2b3df4f] (17919-92800)     return self._publisher.process_lifecycle()
Nov 16 08:32:33 SAT_SERVER pulp: pulp.server.controllers.repository:ERROR: [f2b3df4f] (17919-92800)   File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 572, in process_lifecycle
Nov 16 08:32:33 SAT_SERVER pulp: pulp.server.controllers.repository:ERROR: [f2b3df4f] (17919-92800)     super(PluginStep, self).process_lifecycle()
Nov 16 08:32:33 SAT_SERVER pulp: pulp.server.controllers.repository:ERROR: [f2b3df4f] (17919-92800)   File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 163, in process_lifecycle
Nov 16 08:32:33 SAT_SERVER pulp: pulp.server.controllers.repository:ERROR: [f2b3df4f] (17919-92800)     step.process()
Nov 16 08:32:33 SAT_SERVER pulp: pulp.server.controllers.repository:ERROR: [f2b3df4f] (17919-92800)   File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 256, in process
Nov 16 08:32:33 SAT_SERVER pulp: pulp.server.controllers.repository:ERROR: [f2b3df4f] (17919-92800)     self._process_block()
Nov 16 08:32:33 SAT_SERVER pulp: pulp.server.controllers.repository:ERROR: [f2b3df4f] (17919-92800)   File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 303, in _process_block
Nov 16 08:32:33 SAT_SERVER pulp: pulp.server.controllers.repository:ERROR: [f2b3df4f] (17919-92800)     self.process_main()
Nov 16 08:32:33 SAT_SERVER pulp: pulp.server.controllers.repository:ERROR: [f2b3df4f] (17919-92800)   File "/usr/lib/python2.7/site-packages/pulp_ostree/plugins/distributors/steps.py", line 92, in process_main
Nov 16 08:32:33 SAT_SERVER pulp: pulp.server.controllers.repository:ERROR: [f2b3df4f] (17919-92800)     raise pe
Nov 16 08:32:33 SAT_SERVER pulp: pulp.server.controllers.repository:ERROR: [f2b3df4f] (17919-92800) PulpCodedException: Pulling local refs failed. Reason: GLib.Error('Importing 2eceacd73c4de494ff3f90448467934614cf9f72cb316693db2cad47bd79015e.commit: linkat: No such file or directory', 'g-io-error-quark', 1)
Nov 16 08:32:33 SAT_SERVER pulp: pulp.server.async.tasks:INFO: [f2b3df4f] Task failed : [f2b3df4f-f16f-4956-b6b1-e8304c84cda7] : Pulling local refs failed. Reason: GLib.Error('Importing 2eceacd73c4de494ff3f90448467934614cf9f72cb316693db2cad47bd79015e.commit: linkat: No such file or directory', 'g-io-error-quark', 1)

Comment 3 Sachin Ghai 2017-12-11 07:28:39 UTC

on setting  custom depth =1
==============================
2017-12-11 02:27:14  [foreman-tasks/action] [E] OST0006: Pulling local refs failed. Reason: GLib.Error('No such file or directory', 'g-io-error-quark', 1) (Katello::Errors::PulpError)
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5.32/app/lib/actions/pulp/abstract_async_task.rb:121:in `block in external_task='

on setting  custom depth =-2
==============================
[foreman-tasks/action] [E] PLP0000: GLib.Error('No such metadata object 13fe9e86d640fd257afe831e4b33ad1eb6183d7de2a550dc7397a7b4b1f6ef25.commit', 'g-io-error-quark', 1) (Katello::Errors::PulpError)
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5.32/app/lib/actions/pulp/abstract_async_task.rb:121:in `block in external_task='

Comment 4 Partha Aji 2017-12-14 22:15:05 UTC

I believe this is a duplicate of https://bugzilla.redhat.com/show_bug.cgi?id=1369892#c35

I verified the issue on another QE box. Basic issue is as follows.
When you update the ostree upstream sync policy to something like set a customer depth to 100 in Satellite 2 calls are made to pulp  

1) to update the distributor. Something like

resource - https://<sat fqdn>, action_name, - /pulp/api/v2/repositories/3b5f50a8-04aa-4622-ba2a-34c57846d17a/distributors/3b5f50a8-04aa-4622-ba2a-34c57846d17a//, params - [:put, "{\"distributor_config\":{\"relative_path\":\"Default_Organization/Library/content/dist/rhel/atomic/7/7Server/x86_64/ostree/repo\",\"depth\":100}}", {"content_type"=>"application/json", "accept"=>"application/json"}]


2) Other to update the importer
resource - https://<sat fqdn>, action_name, - /pulp/api/v2/repositories/3b5f50a8-04aa-4622-ba2a-34c57846d17a/importers/ostree_web_importer//, params - [:put, "{\"importer_config\":{\"ssl_validation\":true,\"basic_auth_username\":null,\"basic_auth_password\":null,\"depth\":100,\"feed\":\"https://cdn.redhat.com/content/dist/rhel/atomic/7/7Server/x86_64/ostree/repo\"}}", {"content_type"=>"application/json", "accept"=>"application/json"}]


Assigning it to pulp

notice the depth value is 100 for those. But for some reason accepts 1 and rejects 2 the importer call causing this issue. You can verify this either via pulp admin or running the following in "foreman-rake console"

::Katello.pulp_server.extensions.repository.retrieve("3b5f50a8-04aa-4622-ba2a-34c57846d17a", :details => true)[:importers].first[:config][:depth]

=> 0

Comment 5 pulp-infra@redhat.com 2017-12-18 15:02:36 UTC

The Pulp upstream bug status is at NEW. Updating the external tracker on this bug.

Comment 6 pulp-infra@redhat.com 2017-12-18 15:02:40 UTC

The Pulp upstream bug priority is at Normal. Updating the external tracker on this bug.

Comment 7 pulp-infra@redhat.com 2017-12-18 17:32:27 UTC

The Pulp upstream bug status is at ASSIGNED. Updating the external tracker on this bug.

Comment 8 pulp-infra@redhat.com 2017-12-19 03:05:26 UTC

The Pulp upstream bug status is at POST. Updating the external tracker on this bug.

Comment 9 Bryan Kearney 2018-01-03 17:12:11 UTC

Marking it as a dupe per https://bugzilla.redhat.com/show_bug.cgi?id=1514009#c4

*** This bug has been marked as a duplicate of bug 1369892 ***

Comment 10 Daniel Alley 2018-01-03 20:10:47 UTC

Removing pulp issue # to keep trackers in sync.  See https://bugzilla.redhat.com/show_bug.cgi?id=1369892

Note You need to log in before you can comment on or make changes to this bug.