Bug 1986996 - sync of file:// repos tries to move files from /var/lib/pulp/sync_imports/
Summary: sync of file:// repos tries to move files from /var/lib/pulp/sync_imports/
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Pulp
Version: 6.10.0
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: 6.10.0
Assignee: satellite6-bugs
QA Contact: Stephen Wadeley
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-07-28 16:40 UTC by Tanya Tereshchenko
Modified: 2021-11-16 14:13 UTC (History)
6 users (show)

Fixed In Version: pulpcore-3.14.4
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-11-16 14:12:53 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Pulp Redmine 9146 0 Urgent CLOSED - CURRENTRELEASE sync of file:// repos tries to move files from /var/lib/pulp/sync_imports/ 2021-08-26 13:15:40 UTC
Pulp Redmine 9202 0 Normal CLOSED - CURRENTRELEASE Backport #9146 "sync of file:// repos tries to move files from /var/lib/pulp/sync_imports/" to 3.14.z 2021-10-11 21:07:38 UTC
Red Hat Product Errata RHSA-2021:4702 0 None None None 2021-11-16 14:12:59 UTC

Description Tanya Tereshchenko 2021-07-28 16:40:47 UTC
Description of problem:
When syncing file:// repos it appears that pulp is trying to move files from that directory. In our case we copy files that are world readable, but not writable by the pulp user and see a permission error:

Jul 26 18:27:11 devel2 pulpcore-worker-1: pulp [994770242e2842ecade44f158e7de8d0]: pulpcore.tasking.pulpcore_worker:INFO: Task 4c40418c-8a76-4a83-a6ee-4d2b5326c9d4 failed ([Errno 13] Pe
rmission denied: '/var/lib/pulp/sync_imports/test_repos/zoo/kangaroo-0.2-1.noarch.rpm')
Jul 26 18:27:11 devel2 pulpcore-worker-1: pulp [994770242e2842ecade44f158e7de8d0]: pulpcore.tasking.pulpcore_worker:INFO:   File "/usr/lib/python3.6/site-packages/pulpcore/tasking/pulpc
ore_worker.py", line 272, in _perform_task
Jul 26 18:27:11 devel2 pulpcore-worker-1: result = func(*args, **kwargs)
Jul 26 18:27:11 devel2 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 427, in synchronize
Jul 26 18:27:11 devel2 pulpcore-worker-1: version = dv.create()
Jul 26 18:27:11 devel2 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py", line 151, in create
Jul 26 18:27:11 devel2 pulpcore-worker-1: loop.run_until_complete(pipeline)
Jul 26 18:27:11 devel2 pulpcore-worker-1: File "/usr/lib64/python3.6/asyncio/base_events.py", line 484, in run_until_complete
Jul 26 18:27:11 devel2 pulpcore-worker-1: return future.result()
Jul 26 18:27:11 devel2 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 225, in create_pipeline
Jul 26 18:27:11 devel2 pulpcore-worker-1: await asyncio.gather(*futures)
Jul 26 18:27:11 devel2 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 43, in __call__
Jul 26 18:27:11 devel2 pulpcore-worker-1: await self.run()
Jul 26 18:27:11 devel2 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/artifact_stages.py", line 248, in run
Jul 26 18:27:11 devel2 pulpcore-worker-1: d_artifact.artifact for d_artifact in da_to_save
Jul 26 18:27:11 devel2 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulpcore/app/models/content.py", line 82, in bulk_get_or_create
Jul 26 18:27:11 devel2 pulpcore-worker-1: return super().bulk_create(objs, batch_size=batch_size)
Jul 26 18:27:11 devel2 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/django/db/models/manager.py", line 82, in manager_method
Jul 26 18:27:11 devel2 pulpcore-worker-1: return getattr(self.get_queryset(), name)(*args, **kwargs)
Jul 26 18:27:11 devel2 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/django/db/models/query.py", line 468, in bulk_create
Jul 26 18:27:11 devel2 pulpcore-worker-1: self._batched_insert(objs_with_pk, fields, batch_size, ignore_conflicts=ignore_conflicts)
Jul 26 18:27:11 devel2 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/django/db/models/query.py", line 1204, in _batched_insert
Jul 26 18:27:11 devel2 pulpcore-worker-1: ignore_conflicts=ignore_conflicts,
Jul 26 18:27:11 devel2 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/django/db/models/query.py", line 1186, in _insert
Jul 26 18:27:11 devel2 pulpcore-worker-1: return query.get_compiler(using=using).execute_sql(return_id)
Jul 26 18:27:11 devel2 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1376, in execute_sql
Jul 26 18:27:11 devel2 pulpcore-worker-1: for sql, params in self.as_sql():
Jul 26 18:27:11 devel2 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/django_readonly_field/compiler.py", line 31, in as_sql
Jul 26 18:27:11 devel2 pulpcore-worker-1: return super(ReadonlySQLCompilerMixin, self).as_sql()
Jul 26 18:27:11 devel2 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1320, in as_sql
Jul 26 18:27:11 devel2 pulpcore-worker-1: for obj in self.query.objs
Jul 26 18:27:11 devel2 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1320, in <listcomp>
Jul 26 18:27:11 devel2 pulpcore-worker-1: for obj in self.query.objs
Jul 26 18:27:11 devel2 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1319, in <listcomp>
Jul 26 18:27:11 devel2 pulpcore-worker-1: [self.prepare_value(field, self.pre_save_val(field, obj)) for field in fields]
Jul 26 18:27:11 devel2 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1270, in pre_save_val
Jul 26 18:27:11 devel2 pulpcore-worker-1: return field.pre_save(obj, add=True)
Jul 26 18:27:11 devel2 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulpcore/app/models/fields.py", line 68, in pre_save
Jul 26 18:27:11 devel2 pulpcore-worker-1: return super().pre_save(model_instance, add)
Jul 26 18:27:11 devel2 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/django/db/models/fields/files.py", line 289, in pre_save
Jul 26 18:27:11 devel2 pulpcore-worker-1: file.save(file.name, file.file, save=False)
Jul 26 18:27:11 devel2 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/django/db/models/fields/files.py", line 88, in save
Jul 26 18:27:11 devel2 pulpcore-worker-1: self.name = self.storage.save(name, content, max_length=self.field.max_length)
Jul 26 18:27:11 devel2 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/django/core/files/storage.py", line 54, in save
Jul 26 18:27:11 devel2 pulpcore-worker-1: return self._save(name, content)
Jul 26 18:27:11 devel2 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulpcore/app/models/storage.py", line 68, in _save
Jul 26 18:27:11 devel2 pulpcore-worker-1: file_move_safe(content.temporary_file_path(), full_path)
Jul 26 18:27:11 devel2 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/django/core/files/move.py", line 80, in file_move_safe
Jul 26 18:27:11 devel2 pulpcore-worker-1: os.remove(old_file_name)

These repos are in /var/lib/pulp/sync_imports/ (and settings.py is configured properly).

Version-Release number of selected component (if applicable):
pulpcore 3.14.3

Comment 1 pulp-infra@redhat.com 2021-07-28 17:20:24 UTC
The Pulp upstream bug status is at NEW. Updating the external tracker on this bug.

Comment 2 pulp-infra@redhat.com 2021-07-28 17:20:25 UTC
The Pulp upstream bug priority is at Urgent. Updating the external tracker on this bug.

Comment 3 pulp-infra@redhat.com 2021-08-02 19:06:18 UTC
The Pulp upstream bug status is at ASSIGNED. Updating the external tracker on this bug.

Comment 4 pulp-infra@redhat.com 2021-08-03 20:09:25 UTC
The Pulp upstream bug status is at POST. Updating the external tracker on this bug.

Comment 5 pulp-infra@redhat.com 2021-08-04 00:23:44 UTC
The Pulp upstream bug status is at MODIFIED. Updating the external tracker on this bug.

Comment 6 pulp-infra@redhat.com 2021-08-04 01:52:59 UTC
All upstream Pulp bugs are at MODIFIED+. Moving this bug to POST.

Comment 10 pulp-infra@redhat.com 2021-08-26 13:15:41 UTC
The Pulp upstream bug status is at CLOSED - CURRENTRELEASE. Updating the external tracker on this bug.

Comment 11 pulp-infra@redhat.com 2021-10-11 21:07:39 UTC
The Pulp upstream bug status is at CLOSED - CURRENTRELEASE. Updating the external tracker on this bug.

Comment 12 pulp-infra@redhat.com 2021-10-11 21:07:40 UTC
The Pulp upstream bug priority is at Normal. Updating the external tracker on this bug.

Comment 15 errata-xmlrpc 2021-11-16 14:12:53 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: Satellite 6.10 Release), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:4702


Note You need to log in before you can comment on or make changes to this bug.