Running a force_full sync will not fix missing symlinks in /var/lib/pulp/published/. What you instead need to run is:

    pulp-admin -vvvv rpm repo publish run --repo e3e783b1-7729-4248-92d9-1e399ae60417 --force-full

I don't think Satellite exposes this force_full publish functionality. Maybe it could or should though?

> The Override Config should have  Force Full: True.

I think this might be a bug. I will investigate further and open an upstream bug to fix this in Pulp.

I opened an upstream issue for the last override config not properly showing force_full.

https://pulp.plan.io/issues/4076

I am not linking them though as the issue is unrelated to this one.

I'm tempted to set the component on this issue to 'Katello' since Pulp does provide the ability to fix published symlinks.

>pulp-admin -vvvv rpm repo publish run --repo >e3e783b1-7729-4248-92d9-1e399ae60417 --force-full

>I don't think Satellite exposes this force_full publish functionality. Maybe it >could or should though?

Any drawback of not doing it? IMO, satellite should because we are asking for it to perform a forceful sync.

>Any drawback of not doing it?

I meant any drawback of doing it.

>Running a force_full sync will not fix missing symlinks in /var/lib>>/pulp/published/. What you instead need to run is:

    pulp-admin -vvvv rpm repo publish run --repo >e3e783b1-7729-4248-92d9-1e399ae60417 --force-full

David, I don't have any luck with this, see below 

pulp-admin -u admin -p $pulpAdminPassword -vvvv rpm repo publish run --help
Command: run
Description: triggers an immediate publish of a repository

Available Arguments:

  --bg      - if specified, the client process will end immediately (the task
              will continue to run on the server)
  --repo-id - (required) unique identifier; only alphanumeric, ., -, and _
              allowed

Options
  --force-full - forces to publish from scratch, even if no changes were
                 introduced since last publish

[root@mjia-capsule p]# pulp-admin -u admin -p $pulpAdminPassword -vvvv rpm repo publish run --repo-id e3e783b1-7729-4248-92d9-1e399ae60417 --force-full
+----------------------------------------------------------------------+
      Publishing Repository [e3e783b1-7729-4248-92d9-1e399ae60417]
+----------------------------------------------------------------------+

The following publish configuration options will be used:

Force Full:  True


2018-10-10 19:05:22,987 - DEBUG - sending POST request to /pulp/api/v2/tasks/search/
2018-10-10 19:05:23,101 - INFO - POST request to /pulp/api/v2/tasks/search/ with parameters {"criteria": {"filters": {"state": {"$nin": ["finished", "error", "canceled", "skipped"]}, "tags": {"$all": ["pulp:repository:e3e783b1-7729-4248-92d9-1e399ae60417", "pulp:action:publish"]}}}}
2018-10-10 19:05:23,102 - INFO - Response status : 200 

2018-10-10 19:05:23,102 - INFO - Response body :
 []

2018-10-10 19:05:23,102 - DEBUG - sending POST request to /pulp/api/v2/repositories/e3e783b1-7729-4248-92d9-1e399ae60417/actions/publish/
2018-10-10 19:05:23,217 - INFO - POST request to /pulp/api/v2/repositories/e3e783b1-7729-4248-92d9-1e399ae60417/actions/publish/ with parameters {"override_config": {"force_full": true}, "id": "yum_distributor"}
2018-10-10 19:05:23,217 - INFO - Response status : 202 

2018-10-10 19:05:23,220 - INFO - Response body :
 {
  "spawned_tasks": [
    {
      "_href": "/pulp/api/v2/tasks/75b4af54-36f4-463f-b38d-9f695a15e909/", 
      "task_id": "75b4af54-36f4-463f-b38d-9f695a15e909"
    }
  ], 
  "result": null, 
  "error": null
}

This command may be exited via ctrl+c without affecting the request.


2018-10-10 19:05:24,221 - DEBUG - sending GET request to /pulp/api/v2/tasks/75b4af54-36f4-463f-b38d-9f695a15e909/
2018-10-10 19:05:24,319 - INFO - GET request to /pulp/api/v2/tasks/75b4af54-36f4-463f-b38d-9f695a15e909/ with parameters None
2018-10-10 19:05:24,319 - INFO - Response status : 200 

2018-10-10 19:05:24,320 - INFO - Response body :
 {
  "exception": null, 
  "task_type": "pulp.server.managers.repo.publish.publish", 
  "_href": "/pulp/api/v2/tasks/75b4af54-36f4-463f-b38d-9f695a15e909/", 
  "task_id": "75b4af54-36f4-463f-b38d-9f695a15e909", 
  "tags": [
    "pulp:repository:e3e783b1-7729-4248-92d9-1e399ae60417", 
    "pulp:action:publish"
  ], 
  "finish_time": "2018-10-10T23:05:23Z", 
  "_ns": "task_status", 
  "start_time": "2018-10-10T23:05:23Z", 
  "traceback": "Traceback (most recent call last):\n  File \"/usr/lib/python2.7/site-packages/celery/app/trace.py\", line 240, in trace_task\n    R = retval = fun(*args, **kwargs)\n  File \"/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py\", line 527, in __call__\n    return super(Task, self).__call__(*args, **kwargs)\n  File \"/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py\", line 107, in __call__\n    return super(PulpTask, self).__call__(*args, **kwargs)\n  File \"/usr/lib/python2.7/site-packages/celery/app/trace.py\", line 438, in __protected_call__\n    return self.run(*args, **kwargs)\n  File \"/usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py\", line 1097, in publish\n    dist = model.Distributor.objects.get_or_404(repo_id=repo_id, distributor_id=dist_id)\n  File \"/usr/lib/python2.7/site-packages/pulp/server/db/querysets.py\", line 119, in get_or_404\n    raise pulp_exceptions.MissingResource(**kwargs)\nMissingResource: Missing resource(s): repo_id=e3e783b1-7729-4248-92d9-1e399ae60417, distributor_id=yum_distributor\n", 
  "spawned_tasks": [], 
  "progress_report": {}, 
  "queue": "reserved_resource_worker-2.gsslab.pek.redhat.com.dq", 
  "state": "error", 
  "worker_name": "reserved_resource_worker-2.gsslab.pek.redhat.com", 
  "result": null, 
  "error": {
    "code": "PLP0009", 
    "data": {
      "resources": {
        "repo_id": "e3e783b1-7729-4248-92d9-1e399ae60417", 
        "distributor_id": "yum_distributor"
      }
    }, 
    "description": "Missing resource(s): repo_id=e3e783b1-7729-4248-92d9-1e399ae60417, distributor_id=yum_distributor", 
    "sub_errors": []
  }, 
  "_id": {
    "$oid": "5bbe85b34c079f2d2eaa48ff"
  }, 
  "id": "5bbe85b34c079f2d2eaa48ff"
}


Task Failed

Missing resource(s): repo_id=e3e783b1-7729-4248-92d9-1e399ae60417,
distributor_id=yum_distributor

I forgot that Katello gives the distributor an id other than yum_distributor which won't work with the pulp-admin CLI. You can do it with the API though—although I am not sure it's easier than your method. First you'd have to find out the distributor id with "pulp-admin rpm repo list --details". Then you'd call POST /pulp/api/v2/repositories/e3e783b1-7729-4248-92d9-1e399ae60417/actions/publish/ 
with parameters {"override_config": {"force_full": true}, "id": "my_distributor_id"}.

> Any drawback of not doing it? IMO, satellite should because we are asking for it to perform a forceful sync.

Off the top of my head, I don't see any drawbacks to exposing force_full publishing in Katello or automatically perform a force_full publish after a force_full sync.

matt jia,

Would you be able to provide a tasks export from the satellite?   The way it works is the foreman-task should be kicking off a 2nd publish with force_full set to true.  I'd love to see the task export to confirm if it is doing that or not.

Thanks,

Justin

Created attachment 1494193 [details]
task-export

Pulp provides a way to perform force_full publish which should fix the broken symlinks.

The issue is potentially on the Katello side, or at least investigation should be done there first to see what calls are made to Pulp. Should we move it to a different component?
Maybe it's worth closing this bug completely since the report seems to be 1+ year old.

It looks like this was fixed in Satellite 6.6 as part of https://bugzilla.redhat.com/show_bug.cgi?id=1731060  (although the description is quite different).   Will need to test to confirm.

I've tested this in Satellite 6.6 and can confirm that the symbolic link is recreated after doing a full sync.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:1454