Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1546974

Summary:

Handle Pulp 404 during GenerateApplicability in a clean way

Product:

Red Hat Satellite

Reporter:

Beat Rubischon <brubisch>

Component:

Hosts - Content

Assignee:

Justin Sherrill <jsherril>

Status:

CLOSED ERRATA

QA Contact:

Roman Plevka <rplevka>

Severity:

medium

Docs Contact:

Priority:

medium

Version:

6.2.14

CC:

bbuckingham, bmbouter, cdonnell, daviddavis, dkliban, egolov, ggainey, ipanova, jsherril, pcreech, peter.vreman, rchan, rplevka, ttereshc, wpinheir

Target Milestone:

6.4.0

Keywords:

PrioBumpField, PrioBumpGSS, Triaged

Target Release:

Unused

Hardware:

x86_64

OS:

Linux

Whiteboard:

Fixed In Version:

tfm-rubygem-katello-3.7.0.33-1

Doc Type:

If docs needed, set a value

Doc Text:

Story Points:

---

Clone Of:

Environment:

Last Closed:

2018-10-16 18:55:53 UTC

Type:

Bug

Regression:

---

Mount Type:

---

Documentation:

---

CRM:

Verified Versions:

Category:

---

oVirt Team:

---

RHEL 7.3 requirements from Atomic Host:

Cloudforms Team:

---

Target Upstream Version:

Embargoed:

Bug Depends On:

Bug Blocks:

1122832

Attachments:

Description	Flags
QA_full_traceback	none

Description Beat Rubischon 2018-02-20 09:00:47 UTC

Description of problem:

If an enqueued GenerateApplicability task is executed after a content host is deleted, Pulp returns 404 and the task will end in state paused/error.

Version-Release number of selected component (if applicable):

Satellite 6.2

How reproducible:

rarely

Steps to Reproduce:
1. Register host / install packages / unregister host in a loop on a loaded Satellite

Actual results:

Sporadic failed GenerateApplicability in the queue

Expected results:

No failed tasks in the queue

Additional info:

Comment 3 Satellite Program 2018-02-20 11:19:29 UTC

Upstream bug assigned to cduryee

Comment 5 Satellite Program 2018-02-28 21:19:15 UTC

Moving this bug to POST for triage into Satellite 6 since the upstream issue http://projects.theforeman.org/issues/21797 has been resolved.

Comment 6 Peter Vreman 2018-05-09 09:24:20 UTC

The propsed PRs is only for Katello.
How about the failure in Pulp where the same issue happens.

See below that pulp agent delete is requested at 11:47:05 and finished at 11:47:07. But there was still a generate applicablity task running form 11:46:31 to 11:47:40, that then in the end failed with a nil access because the consumer was deleted.


Sat6 task for GenerateApplicability:
-------------------------------------------------
Started at: 2018-05-02 11:46:30 UTC
Ended at:

3: Actions::Pulp::Consumer::GenerateApplicability (error) [ 69.11s / 0.73s ] Skip

Started at: 2018-05-02 11:46:31 UTC
Ended at: 2018-05-02 11:47:40 UTC
Real time: 69.11s
Execution time (excluding suspended state): 0.73s
Input:

---
uuids:
- ac2d7ee6-344d-4290-bfb1-5e07f7210faf
remote_user: admin
remote_cp_user: admin

Output:
---
pulp_tasks:
- exception: 
  task_type: pulp.server.managers.consumer.applicability.regenerate_applicability_for_consumers
  _href: "/pulp/api/v2/tasks/b7c0e1f5-9371-4070-94cc-c94efe667c6e/"
  task_id: b7c0e1f5-9371-4070-94cc-c94efe667c6e
  tags:
  - pulp:action:consumer_content_applicability_regeneration
  finish_time: '2018-05-02T11:47:39Z'
  _ns: task_status
  start_time: '2018-05-02T11:46:31Z'
  traceback: |
    Traceback (most recent call last):
      File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
        R = retval = fun(*args, **kwargs)
      File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 527, in __call__
        return super(Task, self).__call__(*args, **kwargs)
      File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 107, in __call__
        return super(PulpTask, self).__call__(*args, **kwargs)
      File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 438, in __protected_call__
        return self.run(*args, **kwargs)
      File "/usr/lib/python2.7/site-packages/pulp/server/managers/consumer/applicability.py", line 108, in regenerate_applicability_for_consumers
        manager.regenerate_applicability(profile_hash, content_type, profile_id, repo_id)
      File "/usr/lib/python2.7/site-packages/pulp/server/managers/consumer/applicability.py", line 251, in regenerate_applicability
        profile = unit_profile['profile']
    TypeError: 'NoneType' object has no attribute '__getitem__'
-------------------------------------------------


Sat6 task for Unregister:
-------------------------------------------------
Label: Actions::Katello::Host::Unregister
Status: stopped
Result: success
Started at: 2018-05-02 11:47:05 UTC
Ended at: 2018-05-02 11:47:07 UTC

4: Actions::Candlepin::Consumer::Destroy (success) [ 0.15s / 0.15s ]
6: Actions::Pulp::Consumer::Destroy (success) [ 0.09s / 0.09s ]
Started at: 2018-05-02 11:47:07 UTC
Ended at: 2018-05-02 11:47:07 UTC
Real time: 0.09s
Execution time (excluding suspended state): 0.09s

Input:
---
uuid: ac2d7ee6-344d-4290-bfb1-5e07f7210faf
remote_user: admin
remote_cp_user: admin

Output:
--- {}
-------------------------------------------------




Pulp tasks:
-------------------------------------------------
May  2 11:46:31 li-lc-1578 pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.consumer.applicability.regenerate_applicability_for_consumers[b7c0e1f5-9371-4070-94cc-c94efe667c6e]
May  2 11:47:07 li-lc-1578 pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.consumer.agent.delete_queue[b297b300-9f06-4556-a233-7dac0c374c78] eta:[2018-05-02 11:57:07.181216+00:00]


[crash/LI] root@li-lc-1578:~# sudo pulp-admin tasks details --task-id=b7c0e1f5-9371-4070-94cc-c94efe667c6e
+----------------------------------------------------------------------+
                              Task Details
+----------------------------------------------------------------------+

Operations:       consumer_content_applicability_regeneration
Resources:
State:            Failed
Start Time:       2018-05-02T11:46:31Z
Finish Time:      2018-05-02T11:47:39Z
Result:           N/A
Task Id:          b7c0e1f5-9371-4070-94cc-c94efe667c6e
Progress Report:
Traceback:        Traceback (most recent call last):   File
                  "/usr/lib/python2.7/site-packages/celery/app/trace.py", line
                  240, in trace_task     R = retval = fun(*args, **kwargs)
                  File
                  "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py",
                  line 527, in __call__     return super(Task,
                  self).__call__(*args, **kwargs)   File
                  "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py",
                  line 107, in __call__     return super(PulpTask,
                  self).__call__(*args, **kwargs)   File
                  "/usr/lib/python2.7/site-packages/celery/app/trace.py", line
                  438, in __protected_call__     return self.run(*args,
                  **kwargs)   File
                  "/usr/lib/python2.7/site-packages/pulp/server/managers/consume
                  r/applicability.py", line 108, in
                  regenerate_applicability_for_consumers
                  manager.regenerate_applicability(profile_hash, content_type,
                  profile_id, repo_id)   File
                  "/usr/lib/python2.7/site-packages/pulp/server/managers/consume
                  r/applicability.py", line 251, in regenerate_applicability
                  profile = unit_profile['profile'] TypeError: 'NoneType' object
                  has no attribute '__getitem__'


[crash/LI] root@li-lc-1578:~# sudo pulp-admin tasks details --task-id=b297b300-9f06-4556-a233-7dac0c374c78
+----------------------------------------------------------------------+
                              Task Details
+----------------------------------------------------------------------+

Operations:
Resources:        ac2d7ee6-344d-4290-bfb1-5e07f7210faf (delete_queue)
State:            Successful
Start Time:       2018-05-02T11:57:07Z
Finish Time:      2018-05-02T11:57:07Z
Result:           N/A
Task Id:          b297b300-9f06-4556-a233-7dac0c374c78
Progress Report:
-------------------------------------------------

Peter

Comment 7 Peter Vreman 2018-05-16 15:25:20 UTC

I used a relevant part of the PR to save me some daily hanging generate_applicability tasks during my unittests run. With this i noticed that the patch in PR https://github.com/Katello/katello/pull/7206 for generate_applicability is not enough, because i still got paused tasks because the pulp consumers were gone.

The change in app/lib/actions/pulp/consumer/generate_applicability.rb has only a rescue around the single id and not the array of ids. The call 'pulp_extensions.consumer.regenerate_applicability_by_ids(input[:uuids])' is not wrapped in a rescue block

Comment 8 Evgeni Golov 2018-05-18 08:18:22 UTC

Peter, not sure why you added a needinfo on me -- I can't provide any, as I did not write the patch. Did you mean to NI Chris?

Comment 9 Peter Vreman 2018-05-18 10:25:28 UTC

Evgeni, i added needinfo because you moved it from Dev to QA

Comment 10 Peter Vreman 2018-05-24 07:20:29 UTC

The changes of PR https://github.com/Katello/katello/pull/7206 are already included in both 6.2.15 and 6.3.1. Can this not be closed?

Comment 11 Peter Vreman 2018-05-28 08:30:12 UTC

With 6.3.1 this is still not completly fixed, i still get the backtrace during my OS testsuite run that does many puppet tests (yum installs causing package profiles upload tasks) and also has tests doing system unregister and register tests.

-------------------
ction:

Actions::Pulp::Consumer::GenerateApplicability

Input:

{"uuids"=>["6276fd15-3fe9-48ec-a491-1c721859258c"],
 "remote_user"=>"admin",
 "remote_cp_user"=>"admin"}

Output:

{"pulp_tasks"=>
  [{"exception"=>nil,
    "task_type"=>
     "pulp.server.managers.consumer.applicability.regenerate_applicability_for_consumers",
    "_href"=>"/pulp/api/v2/tasks/9f3371e4-93b7-4dbf-970c-cf35bffb5e68/",
    "task_id"=>"9f3371e4-93b7-4dbf-970c-cf35bffb5e68",
    "tags"=>["pulp:action:consumer_content_applicability_regeneration"],
    "finish_time"=>"2018-05-25T08:23:04Z",
    "_ns"=>"task_status",
    "start_time"=>"2018-05-25T08:21:41Z",
    "traceback"=>
     "Traceback (most recent call last):\n  File \"/usr/lib/python2.7/site-packages/celery/app/trace.py\", line 240, in trace_task\n    R = retval = fun(*args, **kwargs)\n  File \"/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py\", line 527, in __call__\n    return super(Task, self).__call__(*args, **kwargs)\n  File \"/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py\", line 107, in __call__\n    return super(PulpTask, self).__call__(*args, **kwargs)\n  File \"/usr/lib/python2.7/site-packages/celery/app/trace.py\", line 438, in __protected_call__\n    return self.run(*args, **kwargs)\n  File \"/usr/lib/python2.7/site-packages/pulp/server/managers/consumer/applicability.py\", line 108, in regenerate_applicability_for_consumers\n    manager.regenerate_applicability(profile_hash, content_type, profile_id, repo_id)\n  File \"/usr/lib/python2.7/site-packages/pulp/server/managers/consumer/applicability.py\", line 251, in regenerate_applicability\n    profile = unit_profile['profile']\nTypeError: 'NoneType' object has no attribute '__getitem__'\n",
    "spawned_tasks"=>[],
    "progress_report"=>{},
    "queue"=>"reserved_resource_worker-2",
    "state"=>"error",
    "worker_name"=>"reserved_resource_worker-2@li-lc-1578",
    "result"=>nil,
    "error"=>
     {"code"=>"PLP0000",
      "data"=>{},
      "description"=>"'NoneType' object has no attribute '__getitem__'",
      "sub_errors"=>[]},
    "_id"=>{"$oid"=>"5b07c7953270b1fb9d499d6b"},
    "id"=>"5b07c7953270b1fb9d499d6b"}],
 "poll_attempts"=>{"total"=>25, "failed"=>1}}

Exception:

Katello::Errors::PulpError: PLP0000: 'NoneType' object has no attribute '__getitem__'

Backtrace:

/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5.64.2/app/lib/actions/pulp/abstract_async_task.rb:121:in `block in external_task='
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5.64.2/app/lib/actions/pulp/abstract_async_task.rb:119:in `each'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5.64.2/app/lib/actions/pulp/abstract_async_task.rb:119:in `external_task='
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/action/polling.rb:98:in `poll_external_task_with_rescue'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/action/polling.rb:21:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/action/cancellable.rb:13:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5.64.2/app/lib/actions/pulp/abstract_async_task.rb:45:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/action.rb:518:in `block (3 levels) in execute_run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware.rb:18:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware.rb:31:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware.rb:18:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5.64.2/app/lib/actions/middleware/remote_action.rb:16:in `block in run'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5.64.2/app/lib/actions/middleware/remote_action.rb:40:in `block in as_remote_user'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5.64.2/app/models/katello/concerns/user_extensions.rb:21:in `cp_config'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5.64.2/app/lib/actions/middleware/remote_action.rb:27:in `as_cp_user'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5.64.2/app/lib/actions/middleware/remote_action.rb:39:in `as_remote_user'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5.64.2/app/lib/actions/middleware/remote_action.rb:16:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware.rb:18:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/action/progress.rb:30:in `with_progress_calculation'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/action/progress.rb:16:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware.rb:18:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5.64.2/app/lib/actions/middleware/keep_locale.rb:11:in `block in run'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5.64.2/app/lib/actions/middleware/keep_locale.rb:22:in `with_locale'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5.64.2/app/lib/actions/middleware/keep_locale.rb:11:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware.rb:18:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware.rb:31:in `run'
-----------------

Comment 12 pulp-infra@redhat.com 2018-06-08 15:23:25 UTC

The Pulp upstream bug status is at NEW. Updating the external tracker on this bug.

Comment 13 pulp-infra@redhat.com 2018-06-08 15:23:29 UTC

The Pulp upstream bug priority is at Normal. Updating the external tracker on this bug.

Comment 14 pulp-infra@redhat.com 2018-06-11 17:32:37 UTC

The Pulp upstream bug status is at POST. Updating the external tracker on this bug.

Comment 15 pulp-infra@redhat.com 2018-06-12 08:49:09 UTC

The Pulp upstream bug status is at MODIFIED. Updating the external tracker on this bug.

Comment 16 Brad Buckingham 2018-06-12 12:36:02 UTC

Hi Peter,

As mentioned, additional changes were needed in pulp to fully address this bugzilla.  I am moving it back to POST, since that fix is now upstream.

Comment 17 pulp-infra@redhat.com 2018-07-09 15:06:01 UTC

The Pulp upstream bug status is at CLOSED - CURRENTRELEASE. Updating the external tracker on this bug.

Comment 18 Chris Duryee 2018-07-16 11:28:49 UTC

(In reply to Peter Vreman from comment #7)
> I used a relevant part of the PR to save me some daily hanging
> generate_applicability tasks during my unittests run. With this i noticed
> that the patch in PR https://github.com/Katello/katello/pull/7206 for
> generate_applicability is not enough, because i still got paused tasks
> because the pulp consumers were gone.
> 
> The change in app/lib/actions/pulp/consumer/generate_applicability.rb has
> only a rescue around the single id and not the array of ids. The call
> 'pulp_extensions.consumer.regenerate_applicability_by_ids(input[:uuids])' is
> not wrapped in a rescue block

Peter,

This bug had a couple of areas modified and I want to make sure we have them all covered.

How do you reproduce the issue that requires 'pulp_extensions.consumer.regenerate_applicability_by_ids(input[:uuids])' to be wrapped in a rescue block? I can't find any code paths that call the multiple-uuid version of the task, so I'm not sure how to reproduce the issue. Are you invoking the task via rake console?

Comment 19 Peter Vreman 2018-07-16 11:43:34 UTC

Hoi Chris,


Since the upgrade to 6.2.15 i have not seen an backtrace anymore in this part of the Katello code. You can ignore my comment about the multiple_ids

The only other backtrace i see with the generate applicability during my CI test server runs is a pulp backtrace covered in https://pulp.plan.io/issues/3745.

Peter

Comment 24 Roman Plevka 2018-09-20 13:42:08 UTC

FAILED QA
on sat6.4.0-22

- okay, first things first - i'm not quite sure that what i did is valid, so here it is:

- to extend the time window for triggering the race condition (removal pulp consumer just before making a call to pulp for errata applicability) I tweaked the katello registration_manager code (app/services/katello/registration_manager.rb) in a way that i put a `sleep(10)`
between

pulp_consumer_destory(host.content_facet.uuid) if host.content_facet.try(:uuid)
(yeah, there is indeed a typo in the method name in the code)

and
host.subscription_facet.try(:destroy!)

so then, i have my content host and i choose to remove it (using ui).
- this removes the pulp ocnsumer and sleeps for a while before proceeding to the rest of the cleanup.
- now i send a PUT request to api/hosts/123/errata/applicability
to trigger the applicability task (which spawns pulp generate_applicability).

- the pulp returns 404 (as seen in the syslog) but what katello does is far from handling it in a clean way:

Sep 20 09:08:43 satellite.com foreman[241723]: POST: https://satellite.com/pulp/api/v2/consumers/64a83a9b-7035-4038-a978-5d08a38f0dd2/actions/content/regenerate_applicability/: {}: {"content_type"=>"application/json", "accept"=>"application/json"}
                                                                        404 Not Found: {"http_request_method": "POST", "exception": null, "error_message": "Missing resource(s): consumer_id=64a83a9b-7035-4038-a978-5d08a38f0dd2", "_href": "/pulp/api/v2/consumers/64a83a9b-7035-4038-a978-5d08a38f0dd2/actions/content/regenerate_applicability/", "http_status": 404, "error": {"code": "PLP0009", "data": {"resources": {"consumer_id": "64a83a9b-7035-4038-a978-5d08a38f0dd2"}}, "description": "Missing resource(s): consumer_id=64a83a9b-7035-4038-a978-5d08a38f0dd2", "sub_errors": []}, "traceback": null, "resources": {"consumer_id": "64a83a9b-7035-4038-a978-5d08a38f0dd2"}}
Sep 20 09:08:43 intel-lizardhead-02.lab.bos.redhat.com foreman[241723]: Pulp consumer 64a83a9b-7035-4038-a978-5d08a38f0dd2 not found.
Sep 20 09:08:43 intel-lizardhead-02.lab.bos.redhat.com foreman[241723]: undefined method `each' for true:TrueClass (NoMethodError)
...
[i will attach the whole error message as a file]


- the task remains in error state and paused.



- was my tweak putting me into a state where i would never get if i didn't introduce it?
- i tihnk one can hit this with a certain amount of (un)luck even without the tweak in place and many hosts and host actions going on at the same time

Comment 25 Roman Plevka 2018-09-20 13:43:57 UTC

Created attachment 1485152 [details]
QA_full_traceback

Comment 26 pulp-infra@redhat.com 2018-09-20 14:01:59 UTC

Requesting needsinfo from upstream developer ttereshc because the 'FailedQA' flag is set.

Comment 27 Justin Sherrill 2018-09-20 15:35:19 UTC

Katello needs to do some work to handle the 404 properly.

Comment 28 pulp-infra@redhat.com 2018-09-20 16:02:14 UTC

Requesting needsinfo from upstream developer ttereshc because the 'FailedQA' flag is set.

Comment 30 Satellite Program 2018-09-21 02:02:57 UTC

Upstream bug assigned to jsherril

Comment 31 Satellite Program 2018-09-21 02:03:02 UTC

Upstream bug assigned to jsherril

Comment 32 Brad Buckingham 2018-09-21 20:33:08 UTC

The katello changes have been merged using upstream redmine 25003.  Moving BZ to POST.

Comment 34 Roman Plevka 2018-10-03 10:35:09 UTC

VERIFIED
on sat6.4.0-25

the 404 seems to be handled properly now and the task finishes successfully despite the pulp's 404.


...
[deleting the consumer]
Oct  3 06:27:01 cisco-b22m3-01 pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.consumer.agent.delete_queue[7f30c7a8-eeb3-4c90-8ec9-23e40126fe73]  ETA:[2018-10-03 10:37:01.626996+00:00]

[now i trigerred the applicability task for the host]
Oct  3 06:27:05 cisco-b22m3-01 pulp: kombu.transport.qpid:INFO: Connected to qpid with SASL mechanism ANONYMOUS
Oct  3 06:27:05 cisco-b22m3-01 pulp: pulp.server.webservices.middleware.exception:INFO: Missing resource(s): consumer_id=16ef4af4-e46a-4a2c-acae-2fc2d8b4c6b3
Oct  3 06:27:05 cisco-b22m3-01 pulp: django.request:WARNING: Not Found: /pulp/api/v2/consumers/16ef4af4-e46a-4a2c-acae-2fc2d8b4c6b3/actions/content/regenerate_applicability/



in prod.log:
2018-10-03T06:27:05 [I|kat|] POST: https://sat6.redhat.com/pulp/api/v2/consumers/16ef4af4-e46a-4a2c-acae-2fc2d8b4c6b3/actions/content/regenerate_applicability/: {}: {"content_type"=>"application/json", "accept"=>"application/json"}
404 Not Found: {"http_request_method": "POST", "exception": null, "error_message": "Missing resource(s): consumer_id=16ef4af4-e46a-4a2c-acae-2fc2d8b4c6b3", "_href": "/pulp/api/v2/consumers/16ef4af4-e46a-4a2c-acae-2fc2d8b4c6b3/actions/content/regenerate_applicability/", "http_status": 404, "error": {"code": "PLP0009", "data": {"resources": {"consumer_id": "16ef4af4-e46a-4a2c-acae-2fc2d8b4c6b3"}}, "description": "Missing resource(s): consumer_id=16ef4af4-e46a-4a2c-acae-2fc2d8b4c6b3", "sub_errors": []}, "traceback": null, "resources": {"consumer_id": "16ef4af4-e46a-4a2c-acae-2fc2d8b4c6b3"}}


 curl -sku admin:changeme https://sat6.com/foreman_tasks/api/tasks/4e7b7720-4268-4619-a170-e2f9e1cbb5d1 | jq
{
  "id": "4e7b7720-4268-4619-a170-e2f9e1cbb5d1",
  "label": "Actions::Katello::Host::GenerateApplicability",
  "pending": false,
  "action": "Generate applicability {\"services_checked\"=>[\"pulp\", \"pulp_auth\"],\n \"host_ids\"=>[4],\n \"use_queue\"=>false,\n \"current_user_id\"=>4}",
  "username": "admin",
  "started_at": "2018-10-03 10:23:16 UTC",
  "ended_at": "2018-10-03 10:23:17 UTC",
  "state": "stopped",
  "result": "success",
  "progress": 1,
  "input": {
    "services_checked": [
      "pulp",
      "pulp_auth"
    ],
    "host_ids": [
      4
    ],
    "use_queue": false,
    "current_user_id": 4
  },
  "output": {},
  "humanized": {
    "action": "Generate applicability",
    "input": "{\"services_checked\"=>[\"pulp\", \"pulp_auth\"],\n \"host_ids\"=>[4],\n \"use_queue\"=>false,\n \"current_user_id\"=>4}\n",
    "output": "",
    "errors": []
  },
  "cli_example": null
}

Comment 35 Bryan Kearney 2018-10-16 18:55:53 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:2927