Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 2015963

Summary: pulpcore content app doesn't survive PostgreSQL disconnect
Product: Red Hat Satellite Reporter: Evgeni Golov <egolov>
Component: PulpAssignee: Evgeni Golov <egolov>
Status: CLOSED ERRATA QA Contact: Lai <ltran>
Severity: high Docs Contact:
Priority: unspecified    
Version: 6.10.0CC: ahumbe, ggainey, rchan, ttereshc, zhunting
Target Milestone: 6.10.1Keywords: Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: tfm-rubygem-hammer_cli_katello-1.1.2.3-1,tfm-rubygem-katello-4.1.1.38-1 Doc Type: Known Issue
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-11-18 14:40:51 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Evgeni Golov 2021-10-20 13:53:47 UTC 
Description of problem:
When the connection between the pulpcore content app (pulpcore-content.service/pulpcore-content.socket) and the PostgreSQL database is interrupted (e.g. because the DB was restarted), the content app doesn't correctly re-establish the DB connection, leading to content being unavailable to clients.

A restart of the pulpcore-content.service fixes the issue (until the next connection interruption).

This was reported before (at least in BZ#1994923), but never got a good root cause so far.

Additionally, a failed content app is not recognized by "hammer ping" (and friends), leading to the user thinking "everything is fine" if they use the ping endpoint for monitoring.

Version-Release number of selected component (if applicable):


How reproducible:
100%

Steps to Reproduce:
1. deploy satellite
2. systemctl restart postgresql
3. curl -k https://localhost/pulp/content/

Actual results:
500 Internal Server Error

Expected results:
Content listing (possibly empty, but no error)

Additional info:
Comment 2 pulp-infra@redhat.com 2021-10-25 05:08:08 UTC 
The Pulp upstream bug status is at NEW. Updating the external tracker on this bug.
Comment 3 pulp-infra@redhat.com 2021-10-25 05:08:09 UTC 
The Pulp upstream bug priority is at High. Updating the external tracker on this bug.
Comment 4 pulp-infra@redhat.com 2021-10-25 16:12:16 UTC 
The Pulp upstream bug status is at ASSIGNED. Updating the external tracker on this bug.
Comment 5 pulp-infra@redhat.com 2021-10-25 19:07:30 UTC 
The Pulp upstream bug status is at POST. Updating the external tracker on this bug.
Comment 6 pulp-infra@redhat.com 2021-10-25 21:07:30 UTC 
The Pulp upstream bug status is at ASSIGNED. Updating the external tracker on this bug.
Comment 7 pulp-infra@redhat.com 2021-10-25 21:07:31 UTC 
The Pulp upstream bug priority is at Urgent. Updating the external tracker on this bug.
Comment 8 pulp-infra@redhat.com 2021-10-27 19:07:02 UTC 
The Pulp upstream bug status is at POST. Updating the external tracker on this bug.
Comment 9 Bryan Kearney 2021-11-04 13:51:48 UTC 
Upstream bug assigned to egolov
Comment 10 Bryan Kearney 2021-11-04 13:51:49 UTC 
Moving this bug to POST for triage into Satellite since the upstream issue https://projects.theforeman.org/issues/33703 has been resolved.
Comment 11 Brad Buckingham 2021-11-04 23:13:24 UTC 
Moving this to ASSIGNED as the Pulp change is currently in review upstream.
Comment 12 pulp-infra@redhat.com 2021-11-05 14:07:51 UTC 
The Pulp upstream bug status is at MODIFIED. Updating the external tracker on this bug.
Comment 13 Brad Buckingham 2021-11-05 14:30:01 UTC 
Moving to POST as the upstream Pulp PR is now also merged.
Comment 14 pulp-infra@redhat.com 2021-11-05 18:09:39 UTC 
The Pulp upstream bug status is at CLOSED - CURRENTRELEASE. Updating the external tracker on this bug.
Comment 17 Lai 2021-11-09 17:44:44 UTC 
Steps to retest:

1. deploy satellite
2. systemctl restart postgresql
3. curl -k https://localhost/pulp/content/

Expected:
Content should be listed without errors

Actual:
content listed without errors

Verified on 6.10.1_01 with tfm-rubygem-katello-4.1.1.39-1.el7sat.noarch and tfm-rubygem-hammer_cli_katello-1.1.2.3-1.el7sat.noarch

# curl -k https://localhost/pulp/content/

        <!DOCTYPE html>
        <html>
            <body>
                <ul>
                
                    <li><a href="Default_Organization/Library/custom/zoo_prod/zoo_repo/">Default_Organization/Library/custom/zoo_prod/zoo_repo/</a></li>
                
                </ul>
            </body>
        </html>
Comment 22 errata-xmlrpc 2021-11-18 14:40:51 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Satellite Tools 6.10.1 Async Bug Fix Update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:4738