Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 2250206

Summary: Failed to perform incremental restore with "could not connect to database template1: FATAL: could not open file "base/1/XXXXX": No such file or directory" error
Product: Red Hat Satellite Reporter: Hao Chang Yu <hyu>
Component: Satellite MaintainAssignee: satellite6-bugs <satellite6-bugs>
Status: CLOSED DUPLICATE QA Contact: Satellite QE Team <sat-qe-bz-list>
Severity: high Docs Contact:
Priority: unspecified    
Version: 6.13.6CC: arahaman, dhjoshi, jpasqual, rlavi, saydas
Target Milestone: Unspecified   
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-11-20 13:28:03 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Hao Chang Yu 2023-11-17 01:21:54 UTC
Description of problem:
Getting the following error when performing an incremental restore:
~~~
Extract any existing tar files in backup: 
/ Extracting pgsql data                                               [OK]      
--------------------------------------------------------------------------------
REINDEX databases: 
\ Reindexing the databases                                            [FAIL]    
Failed executing runuser - postgres -c "reindexdb -a", exit status 1:
 reindexdb: error: could not connect to database template1: FATAL:  could not open file "base/1/XXXXX": No such file or directory
--------------------------------------------------------------------------------
Scenario [Restore backup] failed.

The following steps ended up in failing state:

  [restore-reindex-databases]

Resolve the failed steps and rerun the command.

If the situation persists and, you are unclear what to do next,
contact Red Hat Technical Support.

In case the failures are false positives, use
--whitelist="restore-reindex-databases"
~~~



How reproducible:
Easy

Steps to Reproduce:
1. Create a full backup from the source Satellite.
2. Create an incremental backup on top of the full backup.
3. Restore the full backup to the target Satellite.
4. Restore the incremental backup to the target Satellite.

Actual results:
Failed executing runuser - postgres -c "reindexdb -a", exit status 1:
 reindexdb: error: could not connect to database template1: FATAL:  could not open file "base/1/XXXXX": No such file or directory

Expected results:
No error

Additional info:
It is caused by the reindexdb step that will always be performed after restoring from a backup. The reindexdb created many new PG data files in the target Satellite. Since those PG data files are not in the source Satellite, they got deleted when performing the incremental backup and caused the above error.

The workaround for this issue is to skip the reindex steps in the restore command.

For example:

Full backup: /backup/satellite-backup-2023-11-16-16-03-49
Incremental backup: /backup/satellite-backup-2023-11-16-16-18-24

~~~
satellite-maintain restore --whitelist="restore-reindex-databases" /backup/satellite-backup-2023-11-16-16-03-49
satellite-maintain restore --whitelist="restore-reindex-databases" /backup/satellite-backup-2023-11-16-16-18-24
~~~

Comment 2 Ron Lavi 2023-11-20 13:28:03 UTC
Looks like this got fixed in 2230934 on Satellite 6.14.0 and proposed to backport into 6.13.z,
therefore closing this as a DUP for now

*** This bug has been marked as a duplicate of bug 2230934 ***