Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1867311 - Upgrade fails when checkpoint_segments postgres parameter configured
Summary: Upgrade fails when checkpoint_segments postgres parameter configured
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Installation
Version: 6.8.0
Hardware: x86_64
OS: Linux
high
high
Target Milestone: 6.8.0
Assignee: Ewoud Kohl van Wijngaarden
QA Contact: Devendra Singh
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-08-08 14:41 UTC by Pavel Moravec
Modified: 2024-10-01 16:45 UTC (History)
8 users (show)

Fixed In Version: foreman-installer-2.1.2.6-1
Doc Type: Known Issue
Doc Text:
Upgrade fails when checkpoint_segments postgres parameter configured
Clone Of:
Environment:
Last Closed: 2020-10-27 13:05:24 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Foreman Issue Tracker 28928 0 Normal Closed checkpoint_segments in PostgreSQL config breaks upgrades 2021-02-20 16:10:38 UTC
Red Hat Bugzilla 1851773 0 unspecified CLOSED When applying tuning configuration via "satellite-installer --tuning <template>" the old configuration still on custom-h... 2021-02-22 00:41:40 UTC
Red Hat Bugzilla 1867753 0 unspecified CLOSED Add check for checkpoint_segments postgres parameter 2023-09-14 06:05:34 UTC
Red Hat Product Errata RHSA-2020:4366 0 None None None 2020-10-27 13:08:18 UTC

Internal Links: 1867753

Description Pavel Moravec 2020-08-08 14:41:19 UTC 
Description of problem:
Upgrading from 6.7 with /etc/foreman-installer/custom-hiera.yaml :

postgresql::server::config_entries:
  checkpoint_segments: 32

to 6.8 fails on starting postgres service:


Aug  8 16:18:34 pmoravec-sat-test sh: 2020-08-08 14:18:34.450 GMT [17329] LOG:  unrecognized configuration parameter "checkpoint_segments" in file "/var/opt/rh/rh-postgresql12/lib/pgsql/data/postgresql.conf" line 760
Aug  8 16:18:34 pmoravec-sat-test sh: 2020-08-08 14:18:34.450 GMT [17329] FATAL:  configuration file "/var/opt/rh/rh-postgresql12/lib/pgsql/data/postgresql.conf" contains errors
Aug  8 16:18:34 pmoravec-sat-test systemd: postgresql.service: main process exited, code=exited, status=1/FAILURE


Further attempt to remove the tuning from custom hiera fails: commenting out the parameter from the yaml file and running satellite-installer does _not_ update the /var/opt/rh/rh-postgresql12/lib/pgsql/data/postgresql.conf file, hence postgresql fails to start again.


Version-Release number of selected component (if applicable):
satellite-6.8.0-0.7.beta.el7sat.noarch
satellite-installer-6.8.0.8-1.el7sat.noarch


How reproducible:
100% I guess?


Steps to Reproduce:
1. Have Sat 6.7 with /etc/foreman-installer/custom-hiera.yaml :
postgresql::server::config_entries:
  checkpoint_segments: 32

2. Run upgrade to 6.8


Actual results:
2. is supposed to fail with above error


Expected results:
2. upgrade to proceed well


Additional info:
Comment 2 Ewoud Kohl van Wijngaarden 2020-08-10 10:10:35 UTC 
This is indeed expected. PostgreSQL has dropped the checkpoint_segments option. See https://www.postgresql.org/docs/9.5/release-9-5.html which states:

> Replace configuration parameter checkpoint_segments with min_wal_size and max_wal_size (Heikki Linnakangas)

The option is not removed due to a technical limitation. The code can only set the value:

https://github.com/puppetlabs/puppetlabs-postgresql/blob/ef64c30b96ee22804de6e0bcf3e83b988c23b22a/manifests/server.pp#L185-L189

And this particular implementation only manages individual ini settings which means unmanaged entries are ignored.

I have submitted https://github.com/puppetlabs/puppetlabs-postgresql/pull/1187 and with that, you should be able to set checkpoint_segments to ~ in custom-hiera.yaml to remove an entry.

By its very nature, we don't touch custom-hiera.yaml at all - this is really supposed to be a user way of setting things. It'd be my recommendation that we add a release note that users who previously followed the tuning guide (which recommended this) update to the new tuning parameter.
Comment 3 Eric Helms 2020-08-10 14:46:26 UTC 
Adding that I think this is should be a docs bug with release notes to inform users who had this configured that they need to:

 1) Remove it from custom-hiera.yaml
 2) Edit /var/opt/rh/rh-postgresql12/lib/pgsql/data/postgresql.conf to remove the checkpoint_segments entry manually
Comment 4 Ewoud Kohl van Wijngaarden 2020-08-10 14:53:27 UTC 
https://bugzilla.redhat.com/show_bug.cgi?id=1851773 is a larger overarching issue. I think it makes sense to keep this open for now since this can actually prevent upgrades while the other might result in a working but unexpected situation.
Comment 5 Chris Roberts 2020-08-10 16:41:47 UTC 
Added a bz to create a foreman-maintain check for this.
Comment 6 Bryan Kearney 2020-08-20 20:01:37 UTC 
Upstream bug assigned to ekohlvan
Comment 7 Bryan Kearney 2020-08-20 20:01:39 UTC 
Upstream bug assigned to ekohlvan
Comment 8 Bryan Kearney 2020-08-24 16:01:40 UTC 
Moving this bug to POST for triage into Satellite since the upstream issue https://projects.theforeman.org/issues/28928 has been resolved.
Comment 12 Devendra Singh 2020-09-28 17:19:12 UTC 
Verified on 6.8.0 Snap17

Verification points:

1- foreman-maintain upgrade "Check and Run" both fail with a proper error message if checkpoint_segments configured.

# foreman-maintain upgrade check --target-version 6.8 -y
Checking for new version of satellite-maintain...
Nothing to update, can't find new version of satellite-maintain.
Running preparation steps required to run the next scenarios
================================================================================
Check whether system has any non Red Hat repositories (e.g.: EPEL) enabled: 
| Checking repositories enabled on the system                         [OK]      
--------------------------------------------------------------------------------


Running Checks before upgrading to Satellite 6.8
================================================================================
Clean old Kernel and initramfs files from tftp-boot:                  [OK]
--------------------------------------------------------------------------------
Check number of fact names in database:                               [OK]
--------------------------------------------------------------------------------
Check for verifying syntax for ISP DHCP configurations:               [OK]
--------------------------------------------------------------------------------
Check whether all services are running:                               [OK]
--------------------------------------------------------------------------------
Check whether all services are running using the ping call:           [OK]
--------------------------------------------------------------------------------
Check for paused tasks:                                               [OK]
--------------------------------------------------------------------------------
Check to verify no empty CA cert requests exist:                      [OK]
--------------------------------------------------------------------------------
Check whether system is self-registered or not:                       [OK]
--------------------------------------------------------------------------------
Check to make sure root(/) partition has enough space:                [OK]
--------------------------------------------------------------------------------
Check to validate candlepin database:                                 [OK]
--------------------------------------------------------------------------------
Check for running tasks:                                              [OK]
--------------------------------------------------------------------------------
Check for old tasks in paused/stopped state:                          [OK]
--------------------------------------------------------------------------------
Check for pending tasks which are safe to delete:                     [OK]
--------------------------------------------------------------------------------
Check for tasks in planning state:                                    [OK]
--------------------------------------------------------------------------------
Check to verify if any hotfix installed on system: 
- Checking for presence of hotfix(es). It may take some time to verify.         
                                                                      [OK]
--------------------------------------------------------------------------------
Check whether system has any non Red Hat repositories (e.g.: EPEL) enabled: 
| Checking repositories enabled on the system                         [OK]      
--------------------------------------------------------------------------------
Check if TMOUT environment variable is set:                           [OK]
--------------------------------------------------------------------------------
Check if any upstream repositories are enabled on system: 
| Checking for presence of upstream repositories                      [OK]      
--------------------------------------------------------------------------------
Check if yum exclude list is configured:                              [OK]
--------------------------------------------------------------------------------
Check for roles that have filters with multiple resources attached:   [OK]
--------------------------------------------------------------------------------
Check for duplicate permissions from database:                        [OK]
--------------------------------------------------------------------------------
Check whether reports have correct associations:                      [OK]
--------------------------------------------------------------------------------
Check if checkpoint_segments configuration exists on the system:      [FAIL]
ERROR: Tuning option 'checkpoint_segments' found.
This option is no longer valid for PostgreSQL 9.5 or newer.
Please remove it from following file and re-run the command.
- /etc/foreman-installer/custom-hiera.yaml
The presence of checkpoint_segments in /etc/foreman-installer/custom-hiera.yaml indicates manual tuning.
Manual tuning can override values provided by the --tuning parameter.
Review /etc/foreman-installer/custom-hiera.yaml for values that are already provided by the built in tuning profiles.
Built in tuning profiles also provide a supported upgrade path.

--------------------------------------------------------------------------------
Validate availability of repositories: 
| Validating availability of repositories for 6.8                               
\ Validating availability of repositories for 6.8                     [OK]      
--------------------------------------------------------------------------------
Scenario [Checks before upgrading to Satellite 6.8] failed.

The following steps ended up in failing state:

  [check-postgresql-checkpoint-segments]

Resolve the failed steps and rerun
the command. In case the failures are false positives,
use --whitelist="check-postgresql-checkpoint-segments"

2. foreman-installer version

# rpm -qa|grep  foreman-installer
foreman-installer-2.1.2.8-1.el7sat.noarch
Comment 15 errata-xmlrpc 2020-10-27 13:05:24 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Satellite 6.8 release), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:4366
Comment 16 errata-xmlrpc 2020-10-27 13:08:16 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Satellite 6.8 release), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:4366
Note You need to log in before you can comment on or make changes to this bug.