Bug 1932772

Summary: foreman: plugin execution times out when DB password can't be found
Product: Red Hat Enterprise Linux 7 Reporter: Evgeni Golov <egolov>
Component: sosAssignee: Jan Jansky <jjansky>
Status: CLOSED ERRATA QA Contact: Miroslav HradĂ­lek <mhradile>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.9CC: agk, bmr, cww, fkrska, jreznik, mhradile, plambri, pmoravec, sbradley, theute
Target Milestone: rcKeywords: OtherQA, ZStream
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: sos-3.9-5.el7_9.3 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-04-27 11:31:08 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Evgeni Golov 2021-02-25 08:45:04 UTC
Description of problem:
The `foreman` plugin tries to read the PostgreSQL password from `/etc/foreman/database.yml` and falls back to an empty string if that can't be found for some reason:

https://github.com/sosreport/sos/blob/00a25deaba41cd34a2143b5324d22a7c35098c1c/sos/plugins/foreman.py#L37-L59

However, when one tries to execute `psql` without a password, it will just sit there and wait for the user to enter one:

```console
# PGPASSWORD= psql -h localhost -p 5432 -U foreman -d foreman -c 'select 1;'
Password for user foreman: 
```

This leads to the plugin timing out:

```
[plugin:foreman] command 'psql -h localhost -p 5432 -U foreman -d foreman  -c 'SELECT table_name, pg_size_pretty(total_bytes) AS total, pg_size_pretty(index_bytes) AS INDEX , pg_size_pretty(toast_bytes) AS toast, pg_size_pretty(table_bytes) AS TABLE FROM ( SELECT *, total_bytes-index_bytes-COALESCE(toast_bytes,0) AS table_bytes FROM (SELECT c.oid,nspname AS table_schema, relname AS TABLE_NAME, c.reltuples AS row_estimate, pg_total_relation_size(c.oid) AS total_bytes, pg_indexes_size(c.oid) AS index_bytes, pg_total_relation_size(reltoastrelid) AS toast_bytes FROM pg_class c LEFT JOIN pg_namespace n ON n.oid = c.relnamespace WHERE relkind = '"'"'r'"'"') a) a order by total_bytes DESC'' timed out after 300s
[plugin:foreman] command 'psql -h localhost -p 5432 -U foreman -d foreman  -c 'WITH prefix_counts AS (SELECT split_part(name,'"'"'::'"'"',1) FROM fact_names) SELECT COUNT(*), split_part AS "fact_name_prefix" FROM prefix_counts GROUP BY split_part ORDER BY count DESC LIMIT 100'' timed out after 600s
[plugin:foreman] command 'psql -h localhost -p 5432 -U foreman -d foreman  -c 'select * from dynflow_schema_info'' timed out after 600s

 Plugin foreman timed out
```

The most trivial fix for this is to pass `--no-password` to `psql`, which will "Never issue a password prompt. If the server requires password authentication and a password is not available from other sources such as a .pgpass file, the connection attempt will fail. This option can be useful in batch jobs and scripts where no user is present to enter a password." and that's exactly the case we have here.

I am linking the `legacy-3.9` code above, as I am experiencing this on EL 7 with sos-3.9-5, but the code in `master` looks (for that part) identical and should suffer from the same issue.

And if you're wondering why we're facing this now, and not since a long time: The plugin works equally on Foreman servers and Foreman proxies (see `packages` definition in the plugin). There is no `/etc/foreman/database.yml` on the proxy, so the password can't be read, which was not a problem until recently as we didn't have a `psql` binary on the proxy. We have one now, but still no `/etc/foreman/database.yml` which leads to the before mentioned password prompt.


Version-Release number of selected component (if applicable):
sos-3.9-5.el7

How reproducible:
100%

Steps to Reproduce:
1. install RHEL7 with PostgreSQL
2. run sosreport --enable-plugins foreman

alternative steps:
Install a Satellite Capsule and add a PostgreSQL server to it (which will be the case in a later Satellite version anyways)

Actual results:
 Plugin foreman timed out

Expected results:
 Plugin correctly executed

Additional info:
Upstream report: https://github.com/sosreport/sos/issues/2421

Comment 13 errata-xmlrpc 2021-04-27 11:31:08 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (sos bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:1385