Bug 2125727

Summary:	Satellite 6.11 fails to install with error change from 'notrun' to ['0'] failed: 'pulpcore-manager migrate --noinput' returned instead of one of [0]
Product:	Red Hat Satellite	Reporter:	Jessica Hanley <jhanley>
Component:	Installer	Assignee:	satellite6-bugs <satellite6-bugs>
Status:	NEW ---	QA Contact:	Satellite QE Team <sat-qe-bz-list>
Severity:	high	Docs Contact:
Priority:	high
Version:	6.11.0	CC:	dhjoshi, ehelms, jfont, jrichards2, saydas, wpinheir
Target Milestone:	Unspecified	Keywords:	Triaged
Target Release:	Unused
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:		Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Jessica Hanley 2022-09-09 21:23:08 UTC

Description of problem:

When the customer attempts to configure Satellite 6.11 on either RHEL 8 or on RHEL 7 (though they would prefer to use RHEL 8) on VMware Nutanix hypervisors, pulpcore-manager returns null instead of zero.  Here's the error:

2022-09-09 16:16:24 [NOTICE] [configure] 1500 configuration steps out of 1630 steps complete.
2022-09-09 16:16:28 [ERROR ] [configure] 'pulpcore-manager migrate --noinput' returned  instead of one of [0]
2022-09-09 16:16:28 [ERROR ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns: change from 'notrun' to ['0'] failed: 'pulpcore-manager migrate --noinput' returned  instead of one of [0]


Version-Release number of selected component (if applicable):

- satellite 6.11
- pulp3

How reproducible:

- it consistently fails in the customer's environment
- it consistently succeeds in red hat's virtual environments


Steps to Reproduce:
1.  install RHEL 8 from ISO file downloaded from RH website
2.  install satellite 6.11 using our install documentation
3.  run this command to configure (i've used dummy values here):

# satellite-installer --scenario satellite --foreman-initial-organization "MYORG" --foreman-initial-location "MYLOCATION"  --foreman-initial-admin-username admin --foreman-initial-admin-password admin -v

Actual results:

the satellite-installer tool fails to configure satellite


Expected results:

the satellite-installer tool should successfully configure satellite


Additional info:

- the satellite server has plenty of memory, CPUs and storage
- the customer reports that this succeeds on their VMware vsan-based hypervisors; it only fails on their Nutanix hypervisors

- the customer also tried this on RHEL 7 and had the same results
- the customer tried a less-busy hypervisor and had the same results
- the customer turned off firewalld and selinux and had the same results
- FIPS isn't enabled

Comment 7 Sayan Das 2023-06-26 09:43:11 UTC

This is just for external visibility :

If upgrade\installer fails with an error like this :

Register/Foreman_smartproxy[satellite.example.com]: Failed to call refresh: Error making PUT request to https://satellite.example.com/api/v2/smart_proxies/1/refresh: Unable to communicate with the Capsule: ERF12-2530 [ProxyAPI::ProxyException]: Unable to detect features ([RestClient::Exceptions::ReadTimeout]: Timed out reading data from server) for Capsule https://satellite.example.com:9090/v2/features Please check the Capsule is configured and running on the host.

And then around the same time, The pulp ( /var/log/messages ) logs are showing a segfault like this i..e 

May 10 09:46:02 satellite kernel: pulpcore-worker[14546]: segfault at 0 ip 00002aabe700bcac sp 00007ffe63972900 error 6 in etree.cpython-38-x86_64-linux-gnu.so[2aabe6f77000+1cc000]
Jun 10 09:46:02 satellite abrt-hook-ccpp: Process 44813 (python3.8) of user 30538 killed by SIGSEGV - dumping core

Then check your locale\LANG settings.

It cannot be en_US.iso885915 but it HAS TO BE en_US.UTF-8 and then only, we can expect that the pulp services as well as the next upgrade\installer execution would work without that error\segfault crash.

Comment 8 Sayan Das 2023-06-26 09:44:42 UTC

Ignore the conflict with dates there i.e. May 10 and Jun 10 .. It's just a copy+paste error .