Bug 1336941
| Summary: | migrate_foreman error upgrading from 6.1 to 6.2 -> error: Some backend services are not running | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat Satellite | Reporter: | Brad Buckingham <bbuckingham> | ||||
| Component: | Upgrades | Assignee: | Partha Aji <paji> | ||||
| Status: | CLOSED ERRATA | QA Contact: | Sachin Ghai <sghai> | ||||
| Severity: | high | Docs Contact: | |||||
| Priority: | unspecified | ||||||
| Version: | 6.2.0 | CC: | cwelton, ehelms, lpramuk, sghai | ||||
| Target Milestone: | Unspecified | Keywords: | Triaged | ||||
| Target Release: | Unused | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | katello-installer-base-3.0.0.32-1 | Doc Type: | If docs needed, set a value | ||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2016-07-27 11:23:52 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Bug Depends On: | |||||||
| Bug Blocks: | 1335807 | ||||||
| Attachments: |
|
||||||
|
Description
Brad Buckingham
2016-05-17 21:04:27 UTC
To do some investigation, I updated /usr/share/katello-installer-base/hooks/pre/30-upgrade.rb to perform the following:
def migrate_foreman
Kafo::Helpers.execute('foreman-rake -- config -k use_pulp_oauth -v true')
exit 1
Kafo::Helpers.execute('foreman-rake db:migrate')
...
...
This caused the upgrade to abort and I was able to confirm the 404 behavior using:
# curl -s -k -u admin:EoxjDuoYdKmMZyPiLhtB94Kz3o82nivu https://localhost/pulp/api/v2//status/
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>404 Not Found</title>
</head><body>
<h1>Not Found</h1>
<p>The requested URL /pulp/api/v2//status/ was not found on this server.</p>
</body></html>
It may be unrelated; however, I also looked over the /etc/httpd/conf.d/* to compare the upgraded system against a 'satellite 6.2' install. I do see a lot of differences in the *.conf files (e.g. pulp.conf). Those differences may be part of the reason for the 404.
*** Bug 1338737 has been marked as a duplicate of this bug. *** Console logs while upgrading sat6.1.9 -> Sat6.2 snap13 on rhel7 still shows similar error as reported in bz description. I'll move this verified only with successful installer. [root@qe-sat6-upgrade-rhel7 ~]# satellite-installer --scenario satellite --upgrade Upgrading... Upgrade Step: stop_services... Upgrade Step: start_databases... Upgrade Step: update_http_conf... Upgrade Step: migrate_pulp... Upgrade Step: fix_pulp_httpd_conf... Upgrade Step: start_httpd... Upgrade Step: start_qpidd... Upgrade Step: start_pulp... Upgrade Step: migrate_candlepin... Upgrade Step: start_tomcat... Upgrade Step: fix_katello_settings_file... Upgrade Step: migrate_foreman... Upgrade Step: migrate_gutterball... Upgrade Step: remove_nodes_distributors... Upgrade Step: Running installer... /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]: Failed to call refresh: /usr/sbin/foreman-rake db:seed returned 1 instead of one of [0] /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]: /usr/sbin/foreman-rake db:seed returned 1 instead of one of [0] Report processor failed: Could not send report to Foreman at https://qe-sat6-upgrade-rhel7.satqe.lab.eng.rdu2.redhat.com/api/reports: SSL_connect returned=1 errno=0 state=SSLv3 read server session ticket A: tlsv1 alert unknown ca Installing Done [100%] [..................................................................] Something went wrong! Check the log for ERROR-level output The full log is at /var/log/foreman-installer/satellite.log Upgrade Step: restart_services... Upgrade Step: db_seed... Upgrade step db_seed failed. Check logs for more information. The error in comment 6 looks very similar to an error that I encountered yesterday. It was raised with Eric and should have been resolved. Eric, can you take a look? This is regarding the following error during db:seed: Report processor failed: Could not send report to Foreman at https://qe-sat6-upgrade-rhel7.satqe.lab.eng.rdu2.redhat.com/api/reports: SSL_connect returned=1 errno=0 state=SSLv3 read server session ticket A: tlsv1 alert unknown ca Sachin, the error reported in comment 6 is different than the original failure raised by this bug. That error is covered by bug 1339901. Verified with upgrade from sat6.1.9 -> sat6.2 GA snap14.2 on rhel7. Upgrade was successfully completed. I don't see any issue with migrate_foreman step. SATELLITE UPGRADE started at: Fri Jun 3 02:18:11 2016 [qe-sat6-upgrade-rhel7.satqe.lab.eng.rdu2.redhat.com] run: satellite-installer --scenario satellite --upgrade [qe-sat6-upgrade-rhel7.satqe.lab.eng.rdu2.redhat.com] out: Upgrading... [qe-sat6-upgrade-rhel7.satqe.lab.eng.rdu2.redhat.com] out: Upgrade Step: stop_services... [qe-sat6-upgrade-rhel7.satqe.lab.eng.rdu2.redhat.com] out: Upgrade Step: start_databases... [qe-sat6-upgrade-rhel7.satqe.lab.eng.rdu2.redhat.com] out: Upgrade Step: update_http_conf... [qe-sat6-upgrade-rhel7.satqe.lab.eng.rdu2.redhat.com] out: Upgrade Step: migrate_pulp... [qe-sat6-upgrade-rhel7.satqe.lab.eng.rdu2.redhat.com] out: Upgrade Step: fix_pulp_httpd_conf... [qe-sat6-upgrade-rhel7.satqe.lab.eng.rdu2.redhat.com] out: Upgrade Step: start_httpd... [qe-sat6-upgrade-rhel7.satqe.lab.eng.rdu2.redhat.com] out: Upgrade Step: start_qpidd... [qe-sat6-upgrade-rhel7.satqe.lab.eng.rdu2.redhat.com] out: Upgrade Step: start_pulp... [qe-sat6-upgrade-rhel7.satqe.lab.eng.rdu2.redhat.com] out: Upgrade Step: migrate_candlepin... [qe-sat6-upgrade-rhel7.satqe.lab.eng.rdu2.redhat.com] out: Upgrade Step: start_tomcat... [qe-sat6-upgrade-rhel7.satqe.lab.eng.rdu2.redhat.com] out: Upgrade Step: fix_katello_settings_file... [qe-sat6-upgrade-rhel7.satqe.lab.eng.rdu2.redhat.com] out: Upgrade Step: migrate_foreman... [qe-sat6-upgrade-rhel7.satqe.lab.eng.rdu2.redhat.com] out: Upgrade Step: remove_nodes_distributors... [qe-sat6-upgrade-rhel7.satqe.lab.eng.rdu2.redhat.com] out: Upgrade Step: Running installer... Done [100%] []Installing Done [100%] [] [qe-sat6-upgrade-rhel7.satqe.lab.eng.rdu2.redhat.com] out: The full log is at /var/log/foreman-installer/satellite.log [qe-sat6-upgrade-rhel7.satqe.lab.eng.rdu2.redhat.com] out: Upgrade Step: restart_services... [qe-sat6-upgrade-rhel7.satqe.lab.eng.rdu2.redhat.com] out: Upgrade Step: db_seed... [qe-sat6-upgrade-rhel7.satqe.lab.eng.rdu2.redhat.com] out: Upgrade Step: import_package_groups (this may take a while) ... [qe-sat6-upgrade-rhel7.satqe.lab.eng.rdu2.redhat.com] out: Upgrade Step: import_rpms (this may take a while) ... [qe-sat6-upgrade-rhel7.satqe.lab.eng.rdu2.redhat.com] out: Upgrade Step: import_distributions (this may take a while) ... [qe-sat6-upgrade-rhel7.satqe.lab.eng.rdu2.redhat.com] out: Upgrade Step: import_puppet_modules (this may take a while) ... [qe-sat6-upgrade-rhel7.satqe.lab.eng.rdu2.redhat.com] out: Upgrade Step: import_subscriptions (this may take a while) ... [qe-sat6-upgrade-rhel7.satqe.lab.eng.rdu2.redhat.com] out: Upgrade Step: elasticsearch_message... [qe-sat6-upgrade-rhel7.satqe.lab.eng.rdu2.redhat.com] out: Elasticsearch has been removed as a dependency. The database files can be removed manually with #rm -rf /var/lib/elasticsearch. Some packages are no longer needed and can be removed: #rpm -e ruby193-rubygem-tire tfm-rubygem-tire elasticsearch sigar-java sigar snappy-java lucene4-contrib lucene4 [qe-sat6-upgrade-rhel7.satqe.lab.eng.rdu2.redhat.com] out: Upgrade Step: remove_docker_v1_content (this may take a while) ... [qe-sat6-upgrade-rhel7.satqe.lab.eng.rdu2.redhat.com] out: Upgrade Step: remove_gutterball... [qe-sat6-upgrade-rhel7.satqe.lab.eng.rdu2.redhat.com] out: Katello upgrade completed! [qe-sat6-upgrade-rhel7.satqe.lab.eng.rdu2.redhat.com] out: SATELLITE UPGRADE finished at: Fri Jun 3 03:07:19 2016 Just to correct: Verification is done with upgrade from sat6.1.9 -> sat6.2 GA snap14.1 on rhel7. Rpm version is 14.2 satellite-6.2.0-14.2.el7sat.noarch Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2016:1501 |