Bug 1368085

Summary: Capsule upgrade from 6.2.0 to 6.2.1 fails to start httpd due to duplicated WSGI daemon definition
Product: Red Hat Satellite Reporter: Julio Entrena Perez <jentrena>
Component: UpgradesAssignee: Stephen Benjamin <stbenjam>
Status: CLOSED ERRATA QA Contact: Sanket Jagtap <sjagtap>
Severity: high Docs Contact:
Priority: urgent    
Version: 6.2.0CC: bbuckingham, bkearney, cdonnell, dojones, egolov, ehelms, ggatward, greartes, jcallaha, kdixon, mhrivnak, mmccune, nshaik, oshtaier, parmstro, sauchter, sghai, sjagtap, stbenjam, sthirugn, xdmoon, zhunting
Target Milestone: UnspecifiedKeywords: PrioBumpField, PrioBumpGSS, PrioBumpPM, PrioBumpQA, Triaged
Target Release: Unused   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: katello-installer-base-3.0.0.69-1 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1405536 (view as bug list) Environment:
Last Closed: 2017-01-26 15:56:01 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1385841, 1405536    
Attachments:
Description Flags
sosreport of capsule
none
capsule upgrade logs from console none

Description Julio Entrena Perez 2016-08-18 10:43:35 UTC
Created attachment 1191845 [details]
sosreport of capsule

Description of problem:
On a 6.2.0 Capsule (installed from scratch, never upgraded from 6.1.x), attempting to upgrade to 6.2.1 fails when httpd fails to start due to duplicated WSGI daemon definition.

Version-Release number of selected component (if applicable):


How reproducible:
Always

Steps to Reproduce:
1. Install a 6.2.0 Capsule from scratch
2. Run "satellite-installer --scenario capsule --upgrade"

Actual results:
Upgrade Step: start_httpd...
Redirecting to /bin/systemctl start  httpd.service
Job for httpd.service failed because the control process exited with error code. See "systemctl status httpd.service" and "journalctl -xe" for details.

Some services failed to start: httpd

Upgrade step start_httpd failed. Check logs for more information.

# systemctl status httpd.service
● httpd.service - The Apache HTTP Server
   Loaded: loaded (/usr/lib/systemd/system/httpd.service; enabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Thu 2016-08-18 06:30:12 EDT; 7min ago
     Docs: man:httpd(8)
           man:apachectl(8)
  Process: 21674 ExecStop=/bin/kill -WINCH ${MAINPID} (code=exited, status=1/FAILURE)
  Process: 21672 ExecStart=/usr/sbin/httpd $OPTIONS -DFOREGROUND (code=exited, status=1/FAILURE)
 Main PID: 21672 (code=exited, status=1/FAILURE)

Aug 18 06:30:12 sat62cap.usersys.redhat.com systemd[1]: Starting The Apache HTTP Server...
Aug 18 06:30:12 sat62cap.usersys.redhat.com httpd[21672]: AH00526: Syntax error on line 28 of /etc/httpd/conf.d/pulp.conf:
Aug 18 06:30:12 sat62cap.usersys.redhat.com httpd[21672]: Name duplicates previous WSGI daemon definition.
Aug 18 06:30:12 sat62cap.usersys.redhat.com systemd[1]: httpd.service: main process exited, code=exited, status=1/FAILURE
Aug 18 06:30:12 sat62cap.usersys.redhat.com kill[21674]: kill: cannot find process ""
Aug 18 06:30:12 sat62cap.usersys.redhat.com systemd[1]: httpd.service: control process exited, code=exited status=1
Aug 18 06:30:12 sat62cap.usersys.redhat.com systemd[1]: Failed to start The Apache HTTP Server.
Aug 18 06:30:12 sat62cap.usersys.redhat.com systemd[1]: Unit httpd.service entered failed state.
Aug 18 06:30:12 sat62cap.usersys.redhat.com systemd[1]: httpd.service failed.


Expected results:
Upgrade to 6.2.1. succeeds.

Additional info:

Comment 1 Julio Entrena Perez 2016-08-18 10:44:24 UTC
(In reply to Julio Entrena Perez from comment #0)
> 
> Steps to Reproduce:
> 1. Install a 6.2.0 Capsule from scratch
> 2. Run "satellite-installer --scenario capsule --upgrade"

Should be:

1. Install a 6.2.0 Capsule from scratch
2. Run "yum update"
3. Run "satellite-installer --scenario capsule --upgrade"

Comment 2 Julio Entrena Perez 2016-08-18 10:50:19 UTC
# pwd
/etc/httpd

# grep -Ri wsgi * | egrep -v "^#|error_log"
conf.d/pulp_content.conf:WSGISocketPrefix run/wsgi
conf.d/pulp_content.conf:WSGIProcessGroup pulp-content
conf.d/pulp_content.conf:WSGIApplicationGroup pulp-content
conf.d/pulp_content.conf:WSGIScriptAlias /pulp/content /usr/share/pulp/wsgi/content.wsgi
conf.d/pulp_content.conf:WSGIDaemonProcess pulp-content user=apache group=apache processes=3 display-name=%{GROUP}
conf.d/pulp_content.conf:WSGIImportScript /usr/share/pulp/wsgi/content.wsgi process-group=pulp-content application-group=pulp-content
conf.d/pulp_content.conf:<Files content.wsgi>
conf.d/pulp_content.conf:    WSGIPassAuthorization On
conf.d/pulp_content.conf:    WSGIProcessGroup pulp-content
conf.d/pulp_content.conf:    WSGIApplicationGroup pulp-content
conf.d/pulp_rpm.conf:# block rewrites all requests to `/pulp/repos/` to the location of the WSGI
conf.d/pulp_rpm.conf:# application, content.wsgi, provided by the Pulp platform. The content.wsgi
conf.d/pulp_rpm.conf:    WSGIAccessScript /usr/share/pulp/wsgi/repo_auth.wsgi
conf.d/pulp_rpm.conf:    WSGIAccessScript /usr/share/pulp/wsgi/repo_auth.wsgi
conf.d/pulp_streamer.conf:    WSGIAccessScript /usr/share/pulp/wsgi/streamer_auth.wsgi
conf.d/pulp_puppet.conf:WSGIDaemonProcess pulp_forge user=apache group=apache processes=3 display-name=%{GROUP}
conf.d/pulp_puppet.conf:WSGIProcessGroup pulp_forge
conf.d/pulp_puppet.conf:WSGIApplicationGroup pulp_forge
conf.d/pulp_puppet.conf:WSGIScriptAlias /api/v1 /usr/share/pulp/wsgi/puppet_forge.wsgi process-group=pulp_forge application-group=pulp_forge
conf.d/pulp_puppet.conf:WSGIScriptAlias /pulp_puppet/forge /usr/share/pulp/wsgi/puppet_forge.wsgi process-group=pulp_forge application-group=pulp_forge
conf.d/pulp_puppet.conf:WSGIScriptAlias /v3 /usr/share/pulp/wsgi/puppet_forge.wsgi process-group=pulp_forge application-group=pulp_forge
conf.d/pulp_puppet.conf:WSGIPassAuthorization On
conf.d/03-crane.conf:  WSGIScriptAlias / "/usr/share/crane/crane.wsgi"
conf.d/05-pulp-http.conf:  DocumentRoot "/usr/share/pulp/wsgi"
conf.d/05-pulp-http.conf:  ## Directories, there should at least be a declaration for /usr/share/pulp/wsgi
conf.d/05-pulp-http.conf:  <Directory "/usr/share/pulp/wsgi">
conf.d/05-pulp-https.conf:  DocumentRoot "/usr/share/pulp/wsgi"
conf.d/05-pulp-https.conf:  ## Directories, there should at least be a declaration for /usr/share/pulp/wsgi
conf.d/05-pulp-https.conf:  <Files "webservices.wsgi">
conf.d/05-pulp-https.conf:  <Directory "/usr/share/pulp/wsgi">
conf.d/05-pulp-https.conf:  WSGIApplicationGroup pulp
conf.d/05-pulp-https.conf:  WSGIDaemonProcess pulp user=apache group=apache processes=3 display-name=%{GROUP}
conf.d/05-pulp-https.conf:  WSGIImportScript /usr/share/pulp/wsgi/webservices.wsgi application-group=pulp process-group=pulp
conf.d/05-pulp-https.conf:  WSGIProcessGroup pulp
conf.d/05-pulp-https.conf:  WSGIScriptAlias /pulp/api "/usr/share/pulp/wsgi/webservices.wsgi"
conf.d/05-pulp-https.conf:  WSGIPassAuthorization On
conf.d/pulp.conf:WSGIProcessGroup pulp
conf.d/pulp.conf:WSGIApplicationGroup pulp
conf.d/pulp.conf:WSGIDaemonProcess pulp user=apache group=apache processes=3 display-name=%{GROUP}
conf.d/pulp.conf:#WSGIRestrictStdin Off
conf.d/pulp.conf:#WSGIRestrictStdout Off
conf.d/pulp.conf:WSGISocketPrefix run/wsgi
conf.d/pulp.conf:WSGIScriptAlias /pulp/api /usr/share/pulp/wsgi/webservices.wsgi
conf.d/pulp.conf:WSGIImportScript /usr/share/pulp/wsgi/webservices.wsgi process-group=pulp application-group=pulp
conf.d/pulp.conf:<Directory /usr/share/pulp/wsgi>
conf.d/pulp.conf:<Files webservices.wsgi>
conf.d/pulp.conf:    WSGIPassAuthorization On
conf.d/pulp.conf:    WSGIProcessGroup pulp
conf.d/pulp.conf:    WSGIApplicationGroup pulp
conf.modules.d/wsgi.load:LoadModule wsgi_module modules/mod_wsgi.so
conf.modules.d/wsgi.conf:# The WSGI Apache module configuration file is being
conf.modules.d/wsgi.conf:<IfModule mod_wsgi.c>
conf.modules.d/wsgi.conf:  WSGISocketPrefix /var/run/wsgi

# grep -Ri wsgidaemon * | egrep -v "^#|error_log"
conf.d/pulp_content.conf:WSGIDaemonProcess pulp-content user=apache group=apache processes=3 display-name=%{GROUP}
conf.d/pulp_puppet.conf:WSGIDaemonProcess pulp_forge user=apache group=apache processes=3 display-name=%{GROUP}
conf.d/05-pulp-https.conf:  WSGIDaemonProcess pulp user=apache group=apache processes=3 display-name=%{GROUP}
conf.d/pulp.conf:WSGIDaemonProcess pulp user=apache group=apache processes=3 display-name=%{GROUP}

Comment 5 Mike McCune 2016-08-22 20:11:49 UTC
WORKAROUND:

# mv /etc/httpd/conf.d/pulp.conf to /tmp
# service httpd restart

Comment 6 Julio Entrena Perez 2016-08-31 15:01:58 UTC
Contents of /etc/httpd/conf.d/ in a freshly installed 6.2.0 Capsule:

  # ls -l /etc/httpd/conf.d/
  total 56
  -rw-r--r--. 1 root root 1232 Aug 19 12:44 03-crane.conf
  -rw-r--r--. 1 root root  812 Aug 19 12:44 05-capsule.conf
  -rw-r--r--. 1 root root  772 Aug 19 12:44 05-pulp-http.conf
  -rw-r--r--. 1 root root 2127 Aug 19 12:44 05-pulp-https.conf
  -rw-r--r--. 1 root root  697 Aug 19 12:45 15-default.conf
  -rw-r--r--. 1 root root 1878 Aug 19 12:44 25-puppet.conf
  -rw-r--r--. 1 root root 2052 Aug 19 12:45 28-katello-reverse-proxy.conf
  -rw-r--r--. 1 root root  747 May 22 16:48 passenger.conf
  -rw-r--r--. 1 root root  747 Aug 19 12:44 pulp_content.conf
  -rw-r--r--. 1 root root  645 Jul 12 11:01 pulp_docker.conf
  -rw-r--r--. 1 root root  936 Aug 19 12:44 pulp_nodes.conf
  -rw-r--r--. 1 root root 1802 Jul 12 10:52 pulp_puppet.conf
  -rw-r--r--. 1 root root 1855 Aug 19 12:44 pulp_rpm.conf
  -rw-r--r--. 1 root root 2451 Aug 19 12:44 pulp_streamer.conf

Contents of /etc/httpd/conf.d/ after running "yum update" (notice new file "pulp.conf" with 755 permissions):

  # ls -l /etc/httpd/conf.d/
  total 60
  -rw-r--r--. 1 root root 1232 Aug 19 12:44 03-crane.conf
  -rw-r--r--. 1 root root  812 Aug 19 12:44 05-capsule.conf
  -rw-r--r--. 1 root root  772 Aug 19 12:44 05-pulp-http.conf
  -rw-r--r--. 1 root root 2127 Aug 19 12:44 05-pulp-https.conf
  -rw-r--r--. 1 root root  697 Aug 19 12:45 15-default.conf
  -rw-r--r--. 1 root root 1878 Aug 19 12:44 25-puppet.conf
  -rw-r--r--. 1 root root 2052 Aug 19 12:45 28-katello-reverse-proxy.conf
  -rw-r--r--. 1 root root  747 May 22 16:48 passenger.conf
  -rwxr-xr-x. 1 root root 3665 Aug  2 21:45 pulp.conf                      <---
  -rw-r--r--. 1 root root  747 Aug 19 12:44 pulp_content.conf
  -rw-r--r--. 1 root root  645 Jul 12 11:01 pulp_docker.conf
  -rw-r--r--. 1 root root  936 Aug 19 12:44 pulp_nodes.conf
  -rw-r--r--. 1 root root 1802 Jul 12 10:52 pulp_puppet.conf
  -rw-r--r--. 1 root root 1855 Aug 19 12:44 pulp_rpm.conf
  -rw-r--r--. 1 root root 2451 Aug 19 12:44 pulp_streamer.conf

  # yum provides /etc/httpd/conf.d/pulp.conf 
  [...]
  pulp-server-2.8.3.4-1.el7sat.noarch : The pulp platform server
  Repo        : rhel-7-server-satellite-capsule-6.2-rpms
  Matched from:
  Filename    : /etc/httpd/conf.d/pulp.conf

Running "satellite-installer --scenario capsule --upgrade" does not remove the new file "pulp.conf" nor the conflicting file "05-pulp-https.conf":

  # satellite-installer --scenario capsule --upgrade
  [...]
  Upgrade Step: start_httpd...
  Redirecting to /bin/systemctl start  httpd.service
  Job for httpd.service failed because the control process exited with error code. See "systemctl status httpd.service" and "journalctl -xe" for details.
  
  Some services failed to start: httpd
  
  Upgrade step start_httpd failed. Check logs for more information.

  # journalctl -u httpd
  Aug 31 10:54:35 sat62cap2.usersys.redhat.com systemd[1]: Starting The Apache HTTP Server...
  Aug 31 10:54:35 sat62cap2.usersys.redhat.com httpd[3717]: AH00526: Syntax error on line 28 of /etc/httpd/conf.d/pulp.conf:
  Aug 31 10:54:35 sat62cap2.usersys.redhat.com httpd[3717]: Name duplicates previous WSGI daemon definition.
  Aug 31 10:54:35 sat62cap2.usersys.redhat.com systemd[1]: httpd.service: main process exited, code=exited, status=1/FAILURE
  Aug 31 10:54:35 sat62cap2.usersys.redhat.com kill[3719]: kill: cannot find process ""
  Aug 31 10:54:35 sat62cap2.usersys.redhat.com systemd[1]: httpd.service: control process exited, code=exited status=1
  Aug 31 10:54:35 sat62cap2.usersys.redhat.com systemd[1]: Failed to start The Apache HTTP Server.
  Aug 31 10:54:35 sat62cap2.usersys.redhat.com systemd[1]: Unit httpd.service entered failed state.
  Aug 31 10:54:35 sat62cap2.usersys.redhat.com systemd[1]: httpd.service failed.

  # grep "WSGIDaemonProcess pulp " *
  05-pulp-https.conf:  WSGIDaemonProcess pulp user=apache group=apache processes=3 display-name=%{GROUP}
  pulp.conf:WSGIDaemonProcess pulp user=apache group=apache processes=3 display-name=%{GROUP}

Comment 7 Julio Entrena Perez 2016-08-31 15:13:01 UTC
Deleting file /etc/httpd/conf.d/pulp.conf and re-running the installer results in the installed completing successfully.

Comment 8 Paul Armstrong 2016-09-22 01:03:15 UTC
This issue is also encountered in upgrading to a capsule from 6.2.1 -> 6.2.2 
I renamed the offending pulp.conf file and re-ran the upgrade without issue.

yum provides /etc/httpd/conf.d/pulp.conf
Loaded plugins: package_upload, product-id, search-disabled-repos, subscription-manager
pulp-server-2.8.3.3-1.el7sat.noarch : The pulp platform server
Repo        : rhel-7-server-satellite-capsule-6.2-rpms
Matched from:
Filename    : /etc/httpd/conf.d/pulp.conf



pulp-server-2.8.3.4-1.el7sat.noarch : The pulp platform server
Repo        : rhel-7-server-satellite-capsule-6.2-rpms
Matched from:
Filename    : /etc/httpd/conf.d/pulp.conf



pulp-server-2.8.3.4-1.el7sat.noarch : The pulp platform server
Repo        : @rhel-7-server-satellite-capsule-6.2-rpms
Matched from:
Filename    : /etc/httpd/conf.d/pulp.conf

Comment 9 Paul Armstrong 2016-09-22 01:08:04 UTC
Really interesting thing...
After the upgrade the file is gone... deleted. even though I renamed it...

[root@capsule conf.d]# mv pulp.conf pulp.conf.error
[root@capsule conf.d]# ls
03-crane.conf    05-pulp-http.conf   15-default.conf  28-katello-reverse-proxy.conf  pulp.conf.error    pulp_docker.conf  pulp_puppet.conf  pulp_streamer.conf
05-capsule.conf  05-pulp-https.conf  25-puppet.conf   passenger.conf                 pulp_content.conf  pulp_nodes.conf   pulp_rpm.conf
[root@capsule conf.d]# satellite-installer --scenario capsule --upgrade
Upgrading...

...
...
after upgrade success....
...

[root@capsule conf.d]# ls
03-crane.conf    05-pulp-http.conf   15-default.conf  28-katello-reverse-proxy.conf  pulp_content.conf  pulp_nodes.conf   pulp_rpm.conf
05-capsule.conf  05-pulp-https.conf  25-puppet.conf   passenger.conf                 pulp_docker.conf   pulp_puppet.conf  pulp_streamer.conf
[root@capsule conf.d]#

Comment 10 Evgeni Golov 2016-09-22 06:11:54 UTC
(In reply to Paul Armstrong from comment #9)
> Really interesting thing...
> After the upgrade the file is gone... deleted. even though I renamed it...

I think this is because Puppet will wipe all unknown files from /etc/httpd/conf.d/ during install.
Same issue as in https://bugzilla.redhat.com/show_bug.cgi?id=1336365

Comment 12 Geoff Gatward 2016-11-01 03:38:30 UTC
+1

Capsule upgrade 6.2.2 -> 6.2.3 failed due to this issue.
Removed /etc/httpd/conf.d/pulp.conf and re-ran satellite-installer and it completed successfully.

Comment 19 Michael Hrivnak 2016-11-14 21:20:48 UTC
I don't think this is a pulp bug. Pulp provides default apache config files, but katello manages the apache config from there. Pulp hasn't made any notable changes to its apache config recently.

I suspect this is a bug in the installer or in the config management for katello.

Brad, I'm hoping you can find the right person to look at it.

Comment 20 Brad Buckingham 2016-11-17 13:47:39 UTC
Hi Stephen, the change needed for this bugzilla is likely within the installer.  Can you take a quick look and if so prioritize it for resolution?  Thanks!

Comment 21 Stephen Benjamin 2016-11-17 15:02:39 UTC
I'm not unaware of any such change.  Do you have a PR or a reason to think it's upstream?

RPM's are restoring files in an upgrade of the package, even though the user explicitly deleted them.  I think that's a bad behavior. It should be possible in the RPM to test that if in an upgrade, you're not putting back a config file a user deleted previously, or at least only creating it as .rpmnew.  It doesn't seem to be a built in %config option but it should be possible to come up with some ugly logic in the spec to do it.

This is similar to the mod_ssl problem we encountered.

Expecting users only ever configure "Listen 443" in ssl.conf really messed us up because puppetlabs-apache puts it ports.conf, so on upgrade you end up with duplicate Listen directives.

That seems to be what's happening here now with pulp.conf, duplicate declarations because the RPM added something back we removed.

I suggested a fix to puppetlabs-apache to make their file named ssl.conf, https://github.com/puppetlabs/puppetlabs-apache/pull/1543.

We could do the same by making where we're putting the pulp config called "pulp.conf" so the RPM doesn't overwrite it, but I'm having a hard time parsing the output of comment #2, what file are we creating in the installer that has the duplicate declaration of WSGI?

Comment 22 Stephen Benjamin 2016-11-21 14:54:21 UTC
Created redmine issue http://projects.theforeman.org/issues/17415 from this bug

Comment 23 Bryan Kearney 2016-11-21 15:09:02 UTC
Upstream bug assigned to stbenjam

Comment 24 Bryan Kearney 2016-11-21 15:09:10 UTC
Upstream bug assigned to stbenjam

Comment 25 Bryan Kearney 2016-12-08 21:09:01 UTC
Moving this bug to POST for triage into Satellite 6 since the upstream issue http://projects.theforeman.org/issues/17415 has been resolved.

Comment 27 Sanket Jagtap 2017-01-13 12:51:37 UTC
Build : Satellite 6.2.7 snap 1

Steps:
1) Installed Satellite 6.2.6 from cdn
2) Installed Capsule from cdn
3) Update satellite repos , upgraded satellite successfully
4) Updated the capsule repo to 6.2.7 ,

Contents of /etc/httpd/conf.d before capsule upgrade
[root@sgi-uv300-02 conf.d]# ll
total 64
-rw-r--r--. 1 root root 1240 Jan 13 02:38 03-crane.conf
-rw-r--r--. 1 root root  820 Jan 13 02:38 05-capsule.conf
-rw-r--r--. 1 root root  783 Jan 13 02:38 05-pulp-http.conf
-rw-r--r--. 1 root root 2148 Jan 13 02:38 05-pulp-https.conf
-rw-r--r--. 1 root root  697 Jan 13 02:38 15-default.conf
-rw-r--r--. 1 root root 1894 Jan 13 02:38 25-puppet.conf
-rw-r--r--. 1 root root 2078 Jan 13 02:38 28-katello-reverse-proxy.conf
-rw-r--r--. 1 root root  747 May 22  2016 passenger.conf
-rwxr-xr-x. 1 root root 3665 Dec 22 10:17 pulp.conf
-rw-r--r--. 1 root root  747 Jan 13 02:38 pulp_content.conf
-rw-r--r--. 1 root root  645 Oct  4 11:19 pulp_docker.conf
-rw-r--r--. 1 root root  936 Jan 13 02:38 pulp_nodes.conf
-rw-r--r--. 1 root root 1802 Oct  4 12:12 pulp_puppet.conf
-rw-r--r--. 1 root root 1855 Jan 13 02:38 pulp_rpm.conf
-rw-r--r--. 1 root root 2451 Jan 13 02:38 pulp_streamer.conf
-rw-r--r--. 1 root root   62 Jan 13 02:38 ssl.conf

5) Upgraded Capsule successfully , 
Upgrade Step: Running installer...
Installing             Done                                               [100%] [...........................................................................................................]
  The full log is at /var/log/foreman-installer/capsule.log
Success!

Upgrade completed!

Contents of /etc/httpd/conf.d after upgrade
ll
total 64
-rw-r--r--. 1 root root 1240 Jan 13 02:38 03-crane.conf
-rw-r--r--. 1 root root  820 Jan 13 02:38 05-capsule.conf
-rw-r--r--. 1 root root  783 Jan 13 02:38 05-pulp-http.conf
-rw-r--r--. 1 root root 2148 Jan 13 02:38 05-pulp-https.conf
-rw-r--r--. 1 root root  697 Jan 13 02:38 15-default.conf
-rw-r--r--. 1 root root 1894 Jan 13 02:38 25-puppet.conf
-rw-r--r--. 1 root root 2078 Jan 13 02:38 28-katello-reverse-proxy.conf
-rw-r--r--. 1 root root  747 May 22  2016 passenger.conf
-rw-r--r--. 1 root root   62 Jan 13 07:40 pulp.conf
-rw-r--r--. 1 root root  747 Jan 13 02:38 pulp_content.conf
-rw-r--r--. 1 root root  645 Oct  4 11:19 pulp_docker.conf
-rw-r--r--. 1 root root  936 Jan 13 02:38 pulp_nodes.conf
-rw-r--r--. 1 root root 1802 Oct  4 12:12 pulp_puppet.conf
-rw-r--r--. 1 root root 1855 Jan 13 02:38 pulp_rpm.conf
-rw-r--r--. 1 root root 2451 Jan 13 02:38 pulp_streamer.conf
-rw-r--r--. 1 root root   62 Jan 13 07:40 ssl.conf

Please confirm the verification steps used are correct.

Comment 28 sthirugn@redhat.com 2017-01-13 13:26:27 UTC
I believe I hit this when testing a capsule upgrade from version 6.1.latest to 6.2.latest.  Can you please test this once? Thanks!

Comment 29 sthirugn@redhat.com 2017-01-13 13:31:11 UTC
(In reply to sthirugn from comment #28)
> I believe I hit this when testing a capsule upgrade from version 6.1.latest
> to 6.2.latest.  Can you please test this once? Thanks!

Disregard my comment, testing from capsule version 6.2.0 to latest should be good.

Comment 30 Stephen Benjamin 2017-01-13 13:52:08 UTC
That should be fine, thanks

Comment 31 Sachin Ghai 2017-01-17 10:23:04 UTC
I got similar issue while upgrading capsule 6.2.6 -> capsule 6.2.7 on rhel6



Upgrade Step: start_httpd...
[Tue Jan 17 05:05:11 2017] [warn] module passenger_module is already loaded, skipping
Syntax error on line 28 of /etc/httpd/conf.d/pulp.conf:
Name duplicates previous WSGI daemon definition.
Starting httpd:                                            [FAILED]
Some services failed to start: httpd

Upgrade step start_httpd failed. Check logs for more information.
[root@qe-capsule-upgrade-rhel6 conf.d]# rpm -qa | grep satellite-capsule
satellite-capsule-6.2.7-1.0.el6sat.noarch
[root@qe-capsule-upgrade-rhel6 conf.d]#

Comment 33 Julio Entrena Perez 2017-01-17 10:30:34 UTC
(In reply to Sachin Ghai from comment #31)
> I got similar issue while upgrading capsule 6.2.6 -> capsule 6.2.7 on rhel6

(In reply to Sanket Jagtap from comment #27)
> Contents of /etc/httpd/conf.d after upgrade
> ll
> total 64
> -rw-r--r--. 1 root root 1240 Jan 13 02:38 03-crane.conf
> -rw-r--r--. 1 root root  820 Jan 13 02:38 05-capsule.conf
> -rw-r--r--. 1 root root  783 Jan 13 02:38 05-pulp-http.conf
> -rw-r--r--. 1 root root 2148 Jan 13 02:38 05-pulp-https.conf
> -rw-r--r--. 1 root root  697 Jan 13 02:38 15-default.conf
> -rw-r--r--. 1 root root 1894 Jan 13 02:38 25-puppet.conf
> -rw-r--r--. 1 root root 2078 Jan 13 02:38 28-katello-reverse-proxy.conf
> -rw-r--r--. 1 root root  747 May 22  2016 passenger.conf
> -rw-r--r--. 1 root root   62 Jan 13 07:40 pulp.conf
> -rw-r--r--. 1 root root  747 Jan 13 02:38 pulp_content.conf
> -rw-r--r--. 1 root root  645 Oct  4 11:19 pulp_docker.conf
> -rw-r--r--. 1 root root  936 Jan 13 02:38 pulp_nodes.conf
> -rw-r--r--. 1 root root 1802 Oct  4 12:12 pulp_puppet.conf
> -rw-r--r--. 1 root root 1855 Jan 13 02:38 pulp_rpm.conf
> -rw-r--r--. 1 root root 2451 Jan 13 02:38 pulp_streamer.conf
> -rw-r--r--. 1 root root   62 Jan 13 07:40 ssl.conf
> 
> Please confirm the verification steps used are correct.

(In reply to Julio Entrena Perez from comment #2)
> # grep -Ri wsgidaemon * | egrep -v "^#|error_log"
[...]
> conf.d/05-pulp-https.conf:  WSGIDaemonProcess pulp user=apache group=apache processes=3 display-name=%{GROUP}
> conf.d/pulp.conf:WSGIDaemonProcess pulp user=apache group=apache processes=3 display-name=%{GROUP}

I'd expect that if files 05-pulp-https.conf and pulp.conf still both contain "WSGIDaemonProcess pulp" then it'll fail.

Comment 34 Sachin Ghai 2017-01-17 11:34:26 UTC
from upgraded sat6.2.7 rhel6 capsule:

]# grep -Ri wsgidaemon * | egrep -v "^#|error_log"
05-pulp-https.conf:  WSGIDaemonProcess pulp user=apache group=apache processes=3 display-name=%{GROUP}
pulp.conf:WSGIDaemonProcess pulp user=apache group=apache processes=3 display-name=%{GROUP}
pulp_content.conf:WSGIDaemonProcess pulp-content user=apache group=apache processes=3 display-name=%{GROUP}
pulp_puppet.conf:WSGIDaemonProcess pulp_forge user=apache group=apache processes=3 display-name=%{GROUP}

Comment 35 Satellite Program 2017-01-17 13:09:14 UTC
Upstream bug assigned to jsherril

Comment 36 Sachin Ghai 2017-01-17 15:44:27 UTC
I was able to reproduce this bug with capsule upgrade from 6.1.11 to 6.2.7 on rhel6 too.


Upgrade Step: start_httpd...
[Tue Jan 17 10:27:27 2017] [warn] module passenger_module is already loaded, skipping
Syntax error on line 28 of /etc/httpd/conf.d/pulp.conf:
Name duplicates previous WSGI daemon definition.
Starting httpd:                                            [FAILED]
Some services failed to start: httpd

Upgrade step start_httpd failed. Check logs for more information.

Comment 37 Justin Sherrill 2017-01-17 15:48:19 UTC
Reassigning to stephen, the original author of this.

Comment 38 Bryan Kearney 2017-01-18 12:55:59 UTC
Per Stephen, moving this back to ON_QA since snap 2 was delivered.

Comment 39 Sachin Ghai 2017-01-23 12:57:14 UTC
Verified this bz with sat6.2.7 snap2.

Capsule upgrade on rhel6 and rhel7 went fine. I don't see the issue pointed in comment 34 and 35. Please see the capsule upgrade logs in attachment

Comment 40 Sachin Ghai 2017-01-23 12:58:01 UTC
Created attachment 1243581 [details]
capsule upgrade logs from console

Comment 41 Bryan Kearney 2017-01-26 15:56:01 UTC
This was delivered in satellite 6.2.7 (https://access.redhat.com/errata/RHBA-2017:0197)

Comment 42 Oleksandr Shtaier 2017-08-16 08:28:24 UTC
Existing automation already handling this