Bug 836395
| Summary: | Apache 503 after running aeolus-upgrade - httpd restart required | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | [Retired] CloudForms Cloud Engine | Reporter: | James Laska <jlaska> | ||||||
| Component: | aeolus-conductor | Assignee: | Steve Linabery <slinaber> | ||||||
| Status: | CLOSED WONTFIX | QA Contact: | Rehana <aeolus-qa-list> | ||||||
| Severity: | medium | Docs Contact: | |||||||
| Priority: | unspecified | ||||||||
| Version: | 1.0.0 | CC: | cpelland, dmacpher, jeckersb, jturner, morazi, rlandy | ||||||
| Target Milestone: | 1.0.1 | ||||||||
| Target Release: | --- | ||||||||
| Hardware: | Unspecified | ||||||||
| OS: | Unspecified | ||||||||
| Whiteboard: | |||||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||||
| Doc Text: |
Conductor traffic passes through httpd+mod_proxy before reaching conductor itself (conductor runs via thin). During upgrade (or running aeolus-restart-services), conductor is restarted. The conductor init script will return successfully before thin has fully loaded the conductor app. Accessing conductor during this brief window will result in an HTTP/503 status code (Temporarily Unavailable). The default behavior of httpd/mod_proxy is to mark a backend worker as disabled for 60 seconds if the backend is determined to be unavailable; once the worker is marked as such, mod_proxy will not attempt to pass any traffic to the backend until the retry period expires. Therefor, if a user attempt to access conductor immediately after it starts, but before it has loaded completely, the user will receive an HTTP/503 error for approximately 60 seconds until the mod_proxy retry interval expires and httpd starts forwarding requests to conductor again.
|
Story Points: | --- | ||||||
| Clone Of: | Environment: | ||||||||
| Last Closed: | 2012-08-08 19:45:14 UTC | Type: | Bug | ||||||
| Regression: | --- | Mount Type: | --- | ||||||
| Documentation: | --- | CRM: | |||||||
| Verified Versions: | Category: | --- | |||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||
| Embargoed: | |||||||||
| Attachments: |
|
||||||||
|
Description
James Laska
2012-06-28 22:20:38 UTC
Created attachment 595153 [details]
aeolus-debug.tgz
Followed through the above reproduction steps. I do get the 'website unavailable' message and then the Apache 503 error if I try access conductor right after running the update script. Refreshing the browser does not help right away. But, after about a minute, refresh returned the conductor interface (according to eck, this is expected - copying chat comment ... <eck> yeah it's 60 seconds - http://httpd.apache.org/docs/2.2/mod/mod_proxy.html#proxypass ) Running >> aeolus-restart-services and then accessing conductor right away shows the same behaviour: unavailable -> 503 -> conductor shows up rpms tested: >> rpm -qa |grep aeolus aeolus-conductor-0.8.33-1.el6cf.noarch rubygem-aeolus-image-0.3.0-12.el6.noarch aeolus-conductor-daemons-0.8.33-1.el6cf.noarch aeolus-conductor-doc-0.8.33-1.el6cf.noarch rubygem-aeolus-cli-0.3.3-2.el6_2.noarch aeolus-configure-2.5.10-1.el6cf.noarch aeolus-all-0.8.33-1.el6cf.noarch
Technical note added. If any revisions are required, please edit the "Technical Notes" field
accordingly. All revisions will be proofread by the Engineering Content Services team.
New Contents:
Conductor traffic passes through httpd+mod_proxy before reaching conductor itself (conductor runs via thin). During upgrade (or running aeolus-restart-services), conductor is restarted. The conductor init script will return successfully before thin has fully loaded the conductor app. Accessing conductor during this brief window will result in an HTTP/503 status code (Temporarily Unavailable). The default behavior of httpd/mod_proxy is to mark a backend worker as disabled for 60 seconds if the backend is determined to be unavailable; once the worker is marked as such, mod_proxy will not attempt to pass any traffic to the backend until the retry period expires. Therefor, if a user attempt to access conductor immediately after it starts, but before it has loaded completely, the user will receive an HTTP/503 error for approximately 60 seconds until the mod_proxy retry interval expires and httpd starts forwarding requests to conductor again.
Should this be closed as the release notes were added or are we trying to track something additional here? (In reply to comment #5) > Should this be closed as the release notes were added or are we trying to > track something additional here? Is there any interest in eliminating, or reducing, the 60 window where conductor may return a HTTP/503? If not, I believe the release note that Dan added documents this behavior. With that in place, we can CLOSED NOTABUG. Since the backend downtime is unpredictable (less than 60, but how much less?) my vote is CLOSED NOTABUG on this. By the power vested in me ... I close this bug and cast it out! I changed my mind and went with CLOSED WONTFIX. I'm waffling between that an NOTABUG. Either way ... we aren't going to fix it. |