Bug 1639676
| Summary: | Unable to persistently set redirect_host for lazy sync to empty value | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | Red Hat Satellite | Reporter: | Pavel Moravec <pmoravec> | ||||||||
| Component: | Pulp | Assignee: | Evgeni Golov <egolov> | ||||||||
| Status: | CLOSED ERRATA | QA Contact: | jcallaha | ||||||||
| Severity: | medium | Docs Contact: | |||||||||
| Priority: | medium | ||||||||||
| Version: | 6.3.4 | CC: | bmbouter, daviddavis, dkliban, egolov, ggainey, ipanova, jcallaha, pcreech, rchan, ttereshc | ||||||||
| Target Milestone: | 6.5.0 | Keywords: | Reopened, Triaged | ||||||||
| Target Release: | Unused | ||||||||||
| Hardware: | x86_64 | ||||||||||
| OS: | Linux | ||||||||||
| Whiteboard: | |||||||||||
| Fixed In Version: | katello-installer-base-3.9.0-0, pulp-2.18.0-0 | Doc Type: | If docs needed, set a value | ||||||||
| Doc Text: | Story Points: | --- | |||||||||
| Clone Of: | Environment: | ||||||||||
| Last Closed: | 2019-05-14 12:38:12 UTC | Type: | Bug | ||||||||
| Regression: | --- | Mount Type: | --- | ||||||||
| Documentation: | --- | CRM: | |||||||||
| Verified Versions: | Category: | --- | |||||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||||
| Embargoed: | |||||||||||
| Attachments: |
|
||||||||||
ewoud++ for the solution: line pulp::lazy_redirect_host: ' ' in /etc/foreman-installer/custom-hiera.yaml is the trick to have in /etc/pulp/server.conf : redirect_host: <empty-spaces-here> that sets the required setting. (In reply to Pavel Moravec from comment #1) > ewoud++ for the solution: > > line > > pulp::lazy_redirect_host: ' ' > > in /etc/foreman-installer/custom-hiera.yaml is the trick to have in > /etc/pulp/server.conf : > > redirect_host: <empty-spaces-here> > > that sets the required setting. So it is just configuration issue and not a bug. Described in KCS 3655581. Upstream bug assigned to egolov Upstream bug assigned to egolov The Pulp upstream bug status is at POST. Updating the external tracker on this bug. The Pulp upstream bug priority is at Normal. Updating the external tracker on this bug. The Pulp upstream bug status is at MODIFIED. Updating the external tracker on this bug. Moving this bug to POST for triage into Satellite 6 since the upstream issue https://projects.theforeman.org/issues/25266 has been resolved. The Pulp upstream bug status is at ON_QA. Updating the external tracker on this bug. The Pulp upstream bug status is at CLOSED - CURRENTRELEASE. Updating the external tracker on this bug. Failed QA in Satellite 6.5.0 Snap 11 Modified the config, performed an upgrade, then checked the config again. The value was stripped. -bash-4.2# grep redirect_host /etc/pulp/server.conf # redirect_host: redirect_host: my.test.host -bash-4.2# -bash-4.2# satellite-installer --upgrade Resetting puppet server version param... Upgrading, to monitor the progress on all related services, please do: foreman-tail | tee upgrade-$(date +%Y-%m-%d-%H%M).log Upgrade Step: stop_services... Redirecting to 'foreman-maintain service' Running Stop Services ================================================================================ ... Upgrade Step 7/7: katello:upgrades:3.11:update_puppet_repos. foreman-rake upgrade:run finished successfully! Upgrade completed! -bash-4.2# grep redirect_host /etc/pulp/server.conf # redirect_host: Requesting needsinfo from upstream developer dkliban, ttereshc, daviddavis because the 'FailedQA' flag is set. I think this is a problem in the installer. Setting NEEDINFO to egolov. Requesting needsinfo from upstream developer dkliban, ttereshc, daviddavis because the 'FailedQA' flag is set. Jake, I think the initial BZ description lead you to a wrong verification path ;) In Pulp < 2.18, we had to explicitly set "redirect_host" to an empty string to make it redirect using the HTTP Host header value (instead of the value of "redirect_host"). With Pulp 2.18, the redirect_host setting behaves "correctly" and does not need to be set to an empty string to trigger the desired behavior. See Pulp changes https://pulp.plan.io/issues/4092 and https://pulp.plan.io/issues/4120 After these were in, we updated the installer in https://projects.theforeman.org/issues/25266. So the correct verification steps would be: 1/ setup a on_demand repo 2/ try to access a (not yet downloaded) file in that repo with a hostname that points at the satellite, but is not the satellites FQDN (using /etc/hosts or something, using HTTP instead of HTTPS would make things easier, as you don't have to care for proper certs in that case 3/ see pulp redirect to the streamer at http://the-name-you-used.example.com/pulp/streamer/… and not http://satellite.example.com/pulp/streamer/… Requesting needsinfo from upstream developer dkliban, ttereshc, daviddavis because the 'FailedQA' flag is set. Created attachment 1536645 [details]
script to create a test repository for this BZ
Created attachment 1536646 [details]
test output from Satellite 6.3 (so before this fix)
Created attachment 1536647 [details]
test output from Satellite 6.5 (with this fix)
To test this, you need a recent Katello and Pulp, e.g. as in the Satellite 6.5 snaps we ship. This BZ has a script attached, that will create and sync a test-repository with on_demand policy: # bash create-test-repo.sh Organization created. Product created. Repository created. New packages: 32 (76.7 KB). After the repository is synced, you can try to access its content using curl: curl -I -X GET --location http://<SOME_NAME>/pulp/repos/Test_Organization/Library/custom/Test_Product/Zoo/Packages/b/bear-4.1-1.noarch.rpm (we only want to see headers, so -I -X GET. we need curl to follow redirects, so --location) An example output for Satellite 6.3 (broken) and Satellite 6.5 (working) is attached. On Satellite 6.5 you can substitute anything for <SOME_NAME> that will reach your apache. I've used localhost and $HOSTNAME. In both cases you see a "Location: http://<SOME_NAME>:80/streamer/…" header, followed by a successful RPM download. On Satellite 6.3 (and 6.4, but I don't have one handy), you'd see that the Location header *always* includes the $HOSTNAME of the machine, not what you've supplied in the original request. Note: if you want to test this with protected (Red Hat) repositories via HTTPS, you'll have to tell curl to use the entitlement certificates of your machine in the request, as otherwise Pulp will not allow you to access the repositories. Note2: Pulp will move the streamed RPMs to their proper location at some point, so you might not see the redirect again if you retry the same RPM at a later point. Just pick another one in that case (a listing can be found at https://jlsherrill.fedorapeople.org/fake-repos/needed-errata/) Verified in Satellite 6.5.0 Snap 15 Followed the verification steps outlined in #17 and #22 First, setup the fake redirect host (note that you only want to have the domain name -bash-4.2# grep "my.sat.host" /etc/pulp/server.conf redirect_host: my.sat.host Second, I added an entry to my client's hosts file for the fake domain, linking back to my satellite's IP After that, I downloaded a package. [root@testhost2 ~]# curl -I -X GET --location http://<my actual satellite>/pulp/repos/Default_Organization/Library/custom/custom/fake/Packages/a/Antelope-10.6.9-1.elfake.noarch.rpm HTTP/1.1 302 Found Date: Wed, 20 Feb 2019 15:34:13 GMT Server: Apache/2.4.6 (Red Hat Enterprise Linux) Content-Length: 0 ETag: "d41d8cd98f00b204e9800998ecf8427e" Location: http://my.sat.host:80/streamer/var/lib/pulp/content/units/rpm/60/bcb129a6ca5274adfd35721c1116069594579c45e86616f4c58ed1ad01fcd1/Antelope-10.6.9-1.elfake.noarch.rpm?policy=eyJleHRlbnNpb25zIjogeyJyZW1vdGVfaXAiOiAiMTAuMTMuMTI5LjEyNyJ9LCAicmVzb3VyY2UiOiAiL3N0cmVhbWVyL3Zhci9saWIvcHVscC9jb250ZW50L3VuaXRzL3JwbS82MC9iY2IxMjlhNmNhNTI3NGFkZmQzNTcyMWMxMTE2MDY5NTk0NTc5YzQ1ZTg2NjE2ZjRjNThlZDFhZDAxZmNkMS9BbnRlbG9wZS0xMC42LjktMS5lbGZha2Uubm9hcmNoLnJwbSIsICJleHBpcmF0aW9uIjogMTU1MDY3Njk0M30%3D;signature=ehgBDmxhTPzCcRGhmykK149x3Xq0R8EUAa9R-8gbGXe34JCVYjIWewYwfN4FgN6iEc40P9KtoQZVrLwUs872gS9ZzyU_1TIYXjYNO1B2OUEZGeZT59CPjResY8vxrbTeFm_-reXjGA3_LaP0unhA14XyczmAOxbzkbfPPb1aJXvoE7sK7dWuv2I-6T7Y1_9Sw9EZ1qCnPNqYzK1ehSA5ggWyN9_7LouH1RwDRpPXd5mjW1h6LO-DJW4dpmM-ng7yl2mocVj85gF1BdXCX3zEe5-48lhInlUL16Ckn9CtOlZ8kccEyPAfq77JRMfwDsX4v3WKYayLGo4We0S7RbFohQ%3D%3D Content-Type: text/html; charset=utf-8 HTTP/1.1 200 OK Date: Wed, 20 Feb 2019 15:34:14 GMT Server: Apache/2.4.6 (Red Hat Enterprise Linux) OpenSSL/1.0.2k-fips Apptime: D=5309 Content-Length: 13032 Strict-Transport-Security: max-age=31536000; includeSubDomains; preload X-Gitproject: (null) Last-Modified: Sat, 08 Sep 2012 20:54:47 GMT Accept-Ranges: bytes Expires: Wed, 20 Feb 2019 16:04:14 GMT Appserver: people02.fedoraproject.org ETag: "32e8-4c936ef4c13c0" Cache-Control: public, s-maxage=86400, max-age=86400 Content-Type: application/x-rpm X-Cache: MISS from <my actual satellite> X-Cache-Lookup: MISS from <my actual satellite>:3128 Via: 1.1 <my actual satellite> (squid/3.5.20) Via: 1.1 <my actual satellite> The redirect completed successfully Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2019:1222 |
Description of problem: Assume a use case where multi-homed Satellite or Capsule needs to be accessed from clients via _different_ alternative hostnames, like caps-altname1 and caps-altname2 and neither is primary FQDN of the Capsule. Then there is no way to permanently configure lazy sync. (I expect the "multi-home clients" use case is just one of few similar ones here, like HA capsules + multihomed Caps is another use case) The cause is, httpd responds with redirect URL (to contact squid) with hostname specified in /etc/pulp/server.conf, [lazy] section, redirect_host property. This property is basically required to have multiple values, or from the code, it must be empty (such that hostname from the request is used directly). BUT redirect_host has hardcoded default value: /usr/lib/python2.7/site-packages/pulp/server/config.py : 'lazy': { 'redirect_host': socket.getfqdn(), So the workaround is to overwrite it to empty value in /etc/pulp/server.conf : [lazy] redirect_host: BUT that will be overwritten by satellite-installer. Making the change permanent via pulp::lazy_redirect_host: '' in /etc/foreman-installer/custom-hiera.yaml is _not_ a solution, since it purges away the "redirect_host: " line completely, hence pulp fallbacks to default primary hostname again. So I dont see a way how to make the config change permanent - either: - pulp default value for redirect_host must be empty - or lazy_redirect_host with empty value in custom hiera must set empty redirect_host in /etc/pulp/server.conf Version-Release number of selected component (if applicable): Sat 6.3.4 How reproducible: 100% Steps to Reproduce: 1. Try to configure Capsule to be accessed from 2 clients, each accessing the Capsule via different alternate hostname each. 2. run satellite-installer (to verify the config is persistent) 3. Try to fetch a package from lazy sync repo Actual results: 3. fails for either or both clients Expected results: 3. works for both clients concurrently Additional info: kudos for egolotov and ewound for some background info here