Bug 1877866
| Summary: | [Assisted-4.6 ] Cluster installation fails with "*" wildcard in "No Proxy" field | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | nshidlin <nshidlin> | ||||||||
| Component: | assisted-installer | Assignee: | vemporop | ||||||||
| assisted-installer sub component: | assisted-service | QA Contact: | Yuri Obshansky <yobshans> | ||||||||
| Status: | CLOSED DEFERRED | Docs Contact: | |||||||||
| Severity: | low | ||||||||||
| Priority: | high | CC: | alazar, aos-bugs, lgamliel, masayag, mfilanov | ||||||||
| Version: | 4.6 | ||||||||||
| Target Milestone: | --- | ||||||||||
| Target Release: | 4.8.0 | ||||||||||
| Hardware: | Unspecified | ||||||||||
| OS: | Unspecified | ||||||||||
| Whiteboard: | AI-Team-Core | ||||||||||
| Fixed In Version: | OCP-Metal-V1.0.12.1 | Doc Type: | Known Issue | ||||||||
| Doc Text: |
Cause:
Several components do not accept an asterisk (*) as a valid no-proxy value.
Consequence:
A user cannot use '*' to bypass proxy for all destinations.
Workaround (if any):
Do not use a proxy at all, or specify in no-proxy all the IP addresses, hosts or domains to be excluded.
Result:
Trying to use '*' results in an error message in the assisted installer.
|
Story Points: | --- | ||||||||
| Clone Of: | Environment: | ||||||||||
| Last Closed: | 2021-05-26 08:51:54 UTC | Type: | Bug | ||||||||
| Regression: | --- | Mount Type: | --- | ||||||||
| Documentation: | --- | CRM: | |||||||||
| Verified Versions: | Category: | --- | |||||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||||
| Embargoed: | |||||||||||
| Bug Depends On: | 1877486, 1947066 | ||||||||||
| Bug Blocks: | |||||||||||
| Attachments: |
|
||||||||||
This bug is blocked by https://bugzilla.redhat.com/show_bug.cgi?id=1877486 - since 'install-config.yaml' doesn't support '*' as a valid value to bypass proxy, the assisted-service cannot use it as well. ealster this ticket should be assigned to slavie This issue has been fixed in OCP so now we can support this on the Assisted-Service. Failed Verification on staging:
{
"release_tag": "v1.0.12_1",
"versions": {
"assisted-ignition-generator": "",
"assisted-installer": "registry-proxy.engineering.redhat.com/rh-osbs/openshift4-assisted-installer-rhel8:v4.6.0-49",
"assisted-installer-controller": "registry-proxy.engineering.redhat.com/rh-osbs/openshift4-assisted-installer-reporter-rhel8:v4.6.0-46",
"assisted-installer-service": "quay.io/app-sre/assisted-service:92e3dad",
"discovery-agent": "registry-proxy.engineering.redhat.com/rh-osbs/openshift4-assisted-installer-agent-rhel8:v4.6.0-47",
"image-builder": "quay.io/app-sre/assisted-iso-create:92e3dad"
}
}
Steps to reproduce:
1. Set http proxy url to a non-existent proxy server
2. Set no proxy value to bypass all destinations "*"
3. Start cluster install
Result:
Cluster install fails with:
2/3 masters(non-bootstrap) installed
2/2 worker nodes failed on "This host failed its installation.
Host failed to install because its installation stage Waiting for ignition took longer than expected 1h0m0s"
Created attachment 1751603 [details]
cluster failed to start installation in v1.0.15.2 logs
Updated behavior:
{
"release_tag": "v1.0.15.2",
"versions": {
"assisted-installer": "registry-proxy.engineering.redhat.com/rh-osbs/openshift4-assisted-installer-rhel8:v4.6.0-82",
"assisted-installer-controller": "registry-proxy.engineering.redhat.com/rh-osbs/openshift4-assisted-installer-reporter-rhel8:v4.6.0-81",
"assisted-installer-service": "quay.io/app-sre/assisted-service:3b47299",
"discovery-agent": "registry-proxy.engineering.redhat.com/rh-osbs/openshift4-assisted-installer-agent-rhel8:v4.6.0-86"
}
}
Steps to reproduce:
1. Set http proxy url to a non-existent proxy server
2. Set no proxy value to bypass all destinations "*"
3. Start cluster install
Result:
Installation fails to start:
Failed generating kubeconfig files for cluster cf83e415-dbf3-4af1-9867-97c47f4603a4: exit status 1.
Reset the installation process to return to the configuration and try again. Some hosts may need to be re-registered by rebooting into the Discovery ISO.
logs are attached
The PR below improves the situation greatly, but the cluster is left with 1. the master that was the bootstrap doesn't come up after it reboots 2. the workers don't come up correctly https://github.com/openshift/assisted-service/pull/1035 I am working on metal3 stuff now, so unlikely to have time to work further on this. An asterisk '*' causes the openshift-install command to fail as below. This is because we will automatically append CIDRs and VIPs to a noProxy value, which results in something like '*,192.168.126.0/24,.test-infra-cluster-assisted-installer.redhat.com,10.128.0.0/14,172.30.0.0/16'. An asterisk in noProxy alone works fine. Created attachment 1768103 [details]
openshift-machine-config-operator.log
After I fixed the assisted service side, two masters of a cluster are installed successfully but the bootstrap fails to download ignition from the API VIP, because machine-config-operator crashes. Log attached.
Blocked by https://bugzilla.redhat.com/show_bug.cgi?id=1947066 (machine-config-operator pod crashes when noProxy is *) Until a fix for the blocking bug https://bugzilla.redhat.com/show_bug.cgi?id=1947066 is released, I've disabled '*' as a valid noProxy value in the assisted installer. So now a user gets a clear error message when trying to enter it, instead of the confusing installer failure. A fix for https://bugzilla.redhat.com/show_bug.cgi?id=1947066 is part of OCP 4.8.0-fc.4, but both fc.4 and fc.5 have been rejected because of issues. So we'll have to wait for the first good OCP version after fc.4. For now we have disabled this option. Will handle re-adding this in epic https://issues.redhat.com/browse/MGMT-6647 |
Created attachment 1714451 [details] screen shot of validation error Description of problem: The helper text for the field: "A comma-separated list of destination domain names, domains, IP addresses or other network CIDRs to exclude proxying. Preface a domain with . to include all subdomains of that domain. Use * to bypass proxy for all destinations." Entering this value in the UI yeilds the following error: "NO Proxy format is not valid: '*'. NO Proxy is a comma-separated list of destination domain names, domains, IP addresses or other network CIDRs. A domain can be prefaced with '.' to include all subdomains of that domain." Attempting to send this field through an API PATCH yeilds the following response: { "code": "400", "href": "", "id": 400, "kind": "Error", "reason": "NO Proxy format is not valid: '*'. NO Proxy is a comma-separated list of destination domain names, domains, IP addresses or other network CIDRs. A domain can be prefaced with '.' to include all subdomains of that domain." } Version-Release number of selected component (if applicable): assisted-installer component versions: { "versions": { "assisted-ignition-generator": "quay.io/ocpmetal/assisted-ignition-generator:v1.0.9.1", "assisted-installer": "quay.io/ocpmetal/assisted-installer:v1.0.9.1", "assisted-installer-controller": "quay.io/ocpmetal/assisted-installer-controller:v1.0.9.1", "assisted-installer-service": "quay.io/app-sre/assisted-service:f0172a8", "discovery-agent": "quay.io/ocpmetal/assisted-installer-agent:v1.0.9.1", "image-builder": "quay.io/app-sre/assisted-iso-create:f0172a8" } } How reproducible: Every time Steps to Reproduce: 1. Create a cluster 2. Click "Download Discovery ISO" and input HTTP Proxy URL and set "No Proxy" to "*" 3. Click "Generate HTTP Proxy" Actual results: "*" value is rejected Expected results: "*" to be a valid option Additional info: