Description of problem: QE has separate Assisted Installer running on PSI http://assisted-installer-ui.apps.ocp.prod.psi.redhat.com/clusters Node discovering failed because agent starts with wrong host. It points to Dev Assisted Installer Beta on PSI. [core@master-0-0 log]$ systemctl status agent ● agent.service Loaded: loaded (/etc/systemd/system/agent.service; enabled; vendor preset: enabled) Active: active (running) since Thu 2020-07-30 19:08:39 UTC; 12min ago Process: 1484 ExecStartPre=/usr/bin/podman run --privileged --rm -v /usr/local/bin:/hostbin quay.io/ocpmetal/agent:latest cp /usr/bin/agent /ho> Main PID: 1961 (agent) Tasks: 13 (limit: 204943) Memory: 311.9M CGroup: /system.slice/agent.service └─1961 /usr/local/bin/agent --host assisted-installer.apps.ocp.prod.psi.redhat.com --port 80 --cluster-id e860dc6f-3b8c-47d4-aa0b-3711> It should point to assisted-installer-ui.apps.ocp.prod.psi.redhat.com/clusters instead of assisted-installer.apps.ocp.prod.psi.redhat.com As result agent throws error: Error registering host: [POST /clusters/{cluster_id}/hosts][400] registerHostBadRequest &{Code:0xc000504150 Href:0xc000504160 ID:0xc00039628c Kind:0xc000504170 Reason:0xc000504180}" file="register_node.go:43" request_id=d6273afa-44de-4ed0-b435-f40110b2edd0 I verified our ansible playbook and it works correct. Here is output: TASK [Creates a new OpenShift bare metal cluster definition.] ************************************************************************************ task path: /home/slave1/jenkins/workspace/ocp-assisted/ocp-edge-qe-demo/linchpin-workspace/hooks/ansible/ocp-edge-setup/assisted-deploy-ocp-cluster.yaml:19 ok: [localhost] => {"changed": false, "connection": "close", "content": "{\"base_dns_domain\":\"qe.lab.redhat.com\",\"cluster_network_cidr\":\"10.128.0.0/14\",\"cluster_network_host_prefix\":23,\"created_at\":\"2020-07-30T19:04:11\",\"host_networks\":null,\"hosts\":null,\"href\":\"/api/assisted-install/v1/clusters/e860dc6f-3b8c-47d4-aa0b-3711ecc3923b\",\"id\":\"e860dc6f-3b8c-47d4-aa0b-3711ecc3923b\",\"image_info\":{\"created_at\":\"0001-01-01T00:00:00\"},\"install_completed_at\":\"2000-01-01T00:00:00\",\"install_started_at\":\"2000-01-01T00:00:00\",\"kind\":\"Cluster\",\"name\":\"seal15-ocp-cluster-0\",\"openshift_version\":\"4.5\",\"org_id\":\"0000000\",\"pull_secret_set\":true,\"service_network_cidr\":\"172.30.0.0/16\",\"ssh_public_key\":\"ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDB/BByoBTYNQj/FtBxAQQnyU5DZOEIuKlUTtR1UElmb8iOOYs67PutK1xYSa+5kCUIdVRChsvq4jNqyoSG3S88ELO1shIuYQSaOM8vAGnH7dQSpXHlJUfux7utog+6oRbMZxApGctqvzn/w7GEMSQB4zoCQBXwTn+B8jXX5UFXffb6yOmjKcflFgG5/W+nUfDKIhmp7CzN6G0qCRdug2brq0dVwdlK5qYS1Rr18Hg9+WYglwcUv1ET2Be1DZCbvBSxPPoHh5QWKss0urNmxiWZdlzazuSgxMSs/Rnvd8uIpogxE9lKCfEdvdbhgem6d5XvKL7JDAq3IDA+cS8/Afu6HABBQt/WsKMUCBaWWA96CROaEprpBxlV/Q8lLRt3DkgycbVGTIq+JVGMuUMOW8USNJ0FD23kGJSRC+cbmjDs6co5AHDFXyPJocIYei6mTMJ+wUM6K0zRqgG8jcZkobK84xNYrTY0h8p4I2d9duPRW7QvdJjq/4MttcPZSCglsCM= root.lab.tlv.redhat.com\",\"status\":\"insufficient\",\"status_info\":\"cluster is insufficient, exactly 3 known master hosts are needed for installation\",\"status_updated_at\":\"2020-07-30T19:04:11\",\"updated_at\":\"2020-07-30T19:04:11\",\"user_id\":\"0000000\"}\n", "content_length": "1422", "content_type": "application/json", "cookies": {"7c441cd171f2243415fc7df491f023c8": "7a6187f5a3bb6ef86c864edb48607164"}, "cookies_string": "7c441cd171f2243415fc7df491f023c8=7a6187f5a3bb6ef86c864edb48607164", "date": "Thu, 30 Jul 2020 19:04:12 GMT", "elapsed": 0, "json": {"base_dns_domain": "qe.lab.redhat.com", "cluster_network_cidr": "10.128.0.0/14", "cluster_network_host_prefix": 23, "created_at": "2020-07-30T19:04:11", "host_networks": null, "hosts": null, "href": "/api/assisted-install/v1/clusters/e860dc6f-3b8c-47d4-aa0b-3711ecc3923b", "id": "e860dc6f-3b8c-47d4-aa0b-3711ecc3923b", "image_info": {"created_at": "0001-01-01T00:00:00"}, "install_completed_at": "2000-01-01T00:00:00", "install_started_at": "2000-01-01T00:00:00", "kind": "Cluster", "name": "seal15-ocp-cluster-0", "openshift_version": "4.5", "org_id": "0000000", "pull_secret_set": true, "service_network_cidr": "172.30.0.0/16", "ssh_public_key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDB/BByoBTYNQj/FtBxAQQnyU5DZOEIuKlUTtR1UElmb8iOOYs67PutK1xYSa+5kCUIdVRChsvq4jNqyoSG3S88ELO1shIuYQSaOM8vAGnH7dQSpXHlJUfux7utog+6oRbMZxApGctqvzn/w7GEMSQB4zoCQBXwTn+B8jXX5UFXffb6yOmjKcflFgG5/W+nUfDKIhmp7CzN6G0qCRdug2brq0dVwdlK5qYS1Rr18Hg9+WYglwcUv1ET2Be1DZCbvBSxPPoHh5QWKss0urNmxiWZdlzazuSgxMSs/Rnvd8uIpogxE9lKCfEdvdbhgem6d5XvKL7JDAq3IDA+cS8/Afu6HABBQt/WsKMUCBaWWA96CROaEprpBxlV/Q8lLRt3DkgycbVGTIq+JVGMuUMOW8USNJ0FD23kGJSRC+cbmjDs6co5AHDFXyPJocIYei6mTMJ+wUM6K0zRqgG8jcZkobK84xNYrTY0h8p4I2d9duPRW7QvdJjq/4MttcPZSCglsCM= root.lab.tlv.redhat.com", "status": "insufficient", "status_info": "cluster is insufficient, exactly 3 known master hosts are needed for installation", "status_updated_at": "2020-07-30T19:04:11", "updated_at": "2020-07-30T19:04:11", "user_id": "0000000"}, "msg": "OK (1422 bytes)", "redirected": false, "server": "nginx", "set_cookie": "7c441cd171f2243415fc7df491f023c8=7a6187f5a3bb6ef86c864edb48607164; path=/; HttpOnly", "status": 201, "url": "http://assisted-installer-ui.apps.ocp.prod.psi.redhat.com/api/assisted-install/v1/clusters", "x_frame_options": "SAMEORIGIN"} TASK [Creates a new OpenShift per-cluster discovery ISO.] **************************************************************************************** task path: /home/slave1/jenkins/workspace/ocp-assisted/ocp-edge-qe-demo/linchpin-workspace/hooks/ansible/ocp-edge-setup/assisted-deploy-ocp-cluster.yaml:41 ok: [localhost] => {"changed": false, "connection": "close", "content": "{\"base_dns_domain\":\"qe.lab.redhat.com\",\"cluster_network_cidr\":\"10.128.0.0/14\",\"cluster_network_host_prefix\":23,\"created_at\":\"2020-07-30T19:04:11\",\"host_networks\":null,\"hosts\":[],\"href\":\"/api/assisted-install/v1/clusters/e860dc6f-3b8c-47d4-aa0b-3711ecc3923b\",\"id\":\"e860dc6f-3b8c-47d4-aa0b-3711ecc3923b\",\"image_info\":{\"created_at\":\"2020-07-30T19:04:13\",\"generator_version\":\"quay.io/ocpmetal/installer-image-build:latest\",\"ssh_public_key\":\"ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDB/BByoBTYNQj/FtBxAQQnyU5DZOEIuKlUTtR1UElmb8iOOYs67PutK1xYSa+5kCUIdVRChsvq4jNqyoSG3S88ELO1shIuYQSaOM8vAGnH7dQSpXHlJUfux7utog+6oRbMZxApGctqvzn/w7GEMSQB4zoCQBXwTn+B8jXX5UFXffb6yOmjKcflFgG5/W+nUfDKIhmp7CzN6G0qCRdug2brq0dVwdlK5qYS1Rr18Hg9+WYglwcUv1ET2Be1DZCbvBSxPPoHh5QWKss0urNmxiWZdlzazuSgxMSs/Rnvd8uIpogxE9lKCfEdvdbhgem6d5XvKL7JDAq3IDA+cS8/Afu6HABBQt/WsKMUCBaWWA96CROaEprpBxlV/Q8lLRt3DkgycbVGTIq+JVGMuUMOW8USNJ0FD23kGJSRC+cbmjDs6co5AHDFXyPJocIYei6mTMJ+wUM6K0zRqgG8jcZkobK84xNYrTY0h8p4I2d9duPRW7QvdJjq/4MttcPZSCglsCM= root.lab.tlv.redhat.com\"},\"install_completed_at\":\"2000-01-01T00:00:00\",\"install_started_at\":\"2000-01-01T00:00:00\",\"kind\":\"Cluster\",\"name\":\"seal15-ocp-cluster-0\",\"openshift_version\":\"4.5\",\"org_id\":\"0000000\",\"pull_secret_set\":true,\"service_network_cidr\":\"172.30.0.0/16\",\"ssh_public_key\":\"ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDB/BByoBTYNQj/FtBxAQQnyU5DZOEIuKlUTtR1UElmb8iOOYs67PutK1xYSa+5kCUIdVRChsvq4jNqyoSG3S88ELO1shIuYQSaOM8vAGnH7dQSpXHlJUfux7utog+6oRbMZxApGctqvzn/w7GEMSQB4zoCQBXwTn+B8jXX5UFXffb6yOmjKcflFgG5/W+nUfDKIhmp7CzN6G0qCRdug2brq0dVwdlK5qYS1Rr18Hg9+WYglwcUv1ET2Be1DZCbvBSxPPoHh5QWKss0urNmxiWZdlzazuSgxMSs/Rnvd8uIpogxE9lKCfEdvdbhgem6d5XvKL7JDAq3IDA+cS8/Afu6HABBQt/WsKMUCBaWWA96CROaEprpBxlV/Q8lLRt3DkgycbVGTIq+JVGMuUMOW8USNJ0FD23kGJSRC+cbmjDs6co5AHDFXyPJocIYei6mTMJ+wUM6K0zRqgG8jcZkobK84xNYrTY0h8p4I2d9duPRW7QvdJjq/4MttcPZSCglsCM= root.lab.tlv.redhat.com\",\"status\":\"insufficient\",\"status_info\":\"cluster is insufficient, exactly 3 known master hosts are needed for installation\",\"status_updated_at\":\"2020-07-30T19:04:11\",\"updated_at\":\"2020-07-30T19:04:13\",\"user_id\":\"0000000\"}\n", "content_type": "application/json", "cookies": {"7c441cd171f2243415fc7df491f023c8": "7a6187f5a3bb6ef86c864edb48607164"}, "cookies_string": "7c441cd171f2243415fc7df491f023c8=7a6187f5a3bb6ef86c864edb48607164", "date": "Thu, 30 Jul 2020 19:05:06 GMT", "elapsed": 53, "json": {"base_dns_domain": "qe.lab.redhat.com", "cluster_network_cidr": "10.128.0.0/14", "cluster_network_host_prefix": 23, "created_at": "2020-07-30T19:04:11", "host_networks": null, "hosts": [], "href": "/api/assisted-install/v1/clusters/e860dc6f-3b8c-47d4-aa0b-3711ecc3923b", "id": "e860dc6f-3b8c-47d4-aa0b-3711ecc3923b", "image_info": {"created_at": "2020-07-30T19:04:13", "generator_version": "quay.io/ocpmetal/installer-image-build:latest", "ssh_public_key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDB/BByoBTYNQj/FtBxAQQnyU5DZOEIuKlUTtR1UElmb8iOOYs67PutK1xYSa+5kCUIdVRChsvq4jNqyoSG3S88ELO1shIuYQSaOM8vAGnH7dQSpXHlJUfux7utog+6oRbMZxApGctqvzn/w7GEMSQB4zoCQBXwTn+B8jXX5UFXffb6yOmjKcflFgG5/W+nUfDKIhmp7CzN6G0qCRdug2brq0dVwdlK5qYS1Rr18Hg9+WYglwcUv1ET2Be1DZCbvBSxPPoHh5QWKss0urNmxiWZdlzazuSgxMSs/Rnvd8uIpogxE9lKCfEdvdbhgem6d5XvKL7JDAq3IDA+cS8/Afu6HABBQt/WsKMUCBaWWA96CROaEprpBxlV/Q8lLRt3DkgycbVGTIq+JVGMuUMOW8USNJ0FD23kGJSRC+cbmjDs6co5AHDFXyPJocIYei6mTMJ+wUM6K0zRqgG8jcZkobK84xNYrTY0h8p4I2d9duPRW7QvdJjq/4MttcPZSCglsCM= root.lab.tlv.redhat.com"}, "install_completed_at": "2000-01-01T00:00:00", "install_started_at": "2000-01-01T00:00:00", "kind": "Cluster", "name": "seal15-ocp-cluster-0", "openshift_version": "4.5", "org_id": "0000000", "pull_secret_set": true, "service_network_cidr": "172.30.0.0/16", "ssh_public_key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDB/BByoBTYNQj/FtBxAQQnyU5DZOEIuKlUTtR1UElmb8iOOYs67PutK1xYSa+5kCUIdVRChsvq4jNqyoSG3S88ELO1shIuYQSaOM8vAGnH7dQSpXHlJUfux7utog+6oRbMZxApGctqvzn/w7GEMSQB4zoCQBXwTn+B8jXX5UFXffb6yOmjKcflFgG5/W+nUfDKIhmp7CzN6G0qCRdug2brq0dVwdlK5qYS1Rr18Hg9+WYglwcUv1ET2Be1DZCbvBSxPPoHh5QWKss0urNmxiWZdlzazuSgxMSs/Rnvd8uIpogxE9lKCfEdvdbhgem6d5XvKL7JDAq3IDA+cS8/Afu6HABBQt/WsKMUCBaWWA96CROaEprpBxlV/Q8lLRt3DkgycbVGTIq+JVGMuUMOW8USNJ0FD23kGJSRC+cbmjDs6co5AHDFXyPJocIYei6mTMJ+wUM6K0zRqgG8jcZkobK84xNYrTY0h8p4I2d9duPRW7QvdJjq/4MttcPZSCglsCM= root.lab.tlv.redhat.com", "status": "insufficient", "status_info": "cluster is insufficient, exactly 3 known master hosts are needed for installation", "status_updated_at": "2020-07-30T19:04:11", "updated_at": "2020-07-30T19:04:13", "user_id": "0000000"}, "msg": "OK (unknown bytes)", "redirected": false, "server": "nginx", "set_cookie": "7c441cd171f2243415fc7df491f023c8=7a6187f5a3bb6ef86c864edb48607164; path=/; HttpOnly", "status": 201, "transfer_encoding": "chunked", "url": "http://assisted-installer-ui.apps.ocp.prod.psi.redhat.com/api/assisted-install/v1/clusters/e860dc6f-3b8c-47d4-aa0b-3711ecc3923b/downloads/image", "x_frame_options": "SAMEORIGIN"} Version-Release number of selected component (if applicable): Assisted Installer UI version refs/heads/master (quay.io/ocpmetal/ocp-metal-ui:6ea08686e6236151f6d9f91fc78c16e431e10f4c) Assisted Installer quay.io/ocpmetal/assisted-installer:latest Assisted Installer Controller quay.io/ocpmetal/assisted-installer-controller:latest Assisted Installer Service quay.io/ocpmetal/bm-inventory:latest Discovery Agent quay.io/ocpmetal/agent:latest Ignition Manifests and Kubeconfig Generate quay.io/ocpmetal/ignition-manifests-and-kubeconfig-generate:latest Image Builder quay.io/ocpmetal/installer-image-build:latest How reproducible: http://assisted-installer-ui.apps.ocp.prod.psi.redhat.com/clusters/e860dc6f-3b8c-47d4-aa0b-3711ecc3923b Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info:
So, finally this is an environmental issue and has been resolved by manual editing the bm-inventory configmap. Probably we need an additional parameter for command: make deploy-all which will allow set INVENTORY_URL. Here is config map yaml before fix kind: ConfigMap apiVersion: v1 metadata: name: bm-inventory-config namespace: assisted-installer-qe selfLink: /api/v1/namespaces/assisted-installer-qe/configmaps/bm-inventory-config uid: fa61eb74-3885-4ddc-a32e-d13c9e711715 resourceVersion: '121716639' creationTimestamp: '2020-07-23T18:57:42Z' labels: app: bm-inventory annotations: kubectl.kubernetes.io/last-applied-configuration: > {"apiVersion":"v1","data":{"AGENT_DOCKER_IMAGE":"quay.io/ocpmetal/agent:latest","BASE_DNS_DOMAINS":"","CONNECTIVITY_CHECK_IMAGE":"quay.io/ocpmetal/connectivity_check:latest","CONTROLLER_IMAGE":"quay.io/ocpmetal/assisted-installer-controller:latest","IMAGE_BUILDER":"quay.io/ocpmetal/installer-image-build:latest","INSTALLER_IMAGE":"quay.io/ocpmetal/assisted-installer:latest","INVENTORY_IMAGE":"quay.io/ocpmetal/inventory:latest","INVENTORY_PORT":"80","INVENTORY_URL":"assisted-installer.apps.ocp.prod.psi.redhat.com","KUBECONFIG_GENERATE_IMAGE":"quay.io/ocpmetal/ignition-manifests-and-kubeconfig-generate:latest","NAMESPACE":"assisted-installer-qe","OPENSHIFT_INSTALL_RELEASE_IMAGE":"quay.io/openshift-release-dev/ocp-release@sha256:eab93b4591699a5a4ff50ad3517892653f04fb840127895bb3609b3cc68f98f3","SELF_VERSION":"quay.io/ocpmetal/bm-inventory:latest"},"kind":"ConfigMap","metadata":{"annotations":{},"labels":{"app":"bm-inventory"},"name":"bm-inventory-config","namespace":"assisted-installer-qe"}} data: KUBECONFIG_GENERATE_IMAGE: 'quay.io/ocpmetal/ignition-manifests-and-kubeconfig-generate:latest' INVENTORY_PORT: '80' INSTALLER_IMAGE: 'quay.io/ocpmetal/assisted-installer:latest' SELF_VERSION: 'quay.io/ocpmetal/bm-inventory:latest' BASE_DNS_DOMAINS: '' INVENTORY_URL: assisted-installer.apps.ocp.prod.psi.redhat.com AGENT_DOCKER_IMAGE: 'quay.io/ocpmetal/agent:latest' IMAGE_BUILDER: 'quay.io/ocpmetal/installer-image-build:latest' INVENTORY_IMAGE: 'quay.io/ocpmetal/inventory:latest' OPENSHIFT_INSTALL_RELEASE_IMAGE: >- quay.io/openshift-release-dev/ocp-release@sha256:eab93b4591699a5a4ff50ad3517892653f04fb840127895bb3609b3cc68f98f3 CONTROLLER_IMAGE: 'quay.io/ocpmetal/assisted-installer-controller:latest' NAMESPACE: assisted-installer-qe CONNECTIVITY_CHECK_IMAGE: 'quay.io/ocpmetal/connectivity_check:latest'
It's not a bug. Please close