Bug 1734100
Summary: | [upi-vmware] bootstrap vm hangs at first boot | ||
---|---|---|---|
Product: | OpenShift Container Platform | Reporter: | Mario Abajo <mabajodu> |
Component: | Documentation | Assignee: | Kathryn Alexander <kalexand> |
Status: | CLOSED EOL | QA Contact: | liujia <jiajliu> |
Severity: | medium | Docs Contact: | Vikram Goyal <vigoyal> |
Priority: | unspecified | ||
Version: | 4.1.z | CC: | adahiya, aos-bugs, bbreard, dornelas, dphillip, dustymabe, hcisneir, imcleod, jiajliu, jligon, jokerman, kalexand, nstielau, rbost, smilner, wsun |
Target Milestone: | --- | Keywords: | Reopened |
Target Release: | --- | Flags: | jiajliu:
needinfo-
|
Hardware: | Unspecified | ||
OS: | Unspecified | ||
Whiteboard: | |||
Fixed In Version: | Doc Type: | If docs needed, set a value | |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | 2020-05-18 06:56:25 UTC | Type: | Bug |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: |
Description
Mario Abajo
2019-07-29 15:44:20 UTC
Mario, Thanks for the report. Has this been brought up with the UPI folks already? (In reply to Steve Milner from comment #1) > Mario, > > Thanks for the report. Has this been brought up with the UPI folks already? Hello Steve, No, not yet, but if you can give me someone to contact i will appreciate it. Thanks His Liu, will you please confirm the PR that Davis opened? Follow up method 2 added in pr16738, add compute-1 node to an existed cluster successfully. # oc get node NAME STATUS ROLES AGE VERSION compute-0 Ready worker 3h45m v1.14.6+194d29900 compute-1 Ready worker 5m23s v1.14.6+194d29900 control-plane-0 Ready master 3h45m v1.14.6+194d29900 Thank you! I've merged the change and am waiting for it to go live. This change is live on docs.openshift: https://docs.openshift.com/container-platform/4.1/installing/installing_vsphere/installing-vsphere.html#installation-vsphere-machines_installing-vsphere and on the portal: https://access.redhat.com/documentation/en-us/openshift_container_platform/4.1/html-single/installing/index#installation-vsphere-machines_installing-vsphere Hi I have to reopen this case as the issue still persist in OCP 4.2 with vsphere 6.5u3 See case 02522835. The solution of adding the "guestinfo.ovfEnv" manage to solve the issue. Davis, do you agree that adding the "guestinfo.ovfEnv" parameter should be required? Its pretty strange, because I've not been able to replicate this issue. There was another github issue: https://github.com/openshift/installer/issues/2537 This is from a configuration deployed via the UPI processed highlighted in the blog post I wrote earlier this year. vmtoolsd --cmd 'info-get guestinfo.ovfEnv' <PropertySection> <Property oe:key="guestinfo.ignition.config.data" oe:value="eyJpZ25pdGlvbiI6eyJjb25maWciOnsiYXBwZW5kIjpbeyJzb3VyY2UiOiJkYXRhOnRleHQvcGxhaW47Y2hhcnNldD11dGYtODtiYXNlNjQsZXlKcFoyNXBkR2x2YmlJNmV5SmpiMjVtYVdjaU9uc2lZWEJ3Wlc1a0lqcGJleUp6YjNWeVkyVWlPaUpvZEhSd2N6b3ZMMkZ3YVMxcGJuUXVkWEJwTG1VeVpTNWliM011Y21Wa2FHRjBMbU52YlRveU1qWXlNeTlqYjI1bWFXY3ZiV0Z6ZEdWeUlpd2lkbVZ5YVdacFkyRjBhVzl1SWpwN2ZYMWRmU3dpYzJWamRYSnBkSGtpT ..omitted.. So, the "guestinfo.ovfEnv" should be created from the vapp properties applied via each deployed template. Would it make more sense to add some information to a troubleshooting section? My two cents on this; up to my knowledge what ignition do to load the ignition data [1] is to read the "ovfenv" parameter, not the "ignition.config.data" as stated in the doc, so we rely on some component of vsphere to fill "ovfenv" with "ignition.config.data" and "ignition.config.data.encoding" parameters but for unknown reason (at least for me) this doesn't always happens. I think that it would be better to not depend on a third party for this process, so we should read the same parameters that we fill. [1] https://github.com/coreos/ignition/blob/befbc8677cc44b8ec089cfc7c5bfe015cfed88cd/internal/providers/vmware/vmware_amd64.go#L65 Jia, does Mario's comment 28 align with your test, and do we need to pass guestinfo.ovfEnv? Davis, we have not created the installation troubleshooting section yet, but if this is not currently required, I can file it as a good addition to the future troubleshooting docs. I did not hit the issue during past v4.1/v4.2/v4.3 upi/vsphere tests. And I have a try for bootstrap node launch again following https://docs.openshift.com/container-platform/4.2/installing/installing_vsphere/installing-vsphere.html step by step. Still works well with two ways. 1. Genetate instal-config/manifests/ignition files(base64 encode). 2. Upload ignition file to s3 and prepare rhcos template(rhcos-42.80.20191002.0 on vsphere 3. Clone vm for bootstrap node as the 1st way in step7 of above doc(https://docs.openshift.com/container-platform/4.2/installing/installing_vsphere/installing-vsphere.html#installation-vsphere-machines_installing-vsphere) ''' On the Customize hardware tab, click VM Options → Advanced. From the Latency Sensitivity list, select High. Click Edit Configuration, and on the Configuration Parameters window, click Add Configuration Params. Define the following parameter names and values: guestinfo.ignition.config.data: Paste the contents of the base64-encoded Ignition config file for this machine type. guestinfo.ignition.config.data.encoding: Specify base64. disk.EnableUUID: Specify TRUE. ''' After above steps(before the vm powoff), checked that [configure]-[Settings]-[vApp Options] page -In "OVF Settings" section, it is disabled with "The OVF enviroment is only available when the vm is powered on". Power on above vm, bootstrap node start successfully, and now OVF enviroment can be checked from above page. <PropertySection> <Property oe:key="guestinfo.ignition.config.data" oe:value=""/> <Property oe:key="guestinfo.ignition.config.data.encoding" oe:value=""/> </PropertySection> And ssh into the bootstrap node successully. 4. Power off above bootsrap vm. 5. Clone vm for bootstrap node as the 2st way in step7 of above doc(https://docs.openshift.com/container-platform/4.2/installing/installing_vsphere/installing-vsphere.html#installation-vsphere-machines_installing-vsphere) ''' On the Customize hardware tab, click VM Options → Advanced. From the Latency Sensitivity list, select High. Alternatively, prior to powering on the virtual machine add via vApp properties: Navigate to a virtual machine from the vCenter Server inventory. On the Configure tab, expand Settings and select vApp options. Scroll down and under Properties apply the configurations from above. ''' After above steps, there are 3 items added into [configure]-[Settings]-[vApp Options]-[Properties] page, and -In "OVF Settings" section, it is disabled with "The OVF enviroment is only available when the vm is powered on". Power on above vm, bootstrap node start successfully, and now OVF enviroment can be checked from above page. <PropertySection> <Property oe:key="disk.EnableUUID" oe:value="TRUE"/> <Property oe:key="guestinfo.ignition.config.data" oe:value="xxxxxxx"/> <Property oe:key="guestinfo.ignition.config.data.encoding" oe:value="base64"/> </PropertySection> And ssh into the bootstrap node successully. |