Bug 1698073
| Summary: | Sometimes vsphere compute and control-plane nodes come up without open-vm-tools | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Hemant Kumar <hekumar> |
| Component: | RHCOS | Assignee: | Steve Milner <smilner> |
| Status: | CLOSED NOTABUG | QA Contact: | Micah Abbott <miabbott> |
| Severity: | unspecified | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 4.1.0 | CC: | bbennett, bbreard, dphillip, dustymabe, imcleod, jligon, nstielau |
| Target Milestone: | --- | ||
| Target Release: | 4.1.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | No Doc Update | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2019-05-07 13:53:19 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
|
Description
Hemant Kumar
2019-04-09 14:42:29 UTC
The machine-os-content for the release is pointing towards: 410.8.20190329.0.
This particular build didn't include open-vm-tools.
I have tested:
"410.8.20190401.0" and "410.8.20190330.0" both of these subsequent builds include open-vm-tools. This image is a one off it seems.
rpm-ostree status
State: idle
AutomaticUpdates: disabled
Deployments:
pivot://registry.svc.ci.openshift.org/openshift/origin-v4.0-2019-04-08-195526@sha256:83e25d1681c42be4e9257e288d79d3d3591301e1d5897caff96cecc38d7878f0
CustomOrigin: Managed by pivot tool
Version: 410.8.20190329.0 (2019-03-29T16:35:54Z)
● pivot://docker-registry-default.cloud.registry.upshift.redhat.com/redhat-coreos/ootpa@sha256:4582398a53ad79e7401be70df54c2ab0afb5543897752def56360b1d4f2d3bd4
CustomOrigin: Provisioned from oscontainer
Version: 410.8.20190326.0 (2019-03-26T15:44:56Z)
@davis - I am still confused why 2 deployments (30 mins apart) come with different versions of RHCOS on compute and control-plane nodes. I understand it picked slightly older version of image but why? The initial boot image is replaced by the machine-os-content supplied by the installer: This happens after ETCD establishes quorum but before kubelet.service starts. [Unit] Description=Pivot Tool ConditionPathExists=/etc/pivot/image-pullspec After=ignition-firstboot-complete.service Before=kubelet.service [Service] # Need oneshot to delay kubelet Type=oneshot ExecStart=/usr/bin/pivot [Install] WantedBy=multi-user.target The bootstrap node's bootkube.sh service pulls the release image, which contains a reference to the MCO (machine-config-operator) and also a reference to a newer machine-os-content. The bootkube.sh service runs the MCO in "bootstrap" mode to generate and serve Ignition to the master machines. https://github.com/openshift/machine-config-operator/blob/9da96326a5ff737869709f5fa2e6c716df4dbaf4/docs/OSUpgrades.md For a while the machine-os-content was pointing towards the 0329 image which is the only one that didn't include open-vm-tools. The subsequent releases does include them. I've tested and verified as much. I'm going to test now. But, I believe the image has been updated.
[dphillip@control-plane-1 ~]$ skopeo inspect --authfile=~/.docker/config.json docker://registry.svc.ci.openshift.org/rhcos/machine-os-content:latest
{
"Name": "registry.svc.ci.openshift.org/rhcos/machine-os-content",
"Tag": "latest",
"Digest": "sha256:ca664d88674d930afd6d727d0d6242668fe3e274f077abbe9d7375854b1bf788",
"RepoTags": [
"latest"
],
"Created": "2019-04-08T22:07:35.285453782Z",
"DockerVersion": "",
"Labels": {
"com.coreos.ostree-commit": "9df99f7dd9e11ba06ef83b006cfe256b4b9ab1e2acc30f47f5a6a4979f6d5f20",
"version": "410.8.20190408.1"
},
"Architecture": "amd64",
"Os": "linux",
"Layers": [
"sha256:4dd39f488cc2725f4711e0a0d66a986fec0aa282bc899633491955824f0551e3"
]
}
It looks like we may be good to go.
[core@control-plane-0 ~]$ rpm-ostree status
State: idle
AutomaticUpdates: disabled
Deployments:
● pivot://registry.svc.ci.openshift.org/openshift/origin-v4.0-2019-04-09-210205@sha256:ca664d88674d930afd6d727d0d6242668fe3e274f077abbe9d7375854b1bf788
CustomOrigin: Managed by pivot tool
Version: 410.8.20190408.1 (2019-04-08T21:44:07Z)
pivot://docker-registry-default.cloud.registry.upshift.redhat.com/redhat-coreos/ootpa@sha256:4582398a53ad79e7401be70df54c2ab0afb5543897752def56360b1d4f2d3bd4
CustomOrigin: Provisioned from oscontainer
Version: 410.8.20190326.0 (2019-03-26T15:44:56Z)
[core@control-plane-0 ~]$ systemctl status vmtoolsd
● vmtoolsd.service - Service for virtual machines hosted on VMware
Loaded: loaded (/usr/lib/systemd/system/vmtoolsd.service; disabled; vendor preset: enabled)
Active: active (running) since Tue 2019-04-09 22:03:58 UTC; 2min 18s ago
Docs: http://github.com/vmware/open-vm-tools
Main PID: 887 (vmtoolsd)
Tasks: 1 (limit: 26213)
Memory: 4.2M
CPU: 52ms
CGroup: /system.slice/vmtoolsd.service
└─887 /usr/bin/vmtoolsd
> It looks like we may be good to go.
Agreed. With the latest images/payload you should be good. Will wait to hear Hemant's findings before resolving this bug.
Hemant, Is this still an issue? The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days |