Bug 1939528

Summary:	Reseting a cluster removes user's created manifests
Product:	OpenShift Container Platform	Reporter:	Constantin Vultur <cvultur>
Component:	assisted-installer	Assignee:	vemporop
assisted-installer sub component:	assisted-service	QA Contact:	Yuri Obshansky <yobshans>
Status:	CLOSED ERRATA	Docs Contact:
Severity:	high
Priority:	high	CC:	aos-bugs, atraeger, itsoiref, masayag, ncarboni, sasha
Version:	4.8
Target Milestone:	---
Target Release:	4.8.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:	AI-Team-Core
Fixed In Version:		Doc Type:	No Doc Update
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2022-01-05 16:11:41 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Constantin Vultur 2021-03-16 15:11:35 UTC

Description of problem:
If a user creates customized content, like manifests and resets the cluster , the manifests are deleted

Version-Release number of selected component (if applicable):
Assisted Installer UI version
quay.io/ocpmetal/ocp-metal-ui:38a0c8d8279ef90099414750371652f701e87170
Assisted Installer UI library version
1.5.12
Assisted Installer
quay.io/ocpmetal/assisted-installer:0645568814701c122bed3ad9a25c9bcdf30215ef
Assisted Installer Controller
quay.io/ocpmetal/assisted-installer-controller:0645568814701c122bed3ad9a25c9bcdf30215ef
Assisted Installer Service
quay.io/ocpmetal/assisted-service:62f6a12f9790dff75397c65cfd652215ab83a088
Discovery Agent
quay.io/ocpmetal/assisted-installer-agent:edbaff3f6b1343b6e51c64d461923ac592820476


How reproducible:


Steps to Reproduce:
1. Create a cluster and add a manifest as per: https://github.com/openshift/assisted-service/blob/master/docs/user-guide/install-customization.md#manifests

In my case I had
$ curl -X GET http://AI/api/assisted-install/v1/clusters/6c648ad3-f6a5-4c18-abd4-99bca732aa7a/manifests
[{"file_name":"openshift/99-openshift-machineconfig-master-registry.yaml","folder":"manifests"}]

2. start install and make installation fail
3. Reset the cluster
4. Check the existing manifests

Actual results:
The manifests list is empty:
$ curl -X GET http://AI/api/assisted-install/v1/clusters/6c648ad3-f6a5-4c18-abd4-99bca732aa7a/manifests
[]


Expected results:
Manifests to still be present, or the user to be warned that the customizations were deleted. 

Additional info:

Comment 1 Moti Asayag 2021-03-16 22:09:20 UTC

The behavior was changed by https://github.com/openshift/assisted-service/pull/711.
Before that PR, ResetCluster used to remove files by pre-defined name.
With this PR, the entire folder (minus logs) is deleted.
The fix should skip manifests files instead of deleting them.

Comment 2 Moti Asayag 2021-03-16 22:12:17 UTC

https://github.com/openshift/assisted-service/blob/master/internal/cluster/cluster.go#L770 is the place to skip the manifest files deletion

Comment 3 Nick Carboni 2021-03-18 13:05:39 UTC

I'm curious if the intent of https://github.com/openshift/assisted-service/pull/711 was explicitly to remove the manifests.
I suppose manifests couldn't really influence the success or failure of an install, but probably worth getting Igal's input.

Comment 4 Ronnie Lazar 2021-04-14 08:48:30 UTC

@atraeger , @itsoiref what do you think?

Comment 5 Ronnie Lazar 2021-05-06 08:14:19 UTC

We need to treat the customizations just like all other configuration and not delete them when the cluster was reset

Comment 6 Moti Asayag 2021-05-06 09:04:33 UTC

(In reply to Ronnie Lazar from comment #5)
> We need to treat the customizations just like all other configuration and
> not delete them when the cluster was reset

That was implemented and merged by https://github.com/openshift/assisted-service/pull/1367

Comment 7 Constantin Vultur 2021-05-17 11:32:58 UTC

Verified with:

"assisted-installer":"quay.io/ocpmetal/assisted-installer:c76ba93cdd5b3e6f2734a0e7b35ab32213e1021a",
"assisted-installer-agent":"quay.io/ocpmetal/assisted-installer-agent:72b14c873597f3883eab9af308fb803b344af2d0",
"assisted-installer-controller":"quay.io/ocpmetal/assisted-installer-controller:c76ba93cdd5b3e6f2734a0e7b35ab32213e1021a",
"assisted-service":"quay.io/ocpmetal/assisted-service:1709a50d59bb8aba03713ade2f4869f86a43b8a3"

Cluster installation was started, manifest were checked and after few minutes the installation was aborted. 
Checked the manifests and did a cluster reset via UI. 

Re-checked the manifests, proxy settings, ignition config overrides and those were still there. 

Marking as Verified

Comment 11 errata-xmlrpc 2022-01-05 16:11:41 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.8.25 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:5209