Description of problem: After draining nodes both pods (prometheus-k8s-0 and prometheus-k8s-1) restarted and all data that was previously collected by metrics removed from a cluster Version-Release number of selected component (if applicable): CNV-4.10 How reproducible: 100% Steps to Reproduce: 1. drain first node where pod prometheus-k8s-0 is running 2. return node back and drain second node where prometheus-k8s-1 is running 3. check metrics on a cluster - old data removed Actual results: Metrics cleared when both pods prometheus-k8s-0 and prometheus-k8s-1 restarted Expected results: Old data should be kept Additional info: I guess some persistent storage should be using for keeping all metrics: https://docs.openshift.com/container-platform/4.9/monitoring/configuring-the-monitoring-stack.html#configuring-persistent-storage
If you haven't configured persistent storage, this is expected. Following the procedure in the docs you linked is a prerequisite to having the data persist.