Description of problem: The long running environment we encounter during upgrade/update/ffwd may have specific issues that are rarely encountered during testing due the the sort running time and few data those test environment have. So we need to have a pre-flight check done. This should be done outside of director framework (no heat/mistral ...) as we want to check both undercloud and overcloud. This shouldn't be done during the upgrade so that it doesn't take unnecessary time in the maintenance window the operator will have for the upgrade/update/ffwd. It should be very easy to read so that anybody can contribute. It should be very easy to extend so that operator could add their own test in whatever language they see fit (shell/python/...whatever) It could trigger purge jobs on the costumer environment provided the purge job exists and the operator agree. It should output human readable report or json like output for further processing.
It should be able to run specific tests based on currently installed OSP version.
Some examples of the common issue are: - db purge not working; - .ini/.json not updated because of costumer modification; - missing package; - undeleted packages;
We heard you :). Please jump in the spec redaction and add your needs, thoughts and so on. We're going to make a cool thing together.
Closing this, since the Validation Framework is now current, and working. We're "just" missing proper validations for some content, but that's another story :).