Description of problem:

If you delete a candlepin consumer without deleting the katello record, the katello record does not disappear after running "foreman-rake katello:clean_backend_objects".

This bug is a less severe bug related to https://bugzilla.redhat.com/show_bug.cgi?id=1271500.

QE note: in the wild, this bug would normally occur if systems were being modified while a satellite was rebooted (see original bz). However, there are some "fake" steps to induce an out-of-sync condition that we can use to replicate this scenario more easily.

Version-Release number of selected component (if applicable): 6.2.0 snap 19.1


How reproducible: every time


Steps to Reproduce:
1. induce a situation where candlepin and katello are out of sync. For purposes of this bug, the best way is to register a Satellite to itself, then delete the host record in candlepin.

Detailed steps:

* set up a sat6, register system to itself
* edit /etc/candlepin/candlepin.conf:
 ** comment out "module.config.adapter_module=org.candlepin.katello.KatelloModule"
 ** change "candlepin.auth.basic.enable=false" to true
* restart tomcat
* run "curl -k https://localhost:8443/candlepin/admin/init" to create candlepin admin user
* run "curl -k -u admin:admin https://localhost:8443/candlepin/consumers/" to grab the UUID of the consumer to delete
* run "curl -k -u admin:admin -X DELETE 'https://localhost:8443/candlepin/consumers/<UUID>'"

* undo your changes to candlepin.conf
* katello-service restart

* ensure that web UI is not totally broken. It is OK if an error is raised when viewing details for the particular deleted host, but "content hosts" list should work still.

* run "foreman-rake katello:clean_backend_objects"

output:

[root@sat-snap-rhel7 ~]# foreman-rake katello:clean_backend_objects
System 1 sat-snap-rhel7.example.com 5a29a0a1-62f2-48f8-bc52-c94534832c02 is partially missing.  Cleaning.


Actual results: system still shows up in katello web UI

Expected results: system should disappear from katello web UI


note: katello:reindex does not seem to help either