Summary:

.has_active_webservices is updated when the subject appliance successfully enables or disables that role. While it should be enabled when the role becomes active, it should be disabled when the role is *requested* to be turned off. Additionally, the flags should be updated when the subject appliance is detected as "not responding".

Also, significantly, the UI provides no indication that the appliances were "not responding" would would have assisted tremendously in debugging issues.


Background:

Client here has deployed a number of new appliances, which were semi-configured, forgotten about, and soon lost database connectivity. It was discovered that these were being sent traffic, as custom code which checks .has_active_webservices indicated they were available.

These appliances were only partially configured/mis-configured, and should not have had those roles enabled. Roles were disabled in the UI, and the UI reported that successful (or at least implicitly: the checkboxes remained unchecked). Unfortunately, the custom code continued to pick these systems up as a target.

Digging further, from the rails console, I noted that the appliance record had last_heartbeat of more than a week ago and were "not responding".

I did not check further, but assume that the .has_active_webservices flag is updated when the appliance reports something is enabled or disabled.