The original out of space issue is temporarily fixed and needs a permanent fix in terms of a cron to delete files older than 15 days.
Our alerting did not seem to catch that there was an issue at all. So we need to figure out why we missed that.
So, the alerting failed because munin was blocked by selinux. I am not sure why/how it happen since there is no log. But I switched the right domain in permissive, and will investigate when I am back.
So, I suspect this was fixed, because I can't see what domain was set in permissive anywhere, ergo closing it.