Description of problem:
Deleting ipsets using firewall-cmd does not remove ipset from the underlying system.

Version-Release number of selected component (if applicable):


How reproducible:
Every time

Steps to Reproduce:

# firewall-cmd --permanent --new-ipset=test --type=hash:net --option=family=inet6
success
# firewall-cmd --permanent --ipset=test --add-entry=::1
success
# systemctl reload firewalld
# firewall-cmd --get-ipsets
test
# firewall-cmd --permanent --delete-ipset=test
success
# firewall-cmd --get-ipsets
test
# systemctl reload firewalld
# firewall-cmd --get-ipsets
# firewall-cmd --get-ipsets | wc -l
0
# ls /etc/firewalld/ipsets/
text.xml.old
# cat /etc/firewalld/ipsets/test.xml.old 
<?xml version="1.0" encoding="utf-8"?>
<ipset type="hash:net">
  <option name="family" value="inet6"/>
  <entry>::1</entry>
</ipset>
# firewall-cmd --reload
success
# ipset --list
Name: test
Type: hash:net
Revision: 6
Header: family inet6 hashsize 1024 maxelem 65536
Size in memory: 1272
References: 0
Number of entries: 1
Members:
::1


Actual results:
/etc/firewalld/ipsets/<ipset>.xml is removed, however the ipset is still present in "ipset --list"

Expected results:
/etc/firewalld/ipsets/<ipset.xml> to be removed and also no longer have ipset present in "ipset --list"

Additional info:
This behavior appears to be present in upstream along with RHEL 8 and RHEL 7. There appears to have been an upstream discussion about this behavior seen in the following conversation.

      https://github.com/firewalld/firewalld/issues/330

As firewalld has the ability to remove all rules from iptable/nftables and reapply all configured rules on a --reload. I feel the same behavior should apply to ipsets as well. I feel that the behavior is at least implied. I can understand that we can only make permanent changes to ipsets using the permanent flag, but I see no reason why we cannot flush all ipsets and reapply configured ipsets on a "--reload" to conform to the behavior that exists in relation to how firewalld interacts with iptables/nftables.