Created attachment 1284722 [details]
Logs

Description of problem:
[RFE] - vdsm should be aware of new bond options in new kernel and should take action upon the change to make the new options as vdsm defaults. 

Unless running the next command after the kernel update we will end up with the next scenario:
vdsm-tool dump-bonding-options
systemctl restart vdsmd supervdsmd.service

Can't update bond with IOError: [Errno 13] Permission denied error on rhel 7.4 with latest kernel.

It's not possible to update the bond with another bond mode on rhel 7.4.

2017-06-04 11:37:42,359+0300 ERROR (jsonrpc/0) [jsonrpc.JsonRpcServer] Internal server error (__init__:623)
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 618, in _handle_request
    res = method(**params)
  File "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py", line 202, in _dynamicMethod
    result = fn(*methodArgs)
  File "/usr/share/vdsm/API.py", line 1433, in setupNetworks
    supervdsm.getProxy().setupNetworks(networks, bondings, options)
  File "/usr/lib/python2.7/site-packages/vdsm/supervdsm.py", line 53, in __call__
    return callMethod()
  File "/usr/lib/python2.7/site-packages/vdsm/supervdsm.py", line 51, in <lambda>
    **kwargs)
  File "<string>", line 2, in setupNetworks
  File "/usr/lib64/python2.7/multiprocessing/managers.py", line 773, in _callmethod
    raise convert_to_error(kind, result)
IOError: [Errno 13] Permission denied

I think that the main problem starts because when creating the bond in the setup networks UI, for example mode 4 or 1, it reported as an custom bond mode, which isn't correct(behave different on previous version). vdsm thinks some bond options are not it's defualt and it's why the bond is a custom bond. 
And after the bond creation, when choosing another bond mode from the list we failing with error on vdsm side.
But, if trying to update the bond with custom bond mode, just switch from 1 to 4, it works. 


Version-Release number of selected component (if applicable):
vdsm-4.20.0-963.git4e2eee6.el7.centos.x86_64
3.10.0-675.el7.x86_64
rhel 7.4

How reproducible:
100%

Steps to Reproduce:
1. Create default bond mode in the setup networks UI on rhel 7.4 
2. Edit the bond and choose from the list, mode 1
3. Update the custom mode field with mode=1

Actual results:
1. Problem starts, bond reported as custom bond mode(regression), vdsm see difference in the default bond options, which changed after kernel update.
2. Failed with vdsm error
3. Works

Expected results:
1. Bond shouldn't be reported as custom bond mode if wasn't selected as custom bond mode. vdsm should recognize new bond options in the kernel and make them as default ones.
2. Should work as expected. 

Additional info:
- Attaching screenshots
- This behaviour happens for all bond modes and it's a regression. 
vdsm see difference between his default bond options and the new kernel bond options and bond created as custom bond mode.