Bug 1667411

Would you please share the vdsm.log from the affected host?
Especially the lines containing hostQos or setupNetworks are relevant,
but best would be the whole vdsm.log and supervdsm.log files, and the relevant part of engine.log.

If possible, can you please check what happens if you change the 50 to another value in
Weighted Share in the QoS in
Compute > Data Centers > xxx > Host Network ?

(In reply to Dominik Holler from comment #1)
> Would you please share the vdsm.log from the affected host?
> Especially the lines containing hostQos or setupNetworks are relevant,
> but best would be the whole vdsm.log and supervdsm.log files, and the
> relevant part of engine.log.
> 

vdsm.log and supervdsm.log attached.

Part of the engine.log:

2019-01-18 14:38:53,772+01 INFO  [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (EE-ManagedThreadFactory-engine-Thread-2919) [36476bcb] Failed to acquire lock and wait lock 'HostEngineLock:{exclusiveLocks='[3592e846-1470-468d-986c-3594af9527cc=VDS_INIT]', sharedLocks=''}'
2019-01-18 14:38:53,872+01 INFO  [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (EE-ManagedThreadFactory-engine-Thread-2919) [36476bcb] Failed to acquire lock and wait lock 'HostEngineLock:{exclusiveLocks='[3592e846-1470-468d-986c-3594af9527cc=VDS_INIT]', sharedLocks=''}'
2019-01-18 14:38:53,966+01 INFO  [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (EE-ManagedThreadFactory-engine-Thread-2919) [36476bcb] Failed to acquire lock and wait lock 'HostEngineLock:{exclusiveLocks='[3592e846-1470-468d-986c-3594af9527cc=VDS_INIT]', sharedLocks=''}'
2019-01-18 14:38:53,998+01 INFO  [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (EE-ManagedThreadFactory-engine-Thread-2919) [36476bcb] Failed to acquire lock and wait lock 'HostEngineLock:{exclusiveLocks='[3592e846-1470-468d-986c-3594af9527cc=VDS_INIT]', sharedLocks=''}'
2019-01-18 14:38:54,002+01 INFO  [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (EE-ManagedThreadFactory-engine-Thread-2919) [36476bcb] Failed to acquire lock and wait lock 'HostEngineLock:{exclusiveLocks='[3592e846-1470-468d-986c-3594af9527cc=VDS_INIT]', sharedLocks=''}'
2019-01-18 14:38:54,335+01 INFO  [org.ovirt.engine.core.bll.network.host.HostSetupNetworksCommand] (default task-354) [c2c9ff12-a918-479b-aaf8-af419a1d81e2] Lock Acquired to object 'EngineLock:{exclusiveLocks='[HOST_NETWORK25dff3be-f2b7-4d42-8240-fd2959d84f5c=HOST_NETWORK]', sharedLocks=''}'
2019-01-18 14:38:54,397+01 WARN  [org.ovirt.engine.core.bll.network.host.HostSetupNetworksCommand] (default task-354) [c2c9ff12-a918-479b-aaf8-af419a1d81e2] Validation of action 'HostSetupNetworks' failed for user bla@******-authz. Reasons: VAR__ACTION__SETUP,VAR__TYPE__NETWORKS,OUTAVERAGELINKSHARE,HOST_OUT_OF_SYNC,DC_OUT_OF_SYNC,$OUT_OF_SYNC_VALUES ${OUTAVERAGELINKSHARE} ${HOST_OUT_OF_SYNC} - null, ${DC_OUT_OF_SYNC} - 50,NETWORK_NOT_IN_SYNC,$NETWORK_NOT_IN_SYNC ovirtmgmt
2019-01-18 14:38:54,398+01 INFO  [org.ovirt.engine.core.bll.network.host.HostSetupNetworksCommand] (default task-354) [c2c9ff12-a918-479b-aaf8-af419a1d81e2] Lock freed to object 'EngineLock:{exclusiveLocks='[HOST_NETWORK25dff3be-f2b7-4d42-8240-fd2959d84f5c=HOST_NETWORK]', sharedLocks=''}'
2019-01-18 14:38:54,401+01 INFO  [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (EE-ManagedThreadFactory-engine-Thread-2919) [36476bcb] Failed to acquire lock and wait lock 'HostEngineLock:{exclusiveLocks='[3592e846-1470-468d-986c-3594af9527cc=VDS_INIT]', sharedLocks=''}'

> If possible, can you please check what happens if you change the 50 to
> another value in
> Weighted Share in the QoS in
> Compute > Data Centers > xxx > Host Network ?

It's a clone from production and we have about 10 QOS policies on that. So I changed the MGMT Qos from 50 to 60, but it didn't help. That was the only Qos set to 50....

Created attachment 1521552 [details]
supervdsm.log

noob mistake, forgot to attach :-&

Created attachment 1521554 [details]
vdsm.log

Thanks for the logfiles. They show that the cluster has switchtype OVS.
Unfortunately QoS is not yet supported on clusters with switchtype OVS.
This is tracked in bug 1380271 and documented in
https://ovirt.org/develop/release-management/features/network/openvswitch/native-openvswitch.html#limitations .

To avoid the out-of-sync state either the switchtype of the cluster or the QoS has to be removed from the management network.

*** This bug has been marked as a duplicate of bug 1380271 ***

Hmm that will be a problem for us in production.

We have two clusters an old a new one. 

The old cluster has limited network bandwith and we have the customer vlans on the same interface of management. That's why we use the Qos.

Also because of all the switch configuration changes for customer networks we would like to use the OVS setup in the new cluster.

Currently as we cannot have both Qos and OVS in the same DC, we will have problems in the old cluster with customers able to use all the bandwith of the management NIC or the scaling issues with the LEGACY network mode....

Why is the Qos only for networks in the comlete DC and not for each cluster?

(In reply to Netbulae from comment #6)
> Hmm that will be a problem for us in production.
> 
> We have two clusters an old a new one. 
> 
> The old cluster has limited network bandwith and we have the customer vlans
> on the same interface of management. That's why we use the Qos.
> 
> Also because of all the switch configuration changes for customer networks
> we would like to use the OVS setup in the new cluster.
> 
> Currently as we cannot have both Qos and OVS in the same DC, we will have
> problems in the old cluster with customers able to use all the bandwith of
> the management NIC or the scaling issues with the LEGACY network mode....
> 
> Why is the Qos only for networks in the comlete DC and not for each cluster?

Would assigning the Management role to a new logical network (instead of ovirtmgmt)
just in the OVS cluster help?

Yes, thanks that works!

I had to remove all the nodes and reinstall them but I do that many times when testing anyway ;-)

(In reply to Netbulae from comment #8)
> Yes, thanks that works!
> 
> I had to remove all the nodes and reinstall them but I do that many times
> when testing anyway ;-)

Thanks for the feedback!

As this is a duplicate of a bug that has been closed for years, can we have QOS for OVS back on the planning at least?

If it's a "won't fix" then sombody should update the page at least:

https://ovirt.org/develop/release-management/features/network/openvswitch/native-openvswitch.html

Network Entities

In the following table, networking entities are listed with their implementation options.

Entity 	Kernel 	OVS 	Remark
QoS 	X 	X


And the link in "Host QoS - OvS supports different parameters for configuring QoS, which are incompatible with the ones used for a Linux bridge." brings me back to 

 bug 1380271


 Dan Kenigsberg 2016-12-05 06:24:36 UTC
 Deferring OvS bugs until they are deemed a priority again.

Also I have an issue removing the ovirtmgmt network assignment for the cluster "Error while executing action: The Management Network ('ovirtmgmt') is mandatory and cannot be removed"

I have it as non required now but it's still assigned.

(In reply to Netbulae from comment #11)
> Also I have an issue removing the ovirtmgmt network assignment for the
> cluster "Error while executing action: The Management Network ('ovirtmgmt')
> is mandatory and cannot be removed"
> 
> I have it as non required now but it's still assigned.

Please ensure that the role Management is assigned to another network in
Compute > Clusters > xxx > Logical Networks > Manage Networks

Yes I have it assigned to ovirtmgmt-OVS and I want to remove the original ovirtmgmt from this cluster so people don't mistakenly use that one when deploying more nodes..