Bug 1323465 - Setup Networks cause ClosedChannelException
Summary: Setup Networks cause ClosedChannelException
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: ovirt-engine
Classification: oVirt
Component: Backend.Core
Version: 3.6.0
Hardware: x86_64
OS: Linux
urgent
medium
Target Milestone: ovirt-3.6.6
: 3.6.6
Assignee: Piotr Kliczewski
QA Contact: Pavol Brilla
URL:
Whiteboard:
: 1324155 1330383 (view as bug list)
Depends On:
Blocks: 1330383 1331396
TreeView+ depends on / blocked
 
Reported: 2016-04-03 10:35 UTC by Michael Burman
Modified: 2021-08-30 12:18 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
: 1331396 (view as bug list)
Environment:
Last Closed: 2016-05-30 10:53:14 UTC
oVirt Team: Infra
Embargoed:
rule-engine: ovirt-3.6.z+
mgoldboi: planning_ack+
mperina: devel_ack+
pstehlik: testing_ack+


Attachments (Terms of Use)
engine log (349.14 KB, application/x-gzip)
2016-04-03 10:35 UTC, Michael Burman
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHV-43226 0 None None None 2021-08-30 12:18:27 UTC
Red Hat Knowledge Base (Solution) 2276551 0 None None None 2019-11-14 07:51:50 UTC
oVirt gerrit 55681 0 'None' 'MERGED' 'timing issue during setup networks' 2019-11-14 07:56:24 UTC
oVirt gerrit 55855 0 'None' 'MERGED' 'vdsbroker: wait for reconnection' 2019-11-14 07:56:24 UTC
oVirt gerrit 55940 0 'None' 'MERGED' 'timing issue during setup networks' 2019-11-14 07:56:24 UTC

Description Michael Burman 2016-04-03 10:35:35 UTC
Created attachment 1142958 [details]
engine log

Description of problem:
Setup Networks operation hangs out for ever and can't be rolled back when changing the ovirtmgmt's interface.
To recover from it we will need to restart the ovirt-engine service.

When changing the ovritmgmt interface via setup networks, operation can't be finished and there is no roll back to last known configuration.

Sometimes operation failing with communication error with engine and host, host moving to connecting state. In the UI operation seems to be failing, although  the change is took place on the host.

Sometimes operation succeeded without any errors.

Sometimes operation failing with connection to peer error and then host rolls back to last network configuration after a very long time out.

Version-Release number of selected component (if applicable):
3.6.5-0.1.el6.noarch

Steps to Reproduce:
1. Move the ovirtmgmt network to other interface on host via setup networks dialog


Actual results:
- Setup Networks operation hangs out for ever and can't be rolled back. Restarting the ovirt-engine service will recover the engine. 

- Operation failed with communication error in the UI engine, but the change took place on host.

- Operation failed with connection to peer error in the UI engine. The change didn't took place on the host and host rolled back to latest network configurations as expected.  

- Operation succeeded. 

Expected results:
In case when we should receive the same IP on the other interface, operation should succeed. 

In case we should receive different IP, we should fail and host should roll back to last configurations. 

Setup Networks operation shouldn't hang out fore ever without failing. In case of any communication or connection issues between host and engine, host should be rolled back to last known network configuration.

Comment 1 Michael Burman 2016-04-03 10:45:42 UTC
Please note, that when engine stuck in such state as described above(hanging for ever with out any time out or failure), we can't approve any other setup networks operations --> "cannot setup networks. another setup network process in progress  on the host". To recover form it we need to restart the engine.

Comment 2 Piotr Kliczewski 2016-04-04 13:57:29 UTC
During setupNetworks we get:

2016-04-04 16:20:57,184 DEBUG [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) [] Unable to process messages: java.nio.channels.ClosedChannelException
        at sun.nio.ch.SocketChannelImpl.ensureReadOpen(SocketChannelImpl.java:257) [rt.jar:1.8.0_71]
        at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:300) [rt.jar:1.8.0_71]
        at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.read(SSLEngineNioHelper.java:50) [vdsm-jsonrpc-java-client.jar:]
        at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.read(SSLClient.java:91) [vdsm-jsonrpc-java-client.jar:]
        at org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient.processIncoming(StompCommonClient.java:103) [vdsm-jsonrpc-java-client.jar:]
        at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.process(ReactorClient.java:204) [vdsm-jsonrpc-java-client.jar:]
        at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:125) [vdsm-jsonrpc-java-client.jar:]
        at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:89) [vdsm-jsonrpc-java-client.jar:]
        at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:65) [vdsm-jsonrpc-java-client.jar:]


which causes:

2016-04-04 16:20:57,249 DEBUG [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) [] Internal server error: java.lang.NullPointerException

Comment 3 Red Hat Bugzilla Rules Engine 2016-04-04 14:52:39 UTC
Bug tickets must have version flags set prior to targeting them to a release. Please ask maintainer to set the correct version flags and only then set the target milestone.

Comment 4 Red Hat Bugzilla Rules Engine 2016-04-04 14:52:39 UTC
Target release should be placed once a package build is known to fix a issue. Since this bug is not modified, the target version has been reset. Please use target milestone to plan a fix for a oVirt release.

Comment 5 Pavol Brilla 2016-05-11 16:02:37 UTC
verified on rhevm-3.6.6.2-0.1.el6.noarch

Comment 6 Martin Perina 2016-05-20 11:53:46 UTC
*** Bug 1324155 has been marked as a duplicate of this bug. ***

Comment 7 Martin Perina 2016-06-06 07:53:13 UTC
*** Bug 1330383 has been marked as a duplicate of this bug. ***


Note You need to log in before you can comment on or make changes to this bug.