Bug 1530944 - Add host failed with PSQLException if host has the same name server entry written twice in the resolv.conf
Summary: Add host failed with PSQLException if host has the same name server entry wri...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: ovirt-engine
Classification: oVirt
Component: BLL.Network
Version: 4.2.0
Hardware: x86_64
OS: Linux
medium
medium
Target Milestone: ovirt-4.2.2
: 4.2.2
Assignee: Alona Kaplan
QA Contact: Michael Burman
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-01-04 09:54 UTC by Michael Burman
Modified: 2018-03-29 11:02 UTC (History)
3 users (show)

Fixed In Version: ovirt-engine-4.2.2
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-03-29 11:02:14 UTC
oVirt Team: Network
Embargoed:
rule-engine: ovirt-4.2+


Attachments (Terms of Use)
Logs (430.35 KB, application/x-gzip)
2018-01-04 09:54 UTC, Michael Burman
no flags Details
too many name server enteries in db (474.29 KB, application/x-gzip)
2018-02-18 12:12 UTC, Michael Burman
no flags Details


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 87508 0 master MERGED engine: Run 'persistAndEnforceNetworkCompliance' in transaction 2018-02-13 22:02:11 UTC
oVirt gerrit 87606 0 ovirt-engine-4.2 MERGED engine: Run 'persistAndEnforceNetworkCompliance' in transaction 2018-02-14 08:46:03 UTC

Description Michael Burman 2018-01-04 09:54:13 UTC
Created attachment 1376750 [details]
Logs

Description of problem:
Add host failed with PSQLException if host has the same name server entry written twice in the resolv.conf.

If the resolv.conf has the same name server entry written twice in the conf, the installation will fail with a PSQLException on the engine.

PL/pgSQL function insertnameserver(uuid,character varying,smallint) line 3 at SQL statement; nested exception is org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "name_server_pkey"
  Detail: Key (dns_resolver_configuration_id, address)=(8c0362a2-00e4-48b5-89ee-8cbdb962baf3, 172.16.17.254) already exists.
  Where: SQL statement "INSERT INTO
    name_server(
      address,
      position,
      dns_resolver_configuration_id)
    VALUES (
      v_address,
      v_position,
      v_dns_resolver_configuration_id)"
PL/pgSQL function insertnameserver(uuid,character varying,smallint) line 3 at SQL statement
	at org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.doTranslate(SQLErrorCodeSQLExceptionTranslator.java:239) [spring-jdbc.jar:4.3.9.RELEASE]
	at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:73) [spring-jdbc.jar:4.3.9.RELEASE]
	at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:1099) [spring-jdbc.jar:4.3.9.RELEASE]
	at org.springframework.jdbc.core.JdbcTemplate.call(JdbcTemplate.java:1135) [spring-jdbc.jar:4.3.9.RELEASE]
	at org.springframework.jdbc.core.simple.AbstractJdbcCall.executeCallInternal(AbstractJdbcCall.java:405) [spring-jdbc.jar:4.3.9.RELEASE]
	at org.springframework.jdbc.core.simple.AbstractJdbcCall.doExecute(AbstractJdbcCall.java:365) [spring-jdbc.jar:4.3.9.RELEASE]
	at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:198) [spring-jdbc.jar:4.3.9.RELEASE]
	at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:135) [dal.jar:]
	at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:130) [dal.jar:]
	at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeModification(SimpleJdbcCallsHandler.java:76) [dal.jar:]
	at org.ovirt.engine.core.dao.network.DnsResolverConfigurationDaoImpl.saveNameServersByDnsResolverConfigurationId(DnsResolverConfigurationDaoImpl.java:66) [dal.jar:]
	at org.ovirt.engine.core.dao.network.DnsResolverConfigurationDaoImpl.save(DnsResolverConfigurationDaoImpl.java:97) [dal.jar:]
	at org.ovirt.engine.core.dao.network.DnsResolverConfigurationDaoImpl.save(DnsResolverConfigurationDaoImpl.java:15) [dal.jar:]
	at org.ovirt.engine.core.dao.VdsDynamicDaoImpl.updateDnsResolverConfiguration(VdsDynamicDaoImpl.java:150) [dal.jar:]
	at org.ovirt.engine.core.vdsbroker.vdsbroker.HostNetworkTopologyPersisterImpl.persistTopology(HostNetworkTopologyPersisterImpl.java:282) [vdsbroker.jar:]
	at org.ovirt.engine.core.vdsbroker.vdsbroker.HostNetworkTopologyPersisterImpl.persistAndEnforceNetworkCompliance(HostNetworkTopologyPersisterImpl.java:95) [vdsbroker.jar:]
	at org.ovirt.engine.core.vdsbroker.vdsbroker.HostNetworkTopologyPersisterImpl.persistAndEnforceNetworkCompliance(HostNetworkTopologyPersisterImpl.java:151) [vdsbroker.jar:]
	at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:684) [vdsbroker.jar:]
	at org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring.refreshVdsRunTimeInfo(HostMonitoring.java:118) [vdsbroker.jar:]
	at org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring.refresh(HostMonitoring.java:85) [vdsbroker.jar:]
	at org.ovirt.engine.core.vdsbroker.VdsManager.refresh(VdsManager.java:267) [vdsbroker.jar:]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [rt.jar:1.8.0_151]
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [rt.jar:1.8.0_151]
	at org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.access$201(ManagedScheduledThreadPoolExecutor.java:383) [javax.enterprise.concurrent.jar:1.0.0.redhat-1]
	at org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.run(ManagedScheduledThreadPoolExecutor.java:534) [javax.enterprise.concurrent.jar:1.0.0.redhat-1]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_151]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_151]
	at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_151]
	at org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:250) [javax.enterprise.concurrent.jar:1.0.0.redhat-1]
	at org.jboss.as.ee.concurrent.service.ElytronManagedThreadFactory$ElytronManagedThread.run(ElytronManagedThreadFactory.java:78)
Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "name_server_pkey"
  Detail: Key (dns_resolver_configuration_id, address)=(8c0362a2-00e4-48b5-89ee-8cbdb962baf3, 172.16.17.254) already exists.
  Where: SQL statement "INSERT INTO
    name_server(
      address,
      position,
      dns_resolver_configuration_id)
    VALUES (
      v_address,
      v_position,
      v_dns_resolver_configuration_id)"

Version-Release number of selected component (if applicable):
4.2.0.2-0.1.el7

How reproducible:
100%

Steps to Reproduce:
1. Try to add host with name server that written twice in the resolv.conf - 
# Generated by NetworkManager
search qa.lab.tlv.redhat.com lab.tlv.redhat.com tlv.redhat.com redhat.com
name server 8.8.8.8
name server 8.8.8.8

Actual results:
Add host failed with multiple exceptions in the engine. This will mess up the db unless removing the host from the engine.

Expected results:
Should work.

Comment 1 Yaniv Kaul 2018-01-04 11:42:49 UTC
Why would you have the same name server set up twice? Sounds like a user error to me? I would not bother fixing it.. (I assume it's an easy fix, but why complicate the code? Doesn't look like a common mistake but an edge case).

Please re-open if I'm wrong and there is such a legit use case for this.

Comment 2 Michael Burman 2018-01-04 11:54:16 UTC
(In reply to Yaniv Kaul from comment #1)
> Why would you have the same name server set up twice? Sounds like a user
> error to me? I would not bother fixing it.. (I assume it's an easy fix, but
> why complicate the code? Doesn't look like a common mistake but an edge
> case).
> 
> Please re-open if I'm wrong and there is such a legit use case for this.

I had 2 nodes provided to me by osp team to certify the ospd12, on the both nodes NetworkManager generated this name server in the resolv.conf, i have no idea why or how. 
Maybe it's a bug on NM side. any how i don't mind closing it. Dan and Alona told me to open it.

Comment 3 Carlos Rodrigues 2018-01-16 22:34:22 UTC
Hi,

I also have same issue but have have about 20 nodes and some ones have duplicate entries on /etc/resolv.conf and this cause the engine stays in loop and create too many entries on name_server table

engine=# select count(*) from name_server;
 count 
-------
 31401
(1 row)

This happens after upgrade of engine from 4.1.0 to 4.2.0 and after upgrade the hosts, i'm not able to regists the host. Its keeping to "activating"/"non-responsive".

After remove duplicate entries from /etc/resolv.conf, i'm able to regists the host but engine should be improved to prevent to stays in loop and create entries on name_server table.

I also know if i can delete 31401 entries from name_server table?

Regards

Comment 4 Michael Burman 2018-02-08 08:54:50 UTC
Hi

Yes you may delete the entries, make sure you are not deleting name_server of hosts that already running in your engine(if you have such).
Deleting the multiple entries in the DB + removing the duplicated name servers in /etc/resolv.conf should work around this bug, which was decided to close as WONTFIX. Just re-add the the server to engine.

Cheers)

Comment 5 Dan Kenigsberg 2018-02-11 11:08:41 UTC
It is not an important use case, and the host-side workaround is simple, but Engine must never get itself it such a horrible condition due to bad value in one host's /etc/resolv.conf.

At the very least, Engine should "uniquify" the dns values reported by Vdsm.

Comment 6 Alona Kaplan 2018-02-12 13:28:20 UTC
The creation of multiple entries in the 'name_server' and 'dns_resolver_configuration' was fixed.
Now, if the resolv.conf will contain duplicate entries the host will stuck in activating state and the log will contain errors regarding the duplicity but the db will be ok.

Comment 7 Michael Burman 2018-02-18 12:10:53 UTC
FailedQA - 4.2.2-0.1.el7 
db still not OK

engine=# select count(*) from name_server;
 count 
-------
  1908
(1 row)

engine=# select count(*) from name_server;
 count 
-------
  1910
(1 row)

Comment 8 Michael Burman 2018-02-18 12:12:44 UTC
Created attachment 1397560 [details]
too many name server enteries in db

Comment 9 Michael Burman 2018-02-18 15:30:22 UTC
I had an issue on my rhvm env and now it fixed. Retesting

Comment 10 Michael Burman 2018-02-20 15:16:58 UTC
Following comment6# - Verified on 4.2.2-0.1.el7

Comment 11 Sandro Bonazzola 2018-03-29 11:02:14 UTC
This bugzilla is included in oVirt 4.2.2 release, published on March 28th 2018.

Since the problem described in this bug report should be
resolved in oVirt 4.2.2 release, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.