+++ This bug was initially created as a clone of Bug #823397 +++ Description of problem: When having an Infiniband HCA VDSM fails to start. Version-Release number of selected component (if applicable): VDSM: From git repository commit 8a14b63fbbafbdb9ee7f85a9b702bff310f4f668 oVirt-engine: From git repository comit b24dddf094e08afa6a2032a487b37476318a872d How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: VDSM fails to starts Expected results: VDSM starts without errors. Additional info: Detailed log: 2012-05-11 03:07:11,292 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (QuartzScheduler_Worker-17) ResourceManager::refreshVdsRunTimeInfo::Failed to refresh VDS , vds = 783eb0ac-9a91-11e1-a8e4-000c29de7759 : xena003, error = DataIntegrityViolationException: CallableStatementCallback; SQL [{call insertvds_interface(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)}]; ERROR: value too long for type character varying(20) Where: SQL statement "INSERT INTO vds_interface(addr, bond_name, bond_type, gateway, id, is_bond, bond_opts, mac_addr, name, network_name, speed, subnet, boot_protocol, type, VDS_ID, vlan_id, mtu, bridged) VALUES(v_addr, v_bond_name, v_bond_type, v_gateway, v_id, v_is_bond, v_bond_opts, v_mac_addr, v_name, v_network_name, v_speed, v_subnet, v_boot_protocol, v_type, v_vds_id, v_vlan_id, v_mtu, v_bridged)" PL/pgSQL function "insertvds_interface" line 3 at SQL statement; nested exception is org.postgresql.util.PSQLException: ERROR: value too long for type character varying(20) Where: SQL statement "INSERT INTO vds_interface(addr, bond_name, bond_type, gateway, id, is_bond, bond_opts, mac_addr, name, network_name, speed, subnet, boot_protocol, type, VDS_ID, vlan_id, mtu, bridged) VALUES(v_addr, v_bond_name, v_bond_type, v_gateway, v_id, v_is_bond, v_bond_opts, v_mac_addr, v_name, v_network_name, v_speed, v_subnet, v_boot_protocol, v_type, v_vds_id, v_vlan_id, v_mtu, v_bridged)" PL/pgSQL function "insertvds_interface" line 3 at SQL statement, continuing. 2012-05-11 03:07:11,300 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (QuartzScheduler_Worker-17) ResourceManager::refreshVdsRunTimeInfo: org.springframework.dao.DataIntegrityViolationException: CallableStatementCallback; SQL [{call insertvds_interface(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)}]; ERROR: value too long for type character varying(20) Where: SQL statement "INSERT INTO vds_interface(addr, bond_name, bond_type, gateway, id, is_bond, bond_opts, mac_addr, name, network_name, speed, subnet, boot_protocol, type, VDS_ID, vlan_id, mtu, bridged) VALUES(v_addr, v_bond_name, v_bond_type, v_gateway, v_id, v_is_bond, v_bond_opts, v_mac_addr, v_name, v_network_name, v_speed, v_subnet, v_boot_protocol, v_type, v_vds_id, v_vlan_id, v_mtu, v_bridged)" PL/pgSQL function "insertvds_interface" line 3 at SQL statement; nested exception is org.postgresql.util.PSQLException: ERROR: value too long for type character varying(20) Where: SQL statement "INSERT INTO vds_interface(addr, bond_name, bond_type, gateway, id, is_bond, bond_opts, mac_addr, name, network_name, speed, subnet, boot_protocol, type, VDS_ID, vlan_id, mtu, bridged) VALUES(v_addr, v_bond_name, v_bond_type, v_gateway, v_id, v_is_bond, v_bond_opts, v_mac_addr, v_name, v_network_name, v_speed, v_subnet, v_boot_protocol, v_type, v_vds_id, v_vlan_id, v_mtu, v_bridged)" PL/pgSQL function "insertvds_interface" line 3 at SQL statement at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:100) [spring-jdbc-2.5.6.SEC02.jar:2.5.6.SEC02] at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc-2.5.6.SEC02.jar:2.5.6.SEC02] at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc-2.5.6.SEC02.jar:2.5.6.SEC02] at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc-2.5.6.SEC02.jar:2.5.6.SEC02] at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:952) [spring-jdbc-2.5.6.SEC02.jar:2.5.6.SEC02] at org.springframework.jdbc.core.JdbcTemplate.call(JdbcTemplate.java:985) [spring-jdbc-2.5.6.SEC02.jar:2.5.6.SEC02] at org.springframework.jdbc.core.simple.AbstractJdbcCall.executeCallInternal(AbstractJdbcCall.java:368) [spring-jdbc-2.5.6.SEC02.jar:2.5.6.SEC02] at org.springframework.jdbc.core.simple.AbstractJdbcCall.doExecute(AbstractJdbcCall.java:342) [spring-jdbc-2.5.6.SEC02.jar:2.5.6.SEC02] at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:164) [spring-jdbc-2.5.6.SEC02.jar:2.5.6.SEC02] at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:124) [engine-dal.jar:] at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeModification(SimpleJdbcCallsHandler.java:37) [engine-dal.jar:] at org.ovirt.engine.core.dao.InterfaceDAODbFacadeImpl.saveInterfaceForVds(InterfaceDAODbFacadeImpl.java:59) [engine-dal.jar:] at org.ovirt.engine.core.vdsbroker.vdsbroker.CollectVdsNetworkDataVDSCommand.persistTopology(CollectVdsNetworkDataVDSCommand.java:110) [vdsbroker-3.1.0-0001.jar:] at org.ovirt.engine.core.vdsbroker.vdsbroker.CollectVdsNetworkDataVDSCommand.persistAndEnforceNetworkCompliance(CollectVdsNetworkDataVDSCommand.java:50) [vdsbroker-3.1.0-0001.jar:] at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:551) [vdsbroker-3.1.0-0001.jar:] at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:412) [vdsbroker-3.1.0-0001.jar:] at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.Refresh(VdsUpdateRunTimeInfo.java:256) [vdsbroker-3.1.0-0001.jar:] at org.ovirt.engine.core.vdsbroker.VdsManager$1.runInTransaction(VdsManager.java:234) [vdsbroker-3.1.0-0001.jar:] at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:168) [engine-utils.jar:] at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:107) [engine-utils.jar:] at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:215) [vdsbroker-3.1.0-0001.jar:] at sun.reflect.GeneratedMethodAccessor134.invoke(Unknown Source) [:1.6.0_24] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.6.0_24] at java.lang.reflect.Method.invoke(Method.java:616) [rt.jar:1.6.0_24] at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:64) [engine-scheduler.jar:] at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz-2.1.2.jar:] at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz-2.1.2.jar:] Caused by: org.postgresql.util.PSQLException: ERROR: value too long for type character varying(20) Where: SQL statement "INSERT INTO vds_interface(addr, bond_name, bond_type, gateway, id, is_bond, bond_opts, mac_addr, name, network_name, speed, subnet, boot_protocol, type, VDS_ID, vlan_id, mtu, bridged) VALUES(v_addr, v_bond_name, v_bond_type, v_gateway, v_id, v_is_bond, v_bond_opts, v_mac_addr, v_name, v_network_name, v_speed, v_subnet, v_boot_protocol, v_type, v_vds_id, v_vlan_id, v_mtu, v_bridged)" PL/pgSQL function "insertvds_interface" line 3 at SQL statement at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2077) at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1810) at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257) at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:498) at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386) at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:379) at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.execute(CachedPreparedStatement.java:297) at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.execute(WrappedPreparedStatement.java:404) at org.springframework.jdbc.core.JdbcTemplate$5.doInCallableStatement(JdbcTemplate.java:987) [spring-jdbc-2.5.6.SEC02.jar:2.5.6.SEC02] at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:936) [spring-jdbc-2.5.6.SEC02.jar:2.5.6.SEC02] ... 22 more --- Additional comment from itzikb on 2012-05-21 03:15:01 EDT --- The problem is that the mac_addr field is VARCHAR(20).It should be VARCHAR(59). --- Additional comment from mkolesni on 2012-05-21 09:40:36 EDT --- Can you provide an example & regex for this type of address? Does it apply to all vNIC types, or only specific types? --- Additional comment from itzikb on 2012-05-22 04:10:59 EDT --- As for example: This is an example of the GUID(HWADDR) of a Infiniband HCA a0:50:00:58:fe:80:00:00:00:00:00:00:00:02:c9:03:00:0d:14:11 --- Additional comment from iheim on 2012-05-29 02:26:49 EDT --- what is the HWADDR itself (and why convert it to GUID? we keep mac addresses as strings today) --- Additional comment from itzikb on 2012-05-29 03:04:47 EDT --- The MAC(HWADDR) is generated by using HUID as described here: http://www.mellanox.com/related-docs/prod_software/guid2mac_checker_user_manual.txt --- Additional comment from acathrow on 2012-06-19 14:23:37 EDT --- If we increase the size to 59 then how do we waht to store a type of interface to in order to differentiate between IB and ethernet. --- Additional comment from itzikb on 2012-08-16 08:42:49 EDT --- Hi, As suggested by Roni L. I added the following line to /etc/vdsm/vdsm.conf: hidden_nics = ib* Restarted vdsmd and it's running without problems. Check with: vdsm-4.10.0-0.313.git8bedc7e.fc17.x86_64 ovirt-engine-3.1.0-2.fc17.noarch This is a workaround so there is still a need to decide whether to change the database schema or not. Itzik --- Additional comment from masayag on 2012-09-23 15:34:43 EDT --- *** Bug 857294 has been marked as a duplicate of this bug. ***
Steps for QA to test: In /usr/lib64/python2.6/site-packages/vdsm/netinfo.py file edit: def ifconfig(): """ Partial parser to ifconfig output """ p = subprocess.Popen([constants.EXT_IFCONFIG, '-a'], close_fds=True, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE) out, err = p.communicate() ifaces = {} for ifaceblock in out.split('\n\n'): if not ifaceblock: continue addr = netmask = hwaddr = '' for line in ifaceblock.splitlines(): if line[0] != ' ': ls = line.split() name = ls[0] if ls[2] == 'encap:Ethernet' and ls[3] == 'HWaddr': hwaddr = ls[4] if hwaddr == "00:14:5E:17:D4:66": hwaddr += ":00:14:5E:17:D4:66" if line.startswith(' inet addr:'): sp = line.split() for col in sp: if ':' not in col: continue k, v = col.split(':') if k == 'addr': addr = v if k == 'Mask': netmask = v ifaces[name] = {'addr': addr, 'netmask': netmask, 'hwaddr': hwaddr} return ifaces if hwaddr == "00:14:5E:17:D4:66": > This is my eth0 MAC hwaddr += ":00:14:5E:17:D4:66" > this will make eth0 MAC longer. This will cause VDSM to report fake MAC.
(In reply to comment #8) > Steps for QA to test: > > In /usr/lib64/python2.6/site-packages/vdsm/netinfo.py file edit: > since you know your nic name, you can alternatively update the dictionary. The following should be added in get() method of netinfo.py, after populating the d['nics'] : d['nics']['eth0']['hwaddr'] = 'a0:50:00:58:fe:80:00:00:00:00:00:00:00:02:c9:03:00:0d:14:11'
Verified on vdsm-4.9.6-38.0.el6_3.x86_64, rhevm-3.1.0-21.el6ev.noarch VDSM service is up and report MAC: 'hwaddr': 'a0:50:00:58:fe:80:00:00:00:00:00:00:00:02:c9:03:00:0d:14:11' engine DB: 58d4db85-9c24-4d0f-aaf7-209a8141743a | eth0 | | ed5f73d4-1c29-11e2-93c2-001122334455 | a0:50:00:58:fe:80:00:00:00:00:00:00:00:02:c9:03:00:0d:14:11 UI have the full MAC as tooltip.