Bug 1154862
| Summary: | RHEV-H upgrade fail with ssh timeout error when trying to copy the ISO to the host | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Virtualization Manager | Reporter: | Marina Kalinin <mkalinin> |
| Component: | ovirt-engine | Assignee: | Nobody <nobody> |
| Status: | CLOSED DUPLICATE | QA Contact: | |
| Severity: | medium | Docs Contact: | |
| Priority: | medium | ||
| Version: | 3.4.1-1 | CC: | ecohen, fdeutsch, gklein, iheim, lpeer, lsurette, oourfali, rbalakri, Rhev-m-bugs, yeylon |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2014-10-21 08:44:23 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Fabian - this sounds familiar to me. Do you happen to remember this issue? oved, yes, I believe this is a dupe of bug 1093882 *** This bug has been marked as a duplicate of bug 1093882 *** |
Description of problem: RHEV-H upgrade fail with ssh timeout error when trying to copy the ISO to the host. Version-Release number of selected component (if applicable): rhevh-6.5-20140930.1.el6ev.iso rhevm-3.4.2-0.2.el6ev.noarch How reproducible: Always reproducible on my environment. But I didn't hear about other cases. Steps to Reproduce: 1. Put host in maintenance. 2. Click "Upgrade" link, use the default option to use 'SSH publick key' option -> Ok. 3. Installation would fail with time out error: Host rhevh4.example.com installation failed. SSH session timeout host 'root.10.11'. Additional info: From engine.log: ~~~ 2014-10-20 16:22:40,079 INFO [org.ovirt.engine.core.bll.InstallerMessages] (org.ovirt.thread.pool-4-thread-10) [5591cfac] Installation 10.10.10.11: Connected to host 10.10.10.11 with SSH key fingerprint: 18:ab:2c:a5:ea:75:d7:0b:80:3f:df:f2:9f:db:00:0b 2014-10-20 16:22:40,086 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-4-thread-10) [5591cfac] Correlation ID: 5591cfac, Call Stack: null, Custom Event ID: -1, Message: Installing Host rhevh4.example.com. Connected to host 10.10.10.11 with SSH key fingerprint: 18:ab:2c:a5:ea:75:d7:0b:80:3f:df:f2:9f:db:00:0b. 2014-10-20 16:22:40,119 INFO [org.ovirt.engine.core.bll.InstallerMessages] (org.ovirt.thread.pool-4-thread-10) [5591cfac] Installation 10.10.10.11: Sending file /usr/share/rhev-hypervisor/rhevh-6.5-20140930.1.el6ev.iso to /data/updates/ovirt-node-image.iso 2014-10-20 16:22:40,127 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-4-thread-10) [5591cfac] Correlation ID: 5591cfac, Call Stack: null, Custom Event ID: -1, Message: Installing Host rhevh4.example.com. Sending file /usr/share/rhev-hypervisor/rhevh-6.5-20140930.1.el6ev.iso to /data/updates/ovirt-node-image.iso. 2014-10-20 16:22:40,127 INFO [org.ovirt.engine.core.utils.ssh.SSHDialog] (org.ovirt.thread.pool-4-thread-10) SSH execute root.10.11 'mkdir -p '/data/updates' 2014-10-20 16:27:40,218 ERROR [org.ovirt.engine.core.bll.OVirtNodeUpgrade] (org.ovirt.thread.pool-4-thread-10) [5591cfac] Timeout during node 10.10.10.11 upgrade: javax.naming.TimeLimitExceede dException: SSH session timeout host 'root.10.11' at org.ovirt.engine.core.utils.ssh.SSHClient.executeCommand(SSHClient.java:499) [utils.jar:] at org.ovirt.engine.core.utils.ssh.SSHClient.sendFile(SSHClient.java:633) [utils.jar:] at org.ovirt.engine.core.utils.ssh.SSHDialog.sendFile(SSHDialog.java:374) [utils.jar:] at org.ovirt.engine.core.bll.OVirtNodeUpgrade.execute(OVirtNodeUpgrade.java:200) [bll.jar:] at org.ovirt.engine.core.bll.InstallVdsCommand.upgradeNode(InstallVdsCommand.java:287) [bll.jar:] at org.ovirt.engine.core.bll.InstallVdsCommand.executeCommand(InstallVdsCommand.java:154) [bll.jar:] at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1133) [bll.jar:] at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1218) [bll.jar:] at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1894) [bll.jar:] at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1238) [bll.jar:] at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:351) [bll.jar:] at org.ovirt.engine.core.bll.MultipleActionsRunner.executeValidatedCommand(MultipleActionsRunner.java:189) [bll.jar:] at org.ovirt.engine.core.bll.MultipleActionsRunner.runCommands(MultipleActionsRunner.java:156) [bll.jar:] at org.ovirt.engine.core.bll.MultipleActionsRunner$2.run(MultipleActionsRunner.java:165) [bll.jar:] at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:97) [utils.jar:] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_65] at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_65] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_65] at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_65] ~~~ - I do not see any ssh connection from the manager opened to the host: # netstat -plant |grep sshd | grep <rhevm-fqdn> - The connection is good, the copy is very fast when connected. - I can ssh to the host from the manager. - When running upgrade, on the host I can see: # grep 'rhevm-ip' /var/log/secure Oct 20 21:00:14 rhevh-4 sshd[32340]: Accepted publickey for root from <rhevm-ip> port 41707 ssh2 - On the manager, sslv3 is disabled on httpd (to prevent poodle attack). Also, sometimes, after I ssh from the manager to the host manually, the upgrade succeeds. Maybe it is something wrong with my specific environment, I do not know. I would appreciate some hints understanding this problem.