Bug 715327 - vdsm: cannot login to node when there are more them 190 targets in iscsi db
Summary: vdsm: cannot login to node when there are more them 190 targets in iscsi db
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: vdsm
Version: 6.1
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: rc
: ---
Assignee: Dan Kenigsberg
QA Contact: Dafna Ron
URL:
Whiteboard: storage
Depends On: 715312
Blocks:
TreeView+ depends on / blocked
 
Reported: 2011-06-22 14:35 UTC by Dafna Ron
Modified: 2011-12-06 07:23 UTC (History)
9 users (show)

Fixed In Version: vdsm-4.9-79
Doc Type: Bug Fix
Doc Text:
Clone Of: 715312
Environment:
Last Closed: 2011-12-06 07:23:06 UTC
Target Upstream Version:


Attachments (Terms of Use)
vdsm log (731.00 KB, application/x-gzip)
2011-06-22 14:35 UTC, Dafna Ron
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2011:1782 0 normal SHIPPED_LIVE new packages: vdsm 2011-12-06 11:55:51 UTC

Description Dafna Ron 2011-06-22 14:35:57 UTC
Created attachment 506023 [details]
vdsm log

+++ This bug was initially created as a clone of Bug #715312 +++

This is a duplicated to a bug opened to iscsi where targets exceed 190 for session and we cannot login to node. 

vdsm error is Error during storage connection: Remove iSCSI node error
when in fact the actual error is that we are unable to login. 

As a result the host will become Non-Operational and backend will show RHEVM internal error message. 

Danken has emplimented a test fix on my host to allow non persistent  connections and cleaning /var/lib/iscsi/nodes/*/*/default

I was then able to connect. 

1) if we can issue some kind of fix we should since 190 targets per 1 connection is not that many. 
2) we should have a correct error message for this issue - where implementation in backend side will also offer a manual solution to administrator. 

Created attachment 506014 [details]
log

Description of problem:

working with the rhevm and iscsi I was trying to connect my host to my iscsi lun. 
I was getting an error for command: /sbin/iscsiadm -m node -T Dafna-blonds-Linux -l -p 10.35.64.10:3260

after some digging we found that we have exceeded 190 targets on the same connection and there for were unable to login. 
 
after removing the nodes under /var/lib/iscsi/nodes/*/*/default and discover again -  we were able to connect again to the storage. 

why are we limited to 190 targets? 


Version-Release number of selected component (if applicable):

iscsi-initiator-utils-6.2.0.872-21.el6.x86_64

How reproducible:


Steps to Reproduce:
1. exceed 190 targets under same connection and try to login to lun
2.
3.
  
Actual results:

you cannot login

Expected results:

exceed 190 nodes limit 

Additional info: vdsm log which can show the error we were getting

Comment 1 Dan Kenigsberg 2011-06-22 20:26:21 UTC
after

/sbin/iscsiadm -m node -T Dafna-blonds-Linux -l -p 10.35.64.10:3260

fails, we return

Thread-1245::INFO::2011-06-22 12:33:11,455::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: connectStorageServer, Return response: {'status': {'message': 'OK', 'code': 0}, 'statuslist': [{'status': 459, 'id': 'e2a868c7-40a4-4168-a9f9-6e31b3f73d3d'}]}

where RemoveiSCSINodeError==459. That's unfortunate - we should return iSCSILoginError==464.

That's a minor bug, but its solution is interesting:


From dec3d671cafbcd5c783f94c6f6b1d33e5e40c430 Mon Sep 17 00:00:00 2001
From: Dan Kenigsberg <danken>
Date: Wed, 22 Jun 2011 23:19:57 +0300
Subject: [PATCH] BZ#715327 connectStorageServer: report original error

When connectStorageServer: fails, we try to clean after ourselves.
However, the error code we report should reflect the original cause of
the error, not a consequential error during cleanup.

Change-Id: I032ee0e059228559698bdd14b1dfe360885c6e35
---
 vdsm/storage/iscsi.py |    4 +++-
 1 files changed, 3 insertions(+), 1 deletions(-)

diff --git a/vdsm/storage/iscsi.py b/vdsm/storage/iscsi.py
index 59a7573..75f1542 100644
--- a/vdsm/storage/iscsi.py
+++ b/vdsm/storage/iscsi.py
@@ -18,6 +18,7 @@ import tempfile
 import logging
 import socket
 import re
+import sys
 
 import constants
 import misc
@@ -471,6 +472,7 @@ def addiSCSINode(ip, port, iqn, tpgt, initiator, username=None, password=None):
         if rc != 0:
             raise se.iSCSILoginError(portal)
     except se.StorageException:
+        exc_class, exc, tb = sys.exc_info()
         try:
             if checkSession(ip, port, iqn, tpgt, username, password):
                 return 0
@@ -482,7 +484,7 @@ def addiSCSINode(ip, port, iqn, tpgt, initiator, username=None, password=None):
         except Exception:
             log.error("Could not remove iscsi node", exc_info=True)
 
-        raise
+        raise exc_class, exc, tb
 
 
 def remiSCSINode(ip, port, iqn, tpgt, username=None, password=None, logout=True):
-- 
1.7.4.4

Comment 2 Dan Kenigsberg 2011-06-23 11:03:31 UTC
http://gerrit.usersys.redhat.com/615

Comment 5 Dafna Ron 2011-07-14 15:35:32 UTC
wasn't able to reproduce. 
verified for vdsm-4.9-81

Comment 6 errata-xmlrpc 2011-12-06 07:23:06 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHEA-2011-1782.html


Note You need to log in before you can comment on or make changes to this bug.