Bugzilla will be upgraded to version 5.0. The upgrade date is tentatively scheduled for 2 December 2018, pending final testing and feedback.
Bug 1401421 - [z-stream clone - 4.0.7] api.vms.get(name) sometimes returns None when the vm exists
[z-stream clone - 4.0.7] api.vms.get(name) sometimes returns None when the vm...
Status: CLOSED ERRATA
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-engine (Show other bugs)
3.6.6
Unspecified Unspecified
medium Severity medium
: ovirt-4.0.7
: ---
Assigned To: Ravi Nori
Gonza
: Reopened, ZStream
Depends On: 1301067
Blocks:
  Show dependency treegraph
 
Reported: 2016-12-05 03:30 EST by rhev-integ
Modified: 2017-03-16 11:30 EDT (History)
17 users (show)

See Also:
Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1301067
Environment:
Last Closed: 2017-03-16 11:30:06 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: Infra
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2017:0542 normal SHIPPED_LIVE Red Hat Virtualization Manager 4.0.7 2017-03-16 15:25:04 EDT

  None (edit)
Description rhev-integ 2016-12-05 03:30:32 EST
+++ This bug is a downstream clone. The original bug is: +++
+++   bug 1301067 +++
======================================================================

Description of problem:
"""
api = API(url=self.url,
          kerberos=True,
          ca_file=self.ca_file,
          filter=True
      )

name = "some-vm-name"
vm = api.vms.get(name)
vm2 = api.vms.get(name)
vm3 = api.vms.get(name)
"""

quite often(subjective, no real measurement) in can happen that one or more of vm, vm2 or vm3 contain None but others have the desired vm object.


Version-Release number of selected component (if applicable):
rpm -qa "*ovirt*"
ovirt-engine-sdk-java-javadoc-3.5.1.0-1.fc22.noarch
libgovirt-0.3.3-1.fc22.x86_64
ovirt-engine-sdk-python-3.6.0.3-1.fc22.noarch
ovirt-engine-cli-3.6.0.2-1.fc22.noarch

server is: Version 3.6.1-0.2.el6


How reproducible:
not always

Steps to Reproduce:
1. see description


Actual results:
api.vms.get(name) sometimes returns None when the vm exists

Expected results:
api.vms.get(name) always returns vm object if the vm exists

Additional info:
If it returns None because of some external error(network, auth issue). Then the API should raise an exception and not return None.

(Originally by Petr Vobornik)
Comment 1 rhev-integ 2016-12-05 03:30:44 EST
same with:
  api.vms.list(query=query)

same call multiple times, different results

(Originally by Petr Vobornik)
Comment 2 rhev-integ 2016-12-05 03:30:52 EST
Any chances you can repeat this with the "debug=True" parameter and share the results? Otherwise it is very difficult to determine if this is caused by a server side issue or by an SDK issue. Also, it would be helpful to have also the server side logs.

(Originally by juan.hernandez)
Comment 3 rhev-integ 2016-12-05 03:31:00 EST
*** Bug 1301066 has been marked as a duplicate of this bug. ***

(Originally by juan.hernandez)
Comment 5 rhev-integ 2016-12-05 03:31:14 EST
attachment 1117228 [details] contains third run of test(failed at 15th attempt). Ve can see there that the result was as follows - so probably issue on server:

 <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
 <vms/>

test:
        name = self.args.name
        for i in range(0, 30):
            vm = self.backend.api.vms.get(name)
            if not vm:
                print "ERROR: attempt failed: %s" % i
                return
            else:
                print "INFO: attempt successful: %s" % i
        return

- first run failed at 2nd attempt
- second run was successful - all 30 attempts

I don't have access to server logs.

(Originally by Petr Vobornik)
Comment 6 rhev-integ 2016-12-05 03:31:21 EST
Yes, looks like a server side issue. Can you repeat the same test using basic (user name and password) authentication instead of Kerberos?

(Originally by juan.hernandez)
Comment 7 rhev-integ 2016-12-05 03:31:28 EST
the API endpoint in the previous example can be used only with Kerberos auth. Luckily the same server has also different API url where basic auth is allowed.

There, I(user which were used with kerberos auth) don't have a the required rights, got:
  status: 400
  reason: Bad Request
  detail: query execution failed due to insufficient permissions.

But the test seems to pass with basic auth, when a different user which was given permissions for the vm is used.

I've tried 500 attempts with the different user and all were successful.

(Originally by Petr Vobornik)
Comment 8 rhev-integ 2016-12-05 03:31:36 EST
OK, it is an server side authentication issue then, I'm moving to the AAA component.

(Originally by juan.hernandez)
Comment 9 rhev-integ 2016-12-05 03:31:43 EST
Ravi - thoughts?

(Originally by Oved Ourfali)
Comment 10 rhev-integ 2016-12-05 03:31:50 EST
Sounds like a network issue performing auth, I will try to reproduce it

(Originally by Ravi Nori)
Comment 11 rhev-integ 2016-12-05 03:31:57 EST
This bug is marked for z-stream, yet the milestone is for a major version, therefore the milestone has been reset.
Please set the correct milestone or drop the z stream flag.

(Originally by rule-engine)
Comment 12 rhev-integ 2016-12-05 03:32:05 EST
I am unable to reproduce this on Master and 3.6 branches (3.6.3 and 3.6.1).
From the attached test run output it does not look like negotiate is failing, just that empty vm list is returned.

Please check on the latest 3.6 and see if it is reproducible, also double check the users permissions.

(Originally by Ravi Nori)
Comment 14 rhev-integ 2016-12-05 03:32:19 EST
The server was upgraded to 3.6.2.6-0.1.el6 in a meanwhile.

I can still reproduce it, see attachment 1128524 [details] 

Tried 3 times.
1. first 30 was OK.
2. failed on 12th attempt (attached)
3. failed on 52nd attempt (max attempt was raised)

I have no idea what user permissions to check. I'm not admin of the server.

(Originally by Petr Vobornik)
Comment 17 rhev-integ 2016-12-05 03:32:39 EST
Do you use mod_auth_kerb? If yes, is it possible for you to test with mod_auth_gssapi?

(Originally by Ondra Machacek)
Comment 18 rhev-integ 2016-12-05 03:32:46 EST
mod_auth_gssapi is available only on RHEL 7 but AFAIK RHEL 7 is not supported with RHEV-M so the answer is no, we cannot test it.

(Originally by Petr Spacek)
Comment 19 rhev-integ 2016-12-05 03:32:54 EST
As it doesn't happen on any other environment, and we can't seem to reproduce it in any way, I'm closing this one as wontfix.
If you're able to reproduce in a new clean environment please let us know.

(Originally by Oved Ourfali)
Comment 26 rhev-integ 2016-12-05 03:33:41 EST
Since we are unable to reproduce this on the listed environment I am closing this as WORKSFORME, please feel free to reopen if the problem reoccurs

(Originally by Ravi Nori)
Comment 27 rhev-integ 2016-12-05 03:33:49 EST
FYI I tried the reproducer from comment 16(only had to change api uri and certificate because it changed since that time, it is updated on the vm) and it reproduced. 

So it si still present on Version 4.0.4.4-0.1.el7ev

If you have issue with it, just ping me.


Reproducer output:
./reproducer-1301067.py 
<sanitized part will all my vms>
Fetching pvoborni-f24-a-ipa1
INFO: attempt successful: 0
INFO: attempt successful: 1
INFO: attempt successful: 2
INFO: attempt successful: 3
INFO: attempt successful: 4
INFO: attempt successful: 5
INFO: attempt successful: 6
INFO: attempt successful: 7
INFO: attempt successful: 8
INFO: attempt successful: 9
INFO: attempt successful: 10
INFO: attempt successful: 11
INFO: attempt successful: 12
INFO: attempt successful: 13
INFO: attempt successful: 14
ERROR: attempt failed: 15

(Originally by Petr Vobornik)
Comment 28 rhev-integ 2016-12-05 03:33:56 EST
This is most probably has the same cause as BZ1396833, so marking as dependency

(Originally by Martin Perina)
Comment 30 Gonza 2017-02-16 10:08:29 EST
Verified with:
python-ovirt-engine-sdk4-4.1.0-1.el7ev.x86_64
Comment 32 errata-xmlrpc 2017-03-16 11:30:06 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2017-0542.html

Note You need to log in before you can comment on or make changes to this bug.