Bug 1025320

Summary: backend search returns only first page when given max=-1
Product: Red Hat Enterprise Virtualization Manager Reporter: Eli Mesika <emesika>
Component: ovirt-engineAssignee: Eli Mesika <emesika>
Status: CLOSED CURRENTRELEASE QA Contact: Jiri Belka <jbelka>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 3.3.0CC: aberezin, acathrow, ederevea, emesika, iheim, lpeer, lyarwood, mkalinin, pablo.iranzo, pbrilla, pep, Rhev-m-bugs, sbonazzo, talayan, tdosek, yeylon
Target Milestone: ---   
Target Release: 3.4.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: infra
Fixed In Version: ovirt-3.4.0-alpha1 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Infra RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Eli Mesika 2013-10-31 13:33:20 UTC
Description of problem:
backend search returns only first page when given max=-1
In that case, backend should return all the records and not only the 1st page as defined by the SearchResultsLimit config value 

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.Add more than 100 faked entities (Hosts, VMs etc)
2.Issue a search query with 'max=-1'
3.

Actual results:
Only first 100 records are retrieved 

Expected results:
All records should be reetrieved 

Additional info:

Comment 1 Sandro Bonazzola 2014-01-14 08:42:39 UTC
ovirt 3.4.0 alpha has been released

Comment 2 Jiri Belka 2014-03-04 17:47:37 UTC
What to test here please?

Comment 3 Eli Mesika 2014-03-05 09:13:49 UTC
See BZ description 

You need to have more than 100 entities (for example : Events)

then do either from REST API or UI search , you will get the 1st 100 chunk
if you send with max=-1 , you shoul get all records

Comment 4 Jiri Belka 2014-03-06 14:50:39 UTC
There's no difference between using 'max=-1' or not using it.

Provide exact steps to verify this BZ or I would need to make it FailedQA.

- 150 vms listed

# curl -k -v -u "admin@internal:123456" -X GET 'https://10.34.60.228/api/vms?search=name=*' 2>/dev/null | grep '<name>t' | perl -e 'use Sort::Naturally; @them=nsort(<STDIN>); print @them;' | tail
        <name>tcms141</name>
        <name>tcms142</name>
        <name>tcms143</name>
        <name>tcms144</name>
        <name>tcms145</name>
        <name>tcms146</name>
        <name>tcms147</name>
        <name>tcms148</name>
        <name>tcms149</name>
        <name>tcms150</name>

- 150 vms listed

# curl -k -v -u "admin@internal:123456" -X GET 'https://10.34.60.228/api/vms?search=name=*&max=-1' 2>/dev/null | grep '<name>t' | perl -e 'use Sort::Naturally; @them=nsort(<STDIN>); print @them;' | tail
        <name>tcms141</name>
        <name>tcms142</name>
        <name>tcms143</name>
        <name>tcms144</name>
        <name>tcms145</name>
        <name>tcms146</name>
        <name>tcms147</name>
        <name>tcms148</name>
        <name>tcms149</name>
        <name>tcms150</name>

Comment 5 Eli Mesika 2014-03-13 01:04:03 UTC
Jiri 

1) You are not using right the max parameter, it should be a URL parameter (";max=<value>" not "?max=<value>"

2) There should not be difference between sending with or without ;max=-1 , the default for API is -1

3) Try to use other values of max , the following examples works with the examples you had provided above

-- show that max works if given a value that is different than -1
 curl -k -v -u "admin@internal:123456" -X GET 'https://10.34.60.228/api/vms?search=name=*;max=1' 2>/dev/null | grep '<name>t' | perl -e 'use Sort::Naturally; @them=nsort(<STDIN>); print @them;' | tail
[emesika@localhost [oVirt] ovirt (1025320)]$ curl -k -v -u "admin@internal:123456" -X GET 'https://10.34.60.228/api/vms;max=1' 2>/dev/null | grep '<name>t' | perl -e 'use Sort::Naturally; @them=nsort(<STDIN>); print @them;' | tail

        <name>tcms1</name>


-- The following 2 examples show that there is no diff between giving max=-1 or omitting it --

[emesika@localhost [oVirt] ovirt (1025320)]$ curl -k -v -u "admin@internal:123456" -X GET 'https://10.34.60.228/api/vms;max=-1' 2>/dev/null | grep '<name>t' | perl -e 'use Sort::Naturally; @them=nsort(<STDIN>); print @them;' | tail
        <name>tcms141</name>
        <name>tcms142</name>
        <name>tcms143</name>
        <name>tcms144</name>
        <name>tcms145</name>
        <name>tcms146</name>
        <name>tcms147</name>
        <name>tcms148</name>
        <name>tcms149</name>
        <name>tcms150</name>


[emesika@localhost [oVirt] ovirt (1025320)]$ curl -k -v -u "admin@internal:123456" -X GET 'https://10.34.60.228/api/vms' 2>/dev/null | grep '<name>t' | perl -e 'use Sort::Naturally; @them=nsort(<STDIN>); print @them;' | tail
        <name>tcms141</name>
        <name>tcms142</name>
        <name>tcms143</name>
        <name>tcms144</name>
        <name>tcms145</name>
        <name>tcms146</name>
        <name>tcms147</name>
        <name>tcms148</name>
        <name>tcms149</name>
        <name>tcms150</name>

Comment 6 Jiri Belka 2014-03-13 09:40:38 UTC
ok, rhevm-backend-3.4.0-0.3.master.el6ev.noarch

* no max, thus all

# curl -k -v -u "admin@internal:123456" -X GET 'https://localhost/api/vms?search=name=tcms2*' 2>/dev/null | grep '<name>t' | p
erl -e 'use Sort::Naturally; @them=nsort(<STDIN>); print @them;' | nl
     1          <name>tcms2</name>
     2          <name>tcms20</name>
     3          <name>tcms21</name>
     4          <name>tcms22</name>
     5          <name>tcms23</name>
     6          <name>tcms24</name>
     7          <name>tcms25</name>
     8          <name>tcms26</name>
     9          <name>tcms27</name>
    10          <name>tcms28</name>
    11          <name>tcms29</name>

* max=5

# curl -k -v -u "admin@internal:123456" -X GET 'https://localhost/api/vms;max=5?search=name=tcms2*' 2>/dev/null | grep '<name>t' | perl -e 'use Sort::Naturally; @them=nsort(<STDIN>); print @them;' | nl
     1          <name>tcms2</name>
     2          <name>tcms20</name>
     3          <name>tcms21</name>
     4          <name>tcms22</name>
     5          <name>tcms23</name>

* max=0

# curl -k -v -u "admin@internal:123456" -X GET 'https://localhost/api/vms;max=0?search=name=tcms2*' 2>/dev/null | grep '<name>t' | perl -e 'use Sort::Naturally; @them=nsort(<STDIN>); print @them;' | nl

* max=-1 (all)

# curl -k -v -u "admin@internal:123456" -X GET 'https://localhost/api/vms;max=-1?search=name=tcms2*' 2>/dev/null | grep '<name>t' | perl -e 'use Sort::Naturally; @them=nsort(<STDIN>); print @them;' | nl
     1          <name>tcms2</name>
     2          <name>tcms20</name>
     3          <name>tcms21</name>
     4          <name>tcms22</name>
     5          <name>tcms23</name>
     6          <name>tcms24</name>
     7          <name>tcms25</name>
     8          <name>tcms26</name>
     9          <name>tcms27</name>
    10          <name>tcms28</name>
    11          <name>tcms29</name>

Comment 7 Juan Hernández 2014-05-20 15:35:15 UTC
*** Bug 1047923 has been marked as a duplicate of this bug. ***

Comment 8 Pablo Iranzo Gómez 2014-05-23 11:19:06 UTC
Hi,
After speaking with Juan, this could potentially ask for all elements on api scripts when using .list() as this will be ignoring maximum page size.

A release note is IMHO required, to let users know the required changes on their scripts to avoid hitting performance issues and/or loading too much clients/server.

Regards,
Pablo

Comment 11 Itamar Heim 2014-06-12 14:08:26 UTC
Closing as part of 3.4.0