Bug 453902 - buglist.cgi ctype=csv dependson list truncated
buglist.cgi ctype=csv dependson list truncated
Status: CLOSED NEXTRELEASE
Product: Bugzilla
Classification: Community
Component: Bugzilla General (Show other bugs)
3.2
All Linux
low Severity medium (vote)
: ---
: ---
Assigned To: PnT DevOps Devs
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2008-07-02 23:13 EDT by FTBFS
Modified: 2013-06-23 22:26 EDT (History)
2 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2008-08-28 00:29:43 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description FTBFS 2008-07-02 23:13:47 EDT
Description of problem:
Querying using buglist.csv?ctype=csv and using a cookie that requests the
'dependson' field, the results list is incomplete.

Example:
#!/bin/bash
cookie='COLUMNLIST=alias%20bug_severity%20priority%20rep_platform%20assigned_to%20bug_status%20resolution%20component%20short_desc%20dependson'
url='https://bugzilla.redhat.com/buglist.cgi?bug_id=440169&ctype=csv'
/usr/bin/curl -k --cookie $cookie $url

resulting csv list has 128 entries in the dependson list.


The same call using python-bugzilla's bugzilla program and xmlrpc call:
$ bugzilla --verbose --debug query -f -b 440169

yields a list with 452 entries.

Version-Release number of selected component (if applicable):
bugzilla 2.18

How reproducible:
always
Comment 1 FTBFS 2008-07-02 23:14:23 EDT
Doh.  I should have filed this under my name. -Matt
Comment 2 Noura El hawary 2008-07-03 02:46:01 EDT
Problem verified, trying the url:

https://bugzilla.redhat.com/buglist.cgi?bug_id=440169&field-1-0-0=bug_id&query_format=advanced&remaction=&type-1-0-0=anyexact&value-1-0-0=440169&query_based_on=&columnlist=bug_severity%2Cpriority%2Cassigned_to%2Cbug_status%2Cresolution%2Cop_sys%2Cdependson%2Cshort_desc

the list returns only 128 bugs in the dependson column which is not the complete
list, I am assuming the problem is coming from the Function bugs.DependsOnList
created using the file bugs-functions.mysql, maybe we need to enlarge the TEXT
to be MEDIUMTEXT or LONGTEXT !!

Noura
Comment 3 Matt Domsch 2008-08-27 21:31:06 EDT
ping
Comment 4 David Lawrence 2008-08-28 00:29:43 EDT
(In reply to comment #2)
> Problem verified, trying the url:
> 
> https://bugzilla.redhat.com/buglist.cgi?bug_id=440169&field-1-0-0=bug_id&query_format=advanced&remaction=&type-1-0-0=anyexact&value-1-0-0=440169&query_based_on=&columnlist=bug_severity%2Cpriority%2Cassigned_to%2Cbug_status%2Cresolution%2Cop_sys%2Cdependson%2Cshort_desc
> 
> the list returns only 128 bugs in the dependson column which is not the complete
> list, I am assuming the problem is coming from the Function bugs.DependsOnList
> created using the file bugs-functions.mysql, maybe we need to enlarge the TEXT
> to be MEDIUMTEXT or LONGTEXT !!

Actually according to the MySQL docs, the GROUP_CONCAT function is limited by the GROUP_CONCAT_MAX_LEN setting which defaults to 1024.

"The result is truncated to the maximum length that is given by the group_concat_max_len system variable, which has a default value of 1024. The value can be set higher, although the effective maximum length of the return value is constrained by the value of max_allowed_packet. The syntax to change the value of group_concat_max_len at runtime is as follows, where val  is an unsigned integer:

SET [SESSION | GLOBAL] group_concat_max_len = val;

Beginning with MySQL 5.0.19, the type returned by GROUP_CONCAT() is always VARCHAR unless group_concat_max_len is greater than 512, in which case, it returns a BLOB. (Previously, it returned a BLOB with group_concat_max_len greater than 512 only if the query included an ORDER BY clause.)"

So I have commited the following change to CVS which fixes the truncation issue.

Index: Bugzilla/DB/Mysql.pm
===================================================================
RCS file: /cvs/qa/rh_bugzilla_3/Bugzilla/DB/Mysql.pm,v
retrieving revision 1.17
diff -u -r1.17 Mysql.pm
--- Bugzilla/DB/Mysql.pm	13 Aug 2008 19:24:55 -0000	1.17
+++ Bugzilla/DB/Mysql.pm	28 Aug 2008 04:24:57 -0000
@@ -67,6 +67,10 @@
     # return their data correctly.
     $self->do("SET NAMES utf8") if Bugzilla->params->{'utf8'};
 
+    # REDHAT EXTENSION START 453902 
+    $self->do("SET GROUP_CONCAT_MAX_LEN = 65536;"); 
+    # REDHAT EXTENSION END 453902
+
     # all class local variables stored in DBI derived class needs to have
     # a prefix 'private_'. See DBI documentation.
     $self->{private_bz_tables_locked} = "";

I set it to 65536 which should be enough for the future.

This will be fixed in the next scheduled update.

Dave

Note You need to log in before you can comment on or make changes to this bug.