Bug 453902 - buglist.cgi ctype=csv dependson list truncated
Summary: buglist.cgi ctype=csv dependson list truncated
Keywords:
Status: CLOSED NEXTRELEASE
Alias: None
Product: Bugzilla
Classification: Community
Component: Bugzilla General
Version: 3.2
Hardware: All
OS: Linux
low
medium
Target Milestone: ---
Assignee: PnT DevOps Devs
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2008-07-03 03:13 UTC by FTBFS
Modified: 2013-06-24 02:26 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2008-08-28 04:29:43 UTC
Embargoed:


Attachments (Terms of Use)

Description FTBFS 2008-07-03 03:13:47 UTC
Description of problem:
Querying using buglist.csv?ctype=csv and using a cookie that requests the
'dependson' field, the results list is incomplete.

Example:
#!/bin/bash
cookie='COLUMNLIST=alias%20bug_severity%20priority%20rep_platform%20assigned_to%20bug_status%20resolution%20component%20short_desc%20dependson'
url='https://bugzilla.redhat.com/buglist.cgi?bug_id=440169&ctype=csv'
/usr/bin/curl -k --cookie $cookie $url

resulting csv list has 128 entries in the dependson list.


The same call using python-bugzilla's bugzilla program and xmlrpc call:
$ bugzilla --verbose --debug query -f -b 440169

yields a list with 452 entries.

Version-Release number of selected component (if applicable):
bugzilla 2.18

How reproducible:
always

Comment 1 FTBFS 2008-07-03 03:14:23 UTC
Doh.  I should have filed this under my name. -Matt

Comment 2 Noura El hawary 2008-07-03 06:46:01 UTC
Problem verified, trying the url:

https://bugzilla.redhat.com/buglist.cgi?bug_id=440169&field-1-0-0=bug_id&query_format=advanced&remaction=&type-1-0-0=anyexact&value-1-0-0=440169&query_based_on=&columnlist=bug_severity%2Cpriority%2Cassigned_to%2Cbug_status%2Cresolution%2Cop_sys%2Cdependson%2Cshort_desc

the list returns only 128 bugs in the dependson column which is not the complete
list, I am assuming the problem is coming from the Function bugs.DependsOnList
created using the file bugs-functions.mysql, maybe we need to enlarge the TEXT
to be MEDIUMTEXT or LONGTEXT !!

Noura

Comment 3 Matt Domsch 2008-08-28 01:31:06 UTC
ping

Comment 4 David Lawrence 2008-08-28 04:29:43 UTC
(In reply to comment #2)
> Problem verified, trying the url:
> 
> https://bugzilla.redhat.com/buglist.cgi?bug_id=440169&field-1-0-0=bug_id&query_format=advanced&remaction=&type-1-0-0=anyexact&value-1-0-0=440169&query_based_on=&columnlist=bug_severity%2Cpriority%2Cassigned_to%2Cbug_status%2Cresolution%2Cop_sys%2Cdependson%2Cshort_desc
> 
> the list returns only 128 bugs in the dependson column which is not the complete
> list, I am assuming the problem is coming from the Function bugs.DependsOnList
> created using the file bugs-functions.mysql, maybe we need to enlarge the TEXT
> to be MEDIUMTEXT or LONGTEXT !!

Actually according to the MySQL docs, the GROUP_CONCAT function is limited by the GROUP_CONCAT_MAX_LEN setting which defaults to 1024.

"The result is truncated to the maximum length that is given by the group_concat_max_len system variable, which has a default value of 1024. The value can be set higher, although the effective maximum length of the return value is constrained by the value of max_allowed_packet. The syntax to change the value of group_concat_max_len at runtime is as follows, where val  is an unsigned integer:

SET [SESSION | GLOBAL] group_concat_max_len = val;

Beginning with MySQL 5.0.19, the type returned by GROUP_CONCAT() is always VARCHAR unless group_concat_max_len is greater than 512, in which case, it returns a BLOB. (Previously, it returned a BLOB with group_concat_max_len greater than 512 only if the query included an ORDER BY clause.)"

So I have commited the following change to CVS which fixes the truncation issue.

Index: Bugzilla/DB/Mysql.pm
===================================================================
RCS file: /cvs/qa/rh_bugzilla_3/Bugzilla/DB/Mysql.pm,v
retrieving revision 1.17
diff -u -r1.17 Mysql.pm
--- Bugzilla/DB/Mysql.pm	13 Aug 2008 19:24:55 -0000	1.17
+++ Bugzilla/DB/Mysql.pm	28 Aug 2008 04:24:57 -0000
@@ -67,6 +67,10 @@
     # return their data correctly.
     $self->do("SET NAMES utf8") if Bugzilla->params->{'utf8'};
 
+    # REDHAT EXTENSION START 453902 
+    $self->do("SET GROUP_CONCAT_MAX_LEN = 65536;"); 
+    # REDHAT EXTENSION END 453902
+
     # all class local variables stored in DBI derived class needs to have
     # a prefix 'private_'. See DBI documentation.
     $self->{private_bz_tables_locked} = "";

I set it to 65536 which should be enough for the future.

This will be fixed in the next scheduled update.

Dave


Note You need to log in before you can comment on or make changes to this bug.