Bug 995404

Summary: resolution of closed bugs not reported
Product: [Fedora] Fedora Reporter: Steve Tyler <stephent98>
Component: python-bugzillaAssignee: Will Woods <wwoods>
Status: CLOSED WONTFIX QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 19CC: bugs.michael, crobinso, dzickus, jskarvad, stephent98, wwoods
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-08-15 15:16:16 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Steve Tyler 2013-08-09 09:30:05 UTC
Description of problem:
Closed bugzilla bugs always have a resolution, yet none of the basic query results contain the resolution:

$ bugzilla query -c grilo-plugins --bug_status=CLOSED -s tracker
#821952 CLOSED     - Bastien Nocera - Make tracker plugin optional

$ bugzilla query -c grilo-plugins --bug_status=CLOSED -s tracker -e
#821952 CLOSED     - Bastien Nocera - Make tracker plugin optional
 +Keywords: []
 +QA Whiteboard: 
 +Status Whiteboard: 
 +Devel Whiteboard: 

$ bugzilla query -c grilo-plugins --bug_status=CLOSED -s tracker -f | grep -i resolution
$ 

$ bugzilla query -c grilo-plugins --bug_status=CLOSED -s tracker --raw | grep -i resolution
ATTRIBUTE[resolution]: WONTFIX

Version-Release number of selected component (if applicable):
python-bugzilla-0.9.0-1.fc19.noarch

How reproducible:
Always.

Steps to Reproduce:
1. See above.

Actual results:
Resolution of closed bug is not reported.

Expected results:
Resolution of closed bug is reported.

Additional info:

Comment 1 Steve Tyler 2013-08-09 13:34:41 UTC
The resolution of closed bugs can, however, be reported with the "--outputformat" option:

$ bugzilla query -c brasero -s tracker --outputformat='Bug %{id} - %{status} %{resolution} - %{assigned_to} - %{summary}'
Bug 796349 - CLOSED WONTFIX - Xavier Lamien - brasero has dependency on tracker
Bug 947529 - NEW  - Xavier Lamien - brasero has dependency on tracker

Comment 2 Cole Robinson 2013-08-15 15:16:16 UTC
Similar to my response in bug 995494, I'm wary of changing the output of the static output options like --oneline, since people might be parsing it and it hasn't changed since creation. If you need custom fields, --outputformat is what you want.

Comment 3 Steve Tyler 2013-08-15 15:32:44 UTC
I don't want to compose a complex output format just to get the bug resolution.

Also, the bug resolution is not reported in the "--full" output either, which makes the name "full" a lie.

BTW, people should not be parsing the human-readable output.
That is what the raw output is for. It is supposed to be machine-readable.

Or is the raw output format as badly designed as the human-readable formats?

Comment 4 Steve Tyler 2013-08-15 15:43:12 UTC
The solution is again[1], a compatibility option -- this is exactly what compilers and interpreters have long done. See for example:
gcc -ansi
bash --posix

[1] See Bug 995494, Comment 9.

Comment 5 Cole Robinson 2013-08-15 16:34:27 UTC
(In reply to Steve Tyler from comment #3)
> I don't want to compose a complex output format just to get the bug
> resolution.
> 

Point taken, but you are the first person to ever complain about that.

> Also, the bug resolution is not reported in the "--full" output either,
> which makes the name "full" a lie.
> 

Yes I agree, the --full output predates me working on python-bugzilla and is poorly specified and incomplete. But there's tons of info it doesn't expose, so it's never been close to 'full'. Maybe we should just change the option description.

> BTW, people should not be parsing the human-readable output.
> That is what the raw output is for. It is supposed to be machine-readable.
> 

I agree, but I guarantee that people _are_ parsing bugzilla output, at least --oneline. And many people depend on this tool for their day to day workflow. There needs to be a very good reason to deviate from that.

> Or is the raw output format as badly designed as the human-readable formats?

Raw is auto generated from the dict(bug) basically. There's likely issues with it as well, because bugzilla value names change quite often so it isn't stable either.

If you have ideas of how to improve things, please send patches

Comment 6 Steve Tyler 2013-08-15 16:53:42 UTC
If people are parsing the output, it is likely to be on hyphens and white-space, so this format would be minimally disruptive:

CLOSED/WONTFIX

That would come back as a single string.

$ bugzilla query -b 995404
#995404 CLOSED     - Will Woods - resolution of closed bugs not reported

$ python
...
>>> s='#995404 CLOSED     - Will Woods - resolution of closed bugs not reported'
>>> p=s.split('-')
>>> p
['#995404 CLOSED     ', ' Will Woods ', ' resolution of closed bugs not reported']
>>> p[0].split()
['#995404', 'CLOSED']

What do you suppose people are doing with their parsed output?

Comment 7 Steve Tyler 2013-08-15 17:15:31 UTC
(In reply to Steve Tyler from comment #6)
> If people are parsing the output, it is likely to be on hyphens and
> white-space, so this format would be minimally disruptive:
...

And they are going to have to hack their way to something that might or might not work -- bug summaries can contain hyphens:

$ bugzilla query -b 966598
#966598 NEW        - David Lehman - Crash in VG creation when installing LVM-on-Intel firmware RAID (19 Beta RC4)

Comment 8 Steve Tyler 2013-08-15 18:06:05 UTC
(In reply to Cole Robinson from comment #5)
...
> I agree, but I guarantee that people _are_ parsing bugzilla output, at least
> --oneline. And many people depend on this tool for their day to day
> workflow. There needs to be a very good reason to deviate from that.
...

I would be interested in seeing an example of one of those parsers, because the --oneline output is also poorly suited to parsing. In particular, the name has spaces in it, and the number of spaces can vary:

$ bugzilla query -b 812134,875944 --oneline
#812134 NEW Cole Robinson virt-manager	[---]  
#875944 CLOSED Brian C. Lane anaconda	[---]  

There is a one-line format that is easy to parse -- CSV (comma-separated values).

Here, Kamil[1] constructed his own format string using semi-colons as separators:

$ bugzilla query --product Fedora --version 18 --reporter kparal --boolean_query 'creation_ts-changedafter-2012-11-29T00:00:00 & creation_ts-changedbefore-2013-01-15T23:59:59' --outputformat='%{id};%{reporter};%{status};%{resolution};%{status_whiteboard};%{blocked}'

881670;Kamil Páral;ASSIGNED;;AcceptedNTH;752665
882147;Kamil Páral;CLOSED;CURRENTRELEASE;abrt_hash:8221c0bcc483b99eea86c39c9f98aea72408f0001067f01dfac72928a53b2b0c RejectedBlocker AcceptedNTH;752665
882152;Kamil Páral;CLOSED;NOTABUG;;[]
...

[1] Bug 905529 - [abrt] python-bugzilla-0.7.0-3.fc18: bugzilla:730:bug_field:TypeError: sequence item 0: expected string, int found

Comment 9 Steve Tyler 2013-08-15 18:16:13 UTC
(In reply to Cole Robinson from comment #5)
...
> If you have ideas of how to improve things, please send patches

If you believe patches are the first step in a development process, you are mistaken. You need to start with a design first -- a first step that was not, evidently, taken when bugzilla was being developed.

BTW, the journalctl command has several well-defined, machine-readable output formats. These two are from the journalctl man page:[1]

1. Journal Export Format
http://www.freedesktop.org/wiki/Software/systemd/export

2. Journal JSON Format
http://www.freedesktop.org/wiki/Software/systemd/json

[1] $ rpm -qf $(which journalctl)
systemd-204-9.fc19.x86_64
Later releases have additional formats.

Comment 10 Don Zickus 2013-08-16 12:39:03 UTC
Hi,

I'll respond here because I added the --oneline code.  I added it for my job at the time, which was RHEL kernel maintainer.  At the time I was juggling thousands on internal patches and needed to associate bz info with them.  I forgot the exact reason why I chose the format I did, but it was mimicing something else I was using at the time and more importantly gave me the bz flags.

Now the maintainers have longed switched to using the bugzilla library to get this info as opposed to the slow commandline way.  So the original group of folks using this have migrated away.  Don't know if others adopted it.

So yes at the time it was being used to parse bugzillas daily and yes it helped improve my workflow.  Was the design crappy?  That is in the eye of the beholder.  It was designed for my workflow and it did its job. :-)  It wasn't meant to replace dozens of other programs/workflows like journalctl did (which required extensive reviews and discussions to end up with its final format).

So using journalctl as a comparision is really silly.  The --oneline was a quick and dirty hack to get me bz info _quickly_ without affecting others using --full.

But like I said the original intent for --oneline is gone, so I won't be too upset if it changes.  Others might.

Cheers,
Don

Comment 11 Steve Tyler 2013-08-16 13:38:28 UTC
Thanks, Don. That is very helpful.

Here are all your python-bugzilla commits as author:
https://git.fedorahosted.org/cgit/python-bugzilla.git/log/?qt=author&q=dzickus%40redhat.com

Here is your first commit message, dated 2008-09-29:

Add new output option --oneline for quick bz summary

This new option allows developers at a glance to quickly see where their bugzilla stands. Not only does it show the normal info like bz, name, state, but also adds flags and CVE info. An example would be 

https://git.fedorahosted.org/cgit/python-bugzilla.git/commit/?id=a717ea885beeb87b5d6b5a6a5fecf6f5d486ec78

Comment 12 Steve Tyler 2013-08-16 14:06:54 UTC
Here are some "--oneline" examples cherry-picked from NEW kernel bugs:
$ bugzilla query -c kernel -t NEW --oneline

#741734 NEW Red Hat Kernel Manager kernel	[beta]  
#835806 NEW Red Hat Kernel Manager kernel	[rc] needinfo- 
#910720 NEW Red Hat Kernel Manager kernel	[rc]  
#995609 NEW Kernel Maintainer List kernel	[---] needinfo? 
#995635 NEW  kernel	[---]  
#997653 NEW Kernel Maintainer List kernel	[---]  

NB: The BZ numbers are not autolinkified, because I copied and pasted them ...
https://bugzilla.redhat.com/docs/en/html/hintsandtips.html