Bug 753113

Summary:	[RFE] : tighter design and better regression testing before clearing beta for release candidate status
Product:	[Fedora] Fedora	Reporter:	collura
Component:	distribution	Assignee:	Adam Williamson <awilliam>
Status:	CLOSED NOTABUG	QA Contact:	Fedora Extras Quality Assurance <extras-qa>
Severity:	medium	Docs Contact:
Priority:	unspecified
Version:	rawhide	CC:	collura, dennis
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2017-11-17 01:31:52 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:

Description collura 2011-11-11 11:31:01 UTC

Description of problem:


/* this project is a huge amount of work and has helped a lot of people.
also, there are SO many lines making up components that errors are fairly likely; though there are only so many components so repeated fails can look similar from the outside.  also that not every user uses every component so it may not be worth holding a release for a low use package. 

further, it seems like (hardware diverse) gui/net/print errors are popping up more often.  its like the fail frequency/duration might have gone up lately as popularity/feature-addition has grown recently (or maybe just too much coffee for me today :') ).

releasing errors past beta however is going to slow adoption and reduce user capability/confidence.

various types of similar errors seem to be filtering into releases intermitantly including for example

  disappearing oxygen-cursor error:
   f15-f16 https://bugzilla.redhat.com/show_bug.cgi?id=707828#c1
   and formerly in
   f13-f14 https://bugzilla.redhat.com/show_bug.cgi?id=612835

  forgetfull brightness settings:
    fc10-fc15 https://bugzilla.redhat.com/show_bug.cgi?id=496975
    fc11-fc12 https://bugzilla.redhat.com/show_bug.cgi?id=500936
    f15 https://bugzilla.redhat.com/show_bug.cgi?id=695087  

  broken network setup:
    fc10-fc12 https://bugzilla.redhat.com/show_bug.cgi?id=540038
    fc15 https://bugzilla.redhat.com/show_bug.cgi?id=715459

  broken printer setup:
    fc12-fc14 https://bugzilla.redhat.com/show_bug.cgi?id=546385
    fc14 https://bugzilla.redhat.com/show_bug.cgi?id=683589

usually the errors seem to have working innards but broken gui
so the functions can still be gotten by the command line 
but we arent going to attract users by shipping broken guis

so maybe we should have tighter release criteria so they get fixed before a release 

the cause might vary but for the last few releases the effect seems to continue so maybe we need to test differently or address standardization without breaking creativity.

i have had similar problems with disappearing oxygen cursors, forgetful lcd brightness, failed printer addition dialogs, broken wifi, and so on that shouldnt not be released to endusers.  it is making this unnecessarily hard for new potential users to adopt as a released product when it is a lot more like a beta product.

complicated things break but whack-a-mole is what us beta testers are for.
we need to spare the endusers. */

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1. test alpha/beta/gama/release/postreleasebugfix for functionality of random capability
2. upgrade production machine if working or file requisite bug reports so that you can later upgrade when fixed
3.
  
Actual results:

 various key features often dont work in a release

Expected results:

 feature works

Additional info:

(the related problem to more pre-release bug squashing being fewer beta testers than release users and more release criteria to test both cause release time delay)

Comment 1 Adam Williamson 2011-11-15 00:50:47 UTC

We can draft new release criteria all day long, that's the easy part. =)

The hard part is achieving them. I'm always plumb in favour of aiming to achieve ever higher quality as we go along, but the track record of the last few releases indicates that the project at a whole is already more or less at its limits in attempting to meet the standards we currently require:

https://fedoraproject.org/wiki/Fedora_Release_Criteria

despite a _huge_ amount of work by the entire QA team and significant work on the part of desktop developers, it was a big struggle just to ensure F15 and F16 met the fairly minimal standards we already require. I suspect that with our current resources and with the current Fedora release cycle, we can't really tighten the criteria further and make it stick: we'd simply wind up delaying releases much further than is currently the case.



-- 
Fedora Bugzappers volunteer triage team
https://fedoraproject.org/wiki/BugZappers

Comment 2 Adam Williamson 2011-11-15 00:51:36 UTC

there is, however, a process if you want to propose specific new release criteria: post a draft to the test mailing list for comments and review. we generally then work on the basis that if there's a clear consensus in support of the proposed new criterion, it gets added to the list.

Comment 3 Adam Williamson 2017-11-17 01:31:52 UTC

I don't think this bug report is achieving anything useful, so I'm closing it. As noted above, we already have a process for *specific* proposals to enhance the criteria and validation process.