Bug 753113 - [RFE] : tighter design and better regression testing before clearing beta for release candidate status
[RFE] : tighter design and better regression testing before clearing beta for...
Product: Fedora
Classification: Fedora
Component: distribution (Show other bugs)
Unspecified Unspecified
unspecified Severity medium
: ---
: ---
Assigned To: Adam Williamson
Fedora Extras Quality Assurance
Depends On:
  Show dependency treegraph
Reported: 2011-11-11 06:31 EST by collura
Modified: 2017-11-16 20:31 EST (History)
2 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2017-11-16 20:31:52 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description collura 2011-11-11 06:31:01 EST
Description of problem:

/* this project is a huge amount of work and has helped a lot of people.
also, there are SO many lines making up components that errors are fairly likely; though there are only so many components so repeated fails can look similar from the outside.  also that not every user uses every component so it may not be worth holding a release for a low use package. 

further, it seems like (hardware diverse) gui/net/print errors are popping up more often.  its like the fail frequency/duration might have gone up lately as popularity/feature-addition has grown recently (or maybe just too much coffee for me today :') ).

releasing errors past beta however is going to slow adoption and reduce user capability/confidence.

various types of similar errors seem to be filtering into releases intermitantly including for example

  disappearing oxygen-cursor error:
   f15-f16 https://bugzilla.redhat.com/show_bug.cgi?id=707828#c1
   and formerly in
   f13-f14 https://bugzilla.redhat.com/show_bug.cgi?id=612835

  forgetfull brightness settings:
    fc10-fc15 https://bugzilla.redhat.com/show_bug.cgi?id=496975
    fc11-fc12 https://bugzilla.redhat.com/show_bug.cgi?id=500936
    f15 https://bugzilla.redhat.com/show_bug.cgi?id=695087  

  broken network setup:
    fc10-fc12 https://bugzilla.redhat.com/show_bug.cgi?id=540038
    fc15 https://bugzilla.redhat.com/show_bug.cgi?id=715459

  broken printer setup:
    fc12-fc14 https://bugzilla.redhat.com/show_bug.cgi?id=546385
    fc14 https://bugzilla.redhat.com/show_bug.cgi?id=683589

usually the errors seem to have working innards but broken gui
so the functions can still be gotten by the command line 
but we arent going to attract users by shipping broken guis

so maybe we should have tighter release criteria so they get fixed before a release 

the cause might vary but for the last few releases the effect seems to continue so maybe we need to test differently or address standardization without breaking creativity.

i have had similar problems with disappearing oxygen cursors, forgetful lcd brightness, failed printer addition dialogs, broken wifi, and so on that shouldnt not be released to endusers.  it is making this unnecessarily hard for new potential users to adopt as a released product when it is a lot more like a beta product.

complicated things break but whack-a-mole is what us beta testers are for.
we need to spare the endusers. */

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1. test alpha/beta/gama/release/postreleasebugfix for functionality of random capability
2. upgrade production machine if working or file requisite bug reports so that you can later upgrade when fixed
Actual results:

 various key features often dont work in a release

Expected results:

 feature works

Additional info:

(the related problem to more pre-release bug squashing being fewer beta testers than release users and more release criteria to test both cause release time delay)
Comment 1 Adam Williamson 2011-11-14 19:50:47 EST
We can draft new release criteria all day long, that's the easy part. =)

The hard part is achieving them. I'm always plumb in favour of aiming to achieve ever higher quality as we go along, but the track record of the last few releases indicates that the project at a whole is already more or less at its limits in attempting to meet the standards we currently require:


despite a _huge_ amount of work by the entire QA team and significant work on the part of desktop developers, it was a big struggle just to ensure F15 and F16 met the fairly minimal standards we already require. I suspect that with our current resources and with the current Fedora release cycle, we can't really tighten the criteria further and make it stick: we'd simply wind up delaying releases much further than is currently the case.

Fedora Bugzappers volunteer triage team
Comment 2 Adam Williamson 2011-11-14 19:51:36 EST
there is, however, a process if you want to propose specific new release criteria: post a draft to the test mailing list for comments and review. we generally then work on the basis that if there's a clear consensus in support of the proposed new criterion, it gets added to the list.
Comment 3 Adam Williamson 2017-11-16 20:31:52 EST
I don't think this bug report is achieving anything useful, so I'm closing it. As noted above, we already have a process for *specific* proposals to enhance the criteria and validation process.

Note You need to log in before you can comment on or make changes to this bug.