Red Hat Bugzilla – Bug 753113
[RFE] : tighter design and better regression testing before clearing beta for release candidate status
Last modified: 2014-02-25 14:28:56 EST
Description of problem:
/* this project is a huge amount of work and has helped a lot of people.
also, there are SO many lines making up components that errors are fairly likely; though there are only so many components so repeated fails can look similar from the outside. also that not every user uses every component so it may not be worth holding a release for a low use package.
further, it seems like (hardware diverse) gui/net/print errors are popping up more often. its like the fail frequency/duration might have gone up lately as popularity/feature-addition has grown recently (or maybe just too much coffee for me today :') ).
releasing errors past beta however is going to slow adoption and reduce user capability/confidence.
various types of similar errors seem to be filtering into releases intermitantly including for example
disappearing oxygen-cursor error:
and formerly in
forgetfull brightness settings:
broken network setup:
broken printer setup:
usually the errors seem to have working innards but broken gui
so the functions can still be gotten by the command line
but we arent going to attract users by shipping broken guis
so maybe we should have tighter release criteria so they get fixed before a release
the cause might vary but for the last few releases the effect seems to continue so maybe we need to test differently or address standardization without breaking creativity.
i have had similar problems with disappearing oxygen cursors, forgetful lcd brightness, failed printer addition dialogs, broken wifi, and so on that shouldnt not be released to endusers. it is making this unnecessarily hard for new potential users to adopt as a released product when it is a lot more like a beta product.
complicated things break but whack-a-mole is what us beta testers are for.
we need to spare the endusers. */
Version-Release number of selected component (if applicable):
Steps to Reproduce:
1. test alpha/beta/gama/release/postreleasebugfix for functionality of random capability
2. upgrade production machine if working or file requisite bug reports so that you can later upgrade when fixed
various key features often dont work in a release
(the related problem to more pre-release bug squashing being fewer beta testers than release users and more release criteria to test both cause release time delay)
We can draft new release criteria all day long, that's the easy part. =)
The hard part is achieving them. I'm always plumb in favour of aiming to achieve ever higher quality as we go along, but the track record of the last few releases indicates that the project at a whole is already more or less at its limits in attempting to meet the standards we currently require:
despite a _huge_ amount of work by the entire QA team and significant work on the part of desktop developers, it was a big struggle just to ensure F15 and F16 met the fairly minimal standards we already require. I suspect that with our current resources and with the current Fedora release cycle, we can't really tighten the criteria further and make it stick: we'd simply wind up delaying releases much further than is currently the case.
Fedora Bugzappers volunteer triage team
there is, however, a process if you want to propose specific new release criteria: post a draft to the test mailing list for comments and review. we generally then work on the basis that if there's a clear consensus in support of the proposed new criterion, it gets added to the list.