Red Hat Bugzilla – Full Text Bug Listing
|Summary:||[RFE] : tighter design and better regression testing before clearing beta for release candidate status|
|Component:||distribution||Assignee:||Adam Williamson <awilliam>|
|Status:||CLOSED NOTABUG||QA Contact:||Fedora Extras Quality Assurance <extras-qa>|
|Fixed In Version:||Doc Type:||Bug Fix|
|Doc Text:||Story Points:||---|
|Last Closed:||2017-11-16 20:31:52 EST||Type:||---|
|oVirt Team:||---||RHEL 7.3 requirements from Atomic Host:|
Description collura 2011-11-11 06:31:01 EST
Description of problem: /* this project is a huge amount of work and has helped a lot of people. also, there are SO many lines making up components that errors are fairly likely; though there are only so many components so repeated fails can look similar from the outside. also that not every user uses every component so it may not be worth holding a release for a low use package. further, it seems like (hardware diverse) gui/net/print errors are popping up more often. its like the fail frequency/duration might have gone up lately as popularity/feature-addition has grown recently (or maybe just too much coffee for me today :') ). releasing errors past beta however is going to slow adoption and reduce user capability/confidence. various types of similar errors seem to be filtering into releases intermitantly including for example disappearing oxygen-cursor error: f15-f16 https://bugzilla.redhat.com/show_bug.cgi?id=707828#c1 and formerly in f13-f14 https://bugzilla.redhat.com/show_bug.cgi?id=612835 forgetfull brightness settings: fc10-fc15 https://bugzilla.redhat.com/show_bug.cgi?id=496975 fc11-fc12 https://bugzilla.redhat.com/show_bug.cgi?id=500936 f15 https://bugzilla.redhat.com/show_bug.cgi?id=695087 broken network setup: fc10-fc12 https://bugzilla.redhat.com/show_bug.cgi?id=540038 fc15 https://bugzilla.redhat.com/show_bug.cgi?id=715459 broken printer setup: fc12-fc14 https://bugzilla.redhat.com/show_bug.cgi?id=546385 fc14 https://bugzilla.redhat.com/show_bug.cgi?id=683589 usually the errors seem to have working innards but broken gui so the functions can still be gotten by the command line but we arent going to attract users by shipping broken guis so maybe we should have tighter release criteria so they get fixed before a release the cause might vary but for the last few releases the effect seems to continue so maybe we need to test differently or address standardization without breaking creativity. i have had similar problems with disappearing oxygen cursors, forgetful lcd brightness, failed printer addition dialogs, broken wifi, and so on that shouldnt not be released to endusers. it is making this unnecessarily hard for new potential users to adopt as a released product when it is a lot more like a beta product. complicated things break but whack-a-mole is what us beta testers are for. we need to spare the endusers. */ Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. test alpha/beta/gama/release/postreleasebugfix for functionality of random capability 2. upgrade production machine if working or file requisite bug reports so that you can later upgrade when fixed 3. Actual results: various key features often dont work in a release Expected results: feature works Additional info: (the related problem to more pre-release bug squashing being fewer beta testers than release users and more release criteria to test both cause release time delay)
Comment 1 Adam Williamson 2011-11-14 19:50:47 EST
We can draft new release criteria all day long, that's the easy part. =) The hard part is achieving them. I'm always plumb in favour of aiming to achieve ever higher quality as we go along, but the track record of the last few releases indicates that the project at a whole is already more or less at its limits in attempting to meet the standards we currently require: https://fedoraproject.org/wiki/Fedora_Release_Criteria despite a _huge_ amount of work by the entire QA team and significant work on the part of desktop developers, it was a big struggle just to ensure F15 and F16 met the fairly minimal standards we already require. I suspect that with our current resources and with the current Fedora release cycle, we can't really tighten the criteria further and make it stick: we'd simply wind up delaying releases much further than is currently the case. -- Fedora Bugzappers volunteer triage team https://fedoraproject.org/wiki/BugZappers
Comment 2 Adam Williamson 2011-11-14 19:51:36 EST
there is, however, a process if you want to propose specific new release criteria: post a draft to the test mailing list for comments and review. we generally then work on the basis that if there's a clear consensus in support of the proposed new criterion, it gets added to the list.
Comment 3 Adam Williamson 2017-11-16 20:31:52 EST
I don't think this bug report is achieving anything useful, so I'm closing it. As noted above, we already have a process for *specific* proposals to enhance the criteria and validation process.