Red Hat Bugzilla – Bug 436312
RFE: FixedIn: Makefile (metadata) tag?
Last modified: 2012-09-26 00:25:47 EDT
Description of problem: What do you think about providing a 'Fixed-In:' entry for the test metadata, which correlates to the bugzillas defined in the metadata? There could be multiple of them for each of the distros, like Bugzilla: 211525 443221 Fixed-In: RHEL5ServerU1 RHEL4U4 F6 When any bug which is known to have been specified Fixed-In fails in a release that is later than the fixed-in release should send up a read flag. bpeck wrote: > I've thought about this. But it becomes a problem when you try and > > determine whats newer? > > > > If something is fixed in RHEL5U1 does it mean it should be fixed in F8? > > How do we determine if F8 is newer than RHEL5U1 without having a > > horrible lookup table? Perhaps it would be better if it was strictly checking only within the same distribution. For example, if it was fixed in RHEL4U2, then it would only care about RHEL4U3+. If it were able to do this, it wouldn't even need to try and figure out if the bug should be fixed in RHEL5 or F5 F8 RHEL7. All distributions would need to be explicitly stated when the fix was made for the particular bug in this case, but that doesn't seem like it would create too much overhead. It would be more cleanly organized this way I think, it might just take up a little more line space to do it. The lookup table wouldn't have to be so terrible. Perhaps it could look something like this? Bugzilla: 44444 55555 FixedIn: 44444 RHEL3-U6 RHEL5Server-U2 RHEL5Client-U2 F7 FixedIn: 55555 RHEL3-U2 RHEL4-GOLD RHEL5Server-U1 RHEL5Client-U1 That would mean (theoretically): BZ#44444 should PASS for RHEL3-U6+, RHEL5Server/Client-U2+, F7+ BZ#55555 should PASS for RHEL3-U2+ RHEL4-GOLD+ RHEL5Server/Client-U1+ Where: U2+ == U2 U3 U4 U5 U6 U7 U8 U9 U?, etc. F7+ == F7 F8 F9 F10 F11, F?, etc. Thoughts?
I've just come across a perfect example of why this feature should be implemented. I wrote a test 5 months back, a simple regression test. The bug was for RHEL 5 releases only. No 4, 3, 2,... This week, someone else ran my old script, when he ran all the tests for logrotate arbitrarily, and was confused about why it failed. Unfortunately I didn't document the test it clear enough that this test was only valid for 5.1+, so he couldn't understand why the test failed. That's my fault. But if we had the ability to selectively append Fixed-In tags per each time the bug is fixed in a particular version release, the impact of such incidents could be lessened. One could have the ability to select and run only tests which are valid for 5.1+ (theoretically). All the rest of 5.y+ releases should then expectedly pass. Results for run on any other distro's would clearly be 'unknown' and the user would need to investigate if they wanted to run the test-case on an undeclared distro. If the test is found to pass on older or newer disto release (or different distro all together, ie RHEL & Fedora), then the meta data could be updated to reflect this new-found fact. We could minimize the number false-negative test-runs we make. My solution mentioned in the above comment doesn't seem perfect. But if we were able find a solid solution for being able to indicate at the distro release level, whether or not a test-case is expected to pass, we'd add the benefit of more consistent and usable results with less headache. The downside is, of course, we'd also be adding a bit of overhead to update the tests for each distribution we expect it to pass for or fail for, but I think it's worth it.
Notice: Legacy RHTS is soon to be retired and replaced by Beaker. As part of this migration process all RHTS bugs need to be re-verified against a Beaker instance by the cut-off date 5pm Monday April 12th UTC-4. Please confirm this bug is still relevant to Beaker by re-verifying it against the stage deployment of Beaker https://beaker-stage.app.eng.bos.redhat.com. To keep this bug open please comment on it If it has not received a comment by that date the bug will be closed/wontfix. After the cutoff date all commented bugs will be moved to the Beaker product. thank you
As far as I know, this is still a valid request. I can not see anyway to accomplish this; if there is new meta data in beaker that would allow me to accomplish this, let me know.
Beaker is not the best place to track this kind of test metadata. Please propose this as a feature for TCMS or another more appropriate tool.