Red Hat Bugzilla – Bug 436312
RFE: FixedIn: Makefile (metadata) tag?
Last modified: 2012-09-26 00:25:47 EDT
Description of problem:
What do you think about providing a 'Fixed-In:' entry for the test metadata,
which correlates to the bugzillas defined in the metadata? There could be
multiple of them for each of the distros, like
Bugzilla: 211525 443221
Fixed-In: RHEL5ServerU1 RHEL4U4 F6
When any bug which is known to have been specified Fixed-In fails in a
release that is later than the fixed-in release should send up a read flag.
> I've thought about this. But it becomes a problem when you try and
> > determine whats newer?
> > If something is fixed in RHEL5U1 does it mean it should be fixed in F8?
> > How do we determine if F8 is newer than RHEL5U1 without having a
> > horrible lookup table?
Perhaps it would be better if it was strictly checking only within the
same distribution. For example, if it was fixed in RHEL4U2, then it would only
care about RHEL4U3+. If it were able to do this, it wouldn't even need to try
and figure out if the bug should be fixed in RHEL5 or F5 F8 RHEL7.
All distributions would need to be explicitly stated when the fix was made for
the particular bug in this case, but that doesn't seem like it would create too
much overhead. It would be more cleanly organized this way I think, it might
just take up a little more line space to do it. The lookup table wouldn't have
to be so terrible.
Perhaps it could look something like this?
Bugzilla: 44444 55555
FixedIn: 44444 RHEL3-U6 RHEL5Server-U2 RHEL5Client-U2 F7
FixedIn: 55555 RHEL3-U2 RHEL4-GOLD RHEL5Server-U1 RHEL5Client-U1
That would mean (theoretically):
BZ#44444 should PASS for RHEL3-U6+, RHEL5Server/Client-U2+, F7+
BZ#55555 should PASS for RHEL3-U2+ RHEL4-GOLD+ RHEL5Server/Client-U1+
U2+ == U2 U3 U4 U5 U6 U7 U8 U9 U?, etc.
F7+ == F7 F8 F9 F10 F11, F?, etc.
I've just come across a perfect example of why this feature should be implemented.
I wrote a test 5 months back, a simple regression test. The bug was for RHEL 5
releases only. No 4, 3, 2,...
This week, someone else ran my old script, when he ran all the tests for
logrotate arbitrarily, and was confused about why it failed.
Unfortunately I didn't document the test it clear enough that this test was only
valid for 5.1+, so he couldn't understand why the test failed. That's my fault.
But if we had the ability to selectively append Fixed-In tags per each time the
bug is fixed in a particular version release, the impact of such incidents could
One could have the ability to select and run only tests which are valid for 5.1+
(theoretically). All the rest of 5.y+ releases should then expectedly pass.
Results for run on any other distro's would clearly be 'unknown' and the user
would need to investigate if they wanted to run the test-case on an undeclared
If the test is found to pass on older or newer disto release (or different
distro all together, ie RHEL & Fedora), then the meta data could be updated to
reflect this new-found fact. We could minimize the number false-negative
test-runs we make.
My solution mentioned in the above comment doesn't seem perfect. But if we were
able find a solid solution for being able to indicate at the distro release
level, whether or not a test-case is expected to pass, we'd add the benefit of
more consistent and usable results with less headache. The downside is, of
course, we'd also be adding a bit of overhead to update the tests for each
distribution we expect it to pass for or fail for, but I think it's worth it.
Legacy RHTS is soon to be retired and replaced by Beaker. As part of
this migration process all RHTS bugs need to be re-verified against a
Beaker instance by the cut-off date
5pm Monday April 12th UTC-4.
Please confirm this bug is still relevant to Beaker by re-verifying it
against the stage deployment of Beaker https://beaker-stage.app.eng.bos.redhat.com.
To keep this bug open please comment on it
If it has not received a comment by that date the bug will be closed/wontfix.
After the cutoff date all commented bugs will be moved to the Beaker
As far as I know, this is still a valid request. I can not see anyway to accomplish this; if there is new meta data in beaker that would allow me to accomplish this, let me know.
Beaker is not the best place to track this kind of test metadata. Please propose this as a feature for TCMS or another more appropriate tool.