Description of problem: The RHEL6 kernel lacks a message that indicates that we have been attacked by Zombies.
Version-Release number of selected component (if applicable): 2.6.32-206.el6
How reproducible: 100%
Steps to Reproduce:
1. Step outside during Zombie attack.
2. Get bitten.
3. Return to office.
Actual results: No indication that you are now a Zombie.
Expected results: Some message indicating that you are a Zombie.
Created attachment 528558 [details]
RHEL6 Zombie patch
I believe this is a blocker.
And hotfix/zstream candidate. Zombies don't respect the schedule and won't wait
How should QA test?
* Can devel provide sample killer Zombies that could be used to attack innocent victims?
* Do we have any spare victims that could be bit, or is that not in our CapEx budget for this year? (preferably located in the Westford office to limit the distance between them and me here in RDU)
* Could testing be performed in one of the partner cages in the lab?
Assuming these criteria are met, I am willing to set qa_ack+.
Adding Gurhan to Cc list. He had knee surgery recently and can't run much.
Will make a great victim.
Cc'ing jburke. this is the kind of automated test we'll want added on tier1.
Oh, perfect. In that case I may want to be there in person to witness. Not sure if the travel budget will allow that though.
It was well established years ago that Fedora will eat Your Brannnnneeee!!! Should we clone this to Fedora as well or is the well known fact that Fedora has been a zombie since the early days sufficient?
I think we should tie this into the keyboard driver and detect if someone just randomly pushes lots of keys at once, clearly that would make them a zombie.
Perhaps, plugging in a network crossover cable, I mean who does that except zombies.
It appears I've already been bitten:
emcnabb 2856 0.0 0.0 0 0 ? Z 09:22 0:09 [chromium-browse] <defunct>
Clearly this is a 0-day invasion. Can I get a hotfix package ASAP?
A release note might be needed too to make things clear. Even though this will
work correctly it's not a zombie survival kit and/or other forms of zombie
Linda's shot at closing this zombie bug wasn't a deathblow.. therefore this bug, much like the zombies we're preparing our defense for, IS ALIIIVVE!
Mike -- I agree. It takes a head shot to put a zombie down.
Created attachment 528649 [details]
RHEL6 Zombie patch
Better patch. Modifies printk code to output Zombie notification.
... isn't this really a security issue? :)
It is. I'm not pulling this patch without the ACK of the building security
Created attachment 528662 [details]
RHEL6 Zombie patch
Changed "ZOMBIE" to more appropriate "OMGZOMBIE".
Has this been cloned for RHEL5 yet? Lots of people haven't upgraded to 6 yet... Actually, simply using 5 still might suggest zombie users...
NACK, we need a new error code for false positives ENOZOMBIE
We'll want to get this patch on the Brew builders as soon as possible. Half of Rel-Eng can't tell the difference between zombie/non-zombie anyway; there's no process defined for it.
Changing severity/priority to match the problem at hand.
Looks like the zombies got to Sly...and non one noticed in the RHEL meeting!
pm clearly can't be trusted anymore. people sneezing in the office. it's happening. reach for baseball bats and your tactical bacon cans people, the time
Sly you should know better. Zombies cannot be easily killed.
Moving to 6.3 ... clalance can still POST.
c'mon now my customers are pressuring me to get this fixed ASAP!!
I'm pretty certain that this will be needed in "Z" stream also.
+1...This whole thing is about the z-stream...we may need a v-stream when the vampire CVE comes in....
GSS approves the fix to restore brains after a zombie attack
Why do I always have to go in and fix Platform/OS? Are those zombie fields?
Setting Customer Facing flag to Yes. Customers have reported zombie processes forever.
I think we need a kbase to accompany this fix. We need to provide guidance on workarounds using improvised anti-zombie weapons likely to be at hand in an Enterprise computing environment.
Also do we need to add a discussion of zombie protection to our SLAs?
(In reply to comment #33)
> Also do we need to add a discussion of zombie protection to our SLAs?
RH Legal is crafting a Zombie-specific ZLA document to release with the Zombie-enabled versions. Of course the ZLA does not mention "best effort" anywhere.
Adding "OtherQA" to this in an effort to "Share the Love" with our partners.
NOTE: I would like to recommend adding an appropriate Zombie SKU to the Partner Entitlements, for those partners needing extra encouragement to help with the testing efforts
The local bokor has provided his ACK.
Has Legal verified no copyright infringement on George A. Romero ("Godfather of all Zombies") works to avoid future law suites by him or his estate?
Incidentally, the first Red Hat Release was known as the "Halloween Release" because it was released on October 31, 1994. I have it on expert authority that Red Hat made a specific number of copies and when we ran out, Mark Ewing instructed the Red Hatters simply to stop answering their phones.
We have interns in the QA Desktop group. I will ack the qe_test_coverage.
Technical note added. If any revisions are required, please edit the "Technical Notes" field
accordingly. All revisions will be proofread by the Engineering Content Services team.
Previously, there was no way to tell whether you'd been bitten by a zombie, so you could unknowingly endanger your colleagues and eat their brains. RHEL6 has been updated to issue a warning of "BRRRAAAAAAIIIIINNNNNSSSSSSSS" so that you are aware that you have indeed been zombified.
Technical Note has been added.
If this note is incorrect, needs to be changed, or needs more Zombie references, please edit the Technical Notes field above, and set the technical_notes flag to '?'
From the perspective of the Docs team; assuming an increasing zombie
demographic, is there not an opportunity cost of not being first to market with
Also, just to clarify the meeting earlier, I suggested a "SWOT analysis", not a
"SWAT analysis". Could whoever called in the airstrike please cancel it. We're
losing far too many members of the control group, and HR were concerned that
their turnover and staff attrition KPI's are going through the roof.
Which Open Sores license is this released under?
Regarding comment #38, is it ethical to propose using interns as zombie fodder?
I don't recall a module in our compliance & ethics training that would apply to this situation...
Bryn, maybe we can switch this to contractor/intern for liability purposes. I think it depends on the relationship that RH has negotiated with the placement services it uses. There is an ambiguity in the contracts or ethics training that it says nothing about Zombies, so it should be ok to use them until otherwise mandated if this becomes an issue.
(In reply to comment #18)
> Has this been cloned for RHEL5 yet?
Shouldn't we focus on RHEL4 first? I don't like the pale green look on it...
What ebout Extended Zombie Support for RHEL 2.1?
Ack. Field reports indicate AS2.1 and RHEL4 have been undead for some time.
I can't guarantee test coverage until 7.0 is verified, then 6.3 and so on. We have a limited amount of contractors/interns, unless we have reqs for more, qe_coverage will be limited.
Patch(es) available on kernel-2.6.32-229.el6
The transition of this bug to ON_QA lined up perfectly with new intern orientation. Great timing.
We started off by entering the training room and giving them a warm "Welcome to Red Hat" speech. After indoctrinating them with propaganda, we quickly unleashed a horde of apocalyptic zombies into the room and locked the door from the outside. Then it was lunchtime, where I had the meat-lovers pizza combo at Ruckus. Very tasty... and a good deal too.
An hour later we returned. Using a taser gun, we stunned them long enough to upgrade the kernel to -236.el6 for each intern, set the "OMGZOMBIES" option in grub, and connected up a serial console. They quickly returned to their senseless moaning about wanting more brains.
Here is sample output. For example, from intern #17:
# uname -a
Linux intern17.victim.rhts.eng.rdu.redhat.com 2.6.32-236.el6.x86_64 #1 SMP Mon Feb 20 10:01:09 EST 2012 x86_64 x86_64 x86_64 GNU/Linux
# cat /proc/cmdline
... KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM OMGZOMBIES
------------[ cut here ]------------
WARNING: at kernel/rh_taint.c:43 bitten_by_zombie+0x28/0x30() (Not tainted)
Hardware name: 2012 Client Platform
... ... ... BRAAAAIIIINNNNSSSSS
Modules linked in:
Pid: 0, comm: swapper Not tainted 2.6.32-236.el6.x86_64 #1
[<ffffffff81069b67>] ? warn_slowpath_common+0x87/0xc0
[<ffffffff81069bff>] ? warn_slowpath_fmt_taint+0x3f/0x50
[<ffffffff81c3e2a6>] ? console_setup+0xb1/0xf5
[<ffffffff810995b8>] ? bitten_by_zombie+0x28/0x30
[<ffffffff81c1fa12>] ? unknown_bootoption+0xe6/0x1ee
[<ffffffff8108e5a7>] ? parse_args+0x167/0x2e0
[<ffffffff814f09f8>] ? printk+0x41/0x49
[<ffffffff81c1f92c>] ? unknown_bootoption+0x0/0x1ee
[<ffffffff81c1fd58>] ? start_kernel+0x201/0x430
[<ffffffff81c1f33a>] ? x86_64_start_reservations+0x125/0x129
[<ffffffff81c1f438>] ? x86_64_start_kernel+0xfa/0x109
---[ end trace a7919e7f17c0a725 ]---
Disabling lock debugging due to kernel taint
[BRRAAIIIINNNSSSSS!] PID hash table entries: 4096 (order: 3, 32768 bytes)
[BRRAAIIIINNNSSSSS!] xsave/xrstor: enabled xstate_bv 0x7, cntxt size 0x340
[BRRAAIIIINNNSSSSS!] Checking aperture...
[BRRAAIIIINNNSSSSS!] No AGP bridge found
So looks good. Unfortunately, pretty much every department at Red Hat and also the University is frustrated we contaminated such a large batch of interns. We're still not sure what to do with all of them, so we'll send them up to Westford since they have more lab space. I'll provide FedEx tracking numbers via email.
Setting to VERIFIED.
We are not sure that upstream hasn't been infected/eaten. We need to test upstream first for 6.4 to see that the fixes are implemented.
Please add steps to verify.
This bug won't die, will it?
Dmitri, steps are still being defined as we get interns to test.
Evan, no. There are always zombies.
The virus is live and released, we can't fail QA now. Please file a new bz or clone this one to address any issues.
(In reply to comment #61)
> The virus is live and released, we can't fail QA now. Please file a new bz
> or clone this one to address any issues.
OMG ... we're now cloning zombies.
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory, and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.
If a closed bug is dead and this bug is unclosed does that mean.... ?!?!?!?!
All right. I just got word that there is an attack slated for our Independence day.
Can we get a fix pls?/????
Should we make the public aware of this errata?
OMG!!! Already happening in Detroit.
It's not just RHEL. :(
Zombies seem to be stumbling into our Cloud products too. (!)
Yes, we now need awareness and warning messages for Virtual Zombies too.
(Oh, the horror!)
REQUEST MANAGEMENT ESCALATION!!
Well anybody alive?
Oh no! Vegetarian zombies now too?
REQUEST MANAGEMENT ESCALATION!!
I think y'allll are ZOMBIEEEEEESSSSSSSSSSSSSSSSSSSSSSSS
Brief documentation of this functionality has been added to <https://access.redhat.com/site/solutions/40594>, «Why is the kernel "tainted" and how are the taint values deciphered?», deferring to the CDC for recommended actions.
I notice on the kbase it says this is a Red Hat extension, does that mean this isn't upstream? If so, shouldn't we work to get it upstream so that the community can benefit from being properly prepared for Zombies and also to benefit from upstream compatibility and maintainability?
You're probably meaning this:
It's there if upstream wants it. :)
"I think we should tie this into the keyboard driver and detect if someone just randomly pushes lots of keys at once, clearly that would make them a zombie."
No, they might also be a cat. :)
Or if we see key logging that a git commit will then follow a random number of keys (From fists) hitting said keyboard.
(In reply to comment #74)
> I notice on the kbase it says this is a Red Hat extension, does that mean
> this isn't upstream?
That's my understanding, yes.
> If so, shouldn't we work to get it upstream so that the community can
> benefit from being properly prepared for Zombies and also to benefit from
> upstream compatibility and maintainability?
I believe this has been attempted before with unfortunately little result
(cf. e.g. comment #55). Maybe it's time to, erm, resurrect that effort?
Do we have a reproducer for this?
(In reply to Ahmed Nazmy from comment #79)
> Do we have a reproducer for this?
Zombies reproduce asexually.
(In reply to Stephen Gallagher from comment #80)
> (In reply to Ahmed Nazmy from comment #79)
> > Do we have a reproducer for this?
> Zombies reproduce asexually.
Damn, this can only be faced by the first human conceived AI
Though a kernel component, I think we could provide some integration with EWS/JWS... WDYT, Coty?
(In reply to Marek Czernek from comment #82)
> Though a kernel component, I think we could provide some integration with
> EWS/JWS... WDYT, Coty?
(In reply to Pavel Raiskup from comment #83)
> Are there still reasons to keep this private?
Sound attracts zombies. Be careful out there.
I think this affects RHEL7 as well..
BTW, if you add 'shadowman' as a kernel parameter in RHEL7 you also get a surprise.