Created attachment 497799 [details]
Description of problem: kernel NULL deref prob (see attached for details)
kernel NULL pointer dereference.
nouveau problem ?
This was compiled from koji source RPM.
Version-Release number of selected component (if applicable):
Steps to Reproduce:
Created attachment 497874 [details]
changed to text plain
Forgot to say - the symptoms (from a user perspective) - hesitation - nothing responding followed by black screen (i think) and then return to gdm login screen
Machine is lenovo t61p - attaching lspci
Created attachment 497903 [details]
output of scripta/ver_linux:
Linux lap1.prv.sapience.com 220.127.116.11-23.fc14.x86_64 #1 SMP Thu May 5 23:06:21 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux
Gnu C 4.5.1
Gnu make 3.82
Linux C Library 2.13
Dynamic linker (ldd) 2.13
Modules Loaded tcp_lp hidp fuse rfcomm sco bnep l2cap vboxnetadp vboxnetflt vboxdrv sunrpc cpufreq_ondemand acpi_cpufreq freq_table mperf capi capifs kernelcapi ipt_MASQUERADE iptable_nat nf_nat iptable_mangle ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables ipv6 kvm_intel kvm uinput snd_hda_codec_analog snd_hda_intel snd_hda_codec snd_hwdep snd_seq snd_seq_device arc4 snd_pcm iwlagn r852 sm_common nand nand_ids e1000e i2c_i801 btusb nand_ecc mtd iTCO_wdt iTCO_vendor_support snd_timer iwlcore thinkpad_acpi bluetooth snd wmi microcode mac80211 cfg80211 rfkill joydev soundcore snd_page_alloc xts gf128mul dm_crypt sdhci_pci sdhci firewire_ohci mmc_core firewire_core yenta_socket crc_itu_t nouveau ttm drm_kms_helper drm i2c_algo_bit i2c_core video
I'm not sure if this is a pure nouveau issue. The attached /var/log/messages seem to show the output of a sysrq-m which can indicate that an OOM occurred. If this is the case, the system was Out-Of-Memory before nouveau caused the BUG.
It is not good that nouveau hits a NULL-pointer dereference, there might need to be an extra check for a kmalloc() or similar. But maybe nouveau assumes that allocating some memory or structures just work, and only hits this BUG when the system does not have any memory free anymore.
Comment #2 could describe an out-of-memory issue as well. Killing the GDM session and restarting/reloading Xorg will likely reset the display (and associated nouveau driver).
Did this problem happen more than once? When yes, can you provide a /var/log/messages that contain some more details from before the BUG?
It happened once - if I remember correctly it was starting virtualbox - which may have caused an OOM ... I have not seen it since then - and have also moved to a newer kernel - i'll play some more and see if I can trigger it with the newer kernels.
We're not using 2.6.38 on any release any more, and certainly never on f14. I'm going to close this bug out. If you're still seeing it on a newer release, please open a new bug.
Which version of VirtualBox did you use at this time?