looks like the uCode just stop responding after some period of time

[   99.618570] iwlagn 0000:06:00.0: iwlagn_tx_agg_start on ra =
74:ea:3a:a5:d3:28 tid = 0
[  177.438914] iwlagn 0000:06:00.0: Error sending REPLY_ADD_STA: time out after
500ms.

you can see aggregation happen on [99.618570} and host command timeout at [177.438914]

What version of uCode you are using? also what the configuration you have? (band/freq/channel/seqcurity/...)

btw, could you try to disable 11n and see if you still have the similar problem?

$modprobe iwlagn 11n_disable=1

Thanks
Wey

Firstly I put your modprobe iwlagn 11n_disable=1 line to my rc.local, and restarted the machine. 

Band: 2.4GHz
freq: 2.412 GHz
channel: 1 (I calculated about freq)
security: WPA-PSK/WPA2-PSK personal

How I can get the informations about uCode?

I tried the tests below under kubuntu 64 bit 11.04. 

cat /sys/bus/pci/drivers/iwlagn/0000:06:00.0/temperature
First test: it's start near 62 and goes up to 67 (Speeed 120-135 Mbit/s) after 3-4 mins it's dropped.
Second probe: It's start from 41 and don't heated so much, near 45. After 2 mins idle dropped form AP. (Speed 120-135Mbit/sec)
Third test: I manually disabled N mode in wifi AP, and tried again. Result is:
Temperature Starts from 47 (Speed 54 MBit/s). It's immediately dropped.

Dmesg below...

[   37.626549] wlan0: RX AssocResp from 74:ea:3a:a5:d3:28 (capab=0x431 status=0 aid=1)
[   37.626554] wlan0: associated
[   37.629871] ADDRCONF(NETDEV_CHANGE): wlan0: link becomes ready
[   38.193680] Intel AES-NI instructions are not detected.
[   38.474189] padlock_aes: VIA PadLock not detected.
[   47.655648] wlan0: no IPv6 routers present
[  111.514322] iwlagn 0000:06:00.0: Error sending POWER_TABLE_CMD: time out after 500ms.
[  111.514332] iwlagn 0000:06:00.0: set power fail, ret = -110
[  112.513871] iwlagn 0000:06:00.0: Error sending REPLY_QOS_PARAM: time out after 500ms.
[  112.513882] iwlagn 0000:06:00.0: Failed to update QoS
[  113.013738] iwlagn 0000:06:00.0: Error sending REPLY_RXON: time out after 500ms.
[  113.013748] iwlagn 0000:06:00.0: Error clearing ASSOC_MSK on BSS (-110)
[  113.513562] iwlagn 0000:06:00.0: Error sending REPLY_RXON: time out after 500ms.
[  113.513569] iwlagn 0000:06:00.0: Error clearing ASSOC_MSK on BSS (-110)
[  114.013285] iwlagn 0000:06:00.0: Error sending REPLY_ADD_STA: time out after 500ms.
[  114.013296] ieee80211 phy0: failed to remove key (0, 74:ea:3a:a5:d3:28) from hardware (-110)
[  114.513141] iwlagn 0000:06:00.0: Error sending REPLY_REMOVE_STA: time out after 500ms.
[  114.513154] iwlagn 0000:06:00.0: Error removing station 74:ea:3a:a5:d3:28
[  115.012896] iwlagn 0000:06:00.0: Error sending REPLY_RXON: time out after 500ms.
[  115.012902] iwlagn 0000:06:00.0: Error clearing ASSOC_MSK on BSS (-110)
[  117.031988] iwlagn 0000:06:00.0: fail to flush all tx fifo queues
[  117.531815] iwlagn 0000:06:00.0: Error sending POWER_TABLE_CMD: time out after 500ms.
[  117.531824] iwlagn 0000:06:00.0: set power fail, ret = -110
[  117.731724] iwlagn 0000:06:00.0: Queue 2 stuck for 10000 ms.
[  117.731734] iwlagn 0000:06:00.0: On demand firmware reload
[  118.031612] iwlagn 0000:06:00.0: Error sending REPLY_RXON: time out after 500ms.
[  118.031621] iwlagn 0000:06:00.0: Error clearing ASSOC_MSK on BSS (-110)
[  118.531367] iwlagn 0000:06:00.0: Error sending REPLY_ADD_STA: time out after 500ms.
[  118.531389] ieee80211 phy0: failed to remove key (1, ff:ff:ff:ff:ff:ff) from hardware (-110)
[  118.531516] cfg80211: All devices are disconnected, going to restore regulatory settings
[  118.531525] cfg80211: Restoring regulatory settings
[  118.531533] cfg80211: Calling CRDA to update world regulatory domain
[  118.539408] cfg80211: Ignoring regulatory request Set by core since the driver uses its own custom regulatory domain 
[  118.539421] cfg80211: World regulatory domain updated:
[  118.539425] cfg80211:     (start_freq - end_freq @ bandwidth), (max_antenna_gain, max_eirp)
[  118.539431] cfg80211:     (2402000 KHz - 2472000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
[  118.539437] cfg80211:     (2457000 KHz - 2482000 KHz @ 20000 KHz), (300 mBi, 2000 mBm)
[  118.539443] cfg80211:     (2474000 KHz - 2494000 KHz @ 20000 KHz), (300 mBi, 2000 mBm)
[  118.539448] cfg80211:     (5170000 KHz - 5250000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
[  118.539453] cfg80211:     (5735000 KHz - 5835000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
[  118.552542] iwlagn 0000:06:00.0: MAC is in deep sleep!.  CSR_GP_CNTRL = 0xFFFFFFFF
[  118.573735] iwlagn 0000:06:00.0: MAC is in deep sleep!.  CSR_GP_CNTRL = 0xFFFFFFFF
[  118.594907] iwlagn 0000:06:00.0: MAC is in deep sleep!.  CSR_GP_CNTRL = 0xFFFFFFFF

Cheers: Balazs
---------------------------------

(In reply to comment #1)
> looks like the uCode just stop responding after some period of time
> 
> [   99.618570] iwlagn 0000:06:00.0: iwlagn_tx_agg_start on ra =
> 74:ea:3a:a5:d3:28 tid = 0
> [  177.438914] iwlagn 0000:06:00.0: Error sending REPLY_ADD_STA: time out after
> 500ms.
> 
> you can see aggregation happen on [99.618570} and host command timeout at
> [177.438914]
> 
> What version of uCode you are using? also what the configuration you have?
> (band/freq/channel/seqcurity/...)
> 
> btw, could you try to disable 11n and see if you still have the similar
> problem?
> 
> $modprobe iwlagn 11n_disable=1
> 
> Thanks
> Wey

for firmware information, please use the following command
$ethtool -i wlanX --- replace "X" with your NIC #

this will tell us what Firmware version you are using

from the log information you provide, both legacy and 11n has similar problem. the uCode just stop responding after a while, all the commands send to uCode got timeout.

Once more thing to check, please make sure the ASPM is program correctly

http://linuxwireless.org/en/users/Documentation/ASPM

Thanks
Wey

You can also check if adding pcie_aspm=off or pcie_aspm=force kernel boot option helps with that hung.

I used ethtool wlan0 command and here is the output:

driver: iwlagn
version: 2.6.38-8-generic
firmware-version: 8.83.5.1 build 33692
bus-info: 0000:06:00.0

I try to add the end of kernel line the pcie_aspm=off and force command. After I tested these options, I write again.

Thanks: Balazs

I tested, and there is no change. I used 

pcie_aspm=off and pcie_aspm=force in the end of the bootloader kernel section, but no success. Still hangs on.

I used the AP in 54MBit BG mode.

Now I turns to install a centos and test again it, and write a bootable flash with fedora live desktop.

I'll check for  ethtool wlan0 results...

question, if you are not passing traffic, can it stay connected?

Thanks
Wey

No.
I just assoicated to the ap (this generate little traffic), but no more network traffic, but it's dropped.

Balazs

(In reply to comment #7)
> question, if you are not passing traffic, can it stay connected?
> 
> Thanks
> Wey

Ok. I put back my AP to bgn mixed mode. 
I installed centos 6, with a fresh install. I running a yum update process, which downloadin 120MB from net, and I'm copying a centos iso 3.9 GB.

Temperature is 72 celsius (highest than previous values), there is no freeze, and other things like in ubuntu.

ethtool wlan0:
No data available

ethtool -i wlan0:
Cannot get driver information: operation not supported

lsmod |grep iw
iwlagn                116422  0 
iwlcore               127880  1 iwlagn
mac80211              146065  2 iwlagn,iwlcore
cfg80211              138993  3 iwlagn,iwlcore,mac80211

dmesg (filtered):

cfg80211: Calling CRDA to update world regulatory domain
iwlagn: Intel(R) Wireless WiFi Link AGN driver for Linux, 1.3.27k
iwlagn: Copyright(c) 2003-2009 Intel Corporation
iwlagn 0000:06:00.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
iwlagn 0000:06:00.0: setting latency timer to 64
iwlagn 0000:06:00.0: Detected Intel Wireless WiFi Link 5100AGN REV=0x54
iwlagn 0000:06:00.0: Tunable channels: 13 802.11bg, 24 802.11a channels
  alloc irq_desc for 38 on node -1
  alloc kstat_irqs on node -1
iwlagn 0000:06:00.0: irq 38 for MSI/MSI-X
phy0: Selected rate control algorithm 'iwl-agn-rs'
i801_smbus 0000:00:1f.3: PCI INT C -> GSI 18 (level, low) -> IRQ 18
ACPI: I/O resource 0000:00:1f.3 [0xefa0-0xefbf] conflicts with ACPI region SMBI [0xefa0-0xefaf]
ACPI: If an ACPI driver is available for this device, you should use it instead of the native driver

cfg80211: Calling CRDA for country: HU

cfg80211: Regulatory domain changed to country: HU
	(start_freq - end_freq @ bandwidth), (max_antenna_gain, max_eirp)
	(2402000 KHz - 2482000 KHz @ 40000 KHz), (N/A, 2000 mBm)
	(5170000 KHz - 5250000 KHz @ 40000 KHz), (N/A, 2000 mBm)
	(5250000 KHz - 5330000 KHz @ 40000 KHz), (N/A, 2000 mBm)
	(5490000 KHz - 5710000 KHz @ 40000 KHz), (N/A, 2700 mBm)

microcode: CPU0 sig=0x20652, pf=0x10, revision=0x8
platform microcode: firmware: requesting intel-ucode/06-25-02
microcode: CPU1 sig=0x20652, pf=0x10, revision=0x8
platform microcode: firmware: requesting intel-ucode/06-25-02
microcode: CPU2 sig=0x20652, pf=0x10, revision=0x8
platform microcode: firmware: requesting intel-ucode/06-25-02
microcode: CPU3 sig=0x20652, pf=0x10, revision=0x8
platform microcode: firmware: requesting intel-ucode/06-25-02
Microcode Update Driver: v2.00 <tigran.co.uk>, Peter Oruba
microcode: CPU0 updated to revision 0x9, date = 2009-11-12 
microcode: CPU1 updated to revision 0x9, date = 2009-11-12 
microcode: CPU2 updated to revision 0x9, date = 2009-11-12 
microcode: CPU3 updated to revision 0x9, date = 2009-11-12 
Microcode Update Driver: v2.00 removed.

iwlagn 0000:06:00.0: firmware: requesting iwlwifi-5000-2.ucode
iwlagn 0000:06:00.0: loaded firmware version 8.24.2.12
Registered led device: iwl-phy0::radio
Registered led device: iwl-phy0::assoc
Registered led device: iwl-phy0::RX
Registered led device: iwl-phy0::TX
ADDRCONF(NETDEV_UP): wlan0: link is not ready

wlan0: deauthenticating from 74:ea:3a:a5:d3:28 by local choice (reason=3)
wlan0: direct probe to AP 74:ea:3a:a5:d3:28 (try 1)
wlan0: direct probe responded
wlan0: authenticate with AP 74:ea:3a:a5:d3:28 (try 1)
wlan0: authenticated
wlan0: associate with AP 74:ea:3a:a5:d3:28 (try 1)
wlan0: RX AssocResp from 74:ea:3a:a5:d3:28 (capab=0x431 status=0 aid=2)
wlan0: associated
ADDRCONF(NETDEV_CHANGE): wlan0: link becomes ready
Intel AES-NI instructions are not detected.
padlock: VIA PadLock not detected.
wlan0: no IPv6 routers present
wlan0: disassociated from 74:ea:3a:a5:d3:28 (Reason: 8)
wlan0: deauthenticating from 74:ea:3a:a5:d3:28 by local choice (reason=3)
wlan0: direct probe to AP 74:ea:3a:a5:d3:28 (try 1)
wlan0: direct probe to AP 74:ea:3a:a5:d3:28 (try 2)
wlan0: direct probe to AP 74:ea:3a:a5:d3:28 (try 3)
wlan0: direct probe to AP 74:ea:3a:a5:d3:28 timed out
wlan0: direct probe to AP 74:ea:3a:a5:d3:28 (try 1)
wlan0: direct probe responded
wlan0: authenticate with AP 74:ea:3a:a5:d3:28 (try 1)
wlan0: authenticated
wlan0: associate with AP 74:ea:3a:a5:d3:28 (try 1)
wlan0: RX AssocResp from 74:ea:3a:a5:d3:28 (capab=0x431 status=0 aid=1)
wlan0: associated
iwlagn 0000:06:00.0: iwl_tx_agg_start on ra = 74:ea:3a:a5:d3:28 tid = 0

copied 4 GB of data without any error.

I try fedora, and post again.

(In reply to comment #6)
> I tested, and there is no change. I used 
> 
> pcie_aspm=off and pcie_aspm=force in the end of the bootloader kernel section,
> but no success. Still hangs on.
> 
> I used the AP in 54MBit BG mode.
> 
> Now I turns to install a centos and test again it, and write a bootable flash
> with fedora live desktop.
> 
> I'll check for  ethtool wlan0 results...

very interesting, using the same NIC, right?

I do notice the firmware is different, you have
iwlagn 0000:06:00.0: firmware: requesting iwlwifi-5000-2.ucode
iwlagn 0000:06:00.0: loaded firmware version 8.24.2.12

on F14/15 and Ubuntu, you have newer firmware. Could you try to use the same firmware on centos 6?

Thanks
Wey

Under fedora 15 bootable liveusb:

lsmod
iwlagn                254115  0 
iwlcore               144189  1 iwlagn
mac80211              234498  2 iwlagn,iwlcore
cfg80211              135850  3 iwlagn,iwlcore,mac80211

under fedora 15:
driver: iwlagn
version: 2.6.38.6-26.rc1.fc15.x86_64
firmware-version: 8.83.5.1 build 33692
bus-info: 0000:06:00.0

previously under ubuntu was:
driver: iwlagn
version: 2.6.38-8-generic
firmware-version: 8.83.5.1 build 33692
bus-info: 0000:06:00.0

The only change: it's associated and I'm running my system near 10 mins without network traffic, but no drop. Temp: 63

I tried to copy a 4 giga iso:
after 435 MB copied, it's dropped from network.
Temp was 67.

dmesg:
[  212.581151] wlan0: associated
[  215.629806] wlan0: deauthenticated from 74:ea:3a:a5:d3:28 (Reason: 2)
[  215.639761] cfg80211: Calling CRDA to update world regulatory domain
[  215.646880] cfg80211: World regulatory domain updated:
[  215.646887] cfg80211:     (start_freq - end_freq @ bandwidth), (max_antenna_gain, max_eirp)
[  215.646894] cfg80211:     (2402000 KHz - 2472000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
[  215.646901] cfg80211:     (2457000 KHz - 2482000 KHz @ 20000 KHz), (300 mBi, 2000 mBm)
[  215.646907] cfg80211:     (2474000 KHz - 2494000 KHz @ 20000 KHz), (300 mBi, 2000 mBm)
[  215.646914] cfg80211:     (5170000 KHz - 5250000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
[  215.646920] cfg80211:     (5735000 KHz - 5835000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
[  225.983242] wlan0: authenticate with 74:ea:3a:a5:d3:28 (try 1)
[  225.985756] wlan0: authenticated
[  225.986069] wlan0: associate with 74:ea:3a:a5:d3:28 (try 1)
[  225.989888] wlan0: RX ReassocResp from 74:ea:3a:a5:d3:28 (capab=0x431 status=0 aid=2)
[  225.989893] wlan0: associated
[  301.033981] iwlagn 0000:06:00.0: Aggregation not enabled for tid 0 because load = 3
[  309.302065] iwlagn 0000:06:00.0: iwlagn_tx_agg_start on ra = 74:ea:3a:a5:d3:28 tid = 0
[  731.780134] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null)
[  731.780153] SELinux: initialized (dev sda1, type ext4), uses xattr
[  835.523742] iwlagn 0000:06:00.0: Error sending REPLY_ADD_STA: time out after 500ms.
[  835.523754] HW problem - can not stop rx aggregation for tid 0
[  836.023526] iwlagn 0000:06:00.0: Error sending REPLY_QOS_PARAM: time out after 500ms.
[  836.023536] iwlagn 0000:06:00.0: Failed to update QoS
[  836.523246] iwlagn 0000:06:00.0: Error sending REPLY_RXON: time out after 500ms.
[  836.523260] iwlagn 0000:06:00.0: Error clearing ASSOC_MSK on BSS (-110)
[  837.023107] iwlagn 0000:06:00.0: Error sending REPLY_RXON: time out after 500ms.
[  838.022658] iwlagn 0000:06:00.0: Stopping AGG while state not ON or starting
[  838.522464] iwlagn 0000:06:00.0: Error sending REPLY_ADD_STA: time out after 500ms.
[  838.522479] ieee80211 phy0: failed to remove key (0, 74:ea:3a:a5:d3:28) from hardware (-110)
[  839.022268] iwlagn 0000:06:00.0: Error sending REPLY_REMOVE_STA: time out after 500ms.
[  839.022280] iwlagn 0000:06:00.0: Error removing station 74:ea:3a:a5:d3:28
[  839.521954] iwlagn 0000:06:00.0: Error sending REPLY_RXON: time out after 500ms.
[  839.521961] iwlagn 0000:06:00.0: Error clearing ASSOC_MSK on BSS (-110)
[  841.523114] iwlagn 0000:06:00.0: fail to flush all tx fifo queues
[  841.568095] iwlagn 0000:06:00.0: Queue 9 stuck for 10000 ms.
[  841.568104] iwlagn 0000:06:00.0: On demand firmware reload
[  842.022917] iwlagn 0000:06:00.0: Error sending REPLY_RXON: time out after 500ms.
[  842.022927] iwlagn 0000:06:00.0: Error clearing ASSOC_MSK on BSS (-110)
[  842.041892] iwlagn 0000:06:00.0: MAC is in deep sleep!.  CSR_GP_CNTRL = 0xFFFFFFFF
[  842.060767] iwlagn 0000:06:00.0: MAC is in deep sleep!.  CSR_GP_CNTRL = 0xFFFFFFFF
[  842.079620] iwlagn 0000:06:00.0: MAC is in deep sleep!.  CSR_GP_CNTRL = 0xFFFFFFFF

so, the same problem, but it occurs later than ubuntu.

Okay. See my last reply, 

A comparision for you:

Fedora 15:
driver: iwlagn
version: 2.6.38.6-26.rc1.fc15.x86_64
firmware-version: 8.83.5.1 build 33692
bus-info: 0000:06:00.0

KUbuntu:
driver: iwlagn
version: 2.6.38-8-generic
firmware-version: 8.83.5.1 build 33692
bus-info: 0000:06:00.0

CentOS:
iwlagn 0000:06:00.0: firmware: requesting iwlwifi-5000-2.ucode
iwlagn 0000:06:00.0: loaded firmware version 8.24.2.12


Yes, I'm using the same wifi card, and various OS to earn more infos. 
I want to get the real problem and a fix for it... 

> on F14/15 and Ubuntu, you have newer firmware. Could you try to use the same
> firmware on centos 6?

Sure, of course. How can I do it? :)

(In reply to comment #10)
> very interesting, using the same NIC, right?
> 
> I do notice the firmware is different, you have
> iwlagn 0000:06:00.0: firmware: requesting iwlwifi-5000-2.ucode
> iwlagn 0000:06:00.0: loaded firmware version 8.24.2.12
> 
> on F14/15 and Ubuntu, you have newer firmware. Could you try to use the same
> firmware on centos 6?
> 
> Thanks
> Wey

yes, I also want to find the real problem and fix it for good :-)

for the older version of firmware, you can download from 
http://intellinuxwireless.org/?n=downloads
click [show older release...] under 5000 series, then choice the -2 version (the same one you have on centos), download it to /lib/firmware directory, please remember rename or remove the newer (-5) version, otherwise, driver will try to load the newer version first.

Another question, what kernel version centos using?

Thanks
Wey

Centos uname -a:

2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011 x86_64 x86_64 x86_64 GNU/Linux

fedora 15 liveusb kernel version:
2.6.38.6-26.rc1.fc15.x86_64

Okay, I find the page and get the iwlwifi-5000-ucode-8.24.2.12.tgz file.

I went to lib/firmware dir, and I find the iwlwifi ucode 2 file too. The size and date is the same, like the downloaded 2-ucode, so I'm just removed the 5-ucode, and restarted the system.
The only problem is, I cannot change the live usb filesystem... It's like readonly...

So I try centos, with the fedora newer firmware, maybe it's reproduce the previous problem...
It's not able to done, because centos cannot load the firmware. 

iwlagn 0000:06:00.0: firmware: requesting iwlwifi-5000-2.ucode
iwlagn 0000:06:00.0: Driver unable to support your firmware API. Driver supports v2, firmware is v0.
iwlagn 0000:06:00.0: Could not read microcode: -22

so now, I must install F15... :) I'm happy...

yeap, sorry about it, at 2.6.32 time, we do not have -5 version of firmware, so the driver will not take the newer firmware.

Really appreciated the help, it is very strange and do want to understand why?

10x

Wey

Okay okay, just a while, I'm installing fedora...

I testing again on installed system with ucode 5, and strange: it's dropped after 6 GB of data.

I rebooted and use the 2ucode firmware:

it's dropped near 1.6 gb data transfer with ucode 2. :) I'm confused...

I verified dmesg, so i'm sure that it used the 2ucode, but still hangs.

[   10.528310] iwlagn: Intel(R) Wireless WiFi Link AGN driver for Linux, in-tree:d
[   10.528314] iwlagn: Copyright(c) 2003-2010 Intel Corporation
[   10.528453] iwlagn 0000:06:00.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
[   10.528507] iwlagn 0000:06:00.0: setting latency timer to 64
[   10.528661] iwlagn 0000:06:00.0: Detected Intel(R) WiFi Link 5100 AGN, REV=0x54
[   10.552377] iwlagn 0000:06:00.0: device EEPROM VER=0x11f, CALIB=0x4
[   10.552380] iwlagn 0000:06:00.0: Device SKU: 0Xb
[   10.552399] iwlagn 0000:06:00.0: Tunable channels: 13 802.11bg, 24 802.11a channels
[   10.552528] iwlagn 0000:06:00.0: irq 43 for MSI/MSI-X
[   10.778611] iwlagn 0000:06:00.0: request for firmware file 'iwlwifi-5000-5.ucode' failed.
[   10.780692] iwlagn 0000:06:00.0: request for firmware file 'iwlwifi-5000-4.ucode' failed.
[   10.782485] iwlagn 0000:06:00.0: request for firmware file 'iwlwifi-5000-3.ucode' failed.
[   11.014475] iwlagn 0000:06:00.0: Firmware has old API version. Expected v5, got v2. New firmware can be obtained from http://www.intellinuxwireless.org.
[   11.014482] iwlagn 0000:06:00.0: loaded firmware version 8.24.2.12
[   11.242190] ieee80211 phy0: Selected rate control algorithm 'iwl-agn-rs'
[   83.937380] iwlagn 0000:06:00.0: iwlagn_tx_agg_start on ra = 74:ea:3a:a5:d3:28 tid = 0
[  389.710252] iwlagn 0000:06:00.0: Error sending REPLY_ADD_STA: time out after 500ms.
[  390.209983] iwlagn 0000:06:00.0: Error sending REPLY_QOS_PARAM: time out after 500ms.
[  390.209993] iwlagn 0000:06:00.0: Failed to update QoS
[  390.709763] iwlagn 0000:06:00.0: Error sending REPLY_RXON: time out after 500ms.
[  390.709777] iwlagn 0000:06:00.0: Error clearing ASSOC_MSK on BSS (-110)
[  391.209558] iwlagn 0000:06:00.0: Error sending REPLY_RXON: time out after 500ms.
[  391.209568] iwlagn 0000:06:00.0: Error clearing ASSOC_MSK on BSS (-110)
[  391.709410] iwlagn 0000:06:00.0: Error sending REPLY_QOS_PARAM: time out after 500ms.
[  391.709422] iwlagn 0000:06:00.0: Failed to update QoS
[  392.209127] iwlagn 0000:06:00.0: Error sending REPLY_RXON: time out after 500ms.
[  392.209138] iwlagn 0000:06:00.0: Error clearing ASSOC_MSK on BSS (-110)
[  392.209160] iwlagn 0000:06:00.0: Stopping AGG while state not ON or starting
[  392.708923] iwlagn 0000:06:00.0: Error sending REPLY_SCAN_CMD: time out after 500ms.
[  393.208682] iwlagn 0000:06:00.0: Error sending REPLY_ADD_STA: time out after 500ms.
[  393.708489] iwlagn 0000:06:00.0: Error sending REPLY_REMOVE_STA: time out after 500ms.
[  393.708501] iwlagn 0000:06:00.0: Error removing station 74:ea:3a:a5:d3:28
[  394.208336] iwlagn 0000:06:00.0: Error sending REPLY_RXON: time out after 500ms.
[  394.208346] iwlagn 0000:06:00.0: Error clearing ASSOC_MSK on BSS (-110)
[  396.209456] iwlagn 0000:06:00.0: fail to flush all tx fifo queues
[  396.546300] iwlagn 0000:06:00.0: Queue 4 stuck for 10000 ms.
[  396.546310] iwlagn 0000:06:00.0: On demand firmware reload
[  396.709225] iwlagn 0000:06:00.0: Error sending REPLY_RXON: time out after 500ms.
[  396.709236] iwlagn 0000:06:00.0: Error clearing ASSOC_MSK on BSS (-110)
[  396.728204] iwlagn 0000:06:00.0: MAC is in deep sleep!.  CSR_GP_CNTRL = 0xFFFFFFFF
[  396.747088] iwlagn 0000:06:00.0: MAC is in deep sleep!.  CSR_GP_CNTRL = 0xFFFFFFFF
[  396.765945] iwlagn 0000:06:00.0: MAC is in deep sleep!.  CSR_GP_CNTRL = 0xFFFFFFFF



(In reply to comment #15)
> yeap, sorry about it, at 2.6.32 time, we do not have -5 version of firmware, so
> the driver will not take the newer firmware.
> 
> Really appreciated the help, it is very strange and do want to understand why?
> 
> 10x
> 
> Wey

ok, it is not because different firmware, what else different, for sure centos has older kernel, does centos you install 32-bits or 64-bits?

btw, is it open or with security? if you kill both NetworkManager and wpa_supplicant and mke manual connect, is the connect still drop?

I guess I will ask you for the full log to see if can spot anything wrong, I am running out of idea :-(. But it do require re-compile the kernel and enable debugging option.

Wey

Centos is 64 bit and also fedora and ubuntu.

The kernel version are different. I now updating my kernel under fedora, so it will may take effect for the system. I try after again.


I used security. Under centos I used gnome-network manager applet, fedora and ubuntu I used kde plasma default network applet.

I may try to connect via command line.

Another idea. There were an drop, when I removed the power cable from notebook. But I'm not sure it depends on.

now I must sleep, because here is midnight...

Tomorrow again, thanx for you!

thanks for helping, I got another few ideas, but all require re-compile the kernel, hope you are up to it.

1. Provide the full debug log, hope we can spot something wrong
2. Generate uCode event log, you need to capture the log from debugfs, then we can have some idea what uCode are doing
3. if you disable security, does it still drop, I guess when you use command line, it will be very easy to control it.

Thanks
Wey

Okay. 

At the thrid point, I've tested without any security. Both with ucode2 and ucode5 firmware it dropped very fast. It dropped only when traffic goes through the interface. So, I think it's a kernel problem.

Can I downgrade kernel, to the version of centos under fedora? They are using the same kernel, or it's developed on a different way?

For enabling debug mode in kernel, kernel recompile, and ucode event log, debugfs tasks you can help me via mail. I think we shouldn't use the bugzilla's system, because it is for sending facts, not howtos.

Any idea? 

Thanks a lot, Balazs.

Note that fedora kernel is compiled with 

CONFIG_IWLWIFI_DEBUG=y
CONFIG_IWLWIFI_DEBUGFS=y
# CONFIG_IWLWIFI_DEVICE_TRACING is not set

To get debug messages is enough to do:

# modprobe -r iwlagn
# modprobe iwlagn debug=0x47fff (or whatever debug option).

Debugfs can be mounted using

# mount -t debugfs debugfs /sys/kernel/debug

then proper file from /sys/kernel/debug/ieee80211/phy0/iwlagn can be read.

RHEL "normal" kernels are compiled without iwlwifi debug, kernel-debug contains these options. RHEL6.0 iwlwifi is 2.6.32 based , RHEL6.1 iwlwifi is 2.6.37 based.

the problem always with fail to send command to uCode, could you load the driver with

modprobe iwlagn debug=0x26

it will only show host command information, so will be less noisy in dmesg log

also, you can disable the wd timer, so firmware will not reload

modprobe iwlagn wd_disable=1

Thanks
Wey

Please provide info requested in comment 22

I sent it to Meanakshi... She said that you're workng on it. You 're sitting on the problem, and doesn't try to fix it.

Try to buy an intel card, and diagnose the problem. I'm trying to help you through 2 months, but there were no success. Solution of the problem is not my main task: I'm not the software developer of intel company. I'm jut a man who want to use the hardware that bought.

Balazs

Hi!

Here are the tests with fedora wireless 3.0.0.+ kernel:

[root@jeefoo-notebook wifitest]# tail -f /var/log/messages
Sep  9 06:44:11 jeefoo-notebook bluetoothd[4003]: Can't open HCI socket: Address family not supported by protocol (97)
Sep  9 06:44:11 jeefoo-notebook bluetoothd[4003]: adapter_ops_setup failed
Sep  9 06:44:11 jeefoo-notebook bluetoothd[4003]: bluetoothd[4003]: Can't open HCI socket: Address family not supported by protocol (97)
Sep  9 06:44:11 jeefoo-notebook bluetoothd[4003]: bluetoothd[4003]: adapter_ops_setup failed
Sep  9 06:44:11 jeefoo-notebook systemd[1]: bluetooth.service: main process exited, code=exited, status=1
Sep  9 06:44:11 jeefoo-notebook NetworkManager[2914]: <warn> bluez error getting default adapter: Message did not receive a reply (timeout by message bus)
Sep  9 06:44:11 jeefoo-notebook pulseaudio[3958]: bluetooth-util.c: Error from ListAdapters reply: org.freedesktop.DBus.Error.NoReply
Sep  9 06:44:11 jeefoo-notebook systemd[1]: Unit bluetooth.service entered failed state.
Sep  9 06:44:39 jeefoo-notebook kernel: [   77.306041] iwlagn 0000:06:00.0: Aggregation not enabled for tid 0 because load = 0
Sep  9 06:44:45 jeefoo-notebook kernel: [   84.009889] iwlagn 0000:06:00.0: Aggregation not enabled for tid 0 because load = 5
Sep  9 06:48:23 jeefoo-notebook kernel: [  301.740092] kworker/u:0 used greatest stack depth: 3640 bytes left
Sep  9 06:48:51 jeefoo-notebook kernel: [  329.800483] iwlagn 0000:06:00.0: Aggregation not enabled for tid 0 because load = 8
Sep  9 06:50:01 jeefoo-notebook kernel: [  399.179735] iwlagn 0000:06:00.0: iwlagn_tx_agg_start on ra = 74:ea:3a:a5:d3:28 tid = 0
Sep  9 06:50:07 jeefoo-notebook dbus-daemon: [system] Activating service name='org.freedesktop.PackageKit' (using servicehelper)
Sep  9 06:50:07 jeefoo-notebook dbus-daemon: [system] Successfully activated service 'org.freedesktop.PackageKit'
Sep  9 06:55:18 jeefoo-notebook dbus-daemon: [system] Activating service name='org.freedesktop.PackageKit' (using servicehelper)
Sep  9 06:55:18 jeefoo-notebook dbus-daemon: [system] Successfully activated service 'org.freedesktop.PackageKit'
Sep  9 07:00:23 jeefoo-notebook dbus-daemon: [system] Activating service name='org.freedesktop.PackageKit' (using servicehelper)
Sep  9 07:00:23 jeefoo-notebook dbus-daemon: [system] Successfully activated service 'org.freedesktop.PackageKit'
Sep  9 07:02:11 jeefoo-notebook kernel: [ 1129.025809] FS-Cache: Loaded
Sep  9 07:02:11 jeefoo-notebook kernel: [ 1129.028889] FS-Cache: Netfs 'cifs' registered for caching
Sep  9 07:02:11 jeefoo-notebook kernel: [ 1129.038580] CIFS VFS: default security mechanism requested.  The default security mechanism will be upgraded from ntlm to ntlmv2 in kernel release 3.1
Sep  9 07:02:34 jeefoo-notebook kernel: [ 1152.329139] CIFS VFS: did not end path lookup where expected namelen is 0
Sep  9 07:05:28 jeefoo-notebook dbus-daemon: [system] Activating service name='org.freedesktop.PackageKit' (using servicehelper)
Sep  9 07:05:28 jeefoo-notebook dbus-daemon: [system] Successfully activated service 'org.freedesktop.PackageKit'
Sep  9 07:10:33 jeefoo-notebook dbus-daemon: [system] Activating service name='org.freedesktop.PackageKit' (using servicehelper)
Sep  9 07:10:33 jeefoo-notebook dbus-daemon: [system] Successfully activated service 'org.freedesktop.PackageKit'
Sep  9 07:15:38 jeefoo-notebook dbus-daemon: [system] Activating service name='org.freedesktop.PackageKit' (using servicehelper)
Sep  9 07:15:38 jeefoo-notebook dbus-daemon: [system] Successfully activated service 'org.freedesktop.PackageKit'
Sep  9 07:19:56 jeefoo-notebook kernel: [ 2193.943422] iwlagn 0000:06:00.0: Error sending REPLY_ADD_STA: time out after 500ms.
Sep  9 07:19:57 jeefoo-notebook kernel: [ 2194.443234] iwlagn 0000:06:00.0: Error sending REPLY_QOS_PARAM: time out after 500ms.
Sep  9 07:19:57 jeefoo-notebook kernel: [ 2194.443244] iwlagn 0000:06:00.0: Failed to update QoS
Sep  9 07:19:57 jeefoo-notebook kernel: [ 2194.942958] iwlagn 0000:06:00.0: Error sending REPLY_RXON: time out after 500ms.
Sep  9 07:19:57 jeefoo-notebook kernel: [ 2194.942969] iwlagn 0000:06:00.0: Error clearing ASSOC_MSK on BSS (-110)
Sep  9 07:19:58 jeefoo-notebook kernel: [ 2195.442763] iwlagn 0000:06:00.0: Error sending REPLY_RXON: time out after 500ms.
Sep  9 07:19:58 jeefoo-notebook kernel: [ 2195.442774] iwlagn 0000:06:00.0: Error clearing ASSOC_MSK on BSS (-110)
Sep  9 07:19:58 jeefoo-notebook kernel: [ 2195.942536] iwlagn 0000:06:00.0: Error sending REPLY_RXON: time out after 500ms.
Sep  9 07:19:58 jeefoo-notebook kernel: [ 2195.942546] iwlagn 0000:06:00.0: Error clearing ASSOC_MSK on BSS (-110)
Sep  9 07:19:58 jeefoo-notebook kernel: [ 2195.942560] iwlagn 0000:06:00.0: Stopping AGG while state not ON or starting
Sep  9 07:19:59 jeefoo-notebook kernel: [ 2196.451326] iwlagn 0000:06:00.0: Error sending REPLY_ADD_STA: time out after 500ms.
Sep  9 07:19:59 jeefoo-notebook kernel: [ 2196.451341] ieee80211 phy0: failed to remove key (0, 74:ea:3a:a5:d3:28) from hardware (-110)
Sep  9 07:19:59 jeefoo-notebook kernel: [ 2196.951158] iwlagn 0000:06:00.0: Error sending REPLY_REMOVE_STA: time out after 500ms.
Sep  9 07:19:59 jeefoo-notebook kernel: [ 2196.951166] iwlagn 0000:06:00.0: Error removing station 74:ea:3a:a5:d3:28
Sep  9 07:20:00 jeefoo-notebook kernel: [ 2197.459905] iwlagn 0000:06:00.0: Error sending REPLY_RXON: time out after 500ms.
Sep  9 07:20:00 jeefoo-notebook kernel: [ 2197.459915] iwlagn 0000:06:00.0: Error clearing ASSOC_MSK on BSS (-110)
Sep  9 07:20:02 jeefoo-notebook kernel: [ 2199.460128] iwlagn 0000:06:00.0: fail to flush all tx fifo queues
Sep  9 07:20:02 jeefoo-notebook kernel: [ 2199.959888] iwlagn 0000:06:00.0: Error sending REPLY_RXON: time out after 500ms.
Sep  9 07:20:02 jeefoo-notebook kernel: [ 2199.959897] iwlagn 0000:06:00.0: Error clearing ASSOC_MSK on BSS (-110)
Sep  9 07:20:02 jeefoo-notebook dbus-daemon: [system] Activating service name='net.reactivated.Fprint' (using servicehelper)
Sep  9 07:20:02 jeefoo-notebook dbus-daemon: [system] Successfully activated service 'net.reactivated.Fprint'
Sep  9 07:20:03 jeefoo-notebook NetworkManager[2914]: <info> (wlan0): roamed from BSSID 74:EA:3A:A5:D3:28 (LAN) to (none) ((none))
Sep  9 07:20:03 jeefoo-notebook kernel: [ 2200.467709] iwlagn 0000:06:00.0: Error sending REPLY_ADD_STA: time out after 500ms.
Sep  9 07:20:03 jeefoo-notebook kernel: [ 2200.467720] ieee80211 phy0: failed to remove key (1, ff:ff:ff:ff:ff:ff) from hardware (-110)
Sep  9 07:20:03 jeefoo-notebook kernel: [ 2200.467889] cfg80211: Calling CRDA to update world regulatory domain
Sep  9 07:20:03 jeefoo-notebook kernel: [ 2200.887488] iwlagn 0000:06:00.0: Queue 4 stuck for 10000 ms.
Sep  9 07:20:03 jeefoo-notebook kernel: [ 2200.887495] iwlagn 0000:06:00.0: On demand firmware reload
Sep  9 07:20:03 jeefoo-notebook kernel: [ 2200.887607] iwlagn 0000:06:00.0: Request scan called when driver not ready.
Sep  9 07:20:03 jeefoo-notebook kernel: [ 2200.887616] ieee80211 phy0: Hardware restart was requested
Sep  9 07:20:03 jeefoo-notebook kernel: [ 2200.888688] cfg80211: World regulatory domain updated:
Sep  9 07:20:03 jeefoo-notebook kernel: [ 2200.888692] cfg80211:     (start_freq - end_freq @ bandwidth), (max_antenna_gain, max_eirp)
Sep  9 07:20:03 jeefoo-notebook kernel: [ 2200.888696] cfg80211:     (2402000 KHz - 2472000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
Sep  9 07:20:03 jeefoo-notebook kernel: [ 2200.888699] cfg80211:     (2457000 KHz - 2482000 KHz @ 20000 KHz), (300 mBi, 2000 mBm)
Sep  9 07:20:03 jeefoo-notebook kernel: [ 2200.888702] cfg80211:     (2474000 KHz - 2494000 KHz @ 20000 KHz), (300 mBi, 2000 mBm)
Sep  9 07:20:03 jeefoo-notebook kernel: [ 2200.888706] cfg80211:     (5170000 KHz - 5250000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
Sep  9 07:20:03 jeefoo-notebook kernel: [ 2200.888709] cfg80211:     (5735000 KHz - 5835000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
Sep  9 07:20:03 jeefoo-notebook kernel: [ 2200.888932] cfg80211: Calling CRDA for country: HU
Sep  9 07:20:03 jeefoo-notebook kernel: [ 2200.893374] cfg80211: Regulatory domain changed to country: HU
Sep  9 07:20:03 jeefoo-notebook kernel: [ 2200.893381] cfg80211:     (start_freq - end_freq @ bandwidth), (max_antenna_gain, max_eirp)
Sep  9 07:20:03 jeefoo-notebook kernel: [ 2200.893387] cfg80211:     (2402000 KHz - 2482000 KHz @ 40000 KHz), (N/A, 2000 mBm)
Sep  9 07:20:03 jeefoo-notebook kernel: [ 2200.893393] cfg80211:     (5170000 KHz - 5250000 KHz @ 40000 KHz), (N/A, 2000 mBm)
Sep  9 07:20:03 jeefoo-notebook kernel: [ 2200.893413] cfg80211:     (5250000 KHz - 5330000 KHz @ 40000 KHz), (N/A, 2000 mBm)
Sep  9 07:20:03 jeefoo-notebook kernel: [ 2200.893420] cfg80211:     (5490000 KHz - 5710000 KHz @ 40000 KHz), (N/A, 2700 mBm)
Sep  9 07:20:03 jeefoo-notebook kernel: [ 2200.906524] iwlagn 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Sep  9 07:20:03 jeefoo-notebook kernel: [ 2200.925407] iwlagn 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Sep  9 07:20:03 jeefoo-notebook kernel: [ 2200.944317] iwlagn 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Sep  9 07:20:03 jeefoo-notebook kernel: [ 2200.963321] iwlagn 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Sep  9 07:20:03 jeefoo-notebook kernel: [ 2200.982728] iwlagn 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Sep  9 07:20:03 jeefoo-notebook kernel: [ 2201.001635] iwlagn 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Sep  9 07:20:03 jeefoo-notebook kernel: [ 2201.020519] iwlagn 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Sep  9 07:20:03 jeefoo-notebook kernel: [ 2201.039397] iwlagn 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Sep  9 07:20:03 jeefoo-notebook kernel: [ 2201.058325] iwlagn 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Sep  9 07:20:03 jeefoo-notebook kernel: [ 2201.077224] iwlagn 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Sep  9 07:20:03 jeefoo-notebook kernel: [ 2201.096131] iwlagn 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Sep  9 07:20:03 jeefoo-notebook kernel: [ 2201.115058] iwlagn 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Sep  9 07:20:03 jeefoo-notebook kernel: [ 2201.133947] iwlagn 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Sep  9 07:20:03 jeefoo-notebook kernel: [ 2201.152900] iwlagn 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Sep  9 07:20:03 jeefoo-notebook kernel: [ 2201.171802] iwlagn 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Sep  9 07:20:03 jeefoo-notebook kernel: [ 2201.190671] iwlagn 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Sep  9 07:20:03 jeefoo-notebook kernel: [ 2201.209585] iwlagn 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Sep  9 07:20:03 jeefoo-notebook kernel: [ 2201.228539] iwlagn 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Sep  9 07:20:04 jeefoo-notebook kernel: [ 2201.247495] iwlagn 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Sep  9 07:20:04 jeefoo-notebook kernel: [ 2201.266408] iwlagn 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Sep  9 07:20:04 jeefoo-notebook kernel: [ 2201.285423] iwlagn 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Sep  9 07:20:04 jeefoo-notebook kernel: [ 2201.304340] iwlagn 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Sep  9 07:20:04 jeefoo-notebook kernel: [ 2201.323279] iwlagn 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Sep  9 07:20:04 jeefoo-notebook kernel: [ 2201.342262] iwlagn 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Sep  9 07:20:04 jeefoo-notebook kernel: [ 2201.361251] iwlagn 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Sep  9 07:20:04 jeefoo-notebook kernel: [ 2201.380195] iwlagn 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Sep  9 07:20:04 jeefoo-notebook kernel: [ 2201.399133] iwlagn 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Sep  9 07:20:04 jeefoo-notebook kernel: [ 2201.418126] iwlagn 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Sep  9 07:20:04 jeefoo-notebook kernel: [ 2201.437109] iwlagn 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Sep  9 07:20:04 jeefoo-notebook kernel: [ 2201.456106] iwlagn 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Sep  9 07:20:04 jeefoo-notebook kernel: [ 2201.475055] iwlagn 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Sep  9 07:20:04 jeefoo-notebook kernel: [ 2201.494049] iwlagn 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Sep  9 07:20:04 jeefoo-notebook kernel: [ 2201.513069] iwlagn 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Sep  9 07:20:04 jeefoo-notebook kernel: [ 2201.532060] iwlagn 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Sep  9 07:20:04 jeefoo-notebook kernel: [ 2201.551060] iwlagn 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Sep  9 07:20:04 jeefoo-notebook kernel: [ 2201.570013] iwlagn 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Sep  9 07:20:04 jeefoo-notebook kernel: [ 2201.588992] iwlagn 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Sep  9 07:20:04 jeefoo-notebook kernel: [ 2201.607970] iwlagn 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Sep  9 07:20:04 jeefoo-notebook kernel: [ 2201.626913] iwlagn 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Sep  9 07:20:09 jeefoo-notebook NetworkManager[2914]: <info> (wlan0): supplicant interface state: completed -> disconnected
Sep  9 07:20:09 jeefoo-notebook kernel: [ 2206.624067] iwlagn 0000:06:00.0: Could not load the INST uCode section
Sep  9 07:20:09 jeefoo-notebook kernel: [ 2206.624077] iwlagn 0000:06:00.0: Failed to start RT ucode: -110
Sep  9 07:20:09 jeefoo-notebook kernel: [ 2206.624110] iwlagn 0000:06:00.0: Unable to initialize device.
Sep  9 07:20:10 jeefoo-notebook kernel: [ 2207.625105] iwlagn 0000:06:00.0: Request scan called when driver not ready.
Sep  9 07:20:11 jeefoo-notebook kernel: [ 2208.626283] iwlagn 0000:06:00.0: Request scan called when driver not ready.
Sep  9 07:20:12 jeefoo-notebook kernel: [ 2209.627335] iwlagn 0000:06:00.0: Request scan called when driver not ready.
Sep  9 07:20:13 jeefoo-notebook kernel: [ 2210.666704] iwlagn 0000:06:00.0: Request scan called when driver not ready.
Sep  9 07:20:14 jeefoo-notebook kernel: [ 2211.667077] iwlagn 0000:06:00.0: Request scan called when driver not ready.
Sep  9 07:20:15 jeefoo-notebook kernel: [ 2212.668144] iwlagn 0000:06:00.0: Request scan called when driver not ready.
Sep  9 07:20:16 jeefoo-notebook kernel: [ 2213.669181] iwlagn 0000:06:00.0: Request scan called when driver not ready.
Sep  9 07:20:17 jeefoo-notebook kernel: [ 2214.670164] iwlagn 0000:06:00.0: Request scan called when driver not ready.
Sep  9 07:20:18 jeefoo-notebook kernel: [ 2215.671223] iwlagn 0000:06:00.0: Request scan called when driver not ready.
Sep  9 07:20:19 jeefoo-notebook kernel: [ 2216.671271] iwlagn 0000:06:00.0: Request scan called when driver not ready.
Sep  9 07:20:20 jeefoo-notebook kernel: [ 2217.672291] iwlagn 0000:06:00.0: Request scan called when driver not ready.
Sep  9 07:20:21 jeefoo-notebook kernel: [ 2218.673378] iwlagn 0000:06:00.0: Request scan called when driver not ready.
Sep  9 07:20:22 jeefoo-notebook kernel: [ 2219.674378] iwlagn 0000:06:00.0: Request scan called when driver not ready.
Sep  9 07:20:23 jeefoo-notebook kernel: [ 2220.675495] iwlagn 0000:06:00.0: Request scan called when driver not ready.
Sep  9 07:20:24 jeefoo-notebook kernel: [ 2221.676510] iwlagn 0000:06:00.0: Request scan called when driver not ready.
Sep  9 07:20:25 jeefoo-notebook NetworkManager[2914]: <warn> (wlan0): link timed out.
Sep  9 07:20:25 jeefoo-notebook NetworkManager[2914]: <info> (wlan0): device state change: activated -> disconnected (reason 'supplicant-timeout') [100 30 11]
Sep  9 07:20:25 jeefoo-notebook NetworkManager[2914]: <info> (wlan0): deactivating device (reason: 11).
Sep  9 07:20:25 jeefoo-notebook NetworkManager[2914]: <info> (wlan0): canceled DHCP transaction, DHCP client pid 3635
Sep  9 07:20:25 jeefoo-notebook avahi-daemon[2901]: Withdrawing address record for 10.1.1.20 on wlan0.
Sep  9 07:20:25 jeefoo-notebook avahi-daemon[2901]: Leaving mDNS multicast group on interface wlan0.IPv4 with address 10.1.1.20.
Sep  9 07:20:25 jeefoo-notebook avahi-daemon[2901]: Interface wlan0.IPv4 no longer relevant for mDNS.
Sep  9 07:20:25 jeefoo-notebook NetworkManager[2914]: <info> Auto-activating connection 'LAN'.
Sep  9 07:20:25 jeefoo-notebook NetworkManager[2914]: <info> Activation (wlan0) starting connection 'LAN'
Sep  9 07:20:25 jeefoo-notebook NetworkManager[2914]: <info> (wlan0): device state change: disconnected -> prepare (reason 'none') [30 40 0]
Sep  9 07:20:25 jeefoo-notebook NetworkManager[2914]: <info> Activation (wlan0) Stage 1 of 5 (Device Prepare) scheduled...
Sep  9 07:20:25 jeefoo-notebook NetworkManager[2914]: <info> Activation (wlan0) Stage 1 of 5 (Device Prepare) started...
Sep  9 07:20:25 jeefoo-notebook NetworkManager[2914]: <info> Activation (wlan0) Stage 2 of 5 (Device Configure) scheduled...
Sep  9 07:20:25 jeefoo-notebook NetworkManager[2914]: <info> Activation (wlan0) Stage 1 of 5 (Device Prepare) complete.
Sep  9 07:20:25 jeefoo-notebook NetworkManager[2914]: <info> Activation (wlan0) Stage 2 of 5 (Device Configure) starting...
Sep  9 07:20:25 jeefoo-notebook NetworkManager[2914]: <info> (wlan0): device state change: prepare -> config (reason 'none') [40 50 0]
Sep  9 07:20:25 jeefoo-notebook dbus-daemon: [system] Activating service name='org.freedesktop.nm_dispatcher' (using servicehelper)
Sep  9 07:20:25 jeefoo-notebook NetworkManager[2914]: <info> Activation (wlan0/wireless): access point 'LAN' has security, but secrets are required.
Sep  9 07:20:25 jeefoo-notebook NetworkManager[2914]: <info> (wlan0): device state change: config -> need-auth (reason 'none') [50 60 0]
Sep  9 07:20:25 jeefoo-notebook NetworkManager[2914]: <info> Activation (wlan0) Stage 2 of 5 (Device Configure) complete.
Sep  9 07:20:25 jeefoo-notebook NetworkManager[2914]: <info> Activation (wlan0) Stage 1 of 5 (Device Prepare) scheduled...
Sep  9 07:20:25 jeefoo-notebook NetworkManager[2914]: <info> Activation (wlan0) Stage 1 of 5 (Device Prepare) started...
Sep  9 07:20:25 jeefoo-notebook NetworkManager[2914]: <info> (wlan0): device state change: need-auth -> prepare (reason 'none') [60 40 0]
Sep  9 07:20:25 jeefoo-notebook NetworkManager[2914]: <info> Activation (wlan0) Stage 2 of 5 (Device Configure) scheduled...
Sep  9 07:20:25 jeefoo-notebook NetworkManager[2914]: <info> Activation (wlan0) Stage 1 of 5 (Device Prepare) complete.
Sep  9 07:20:25 jeefoo-notebook NetworkManager[2914]: <info> Activation (wlan0) Stage 2 of 5 (Device Configure) starting...
Sep  9 07:20:25 jeefoo-notebook NetworkManager[2914]: <info> (wlan0): device state change: prepare -> config (reason 'none') [40 50 0]
Sep  9 07:20:25 jeefoo-notebook NetworkManager[2914]: <info> Activation (wlan0/wireless): connection 'LAN' has security, and secrets exist.  No new secrets needed.
Sep  9 07:20:25 jeefoo-notebook NetworkManager[2914]: <info> Config: added 'ssid' value 'LAN'
Sep  9 07:20:25 jeefoo-notebook NetworkManager[2914]: <info> Config: added 'scan_ssid' value '1'
Sep  9 07:20:25 jeefoo-notebook NetworkManager[2914]: <info> Config: added 'key_mgmt' value 'WPA-PSK'
Sep  9 07:20:25 jeefoo-notebook NetworkManager[2914]: <info> Config: added 'psk' value '<omitted>'
Sep  9 07:20:25 jeefoo-notebook NetworkManager[2914]: <info> Config: added 'proto' value 'WPA RSN'
Sep  9 07:20:25 jeefoo-notebook NetworkManager[2914]: <info> Activation (wlan0) Stage 2 of 5 (Device Configure) complete.
Sep  9 07:20:25 jeefoo-notebook NetworkManager[2914]: <info> Config: set interface ap_scan to 1
Sep  9 07:20:25 jeefoo-notebook dbus-daemon: [system] Successfully activated service 'org.freedesktop.nm_dispatcher'
Sep  9 07:20:25 jeefoo-notebook kernel: [ 2222.676752] iwlagn 0000:06:00.0: Request scan called when driver not ready.
Sep  9 07:20:26 jeefoo-notebook kernel: [ 2223.677017] iwlagn 0000:06:00.0: Request scan called when driver not ready.
Sep  9 07:20:27 jeefoo-notebook kernel: [ 2224.678026] iwlagn 0000:06:00.0: Request scan called when driver not ready.
Sep  9 07:20:28 jeefoo-notebook kernel: [ 2225.678146] iwlagn 0000:06:00.0: Request scan called when driver not ready.
Sep  9 07:20:29 jeefoo-notebook kernel: [ 2226.678234] iwlagn 0000:06:00.0: Request scan called when driver not ready.
Sep  9 07:20:30 jeefoo-notebook kernel: [ 2227.679460] iwlagn 0000:06:00.0: Request scan called when driver not ready.
Sep  9 07:20:31 jeefoo-notebook kernel: [ 2228.680529] iwlagn 0000:06:00.0: Request scan called when driver not ready.
Sep  9 07:20:32 jeefoo-notebook kernel: [ 2229.681694] iwlagn 0000:06:00.0: Request scan called when driver not ready.
Sep  9 07:20:33 jeefoo-notebook kernel: [ 2230.682751] iwlagn 0000:06:00.0: Request scan called when driver not ready.
Sep  9 07:20:34 jeefoo-notebook kernel: [ 2231.683787] iwlagn 0000:06:00.0: Request scan called when driver not ready.
Sep  9 07:20:35 jeefoo-notebook kernel: [ 2232.685486] iwlagn 0000:06:00.0: Request scan called when driver not ready.
Sep  9 07:20:36 jeefoo-notebook kernel: [ 2233.686488] iwlagn 0000:06:00.0: Request scan called when driver not ready.
Sep  9 07:20:37 jeefoo-notebook kernel: [ 2234.687500] iwlagn 0000:06:00.0: Request scan called when driver not ready.
Sep  9 07:20:38 jeefoo-notebook kernel: [ 2235.687941] iwlagn 0000:06:00.0: Request scan called when driver not ready.
Sep  9 07:20:39 jeefoo-notebook kernel: [ 2236.688062] iwlagn 0000:06:00.0: Request scan called when driver not ready.
Sep  9 07:20:40 jeefoo-notebook kernel: [ 2237.689171] iwlagn 0000:06:00.0: Request scan called when driver not ready.
Sep  9 07:20:41 jeefoo-notebook kernel: [ 2238.690201] iwlagn 0000:06:00.0: Request scan called when driver not ready.
Sep  9 07:20:42 jeefoo-notebook kernel: [ 2239.691427] iwlagn 0000:06:00.0: Request scan called when driver not ready.
Sep  9 07:20:43 jeefoo-notebook kernel: [ 2240.692526] iwlagn 0000:06:00.0: Request scan called when driver not ready.
Sep  9 07:20:44 jeefoo-notebook dbus-daemon: [system] Activating service name='org.freedesktop.PackageKit' (using servicehelper)
Sep  9 07:20:44 jeefoo-notebook dbus-daemon: [system] Successfully activated service 'org.freedesktop.PackageKit'
Sep  9 07:20:44 jeefoo-notebook kernel: [ 2241.693755] iwlagn 0000:06:00.0: Request scan called when driver not ready.
Sep  9 07:20:45 jeefoo-notebook kernel: [ 2242.695406] iwlagn 0000:06:00.0: Request scan called when driver not ready.
Sep  9 07:20:46 jeefoo-notebook kernel: [ 2243.695598] iwlagn 0000:06:00.0: Request scan called when driver not ready.
Sep  9 07:20:47 jeefoo-notebook kernel: [ 2244.696680] iwlagn 0000:06:00.0: Request scan called when driver not ready.
Sep  9 07:20:48 jeefoo-notebook kernel: [ 2245.697692] iwlagn 0000:06:00.0: Request scan called when driver not ready.
Sep  9 07:20:49 jeefoo-notebook kernel: [ 2246.697850] iwlagn 0000:06:00.0: Request scan called when driver not ready.
Sep  9 07:20:50 jeefoo-notebook NetworkManager[2914]: <warn> Activation (wlan0/wireless): association took too long.
Sep  9 07:20:50 jeefoo-notebook NetworkManager[2914]: <info> (wlan0): device state change: config -> need-auth (reason 'none') [50 60 0]
Sep  9 07:20:50 jeefoo-notebook NetworkManager[2914]: <warn> Activation (wlan0/wireless): asking for new secrets
Sep  9 07:20:50 jeefoo-notebook NetworkManager[2914]: <info> (wlan0): supplicant interface state: disconnected -> inactive

cheers Balazs

On 09/08/2011 08:34 PM, Venkataraman, Meenakshi wrote:
>
> Hi Balazs,
>
>  
>
> Can you please run a quick test with the watchdog timer disabled?
>
>  
>
> Copied below is Wey-Yi’s comment from bugzilla:
>
>  
>
> --snip--
>
> the problem always with fail to send command to uCode, could you load the
>
> driver with
>
>  
>
> modprobe iwlagn debug=0x26
>
>  
>
> it will only show host command information, so will be less noisy in dmesg log
>
>  
>
> also, you can disable the wd timer, so firmware will not reload
>
>  
>
> modprobe iwlagn wd_disable=1
>
>  
>
> --snip--
>
>  
>
> Thanks,
>
> Meenakshi

(In reply to comment #24)
> I sent it to Meanakshi... 
Who is Meanakasi? If you send someone information in private email, how do you expect I will know you provided requested info?

> She said that you're workng on it. 
No, I do not, bug is not assigned to me. I thought Intel is looking at it and need info requested in comment 22.

> You 're sitting on
> the problem, and doesn't try to fix it.
Yes, I have tons of other things to do. 

> Try to buy an intel card, and diagnose the problem. 
I have 5100 and it works with any APs with any configuration I use.

Ok, let's open this bug for now.

Sorry people...that went out of hand.

I'm working on this bug offline and exchanged several emails with data from Balazs, as we're not able to reproduce the problem with any of our test setups here either.

I've also looked at the logs s/he provided, and the only thing I can isolate is that for some reason on his/her a queue is stuck, there's an error sending a host command, and then we lose the PCI bus; requiring a reboot to restore the WiFi back.

Now, my search on this issue revealed that this problem has existed for a while before Balazs reported the problem. Since we're unable to reproduce it here, I'm trying to gather all information I can to characterize the problem.

We'll hopefully get back to all users experiencing this problem with a fix soon. Thanks for your patience & sorry for the confusion. I'll request for info on the mailing list in future.

Thanks,
Meenakshi

I also have another question -- do you see this problem after you suspend/resume your system? Or do you run into it as soon as you boot up?

Hi!

Under fedora 15, the problem when I trying to suspend or shutdown the laptop, I cannot do that.

The suspend don't works, it's come back immediately.
When I issue a shutdown command, it's freezing near unmounting filesystems.
I think there is something which holds the system to execute the shutdown.

Before the time, with fedora 14, there were no problem with these functions, and I used them often.

Additional info:
In my desktop computer, on f15, the suspend do the same, but the shutdown always works well.
In f14, all went fine.

Thankx Balazs.

Thanks Balazs,

Are you talking about problems related to the Intel WiFi card during system suspend, hibernate, and resume?

Meenakshi

Maybe, but I cannot be sure.

I think this is unrelated to suspend/hibernate, since that not work on F-15 on  Balazs system, but iwlwifi issue is present.

Agree with Stanislaw, Meenakshi, I also don't think it has anything to do with 
D3 state.

Thanks
Wey

I'm facing something similar. 

My wireless card:
03:00.0 Network controller: Intel Corporation PRO/Wireless 5100 AGN [Shiloh] Network Connection



I was able to cure it by:

modprobe iwlagn iwlagn wd_disable=1


The reason I did this was because once I had a panic which said something about 
MAC going into deep sleep. A subsequent search on forums showed wd_disable=1
to be the workaround.

Hope it helps someone. However, a fix is really desirable.

hmm, the idea of watchdog timer is try to recover from firmware not responding for a period of time. Looks like it create more problems than fixing it. Maybe it is not a good idea for all the devices.

I am not sure watchdog timer is the only problem here, but I will submit patch to enable/disable per devices base. thought?

Thanks
Wey

Its so interesting, just after you posted the message about not being sure whether the watchdog timer is the only problem, my network card crashed again!
So, as you rightly suspect, the problem lies elsewhere.

Having said that, I'd still like to note that earlier, where my network card would crash during boot time (and by the time I logged into gnome, there was no sign of wlan0) - now this problem occurred about 1.5 days after I set wd_disable=1.

So wd_disable=1 does seem to cure the symptom in some way but not the cause.
Maybe thats a hint to whats going on. I dont know.

yup, I think the watch dog timer try to treat the problem too aggressive. my guess it is more or less the Firmware get into a bad state which may or may not get out of it by itself, by trigger the watch dog timer and reload the firmware, sometime make the case even worse. But it is just my guess, the bad news is I have no idea how to reproduce it on the system I have.

Wey

I can again set wd_disable=1 and try to reproduce it in case you want some more
logs. Let me know.

I can again set wd_disable=0 and try to reproduce it in case you want some more
logs. Let me know.

sure, if you can test with both and make sure it is not just coincedence.

Thanks
Wey

(In reply to comment #35)
> hmm, the idea of watchdog timer is try to recover from firmware not responding
> for a period of time. Looks like it create more problems than fixing it.
Actually this looks more like reset procedure problem than watchdog itself.

> I am not sure watchdog timer is the only problem here, but I will submit patch
> to enable/disable per devices base. thought?
I would be better to disable it completely. We have it because it allow make device functional when firmware hangs (at least with some cases). Disable it and fix firmware would be much more desirable.

Also I'm not sure if Sisir problem is the same as jeefoo has.

Tested. I think my card working with wd_disable=1 was a coincidence because yesterday even with wd_disable=1, I had to try logging for like 15 minutes to get one "good" login with wireless working. Very frustrating.

Maybe Sisir and me the two person on the world, whom have a problem with this intel wifi card. :) I put back to my notebook the previous realtek card, and I'm using it satisfied, because by the time the realtek driver was fixed, and working almost fully stable. I'm waiting for the solution, a new firmware, etc...

Balazs

Looks like this very same issue: http://lkml.org/lkml/2011/10/19/74
Incidentally Wey commented on it, then apparently it got dropped.

I believe I'm hitting this problem (at irregular intervals) on F16:

kernel-3.1.5-1
firmware-version: 9.221.4.1 build 25532

setting wd_disable got me back on the network, for now at least. Please let me know if you would like me to run any additional debugging.

(In reply to comment #45)
> I believe I'm hitting this problem (at irregular intervals) on F16:
> kernel-3.1.5-1
> firmware-version: 9.221.4.1 build 25532
> setting wd_disable got me back on the network, for now at least. Please let me
> know if you would like me to run any additional debugging.

Interesting, in breeding-edge tree, the wd_disable is ON for both 1000 and 5000. so what you seeing is, we also need to disable for 6000

Thanks
Wey

You mean the wifi device? lscpi reports it as

03:00.0 Network controller: Intel Corporation Centrino Ultimate-N 6300 (rev 35)

And in my syslog I see lines like

Dec 14 16:18:28 localhost kernel: [36144.991596] iwlagn 0000:03:00.0: Error sending REPLY_ADD_STA: time out after 2000ms.
Dec 14 16:18:28 localhost kernel: [36144.991605] iwlagn 0000:03:00.0: Adding station ff:ff:ff:ff:ff:ff failed.

I wonder how watchdog is breaking things. Looking at comment 36, I think driver has some race conditions, which are better reproducible with watchdog enabled.

In David case seems some current (post 3.0 or 3.1) driver changes cause breakage, but watchdog in form we have it now in the driver, is there since 2.6.38.

David, what was the latest kernel version where iwlwifi works for you without wd_disable=1 option?

A bit hard to tell since my laptop goes several hours before a problem occurres. But looking trough my log files, first time I saw the  " REPLY_ADD_STA: time out" message was with the 3.1.5-1 kernel while I did not see it with 3.1.4-1 or earlier.

So perhaps problem you are seeing is regression caused by ht40 changes (bug 767173), fixed in 3.1.5-6.fc16

I tried loading the module without wd_disable with kernel-3.1.5-6 and it worked fine for the remainder of the day.
The next day however, after resuming from suspend, wifi wouldn't work and in syslog I saw the following lines:

 Dec 21 09:39:45 localhost kernel: [12026.707368] iwlagn 0000:03:00.0: Queue 2 stuck for 2000 ms.
Dec 21 09:39:45 localhost kernel: [12026.707373] iwlagn 0000:03:00.0: On demand firmware reload
Dec 21 09:39:45 localhost kernel: [12026.707479] ieee80211 phy0: Hardware restart was requested
Dec 21 09:39:45 localhost kernel: [12026.707541] iwlagn 0000:03:00.0: L1 Enabled; Disabling L0S
Dec 21 09:39:45 localhost kernel: [12026.707708] iwlagn 0000:03:00.0: Radio type=0x0-0x3-0x1

Does this sound like it's related? Back now with wd_disable=1 and network works.

Maybe not surprisingly, the same(?) problem occurs also with the iwlwifi driver.

The following messages where seen in syslog:

Feb 20 12:23:42 localhost kernel: [ 3799.105646] iwlwifi 0000:03:00.0: Error sending REPLY_ADD_STA: time out after 2000ms.
Feb 20 12:23:42 localhost kernel: [ 3799.105652] iwlwifi 0000:03:00.0: Current CMD queue read_ptr 6 write_ptr 7
Feb 20 12:23:42 localhost kernel: [ 3799.105657] iwlwifi 0000:03:00.0: Adding station ff:ff:ff:ff:ff:ff failed.
Feb 20 12:23:42 localhost kernel: [ 3799.114771] iwlwifi 0000:03:00.0: ACTIVATE a non DRIVER active station id 15 addr ff:ff:ff:ff:ff:ff
Feb 20 12:23:42 localhost kernel: [ 3799.114784] iwlwifi 0000:03:00.0: HCMD_ACTIVE already clear for command REPLY_REMOVE_STA

And after that, the WLAN disconnected.

Reloaded the iwlwifi driver with wd_disable=1 and so far so good...

Maybe worth mentioning that comment 52 is on the 3.2.6-3.fc16 kernel

And acutally, with the 3.2.6-3.fc16.x86_64 kernel, I've gotten this problem twice in one afternoon now although I've set wd_disable=1

Test kernels w/ "mac80211: Fix a rwlock bad magic bug" reverted are building here:

http://koji.fedoraproject.org/koji/taskinfo?taskID=3804990

Please try to replicate the issue with those kernels, and post the results here...thanks!

Test kernel still working fine after a full working day. 

Also maybe worth noting is that I have _not_ set wd_disable=1

Been running the 3.2.7-1.fc16 kernel for more then a day now without any issues. Can we hope that this issue has been solved?

Hi All!

I decided to put back again my intel wifi link 5100 card and see what's news.
The problem still exists, like previously.

My question is: will the driver fixed in the future, or it cannot be fixed. Do you have new information about this bug, and is there any idea, to solve the problem?

The replacement card rtl8192SE wifi was worked stable, but very slow when copying largfe files
Intel 5100 goes up to 4-5MB/sec, but after few seconds it's hang with the following:

Thanks, Balazs

Mar 17 21:07:36 jeefoo-notebook kernel: [  284.611029] iwlwifi 0000:06:00.0: Error sending REPLY_ADD_STA: time out after 2000ms.
Mar 17 21:07:36 jeefoo-notebook kernel: [  284.611036] iwlwifi 0000:06:00.0: Current CMD queue read_ptr 246 write_ptr 250
Mar 17 21:07:38 jeefoo-notebook kernel: [  286.610352] iwlwifi 0000:06:00.0: Error sending REPLY_ADD_STA: time out after 2000ms.
Mar 17 21:07:38 jeefoo-notebook kernel: [  286.610361] iwlwifi 0000:06:00.0: Current CMD queue read_ptr 246 write_ptr 253
Mar 17 21:07:40 jeefoo-notebook kernel: [  288.609592] iwlwifi 0000:06:00.0: Error sending REPLY_QOS_PARAM: time out after 2000ms.
Mar 17 21:07:40 jeefoo-notebook kernel: [  288.609602] iwlwifi 0000:06:00.0: Current CMD queue read_ptr 246 write_ptr 0
Mar 17 21:07:40 jeefoo-notebook kernel: [  288.609608] iwlwifi 0000:06:00.0: Failed to update QoS
Mar 17 21:07:42 jeefoo-notebook kernel: [  290.608839] iwlwifi 0000:06:00.0: Error sending REPLY_RXON: time out after 2000ms.
Mar 17 21:07:42 jeefoo-notebook kernel: [  290.608847] iwlwifi 0000:06:00.0: Current CMD queue read_ptr 246 write_ptr 3
Mar 17 21:07:42 jeefoo-notebook kernel: [  290.608851] iwlwifi 0000:06:00.0: Error clearing ASSOC_MSK on BSS (-110)
Mar 17 21:07:44 jeefoo-notebook kernel: [  292.608087] iwlwifi 0000:06:00.0: Error sending REPLY_RXON: time out after 2000ms.
Mar 17 21:07:44 jeefoo-notebook kernel: [  292.608094] iwlwifi 0000:06:00.0: Current CMD queue read_ptr 246 write_ptr 6
Mar 17 21:07:44 jeefoo-notebook kernel: [  292.608098] iwlwifi 0000:06:00.0: Error clearing ASSOC_MSK on BSS (-110)
Mar 17 21:07:46 jeefoo-notebook kernel: [  294.607288] iwlwifi 0000:06:00.0: Error sending REPLY_RXON: time out after 2000ms.
Mar 17 21:07:46 jeefoo-notebook kernel: [  294.607297] iwlwifi 0000:06:00.0: Current CMD queue read_ptr 246 write_ptr 9
Mar 17 21:07:46 jeefoo-notebook kernel: [  294.607304] iwlwifi 0000:06:00.0: Error clearing ASSOC_MSK on BSS (-110)
Mar 17 21:07:48 jeefoo-notebook kernel: [  296.606570] iwlwifi 0000:06:00.0: Error sending REPLY_ADD_STA: time out after 2000ms.
Mar 17 21:07:48 jeefoo-notebook kernel: [  296.606577] iwlwifi 0000:06:00.0: Current CMD queue read_ptr 246 write_ptr 12
Mar 17 21:07:48 jeefoo-notebook kernel: [  296.606584] ieee80211 phy0: failed to remove key (0, 74:ea:3a:a5:d3:28) from hardware (-110)
Mar 17 21:07:50 jeefoo-notebook kernel: [  298.605823] iwlwifi 0000:06:00.0: Error sending REPLY_REMOVE_STA: time out after 2000ms.
Mar 17 21:07:50 jeefoo-notebook kernel: [  298.605829] iwlwifi 0000:06:00.0: Current CMD queue read_ptr 246 write_ptr 15
Mar 17 21:07:50 jeefoo-notebook kernel: [  298.605835] iwlwifi 0000:06:00.0: Error removing station 74:ea:3a:a5:d3:28
Mar 17 21:07:52 jeefoo-notebook kernel: [  300.621071] iwlwifi 0000:06:00.0: Error sending REPLY_ADD_STA: time out after 2000ms.
Mar 17 21:07:52 jeefoo-notebook kernel: [  300.621078] iwlwifi 0000:06:00.0: Current CMD queue read_ptr 246 write_ptr 18
Mar 17 21:07:52 jeefoo-notebook kernel: [  300.621089] ieee80211 phy0: failed to remove key (1, ff:ff:ff:ff:ff:ff) from hardware (-110)
Mar 17 21:07:52 jeefoo-notebook kernel: [  300.621235] cfg80211: Calling CRDA to update world regulatory domain
Mar 17 21:07:52 jeefoo-notebook kernel: [  301.091795] iwlwifi 0000:06:00.0: No space in command queue
Mar 17 21:07:52 jeefoo-notebook kernel: [  301.091808] iwlwifi 0000:06:00.0: Restarting adapter queue is full
Mar 17 21:07:52 jeefoo-notebook kernel: [  301.091832] iwlwifi 0000:06:00.0: Error sending REPLY_LEDS_CMD: enqueue_hcmd failed: -28
Mar 17 21:07:54 jeefoo-notebook kernel: [  303.093120] iwlwifi 0000:06:00.0: fail to flush all tx fifo queues
Mar 17 21:07:54 jeefoo-notebook kernel: [  303.112038] iwlwifi 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Mar 17 21:07:54 jeefoo-notebook kernel: [  303.130788] iwlwifi 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Mar 17 21:07:54 jeefoo-notebook kernel: [  303.149703] iwlwifi 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Mar 17 21:07:54 jeefoo-notebook kernel: [  303.168498] iwlwifi 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Mar 17 21:07:54 jeefoo-notebook kernel: [  303.187403] iwlwifi 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Mar 17 21:07:54 jeefoo-notebook kernel: [  303.206310] iwlwifi 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Mar 17 21:07:54 jeefoo-notebook kernel: [  303.225211] iwlwifi 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Mar 17 21:07:54 jeefoo-notebook kernel: [  303.244185] iwlwifi 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Mar 17 21:07:54 jeefoo-notebook kernel: [  303.263087] iwlwifi 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Mar 17 21:07:54 jeefoo-notebook kernel: [  303.281982] iwlwifi 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Mar 17 21:07:54 jeefoo-notebook kernel: [  303.301005] iwlwifi 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Mar 17 21:07:54 jeefoo-notebook kernel: [  303.319908] iwlwifi 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Mar 17 21:07:54 jeefoo-notebook kernel: [  303.338821] iwlwifi 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Mar 17 21:07:54 jeefoo-notebook kernel: [  303.357721] iwlwifi 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Mar 17 21:07:54 jeefoo-notebook kernel: [  303.376620] iwlwifi 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Mar 17 21:07:54 jeefoo-notebook kernel: [  303.395524] iwlwifi 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Mar 17 21:07:54 jeefoo-notebook kernel: [  303.414424] iwlwifi 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Mar 17 21:07:54 jeefoo-notebook kernel: [  303.433415] iwlwifi 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Mar 17 21:07:54 jeefoo-notebook kernel: [  303.452382] iwlwifi 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Mar 17 21:07:54 jeefoo-notebook kernel: [  303.471172] iwlwifi 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Mar 17 21:07:54 jeefoo-notebook kernel: [  303.471297] ieee80211 phy0: Hardware restart was requested
Mar 17 21:07:54 jeefoo-notebook kernel: [  303.471349] iwlwifi 0000:06:00.0: L1 Disabled; Enabling L0S
Mar 17 21:07:54 jeefoo-notebook kernel: [  303.471416] cfg80211: World regulatory domain updated:
Mar 17 21:07:54 jeefoo-notebook kernel: [  303.471419] cfg80211:   (start_freq - end_freq @ bandwidth), (max_antenna_gain, max_eirp)
Mar 17 21:07:54 jeefoo-notebook kernel: [  303.471421] cfg80211:   (2402000 KHz - 2472000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
Mar 17 21:07:54 jeefoo-notebook kernel: [  303.471424] cfg80211:   (2457000 KHz - 2482000 KHz @ 20000 KHz), (300 mBi, 2000 mBm)
Mar 17 21:07:54 jeefoo-notebook kernel: [  303.471426] cfg80211:   (2474000 KHz - 2494000 KHz @ 20000 KHz), (300 mBi, 2000 mBm)
Mar 17 21:07:54 jeefoo-notebook kernel: [  303.471428] cfg80211:   (5170000 KHz - 5250000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
Mar 17 21:07:54 jeefoo-notebook kernel: [  303.471430] cfg80211:   (5735000 KHz - 5835000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
Mar 17 21:07:54 jeefoo-notebook kernel: [  303.471442] cfg80211: Calling CRDA for country: HU
Mar 17 21:07:54 jeefoo-notebook kernel: [  303.490154] iwlwifi 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Mar 17 21:07:54 jeefoo-notebook kernel: [  303.508932] iwlwifi 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Mar 17 21:07:54 jeefoo-notebook kernel: [  303.513571] cfg80211: Regulatory domain changed to country: HU
Mar 17 21:07:54 jeefoo-notebook kernel: [  303.513577] cfg80211:   (start_freq - end_freq @ bandwidth), (max_antenna_gain, max_eirp)
Mar 17 21:07:54 jeefoo-notebook kernel: [  303.513583] cfg80211:   (2402000 KHz - 2482000 KHz @ 40000 KHz), (N/A, 2000 mBm)
Mar 17 21:07:54 jeefoo-notebook kernel: [  303.513588] cfg80211:   (5170000 KHz - 5250000 KHz @ 40000 KHz), (N/A, 2000 mBm)
Mar 17 21:07:54 jeefoo-notebook kernel: [  303.513593] cfg80211:   (5250000 KHz - 5330000 KHz @ 40000 KHz), (N/A, 2000 mBm)
Mar 17 21:07:54 jeefoo-notebook kernel: [  303.513597] cfg80211:   (5490000 KHz - 5710000 KHz @ 40000 KHz), (N/A, 2700 mBm)
Mar 17 21:07:54 jeefoo-notebook kernel: [  303.527883] iwlwifi 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Mar 17 21:07:54 jeefoo-notebook kernel: [  303.527883] iwlwifi 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Mar 17 21:07:54 jeefoo-notebook kernel: [  303.527883] iwlwifi 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Mar 17 21:07:54 jeefoo-notebook kernel: [  303.527883] iwlwifi 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Mar 17 21:07:54 jeefoo-notebook kernel: [  303.527883] iwlwifi 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Mar 17 21:07:54 jeefoo-notebook kernel: [  303.527978] iwlwifi 0000:06:00.0: Radio type=0x1-0x2-0x0
Mar 17 21:07:54 jeefoo-notebook kernel: [  303.546756] iwlwifi 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Mar 17 21:08:00 jeefoo-notebook kernel: [  309.203793] iwlwifi 0000:06:00.0: Could not load the INST uCode section
Mar 17 21:08:00 jeefoo-notebook kernel: [  309.203801] iwlwifi 0000:06:00.0: Failed to start RT ucode: -110
Mar 17 21:08:00 jeefoo-notebook kernel: [  309.222571] iwlwifi 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Mar 17 21:08:00 jeefoo-notebook kernel: [  309.241338] iwlwifi 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Mar 17 21:08:00 jeefoo-notebook kernel: [  309.260091] iwlwifi 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Mar 17 21:08:00 jeefoo-notebook kernel: [  309.278897] iwlwifi 0000:06:00.0: MAC is in deep sleep!. CSR_GP_CNTRL = 0xFFFFFFFF
Mar 17 21:08:00 jeefoo-notebook kernel: [  309.579773] iwlwifi 0000:06:00.0: Unable to initialize device.
Mar 17 21:08:00 jeefoo-notebook NetworkManager[896]: <info> (wlan1): roamed from BSSID 74:EA:3A:A5:D3:28 (LAN) to (none) ((none))
Mar 17 21:08:00 jeefoo-notebook NetworkManager[896]: NetworkManager[896]: <info> (wlan1): roamed from BSSID 74:EA:3A:A5:D3:28 (LAN) to (none) ((none))
Mar 17 21:08:00 jeefoo-notebook NetworkManager[896]: <info> (wlan1): supplicant interface state: completed -> disconnected
Mar 17 21:08:00 jeefoo-notebook NetworkManager[896]: NetworkManager[896]: <info> (wlan1): supplicant interface state: completed -> disconnected
Mar 17 21:08:01 jeefoo-notebook kernel: [  310.580742] iwlwifi 0000:06:00.0: Request scan called when driver not ready.
Mar 17 21:08:02 jeefoo-notebook dbus-daemon[963]: ** (upowerd:1466): WARNING **: Property get or set does not have an interface string as first arg
Mar 17 21:08:02 jeefoo-notebook dbus-daemon[963]: ** (upowerd:1466): WARNING **: Property get or set does not have an interface string as first arg
Mar 17 21:08:02 jeefoo-notebook kernel: [  311.581816] iwlwifi 0000:06:00.0: Request scan called when driver not ready.
Mar 17 21:08:03 jeefoo-notebook kernel: [  312.582827] iwlwifi 0000:06:00.0: Request scan called when driver not ready.
Mar 17 21:08:04 jeefoo-notebook kernel: [  313.583790] iwlwifi 0000:06:00.0: Request scan called when driver not ready.
Mar 17 21:08:05 jeefoo-notebook kernel: [  314.584797] iwlwifi 0000:06:00.0: Request scan called when driver not ready.
Mar 17 21:08:11 jeefoo-notebook kernel: [  320.589691] iwlwifi 0000:06:00.0: Request scan called when driver not ready.
Mar 17 21:08:12 jeefoo-notebook kernel: [  321.590710] iwlwifi 0000:06:00.0: Request scan called when driver not ready.
Mar 17 21:08:13 jeefoo-notebook kernel: [  322.591775] iwlwifi 0000:06:00.0: Request scan called when driver not ready.
Mar 17 21:08:14 jeefoo-notebook kernel: [  323.592786] iwlwifi 0000:06:00.0: Request scan called when driver not ready.
Mar 17 21:08:15 jeefoo-notebook NetworkManager[896]: <warn> (wlan1): link timed out.
Mar 17 21:08:15 jeefoo-notebook NetworkManager[896]: <info> (wlan1): device state change: activated -> disconnected (reason 'supplicant-timeout') [100 30 11]
Mar 17 21:08:15 jeefoo-notebook NetworkManager[896]: NetworkManager[896]: <warn> (wlan1): link timed out.
Mar 17 21:08:15 jeefoo-notebook NetworkManager[896]: NetworkManager[896]: <info> (wlan1): device state change: activated -> disconnected (reason 'supplicant-timeout') [100 30 11]
Mar 17 21:08:15 jeefoo-notebook NetworkManager[896]: <info> (wlan1): deactivating device (reason 'supplicant-timeout') [11]
Mar 17 21:08:15 jeefoo-notebook NetworkManager[896]: NetworkManager[896]: <info> (wlan1): deactivating device (reason 'supplicant-timeout') [11]
Mar 17 21:08:15 jeefoo-notebook NetworkManager[896]: <info> (wlan1): canceled DHCP transaction, DHCP client pid 1239
Mar 17 21:08:15 jeefoo-notebook NetworkManager[896]: NetworkManager[896]: <info> (wlan1): canceled DHCP transaction, DHCP client pid 1239
Mar 17 21:08:15 jeefoo-notebook avahi-daemon[928]: Withdrawing address record for 10.1.1.22 on wlan1.
Mar 17 21:08:15 jeefoo-notebook avahi-daemon[928]: Leaving mDNS multicast group on interface wlan1.IPv4 with address 10.1.1.22.
Mar 17 21:08:15 jeefoo-notebook avahi-daemon[928]: Interface wlan1.IPv4 no longer relevant for mDNS.
Mar 17 21:08:16 jeefoo-notebook dbus-daemon[963]: dbus[963]: [system] Activating service name='org.freedesktop.nm_dispatcher' (using servicehelper)
Mar 17 21:08:16 jeefoo-notebook dbus[963]: [system] Activating service name='org.freedesktop.nm_dispatcher' (using servicehelper)
Mar 17 21:08:16 jeefoo-notebook NetworkManager[896]: <info> (wlan1): supplicant interface state: disconnected -> inactive
Mar 17 21:08:16 jeefoo-notebook NetworkManager[896]: NetworkManager[896]: <info> (wlan1): supplicant interface state: disconnected -> inactive
Mar 17 21:08:16 jeefoo-notebook NetworkManager[896]: <info> Auto-activating connection 'Auto LAN'.
Mar 17 21:08:16 jeefoo-notebook NetworkManager[896]: NetworkManager[896]: <info> Auto-activating connection 'Auto LAN'.
Mar 17 21:08:16 jeefoo-notebook NetworkManager[896]: <info> Activation (wlan1) starting connection 'Auto LAN'
Mar 17 21:08:16 jeefoo-notebook NetworkManager[896]: <info> (wlan1): device state change: disconnected -> prepare (reason 'none') [30 40 0]
Mar 17 21:08:16 jeefoo-notebook NetworkManager[896]: NetworkManager[896]: <info> Activation (wlan1) starting connection 'Auto LAN'
Mar 17 21:08:16 jeefoo-notebook NetworkManager[896]: NetworkManager[896]: <info> (wlan1): device state change: disconnected -> prepare (reason 'none') [30 40 0]
Mar 17 21:08:16 jeefoo-notebook NetworkManager[896]: <info> Activation (wlan1) Stage 1 of 5 (Device Prepare) scheduled...
Mar 17 21:08:16 jeefoo-notebook NetworkManager[896]: NetworkManager[896]: <info> Activation (wlan1) Stage 1 of 5 (Device Prepare) scheduled...
Mar 17 21:08:16 jeefoo-notebook NetworkManager[896]: <info> Activation (wlan1) Stage 1 of 5 (Device Prepare) started...
Mar 17 21:08:16 jeefoo-notebook NetworkManager[896]: NetworkManager[896]: <info> Activation (wlan1) Stage 1 of 5 (Device Prepare) started...
Mar 17 21:08:16 jeefoo-notebook NetworkManager[896]: <info> Activation (wlan1) Stage 2 of 5 (Device Configure) scheduled...
Mar 17 21:08:16 jeefoo-notebook NetworkManager[896]: NetworkManager[896]: <info> Activation (wlan1) Stage 2 of 5 (Device Configure) scheduled...

This is regression. Find two consecutive kernel version where things work and problem start to happen i.e 2.6.37 and 2.6.38 (you will need to compile kernel from kernel.org). Then attach logs from working kernel and broken kernel as described here:
https://bugzilla.redhat.com/show_bug.cgi?id=733269#c3
Logs on broken kernel should contains at least first "time out" error, logs on working kernel should be captured the same duration, hence I would be able to compere them.

I can confirm that I as well has again started to see this problem with kernel-3.3.0-4.fc16 
I did not see the problem with any kernel between 3.2.7-1.fc16 and 3.2.10-3,fc16

the error-messages I'm getting 
Mar 30 10:04:55 localhost kernel: [166355.607522] iwlwifi 0000:03:00.0: U iwl_send_cmd_sync Error sending REPLY_ADD_STA: time out after 2000ms.
Mar 30 10:04:55 localhost kernel: [166355.607530] iwlwifi 0000:03:00.0: U iwl_send_cmd_sync Current CMD queue read_ptr 234 write_ptr 235
Mar 30 10:04:55 localhost kernel: [166355.607536] iwlwifi 0000:03:00.0: Adding station ff:ff:ff:ff:ff:ff failed.
Mar 30 10:04:55 localhost kernel: [166355.611198] iwlwifi 0000:03:00.0: ACTIVATE a non DRIVER active station id 15 addr ff:ff:ff:ff:ff:ff
Mar 30 10:04:55 localhost kernel: [166355.611214] iwlwifi 0000:03:00.0: HCMD_ACTIVE already clear for command REPLY_REMOVE_STA

Regarding the debug flag 0x47ffffff for the iwlwifi module: For me this problem (usually) occurs after a day or so of work and there is _quite_ a lot of debug info generated. I'll try to keep it running today though and hope my disk can keep up with it.

David, since your iwlwifi device problems started to happen with 3.1, and issue originally reported here happen with kernel 2.6.38, they must be a different issues. 

It's not good to mix different issues in the same bug report, please open a separate bug for your case.

Can we get logs requested in comment 59 ?

Without logs requested above we can not fix this bug.