Bug 158725

Another machine with only one 3c59x works:
2.6.11-1.27_FC3:
3c59x: Donald Becker and others. www.scyld.com/network/vortex.html
0000:02:08.0: 3Com PCI 3c905C Tornado at 0x3000. Vers LK1.1.19

2.6.11-1.27_FC3 #1 Tue May 17 20:27:37 EDT 2005 i686 athlon i386 GNU/Linux

The machine which had problems is i586 if that make some difference. Main
difference is it's with two identical cards (3c905B) and this only one 3c509C
seems to work..

I think this has been fixed upstream.  I've included that patch in the test 
kernels here: 
 
   http://people.redhat.com/linville/kernels/fc3/ 
 
Would you mind giving them a test just to be sure?  Thanks! 

Sorry, cannot test those. Machine which I had troubles is i586 machine, with no
development enviroment (gateway device). And no space for installing gcc and
etc. But if those are coming to next kernel release then i can report what
happened and if it works we can close this bug.

Jani - the linked page contains finished RPMs not just source RPMs. You wouldn't
need to build them - just install them.

Well those are i686 optimized and my machine in question is i586. And If I'm
right those kernels won't run properly in my machine..

[root@gw ~]# rpm -ivh kernel-2.6.11-1.29_FC3.jwltest.11.i686.rpm
Preparing...                ########################################### [100%]
        package kernel-2.6.11-1.29_FC3.jwltest.11 is intended for a i686
architecture

Ok, Won't even want to install it without force, so... And a little bit more
info again:

2.6.11-1.27_FC3 #1 Tue May 17 20:27:37 EDT 2005 i686 athlon i386 GNU/Linux:
3c59x: Donald Becker and others. www.scyld.com/network/vortex.html
0000:00:0d.0: 3Com PCI 3c905C Tornado at 0xe400. Vers LK1.1.19
0000:00:0f.0: 3Com PCI 3c905C Tornado at 0xe800. Vers LK1.1.19

Works, so it seems to do more with 3c905B Cyclone cards.

I have added i586 kernels to the location from comment 3.  Please give them a 
try and post the results.  Thanks! 

2.6.11-1.29_FC3.jwltest.11 #1 Fri May 27 09:29:05 EDT 2005 i586 i586 i386 GNU/Linux:

PCI: Enabling device 0000:00:06.0 (0000 -> 0003)
3c59x: Donald Becker and others. www.scyld.com/network/vortex.html
0000:00:06.0: 3Com PCI 3c905B Cyclone 100baseTx at 0x8000. Vers LK1.1.19
PCI: Setting latency timer of device 0000:00:06.0 to 64
*** EEPROM MAC address is invalid.
3c59x: vortex_probe1 fails.  Returns -22
3c59x: probe of 0000:00:06.0 failed with error -22
PCI: Enabling device 0000:00:07.0 (0000 -> 0003)
0000:00:07.0: 3Com PCI 3c905B Cyclone 100baseTx at 0x8080. Vers LK1.1.19
PCI: Setting latency timer of device 0000:00:07.0 to 64
*** EEPROM MAC address is invalid.
3c59x: vortex_probe1 fails.  Returns -22
3c59x: probe of 0000:00:07.0 failed with error -22

Oh, it seems that the problem lies somewhere else.. :(

I have the same problem, but with a different chip: 3c556B.  (Built into my
ThinkPad T21c)

Here is my interesting datapoint.  If ACPI is off (acpi=off in kernel
commandline) the driver loads fine.  If it boots normally and loads with acpi
active, I get the above error.

Tested with acpi=off and didn't work. I think that my gw machine don't even have
acpi capabilities. So that workaround doesn't work with my machine in question.

Same problem here. Thinkpad T21 with the 3Com Ethernet+winmodem mini-PCI card,
running an up to date Fedora Core 3 (kernel 2.6.11-1.27_FC3). I'm getting the
"EEPROM MAC address is invalid" error and no network. 

With "acpi=off" the network is OK, but advanced power management is no longer
available.

I also tried kernel kernel-2.6.11-1.31_FC3.jwltest.12 -- no change, the problem
is still there.

The problems that go away w/ "acpi=off" generally indicate a BIOS problem.  I 
don't have a lot to offer there, other than to suggest trying to find a BIOS 
update...sorry! 
 
Unfortunately, the "EEPROM MAC addres is invalid" message actually doesn't do 
much to narrow down the exact problem.  I would like for you to add a line 
to /etc/modules.conf: 
 
   options 3c59x debug=7 
 
Then, execute these commands: 
 
   # ifdown eth0 
   # modprobe -r 3c59x 
   # dmesg -c 
   # ifup eth0 
   # dmesg 
 
Finally, attach the output of that last "dmesg" command to this bug.  
Hopefully that will be informative...? :-) Thanks! 

Alas, this is the latest BIOS from IBM...

With acpi=on, the debug=7 printk output from "modprobe 3c59x" is as follows:

ACPI: PCI interrupt 0000:00:03.0[A] -> GSI 5 (level, low) -> IRQ 5
3c59x: Donald Becker and others. www.scyld.com/network/vortex.html
See Documentation/networking/vortex.txt
0000:00:03.0: 3Com PCI 3c556B Laptop Hurricane at 0x1800. Vers LK1.1.19
 ff:ff:ff:ff:ff:ff<3>*** EEPROM MAC address is invalid.
3c59x: vortex_probe1 fails.  Returns -22
3c59x: probe of 0000:00:03.0 failed with error -22

With acpi=off, it's as follows:

PCI: Found IRQ 5 for device 0000:00:03.0
PCI: Sharing IRQ 5 with 0000:00:03.1
3c59x: Donald Becker and others. www.scyld.com/network/vortex.html
See Documentation/networking/vortex.txt
0000:00:03.0: 3Com PCI 3c556B Laptop Hurricane at 0x1800. Vers LK1.1.19
 00:00:86:4b:41:3b, IRQ 5
  product code 0000 rev aa.8 date 03-01-00
0000:00:03.0: CardBus functions mapped e8101000->d8968000
  Internal config register is 80600040, transceivers 0x40.
  8K byte-wide RAM 5:3 Rx:Tx split, MII interface.
  MII transceiver found at address 0, status 786d.
  Enabling bus-master transmits and whole-frame receives.
0000:00:03.0: scatter/gather enabled. h/w checksums enabled

As before, this is a ThinkPad T21 running 2.6.11-1.27_FC3.

The problem persists with "acpi=noirq acpi_irq_nobalance pci=noacpi acpi_osi=
pnpacpi=off".

Is there any other ACPI functionality that can be disabled while keeping the
basic ACPI power management functions (e.g., plain "poweroff")?

Well I'd like to point out that my machine don't have ACPI (or i think so), so
that cannot be the reason for problems. acpi=off doesn't workaround with me.

With normal boot:
ACPI: Unable to locate RSDP
ACPI: Subsystem revision 20050211
ACPI: Interpreter disabled.
pnp: PnP ACPI: disabled

Debug information:
3c59x: Donald Becker and others. www.scyld.com/network/vortex.html
See Documentation/networking/vortex.txt
0000:00:06.0: 3Com PCI 3c905B Cyclone 100baseTx at 0x8000. Vers LK1.1.19
 ff:ff:ff:ff:ff:ff<3>*** EEPROM MAC address is invalid.
3c59x: vortex_probe1 fails.  Returns -22
3c59x: probe of 0000:00:06.0 failed with error -22
See Documentation/networking/vortex.txt
0000:00:07.0: 3Com PCI 3c905B Cyclone 100baseTx at 0x8080. Vers LK1.1.19
 ff:ff:ff:ff:ff:ff<3>*** EEPROM MAC address is invalid.
3c59x: vortex_probe1 fails.  Returns -22
3c59x: probe of 0000:00:07.0 failed with error -22

.. Not so helpfull, but consistent with previous debug messages..
And with that old working kernel:

3c59x: Donald Becker and others. www.scyld.com/network/vortex.html
See Documentation/networking/vortex.txt
0000:00:06.0: 3Com PCI 3c905B Cyclone 100baseTx at 0x8000. Vers LK1.1.19
 00:a0:24:a6:41:31, IRQ 15
  product code 4e4b rev 00.9 date 04-29-98
Full duplex capable
  Internal config register is 1000000, transceivers 0xa.
  8K byte-wide RAM 5:3 Rx:Tx split, autoselect/10baseT interface.
  MII transceiver found at address 24, status 784d.
  Enabling bus-master transmits and whole-frame receives.
0000:00:06.0: scatter/gather enabled. h/w checksums enabled
See Documentation/networking/vortex.txt
0000:00:07.0: 3Com PCI 3c905B Cyclone 100baseTx at 0x8080. Vers LK1.1.19
 00:a0:24:a6:3d:81, IRQ 11
  product code 4e4b rev 00.9 date 04-29-98
Full duplex capable
  Internal config register is 1000000, transceivers 0xa.
  8K byte-wide RAM 5:3 Rx:Tx split, autoselect/10baseT interface.
  MII transceiver found at address 24, status 784d.
  Enabling bus-master transmits and whole-frame receives.
0000:00:07.0: scatter/gather enabled. h/w checksums enabled

redhat-bugs2eran, did you try the test kernels from comment 3?  
They are the only thing likely to help w/ the "acpi=off" issue.  Please let me 
know the results of testing with those...thanks! 

Jani,  
  
FWIW, there is _no_ difference in the 3c59x driver between  
kernel-2.6.11-1.14_FC3 and kernel-2.6.11-1.27_FC3...I'm at a loss to explain 
what you are seeing... 

Yes, I get the same behavior and debug output with
kernel-2.6.11-1.31_FC3.jwltest.12 (as mentioned in comment 11).

I confirm to have the same problem with FC4 ( IBM Thinkpad A21p).

I am using the acpi=off switch since FC2 was released, it used to work until I
installed FC4. I did not test 2.6.11-1.27_FC3, but 2.6.11-1.14 was fine. At the
moment it does not matter which ACPI setting I pass zo the kernel.

For the record, I get the same effect (3c59x works only with acpi=off on
ThinkPad T21) also with FC4 (kernel 2.6.11-1.1369.FC4) and with vanilla 2.6.11.11.

Jani, were you running kernel-2.6.11-1.27_FC3 and kernel-2.6.11-1.14_FC3 with
identical kernel parameters?
Looking at te SPEC files, there seem to be few differences between those two
kernels. It should be possible to experimentally perform a binary search on the
set of patches added and removed between these kernels, to find the patch
affecting 3c59x. That would probably give us a big hint about the source of the
problem. Linville, is it worth the effort? Jani, I can't do it myself since I
don't get the kernel-dependent behavior you got.

Meanwhile, I added a few printk()s to the driver, and I see that with acpi=on
it's getting 0xFFFF on all inw() io reads from the EEPROM; the MAC address is
just part of that. Also, calling pci_set_power_state(pdev,PCI_D0) early in
vortex_probe1() has no effect, and neither does increasing the udelay() in the
EEPROM read loop of vortex_probe1().

Here's the kernel.org Bugzilla bug about this:
  http://bugzilla.kernel.org/show_bug.cgi?id=1188
No solution there, but a few more data points.

BTW, the problem remains in vanilla kernel 2.6.12.

Yes. Parameters where the same. "ro root=LABEL=/ vga=1"
And that patches testing which one does the breaking sounds that it is too much
work.

Some more mind boggling things. I've identical machine (or i think so that they
are identical) tested it after FC4 upgrade. The kernel detects both network
adapters and it's ok. From that I got the idea to try FC4 kernel on that not
working machine (otherwise FC3 packages) and it didn't work. Uh. And still that
other machine works with that old kernel so it can't be hardware fault (entirely).

Someone (Eran Tromer?) for whom "acpi=off" improves this problem please 
execute the following command as root: 
 
   cat /proc/acpi/dsdt > dsdt.dat 
 
Then, attach dsdt.dat to this bug...no promises... :-)  Please also state the 
make/model of the laptop in use.  Thanks! 

Created attachment 115687 [details]
DSDT on ThinkPad T21

Content of /proc/acpi/dsdt on IBM ThinkPad T21 (model 2647-5BG) with 3Com
3c556B+winmodem mini-PCI card.

Created attachment 115717 [details]
Corrected(?) DSDT for ThinkPad T21

I think/hope this will work better...at least it compiles w/o warnings on the
Intel ASL compiler...

Created attachment 115718 [details]
mkinitrd

mkinitrd w/ support for DSDT override

Ok, Eran, are you ready to try this? :-)   
   
Start by downloading the attachment from comment 26.  I'll presume that you   
save it as /etc/DSDT.aml.  
  
Next download the attachment from comment 27 and save it as /sbin/mkinitrd.   
You probably want to save a backup of your original /sbin/mkinitrd just in 
case... 
 
Now, edit /etc/sysconfig/kernel and add a line like this: 
 
   ACPI_DSDT=/etc/DSDT.aml 
 
Customize it to match where you saved the attachment from comment 26, of 
course. 
 
It is very important that you complete _all_ of the steps above before 
proceeding to the next step.  Please make sure you have done that, or the test 
will be pointless.  This means you... :-) 
 
Finally, download and install an appropriate kernel from here: 
 
   http://people.redhat.com/linville/kernels/fc3/ 
 
The installation process will only create the initrd correctly if you have 
already completed the previous steps.  If you didn't believe me before, start 
again now... :-) 
 
Once that kernel is installed, please reboot and test your NIC.  Please post 
the results here.  Thanks! 
 
P.S.  If it doesn't work, please attach your initrd image so that I can make 
sure it got created properly... 

This laptop is already upgraded to FC4, so instead of using your RPMs I applied
your jwltest-acpi-dsdt-initrd.patch to what I had at hand, namely vanilla
2.6.12+swsusp2. Also, your patched mkinitrd 4.1.18 doesn't generate a usable
initrd for the vanilla kernel, so I merged the relevant changes into my current
mkinitrd (4.2.15). After generating the initrd image and rebooting, low and
behold: my /proc/acpi/dsdt is now identical to that in comment 26.

But 3c59x still gets only 0xFFFF's when reading the EEPROM...

BTW, I didn't see any mention of the 3Com card in the disassembled (iasl -d)
DSDT. What did you change that should have affected it?

Here's progress and a workaround.

Right after boot with acpi=on (and initrd=/bin/bash, just in case), vortex-diag
shows that all registers of the 3c556B (not just the EEPROM) contain 0xFF, which
looks very much like a powerdown issue. And indeed, at this point pci-config
says that this device is in D3 sleep! After running "pci-config -W -#5" to wake
it up into D0, the registers shown by vortex-diag look fine, and the 3c59x
module loads cleanly.


Hence, the following kludgy workaround:

1. Install pci-config (e.g., from the net-tools RPM).
2. Run /sbin/lspci and identify the PCI device of the 3Com chip (e.g., "00:03.0").
3. Run /usr/sbin/pci-config and identify the corresponding device# (e.g., #5).
4. Append the following to /etc/modprobe.conf as one line, using the dabove device#:
install 3c59x /usr/sbin/pci-config -W -#5 > /dev/null && /sbin/modprobe
--ignore-install 3c59x
5. Reboot (or "rmmod 3c59x && modprobe 3c59x").


Remaining questions:
Why does the NIC come up in D3 state in some kernel configurations?
Why does /sys/bus/pci/devices/0000:00:03.0/power/state lie about it, by saying
"0" at a point where pci-config says "D3" about the same device?
Is it a BIOS issue or a card issue, and should the 3c59x driver pull the card to D0?
I actually tried doing that back in comment 21, by calling
pci_set_power_state(pdev,PCI_D0) at the beginning of vortex_probe1(). That
didn't work; why?

re: comment 29...the idea was to clean-up the compiler warnings from your  
original DSDT...unfortunately it sounds like that was not enough to clear-up  
the problem...at least it sounds like the kernel patch works, so that might be  
handy elsewhere... :-)  
  
re: comment 30...where did you get pci-config?  I don't see it in any version  
of net-tools that I have readily available.  I'd like to see what it is doing.   
The kernel driver should be putting the 3c59x card into D0 (indirectly, by  
calling pci_enable_device in vortex_init_one) already, so I'm not sure why it  
doesn't seem to "stick".  
  
The fact that "acpi=noirq" helps the issue definitely indicates a BIOS issue.   
The kernel/driver may be able to work around it, but the BIOS definitely seems  
to have some sort of problem.  

Could there be a setting in your BIOS configuration that is effecting the 
initial power state of your 3c556? 
 
It looks like to me that the card is not starting-out in D0.  The code makes 
calls to set the power state to D0, but they will not do anything if the state 
is already believed to be D0.  It looks like the assumption is that the card 
will be in D0 after boot. 

I can't find any related BIOS setting. The only thing remotely related is some
PCI power saving option, which doesn't affect this.

Is it sensible to assume everything is in D0 after boot?

BTW, I now see there's another workaround here:
http://www.thinkwiki.org/wiki/Problem_with_3Com_10/100_Ethernet_card_not_being_recognized
Haven't tested it.

re: comment 31
My mistake, pci-config is from Fedora Extra's netdiag package.
But "acpi=noirq" does *not* make any difference for me.

Should have said "acpi=off" in comment 31... 

OK, then maybe the card boots into D3 and the BIOS switches it to D0 when using
APM (because the OS can't do it itself) but not in ACPI (because then the driver
can do it)?

Created attachment 115766 [details]
jwltest-3c59x-hack.patch

Patch that prints some PM-related info when probing the 3c59x device and tries
to put it into D0 state if there seems to be confusion about the actual state.

Eran, please apply the patch from comment 37 to the kernel you are using. 
 
Everyone still using FC3, please try the kernels at the location here: 
 
   http://people.redhat.com/linville/kernels/fc3/ 
 
Everyone please attach the 3c59x-related output from dmesg 
and/or /var/log/messages after booting w/ the kernels above.  Also, please let 
me know if the above patches make the devices work for you...thanks! 

While waiting for the kernel compile, another recent data point:
After one reboot [1], the card came up in a state where sending it to D0 using
pci-config did *not* fix it (i.e., all registers still read 0xFFFF). Neither did
sending it to D3 and then back to D0. However, at this point the ThinkWiki
script linked above *did* work [2].

This seems to suggest that the card is not only coming up not just in the wrong
PM state, but in an arbitrarily silly state. So even if your current patch
helps, it might not solve the problem completely.

[1] First boot after I let the batteries run out completely with auto-suspend
disable (to reset the battery's concept of its own capacity). Probably
unrelated, but just in case.

[2] But note that the ThinkWiki script assumes that the io port number is
lspci's output is decimal and converts it into hex, whereas in my lspci it's
decimal in the first place! Better fix that before running.

The patch doesn't work. It's indeed switching the NIC to D0, but this causes the
NIC to forget its IO port. Here's what happens:

==================================
(booted patched kernel with init=/bin/bash)

# /usr/sbin/pci-config -#5
...
Device #5 at bus 0 device/function 3/0.
  605610b7 02100013 02000020 00805008 00001801 e8101400 e8101000 00000000
...
  Base Address 0: I/O at 00001800.
  Base Address 1: Memory at e8101400.
  Base Address 2: Memory at e8101000.
  Extended capabilities, first structure at offset 0x50.
  Extended PCI capability type 1 at 0x50, next 0.
   Power management entry ver. 2: Capabilities fe02, Ctrl 4103, Event 2900.
   Power state D3.

# /sbin/modprobe 3c59x ; dmesg
...
3c59x: pdev->current_state 4
3c59x: pmcsr state 3
PCI: Enabling device 0000:00:03.0 (0000 -> 0003)
ACPI: PCI Interrupt Link [LNKC] enabled at IRQ 7
PCI: setting IRQ 7 as level-triggered
ACPI: PCI Interrupt 0000:00:03.0[A] -> Link [LNKC] -> GSI 7 (level, low) -> IRQ 7
3c59x: Donald Becker and others. www.scyld.com/network/vortex.html
0000:00:03.0: 3Com PCI 3c556B Laptop Hurricane at 0x1400. Vers LK1.1.19
PCI: Setting latency timer of device 0000:00:03.0 to 64
*** EEPROM MAC address is invalid.
3c59x: vortex_probe1 fails.  Returns -22
[42949599.310000] 3c59x: probe of 0000:00:03.0 failed with error -22

# /usr/sbin/pci-config -#5
...
Device #5 at bus 0 device/function 3/0.
  605610b7 02100003 02000020 00804000 00000001 00000000 00000000 00000000
...
  Base Address 0: I/O at 00000000.
  Extended capabilities, first structure at offset 0x50.
  Extended PCI capability type 1 at 0x50, next 0.
   Power management entry ver. 2: Capabilities fe02, Ctrl 4100, Event 2900.
   Power state D0.
[==================================

So it has switched to D0, but lost the device registers. If you now manually set
the IO port address it will work (but is still missing some state, such as the
memory addresses):

==================================
(continue above)

# /sbin/setpci -H1 -d 10b7:6056 BASE_ADDRESS_0=0x1800

# /sbin/rmmod 3c59x; /sbin/modprobe 3c59x ; dmesg

3c59x: pdev->current_state 0
3c59x: pmcsr state 0
ACPI: PCI Interrupt 0000:00:03.0[A] -> Link [LNKC] -> GSI 7 (level, low) -> IRQ 7
3c59x: Donald Becker and others. www.scyld.com/network/vortex.html
0000:00:03.0: 3Com PCI 3c556B Laptop Hurricane at 0x1800. Vers LK1.1.19
==================================

By contrast, my workaround (if executed before anything else touches the card)
preserves those registers:

==================================
(booted patched kernel with init=/bin/bash)

# /usr/sbin/pci-config -#5
...
Device #5 at bus 0 device/function 3/0.
  605610b7 02100000 02000020 00800000 00001401 18000000 18000080 00000000
...
  Base Address 0: I/O at 00001400.
  Base Address 1: Memory at 18000000.
  Base Address 2: Memory at 18000080.
  Extended capabilities, first structure at offset 0x50.
  Extended PCI capability type 1 at 0x50, next 0.
   Power management entry ver. 2: Capabilities fe02, Ctrl 4103, Event 2900.
   Power state D3.

# /usr/sbin/pci-config -W -#5

# /usr/sbin/pci-config -#5
...
Device #5 at bus 0 device/function 3/0.
  605610b7 02100000 02000020 00800000 00001401 18000000 18000080 00000000
...
  Base Address 0: I/O at 00001400.
  Base Address 1: Memory at 18000000.
  Base Address 2: Memory at 18000080.
  Extended capabilities, first structure at offset 0x50.
  Extended PCI capability type 1 at 0x50, next 0.
   Power management entry ver. 2: Capabilities fe02, Ctrl 4100, Event 2900.
   Power state D0.

# /sbin/modprobe 3c59x ; dmesg

3c59x: pdev->current_state 4
3c59x: pmcsr state 3
PCI: Enabling device 0000:00:03.0 (0000 -> 0003)
PCI: setting IRQ 7 as level-triggered
ACPI: PCI Interrupt 0000:00:03.0[A] -> Link [LNKC] -> GSI 7 (level, low) -> IRQ 7
3c59x: Donald Becker and others. www.scyld.com/network/vortex.html
0000:00:03.0: 3Com PCI 3c556B Laptop Hurricane at 0x1800. Vers LK1.1.19
PCI: Setting latency timer of device 0000:00:03.0 to 64

(module works)
==================================

The catch is that you have to run pci-config -W before anything switches the
device to D0, which is pretty tough with the Fedora init process. The ThinkWiki
script, by contrast, implicitly *relies* on something else (what?) switching the
device to D0 during the boot process, and then (partially) recovers from the
loss of device registers values.

The patch, then, should switch to D0 and then set the registers -- either to
what it thinks their value should be, or from a copy it made before switching to
D0. For the latter, you may want to have a look at the acpi_wake() function of
pci-config (ftp://ftp.scyld.com/pub/diag/pci-config.c).

At normal boot:
3c59x: pdev->current_state 4
3c59x: pmcsr state 3
PCI: Enabling device 0000:00:06.0 (0000 -> 0003)
3c59x: Donald Becker and others. www.scyld.com/network/vortex.html
0000:00:06.0: 3Com PCI 3c905B Cyclone 100baseTx at 0x8000. Vers LK1.1.19
PCI: Setting latency timer of device 0000:00:06.0 to 64
*** EEPROM MAC address is invalid.
3c59x: vortex_probe1 fails.  Returns -22
3c59x: probe of 0000:00:06.0 failed with error -22
3c59x: pdev->current_state 4
3c59x: pmcsr state 3
PCI: Enabling device 0000:00:07.0 (0000 -> 0003)
0000:00:07.0: 3Com PCI 3c905B Cyclone 100baseTx at 0x8080. Vers LK1.1.19
PCI: Setting latency timer of device 0000:00:07.0 to 64
*** EEPROM MAC address is invalid.
3c59x: vortex_probe1 fails.  Returns -22
3c59x: probe of 0000:00:07.0 failed with error -22

After boot:
[root@gw ~]# pci-config -W -#3
pci-config.c:v2.03 4/15/2002 Donald Becker (becker)
 http://www.scyld.com/diag/index.html
Device #3 at bus 0 device/function 7/0.
  905510b7 02100003 02000024 00004000 00000001 00000000 00000000 00000000
  00000000 00000000 00000000 905510b7 00000000 000000dc 00000000 0a0a0100
  00000000 00000000 00000000 00000000 00000000 00000040 00000000 00000000
  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
  00000000 00000000 00000000 00000000 00000000 00000000 00000000 76010001
  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
  Base Address 0: I/O at 00000000.
  Extended capabilities, first structure at offset 0xdc.
  Extended PCI capability type 1 at 0xdc, next 0.
   Power management entry ver. 1: Capabilities 7601, Ctrl 0000, Event 0000.
   Power state D0.
  Extended capabilities, first structure at offset 0xdc.
  Waking up an ACPI device.  Currently powered up, I/O 0x1 IRQ 0.
  Updating the power state of 0000->0000.

=> Where are Base addresses? Something in the boot messes them up?

After boot with init=/bin/sh and /sbin/modprobe 3c59x:
3c59x: pdev->current_state 4
3c59x: pmcsr state 0
3c59x: Donald Becker and others. www.scyld.com/network/vortex.html
0000:00:06.0: 3Com PCI 3c905B Cyclone 100baseTx at 0x8000. Vers LK1.1.19
3c59x: pdev->current_state 4
3c59x: pmcsr state 0
0000:00:07.0: 3Com PCI 3c905B Cyclone 100baseTx at 0x8080. Vers LK1.1.19

Works! So something in the normal boot messes it up:)

More testing with normal failure boot.. i did set the base addesses
again with setpci. Same ones for the both (that totally crashed)
the machine (looked the normal base addresses from the working kernel)
and now after that crash i got both of them working at the boot.
Power states where:
3c59x: pdev->current_state 4
3c59x: pmcsr state 0

rmmod and modprobe agein, states:
3c59x: pdev->current_state 0
3c59x: pmcsr state 0

Still works.. With this working boot there are now lines like:
PCI: Enabling device 0000:00:06.0 (0000 -> 0003)
PCI: Enabling device 0000:00:07.0 (0000 -> 0003)

Ok. I rebooted the machine couple of time, even with power cord off
and it still works and detects the cards right. Before the boot
with that new kernel machine has been power off couple of days so
i cannot figure out why this happened and what fixed it, and why
the older kernel worked all the time.... Weird uh? :)

Created attachment 115811 [details]
jwltest-3c59x-hack.patch

Eran, please use the above patch instead of the previous one, and repeat your 
tests. 
 
Jani, I've got pre-built FC3 kernels at the same location as in comment 38.  
Please use them to repeat your test as well. 
 
This version should save and restore the config state.  Please post the 
results...thanks! 

The patch in comment 42 works perfectly for me: the module loads smoothly, and
the pci-config outputs look like they should (all configuration preserved).

The dmesg output is
  3c59x: pdev->current state 4
  3c59x: pmcsr state 3
on first module load, and both 0 if it's rmmodded and loaded again.

John, I'll test it next week, i'm not at work before that (and the machine is
there). Mon/Tue depending on the hangover. We have midsommer festival coming and
I'm starting it tomorrow! :)

I'm working on a more permanent patch...still deciding whether this should be  
handled by the 3c59x driver or if a patch to the PCI subsystem would be more  
appropriate...  

Created attachment 115900 [details]
jwltest-pci-d3hot-boot.patch

A more systemic approach...

I have FC3 test kernels w/ the patch from comment 47 available here: 
 
   http://people.redhat.com/linville/kernels/fc3/ 
 
Please give those (or your own w/ the patch) a try and let me know the 
results.  Thanks! 

The systematic patch of comment 47 works perfectly here (PCI PM trivia indeed...).
Maybe add a kernel cmdline parameter for disabling this new behavior, just in
case it breaks something?

I'm working on an upstream solution, based mostly on the last patch.  I'll 
keep this open until that gets somewhere...thanks for the help! 

Created attachment 116035 [details]
pci-enable-d3hot.patch

This is what I plan to champion upstream.  I would appreciate some testing! 
Pre-built FC3/4 test kernels coming soon...

FC3 test kernels here: 
 
   http://people.redhat.com/linville/kernels/fc3/ 
 
FC4 test kernels here: 
 
   http://people.redhat.com/linville/kernels/fc4/ 
 
Please give those a try and let me know the results...thanks! 

The new patch from comment 51 works for normal boot, but breaks resume from
software suspend. That's because in 3c59x.c,
  vortex_resume() calls 
  vortex_up() which calls 
  pci_set_power_state(..., PCI_D0)
which doesn't really switch from D3 to D0 because the kernel has a wrong mental
image of the device's state. The previous patch from comment 47 did handle this.

Tested with 2.6.12.1+suspend2.

Not really sure I understand "the kernel has a wrong mental image of the   
device's state"?  The code in the patch from comment 47 really should only be   
invoked the first time the driver loads anyway.  So for ->resume, the call to   
pci_set_power_state shouldn't be any different with either patch.  Am I   
misreading something?  Is it possible that neither patch resumes properly?   
   
Besides that, vortex_up later calls pci_enable_device, which will also call   
pci_set_power_state (through pci_enable_device_bars), after which it will hit   
the code to restore the BARs anyway.  
  
I'm prepared to believe that 3c59x is still broken in some way w/ the patch  
from comment 51, but I'm thinking that those are different problems?  Could 
you attach the results of "lspci -nv" from both before and after a failed 
resume?  Thanks! 

About "mental image", I meant 
  pdev->current_state 
vs.
  pci_read_config_word(pdev, pm + PCI_PM_CTRL, &pmcsr) & PCI_PM_CTRL_STATE_MASK
.

Anyway, sorry, my mistake. Resume worked with the older patches, but *neither*
if these two patches. After resume the device now has correct registers, but is
left at D3. Removing and reloading the module does not change this.


Output of pci -nv immediately after boot and successful loading:
00:03.1 Communication controller: 3Com Corporation Mini PCI 56k Winmodem (rev 20)
        Subsystem: 3Com Corporation: Unknown device 6159
        Flags: medium devsel, IRQ 7
        I/O ports at 2000 [size=256]
        Memory at e8101c00 (32-bit, non-prefetchable) [size=256]
        Memory at e8101800 (32-bit, non-prefetchable) [size=128]
        Capabilities: [50] Power Management version 2

After resume:
00:05.0 Multimedia audio controller: Cirrus Logic CS 4614/22/24 [CrystalClear
SoundFusion Audio Accelerator] (rev 01)
        Subsystem: IBM: Unknown device 0153
        Flags: bus master, slow devsel, latency 64, IRQ 4
        Memory at e8100000 (32-bit, non-prefetchable) [size=4K]
        Memory at e8000000 (32-bit, non-prefetchable) [size=1M]
        Capabilities: [40] Power Management version 2

After rmmod 3c59x && modprobe 3c59x:
00:05.0 Multimedia audio controller: Cirrus Logic CS 4614/22/24 [CrystalClear
SoundFusion Audio Accelerator] (rev 01)
        Subsystem: IBM: Unknown device 0153
        Flags: bus master, slow devsel, latency 64, IRQ 4
        Memory at e8100000 (32-bit, non-prefetchable) [size=4K]
        Memory at e8000000 (32-bit, non-prefetchable) [size=1M]
        Capabilities: [40] Power Management version 2


Likwise, immediately after boot and successful loading:
Device #5 at bus 0 device/function 3/0.
  605610b7 02100007 02000020 00804000 00001401 18000000 18000080 00000000
  00000000 00000000 00000000 635610b7 00000000 00000050 00000000 0a0a0100
...
  Base Address 0: I/O at 00001400.
  Base Address 1: Memory at 18000000.
  Base Address 2: Memory at 18000080.
  Extended capabilities, first structure at offset 0x50.
  Extended PCI capability type 1 at 0x50, next 0.
   Power management entry ver. 2: Capabilities fe02, Ctrl 4000, Event 2900.
   Power state D0.

After resume:
Device #5 at bus 0 device/function 3/0.
  605610b7 02100000 02000020 00800000 00001401 18000000 18000080 00000000
  00000000 00000000 00000000 635610b7 00000000 00000050 00000000 0a0a0100
...
  Base Address 0: I/O at 00001400.
  Base Address 1: Memory at 18000000.
  Base Address 2: Memory at 18000080.
  Extended capabilities, first structure at offset 0x50.
  Extended PCI capability type 1 at 0x50, next 0.
   Power management entry ver. 2: Capabilities fe02, Ctrl 4103, Event 2900.
   Power state D3.

After rmmod 3c59x && modprobe 3c59x:
Device #5 at bus 0 device/function 3/0.
  605610b7 02100003 02000020 00804000 00001401 18000000 18000080 00000000
  00000000 00000000 00000000 635610b7 00000000 00000050 00000000 0a0a0100
...
  Base Address 0: I/O at 00001400.
  Base Address 1: Memory at 18000000.
  Base Address 2: Memory at 18000080.
  Extended capabilities, first structure at offset 0x50.
  Extended PCI capability type 1 at 0x50, next 0.
   Power management entry ver. 2: Capabilities fe02, Ctrl 4103, Event 2900.
   Power state D3.

Eran, your first lspci -nv above is for 0:3.1 (i.e. "lspci -nv -s 0:3.1") 
which is your 3c59x device.  But the later ones are all for 0:5.0, which is a 
sound device. 
 
Would you mind re-running the test and posting the correct output for device 
0:3.1? 

Oops! Copy&paste error in all 3 lspci snippets. I must be tired today...
Here's the relevant output:

Output of pci -nv immediately after boot and successful loading:
00:03.0 Ethernet controller: 3Com Corporation 3c556B CardBus [Tornado] (rev 20)
        Subsystem: 3Com Corporation: Unknown device 6356
        Flags: bus master, medium devsel, latency 64, IRQ 7
        I/O ports at 1400 [size=256]
        Memory at 18000000 (32-bit, non-prefetchable) [size=128]
        Memory at 18000080 (32-bit, non-prefetchable) [size=128]
        Capabilities: [50] Power Management version 2

After resume:
00:03.0 Ethernet controller: 3Com Corporation 3c556B CardBus [Tornado] (rev 20)
        Subsystem: 3Com Corporation: Unknown device 6356
        Flags: medium devsel, IRQ 7
        I/O ports at 1400 [disabled] [size=256]
        Memory at 18000000 (32-bit, non-prefetchable) [disabled] [size=128]
        Memory at 18000080 (32-bit, non-prefetchable) [disabled] [size=128]
        Capabilities: [50] Power Management version 2

After rmmod 3c59x && modprobe 3c59x:
00:03.0 Ethernet controller: 3Com Corporation 3c556B CardBus [Tornado] (rev 20)
        Subsystem: 3Com Corporation: Unknown device 6356
        Flags: medium devsel, IRQ 7
        I/O ports at 1400 [size=256]
        Memory at 18000000 (32-bit, non-prefetchable) [size=128]
        Memory at 18000080 (32-bit, non-prefetchable) [size=128]
        Capabilities: [50] Power Management version 2

Regarding the top of comment 55, are you sure that is happening?  Is the 
device going to D3hot through some other mechanism than the driver calling 
pci_set_power_state?  After the initial load of the driver (when current_state 
defaults to D3cold) then current_state should be updated for every call to 
pci_set_power_state.  So, the kernel should know the correct power state for 
the device. 
 
To be clear, you are saying that after ->resume the 3c59x is unable to 
communicate on the network?  It is still properly configured w.r.t. network 
settings, correct? 

The suspend+resume affects the device's power state. The way I read it,
vortex_suspend() does pci_set_power_state() to PCI_D3hot, but upon resume from
suspend-to-disk the device is in D3cold, so the kernel is again out of sync
again. Not sure why D3hot vs. D3cold should matter, though.

With the latest patch, after suspend-resume the network interface doesn't work
even thoguh correctly configured. Moreover, the NIC is in D3, and rmmod+modprobe
won't revive it. But the module can be successfully loaded after manually
reviving the device using "pci-config -W" and seeting its registers via setpci.
So basically, after resume it behaves exactly like an unpatched kernel after
normal boot.

Created attachment 116214 [details]
pci-enable-d3hot.patch

A more robust implementation, based on upstream feedback...

I don't know if this will help w/ the resume stuff or not, but I would
definitely like to hear some test results both for boot and for resume.

New FC3 and FC4 test kernels w/ above patch available at the same locations as 
in comment 52.  Please give them a try and post the results...thanks! 

Okay, back from the festival.. and telecommuting because the flu i got..

Tested kernel-2.6.11-1.35_FC3.jwltest.18.i586.rpm and it worked, no problems
which i noticed.. 

The patch of comment 60 has the same problem as the previous one: works fine
after boot, but after resume the device is left in D3 (and rmmod+modprobe
doesn't wake it up).

Created attachment 116271 [details]
jwltest-3c59x-resume.patch

Attempted clean-up of 3c59x PM stuff...

FC4 test kernels w/ above patch available here: 
 
   http://people.redhat.com/linville/kernels/fc4/ 
 
Please give these (or at least the patch from comment 64) a try to see if it 
helps the resume issue...thanks! 

The patch from comment 64 behaves like the previous one: boots fine but stays in
D3 after resume.

Sorry, I meant it does that after *adding* the patch of comment 64 on top of the
previous one.

Created attachment 116374 [details]
jwltest-3c59x-resume-debug.patch

This should add some debugging output to the ->resume routine...

Running mainline 2.6.12.1 + swsusp2-2.1.9.5  + pci-enable-d3hot.patch +
jwltest-3c59x-resume.patch + jwltest-3c59x-resume-debug.patch:

On boot:

3c59x: Calling pci_enable_device
ACPI: PCI Interrupt 0000:00:03.0[A] -> Link [LNKC] -> GSI 7 (level, low) -> IRQ 7
3c59x: VORTEX_PCI(vp)->current_state 0
3c59x: pmcsr state 0
3c59x: Calling pci_restore_state

On resume:

3c59x: Entering vortex_resume
3c59x: pdev->current_state 0     // How come? --Eran
3c59x: pmcsr state 3
3c59x: Calling pci_enable_device
ACPI: PCI Interrupt 0000:00:03.0[A] -> Link [LNKC] -> GSI 7 (level, low) -> IRQ 7
3c59x: VORTEX_PCI(vp)->current_state 0
3c59x: pmcsr state 3
3c59x: Calling pci_restore_state
eth0: command 0x5800 did not complete! Status=0xffff
eth0: command 0x2804 did not complete! Status=0xffff
3c59x: Exiting vortex_resume
3c59x: pdev->current_state 0
3c59x: pmcsr state 3

http://lists.suspend2.net/lurker/message/20050623.124303.63a27d55.en.html 
 
The link indicates that there may be problems w/ swsusp2-2.1.9.5.  Could I get 
you to try one of my kernels? 
 
   http://people.redhat.com/linville/kernels/fc4/ 
 
Short of that, perhaps you could try using swsusp2-2.1.9 instead? 
 
I don't see any reason that current_state should be 0 when ->resume is called.  
The swsusp2 patch is pretty big, so it may take quite a bit of analysis.  If 
you could test my FC4-based kernel, that may help.  Thanks! 

John, I would have used your kernels if it could suspend to disk... It has
neither swsusp nor swsusp2 enabled.

Is suspend-to-ram insufficient for testing? 

Plain S3 suspend-to-RAM doesn't trigger the problem with either kernel.

Moreover, you can tell swsusp2 to go to S3 instead of S4, so it goes through
exactly the same routine (including freezing and saving the state to disk and
then reading and restoring it) except that instead of actually reboot, it just
goes in and out of suspend-to-RAM. This doesn't trigger the problem either.

So it seems like an actual reboot is needed between suspend and resume to
trigger the problem.

Created attachment 116436 [details]
pci-d3hot-d0.patch

Latest version of the patch for upstream...this one moves the guts to a new
function that gets called from pci_set_power_state, but only on D3hot->D0
transitions and only for devices that have the "no soft reset" bit cleared in
the PM control register.

More test kernels w/ above patch at same location as in comment 52.  I would 
appreciate some quick testing...thanks! 

The latest patch boots nicely, but after resume still comes up in D3.
Debug output during resume:

3c59x: Entering vortex_resume
3c59x: pdev->current_state 0
3c59x: pmcsr state 3
3c59x: Exiting vortex_resume
3c59x: pdev->current_state 0
3c59x: pmcsr state 3

I added a printk block to vortex_suspend. Here's the self-explanatory output
during suspend:

3c59x: Quitting vortex_suspend
3c59x: pdev->current_state 0
3c59x: pmcsr state 0

This is odd. Weren't both supposed to be D3hot at this point? This is before the
kernel data snapshot is taken, so it would explain why pdev->current_state is 0
after resume.

Upgraded to suspend2 2.1.9.8, BTW.

Eran, thanks once again for the quick testing! 
 
The new info on vortex_suspend is interesting...I'll have to get back to you 
on that... 

Created attachment 116488 [details]
pci-d3hot-d0.patch

One more tweak...*sigh*...

2.6.12.1 + swsusp2-2.1.9.8 + latest pci-d3hot-d0.patch +
jwltest-3c59x-resume.patch + jwltest-3c59x-resume-debug.patch:

As before, boots nicely but after resume comes up in D3.

An update has been released for Fedora Core 3 (kernel-2.6.12-1.1372_FC3) which
may contain a fix for your problem.   Please update to this new kernel, and
report whether or not it fixes your problem.

If you have updated to Fedora Core 4 since this bug was opened, and the problem
still occurs with the latest updates for that release, please change the version
field of this bug to 'fc4'.

Thank you.

Hello,

I am still experiencing this problem under FC4, kernel 2.6.12-1.1398_FC4. 
However, all the FC3 kernels always worked (and still do) for me.

Thanks.

Hello,

kernel-2.6.12-1.1372_FC3 does not bring up the 3C556B network interface on my
IBM TP A21p when ACPI is enabled, whereas the 2.6.11-1.35_FC3.jwltest.20 kernel
does. An extract from the messages file for the failure is included below. 

Jul 18 23:01:31 obelix kernel: ACPI: PCI Interrupt 0000:00:03.0[A] -> Link
[LNKC] -> GSI 11 (level, low) -> IRQ 11
Jul 18 23:01:31 obelix kernel: 3c59x: Donald Becker and others.
www.scyld.com/network/vortex.html
Jul 18 23:01:31 obelix kernel: 0000:00:03.0: 3Com PCI 3c556B Laptop Hurricane at
0x1800. Vers LK1.1.19
Jul 18 23:01:31 obelix kernel: *** EEPROM MAC address is invalid.
Jul 18 23:01:31 obelix kernel: 3c59x: vortex_probe1 fails.  Returns -22
Jul 18 23:01:31 obelix kernel: 3c59x: probe of 0000:00:03.0 failed with error -22
Jul 18 23:01:31 obelix kernel: ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 11
Jul 18 23:01:31 obelix kernel: ACPI: PCI Interrupt 0000:00:05.0[A] -> Link
[LNKA] -> GSI 11 (level, low) -> IRQ 11

I'm still working on getting the fix for this upstream.  Only then will it 
make its way into Fedora.  Please bear with us!  Thanks... 

A version of this patch is upstream as of kernel version 2.6.14-rc1.  This 
will eventually make it into a future Fedora (probably FC5).  Closing this as 
UPSTREAM...thanks!