Bug 698743 - DVD installer hangs: "BUG: soft lockup - CPU#0 stuck for 68s! [kswapd0:52]"
Summary: DVD installer hangs: "BUG: soft lockup - CPU#0 stuck for 68s! [kswapd0:52]"
Keywords:
Status: CLOSED NEXTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: kernel
Version: 15
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Kernel Maintainer List
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2011-04-21 16:40 UTC by Steve Tyler
Modified: 2012-06-04 19:03 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2012-06-04 19:03:04 UTC
Type: ---


Attachments (Terms of Use)
screenshot showing kernel call traces on installer syslog console (2.39 MB, image/jpeg)
2011-04-21 16:44 UTC, Steve Tyler
no flags Details
screenshot: BUG: soft lockup - CPU#0 stuck for 67s! [kswapd0:52] (2.40 MB, image/jpeg)
2011-04-21 16:58 UTC, Steve Tyler
no flags Details
dmesg just before package installation; captured from installer console (55.13 KB, text/plain)
2011-04-21 18:43 UTC, Steve Tyler
no flags Details

Description Steve Tyler 2011-04-21 16:40:27 UTC
Description of problem:
During a bare-metal install, the DVD installer hangs during package installation.
The keyboard is responsive (caps lock, num lock control lights).
However, it is not possible to switch to an installer console.

In a second run, I switched to the syslog console before the hang.
The kernel is periodically dumping call traces referring to kswapd.

The DVD passes media check.

The target system has ~3.7 GB physical memory and two disks, each with a swap partition.

Version-Release number of selected component (if applicable):
F15-Beta-Final DVD x86_64

How reproducible:
Twice.

Steps to Reproduce:
1. In custom layout, specify a preexisting partition to be formatted as ext4 root. The partition is 12 GB.
2. Use preexisting swap without formatting.
3. Specify a default desktop install.

Actual results:
Hangs during package installation.

Expected results:
Install completes.

Additional info:
Screenshot to follow.

Comment 1 Steve Tyler 2011-04-21 16:44:35 UTC
Created attachment 493923 [details]
screenshot showing kernel call traces on installer syslog console

Comment 2 Steve Tyler 2011-04-21 16:58:12 UTC
Created attachment 493936 [details]
screenshot: BUG: soft lockup - CPU#0 stuck for 67s! [kswapd0:52]

Comment 3 Steve Tyler 2011-04-21 17:03:55 UTC
BUG: soft lockup - CPU#0 stuck for 67s! [kswapd0:52]

Comment 4 Steve Tyler 2011-04-21 18:34:30 UTC
Workaround report:
1. disable one swap partition:
   failed with call traces displayed on syslog console
2. disable all swap partitions:
   failed but no call traces were displayed
3. disable all swap partitions and do a minimal install:
   SUCCESS, system boots to root login

Comment 5 Steve Tyler 2011-04-21 18:43:03 UTC
Created attachment 493964 [details]
dmesg just before package installation; captured from installer console

FWIW: "swapon -s"

Filename				Type		Size	Used	Priority
/dev/sda8                               partition	1835004	0	-1
/dev/sdb7                               partition	3999964	0	-2

Comment 6 Steve Tyler 2011-04-22 14:39:41 UTC
(In reply to comment #4)
...
> 3. disable all swap partitions and do a minimal install:
>    SUCCESS, system boots to root login

After completing the minimal install, I manually installed enough packages to run the gnome desktop.

With one swap partition enabled, the system seems to be running normally.

$ uname -a
Linux fir 2.6.38.2-9.fc15.x86_64 #1 SMP Wed Mar 30 16:55:57 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux

$ free
             total       used       free     shared    buffers     cached
Mem:       3799600     758540    3041060          0      20800     200948
-/+ buffers/cache:     536792    3262808
Swap:      3999964          0    3999964

$ swapon -s
Filename				Type		Size	Used	Priority
/dev/sdb7                               partition	3999964	0	-1

Comment 7 Steve Tyler 2011-04-22 15:12:58 UTC
(In reply to comment #4)
> Workaround report:
> 1. disable one swap partition:
>    failed with call traces displayed on syslog console
> 2. disable all swap partitions:
>    failed but no call traces were displayed
> 3. disable all swap partitions and do a minimal install:
>    SUCCESS, system boots to root login

This also succeeds:
leave both swap partitions enabled, do a minimal install
(The previous tests were targeting a partition on sdb, but since I have that running, I did this last test to a spare partition on sda, if that matters.)

Comment 8 Steve Tyler 2011-04-22 16:12:48 UTC
Two more tests:
Both tests targeted sda7 with a full, default graphical install.
Both swap partitions were enabled.

1: Running "top" in the installer's shell console to monitor memory usage
   (default 3 second updates)
   SUCCESS: The install completed and booted to firstboot.
   Memory usage peaked around 3.2 GB, IIRC.
   Swap was never used.

2: Monitoring the installer's syslog console
   FAIL: The install hung.
   kswapd messages were displayed.
   Caps-lock and num-lock did not control the lights.
   It was not possible to switch to another console.
   Ctrl-alt-del switched to the installer's 1 console,
   but was unresponsive after that.
   Rebooted with the reset button.

Comment 9 Sandro Mathys 2011-08-23 12:47:12 UTC
Has anyone ever found a better solution to this issue? We can reproduce it on a few dozen Dell Optiplex 755. Keeping top running on tty2 is a valid workaround - don't understand why that help, though.

Comment 10 Josh Boyer 2012-06-04 18:49:38 UTC
We can't fix F15 install media, nor F16 at this point.  While we can't fix F17 install media, it would be good to know if this still happens with that so we can move this bug to rawhide and work on it for F18 if so.

Comment 11 Sandro Mathys 2012-06-04 18:54:43 UTC
I think we never saw this issue in F16 or F17. FWIW, back in F15: the "keep top running in tty2" workaround only helped in about 3 out of 4 installations - not exactly sure whether some systems were more often affected than others or if it was just the number of tries per system, though.

Comment 12 Josh Boyer 2012-06-04 19:03:04 UTC
(In reply to comment #11)
> I think we never saw this issue in F16 or F17. FWIW, back in F15: the "keep
> top running in tty2" workaround only helped in about 3 out of 4
> installations - not exactly sure whether some systems were more often
> affected than others or if it was just the number of tries per system,
> though.

OK.  Thanks for letting us know.  If anyone does see this in F17/rawhide, please reopen or open a new bug.


Note You need to log in before you can comment on or make changes to this bug.