Bug 999647

Summary: [abrt] wget-1.14-5.fc18: map_html_tags: Process /usr/bin/wget was killed by signal 7 (SIGBUS)
Product: [Fedora] Fedora Reporter: Wim ten Have <whaveten>
Component: wgetAssignee: Tomáš Hozza <thozza>
Status: CLOSED WONTFIX QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 18CC: micah, thozza, whaveten
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Unspecified   
Whiteboard: abrt_hash:518e003d0eed1987a9d6e321c7d0e9096ec45843
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-02-05 23:13:42 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
File: backtrace
none
File: cgroup
none
File: core_backtrace
none
File: dso_list
none
File: environ
none
File: limits
none
File: maps
none
File: open_fds
none
File: proc_pid_status none

Description Wim ten Have 2013-08-21 18:59:06 UTC 
Version-Release number of selected component:
wget-1.14-5.fc18

Additional info:
reporter:       libreport-2.1.5
backtrace_rating: 4
cmdline:        wget -r http://www.makerwise.com/
crash_function: map_html_tags
executable:     /usr/bin/wget
kernel:         3.10.7-100.fc18.x86_64
runlevel:       N 5
uid:            1500
var_log_messages: Aug 21 20:38:43 kruimel abrt[25621]: Saved core dump of pid 25600 (/usr/bin/wget) to /var/tmp/abrt/ccpp-2013-08-21-20:38:43-25600 (1187840 bytes)

Truncated backtrace:
Thread no. 1 (3 frames)
 #0 map_html_tags at html-parse.c:1144
 #1 get_urls_html at html-url.c:731
 #2 retrieve_tree at recur.c:369
Comment 1 Wim ten Have 2013-08-21 18:59:10 UTC 
Created attachment 788965 [details]
File: backtrace
Comment 2 Wim ten Have 2013-08-21 18:59:13 UTC 
Created attachment 788966 [details]
File: cgroup
Comment 3 Wim ten Have 2013-08-21 18:59:16 UTC 
Created attachment 788967 [details]
File: core_backtrace
Comment 4 Wim ten Have 2013-08-21 18:59:19 UTC 
Created attachment 788968 [details]
File: dso_list
Comment 5 Wim ten Have 2013-08-21 18:59:22 UTC 
Created attachment 788969 [details]
File: environ
Comment 6 Wim ten Have 2013-08-21 18:59:25 UTC 
Created attachment 788970 [details]
File: limits
Comment 7 Wim ten Have 2013-08-21 18:59:28 UTC 
Created attachment 788971 [details]
File: maps
Comment 8 Wim ten Have 2013-08-21 18:59:31 UTC 
Created attachment 788972 [details]
File: open_fds
Comment 9 Wim ten Have 2013-08-21 18:59:34 UTC 
Created attachment 788973 [details]
File: proc_pid_status
Comment 10 Tomáš Hozza 2013-09-02 15:33:58 UTC 
Hi.

Is there any chance you could attach the core dump from the crash? I'm not
able to figure out what caused the problem from the backtrace.

Also when running the same command as you did does not trigger a crash.

Thanks
Comment 11 Wim ten Have 2013-09-02 15:59:32 UTC 
Rerun the command with 20 sessions in parallel.  We use wget(1) to generate a sudden httpd network load for measurement.  The problem reproduces any so often.

Just a thought, because i run -r towards same local directory perhaps there is race condition there on something.

<wtenhave@kruimel:80> cat os_info 
NAME=Fedora
VERSION="18 (Spherical Cow)"
ID=fedora
VERSION_ID=18
PRETTY_NAME="Fedora 18 (Spherical Cow)"
ANSI_COLOR="0;34"
CPE_NAME="cpe:/o:fedoraproject:fedora:18"

<wtenhave@kruimel:73> cat var_log_messages 
Sep  2 17:42:09 kruimel abrt[28084]: Saved core dump of pid 28082 (/usr/bin/wget) to /var/tmp/abrt/ccpp-2013-09-02-17:42:08-28082 (983040 bytes)

<wtenhave@kruimel:64> cat core_backtrace
{   "signal": 7
,   "executable": "/usr/bin/wget"
,   "stacktrace":
      [ {   "crash_thread": true
        ,   "frames":
              [ {   "address": 4283942
                ,   "build_id": "7cdd8049a8f298ed939873cbe787c95e6613181a"
                ,   "build_id_offset": 89638
                ,   "file_name": "/usr/bin/wget"
                ,   "fingerprint": "b160114d1c9c4bca36d53b9a35ad7e319b3f2335"
                }
              , {   "address": 4288637
                ,   "build_id": "7cdd8049a8f298ed939873cbe787c95e6613181a"
                ,   "build_id_offset": 94333
                ,   "file_name": "/usr/bin/wget"
                ,   "fingerprint": "0b5d494b31962a7e3f856a34af8aaa85431fb635"
                }
              , {   "address": 4344186
                ,   "build_id": "7cdd8049a8f298ed939873cbe787c95e6613181a"
                ,   "build_id_offset": 149882
                ,   "file_name": "/usr/bin/wget"
                ,   "fingerprint": "a88e04f1f74a11187d88c537bb8ce72a09e2476b"
                }
              , {   "address": 4218837
                ,   "build_id": "7cdd8049a8f298ed939873cbe787c95e6613181a"
                ,   "build_id_offset": 24533
                ,   "file_name": "/usr/bin/wget"
                ,   "fingerprint": "1f7076c12a4391c4558bccd7acf2a383175811b9"
                } ]
        } ]
}

<wtenhave@kruimel:65> cat dso_list
/usr/lib64/libssl.so.1.0.1e openssl-libs-1:1.0.1e-4.fc18.x86_64 (Fedora Project) 1374347727
/usr/lib64/librt-2.16.so glibc-2.16-33.fc18.x86_64 (Fedora Project) 1374347717
/usr/lib64/libkrb5.so.3.3 krb5-libs-1.10.3-17.fc18.x86_64 (Fedora Project) 1374347727
/usr/lib64/libdl-2.16.so glibc-2.16-33.fc18.x86_64 (Fedora Project) 1374347717
/usr/lib64/gconv/gconv-modules.cache glibc-2.16-33.fc18.x86_64 (Fedora Project) 1374347717
/usr/lib64/libkrb5support.so.0.1 krb5-libs-1.10.3-17.fc18.x86_64 (Fedora Project) 1374347727
/usr/lib64/ld-2.16.so glibc-2.16-33.fc18.x86_64 (Fedora Project) 1374347717
/usr/bin/wget wget-1.14-5.fc18.x86_64 (Fedora Project) 1374681278
/usr/lib64/libuuid.so.1.3.0 libuuid-2.22.2-6.fc18.x86_64 (Fedora Project) 1374347730
/usr/lib64/libnss_dns-2.16.so glibc-2.16-33.fc18.x86_64 (Fedora Project) 1374347717
/usr/lib64/libc-2.16.so glibc-2.16-33.fc18.x86_64 (Fedora Project) 1374347717
/usr/lib64/libpthread-2.16.so glibc-2.16-33.fc18.x86_64 (Fedora Project) 1374347717
/usr/lib64/libz.so.1.2.7 zlib-1.2.7-9.fc18.x86_64 (Fedora Project) 1374346304
/usr/lib64/libnss_files-2.16.so glibc-2.16-33.fc18.x86_64 (Fedora Project) 1374347717
/usr/lib64/libselinux.so.1 libselinux-2.1.12-7.3.fc18.x86_64 (Fedora Project) 1374347724
/usr/lib64/libresolv-2.16.so glibc-2.16-33.fc18.x86_64 (Fedora Project) 1374347717
/usr/lib64/libcom_err.so.2.1 libcom_err-1.42.5-1.fc18.x86_64 (Fedora Project) 1374346304
/usr/lib64/libidn.so.11.6.9 libidn-1.26-1.fc18.x86_64 (Fedora Project) 1374347762
/usr/lib64/libgssapi_krb5.so.2.2 krb5-libs-1.10.3-17.fc18.x86_64 (Fedora Project) 1374347727
/usr/lib64/libpcre.so.1.0.1 pcre-8.31-5.fc18.x86_64 (Fedora Project) 1374347724
/usr/lib/locale/locale-archive glibc-common-2.16-33.fc18.x86_64 (Fedora Project) 1374347712
/usr/lib64/libcrypto.so.1.0.1e openssl-libs-1:1.0.1e-4.fc18.x86_64 (Fedora Project) 1374347727
/usr/lib64/libkeyutils.so.1.4 keyutils-libs-1.5.5-3.fc18.x86_64 (Fedora Project) 1374346307
/usr/lib64/libk5crypto.so.3.1 krb5-libs-1.10.3-17.fc18.x86_64 (Fedora Project) 1374347727
/usr/lib64/libgcc_s-4.7.2-20121109.so.1 libgcc-4.7.2-8.fc18.x86_64 (Fedora Project) 1374346295

<wtenhave@kruimel:68> cat maps
00400000-0045e000 r-xp 00000000 fd:02 932619                             /usr/bin/wget
0065d000-0065e000 r--p 0005d000 fd:02 932619                             /usr/bin/wget
0065e000-00662000 rw-p 0005e000 fd:02 932619                             /usr/bin/wget
00662000-00669000 rw-p 00000000 00:00 0 
00861000-00865000 rw-p 00061000 fd:02 932619                             /usr/bin/wget
02794000-027b5000 rw-p 00000000 00:00 0                                  [heap]
3555200000-3555220000 r-xp 00000000 fd:02 973792                         /usr/lib64/ld-2.16.so
3555420000-3555421000 r--p 00020000 fd:02 973792                         /usr/lib64/ld-2.16.so
3555421000-3555422000 rw-p 00021000 fd:02 973792                         /usr/lib64/ld-2.16.so
3555422000-3555423000 rw-p 00000000 00:00 0 
3555a00000-3555bad000 r-xp 00000000 fd:02 973793                         /usr/lib64/libc-2.16.so
3555bad000-3555dad000 ---p 001ad000 fd:02 973793                         /usr/lib64/libc-2.16.so
3555dad000-3555db1000 r--p 001ad000 fd:02 973793                         /usr/lib64/libc-2.16.so
3555db1000-3555db3000 rw-p 001b1000 fd:02 973793                         /usr/lib64/libc-2.16.so
3555db3000-3555db8000 rw-p 00000000 00:00 0 
3555e00000-3555e03000 r-xp 00000000 fd:02 973805                         /usr/lib64/libdl-2.16.so
3555e03000-3556002000 ---p 00003000 fd:02 973805                         /usr/lib64/libdl-2.16.so
3556002000-3556003000 r--p 00002000 fd:02 973805                         /usr/lib64/libdl-2.16.so
3556003000-3556004000 rw-p 00003000 fd:02 973805                         /usr/lib64/libdl-2.16.so
3556200000-3556216000 r-xp 00000000 fd:02 973794                         /usr/lib64/libpthread-2.16.so
3556216000-3556416000 ---p 00016000 fd:02 973794                         /usr/lib64/libpthread-2.16.so
3556416000-3556417000 r--p 00016000 fd:02 973794                         /usr/lib64/libpthread-2.16.so
3556417000-3556418000 rw-p 00017000 fd:02 973794                         /usr/lib64/libpthread-2.16.so
3556418000-355641c000 rw-p 00000000 00:00 0 
3556a00000-3556a07000 r-xp 00000000 fd:02 973843                         /usr/lib64/librt-2.16.so
3556a07000-3556c06000 ---p 00007000 fd:02 973843                         /usr/lib64/librt-2.16.so
3556c06000-3556c07000 r--p 00006000 fd:02 973843                         /usr/lib64/librt-2.16.so
3556c07000-3556c08000 rw-p 00007000 fd:02 973843                         /usr/lib64/librt-2.16.so
3556e00000-3556e15000 r-xp 00000000 fd:02 973797                         /usr/lib64/libz.so.1.2.7
3556e15000-3557014000 ---p 00015000 fd:02 973797                         /usr/lib64/libz.so.1.2.7
3557014000-3557015000 r--p 00014000 fd:02 973797                         /usr/lib64/libz.so.1.2.7
3557015000-3557016000 rw-p 00015000 fd:02 973797                         /usr/lib64/libz.so.1.2.7
3557200000-3557215000 r-xp 00000000 fd:02 973821                         /usr/lib64/libgcc_s-4.7.2-20121109.so.1
3557215000-3557414000 ---p 00015000 fd:02 973821                         /usr/lib64/libgcc_s-4.7.2-20121109.so.1
3557414000-3557415000 r--p 00014000 fd:02 973821                         /usr/lib64/libgcc_s-4.7.2-20121109.so.1
3557415000-3557416000 rw-p 00015000 fd:02 973821                         /usr/lib64/libgcc_s-4.7.2-20121109.so.1
3557600000-355765c000 r-xp 00000000 fd:02 973830                         /usr/lib64/libpcre.so.1.0.1
355765c000-355785c000 ---p 0005c000 fd:02 973830                         /usr/lib64/libpcre.so.1.0.1
355785c000-355785d000 r--p 0005c000 fd:02 973830                         /usr/lib64/libpcre.so.1.0.1
355785d000-355785e000 rw-p 0005d000 fd:02 973830                         /usr/lib64/libpcre.so.1.0.1
3557a00000-3557a1f000 r-xp 00000000 fd:02 973831                         /usr/lib64/libselinux.so.1
3557a1f000-3557c1e000 ---p 0001f000 fd:02 973831                         /usr/lib64/libselinux.so.1
3557c1e000-3557c1f000 r--p 0001e000 fd:02 973831                         /usr/lib64/libselinux.so.1
3557c1f000-3557c20000 rw-p 0001f000 fd:02 973831                         /usr/lib64/libselinux.so.1
3557c20000-3557c22000 rw-p 00000000 00:00 0 
3557e00000-3557e16000 r-xp 00000000 fd:02 961046                         /usr/lib64/libresolv-2.16.so
3557e16000-3558015000 ---p 00016000 fd:02 961046                         /usr/lib64/libresolv-2.16.so
3558015000-3558016000 r--p 00015000 fd:02 961046                         /usr/lib64/libresolv-2.16.so
3558016000-3558017000 rw-p 00016000 fd:02 961046                         /usr/lib64/libresolv-2.16.so
3558017000-3558019000 rw-p 00000000 00:00 0 
3559a00000-3559a04000 r-xp 00000000 fd:02 973819                         /usr/lib64/libuuid.so.1.3.0
3559a04000-3559c03000 ---p 00004000 fd:02 973819                         /usr/lib64/libuuid.so.1.3.0
3559c03000-3559c04000 r--p 00003000 fd:02 973819                         /usr/lib64/libuuid.so.1.3.0
3559c04000-3559c05000 rw-p 00004000 fd:02 973819                         /usr/lib64/libuuid.so.1.3.0
355e200000-355e39b000 r-xp 00000000 fd:02 971535                         /usr/lib64/libcrypto.so.1.0.1e
355e39b000-355e59b000 ---p 0019b000 fd:02 971535                         /usr/lib64/libcrypto.so.1.0.1e
355e59b000-355e5b5000 r--p 0019b000 fd:02 971535                         /usr/lib64/libcrypto.so.1.0.1e
355e5b5000-355e5c0000 rw-p 001b5000 fd:02 971535                         /usr/lib64/libcrypto.so.1.0.1e
355e5c0000-355e5c5000 rw-p 00000000 00:00 0 
355ea00000-355ea03000 r-xp 00000000 fd:02 973834                         /usr/lib64/libcom_err.so.2.1
355ea03000-355ec02000 ---p 00003000 fd:02 973834                         /usr/lib64/libcom_err.so.2.1
355ec02000-355ec03000 r--p 00002000 fd:02 973834                         /usr/lib64/libcom_err.so.2.1
355ec03000-355ec04000 rw-p 00003000 fd:02 973834                         /usr/lib64/libcom_err.so.2.1
355f200000-355f228000 r-xp 00000000 fd:02 973833                         /usr/lib64/libk5crypto.so.3.1
355f228000-355f428000 ---p 00028000 fd:02 973833                         /usr/lib64/libk5crypto.so.3.1
355f428000-355f429000 r--p 00028000 fd:02 973833                         /usr/lib64/libk5crypto.so.3.1
355f429000-355f42a000 rw-p 00029000 fd:02 973833                         /usr/lib64/libk5crypto.so.3.1
355f42a000-355f42b000 rw-p 00000000 00:00 0 
355f600000-355f603000 r-xp 00000000 fd:02 961119                         /usr/lib64/libkeyutils.so.1.4
355f603000-355f802000 ---p 00003000 fd:02 961119                         /usr/lib64/libkeyutils.so.1.4
355f802000-355f803000 r--p 00002000 fd:02 961119                         /usr/lib64/libkeyutils.so.1.4
355f803000-355f804000 rw-p 00003000 fd:02 961119                         /usr/lib64/libkeyutils.so.1.4
355fa00000-355fa0a000 r-xp 00000000 fd:02 973832                         /usr/lib64/libkrb5support.so.0.1
355fa0a000-355fc09000 ---p 0000a000 fd:02 973832                         /usr/lib64/libkrb5support.so.0.1
355fc09000-355fc0a000 r--p 00009000 fd:02 973832                         /usr/lib64/libkrb5support.so.0.1
355fc0a000-355fc0b000 rw-p 0000a000 fd:02 973832                         /usr/lib64/libkrb5support.so.0.1
355fe00000-355fe41000 r-xp 00000000 fd:02 973836                         /usr/lib64/libgssapi_krb5.so.2.2
355fe41000-3560040000 ---p 00041000 fd:02 973836                         /usr/lib64/libgssapi_krb5.so.2.2
3560040000-3560041000 r--p 00040000 fd:02 973836                         /usr/lib64/libgssapi_krb5.so.2.2
3560041000-3560043000 rw-p 00041000 fd:02 973836                         /usr/lib64/libgssapi_krb5.so.2.2
3560600000-35606d9000 r-xp 00000000 fd:02 973835                         /usr/lib64/libkrb5.so.3.3
35606d9000-35608d9000 ---p 000d9000 fd:02 973835                         /usr/lib64/libkrb5.so.3.3
35608d9000-35608e3000 r--p 000d9000 fd:02 973835                         /usr/lib64/libkrb5.so.3.3
35608e3000-35608e5000 rw-p 000e3000 fd:02 973835                         /usr/lib64/libkrb5.so.3.3
3560e00000-3560e5d000 r-xp 00000000 fd:02 973869                         /usr/lib64/libssl.so.1.0.1e
3560e5d000-356105c000 ---p 0005d000 fd:02 973869                         /usr/lib64/libssl.so.1.0.1e
356105c000-3561060000 r--p 0005c000 fd:02 973869                         /usr/lib64/libssl.so.1.0.1e
3561060000-3561066000 rw-p 00060000 fd:02 973869                         /usr/lib64/libssl.so.1.0.1e
3573c00000-3573c32000 r-xp 00000000 fd:02 973826                         /usr/lib64/libidn.so.11.6.9
3573c32000-3573e31000 ---p 00032000 fd:02 973826                         /usr/lib64/libidn.so.11.6.9
3573e31000-3573e32000 r--p 00031000 fd:02 973826                         /usr/lib64/libidn.so.11.6.9
3573e32000-3573e33000 rw-p 00032000 fd:02 973826                         /usr/lib64/libidn.so.11.6.9
7fc751a87000-7fc751a8c000 r-xp 00000000 fd:02 918366                     /usr/lib64/libnss_dns-2.16.so
7fc751a8c000-7fc751c8c000 ---p 00005000 fd:02 918366                     /usr/lib64/libnss_dns-2.16.so
7fc751c8c000-7fc751c8d000 r--p 00005000 fd:02 918366                     /usr/lib64/libnss_dns-2.16.so
7fc751c8d000-7fc751c8e000 rw-p 00006000 fd:02 918366                     /usr/lib64/libnss_dns-2.16.so
7fc751c8e000-7fc751c9a000 r-xp 00000000 fd:02 918368                     /usr/lib64/libnss_files-2.16.so
7fc751c9a000-7fc751e99000 ---p 0000c000 fd:02 918368                     /usr/lib64/libnss_files-2.16.so
7fc751e99000-7fc751e9a000 r--p 0000b000 fd:02 918368                     /usr/lib64/libnss_files-2.16.so
7fc751e9a000-7fc751e9b000 rw-p 0000c000 fd:02 918368                     /usr/lib64/libnss_files-2.16.so
7fc751e9b000-7fc75828b000 r--p 00000000 fd:02 936630                     /usr/lib/locale/locale-archive
7fc75828b000-7fc758296000 rw-p 00000000 00:00 0 
7fc7582b4000-7fc7582bb000 rw-p 00000000 fd:03 34734094                   /home/wtenhave/P/www.makerwise.com/3d-printer/formlabs/form-1/index.html
7fc7582bb000-7fc7582c2000 r--s 00000000 fd:02 918653                     /usr/lib64/gconv/gconv-modules.cache
7fc7582c2000-7fc7582c3000 rw-p 00000000 00:00 0 
7fffa37b6000-7fffa37d8000 rw-p 00000000 00:00 0                          [stack]
7fffa37fe000-7fffa3800000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]

<wtenhave@kruimel:70> cat limits 
Limit                     Soft Limit           Hard Limit           Units     
Max cpu time              unlimited            unlimited            seconds   
Max file size             unlimited            unlimited            bytes     
Max data size             unlimited            unlimited            bytes     
Max stack size            8388608              unlimited            bytes     
Max core file size        0                    unlimited            bytes     
Max resident set          unlimited            unlimited            bytes     
Max processes             1024                 46854                processes 
Max open files            1024                 4096                 files     
Max locked memory         65536                65536                bytes     
Max address space         unlimited            unlimited            bytes     
Max file locks            unlimited            unlimited            locks     
Max pending signals       46854                46854                signals   
Max msgqueue size         819200               819200               bytes     
Max nice priority         0                    0                    
Max realtime priority     0                    0                    
Max realtime timeout      unlimited            unlimited            us        

<wtenhave@kruimel:71> cat proc_pid_status 
Name:   wget
State:  S (sleeping)
Tgid:   28082
Pid:    28082
PPid:   27925
TracerPid:      0
Uid:    1500    1500    1500    1500
Gid:    100     100     100     100
FDSize: 64
Groups: 10 18 100 
VmPeak:   153092 kB
VmSize:   152972 kB
VmLck:         0 kB
VmPin:         0 kB
VmHWM:      2032 kB
VmRSS:      2016 kB
VmData:      296 kB
VmStk:       140 kB
VmExe:       376 kB
VmLib:      6372 kB
VmPTE:       200 kB
VmSwap:        0 kB
Threads:        1
SigQ:   0/46854
SigPnd: 0000000000000000
ShdPnd: 0000000000000000
SigBlk: 0000000000000000
SigIgn: 0000000000001000
SigCgt: 0000000188000201
CapInh: 0000000000000000
CapPrm: 0000000000000000
CapEff: 0000000000000000
CapBnd: 0000003fffffffff
Seccomp:        0
Cpus_allowed:   ff
Cpus_allowed_list:      0-7
Mems_allowed:   00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000001
Mems_allowed_list:      0
voluntary_ctxt_switches:        56
nonvoluntary_ctxt_switches:     32

<wtenhave@kruimel:82> rpm -qi wget
Name        : wget
Version     : 1.14
Release     : 5.fc18
Architecture: x86_64
Install Date: Wed 24 Jul 2013 05:54:38 PM CEST
Group       : Applications/Internet
Size        : 2046550
License     : GPLv3+
Signature   : RSA/SHA256, Mon 15 Jul 2013 05:25:20 PM CEST, Key ID ff01125cde7f38bd
Source RPM  : wget-1.14-5.fc18.src.rpm
Build Date  : Mon 15 Jul 2013 09:55:22 AM CEST
Build Host  : buildvm-08.phx2.fedoraproject.org
Relocations : (not relocatable)
Packager    : Fedora Project
Vendor      : Fedora Project
URL         : http://www.gnu.org/software/wget/
Summary     : A utility for retrieving files using the HTTP or FTP protocols
Description :
GNU Wget is a file retrieval utility which can use either the HTTP or
FTP protocols. Wget features include the ability to work in the
background while you are logged out, recursive retrieval of
directories, file name wildcard matching, remote file timestamp
storage and comparison, use of Rest with FTP servers and Range with
HTTP servers to retrieve files over slow or unstable connections,
support for Proxy servers, and configurability.

<wtenhave@kruimel:85> gdb `which wget` coredump
GNU gdb (GDB) Fedora (7.5.1-38.fc18)
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /usr/bin/wget...Reading symbols from /usr/lib/debug/usr/bin/wget.debug...done.
done.
[New LWP 28082]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `wget -r http://www.makerwise.com/'.
Program terminated with signal 7, Bus error.
#0  map_html_tags (text=<optimized out>, size=<optimized out>, 
    mapfun=mapfun@entry=0x416590 <collect_tags_mapper>, maparg=maparg@entry=0x7fffa37d6060, flags=2, 
    allowed_tags=allowed_tags@entry=0x0, allowed_attributes=0x2796a70) at html-parse.c:1006
1006                    while (*p != quote_char)
Missing separate debuginfos, use: debuginfo-install keyutils-libs-1.5.5-3.fc18.x86_64 krb5-libs-1.10.3-17.fc18.x86_64 libcom_err-1.42.5-1.fc18.x86_64 libgcc-4.7.2-8.fc18.x86_64 libselinux-2.1.12-7.3.fc18.x86_64 pcre-8.31-5.fc18.x86_64
(gdb) bt
#0  map_html_tags (text=<optimized out>, size=<optimized out>, 
    mapfun=mapfun@entry=0x416590 <collect_tags_mapper>, maparg=maparg@entry=0x7fffa37d6060, flags=2, 
    allowed_tags=allowed_tags@entry=0x0, allowed_attributes=0x2796a70) at html-parse.c:1006
#1  0x000000000041707d in get_urls_html (
    file=0x2798a90 "www.makerwise.com/3d-printer/formlabs/form-1/index.html", url=<optimized out>, 
    meta_disallow_follow=meta_disallow_follow@entry=0x7fffa37d614b, iri=0x279ff90) at html-url.c:731
#2  0x000000000042497a in retrieve_tree (start_url_parsed=start_url_parsed@entry=0x2795090, pi=pi@entry=0x0)
    at recur.c:369
#3  0x0000000000405fd5 in main (argc=3, argv=0x7fffa37d6418) at main.c:1552
(gdb) list
1001                    char quote_char = *p;
1002                    attr_raw_value_begin = p;
1003                    ADVANCE (p);
1004                    attr_value_begin = p; /* <foo bar="baz"> */
1005                                          /*           ^     */
1006                    while (*p != quote_char)
1007                      {
1008                        if (!newline_seen && *p == '\n')
1009                          {
1010                            /* If a newline is seen within the quotes, it
(gdb) print p
$1 = 0x7fc7582b8000 <Address 0x7fc7582b8000 out of bounds>

If the above does not help you further, i saved the core file.  In case you like i can provide.  Drop me a line.

Regards,
-Wim ten Have.
Comment 12 Wim ten Have 2013-09-02 16:07:53 UTC 
I forgot to add in the above;  (outcome of my parallel exercise)

FINISHED --2013-09-02 17:45:27--
Total wall clock time: 3m 25s
Downloaded: 425 files, 11M in 6.4s (1.74 MB/s)

[22]   Exit 8                        wget -r http://www.makerwise.com/
[21] + Bus error                     wget -r http://www.makerwise.com/ (core dumped)
[20] + Exit 8                        wget -r http://www.makerwise.com/
[19] + Exit 8                        wget -r http://www.makerwise.com/
[18] + Bus error                     wget -r http://www.makerwise.com/ (core dumped)
[17] + Exit 8                        wget -r http://www.makerwise.com/
[16] + Bus error                     wget -r http://www.makerwise.com/ (core dumped)
[15] + Exit 8                        wget -r http://www.makerwise.com/
[14] + Exit 8                        wget -r http://www.makerwise.com/
[13] + Exit 8                        wget -r http://www.makerwise.com/
[12] + Exit 8                        wget -r http://www.makerwise.com/
[11] + Exit 8                        wget -r http://www.makerwise.com/
[10] + Exit 8                        wget -r http://www.makerwise.com/
[9]  + Exit 8                        wget -r http://www.makerwise.com/
[8]  + Exit 8                        wget -r http://www.makerwise.com/
[7]  + Exit 8                        wget -r http://www.makerwise.com/
[6]  + Exit 8                        wget -r http://www.makerwise.com/
[5]  + Exit 8                        wget -r http://www.makerwise.com/
[4]  + Exit 8                        wget -r http://www.makerwise.com/
[3]  + Exit 8                        wget -r http://www.makerwise.com/
[2]  + Bus error                     wget -r http://www.makerwise.com/ (core dumped)
[1]  + Exit 8                        wget -r http://www.makerwise.com/

Regards,
-Wim ten Have.
Comment 13 Tomáš Hozza 2013-09-02 16:19:39 UTC 
(In reply to Wim ten Have from comment #11)
> Rerun the command with 20 sessions in parallel.  We use wget(1) to generate
> a sudden httpd network load for measurement.  The problem reproduces any so
> often.

Knowing that you are running multiple wget instances in the same directory to
get the same site recursively helps.

> Just a thought, because i run -r towards same local directory perhaps there
> is race condition there on something.

wget distributed in Fedora is single process/single thread. But there has to be
some race condition among those multiple wget processes.
Comment 14 Tomáš Hozza 2013-09-03 07:01:55 UTC 
So, the problem is indeed caused by race condition on downloaded files. Wget
uses mmap if it is available. Unfortunately it is not defined what will happen
if someone outside the process modifies the mmap-ed file. Also wget is getting
the size of file and then mapping it into memory. If another wget rewrites
the file between the stat() call and mmap() it can cause a problem.

Personally I think that you are misusing wget for what it was not intended.
A workaround in your case would be to use '-P' option so each wget creates its
own directory and don't conflict with another instances.

Something like

for i in `seq 20`; do
    wget -P $i -r http://www.makerwise.com/
done


It tried not to use mmap() but rather read() and wget works just fine with
it, but using read() is slower than using mmap().

I'll have to discuss this with upstream to see if this should be "fixed" at all.
Comment 15 Tomáš Hozza 2013-09-03 07:07:14 UTC 
(In reply to Tomas Hozza from comment #14)
>     wget -P $i -r http://www.makerwise.com/
     wget -P $i -r http://www.makerwise.com/ &
Comment 16 Wim ten Have 2013-09-03 07:43:02 UTC 
I understand that -P or running it to multiple destination directory trees won't cause the problem.

Personally i think people are free to use any command for whatever they think it fits O:-) .. and it should not undocumented hit a SIGSEGV/SIGBUS/SIGILL/SIGFPE.

For a simpel solution, a minor change (suggestion) is to extend the destination to wget with a mkdtemp(3) or mkdir(".wget.PID") under its C(main) core where post completion specific fetched and stored contents are moved to given destination or current if none.

Regards,
-Wim ten Have.
Comment 17 Tomáš Hozza 2013-09-03 07:56:24 UTC 
(In reply to Wim ten Have from comment #16)
> I understand that -P or running it to multiple destination directory trees
> won't cause the problem.
>
> Personally i think people are free to use any command for whatever they
> think it fits O:-) .. and it should not undocumented hit a
> SIGSEGV/SIGBUS/SIGILL/SIGFPE.

Yes, that's true. I was just trying to say that I don't think anyone thought
of this before.

> For a simpel solution, a minor change (suggestion) is to extend the
> destination to wget with a mkdtemp(3) or mkdir(".wget.PID") under its
> C(main) core where post completion specific fetched and stored contents are
> moved to given destination or current if none.

This looks like a good idea. Unfortunately I think you'd end up with the same
issue when moving/renaming the directory (race condition among multiple wget
instances trying to do the same thing).
Comment 18 Tomáš Hozza 2013-09-09 10:14:37 UTC 
After discussion in the upstream, good solution for the issue would be to
use read() instead of mmap() and calculate/verify the checksum of file
when downloading/reading to detect possible corruption.
Comment 19 Fedora End Of Life 2013-12-21 15:36:15 UTC 
This message is a reminder that Fedora 18 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 18. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '18'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 18's end of life.

Thank you for reporting this issue and we are sorry that we may not be 
able to fix it before Fedora 18 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior to Fedora 18's end of life.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.
Comment 20 Fedora End Of Life 2014-02-05 23:13:42 UTC 
Fedora 18 changed to end-of-life (EOL) status on 2014-01-14. Fedora 18 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.