This service will be undergoing maintenance at 00:00 UTC, 2016-09-28. It is expected to last about 1 hours
Bug 116337 - aborting due to signal 6 (abort)
aborting due to signal 6 (abort)
Status: CLOSED DUPLICATE of bug 165367
Product: Red Hat Enterprise Linux 3
Classification: Red Hat
Component: squid (Show other bugs)
3.0
i386 Linux
high Severity high
: ---
: ---
Assigned To: Martin Stransky
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2004-02-20 03:04 EST by Dan Barber
Modified: 2007-11-30 17:07 EST (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2005-10-11 05:42:03 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:


Attachments (Terms of Use)
patch to try fix a few shortcommings... (319.82 KB, patch)
2004-03-03 09:16 EST, Neil Horman
no flags Details | Diff
allow syslogging to work (2.70 KB, patch)
2004-03-19 14:48 EST, Jason Woodward
no flags Details | Diff

  None (edit)
Description Dan Barber 2004-02-20 03:04:06 EST
Description of problem:
I'm seeing squid continuously starting/restarting/crashing in my log
files.  The logpriority is at warning, so I'm assuming this is not a
normal occurrence.

This is on an smp Dell 1655 blade server.

Can anyone tell me what might be causing this?  The logs are showing a
child process exiting due to signal 6. 

man 7 signal shows that signal 6 is
SIGABRT       6       Core    Abort signal from abort(3)


Log messages:

Feb 18 05:31:18 webc1 squid[13403]: storeDirWriteCleanLogs:
Starting...Feb 18 05:31:18 webc1 squid[13403]: WARNING: Closing open
FD   11
Feb 18 05:31:18 webc1 squid[13403]:     65536 entries written so far.
Feb 18 05:31:18 webc1 squid[13403]:    131072 entries written so far.
Feb 18 05:31:18 webc1 squid[13403]:    196608 entries written so far.
Feb 18 05:31:18 webc1 squid[13403]:    262144 entries written so far.
Feb 18 05:31:18 webc1 squid[13403]:    327680 entries written so far.
Feb 18 05:31:18 webc1 squid[13403]:   Finished.  Wrote 360001 entries.
Feb 18 05:31:18 webc1 squid[13403]:   Took 0.3 seconds
(1087258.3entries/sec).
Feb 18 05:31:18 webc1 squid[9313]: Squid Parent: child process
13403exited due to signal 6
Feb 18 05:31:21 webc1 squid[13407]: Starting Squid Cache
version2.5.STABLE3 for i686-pc-linux-gnu...
Feb 18 05:31:21 webc1 squid[13407]: Process ID 13407Feb 18 05:31:21
webc1 squid[13407]: With 1024 file descriptors available
Feb 18 05:31:21 webc1 squid[13407]: DNS Socket created at 0.0.0.0,
port32780, FD 5


2004/02/19 17:10:27| Starting Squid Cache version 2.5.STABLE4 for
i686-pc-linux-gnu...
2004/02/19 17:10:27| Process ID 13474
2004/02/19 17:10:27| With 1024 file descriptors available
2004/02/19 17:10:27| DNS Socket created at 0.0.0.0, port 32786, FD 5
2004/02/19 17:10:27| Adding nameserver 10.10.20.100 from /etc/resolv.conf
2004/02/19 17:10:27| Adding nameserver 10.10.20.101 from /etc/resolv.conf
2004/02/19 17:10:27| Adding nameserver 10.10.20.102 from /etc/resolv.conf
2004/02/19 17:10:27| Adding nameserver 10.10.20.103 from /etc/resolv.conf
2004/02/19 17:10:27| Unlinkd pipe opened on FD 10
2004/02/19 17:10:27| Swap maxSize 33554432 KB, estimated 2581110 objects
2004/02/19 17:10:27| Target number of buckets: 129055
2004/02/19 17:10:27| Using 131072 Store buckets
2004/02/19 17:10:27| Max Mem  size: 524288 KB
2004/02/19 17:10:27| Max Swap size: 33554432 KB
2004/02/19 17:10:27| Store logging disabled
2004/02/19 17:10:27| Rebuilding storage in /var/spool/squid (DIRTY)
2004/02/19 17:10:27| Using Round Robin store dir selection
2004/02/19 17:10:27| Current Directory is /
2004/02/19 17:10:27| Loaded Icons.
2004/02/19 17:10:27| Accepting HTTP connections at 0.0.0.0, port 80,
FD 11.
2004/02/19 17:10:27| Accepting ICP messages at 0.0.0.0, port 3130, FD 13.
2004/02/19 17:10:27| Accepting SNMP messages on port 3401, FD 14.
2004/02/19 17:10:27| WCCP Disabled.
2004/02/19 17:10:27| Ready to serve requests.
2004/02/19 17:10:27| Configuring Sibling webc1.nocout.navy.mil/80/3130
2004/02/19 17:10:27| Store rebuilding is  0.8% complete
FATAL: Received Segment Violation...dying.
2004/02/19 17:10:33| Not currently OK to rewrite swap log.
2004/02/19 17:10:33| storeDirWriteCleanLogs: Operation aborted.
CPU Usage: 6.010 seconds = 2.900 user + 3.110 sys
Maximum Resident Size: 0 KB
Page faults with physical i/o: 516
Memory usage for squid via mallinfo():
  total space in arena:   32474 KB
  Ordinary blocks:        32456 KB      2 blks
  Small blocks:               0 KB      0 blks
  Holding blocks:           716 KB      2 blks
  Free Small blocks:          0 KB
  Free Ordinary blocks:      18 KB
  Total in use:           33172 KB 102%
  Total free:                18 KB 0%

Feb 19 17:10:27 webc2 squid[13474]: Using Round Robin store dir selection
Feb 19 17:10:27 webc2 squid[13474]: Current Directory is /
Feb 19 17:10:27 webc2 squid[13474]: Loaded Icons.
Feb 19 17:10:27 webc2 squid[13474]: Accepting HTTP connections at
0.0.0.0, port 80, FD 11.
Feb 19 17:10:27 webc2 squid[13474]: Accepting ICP messages at 0.0.0.0,
port 3130, FD 13.
Feb 19 17:10:27 webc2 squid[13474]: Accepting SNMP messages on port
3401, FD 14.
Feb 19 17:10:27 webc2 squid[13474]: WCCP Disabled.
Feb 19 17:10:27 webc2 squid[13474]: Ready to serve requests.
Feb 19 17:10:27 webc2 squid[13474]: Configuring Sibling
webc1.nocout.navy.mil/80/3130
Feb 19 17:10:27 webc2 squid[13474]: Store rebuilding is  0.8% complete
Feb 19 17:10:30 webc2 squid[13474]: access 1077210630.391    667
205.72.1.21 TCP_MISS/304 267 GET
http://images.bestbuy.com/BestBuy_US/images/global/misc/gl_stripes_top_333399.gif
- TIMEOUT_DIRECT/63.215.124.23 image/gif
Feb 19 17:10:31 webc2 squid[13474]: access 1077210631.306    658
205.106.99.21 TCP_MISS/200 2241 GET
http://thumbs.ebaystatic.com/pict/24604322966464.jpg -
DIRECT/64.124.29.189 image/jpeg
Feb 19 17:10:31 webc2 squid[13474]: access 1077210631.382    664
205.72.1.21 TCP_MISS/304 240 GET
http://us.i1.yimg.com/us.yimg.com/i/us/nws/main5.gif -
DIRECT/63.215.124.46 image/gif
Feb 19 17:10:32 webc2 squid[13474]: access 1077210632.122    692
205.106.99.21 TCP_MISS/200 1879 GET
http://thumbs.ebaystatic.com/pict/24606385106464.jpg -
TIMEOUT_DIRECT/64.124.29.188 image/jpeg
Feb 19 17:10:32 webc2 squid[13474]: access 1077210632.199   1048
205.106.145.42 TCP_MISS/200 5626 GET
http://www.epicgals.com/am33/affiliates/ - DIRECT/66.228.210.150 text/html
Feb 19 17:10:32 webc2 squid[13474]: access 1077210632.211    686
205.72.1.21 TCP_MISS/302 263 GET
http://www.sharperimage.com/ionicbreeze - TIMEOUT_DIRECT/64.41.222.72
text/html
Feb 19 17:10:32 webc2 squid[13474]: access 1077210632.268    678
205.106.99.21 TCP_MISS/200 2445 GET
http://www.ebay.com/sfpromo/allthingsautoV2.html -
TIMEOUT_DIRECT/66.135.192.88 text/html
Feb 19 17:10:32 webc2 squid[13474]: access 1077210632.345    623
205.72.1.21 TCP_MISS/200 943 GET
http://www.sonystyle.com/intershoproot/eCS/Store/en/imagesOnline/header/hdr_btn_support_hi.gif
- DIRECT/129.33.21.28 image/gif
Feb 19 17:10:32 webc2 squid[13474]: access 1077210632.766   2033
205.72.1.21 TCP_MISS/200 12475 GET
http://images.overstock.com/f/102/3117/8h/www.overstock.com/images/products/muze/dvd/205745.JPG
- DIRECT/63.215.124.45 image/jpeg
Feb 19 17:10:32 webc2 squid[13474]: access 1077210632.901    677
205.106.99.21 TCP_MISS/200 2053 GET
http://thumbs.ebaystatic.com/pict/24606201056464.jpg -
TIMEOUT_DIRECT/64.124.29.220 image/jpeg
Feb 19 17:10:32 webc2 squid[13474]: access 1077210632.962    643
205.72.1.21 TCP_MISS/200 2934 GET
http://www.biotimeinc.com/ProductSub.gif -
TIMEOUT_DIRECT/216.205.146.188 image/gif
Feb 19 17:10:33 webc2 squid[13474]: access 1077210633.081   1055
205.106.145.42 TCP_MISS/200 6029 GET
http://www.epicgals.com/am33/affiliates/images/011t.jpg -
TIMEOUT_DIRECT/66.228.210.148 image/jpeg
Feb 19 17:10:33 webc2 squid[13474]: access 1077210633.282    671
205.72.1.21 TCP_MISS/304 199 GET
http://us.js1.yimg.com/us.yimg.com/lib/hdr/ygma.css -
DIRECT/166.90.208.153 text/css
Feb 19 17:10:33 webc2 squid[13474]: access 1077210633.384   1059
205.106.145.42 TCP_MISS/200 5525 GET
http://www.epicgals.com/am33/affiliates/images/003t.jpg -
TIMEOUT_DIRECT/66.228.210.150 image/jpeg
Feb 19 17:10:33 webc2 squid[13474]: access 1077210633.500    684
205.72.1.21 TCP_MISS/302 851 GET
http://www.sharperimage.com/ionicbreeze/ - TIMEOUT_DIRECT/64.41.222.72
text/html
Feb 19 17:10:33 webc2 squid[13474]: Not currently OK to rewrite swap log.
Feb 19 17:10:33 webc2 squid[13474]: storeDirWriteCleanLogs: Operation
aborted.
Feb 19 17:10:33 webc2 squid[13200]: Squid Parent: child process 13474
exited due to signal 6
Feb 19 17:10:33 webc2 squid[13200]: Exiting due to repeated, frequent
failures

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.  Start squid on 1655
2.  Load under normal load
3.  Watch logs and see failures and crashes
  
Actual results:
See logs in Description 

Expected results:
Normal occurance as not shown from logs in Description 

Additional info:
Comment 1 Jay Fenlason 2004-02-20 14:31:11 EST
Can you attach your squid.conf file to this so I'll have a chance of
debugging it?  The "FATAL: Received Segment Violation...dying."
certainly appears to be an error, but without the config file, I can't
even try to replicate it.
Comment 2 Dan Barber 2004-02-21 02:41:19 EST
http_port 80
icp_port 3130
cache_peer webc1.nocout.xxxx.xxx sibling 80 3130
hierarchy_stoplist cgi-bin ?
cache_mem  512 MB
cache_swap_low  72
cache_swap_high 75
maximum_object_size 32768 KB
minimum_object_size 0 KB
cache_dir diskd /var/spool/squid 32768 32 1024
store_dir_select_algorithm round-robin
cache_store_log none
debug_options ALL,1
ftp_user ftpuser@xxxx.xxx
acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl webc1.nocout.xxxx.xxx src 10.10.20.113/255.255.255.255
acl nocman src 10.10.20.120/255.255.255.255
acl snmppublic snmp_community pubinner
acl Safe_ports port 80 21 443 563 70 210 1025-65535
http_access deny !Safe_ports
http_access allow all
icp_access allow webc1.nocout.xxxx.xxx
icp_access deny all
miss_access allow all
cache_mgr webmaster
cachemgr_passwd secret all
cache_effective_user squid
cache_effective_group squid
visible_hostname webc2.ior.xxxx.xxx
httpd_accel_host virtual
httpd_accel_port 80
httpd_accel_with_proxy on
httpd_accel_uses_host_header on
dns_testnames infosec.xxxx.xxx spawar.xxxx.xxx
logfile_rotate 5
buffered_logs on
always_direct allow nocman
snmp_port 3401
snmp_access allow snmppublic localhost
snmp_access deny all
memory_pools on
log_icp_queries off
ipcache_size 4192
fqdncache_size 4192
Comment 3 Neil Horman 2004-02-23 10:13:04 EST
It would also be helpful if you could enable core file dumps and
setuid cores.  If you could attach the core here it would be helpful
in debugging this.  Thanks!
Comment 4 Neil Horman 2004-03-03 09:16:49 EST
Created attachment 98238 [details]
patch to try fix a few shortcommings...

Based on the last core that jason (woodward@tds-incorp.com) sent me regarding
this bug, This patch should clear up a few things.  The core file submitted
clearly indicated that the segfault occured inside logfilePrintf().  My guess
is that some function was calling this with an improperly constructed format
string, or an invalid requisite argument.  I was unfortunately unable to find
the problem, but I was able to relieve the segfault by rewriting the
logfilePrintf function as a macro in this patch.  I understand this is a
completely undesireable fix, but since this group is under a deadline, I
thought this might help track down the real problem, or at least patch it up
until a better fix is constructed.

Also, this patch corrects something else that I noticed while investigating
this problem.  The assert macro is used quite a bit in this app, but since it
runs a child as a daemon, any error messages reported due to failed asserts get
lost.  This patch also adds a wrapper around assert to copy an error message
into syslog anytime an assert fails.

While I'm writing here, can I ask why squid is built with -fPIE and -pie turned
on?  This causes it to be reported as a shared object, and makes for difficult
debugging, since you have to manually load it at the appropriate address in the
debugger.
Comment 5 Jason Woodward 2004-03-19 14:48:32 EST
Created attachment 98692 [details]
allow syslogging to work

The attached patch seems to fix our syslog issues on top of the fix the last
patch gave us.
Comment 6 Mindaugas Riauba 2004-07-16 01:01:44 EDT
While testing one setup we also got this error message. While googling 
around I found suggestion that upgrading to STABLE4 may solve the 
problem: http://www.mail-archive.com/squid-users@squid-
cache.org/msg08444.html
By the way our test was performed on x86_64 dual Opteron system.
Comment 7 Bob Gorman 2005-07-29 10:52:20 EDT
I get this under RHEL3U5.

It started when I upgraded
from: squid-2.5.STABLE3-6.3E.8
  to: squid-2.5.STABLE3-6.3E.13

I backed down to:
    squid-2.5.STABLE3-6.3E.9

And the problem has cleared up.
Comment 8 Martin Stransky 2005-09-13 08:58:46 EDT
Could you please create and attach a strack trace when squid crashes? How to is
here - http://people.redhat.com/stransky/squid.html
Comment 9 Martin Stransky 2005-09-30 04:54:54 EDT
Due to PIE gdb can't read symbols from the debug package. If your squid crashes
(and it isn't a problem with assertion) and you can't obtain a stack-trace there
are new packages which aren't compiled with PIE:

http://people.redhat.com/stransky/squid.html
Comment 10 Martin Stransky 2005-10-11 05:42:03 EDT

*** This bug has been marked as a duplicate of 165367 ***

Note You need to log in before you can comment on or make changes to this bug.