Hide Forgot
abrt version: 1.1.18 architecture: i686 Attached file: backtrace, 19875 bytes cmdline: (squid) -f /etc/squid/squid.conf component: squid Attached file: coredump, 55787520 bytes crash_function: death executable: /usr/sbin/squid kernel: 2.6.35.13-92.fc14.i686.PAE package: squid-7:3.1.12-1.fc14 rating: 4 reason: Process /usr/sbin/squid was killed by signal 6 (SIGABRT) release: Fedora release 14 (Laughlin) time: 1310644432 uid: 0 comment ----- Strange things I have in my configuration: - IPv6 setup - URL redirector - Transparent redirector (configured, but in use anymore) How to reproduce ----- 1. Squid is running 2. Suddenly, it dies: "Squid Parent: child process XXX exited due to signal 6 with status 0" 3. Happens many times a day
Created attachment 512873 [details] File: backtrace
May I have your conf file? Thanks, Jiri
And what is said in cache.log?
I am currently running squid-3.1.15-1.fc14.i686 Logs from the last month do not show anymore this problem, but last week I had to clean the disk cache, due to some inconsistencies, and since then, I had no more problems. Not sure if the problems are related, though. Anyway, here is an excerpt from cache.log: 2011/09/24 17:58:01| Starting Squid Cache version 3.1.15 for i386-redhat-linux-gnu... 2011/09/24 17:58:01| Process ID 952 2011/09/24 17:58:01| With 1024 file descriptors available 2011/09/24 17:58:01| Initializing IP Cache... 2011/09/24 17:58:01| DNS Socket created at [::], FD 7 2011/09/24 17:58:01| DNS Socket created at 0.0.0.0, FD 8 2011/09/24 17:58:01| Adding domain home.jonny.eng.br from /etc/resolv.conf 2011/09/24 17:58:01| Adding nameserver 127.0.0.1 from /etc/resolv.conf 2011/09/24 17:58:01| helperOpenServers: Starting 5/5 'redir.pl' processes 2011/09/24 17:58:01| User-Agent logging is disabled. 2011/09/24 17:58:01| Referer logging is disabled. 2011/09/24 17:58:01| Unlinkd pipe opened on FD 23 2011/09/24 17:58:01| Local cache digest enabled; rebuild/rewrite every 3600/3600 sec 2011/09/24 17:58:01| Store logging disabled 2011/09/24 17:58:01| Swap maxSize 33554432 + 32768 KB, estimated 2583630 objects 2011/09/24 17:58:01| Target number of buckets: 129181 2011/09/24 17:58:01| Using 131072 Store buckets 2011/09/24 17:58:01| Max Mem size: 32768 KB 2011/09/24 17:58:01| Max Swap size: 33554432 KB 2011/09/24 17:58:01| Version 1 of swap file with LFS support detected... 2011/09/24 17:58:01| Rebuilding storage in /var/spool/squid (CLEAN) 2011/09/24 17:58:01| Using Least Load store dir selection 2011/09/24 17:58:01| Set Current Directory to /var/spool/squid 2011/09/24 17:58:01| Loaded Icons. 2011/09/24 17:58:01| Accepting HTTP connections at [::]:3128, FD 26. 2011/09/24 17:58:01| Accepting intercepted HTTP connections at 0.0.0.0:3129, FD 27. 2011/09/24 17:58:01| HTCP Disabled. 2011/09/24 17:58:01| Squid plugin modules loaded: 0 2011/09/24 17:58:01| Adaptation support is off. 2011/09/24 17:58:01| Ready to serve requests. 2011/09/24 17:58:01| Store rebuilding is 0.81% complete 2011/09/24 17:58:02| Done reading /var/spool/squid swaplog (505404 entries) 2011/09/24 17:58:02| Finished rebuilding storage from disk. 2011/09/24 17:58:02| 505404 Entries scanned 2011/09/24 17:58:02| 0 Invalid entries. 2011/09/24 17:58:02| 0 With invalid flags. 2011/09/24 17:58:02| 505404 Objects loaded. 2011/09/24 17:58:02| 0 Objects expired. 2011/09/24 17:58:02| 0 Objects cancelled. 2011/09/24 17:58:02| 0 Duplicate URLs purged. 2011/09/24 17:58:02| 0 Swapfile clashes avoided. 2011/09/24 17:58:02| Took 1.56 seconds (324402.60 objects/sec). 2011/09/24 17:58:02| Beginning Validation Procedure 2011/09/24 17:58:02| 524288 Entries Validated so far. 2011/09/24 17:58:02| Completed Validation Procedure 2011/09/24 17:58:02| Validated 1010827 Entries 2011/09/24 17:58:02| store_swap_size = 29194244 2011/09/24 17:58:03| storeLateRelease: released 0 objects 2011/09/24 18:02:09| WARNING: swapfile header inconsistent with available data FATAL: Received Segment Violation...dying. 2011/09/24 18:02:09| storeDirWriteCleanLogs: Starting... 2011/09/24 18:02:09| WARNING: Closing open FD 26 2011/09/24 18:02:09| 65536 entries written so far. 2011/09/24 18:02:09| 131072 entries written so far. 2011/09/24 18:02:09| 196608 entries written so far. 2011/09/24 18:02:09| 262144 entries written so far. 2011/09/24 18:02:09| 327680 entries written so far. 2011/09/24 18:02:09| 393216 entries written so far. 2011/09/24 18:02:09| 458752 entries written so far. 2011/09/24 18:02:09| Finished. Wrote 505569 entries. 2011/09/24 18:02:09| Took 0.10 seconds (5130961.20 entries/sec). CPU Usage: 3.426 seconds = 1.844 user + 1.583 sys Maximum Resident Size: 293344 KB Page faults with physical i/o: 0 Memory usage for squid via mallinfo(): total space in arena: 64816 KB Ordinary blocks: 64675 KB 30 blks Small blocks: 0 KB 1 blks Holding blocks: 5308 KB 7 blks Free Small blocks: 0 KB Free Ordinary blocks: 140 KB Total in use: 69983 KB 108% Total free: 140 KB 0% 2011/09/24 18:02:13| Starting Squid Cache version 3.1.15 for i386-redhat-linux-gnu... 2011/09/24 18:02:13| Process ID 1097 2011/09/24 18:02:13| With 1024 file descriptors available
The first findings: - the source of troubles are url_rewrite* options. to be continued ...
the backtrace is cache swapin related, not url_rewrite.
Henrik, I've sent in private my squid.conf to Jiri, maybe he has seen something interesting there. I'll send it to you too. But I agree with you. Taking a further look into the backtrace, and to the troubles I had with squid-3.1.15-1.fc14.i686, probably some check is missing in the Swap Validation routines. This machine is my desktop and also my lab. It crashes sometimes, and the filesystem may be left in an inconsistent state that squid3 is not able to deal with. Thanks in advance for your help!
Yes, the c#5 is nonsense. I've written the comment based only on the first experience with the conf file. I agree with the possible filesystem issue.
I'm decided to close this bug in accordance with your private mail from October 4 2011. The decision is based on oncoming EOL of F14. If the bug occurs again on higher Fedora release I'll await reopening the bug.
Hello We run squid here at the university on an IBMx3650 M3 with 32 GB of RAM on a RAID0 ( 4/146GB with two disks paired and mirrored) There was a power surge at the university this afternoon and the computer did not shutdown normally. None of the machines however suffered any damage as a result of the surge (the central powerbackup however did). When the machine was turned on, squid started up but incessantly kept crashing with 100%cpu usage with the following error at cache.log 2013/03/16 16:35:03| WARNING: swapfile header inconsistent with available data FATAL: Received Segment Violation...dying. 2013/03/16 16:35:03| storeDirWriteCleanLogs: Starting... 2013/03/16 16:35:03| WARNING: Closing open FD 17 2013/03/16 16:35:03| 65536 entries written so far. 2013/03/16 16:35:03| 131072 entries written so far. 2013/03/16 16:35:03| 196608 entries written so far. 2013/03/16 16:35:03| 262144 entries written so far. 2013/03/16 16:35:03| 327680 entries written so far. 2013/03/16 16:35:03| 393216 entries written so far. 2013/03/16 16:35:03| 458752 entries written so far. 2013/03/16 16:35:03| 524288 entries written so far. 2013/03/16 16:35:03| 589824 entries written so far. 2013/03/16 16:35:03| 655360 entries written so far. 2013/03/16 16:35:03| 720896 entries written so far. 2013/03/16 16:35:03| 786432 entries written so far. 2013/03/16 16:35:03| Finished. Wrote 833156 entries. 2013/03/16 16:35:03| Took 0.24 seconds (3470037.48 entries/sec). CPU Usage: 14.707 seconds = 12.713 user + 1.994 sys Maximum Resident Size: 1110848 KB Page faults with physical i/o: 0 Memory usage for squid via mallinfo(): total space in arena: 291852 KB Ordinary blocks: 291711 KB 50 blks Small blocks: 0 KB 5 blks Holding blocks: 15260 KB 10 blks Free Small blocks: 0 KB Free Ordinary blocks: 140 KB Total in use: 306971 KB 105% Total free: 141 KB 0% Any help will be much appreciated! Thank you
UPDATE: So we looked into squid's cache folder and found that there was a 'swap.state' and a 'swap.state.last-clean'. We proceeded by removing both this files in a hope that squid would recreate them. This didn't happen and squid created a new 'swap.state' and a 'swap.state.new'. But the crashes persisted. So we decided to remove the cache folder entirely and use the squid -z command to recreate the cache. This did the trick it seems, squid is running fine as before. Also, forgot to mention this earlier but we're running RHEL 6.4 (Santiago) Thank you