Bug 53590 - malloc frenzy until kernel kills process
Summary: malloc frenzy until kernel kills process
Alias: None
Product: Red Hat Linux
Classification: Retired
Component: wget   
(Show other bugs)
Version: 7.1
Hardware: i386
OS: Linux
Target Milestone: ---
Assignee: Trond Eivind Glomsrxd
QA Contact:
Depends On:
TreeView+ depends on / blocked
Reported: 2001-09-12 13:11 UTC by j. alan eldridge
Modified: 2008-05-01 15:38 UTC (History)
0 users

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2001-12-06 19:32:59 UTC
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

Description j. alan eldridge 2001-09-12 13:11:45 UTC
Description of Problem:

When recursively descending large web sites (mirroring), wget will 
sometimes get into a state where it seems to get stuck in a loop (I 
usually use -nc so this consists of "xxx already there, will not 
retrieve, yyy already there will not retrieve, xxx already...").

I can't provide a meaningful test case that is of usable size. I'm going 
to go on a spelunking mission in the source to see if I can track it down 
using just my understanding of the source code (I've patched this sucker 
a number of times, so I'm already familiar).

Right now, suggest you either (1) wait a few days until I either come up 
with something or concede concede defeat or (2) try the same course.

TODO: see the most recent issue of MSDN. There's an article about a 
program for IIS (which would translate to a simple module or CGI for a 
real web server) that uses statistical analysis to kick mirroring 
programs off the site. I'm going to patch wget to select a random number 
from a range of "time-to-wait-between-connect" values to defeat it. Also, 
note the irresponsible and brain-dead suggestion that it would be best to 
only use the class C net address of the client to identify visitors, 
since DHCP defeats it otherwise. E.g.: I have a 4 bit netblock on a 
registered domain (geeksrus.net). Someone else I've never heard of on the 
same class C gets me banned from a website. Or you're on DHCP, so one guy 
bans 252 other people. Nice.


Version-Release number of selected component (if applicable):

How Reproducible:

Steps to Reproduce:

Actual Results:

Expected Results:

Additional Information:

Comment 1 j. alan eldridge 2001-09-12 13:12:47 UTC
Just a test to see if I can submit a followup w/o logging in again.

Comment 2 Trond Eivind Glomsrxd 2001-09-12 14:55:12 UTC
I need a meaningful test case to look into it...

Comment 3 Trond Eivind Glomsrxd 2001-12-06 19:32:54 UTC
Still needing testcase... I've never seen it happen

Comment 4 Trond Eivind Glomsrxd 2002-07-25 22:45:38 UTC
Closing due to lack of feedback and reproducability.

Note You need to log in before you can comment on or make changes to this bug.