462451 – wget -r only gets index.html and robots.txt

Bug 462451 - wget -r only gets index.html and robots.txt

Summary: wget -r only gets index.html and robots.txt

Keywords:
Status:	CLOSED NOTABUG
Alias:	None
Product:	Fedora
Classification:	Fedora
Component:	wget
Sub Component:
Version:	rawhide
Hardware:	All
OS:	Linux
Priority:	medium
Severity:	medium
Target Milestone:	---
Assignee:	Karsten Hopp
QA Contact:	Fedora Extras Quality Assurance
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+	depends on / blocked

Reported:	2008-09-16 13:21 UTC by Patrice Dumas
Modified:	2008-09-25 15:35 UTC (History)
CC List:	1 user (show)
Fixed In Version:
Doc Type:	Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed:	2008-09-25 15:35:56 UTC
Type:	---
Embargoed:
Dependent Products:

Attachments	(Terms of Use)

Description Patrice Dumas 2008-09-16 13:21:14 UTC

Description of problem:

This may be the same than Bug 286161, but the patch that was
problematic shouldn't be there.

For example (beware that this is a scratch build, so will 
disappear in 1 week):
wget -r -np http://koji.fedoraproject.org/scratch/pertusus/task_827728/



$ wget -r -np http://koji.fedoraproject.org/scratch/pertusus/task_827728/
--2008-09-16 15:20:42--  http://koji.fedoraproject.org/scratch/pertusus/task_827728/
Resolving koji.fedoraproject.org... 209.132.176.125
Connecting to koji.fedoraproject.org|209.132.176.125|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 3607 (3.5K) [text/html]
Saving to: `koji.fedoraproject.org/scratch/pertusus/task_827728/index.html'

100%[============================================================================================>] 3,607       --.-K/s   in 0s      

2008-09-16 15:20:42 (45.8 MB/s) - `koji.fedoraproject.org/scratch/pertusus/task_827728/index.html' saved [3607/3607]

Loading robots.txt; please ignore errors.
--2008-09-16 15:20:42--  http://koji.fedoraproject.org/robots.txt
Connecting to koji.fedoraproject.org|209.132.176.125|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 26 [text/plain]
Saving to: `koji.fedoraproject.org/robots.txt'

100%[============================================================================================>] 26          --.-K/s   in 0s      

2008-09-16 15:20:42 (727 KB/s) - `koji.fedoraproject.org/robots.txt' saved [26/26]

FINISHED --2008-09-16 15:20:42--
Downloaded: 2 files, 3.5K in 0s (31.5 MB/s)


Version-Release number of selected component (if applicable):

wget-1.11.4-1.fc10.i386

How reproducible:


Steps to Reproduce:
1.
2.
3.
  
Actual results:


Expected results:


Additional info:

Comment 1 Micah Cowan 2008-09-16 16:14:32 UTC

Not a bug. Contents of robots.txt file:

User-agent: *
Disallow: /

Please see http://wget.addictivecode.org/FrequentlyAskedQuestions#robots

Comment 2 Patrice Dumas 2008-09-25 15:35:56 UTC

Sorry for the RTFM bug report...

Note You need to log in before you can comment on or make changes to this bug.