Bug 1425097 - F26 wget does not honor .netrc
Summary: F26 wget does not honor .netrc
Alias: None
Product: Fedora
Classification: Fedora
Component: wget
Version: 26
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
Assignee: Tomáš Hozza 🤓
QA Contact: Fedora Extras Quality Assurance
Depends On:
TreeView+ depends on / blocked
Reported: 2017-02-20 14:48 UTC by Bruce Jerrick
Modified: 2017-06-09 19:22 UTC (History)
3 users (show)

Fixed In Version: wget-1.19.1-3.fc26
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Last Closed: 2017-06-09 19:22:27 UTC

Attachments (Terms of Use)
Fix dereferences, so .netrc is read. (481 bytes, patch)
2017-02-20 14:48 UTC, Bruce Jerrick
no flags Details | Diff
Add --netrc/--no-netrc option. (483 bytes, patch)
2017-02-20 14:49 UTC, Bruce Jerrick
no flags Details | Diff
Patch to 1.19.1 to add user/passwd and search_netrc traces, for testing. (991 bytes, patch)
2017-02-20 23:09 UTC, Bruce Jerrick
no flags Details | Diff

Description Bruce Jerrick 2017-02-20 14:48:19 UTC
Created attachment 1255721 [details]
Fix dereferences, so .netrc is read.

Description of problem:
'wget' does not actually honor ~/.netrc as advertised, due to a minor
  coding error.
Also, it does not support '--netrc' or '--no-netrc' options (which may be
intentional, but it seems that it should).

Version-Release number of selected component:

How reproducible:

Steps to Reproduce:
1. Run wget to get your favorite password-protected download, with the
   site's username and passowrd included in ~/.netrc .
2. ls -ltu ~/.netrc

Actual results:
The download will fail due with "401 Unauthorized", and
  ~/.netrc will not have been accessed.

Expected results:
Successful download, and atime of ~/.netrc updated.

Additional info:
Patches included as attachments -- one to fix reading .netrc, the
  other to add the '--netrc/--no-netrc' option.  (Both are one-liners.)

Comment 1 Bruce Jerrick 2017-02-20 14:49:56 UTC
Created attachment 1255722 [details]
Add --netrc/--no-netrc option.

Comment 2 Bruce Jerrick 2017-02-20 15:03:25 UTC
Note that 'user' and 'password' are used slightly differenty in http.c vs.
ftp.c, hence the difference in dereferencing when determining whether they
are set.

Comment 3 Tomáš Hozza 🤓 2017-02-20 16:45:16 UTC
Do you see the same issue with 1.19.1?

Comment 4 Bruce Jerrick 2017-02-20 21:54:35 UTC
Give me a few minutes to get a spec file hacked up to build the source
tarball from ftp://ftp.gnu.org/gnu/wget/ (I don't see 1.19.1 in the F26
repo yet).
But in any case, 1.19.1 still seems wrong -- in http.c, the 20 lines or so
preceding the conditional call to search_netrc assumes that 'user' and
'passwd' are both non-NULL, because no matter what the outcome of the two if/else chains, they're both dereferenced by *user and *passwd .
So !user and !passwd are always false (if either was true, i.e., if either pointer was NULL, there would have been a segfault in the previous lines,
So the user/passwd test boils down to just !*passwd, if Mr. Boole has
anything to say about it.
(There may have been some confusion because similar code in ftp.c does
allow them to be NULL .)

From src/http.c:

   1878   /* Find the username with priority */
   1879   if (u->user)
   1880     *user = u->user;
   1881   else if (opt.user && (opt.use_askpass || opt.ask_passwd))
   1882     *user = opt.user;
   1883   else if (opt.http_user)
   1884     *user = opt.http_user;
   1885   else if (opt.user)
   1886     *user = opt.user;
   1887   else
   1888     *user = NULL;
   1890   /* Find the password with priority */
   1891   if (u->passwd)
   1892     *passwd = u->passwd;
   1893   else if (opt.passwd && (opt.use_askpass || opt.ask_passwd))
   1894     *passwd = opt.passwd;
   1895   else if (opt.http_passwd)
   1896     *passwd = opt.http_passwd;
   1897   else if (opt.passwd)
   1898     *passwd = opt.passwd;
   1899   else
   1900     *passwd = NULL;
   1902   /* Check for ~/.netrc if none of the above match */
   1903   if (opt.netrc && (!user || (!passwd || !*passwd)))
   1904     search_netrc (u->host, (const char **) user, (const char **) passwd, 0);

Comment 5 Bruce Jerrick 2017-02-20 22:51:39 UTC
OK, here's the results:

The 1.19.1 code works in the usual case where it has to get both user and passwd
from .netrc .

But unlike my patch, it does not work in the case where only "login" (user) is in
.netrc, and only the password is given on the command-line (via the '--password' option; one can't give just a password with the '"http://username:password@site/"
syntax ("Invalid user name")).
(Tested empirically; I trust Mr. Boole but not myself after 18 hours uptime.)

And there really ought to be a '--no-netrc' option (my main.c patch).

Comment 6 Bruce Jerrick 2017-02-20 23:05:57 UTC
I'm attaching a patch that adds traces and an early exit to http.c, to make testing easy.
(BTW, this isn't an "issue", it's a "problem" or a "bug".  "Issue" is a
euphemism dreamed up by Redmond marketeers.)

Comment 7 Bruce Jerrick 2017-02-20 23:09:07 UTC
Created attachment 1255894 [details]
Patch to 1.19.1 to add user/passwd and search_netrc traces, for testing.

This should tell everything one needs to know, without the need for
an http server.

Comment 8 Tomáš Hozza 🤓 2017-02-21 12:52:58 UTC
For the record, the 1.19.1 is available since 17-Feb.


Did you propose your fix to the upstream? If not, I can review your changes and report it to the upstream.

Comment 9 Bruce Jerrick 2017-02-22 16:34:39 UTC
(In reply to Tomas Hozza from comment #8)
> ...
> Did you propose your fix to the upstream? If not, I can review your changes
> and report it to the upstream.

No, I haven't -- please do.
Thank you.

Comment 10 Fedora End Of Life 2017-02-28 12:23:38 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 26 development cycle.
Changing version to '26'.

Comment 11 Tomáš Hozza 🤓 2017-05-12 17:18:35 UTC
I just sent changes to the upstream. Will wait for their response.

Comment 13 Fedora Update System 2017-06-02 11:27:30 UTC
wget-1.19.1-3.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2017-38c3781b89

Comment 14 Tomáš Hozza 🤓 2017-06-02 11:28:03 UTC
fixed in:

Comment 15 Fedora Update System 2017-06-04 19:40:33 UTC
wget-1.19.1-3.fc26 has been pushed to the Fedora 26 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-38c3781b89

Comment 16 Fedora Update System 2017-06-09 19:22:27 UTC
wget-1.19.1-3.fc26 has been pushed to the Fedora 26 stable repository. If problems still persist, please make note of it in this bug report.

Note You need to log in before you can comment on or make changes to this bug.