Bug 1601170 - [abrt] youtube-dl: _really_load(): cookiejar.py:2063:_really_load:http.cookiejar.LoadError: invalid Netscape format cookies file '/home/daddy/.mozilla/firefox/8c3de6f0.default/cookies-1.txt': 'www.lo'
Summary: [abrt] youtube-dl: _really_load(): cookiejar.py:2063:_really_load:http.cookie...
Keywords:
Status: CLOSED EOL
Alias: None
Product: Fedora
Classification: Fedora
Component: youtube-dl
Version: 27
Hardware: x86_64
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Till Maas
QA Contact: Fedora Extras Quality Assurance
URL: https://retrace.fedoraproject.org/faf...
Whiteboard: abrt_hash:bc67f1cf2c3c73728c2641cae88...
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-07-14 13:25 UTC by David W. Legg
Modified: 2018-11-30 22:11 UTC (History)
5 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2018-11-30 22:11:37 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
File: backtrace (9.28 KB, text/plain)
2018-07-14 13:25 UTC, David W. Legg
no flags Details
File: cgroup (319 bytes, text/plain)
2018-07-14 13:25 UTC, David W. Legg
no flags Details
File: cpuinfo (1.14 KB, text/plain)
2018-07-14 13:25 UTC, David W. Legg
no flags Details
File: environ (3.63 KB, text/plain)
2018-07-14 13:25 UTC, David W. Legg
no flags Details
File: mountinfo (3.37 KB, text/plain)
2018-07-14 13:25 UTC, David W. Legg
no flags Details
File: namespaces (129 bytes, text/plain)
2018-07-14 13:25 UTC, David W. Legg
no flags Details
File: open_fds (244 bytes, text/plain)
2018-07-14 13:25 UTC, David W. Legg
no flags Details

Description David W. Legg 2018-07-14 13:25:19 UTC
Version-Release number of selected component:
youtube-dl-2018.05.18-1.fc27

Additional info:
reporter:       libreport-2.9.3
cmdline:        /usr/bin/python3 /usr/bin/youtube-dl --user-agent 'Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:61.0) Gecko/20100101 Firefox/61.0' https://www.itv.com/hub/the-bletchley-circle/2a1939a0004 --cookies /home/daddy/.mozilla/firefox/8c3de6f0.default/cookies-1.txt
crash_function: _really_load
exception_type: ValueError
executable:     /usr/bin/youtube-dl
kernel:         4.17.3-100.fc27.x86_64
runlevel:       N 5
type:           Python3
uid:            1001

Truncated backtrace:
#1 _really_load in /usr/lib64/python3.6/http/cookiejar.py:2023

Comment 1 David W. Legg 2018-07-14 13:25:25 UTC
Created attachment 1458881 [details]
File: backtrace

Comment 2 David W. Legg 2018-07-14 13:25:26 UTC
Created attachment 1458882 [details]
File: cgroup

Comment 3 David W. Legg 2018-07-14 13:25:27 UTC
Created attachment 1458883 [details]
File: cpuinfo

Comment 4 David W. Legg 2018-07-14 13:25:28 UTC
Created attachment 1458884 [details]
File: environ

Comment 5 David W. Legg 2018-07-14 13:25:30 UTC
Created attachment 1458885 [details]
File: mountinfo

Comment 6 David W. Legg 2018-07-14 13:25:31 UTC
Created attachment 1458886 [details]
File: namespaces

Comment 7 David W. Legg 2018-07-14 13:25:32 UTC
Created attachment 1458887 [details]
File: open_fds

Comment 8 Matěj Cepl 2018-07-16 09:21:10 UTC
Is /home/daddy/.mozilla/firefox/8c3de6f0.default your real profile used by FIrefox (look at about:support)?

Comment 9 David W. Legg 2018-07-16 14:00:15 UTC
Maybe not.  I was just reporting a crash.

Comment 10 Matěj Cepl 2018-07-16 15:00:21 UTC
(In reply to David W. Legg from comment #9)
> Maybe not.  I was just reporting a crash.

I don't understand. What does "Maybe not" mean? Is it your profile in Firefox or not? What happens if you login with this profile (kill all your instances of Firefox and run it again from the terminal as firefox -no-remote -P) to www.itv.com and then rerun the command again?

Comment 11 David W. Legg 2018-07-16 15:13:05 UTC
I checked, and /home/daddy/.mozilla/firefox/8c3de6f0.default is my default real profile.

Comment 12 David W. Legg 2018-07-16 15:16:49 UTC
Ran the command again:

/usr/bin/python3 /usr/bin/youtube-dl --user-agent 'Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:61.0) Gecko/20100101 Firefox/61.0' https://www.itv.com/hub/the-bletchley-circle/2a1939a0004 --cookies
Usage: youtube-dl [OPTIONS] URL [URL...]

youtube-dl: error: --cookies option requires 1 argument
[daddy@arcturus ~]$ /usr/bin/python3 /usr/bin/youtube-dl --user-agent 'Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:61.0) Gecko/20100101 Firefox/61.0' https://www.itv.com/hub/the-bletchley-circle/2a1939a0004 --cookies /home/daddy/.mozilla/firefox/8c3de6f0.default/cookies-1.txt
/usr/lib64/python3.6/http/cookiejar.py:2061: UserWarning: http.cookiejar bug!
Traceback (most recent call last):
  File "/usr/lib64/python3.6/http/cookiejar.py", line 2023, in _really_load
    line.split("\t")
ValueError: not enough values to unpack (expected 7, got 1)

  _warn_unhandled_exception()
Traceback (most recent call last):
  File "/usr/lib64/python3.6/http/cookiejar.py", line 2023, in _really_load
    line.split("\t")
ValueError: not enough values to unpack (expected 7, got 1)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/bin/youtube-dl", line 11, in <module>
    load_entry_point('youtube-dl==2018.5.18', 'console_scripts', 'youtube-dl')()
  File "/usr/lib/python3.6/site-packages/youtube_dl/__init__.py", line 471, in main
    _real_main(argv)
  File "/usr/lib/python3.6/site-packages/youtube_dl/__init__.py", line 438, in _real_main
    with YoutubeDL(ydl_opts) as ydl:
  File "/usr/lib/python3.6/site-packages/youtube_dl/YoutubeDL.py", line 413, in __init__
    self._setup_opener()
  File "/usr/lib/python3.6/site-packages/youtube_dl/YoutubeDL.py", line 2302, in _setup_opener
    self.cookiejar.load()
  File "/usr/lib64/python3.6/http/cookiejar.py", line 1784, in load
    self._really_load(f, filename, ignore_discard, ignore_expires)
  File "/usr/lib64/python3.6/http/cookiejar.py", line 2063, in _really_load
    (filename, line))
http.cookiejar.LoadError: invalid Netscape format cookies file '/home/daddy/.mozilla/firefox/8c3de6f0.default/cookies-1.txt': 'www.lo'

Comment 13 Matěj Cepl 2018-07-16 19:36:41 UTC
(In reply to David W. Legg from comment #12)
>   File "/usr/lib/python3.6/site-packages/youtube_dl/YoutubeDL.py", line 413,
> in __init__
>     self._setup_opener()
>   File "/usr/lib/python3.6/site-packages/youtube_dl/YoutubeDL.py", line
> 2302, in _setup_opener
>     self.cookiejar.load()
>   File "/usr/lib64/python3.6/http/cookiejar.py", line 1784, in load
>     self._really_load(f, filename, ignore_discard, ignore_expires)
>   File "/usr/lib64/python3.6/http/cookiejar.py", line 2063, in _really_load
>     (filename, line))
> http.cookiejar.LoadError: invalid Netscape format cookies file
> '/home/daddy/.mozilla/firefox/8c3de6f0.default/cookies-1.txt': 'www.lo'

Two thoughts:

1. in this part youtube-dl is out of scope of the problem, the bug is in the standard library http.cookiejar. Which seems pretty strong opinion (these modules are usually rather well tested), so my other idea is

2. file /home/daddy/.mozilla/firefox/8c3de6f0.default/cookies-1.txt is somehow corrupted. Moreover, with my Firefox, I don't have cookies*.txt file at all, it is cookies.sqlite. Could you take a look at that file /home/daddy/.mozilla/firefox/8c3de6f0.default/cookies-1.txt with some text editor (gedit, vim, emacs, nano, whatever)? It should be just a plain text file after all.

What happens if you just remove the file (or move it somewhere else)?

Comment 14 David W. Legg 2018-07-17 08:53:25 UTC
The cookies-1.txt file looks like this:

# HTTP Cookie File
# http://www.netscape.com/newsref/std/cookie_spec.html
# This is a generated file!  Do not edit.
# To delete cookies, use the Cookie Manager.

.google.com	TRUE	/	FALSE	1270839741	PREF	ID=325f844d634cc4cc:TB=2:TM=1175859361:LM=1207767740:DV=AA:GM=1:S=maFwj2lf6pIyWnHE
...
etc.

To be honest I would just like youtube-dl to start working again with www.itv.com.  This bug was reported by the automatic crash reporter.  I let it do it because, obviously, the program should not crash at all.

Comment 15 Matěj Cepl 2018-07-17 10:04:54 UTC
(In reply to David W. Legg from comment #14)
> To be honest I would just like youtube-dl to start working again with
> www.itv.com.  This bug was reported by the automatic crash reporter.  I let
> it do it because, obviously, the program should not crash at all.

What about (with Firefox being turned off)

sed -i.bak -e '/^\S\+itv.com/d' -e '/^\S\+www.lo/d' \
    /home/daddy/.mozilla/firefox/8c3de6f0.default/cookies-1.txt

Does it help?

Comment 16 David W. Legg 2018-07-17 14:04:58 UTC
The sed command returns nothing.

Comment 17 Matěj Cepl 2018-07-17 14:38:31 UTC
(In reply to David W. Legg from comment #16)
> The sed command returns nothing.

Sorry, of course, it doesn't. It CHANGES that file by removing all lines for itv.com and www.lo.

Comment 18 Matěj Cepl 2018-07-17 14:38:54 UTC
cookies-1.txt.bak is the original version

Comment 19 Ben Cotton 2018-11-27 14:58:56 UTC
This message is a reminder that Fedora 27 is nearing its end of life.
On 2018-Nov-30  Fedora will stop maintaining and issuing updates for
Fedora 27. It is Fedora's policy to close all bug reports from releases
that are no longer maintained. At that time this bug will be closed as
EOL if it remains open with a Fedora  'version' of '27'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 27 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 20 Ben Cotton 2018-11-30 22:11:37 UTC
Fedora 27 changed to end-of-life (EOL) status on 2018-11-30. Fedora 27 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.


Note You need to log in before you can comment on or make changes to this bug.