Bug 1187934 - Feature Request: Please revert back to one file youtube-dl mode
Summary: Feature Request: Please revert back to one file youtube-dl mode
Keywords:
Status: CLOSED UPSTREAM
Alias: None
Product: Fedora
Classification: Fedora
Component: youtube-dl
Version: 20
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Till Maas
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-01-31 14:29 UTC by Ali Akcaagac
Modified: 2015-02-10 13:02 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-02-10 13:02:23 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Ali Akcaagac 2015-01-31 14:29:40 UTC
Please revert back to youtube-dl one file mode and the possibility to youtube-dl -U anytime I like.

Rasons why the current change is bad:

It takes days if not weeks once a newer version of youtube-dl as package shows up (and then the contents is mostly old again). I - and probably many others too - make have usage of the youtube-dl -U feature whenever its required. Some sites such as youtube and others change far to quickly to keep track of RPM packages. So a youtube-dl -U often already provides the updates.

The split up mess in the python directory was kinda unnecessary. youtube-dl -U doesn't work anymore either. It explicitly tells you to wait for packages.

Thanks for having communicated this with your userbase before the changes.

Comment 1 nucleo 2015-01-31 16:28:55 UTC
> All program binaries and program libraries included in Fedora packages must be built from the source code that is included in the source package. 
http://fedoraproject.org/wiki/Packaging:Guidelines#No_inclusion_of_pre-built_binaries_or_libraries

More frequent updates possible if install youtube_dl with pip which also provides upgrades or with manual installing and updating youtube-dl binary.

Comment 2 Michael Schwendt 2015-01-31 17:54:20 UTC
Applying updates via "youtube-dl -U" and getting out-of-sync with the distributions package management gives reason to be worried. If you don't want a distributor's official (and possibly customised and tested) updates for a package, just don't install youtube-dl as a package. Download it manually and run it from somewhere in /usr/local, /opt or $HOME. Then you're entirely on your own and don't need to complain about what the distributor offers.

That's IMHO.

Same applies to other software, such as Perl & Python Modules, overriding C/C++ shared system libraries, or turning on Firefox Auto Updates.

Btw, bugzilla seldomly is a place for a longer discussion (such as bug 1176454). Prefer devel@ list: https://lists.fedoraproject.org/mailman/listinfo/devel

Comment 3 Till Maas 2015-01-31 23:04:12 UTC
(In reply to Ali Akcaagac from comment #0)

> Thanks for having communicated this with your userbase before the changes.

How can I reach the youtube-dl userbase to discuss this?

Comment 4 Christopher Meng 2015-02-08 04:22:34 UTC
Nucleo you've done a really nice sh*t to my packages, congratulation. And next time please don't mention any words of archlinux, I DON'T CARE ABOUT ARCHLINUX FROM THE DAY I JOINED FEDORA, AND I DON'T LIKE THEIR SIMPLE AND SO CALLED AGILE WAY.

Till have you asked any other maintainers or users before applying any patch from any people?

I didn't want to inflict some weird behaviors on users of my packages, but you all did a nice job.

The reason of using one file is pretty obvious, as under the current way users can't use -U option.

I should tell you that packaging work for some packages is pretty hard, the best example is youtube-dl, I've kept updating this for nearly one year from 2013, but I still can't fully catch up with upstream as they do a rapid releasing.

Another package of mine is bleachbit, it also includes updates notification function, fortunately I can disable it during building because bleachbit often releases one update in months, I have enough time to update it.

So in this case, youtube-dl users should be able to update by themselves. I was tired then I only updated it once a week/2 weeks in mid-2014. And I'D ALREADY KNOW THAT -U WON'T WORK IF I CHANGED THE SPEC.

I'm gonna orphan this package since people in Fedora doesn't repect opinions of others, and conflicts you are bringing.

Comment 5 Till Maas 2015-02-08 06:45:46 UTC
(In reply to Christopher Meng from comment #4)
> Nucleo you've done a really nice sh*t to my packages, congratulation. And
> next time please don't mention any words of archlinux, I DON'T CARE ABOUT
> ARCHLINUX FROM THE DAY I JOINED FEDORA, AND I DON'T LIKE THEIR SIMPLE AND SO
> CALLED AGILE WAY.

Christopher, please watch your language. If you disagree with something, you can say it politely.

> Till have you asked any other maintainers or users before applying any patch
> from any people?

The bug was open for a month, so anyone was able to comment to it, including you. Also I was pointed to this behaviour by someone else.
 
> I didn't want to inflict some weird behaviors on users of my packages, but
> you all did a nice job.
> 
> The reason of using one file is pretty obvious, as under the current way
> users can't use -U option.

From what I understand, the only reason it does not work, is because there is an if statement preventing this, not a technical reason. However, if only impolite people would like this to be working, there is little motivation for anyone to do this.

How much time did you spend debugging bugs in youtube-dl? I do not remember any comment from you on the various abrt bugs here in bugzilla. Having a properly install python youtube-dl makes debugging easier.

> I should tell you that packaging work for some packages is pretty hard, the
> best example is youtube-dl, I've kept updating this for nearly one year from
> 2013, but I still can't fully catch up with upstream as they do a rapid
> releasing.

Packaging is more than just updating the file. Also, do you realize that you did not update youtube-dl for more than 5 months now? Btw. the usual behaviour is to thank the co-maintainers for taking care of packages, especially if you just disappear.

> So in this case, youtube-dl users should be able to update by themselves. I
> was tired then I only updated it once a week/2 weeks in mid-2014. And I'D
> ALREADY KNOW THAT -U WON'T WORK IF I CHANGED THE SPEC.

So, why did you keep this knowlegde to yourself, given that the bug report with the patch was open for a month?

> I'm gonna orphan this package since people in Fedora doesn't repect opinions
> of others, and conflicts you are bringing.

This is completely exaggerated. The bug reporter was asked to bring this topic to fedora-devel, so we can discuss it properly. This is exactly the opposite of ignoring opinions. It all happened in a friendly way by the people supporting the change. Only you and the bug reporter used offensive language, which is something you won't find acceptance in the Fedora community.

However, if you cannot co-operate without being offensive, orphaning packages is the right way to move forward, but this needs to be done in package db:
https://admin.fedoraproject.org/pkgdb/package/youtube-dl

If you like, I can take care of this your you.

Comment 6 Michael Schwendt 2015-02-08 11:37:00 UTC
Comments from just a sporadic user of youtube-dl:


> I should tell you that packaging work for some packages is pretty hard,
> the best example is youtube-dl,

Define "hard". If you spent time on

 - debugging the reported issues
 - forwarding issues and/or patches upstream

it would be even "harder":

  https://fedoraproject.org/wiki/Package_maintainer_responsibilities

The possibility to modify the Python source files directly can benefit trouble-shooting attempts.

Trying to update the package weekly (or even more frequently) asks for automation. You cannot automate the testing, however, where you depend on tester feedback unless you push to stable without feedback. And it's highly questionable anyway whether there needs to be frequent updates that don't fix any open issues reported at Fedora and even may introduce new issues instead. The package does not include any NEWS file or changelog document that sums up what has changed in a new release. The Fedora update tickets don't give any hint either.

Currently, there are 980 open issues listed at github and 2,990 closed ones, so even the decision _when_ to update to _which release_ may be a difficult one. Too frequent updates could get annoying for the users, who are affected by a bug that is not being acted on by Fedora maintainers.


> I've kept updating this for nearly one year from 2013, but I still
> can't fully catch up with upstream as they do a rapid releasing.

The Fedora Build System shows only activity from other co-maintainers for many months:

  http://koji.fedoraproject.org/koji/packageinfo?packageID=5677

Same for updates:

  https://admin.fedoraproject.org/updates/search/youtube-dl

It seems there has been a bit of co-maintenance, but only a very few issues have been reported by the current maintainers:

  https://github.com/rg3/youtube-dl/issues/created_by/cicku
  https://github.com/rg3/youtube-dl/issues/created_by/mcepl
  https://github.com/rg3/youtube-dl/issues/created_by/rahulsundaram
  https://github.com/rg3/youtube-dl/issues/created_by/tyll

So, any effort spent on squashing the bugs, which reportedly affect users of the Fedora packages, could be a good thing. Installing individual source files instead of a single zip archive sounds reasonable and helpful.

For example, especially with scripting languages it can be convenient to ask a bug reporter to apply a test patch file directly instead of having to provide a scratch-build of a full package. If, however, you never talk to your bug reporters, you would not have any interest in activity like that.

Comment 7 Ali Akcaagac 2015-02-08 12:20:50 UTC
I don't want to get involved into politics here because I'm more concerned about the sofware itself. I also don't intend to spent time on a developers mailinglist discussing this, while the patch has already been swallowed. A conversation would have made sense before applying of any patches.

For me as person I found a "solution" - at least for Fedora 20 - I downgraded to the last "onefile" package and put an exclude=youtube-dl in the yum.conf file. The first and hopefully only package requiring this drastic decision.

The thing why I opened this bugreport and left some comments on ...

https://bugzilla.redhat.com/show_bug.cgi?id=1176454

... was because of the nature of this software.

I understand the Fedora politics and I do agree that software inside Fedora should be able to be compiled, to be able to be reviewed, to be able to be dealt with in case of bugs.

Unfortunately this single "package" is a special sort of nature. You - the people of Fedora - can't deal with all the bugreports it is getting and please allow me to explain:

This software is based on either API provided by websites such as maybe "youtube" or on hacks (grabbing content out of downloaded html text pages). The amount of pages and content offerers are quite huge.

-bash-4.2$ youtube-dl --list-extractors | wc -l
553
-bash-4.2$ 

553 pages as of the current last version I have. All these websites are permanently changed and you might receive plenty of bugreports here at bugzilla where people complain about XYZ webpage not working anymore.

With the recent changes people are "forced" and "bound" to depend on Fedora packages - whenever they show up. This means. People probably already have an old package (the last one from fedora is 25-01-2015 while the current one in upstream ist 06-02-2015). This means 12 days of changes since last youtube-dl release. During this time there might have been plenty of changes and updates within this software and/or websites (who changed things).

Now say I want to download a file from vimeo today with the version from 25-01-2015 and it fails. I come here report a bug. 10 other people do so as well.

So how does your bugreport and debugging will look like ?

Most likely you won't spent a second debugging anything. You might pack the upstream version in hope that everything works as expected again. The time difference between the reporting of the bug and releasing a new package might cause plenty of changes in other areas again. This cause a permanent catching up game here.

This is no software like "picard" or "calibre" or some other python package which you can offer as package and leave alone for weeks. This package here probably requires a high frequency of dealing around. A quite common and wide used package that needs high attention. The time you release a new package for fedora makes the package old again, because meanwhile new changes happened (not just the software but also on websites offering content).

The debugging argument is something I see with a smiling face here (with one eye closed). Using fedora for quite a few years I can look back and say that plenty of packages haven't seen the "debugging" and "caring" as they require. Where new patches made things more worse than the package originally was (when it was introduced to fedora or released with the stable fedora release). I remember where cups received changes (on a stable version of fedora) rendering printing for me useless or where the "sort" command received some "love", which caused sorting to end up in a fiasco (wrong sorting: a few days of my work had to be reviewed). I just wanted to throw the last few sentences in here with a smile.

Comment 8 Ali Akcaagac 2015-02-08 12:29:55 UTC
An alternative approach would be to download the sources for youtube-dl, review, debug and build them (as explained in the releases document) and offer it as "onefile" again (as it's intended to be) leaving the -U option on demand for people who like to update on their own.

Comment 9 nucleo 2015-02-08 12:32:03 UTC
(In reply to Christopher Meng from comment #4)
> Nucleo you've done a really nice sh*t to my packages, congratulation. And
> next time please don't mention any words of archlinux, I DON'T CARE ABOUT
> ARCHLINUX FROM THE DAY I JOINED FEDORA, AND I DON'T LIKE THEIR SIMPLE AND SO
> CALLED AGILE WAY.
> 
> Till have you asked any other maintainers or users before applying any patch
> from any people?
> 
> I didn't want to inflict some weird behaviors on users of my packages, but
> you all did a nice job.
> 
> The reason of using one file is pretty obvious, as under the current way
> users can't use -U option.
> 
> I should tell you that packaging work for some packages is pretty hard, the
> best example is youtube-dl, I've kept updating this for nearly one year from
> 2013, but I still can't fully catch up with upstream as they do a rapid
> releasing.
> 
> Another package of mine is bleachbit, it also includes updates notification
> function, fortunately I can disable it during building because bleachbit
> often releases one update in months, I have enough time to update it.
> 
> So in this case, youtube-dl users should be able to update by themselves. I
> was tired then I only updated it once a week/2 weeks in mid-2014. And I'D
> ALREADY KNOW THAT -U WON'T WORK IF I CHANGED THE SPEC.
> 
> I'm gonna orphan this package since people in Fedora doesn't repect opinions
> of others, and conflicts you are bringing.

This sh*t is exactly what youtube-dl upstream recommends 
https://github.com/rg3/youtube-dl/blob/master/README.md#faq

> How do I update youtube-dl?
> If you've followed our manual installation instructions, you can simply run youtube-dl -U (or, on Linux, sudo youtube-dl -U).

So this update method is for manual installations.

> If you have installed youtube-dl using a package manager like apt-get or yum, use the standard system update mechanism to update. 

> As a last resort, you can also uninstall the version installed by your package manager and follow our manual installation instructions.

Comment 10 Michael Schwendt 2015-02-08 13:58:08 UTC
Re: comment 7

> I also don't intend to spent time on a developers mailinglist
> discussing this, 

Still that would be the only way to raise awareness of what you consider to be a problem.


> A conversation would have made sense before applying of any patches.

In case of disagreement, possibly. For some packages at Fedora, co-maintainers stay in contact easily via an email group alias that exists for all packages. For other packages, the used communication channel is different.

Btw, it would still be possible to revert a patch. ;-)


> You - the people of Fedora - can't deal with all the bugreports
> it is getting

I had opened an upstream youtube-dl ticket back in 2013, and still haven't had any response in the ticket. That's not encouraging either.

At  http://bugz.fedoraproject.org/youtube-dl  there are only 16 open tickets currently. That's not much. Impossible to draw further conclusions based on that. Just: There are users of Fedora affected by those bugs - crashes even - and the open tickets don't give any hint whether any of the frequent updates have fixed the bugs. => It could be that the frequently updated software still fails for those users.


> With the recent changes people are "forced" and "bound" to depend on Fedora
> packages - whenever they show up.

Nobody is being "forced" in any way.


> This means. People probably already have an old package (the last one from
> fedora is 25-01-2015 while the current one in upstream ist 06-02-2015). 

The "old" one may work good enough for the average user. Nowhere is written that it must be the latest.


> I come here report a bug. 10 other people do so as well.

The opposite could also happen. 10 users running -U end up with an update that doesn't work on Fedora anymore either due to regression or need of updating dependencies.

Discussing it like that doesn't lead anywhere.


> The debugging argument is something I see with a smiling face here (with
> one eye closed). Using fedora for quite a few years I can look back and
> say that plenty of packages haven't seen the "debugging" and "caring"
> as they require. 

Are specific bugzilla ticket numbers available for that?
Have you asked/told anyone other than the ticket assigne?

Other than that, it is no secret that some packages are not well-maintained, especially not in bugzilla. More maintainers and testers (in particular during the development and testing period of the distribution) could be beneficial. Raising awareness of poorly maintained packages could be helpful, too. Only complaining here about unrelated packages doesn't lead anywhere either.


> I just wanted to throw the last few sentences in here with a smile.

Throw as much as you like, even if it's not productive.

Comment 11 Till Maas 2015-02-10 13:02:23 UTC
I recommend that you ask upstream to enable -U for youtube for multi-file youtube files as well. But I do not see why you want to have both updates from Fedora and upstream, because this only leads to conflicts, e.g. when you ran youtube-dl -U and an older update is moved from testing to updates, it will overwrite your manual update.

Also you are still welcomed to discuss this on the devel list, but I do not see any value in keeping this bug open.


Note You need to log in before you can comment on or make changes to this bug.