The durations were previously set to "LIVE", "UPCOMING", etc. and
would not be updated once the livestream was over or the upcoming
video was published.
E.g. line 548, AttributeError: 'NoneType' object has no attribute 'lower'
When upload dates are unavailable, make ones up which give the
correct video order
YouTube disabled the browse_ajax endpoint. Quick fix for first
channel page. In general, this will be more robust against api
endpoint changes. As YouTube continues to change api endpoints,
at least the subscriptions and first page will no longer break.
Signed-off-by: Jesús <heckyel@hyperbola.info>
It would be 30 since the old method looked to see where the latest
video in the database is in the new batch of videos. New method
finds the first video in the new batch which is in the database.
The try statement was missing an except clause. So if there
was an exception such as the tor browser being closed or
getting a 429 error during the request, the workers would
crash until there were none left to handle checking.
These occur when too many requests are coming from a Tor exit node.
Before, there would be an error page with an exception instructing users to report the issue.
But this is an expected and persistent issue.
Commits in this branch are prefixed with "Extraction:"
This branch refactors data extraction. All such functionality has been moved to the yt_data_extract module.
Responses from requests are given to the module and it parses them into a consistent, more useful format.
The dependency on youtube-dl has also been dropped and this functionality has been built from scratch for these reasons:
(1) I've noticed youtube-dl breaks more often than invidious (which uses watch page extraction built from scratch) in response to changes from Youtube, so I'm hoping what I wrote will also be less brittle.
(2) Such breakage is inconvenient because I have to manually merge the fixes since I had to make changes to youtube-dl to make it do things such as extracting related videos.
(3) I have no control over error handling and request pooling with youtube-dl, since it does all the requests (these would require intrusive changes I don't want to maintain).
(4) I will now be able to finally display the number of comments and whether comments are disabled without making additional requests.
