26 Commits

Author SHA1 Message Date
James Taylor
3a081a9c46 Automatically change tor circuit once if ip is blocked
Use stem library to send a new identity signal via the tor
control port.

See #20
2020-10-25 11:15:59 -07:00
James Taylor
bcaec7b7d3 fetch_url: move content decoding to before 429 check
Content should be decoded before we try reading page to check for
429. Lucky that it worked at all before.
2020-10-25 10:54:15 -07:00
James Taylor
5f4884dce8 Put vid title at end of download urls so downloads w/ that filename 2020-10-22 14:30:33 -07:00
zrose584
4f46e14ad0 fix style 2020-10-09 18:25:17 +02:00
zrose584
f8d9be2d5a settings.py: add 'proxy_images' 2020-10-07 19:28:52 +02:00
James Taylor
5f5034e826 Increase max redirects for video routing 2020-09-22 20:59:20 -07:00
user938120
3ffe5d0120 Add setting for tor_port 2020-09-19 12:23:30 +00:00
James Taylor
e9989af03a Add tor video routing
Includes non-tor video routing by default, so no more chances
of the browser leaking headers or user agent to googlevideo
Adjust settings upgrade system to facilitate change to route_tor
setting.
Add some more space on settings page for dropdown settings so does
not overflow due to options with long names.
Closes #7
2020-09-18 14:40:21 -07:00
James Taylor
bd255a9ab0 Don't use brotli if import fails 2020-08-11 21:57:40 -07:00
James Taylor
56e7751da7 Fix failure to parse comments when there's one from deleted channel
Specifically, fix failures when any of the fields from the parsed
comment are None, such as author, author_url, etc.
(failure due to string concatenation when building urls).
2020-03-08 16:17:04 -07:00
James Taylor
0989d55db1 Ensure urllib3 and python version not leaked during video file check
This is likely not a big deal since it is already assumed that video file server logs are not plugged into
Google's tracking infrastructure, but it doesn't hurt to give less info.
2020-02-18 22:17:50 -08:00
James Taylor
f253247949 Fix MaxRetryError when checking for video URL access
The default urllib3 max redirect amount was set to 3. Change it to 10 and
do not fail if there is a problem with checking for URL access. Just print
the error to the console and proceed.

Also add an unrelated remark about the bcptr=9999999999 parameter in watch.py
2020-02-18 11:47:23 -08:00
James Taylor
7c2736aa26 Check for 403 errors and fallback on Invidious
403 errors on the video urls happen typically when a video has copyrighted content or was livestreamed originally. They appear to not happen (or at least happen less frequently) if the Tor exit node used ipv6, however.
2020-02-01 15:09:37 -08:00
James Taylor
f787e4e202 Give a proper error message for 429 errors
These occur when too many requests are coming from a Tor exit node.
Before, there would be an error page with an exception instructing users to report the issue.
But this is an expected and persistent issue.
2020-01-31 20:06:15 -08:00
James Taylor
d1d908d5b1 Extraction: Move html post processing stuff from yt_data_extract to util 2019-12-19 19:48:53 -08:00
James Taylor
98777ee825 Extraction: Rewrite item_extraction for better error handling and readability, rename extracted names for more consistency 2019-12-18 19:39:16 -08:00
James Taylor
4c07546e7a Extraction: Replace youtube-dl with custom-built watch page extraction 2019-10-17 19:58:13 -07:00
James Taylor
61c50e0b54 Extraction: Move comment extraction to yt_data_extract 2019-09-19 11:41:16 -07:00
James Taylor
fb1a3531c5 Extraction: Fix url prefixing 2019-09-08 17:20:02 -07:00
James Taylor
2e75c6d960 Merge flask framework and other stuff from master 2019-08-09 22:01:04 -07:00
James Taylor
e00c3cf99f Remove ad-hoc response saving from code, create a debug setting for fetch_url 2019-07-23 23:57:45 -07:00
James Taylor
27ee2990e9 Overhaul refresh system, make it asynchronous 2019-06-09 16:03:20 -07:00
James Taylor
ae5fd9eb00 Make thumbnails work and other stuff 2019-06-05 00:41:15 -07:00
James Taylor
89daa27be2 prevent urllib3 from decoding content 2019-02-26 17:15:51 -08:00
James Taylor
1e1f55c9e5 Use persistent connections 2019-02-25 20:47:49 -08:00
James Taylor
b32330be4f refactor common.py into 3 files 2019-02-21 21:32:31 -08:00