35 Commits

Author SHA1 Message Date
e4af99fd17
Revert "Add support for more qualities, merging video+audio using MSE"
This reverts commit d56df02e7b1eba86baf511289208295b1f6c5a50.
2021-08-29 18:48:01 -05:00
James Taylor
d56df02e7b
Add support for more qualities, merging video+audio using MSE
Signed-off-by: Jesús <heckyel@hyperbola.info>
2021-08-29 18:44:26 -05:00
James Taylor
2039972ab3
Fix (dis)like, music list extraction due to YouTube changes (again)
YouTube reverted the changes they made that prompted f9f5d5ba.

In case they change their minds again, this adds support for both
formats.

The liberal_update and conservative_update functions needed to be
modified to handle the cases of empty lists, so that
a successfully extracted 'music_list': [{'Author':...},...] will
not be overwritten by 'music_list': [] in the calls to
liberal_dict_update.

Signed-off-by: Jesús <heckyel@hyperbola.info>
2021-08-09 12:13:52 -05:00
James Taylor
f27105fa7f
New age restriction bypass method since get_video_info was disabled
From
https://github.com/yt-dlp/yt-dlp/issues/574#issuecomment-887171136

Signed-off-by: Jesús <heckyel@hyperbola.info>
2021-07-28 23:48:54 -05:00
James Taylor
54b39f1303
Fix missing likes, dislikes, & music list due to Youtube changes
Also moves some microformat extraction from
_extract_watch_info_mobile to extract_watch_info where it belongs.
_extract_watch_info_mobile is really only for stuff visible on the
page, and thus specialized for either mobile or desktop.

Signed-off-by: Jesús <heckyel@hyperbola.info>
2021-07-28 23:47:41 -05:00
7fd2c3474f
Capitalize name app 2021-06-10 16:41:45 -05:00
James Taylor
31fe1dac55
Fix signature decryption due to new base.js minifier rules
YouTube now includes e.g. {"fe": ...} instead of just {fe: ...}
in the javascript object entries in the object holding the
operation definitions.

Fixes #2

Signed-off-by: Jesús <heckyel@hyperbola.info>
2021-02-23 17:17:06 -05:00
James Taylor
5edcaa4f9d
Improve ytInitialPlayerResponse extraction
Makes it work if there are additional javascripts statements
after the playerResponse variable

Signed-off-by: Jesús <heckyel@hyperbola.info>
2020-12-17 11:00:04 -05:00
James Taylor
9d0be82e74 Always extract from html watch page to get base.js url
Youtube removed the url from the pbj responses. They are now
only in the html page. Replaces previous fix for the missing
base.js issue.
2020-12-12 23:11:54 -08:00
James Taylor
6443cedf62 Retrieve base.js url from html watch page when it's missing
Fixes failure mode 3 in #22
2020-12-09 17:08:12 -08:00
James Taylor
0589cfb8f7 yt_data_ext: watch playlist: Fix missing author_url if no author_id
Embedded playlist info was missing author_url key if author_id was
None. This caused KeyError in watch.py when it expected that key

Closes #37
2020-11-08 10:08:20 -08:00
James Taylor
f8b6db1480 Redo fix for failure mode 1 in issue #22
Previous fix didn't work. Should work now. The non-embedded player
response can still be present but the urls will be missing.
2020-10-21 22:42:07 -07:00
zrose584
a27b575380 remove trailing whitespaces 2020-10-21 10:35:01 +02:00
James Taylor
c9d0f685a4 Use get_video_info to get video urls if player response missing
Fixes failure mode 1 in #22
2020-10-19 13:53:57 -07:00
James Taylor
20152a6316 Specify video height in html so page doesn't shift down after load
Use true video height extracted from youtube to handle videos
shorter than their quality size. (e.g. widescreen videos)
2020-09-24 18:50:54 -07:00
James Taylor
803c901445 Fix hls_manifest_url not included when there's no other formats
Since there are no formats, it was retrying with the
non-embedded playerResponse, which resulted in the
hls_manifest_urls from the embedded player_response being
overwritten with None. So use conservative_update instead
2020-06-28 18:18:04 -07:00
James Taylor
aa3e5aa441 Add dialog for copying urls to external player for livestreams
Also for livestreams which are over whose other sources
aren't present or aren't ready yet.
2020-06-28 17:52:24 -07:00
James Taylor
6e14a8547d Handle case where embedded player response missing
Change so it extracts other stuff from regular playerResponse
Extract formats from embedded player response, but fallback to
regular one if that doesn't work.
Sometimes there is no 'player' at top_level and the urls are in
the regular playerResponse
2020-06-28 13:18:54 -07:00
James Taylor
0b5d6fe1ed Do not override previous playability error if unknown 2020-06-28 12:46:04 -07:00
James Taylor
b4450ec4bb Fix previously live videos labeled as live 2020-05-29 15:34:33 -07:00
James Taylor
bdac6a2302 Fix broken signature decryption
The base.js url format changed, so the identifier at the end
was no longer unique. So it was using the wrong cached decryption
function

Changes the identifier to just be the whole url so
this won't happen again.
2020-05-27 12:15:41 -07:00
James Taylor
85db7e46ed Fix urls sometimes not extracted due to youtube changes
The 'cipher' parameter which contains the url is sometimes called
'signatureCipher' instead now.
2020-05-27 11:56:30 -07:00
James Taylor
f1f77c4d77 Fix error getting exit node ip if format urls are None 2020-05-27 11:14:52 -07:00
James Taylor
b2f482f1fb Fix comment count & disabled extraction not working sometimes
because of A/B test.
2020-04-10 13:57:11 -07:00
James Taylor
3e09193eaf Fix exception due to missing 'playlist' key in extracted info
Happens when there's an error on the page and there was no
visible stuff on the page. 'playlist' wasn't set to None in that
case.
2020-04-05 17:27:43 -07:00
James Taylor
4d9d8cec6f Fix error when there's a video format with mimetype class of 'text' 2020-04-04 22:53:49 -07:00
James Taylor
5554d5afff Add playlist sidebar for videos in playlist, including autoplay 2020-04-04 22:52:09 -07:00
James Taylor
8c2b81094e yt_data_extract: fix missing variables in info for unavailable videos
'ip_address' was not set when no formats are available
'allowed_countries' was set to None rather than [] in extract_desktop_info which it turns out is the function that gets used in these cases
2020-02-17 20:15:59 -08:00
James Taylor
9f090dbbf8 Watch page: add info box with allowed countries and tor exit node
Should help with debugging various content blocks
2020-02-01 16:16:49 -08:00
James Taylor
e364927f83 yt_data_extract: parse mimeType field for codecs
the youtube-dl formats table doesn't have all the necessary information
2020-02-01 14:23:50 -08:00
James Taylor
b2a1f4ecfb Fix signature decryption.
The function body regex was capturing some unrelated new code before the actual function body. Example:

`function(a){a=a.split("");var b=[function(c,d){d=(d%c.length+c.length)%c.length;c.splice(-d).reverse().forEach(function(e){return c.unshift(e)}`

If you look closely, the closing bracket doesn't match the opening one. I have added `{` to the `[^\}]+` part to make sure it only captures matching brackets. Additionally, I've added `return a\.join\(""\)` to the end for good measure.
2020-01-24 14:11:59 -08:00
James Taylor
a9f67d4630 Fix regression: date extraction broken. Move constants to correct file in yt_data_extract 2019-12-20 18:48:40 -08:00
James Taylor
6b7a1212e3 Extraction: Move non-stateful signature decryption functionality into yt_data_extract 2019-12-19 21:28:21 -08:00
James Taylor
4a3529df95 Extraction: Move stuff around in files and put underscores in front of internal helper function names
Move get_captions_url in watch_extraction to bottom next to other exported, public functions
2019-12-19 20:12:37 -08:00
James Taylor
76376b29a0 Extraction: Split yt_data_extract.py into multiple files 2019-12-19 19:29:47 -08:00