Commit Graph

104 Commits

Author SHA1 Message Date
97972d6fa3 Fix like count extraction 2024-01-22 06:35:46 +08:00
5f3b90ad45 Fix channel about tab 2024-01-22 06:29:42 +08:00
51a1693789 Fix comment count extraction due to 'K/M' postfixes
YouTube now displays 2K comments instead of 2359, for instance
2024-01-22 05:59:11 +08:00
d591956baa ylist: show 100 videos per page instead of 20
Also add an option to the internal playlist ctoken function
for filtering out shorts, to be used in future anti-shorts features
2024-01-22 05:21:12 +08:00
Jesus
83af4ab0d7 Fix comment count not extracted sometimes
YouTube created a new key 'commentCount' in addition to 'headerText'
2023-09-11 04:15:25 +08:00
Jesus
5594d017e2 Fix related vids, like_count, playlist sometimes missing
Cause is that some pages have the onResponseReceivedEndpoints key
at the top level with useless stuff in it, and the extract_items
function was searching in that instead of the 'contents' key.

Change to use if blocks instead of elif blocks in the
extract_items function.
2023-09-11 04:13:56 +08:00
Jesus
cb4ceefada Filter out translated audio tracks
See comment in code
2023-09-11 04:06:11 +08:00
Jesus E
cd7624f2cb Set hqdefault thumnail images 2023-06-18 19:45:34 -04:00
Jesus E
21224c8dae watch_extraction.py: fix conditional 2023-06-17 16:25:34 -04:00
Jesus E
0f4bf45cde Fix minor formatting issues 2023-06-17 16:14:59 -04:00
Jesus E
d7f934b7b2 Merge short and video parsing even further
Use multi_get and multi_deep_get for tag differences
Replace the duration check with conservative_update
2023-06-17 16:14:02 -04:00
Jesus E
a4299dc917 Merge short and video parsing 2023-06-17 16:10:59 -04:00
Jesus E
e6fd9b40f4 Fix parsing shorts
Add check for extracting duration for shorts
Make short duration extraction stricter
Fix handling shorts with no views
2023-06-17 16:08:52 -04:00
Jesus E
f322035d4a Add functional but preliminary channel tab support
Add channel tabs to the channel template and script
Update continuation token to request different tabs

Add support for 'reelItemRenderer' format required to extract shorts
2023-06-17 16:05:40 -04:00
Jesus E
74907a8183 Music list extraction: read from SONG field
This one is used when there is no corresponding YouTube video
for the track
2023-05-28 21:45:20 -04:00
Jesus E
aa57ace742 Fix music list extraction
Closes #160
2023-05-28 21:42:13 -04:00
Jesus E
e54596f3e9 Partially fix age restricted videos
Does not work for videos that require decryption because
decryption is not working (giving 403) for some reason.

Related invidious issue for decryption not working:
https://github.com/iv-org/invidious/issues/3245

Partial fix for #146
2023-05-28 21:30:51 -04:00
Jesus E
68752000f0 Update channel to new ctoken format
Huge thanks to @michaelweiser

Different sortings still don't work for videos and playlists
2023-05-28 21:04:36 -04:00
Jesus E
7b60751e99 Fix failure to detect vp9.2 and mp4v.20.3 codecs 2023-05-28 20:47:47 -04:00
Jesus E
9890617098 Fix fmt extraction mime_type regex failure as well as exceptions 2023-05-28 20:44:30 -04:00
Jesus E
0f78f07875 Remove leftover print statement 2023-05-28 20:40:25 -04:00
Jesus E
08545a29df Fix likes count 2023-05-28 20:39:11 -04:00
Jesús
1fbc0cdd46 Fix preview_thumbnails
use 'deep_get' for storyboard
2022-05-30 22:45:08 +08:00
James Taylor
79fd2966cd Extract captions base_url using different method when missing
The base url will be randomly missing.

Take one of the listed captions urls which already
has the &lang and automatic specifiers. Then remove these
specifiers.

Signed-off-by: Jesús <heckyel@hyperbola.info>
2022-03-30 00:41:30 +08:00
James Taylor
dcd4b0f0ae Fix exception when _captions_base_url is not present
Signed-off-by: Jesús <heckyel@hyperbola.info>
2022-03-30 00:37:43 +08:00
zrose584
a5ef801c07 handle missing storyboard
Signed-off-by: Jesús <heckyel@hyperbola.info>
2022-01-17 09:01:09 -05:00
zrose584
63c92e0c4e add preview thumbnails
Signed-off-by: Jesús <heckyel@hyperbola.info>
2022-01-09 16:39:50 -05:00
Jesús
a1d3cc5045 update formats 2021-12-27 13:05:54 -05:00
Jesús
92067638b1 Disable dislikes
Ref: https://blog.youtube/news-and-events/update-to-youtube/
2021-12-26 13:29:55 -05:00
Jesús
f3469b1ff4 Revert "Usage hqdefault thumbnail in related videos"
This reverts commit a0c3ca0159.
2021-09-14 16:35:04 -05:00
Jesús
a0c3ca0159 Usage hqdefault thumbnail in related videos 2021-09-14 15:58:13 -05:00
James Taylor
9c7e93ecf8 Redo av codec settings & selections to accomodate webm
Allows for ranked preferences for h264, av1, and vp9 codecs in
settings, along with equal preferences which are tiebroken using
smaller file size.

For each quality, gives av-merge a list of video sources
and audio sources sorted based on preference & file size. It
will pick the first one that the browser supports.

Closes #84

Signed-off-by: Jesús <heckyel@hyperbola.info>
2021-09-06 16:18:11 -05:00
James Taylor
2ab5b96178 Fix videos added to playlist from channel missing author_id
Signed-off-by: Jesús <heckyel@hyperbola.info>
2021-08-31 18:29:37 -05:00
James Taylor
7c79f530a5 Support more audio and video qualities
Adds support for AV1-encoded videos, which includes any videos
above 1080p. These weren't getting included because they did
not have a quality entry in the format table at the top of
watch_extraction.py. So get the quality from the quality
labels of the format if it's not there.

Because YouTube often includes BOTH AV1 and H.264 (AVC) for each
quality, after these are included, there will be way too many
quality options and the code needs to choose which one to use.
The choice is somewhat hard: AV1 is encoded in fewer bytes than
H.264 and is patent-free, however, it has less hardware support,
so might be more difficult to play. For instance, on my system,
AV1 does not work on 1080p, but H.264 does. Adds a setting about
which to prefer, set to H.264 as the default.

Also adds support for the lower quality mp4 audio quality, which
now gets used at 144p to save network bandwidth. For similar
reasons, this was not getting included because it did not
have an audio_bitrate entry in the table. Prefer bitrate
instead for the quality.

Signed-off-by: Jesús <heckyel@hyperbola.info>
2021-08-31 16:40:19 -05:00
James Taylor
c9a75042d2 Add support for more qualities, merging video+audio using MSE
Signed-off-by: Jesús <heckyel@hyperbola.info>
2021-08-29 18:48:56 -05:00
Jesús
e4af99fd17 Revert "Add support for more qualities, merging video+audio using MSE"
This reverts commit d56df02e7b.
2021-08-29 18:48:01 -05:00
James Taylor
d56df02e7b Add support for more qualities, merging video+audio using MSE
Signed-off-by: Jesús <heckyel@hyperbola.info>
2021-08-29 18:44:26 -05:00
James Taylor
e76257f8b7 Fix comment reply url extraction due to youtube changes
Signed-off-by: Jesús <heckyel@hyperbola.info>
2021-08-23 18:42:16 -05:00
James Taylor
4e556efa3d Fix comments extraction due to new response continuation key name
Signed-off-by: Jesús <heckyel@hyperbola.info>
2021-08-23 18:40:52 -05:00
James Taylor
40fcee52c0 Fix description extraction in search results
Signed-off-by: Jesús <heckyel@hyperbola.info>
2021-08-09 12:29:01 -05:00
James Taylor
2039972ab3 Fix (dis)like, music list extraction due to YouTube changes (again)
YouTube reverted the changes they made that prompted f9f5d5ba.

In case they change their minds again, this adds support for both
formats.

The liberal_update and conservative_update functions needed to be
modified to handle the cases of empty lists, so that
a successfully extracted 'music_list': [{'Author':...},...] will
not be overwritten by 'music_list': [] in the calls to
liberal_dict_update.

Signed-off-by: Jesús <heckyel@hyperbola.info>
2021-08-09 12:13:52 -05:00
James Taylor
3dee7ea0d1 Switch to new comments api now that old one is being disabled
watch_comment api periodically gives the error "Top level
comments mweb servlet is turned down."

The continuation items for the new api are in a different
arrangement in the json, so changes were necessary to the
extract_items function.

Signed-off-by: Jesús <heckyel@hyperbola.info>
2021-08-09 12:10:42 -05:00
James Taylor
f27105fa7f New age restriction bypass method since get_video_info was disabled
From
https://github.com/yt-dlp/yt-dlp/issues/574#issuecomment-887171136

Signed-off-by: Jesús <heckyel@hyperbola.info>
2021-07-28 23:48:54 -05:00
James Taylor
54b39f1303 Fix missing likes, dislikes, & music list due to Youtube changes
Also moves some microformat extraction from
_extract_watch_info_mobile to extract_watch_info where it belongs.
_extract_watch_info_mobile is really only for stuff visible on the
page, and thus specialized for either mobile or desktop.

Signed-off-by: Jesús <heckyel@hyperbola.info>
2021-07-28 23:47:41 -05:00
Jesús
7fd2c3474f Capitalize name app 2021-06-10 16:41:45 -05:00
James Taylor
a9edc54aac Use extract_approx_int for comment likes
Full digits no longer available

Closes #64

Signed-off-by: Jesús <heckyel@hyperbola.info>
2021-06-10 12:47:09 -05:00
James Taylor
e44647f106 Fix comment like extraction due to Youtube changes
Variable name changed from likeCount to voteCount

Signed-off-by: Jesús <heckyel@hyperbola.info>
2021-05-17 22:08:09 -05:00
James Taylor
f0cd170767 Fix videos added to playlist from channel page not having author
Information from additional_info was being overrided with None.

Signed-off-by: Jesús <heckyel@hyperbola.info>
2021-05-17 22:02:03 -05:00
James Taylor
0ee448c186 Channel about: Add http:// to links without it
So that the link is not interpretted as a relative link

Signed-off-by: Jesús <heckyel@hyperbola.info>
2021-05-06 15:21:36 -05:00
James Taylor
e549b5f67c Channel: Allow going to next pages of playlists page
Uses previous and next buttons. Now can view more than just
first page of playlists page

Signed-off-by: Jesús <heckyel@hyperbola.info>
2021-03-15 22:22:15 -05:00