45 Commits

Author SHA1 Message Date
f629565e77 bump to v0.4.1
All checks were successful
git-sync-with-mirror / git-sync (push) Successful in 13s
CI / test (push) Successful in 48s
2026-03-22 21:27:50 -05:00
1f8c13adff feat: improve 429 handling with Tor support and clean CI
All checks were successful
git-sync-with-mirror / git-sync (push) Successful in 11s
CI / test (push) Successful in 50s
- Retry with new Tor identity on 429
- Improve error logging
- Remove .build.yml and .drone.yml
2026-03-22 21:25:57 -05:00
6a68f06645 Release v0.4.0 - HD Thumbnails, YouTube 2024+ Support, and yt-dlp Integration
Some checks failed
CI / test (push) Failing after 1m19s
Major Features:
- HD video thumbnails (hq720.jpg) with automatic fallback to lower qualities
- HD channel avatars (240x240 instead of 88x88)
- YouTube 2024+ lockupViewModel support for channel playlists
- youtubei/v1/browse API integration for channel playlist tabs
- yt-dlp integration for multi-language audio and subtitles

Bug Fixes:
- Fixed undefined `abort` import in playlist.py
- Fixed undefined functions in proto.py (encode_varint, bytes_to_hex, succinct_encode)
- Fixed missing `traceback` import in proto_debug.py
- Fixed blurry playlist thumbnails using default.jpg instead of HD versions
- Fixed channel playlists page using deprecated pbj=1 format

Improvements:
- Automatic thumbnail fallback system (hq720 → sddefault → hqdefault → mqdefault → default)
- JavaScript thumbnail_fallback() handler for 404 errors
- Better thumbnail quality across all pages (watch, channel, playlist, subscriptions)
- Consistent HD avatar display for all channel items
- Settings system automatically adds new settings without breaking user config

Files Modified:
- youtube/watch.py - HD thumbnails for related videos and playlist items
- youtube/channel.py - HD thumbnails for channel playlists, youtubei API integration
- youtube/playlist.py - HD thumbnails, fixed abort import
- youtube/util.py - HD thumbnail URLs, avatar HD upgrade, prefix_url improvements
- youtube/comments.py - HD video thumbnail
- youtube/subscriptions.py - HD thumbnails, fixed abort import
- youtube/yt_data_extract/common.py - lockupViewModel support, extract_lockup_view_model_info()
- youtube/yt_data_extract/everything_else.py - HD playlist thumbnails
- youtube/proto.py - Fixed undefined function references
- youtube/proto_debug.py - Added traceback import
- youtube/static/js/common.js - thumbnail_fallback() handler
- youtube/templates/*.html - Added onerror handlers for thumbnail fallback
- youtube/version.py - Bump to v0.4.0

Technical Details:
- All thumbnail URLs now use hq720.jpg (1280x720) when available
- Fallback handled client-side via JavaScript onerror handler
- Server-side avatar upgrade via regex in util.prefix_url()
- lockupViewModel parser extracts contentType, metadata, and first_video_id
- Channel playlist tabs now use youtubei/v1/browse instead of deprecated pbj=1
- Settings version system ensures backward compatibility
2026-03-22 20:50:03 -05:00
84e1acaab8 yt-dlp 2026-03-22 14:17:23 -05:00
Jesus
ed4b05d9b6 Bump version to v0.3.2 2025-03-08 16:41:58 -05:00
Jesus
6f88b1cec6 Refactor extract_info in watch.py to improve client flexibility
Introduce primary_client, fallback_client, and last_resort_client variables for better configurability.
Replace hardcoded 'android_vr' with primary_client in fetch_player_response call.
2025-03-08 16:40:51 -05:00
Jesus
03451fb8ae fix: prevent error when closing avMerge if not a function 2025-03-08 16:39:37 -05:00
Jesus
e45c3fd48b Add styles error in player 2025-03-08 16:38:31 -05:00
Jesus
1153ac8f24 Fix NoneType inside comments.py
Bug:

Traceback (most recent call last):
  File "/home/rusian/yt-local/youtube/comments.py", line 180, in video_comments
    post_process_comments_info(comments_info)
  File "/home/rusian/yt-local/youtube/comments.py", line 81, in post_process_comments_info
    comment['author'] = strip_non_ascii(comment['author'])
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/rusian/yt-local/youtube/util.py", line 843, in strip_non_ascii
    stripped = (c for c in string if 0 < ord(c) < 127)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: 'NoneType' object is not iterable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "src/gevent/greenlet.py", line 900, in gevent._gevent_cgreenlet.Greenlet.run
  File "/home/rusian/yt-local/youtube/comments.py", line 195, in video_comments
    comments_info['error'] = 'YouTube blocked the request. IP address: %s' % e.ip
                                                                             ^^^^
AttributeError: 'TypeError' object has no attribute 'ip'
2025-03-08T01:25:47Z <Greenlet at 0x7f251e5279c0: video_comments('hcm55lU9knw', 0, lc='')> failed with AttributeError
2025-03-08 16:37:33 -05:00
Jesus
c256a045f9 Bump version to v0.3.1 2025-03-08 16:34:29 -05:00
Jesus
98603439cb Improve buffer management for different platforms
- Introduced `BUFFER_CONFIG` to define buffer sizes for various systems (webOS, Samsung Tizen, Android TV, desktop).
- Added `detectSystem()` function to determine the platform based on `navigator.userAgent`.
- Updated `Stream` constructor to use platform-specific buffer sizes dynamically.
- Added console log for debugging detected system and applied buffer size.
2025-03-08 16:32:26 -05:00
Jesus
a6ca011202 version v0.3.0 2025-03-08 16:28:39 -05:00
Jesus
114c2572a4 Renew plyr UI and simplify elements 2025-03-08 16:28:27 -05:00
f64b362603 update logic plyr-start.js 2025-03-03 08:20:41 +08:00
2fd7910194 version 0.2.21 2025-03-02 06:24:03 +08:00
c2e53072f7 update dependencies 2025-03-01 04:58:31 +08:00
c2986f3b14 Refactoring get_app_version 2025-03-01 04:06:11 +08:00
57854169f4 minor fix deprecation warning
tests/test_util.py: 14 warnings
  /home/runner/work/yt-local/youtube-local/youtube/util.py:321: DeprecationWarning: HTTPResponse.getheader() is deprecated and will be removed in urllib3 v2.1.0. Instead use HTTPResponse.headers.get(name, default).
    response.getheader('Content-Encoding', default='identity'))
2025-03-01 01:12:09 +08:00
3217305f9f version 0.2.20 2025-02-28 11:04:06 +08:00
639aadd2c1 Remove gather_googlevideo_domains setting
This was an old experiment to collect googlevideo domains to see
if there was a pattern that could correlate to IP address to
look for workarounds for 403 errors

Can bug out if enabled and if failed to get any vidoe urls,
so remove since it is obsolete and some people are enabling it

See #218
2025-02-28 10:58:29 +08:00
7157df13cd Remove params to fetch_player_response 2025-02-28 10:58:15 +08:00
630e0137e0 Increase playlist count to 1000 by default if cannot get video count
This way, buttons will still appear even if there is a failure
to read playlist metadata

Fixes #220# Please enter the commit message for your changes. Lines starting
2025-02-28 10:51:51 +08:00
a0c51731af channel.py: Catch FetchError
Should catch this error to fail gracefully

See #227
2025-02-28 10:51:29 +08:00
d361996fc0 util: use visitorData for api request
watch: use android_vr client to get player data
2025-02-28 10:43:14 +08:00
Jesus
4ef7dda14a version 0.2.19 2024-10-11 11:25:12 +08:00
Jesus
ee31cedae0 Revert "Refactoring code and reuse INNERTUBE_CLIENTS"
This reverts commit 8af98968dd.
2024-10-11 11:22:36 +08:00
d3b0cb5e13 workflows: update git sync actions 2024-08-05 09:23:38 +08:00
0a79974d11 Add sync to c.fridu.us and sourcehut 2024-08-05 05:27:58 +08:00
4e327944a0 Add CI 2024-07-15 10:39:00 +08:00
09a437f7fb v0.2.18 2024-07-09 13:10:10 +08:00
3cbe18aac0 Fix cves
CVE-2024-34064
CVE-2024-34069
CVE-2024-37891
2024-07-09 13:03:36 +08:00
Jesus
62418f8e95 Switch to android test suite client by default
Invidious' solution to the destruction of the android client:
https://github.com/iv-org/invidious/pull/4650

Fixes #207
2024-06-11 10:46:25 +08:00
bfd3760969 Release v0.2.17 2024-04-29 01:00:13 +08:00
efd89b2e64 set ios client 2024-04-27 09:54:42 +08:00
0dc1747178 update version 0.2.16 2024-04-21 13:16:18 +08:00
8577164785 update client params 2024-04-21 13:14:08 +08:00
8af98968dd Refactoring code and reuse INNERTUBE_CLIENTS 2024-04-21 13:13:19 +08:00
8f00cbcdd6 update
update android_music client
2024-04-21 11:21:35 +08:00
af75551bc2 update
update android client
2024-04-21 11:18:42 +08:00
3a6cc1e44f version 0.2.15 2024-04-08 07:25:50 +08:00
7664b5f0ff normalize css 2024-04-08 07:12:03 +08:00
ec5d236cad fix color dark theme 2024-04-08 07:10:03 +08:00
d6b7a255d0 v0.2.14 2024-04-07 11:52:53 +08:00
22bc7324db css normalize 2024-04-07 11:50:53 +08:00
48e8f271e7 update styles to modern 2024-04-07 11:44:19 +08:00
48 changed files with 2331 additions and 361 deletions

View File

@@ -1,12 +0,0 @@
image: debian/buster
packages:
- python3-pip
- virtualenv
tasks:
- test: |
cd yt-local
virtualenv -p python3 venv
source venv/bin/activate
python --version
pip install -r requirements-dev.txt
pytest

View File

@@ -1,10 +0,0 @@
kind: pipeline
name: default
steps:
- name: test
image: python:3.7.3
commands:
- pip install --upgrade pip
- pip install -r requirements-dev.txt
- pytest

23
.gitea/workflows/ci.yaml Normal file
View File

@@ -0,0 +1,23 @@
name: CI
on: [push, pull_request]
jobs:
test:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: 3.11
- name: Install dependencies
run: |
pip install --upgrade pip
pip install -r requirements-dev.txt
- name: Run tests
run: pytest

View File

@@ -0,0 +1,40 @@
name: git-sync-with-mirror
on:
push:
branches: [ master ]
workflow_dispatch:
jobs:
git-sync:
runs-on: ubuntu-latest
steps:
- name: git-sync
env:
git_sync_source_repo: git@git.fridu.us:heckyel/yt-local.git
git_sync_destination_repo: ssh://git@c.fridu.us/software/yt-local.git
if: env.git_sync_source_repo && env.git_sync_destination_repo
uses: astounds/git-sync@v1
with:
source_repo: git@git.fridu.us:heckyel/yt-local.git
source_branch: "master"
destination_repo: ssh://git@c.fridu.us/software/yt-local.git
destination_branch: "master"
source_ssh_private_key: ${{ secrets.GIT_SYNC_SOURCE_SSH_PRIVATE_KEY }}
destination_ssh_private_key: ${{ secrets.GIT_SYNC_DESTINATION_SSH_PRIVATE_KEY }}
- name: git-sync-sourcehut
env:
git_sync_source_repo: git@git.fridu.us:heckyel/yt-local.git
git_sync_destination_repo: git@git.sr.ht:~heckyel/yt-local
if: env.git_sync_source_repo && env.git_sync_destination_repo
uses: astounds/git-sync@v1
with:
source_repo: git@git.fridu.us:heckyel/yt-local.git
source_branch: "master"
destination_repo: git@git.sr.ht:~heckyel/yt-local
destination_branch: "master"
source_ssh_private_key: ${{ secrets.GIT_SYNC_SOURCE_SSH_PRIVATE_KEY }}
destination_ssh_private_key: ${{ secrets.GIT_SYNC_DESTINATION_SSH_PRIVATE_KEY }}
continue-on-error: true

137
.gitignore vendored
View File

@@ -1,5 +1,128 @@
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class
# C extensions
*.so
# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
pip-wheel-metadata/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST
# PyInstaller
*.manifest
*.spec
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
# Translations
*.mo
# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal
# Flask stuff:
instance/
.webassets-cache
# Scrapy stuff:
.scrapy
# Sphinx documentation
docs/_build/
# PyBuilder
target/
# Jupyter Notebook
.ipynb_checkpoints
# IPython
profile_default/
ipython_config.py
# pyenv
.python-version
# pipenv
Pipfile.lock
# PEP 582
__pypackages__/
# Celery stuff
celerybeat-schedule
celerybeat.pid
# SageMath parsed files
*.sage.py
# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/
*venv*
# Spyder project settings
.spyderproject
.spyproject
# Rope project settings
.ropeproject
# mkdocs documentation
/site
# mypy
.mypy_cache/
.dmypy.json
dmypy.json
# Pyre type checker
.pyre/
# Project specific
debug/
data/
python/
@@ -11,5 +134,17 @@ get-pip.py
latest-dist.zip
*.7z
*.zip
*venv*
# Editor specific
flycheck_*
.vscode/
.idea/
*.swp
*.swo
*~
.DS_Store
# Temporary files
*.tmp
*.bak
*.orig

210
Makefile Normal file
View File

@@ -0,0 +1,210 @@
# yt-local Makefile
# Automated tasks for development, translations, and maintenance
.PHONY: help install dev clean test i18n-extract i18n-init i18n-update i18n-compile i18n-stats i18n-clean setup-dev lint format backup restore
# Variables
PYTHON := python3
PIP := pip3
LANG_CODE ?= es
VENV_DIR := venv
PROJECT_NAME := yt-local
## Help
help: ## Show this help message
@echo "$(PROJECT_NAME) - Available tasks:"
@echo ""
@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf " %-20s %s\n", $$1, $$2}'
@echo ""
@echo "Examples:"
@echo " make install # Install dependencies"
@echo " make dev # Run development server"
@echo " make i18n-extract # Extract strings for translation"
@echo " make i18n-init LANG_CODE=fr # Initialize French"
@echo " make lint # Check code style"
## Installation and Setup
install: ## Install project dependencies
@echo "[INFO] Installing dependencies..."
$(PIP) install -r requirements.txt
@echo "[SUCCESS] Dependencies installed"
setup-dev: ## Complete development setup
@echo "[INFO] Setting up development environment..."
$(PYTHON) -m venv $(VENV_DIR)
./$(VENV_DIR)/bin/pip install -r requirements.txt
@echo "[SUCCESS] Virtual environment created in $(VENV_DIR)"
@echo "[INFO] Activate with: source $(VENV_DIR)/bin/activate"
requirements: ## Update and install requirements
@echo "[INFO] Installing/updating requirements..."
$(PIP) install --upgrade pip
$(PIP) install -r requirements.txt
@echo "[SUCCESS] Requirements installed"
## Development
dev: ## Run development server
@echo "[INFO] Starting development server..."
@echo "[INFO] Server available at: http://localhost:9010"
$(PYTHON) server.py
run: dev ## Alias for dev
## Testing
test: ## Run tests
@echo "[INFO] Running tests..."
@if [ -d "tests" ]; then \
$(PYTHON) -m pytest -v; \
else \
echo "[WARN] No tests directory found"; \
fi
test-cov: ## Run tests with coverage
@echo "[INFO] Running tests with coverage..."
@if command -v pytest-cov >/dev/null 2>&1; then \
$(PYTHON) -m pytest -v --cov=$(PROJECT_NAME) --cov-report=html; \
else \
echo "[WARN] pytest-cov not installed. Run: pip install pytest-cov"; \
fi
## Internationalization (i18n)
i18n-extract: ## Extract strings for translation
@echo "[INFO] Extracting strings for translation..."
$(PYTHON) manage_translations.py extract
@echo "[SUCCESS] Strings extracted to translations/messages.pot"
i18n-init: ## Initialize new language (use LANG_CODE=xx)
@echo "[INFO] Initializing language: $(LANG_CODE)"
$(PYTHON) manage_translations.py init $(LANG_CODE)
@echo "[SUCCESS] Language $(LANG_CODE) initialized"
@echo "[INFO] Edit: translations/$(LANG_CODE)/LC_MESSAGES/messages.po"
i18n-update: ## Update existing translations
@echo "[INFO] Updating existing translations..."
$(PYTHON) manage_translations.py update
@echo "[SUCCESS] Translations updated"
i18n-compile: ## Compile translations to binary .mo files
@echo "[INFO] Compiling translations..."
$(PYTHON) manage_translations.py compile
@echo "[SUCCESS] Translations compiled"
i18n-stats: ## Show translation statistics
@echo "[INFO] Translation statistics:"
@echo ""
@for lang_dir in translations/*/; do \
if [ -d "$$lang_dir" ] && [ "$$lang_dir" != "translations/*/" ]; then \
lang=$$(basename "$$lang_dir"); \
po_file="$$lang_dir/LC_MESSAGES/messages.po"; \
if [ -f "$$po_file" ]; then \
total=$$(grep -c "^msgid " "$$po_file" 2>/dev/null || echo "0"); \
translated=$$(grep -c "^msgstr \"[^\"]\+\"" "$$po_file" 2>/dev/null || echo "0"); \
fuzzy=$$(grep -c "^#, fuzzy" "$$po_file" 2>/dev/null || echo "0"); \
if [ "$$total" -gt 0 ]; then \
percent=$$((translated * 100 / total)); \
echo " [STAT] $$lang: $$translated/$$total ($$percent%) - Fuzzy: $$fuzzy"; \
else \
echo " [STAT] $$lang: No translations yet"; \
fi; \
fi \
fi \
done
@echo ""
i18n-clean: ## Clean compiled translation files
@echo "[INFO] Cleaning compiled .mo files..."
find translations/ -name "*.mo" -delete
@echo "[SUCCESS] .mo files removed"
i18n-workflow: ## Complete workflow: extract → update → compile
@echo "[INFO] Running complete translation workflow..."
@make i18n-extract
@make i18n-update
@make i18n-compile
@make i18n-stats
@echo "[SUCCESS] Translation workflow completed"
## Code Quality
lint: ## Check code with flake8
@echo "[INFO] Checking code style..."
@if command -v flake8 >/dev/null 2>&1; then \
flake8 youtube/ --max-line-length=120 --ignore=E501,W503,E402 --exclude=youtube/ytdlp_service.py,youtube/ytdlp_integration.py,youtube/ytdlp_proxy.py; \
echo "[SUCCESS] Code style check passed"; \
else \
echo "[WARN] flake8 not installed (pip install flake8)"; \
fi
format: ## Format code with black (if available)
@echo "[INFO] Formatting code..."
@if command -v black >/dev/null 2>&1; then \
black youtube/ --line-length=120 --exclude='ytdlp_.*\.py'; \
echo "[SUCCESS] Code formatted"; \
else \
echo "[WARN] black not installed (pip install black)"; \
fi
check-deps: ## Check installed dependencies
@echo "[INFO] Checking dependencies..."
@$(PYTHON) -c "import flask_babel; print('[OK] Flask-Babel:', flask_babel.__version__)" 2>/dev/null || echo "[ERROR] Flask-Babel not installed"
@$(PYTHON) -c "import flask; print('[OK] Flask:', flask.__version__)" 2>/dev/null || echo "[ERROR] Flask not installed"
@$(PYTHON) -c "import yt_dlp; print('[OK] yt-dlp:', yt_dlp.__version__)" 2>/dev/null || echo "[ERROR] yt-dlp not installed"
## Maintenance
backup: ## Create translations backup
@echo "[INFO] Creating translations backup..."
@timestamp=$$(date +%Y%m%d_%H%M%S); \
tar -czf "translations_backup_$$timestamp.tar.gz" translations/ 2>/dev/null || echo "[WARN] No translations to backup"; \
if [ -f "translations_backup_$$timestamp.tar.gz" ]; then \
echo "[SUCCESS] Backup created: translations_backup_$$timestamp.tar.gz"; \
fi
restore: ## Restore translations from backup
@echo "[INFO] Restoring translations from backup..."
@if ls translations_backup_*.tar.gz 1>/dev/null 2>&1; then \
latest_backup=$$(ls -t translations_backup_*.tar.gz | head -1); \
tar -xzf "$$latest_backup"; \
echo "[SUCCESS] Restored from: $$latest_backup"; \
else \
echo "[ERROR] No backup files found"; \
fi
clean: ## Clean temporary files and caches
@echo "[INFO] Cleaning temporary files..."
find . -type f -name "*.pyc" -delete
find . -type d -name "__pycache__" -delete
find . -type f -name "*.mo" -delete
find . -type d -name ".pytest_cache" -delete
find . -type f -name ".coverage" -delete
find . -type d -name "htmlcov" -delete
@echo "[SUCCESS] Temporary files removed"
distclean: clean ## Clean everything including venv
@echo "[INFO] Cleaning everything..."
rm -rf $(VENV_DIR)
@echo "[SUCCESS] Complete cleanup done"
## Project Information
info: ## Show project information
@echo "[INFO] $(PROJECT_NAME) - Project information:"
@echo ""
@echo " [INFO] Directory: $$(pwd)"
@echo " [INFO] Python: $$($(PYTHON) --version)"
@echo " [INFO] Pip: $$($(PIP) --version | cut -d' ' -f1-2)"
@echo ""
@echo " [INFO] Configured languages:"
@for lang_dir in translations/*/; do \
if [ -d "$$lang_dir" ] && [ "$$lang_dir" != "translations/*/" ]; then \
lang=$$(basename "$$lang_dir"); \
echo " - $$lang"; \
fi \
done
@echo ""
@echo " [INFO] Main files:"
@echo " - babel.cfg (i18n configuration)"
@echo " - manage_translations.py (i18n CLI)"
@echo " - youtube/i18n_strings.py (centralized strings)"
@echo " - youtube/ytdlp_service.py (yt-dlp integration)"
@echo ""
# Default target
.DEFAULT_GOAL := help

7
babel.cfg Normal file
View File

@@ -0,0 +1,7 @@
[python: youtube/**.py]
keywords = lazy_gettext:1,2 _l:1,2
[python: server.py]
[python: settings.py]
[jinja2: youtube/templates/**.html]
extensions=jinja2.ext.i18n
encoding = utf-8

113
manage_translations.py Normal file
View File

@@ -0,0 +1,113 @@
#!/usr/bin/env python3
"""
Translation management script for yt-local
Usage:
python manage_translations.py extract # Extract strings to messages.pot
python manage_translations.py init es # Initialize Spanish translation
python manage_translations.py update # Update all translations
python manage_translations.py compile # Compile translations to .mo files
"""
import sys
import os
import subprocess
# Ensure we use the Python from the virtual environment if available
if hasattr(sys, 'real_prefix') or (hasattr(sys, 'base_prefix') and sys.base_prefix != sys.prefix):
# Already in venv
pass
else:
# Try to activate venv
venv_path = os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), 'venv')
if os.path.exists(venv_path):
venv_bin = os.path.join(venv_path, 'bin')
if os.path.exists(venv_bin):
os.environ['PATH'] = venv_bin + os.pathsep + os.environ['PATH']
def run_command(cmd):
"""Run a shell command and print output"""
print(f"Running: {' '.join(cmd)}")
# Use the pybabel from the same directory as our Python executable
if cmd[0] == 'pybabel':
import os
pybabel_path = os.path.join(os.path.dirname(sys.executable), 'pybabel')
if os.path.exists(pybabel_path):
cmd = [pybabel_path] + cmd[1:]
result = subprocess.run(cmd, capture_output=True, text=True)
if result.stdout:
print(result.stdout)
if result.stderr:
print(result.stderr, file=sys.stderr)
return result.returncode
def extract():
"""Extract translatable strings from source code"""
print("Extracting translatable strings...")
return run_command([
'pybabel', 'extract',
'-F', 'babel.cfg',
'-k', 'lazy_gettext',
'-k', '_l',
'-o', 'translations/messages.pot',
'.'
])
def init(language):
"""Initialize a new language translation"""
print(f"Initializing {language} translation...")
return run_command([
'pybabel', 'init',
'-i', 'translations/messages.pot',
'-d', 'translations',
'-l', language
])
def update():
"""Update existing translations with new strings"""
print("Updating translations...")
return run_command([
'pybabel', 'update',
'-i', 'translations/messages.pot',
'-d', 'translations'
])
def compile_translations():
"""Compile .po files to .mo files"""
print("Compiling translations...")
return run_command([
'pybabel', 'compile',
'-d', 'translations'
])
def main():
if len(sys.argv) < 2:
print(__doc__)
sys.exit(1)
command = sys.argv[1]
if command == 'extract':
sys.exit(extract())
elif command == 'init':
if len(sys.argv) < 3:
print("Error: Please specify a language code (e.g., es, fr, de)")
sys.exit(1)
sys.exit(init(sys.argv[2]))
elif command == 'update':
sys.exit(update())
elif command == 'compile':
sys.exit(compile_translations())
else:
print(f"Unknown command: {command}")
print(__doc__)
sys.exit(1)
if __name__ == '__main__':
main()

View File

@@ -1,21 +1,5 @@
blinker==1.7.0
Brotli==1.1.0
cachetools==5.3.3
click==8.1.7
defusedxml==0.7.1
Flask==3.0.2
gevent==24.2.1
greenlet==3.0.3
iniconfig==2.0.0
itsdangerous==2.1.2
Jinja2==3.1.3
MarkupSafe==2.1.5
packaging==24.0
pluggy==1.4.0
PySocks==1.7.1
pytest==8.1.1
stem==1.8.2
urllib3==2.2.1
Werkzeug==3.0.1
zope.event==5.0
zope.interface==6.2
# Include all production requirements
-r requirements.txt
# Development requirements
pytest>=6.2.1

View File

@@ -1,17 +1,12 @@
blinker==1.7.0
Brotli==1.1.0
cachetools==5.3.3
click==8.1.7
defusedxml==0.7.1
Flask==3.0.2
gevent==24.2.1
greenlet==3.0.3
itsdangerous==2.1.2
Jinja2==3.1.3
MarkupSafe==2.1.5
PySocks==1.7.1
stem==1.8.2
urllib3==2.2.1
Werkzeug==3.0.1
zope.event==5.0
zope.interface==6.2
Flask>=1.0.3
Flask-Babel>=4.0.0
Babel>=2.12.0
gevent>=1.2.2
Brotli>=1.0.7
PySocks>=1.6.8
urllib3>=1.24.1
defusedxml>=0.5.0
cachetools>=4.0.0
stem>=1.8.0
yt-dlp>=2026.01.01
requests>=2.25.0

View File

@@ -279,6 +279,16 @@ if __name__ == '__main__':
print('Starting httpserver at http://%s:%s/' %
(ip_server, settings.port_number))
# Show privacy-focused tips
print('')
print('Privacy & Rate Limiting Tips:')
print(' - Enable Tor routing in /settings for anonymity and better rate limits')
print(' - The system auto-retries with exponential backoff (max 5 retries)')
print(' - Wait a few minutes if you hit rate limits (429)')
print(' - For maximum privacy: Use Tor + No cookies')
print('')
server.serve_forever()
# for uwsgi, gunicorn, etc.

View File

@@ -296,6 +296,17 @@ Archive: https://archive.ph/OZQbN''',
'category': 'interface',
}),
('language', {
'type': str,
'default': 'en',
'comment': 'Interface language',
'options': [
('en', 'English'),
('es', 'Español'),
],
'category': 'interface',
}),
('embed_page_mode', {
'type': bool,
'label': 'Enable embed page',
@@ -322,13 +333,6 @@ Archive: https://archive.ph/OZQbN''',
'comment': '',
}),
('gather_googlevideo_domains', {
'type': bool,
'default': False,
'comment': '''Developer use to debug 403s''',
'hidden': True,
}),
('debugging_save_responses', {
'type': bool,
'default': False,
@@ -336,9 +340,18 @@ Archive: https://archive.ph/OZQbN''',
'hidden': True,
}),
('ytdlp_enabled', {
'type': bool,
'default': True,
'comment': '''Enable yt-dlp integration for multi-language audio and subtitles''',
'hidden': False,
'label': 'Enable yt-dlp integration',
'category': 'playback',
}),
('settings_version', {
'type': int,
'default': 5,
'default': 6,
'comment': '''Do not change, remove, or comment out this value, or else your settings may be lost or corrupted''',
'hidden': True,
}),
@@ -419,11 +432,20 @@ def upgrade_to_5(settings_dict):
return new_settings
def upgrade_to_6(settings_dict):
new_settings = settings_dict.copy()
if 'gather_googlevideo_domains' in new_settings:
del new_settings['gather_googlevideo_domains']
new_settings['settings_version'] = 6
return new_settings
upgrade_functions = {
1: upgrade_to_2,
2: upgrade_to_3,
3: upgrade_to_4,
4: upgrade_to_5,
5: upgrade_to_6,
}

View File

@@ -0,0 +1,74 @@
# Spanish translations for yt-local.
# Copyright (C) 2026 yt-local
# This file is distributed under the same license as the yt-local project.
#
msgid ""
msgstr ""
"Project-Id-Version: PROJECT VERSION\n"
"Report-Msgid-Bugs-To: EMAIL@ADDRESS\n"
"POT-Creation-Date: 2026-03-22 15:05-0500\n"
"PO-Revision-Date: 2026-03-22 15:06-0500\n"
"Last-Translator: \n"
"Language: es\n"
"Language-Team: es <LL@li.org>\n"
"Plural-Forms: nplurals=2; plural=(n != 1);\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=utf-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Generated-By: Babel 2.18.0\n"
#: youtube/templates/base.html:38
msgid "Type to search..."
msgstr "Escribe para buscar..."
#: youtube/templates/base.html:39
msgid "Search"
msgstr "Buscar"
#: youtube/templates/base.html:45
msgid "Options"
msgstr "Opciones"
#: youtube/templates/base.html:47
msgid "Sort by"
msgstr "Ordenar por"
#: youtube/templates/base.html:50
msgid "Relevance"
msgstr "Relevancia"
#: youtube/templates/base.html:54 youtube/templates/base.html:65
msgid "Upload date"
msgstr "Fecha de subida"
#: youtube/templates/base.html:58
msgid "View count"
msgstr "Número de visualizaciones"
#: youtube/templates/base.html:62
msgid "Rating"
msgstr "Calificación"
#: youtube/templates/base.html:68
msgid "Any"
msgstr "Cualquiera"
#: youtube/templates/base.html:72
msgid "Last hour"
msgstr "Última hora"
#: youtube/templates/base.html:76
msgid "Today"
msgstr "Hoy"
#: youtube/templates/base.html:80
msgid "This week"
msgstr "Esta semana"
#: youtube/templates/base.html:84
msgid "This month"
msgstr "Este mes"
#: youtube/templates/base.html:88
msgid "This year"
msgstr "Este año"

75
translations/messages.pot Normal file
View File

@@ -0,0 +1,75 @@
# Translations template for PROJECT.
# Copyright (C) 2026 ORGANIZATION
# This file is distributed under the same license as the PROJECT project.
# FIRST AUTHOR <EMAIL@ADDRESS>, 2026.
#
#, fuzzy
msgid ""
msgstr ""
"Project-Id-Version: PROJECT VERSION\n"
"Report-Msgid-Bugs-To: EMAIL@ADDRESS\n"
"POT-Creation-Date: 2026-03-22 15:05-0500\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
"Language-Team: LANGUAGE <LL@li.org>\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=utf-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Generated-By: Babel 2.18.0\n"
#: youtube/templates/base.html:38
msgid "Type to search..."
msgstr ""
#: youtube/templates/base.html:39
msgid "Search"
msgstr ""
#: youtube/templates/base.html:45
msgid "Options"
msgstr ""
#: youtube/templates/base.html:47
msgid "Sort by"
msgstr ""
#: youtube/templates/base.html:50
msgid "Relevance"
msgstr ""
#: youtube/templates/base.html:54 youtube/templates/base.html:65
msgid "Upload date"
msgstr ""
#: youtube/templates/base.html:58
msgid "View count"
msgstr ""
#: youtube/templates/base.html:62
msgid "Rating"
msgstr ""
#: youtube/templates/base.html:68
msgid "Any"
msgstr ""
#: youtube/templates/base.html:72
msgid "Last hour"
msgstr ""
#: youtube/templates/base.html:76
msgid "Today"
msgstr ""
#: youtube/templates/base.html:80
msgid "This week"
msgstr ""
#: youtube/templates/base.html:84
msgid "This month"
msgstr ""
#: youtube/templates/base.html:88
msgid "This year"
msgstr ""

View File

@@ -7,12 +7,36 @@ import settings
import traceback
import re
from sys import exc_info
from flask_babel import Babel
yt_app = flask.Flask(__name__)
yt_app.config['TEMPLATES_AUTO_RELOAD'] = True
yt_app.url_map.strict_slashes = False
# yt_app.jinja_env.trim_blocks = True
# yt_app.jinja_env.lstrip_blocks = True
# Configure Babel for i18n
import os
yt_app.config['BABEL_DEFAULT_LOCALE'] = 'en'
# Use absolute path for translations directory to avoid issues with package structure changes
_app_root = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
yt_app.config['BABEL_TRANSLATION_DIRECTORIES'] = os.path.join(_app_root, 'translations')
def get_locale():
"""Determine the best locale based on user preference or browser settings"""
# Check if user has a language preference in settings
if hasattr(settings, 'language') and settings.language:
locale = settings.language
print(f'[i18n] Using user preference: {locale}')
return locale
# Otherwise, use browser's Accept-Language header
# Only match languages with available translations
locale = request.accept_languages.best_match(['en', 'es'])
print(f'[i18n] Using browser language: {locale}')
return locale or 'en'
babel = Babel(yt_app, locale_selector=get_locale)
yt_app.add_url_rule('/settings', 'settings_page', settings.settings_page, methods=['POST', 'GET'])
@@ -113,15 +137,28 @@ def error_page(e):
error_message += '\n\nExit node IP address: ' + exc_info()[1].ip
return flask.render_template('error.html', error_message=error_message, slim=slim), 502
elif exc_info()[0] == util.FetchError and exc_info()[1].error_message:
# Handle specific error codes with user-friendly messages
error_code = exc_info()[1].code
error_msg = exc_info()[1].error_message
if error_code == '400':
error_message = (f'Error: Bad Request (400)\n\n{error_msg}\n\n'
'This usually means the URL or parameters are invalid. '
'Try going back and trying a different option.')
elif error_code == '404':
error_message = 'Error: The page you are looking for isn\'t here.'
else:
error_message = f'Error: {error_code} - {error_msg}'
return (flask.render_template(
'error.html',
error_message=exc_info()[1].error_message,
error_message=error_message,
slim=slim
), 502)
elif (exc_info()[0] == util.FetchError
and exc_info()[1].code == '404'
):
error_message = ('Error: The page you are looking for isn\'t here. ¯\_(ツ)_/¯')
error_message = ('Error: The page you are looking for isn\'t here.')
return flask.render_template('error.html',
error_code=exc_info()[1].code,
error_message=error_message,

View File

@@ -33,53 +33,75 @@ headers_mobile = (
real_cookie = (('Cookie', 'VISITOR_INFO1_LIVE=8XihrAcN1l4'),)
generic_cookie = (('Cookie', 'VISITOR_INFO1_LIVE=ST1Ti53r4fU'),)
# added an extra nesting under the 2nd base64 compared to v4
# added tab support
# changed offset field to uint id 1
# FIXED 2026: YouTube changed continuation token structure (from Invidious commit a9f8127)
# Sort values for YouTube API (from Invidious): 2=popular, 4=newest, 5=oldest
def channel_ctoken_v5(channel_id, page, sort, tab, view=1):
new_sort = (2 if int(sort) == 1 else 1)
# Map sort values to YouTube API values (Invidious values)
# Input: sort=3 (newest), sort=4 (newest no shorts)
# YouTube expects: 4=newest
sort_mapping = {'1': 2, '2': 5, '3': 4, '4': 4} # 4 is newest without shorts
new_sort = sort_mapping.get(sort, 4)
offset = 30*(int(page) - 1)
if tab == 'videos':
tab = 15
elif tab == 'shorts':
tab = 10
elif tab == 'streams':
tab = 14
# Build continuation token using Invidious structure
# The structure is: base64(protobuf({
# 80226972: {
# 2: channel_id,
# 3: base64(protobuf({
# 110: {
# 3: {
# tab: {
# 1: {
# 1: base64(protobuf({
# 1: base64(protobuf({
# 2: "ST:" + base64(offset_varint)
# }))
# }))
# },
# 2: base64(protobuf({1: UUID}))
# 4: sort_value
# 8: base64(protobuf({
# 1: UUID
# 3: sort_value
# }))
# }
# }
# }
# }))
# }
# }))
# UUID placeholder
uuid_proto = proto.string(1, "00000000-0000-0000-0000-000000000000")
# Offset encoding
offset_varint = proto.uint(1, offset)
offset_encoded = proto.string(2, proto.unpadded_b64encode(offset_varint))
offset_wrapper = proto.string(1, proto.unpadded_b64encode(offset_encoded))
offset_base = proto.string(1, proto.unpadded_b64encode(offset_wrapper))
# Sort value varint
sort_varint = proto.uint(4, new_sort)
# Embedded message with UUID and sort
embedded_inner = uuid_proto + proto.uint(3, new_sort)
embedded_encoded = proto.string(8, proto.unpadded_b64encode(embedded_inner))
# Combine: uuid_wrapper + sort_varint + embedded
tab_inner_content = offset_base + uuid_proto + sort_varint + embedded_encoded
tab_inner = proto.string(1, proto.unpadded_b64encode(tab_inner_content))
tab_wrapper = proto.string(tab, tab_inner)
inner_container = proto.string(3, tab_wrapper)
outer_container = proto.string(110, inner_container)
encoded_inner = proto.percent_b64encode(outer_container)
pointless_nest = proto.string(80226972,
proto.string(2, channel_id)
+ proto.string(3,
proto.percent_b64encode(
proto.string(110,
proto.string(3,
proto.string(tab,
proto.string(1,
proto.string(1,
proto.unpadded_b64encode(
proto.string(1,
proto.string(1,
proto.unpadded_b64encode(
proto.string(2,
b"ST:"
+ proto.unpadded_b64encode(
proto.uint(1, offset)
)
)
)
)
)
)
)
# targetId, just needs to be present but
# doesn't need to be correct
+ proto.string(2, "63faaff0-0000-23fe-80f0-582429d11c38")
)
# 1 - newest, 2 - popular
+ proto.uint(3, new_sort)
)
)
)
)
)
+ proto.string(3, encoded_inner)
)
return base64.urlsafe_b64encode(pointless_nest).decode('ascii')
@@ -161,11 +183,6 @@ def channel_ctoken_v4(channel_id, page, sort, tab, view=1):
# SORT:
# videos:
# Popular - 1
# Oldest - 2
# Newest - 3
# playlists:
# Oldest - 2
# Newest - 3
# Last video added - 4
@@ -292,7 +309,7 @@ def get_number_of_videos_channel(channel_id):
try:
response = util.fetch_url(url, headers_mobile,
debug_name='number_of_videos', report_text='Got number of videos')
except urllib.error.HTTPError as e:
except (urllib.error.HTTPError, util.FetchError) as e:
traceback.print_exc()
print("Couldn't retrieve number of videos")
return 1000
@@ -389,7 +406,12 @@ def post_process_channel_info(info):
info['avatar'] = util.prefix_url(info['avatar'])
info['channel_url'] = util.prefix_url(info['channel_url'])
for item in info['items']:
item['thumbnail'] = "https://i.ytimg.com/vi/{}/hqdefault.jpg".format(item['id'])
# For playlists, use first_video_id for thumbnail, not playlist id
if item.get('type') == 'playlist' and item.get('first_video_id'):
item['thumbnail'] = "https://i.ytimg.com/vi/{}/hq720.jpg".format(item['first_video_id'])
elif item.get('type') == 'video':
item['thumbnail'] = "https://i.ytimg.com/vi/{}/hq720.jpg".format(item['id'])
# For channels and other types, keep existing thumbnail
util.prefix_urls(item)
util.add_extra_html_info(item)
if info['current_tab'] == 'about':
@@ -398,11 +420,20 @@ def post_process_channel_info(info):
info['links'][i] = (text, util.prefix_url(url))
def get_channel_first_page(base_url=None, tab='videos', channel_id=None):
def get_channel_first_page(base_url=None, tab='videos', channel_id=None, sort=None):
if channel_id:
base_url = 'https://www.youtube.com/channel/' + channel_id
return util.fetch_url(base_url + '/' + tab + '?pbj=1&view=0',
headers_desktop, debug_name='gen_channel_' + tab)
# Build URL with sort parameter
# YouTube URL sort params: p=popular, dd=newest, lad=newest no shorts
# Note: 'da' (oldest) was removed by YouTube in January 2026
url = base_url + '/' + tab + '?pbj=1&view=0'
if sort:
# Map sort values to YouTube's URL parameter values
sort_map = {'3': 'dd', '4': 'lad'}
url += '&sort=' + sort_map.get(sort, 'dd')
return util.fetch_url(url, headers_desktop, debug_name='gen_channel_' + tab)
playlist_sort_codes = {'2': "da", '3': "dd", '4': "lad"}
@@ -416,7 +447,6 @@ def get_channel_page_general_url(base_url, tab, request, channel_id=None):
page_number = int(request.args.get('page', 1))
# sort 1: views
# sort 2: oldest
# sort 3: newest
# sort 4: newest - no shorts (Just a kludge on our end, not internal to yt)
default_sort = '3' if settings.include_shorts_in_channel else '4'
sort = request.args.get('sort', default_sort)
@@ -483,17 +513,15 @@ def get_channel_page_general_url(base_url, tab, request, channel_id=None):
else:
num_videos_call = (get_number_of_videos_general, base_url)
# Use ctoken method, which YouTube changes all the time
if channel_id and not default_params:
if sort == 4:
_sort = 3
# For page 1, use the first-page method which won't break
# Pass sort parameter directly (2=oldest, 3=newest, etc.)
if page_number == 1:
# Always use first-page method for page 1 with sort parameter
page_call = (get_channel_first_page, base_url, tab, None, sort)
else:
_sort = sort
page_call = (get_channel_tab, channel_id, page_number, _sort,
tab, view, ctoken)
# Use the first-page method, which won't break
else:
page_call = (get_channel_first_page, base_url, tab)
# For page 2+, we can't paginate without continuation tokens
# This is a YouTube limitation, not our bug
flask.abort(404, 'Pagination not available for this sort option. YouTube removed this feature.')
tasks = (
gevent.spawn(*num_videos_call),
@@ -512,7 +540,14 @@ def get_channel_page_general_url(base_url, tab, request, channel_id=None):
})
continuation=True
elif tab == 'playlists' and page_number == 1:
polymer_json = util.fetch_url(base_url+ '/playlists?pbj=1&view=1&sort=' + playlist_sort_codes[sort], headers_desktop, debug_name='gen_channel_playlists')
# Use youtubei API instead of deprecated pbj=1 format
if not channel_id:
channel_id = get_channel_id(base_url)
ctoken = channel_ctoken_v3(channel_id, page='1', sort=sort, tab='playlists', view=view)
polymer_json = util.call_youtube_api('web', 'browse', {
'continuation': ctoken,
})
continuation = True
elif tab == 'playlists':
polymer_json = get_channel_tab(channel_id, page_number, sort,
'playlists', view)

View File

@@ -53,7 +53,7 @@ def request_comments(ctoken, replies=False):
'hl': 'en',
'gl': 'US',
'clientName': 'MWEB',
'clientVersion': '2.20240328.08.00',
'clientVersion': '2.20210804.02.00',
},
},
'continuation': ctoken.replace('=', '%3D'),
@@ -78,7 +78,7 @@ def single_comment_ctoken(video_id, comment_id):
def post_process_comments_info(comments_info):
for comment in comments_info['comments']:
comment['author'] = strip_non_ascii(comment['author'])
comment['author'] = strip_non_ascii(comment['author']) if comment.get('author') else ""
comment['author_url'] = concat_or_none(
'/', comment['author_url'])
comment['author_avatar'] = concat_or_none(
@@ -150,7 +150,7 @@ def post_process_comments_info(comments_info):
util.URL_ORIGIN, '/watch?v=', comments_info['video_id'])
comments_info['video_thumbnail'] = concat_or_none(
settings.img_prefix, 'https://i.ytimg.com/vi/',
comments_info['video_id'], '/hqdefault.jpg'
comments_info['video_id'], '/hq720.jpg'
)
@@ -189,10 +189,10 @@ def video_comments(video_id, sort=0, offset=0, lc='', secret_key=''):
comments_info['error'] += '\n\n' + e.error_message
comments_info['error'] += '\n\nExit node IP address: %s' % e.ip
else:
comments_info['error'] = 'YouTube blocked the request. IP address: %s' % e.ip
comments_info['error'] = 'YouTube blocked the request. Error: %s' % str(e)
except Exception as e:
comments_info['error'] = 'YouTube blocked the request. IP address: %s' % e.ip
comments_info['error'] = 'YouTube blocked the request. Error: %s' % str(e)
if comments_info.get('error'):
print('Error retrieving comments for ' + str(video_id) + ':\n' +

View File

@@ -11,17 +11,10 @@ import subprocess
def app_version():
def minimal_env_cmd(cmd):
# make minimal environment
env = {}
for k in ['SYSTEMROOT', 'PATH']:
v = os.environ.get(k)
if v is not None:
env[k] = v
env = {k: os.environ[k] for k in ['SYSTEMROOT', 'PATH'] if k in os.environ}
env.update({'LANGUAGE': 'C', 'LANG': 'C', 'LC_ALL': 'C'})
env['LANGUAGE'] = 'C'
env['LANG'] = 'C'
env['LC_ALL'] = 'C'
out = subprocess.Popen(
cmd, stdout=subprocess.PIPE, env=env).communicate()[0]
out = subprocess.Popen(cmd, stdout=subprocess.PIPE, env=env).communicate()[0]
return out
subst_list = {
@@ -31,24 +24,21 @@ def app_version():
}
if os.system("command -v git > /dev/null 2>&1") != 0:
subst_list
else:
if call(["git", "branch"], stderr=STDOUT,
stdout=open(os.devnull, 'w')) != 0:
subst_list
else:
# version
describe = minimal_env_cmd(["git", "describe", "--always"])
return subst_list
if call(["git", "branch"], stderr=STDOUT, stdout=open(os.devnull, 'w')) != 0:
return subst_list
describe = minimal_env_cmd(["git", "describe", "--tags", "--always"])
git_revision = describe.strip().decode('ascii')
# branch
branch = minimal_env_cmd(["git", "branch"])
git_branch = branch.strip().decode('ascii').replace('* ', '')
subst_list = {
"version": __version__,
subst_list.update({
"branch": git_branch,
"commit": git_revision
}
})
return subst_list

112
youtube/i18n_strings.py Normal file
View File

@@ -0,0 +1,112 @@
#!/usr/bin/env python3
"""
Centralized i18n strings for yt-local
This file contains static strings that need to be translated but are used
dynamically in templates or generated content. By importing this module,
these strings get extracted by babel for translation.
"""
from flask_babel import lazy_gettext as _l
# Settings categories
CATEGORY_NETWORK = _l('Network')
CATEGORY_PLAYBACK = _l('Playback')
CATEGORY_INTERFACE = _l('Interface')
# Common setting labels
ROUTE_TOR = _l('Route Tor')
DEFAULT_SUBTITLES_MODE = _l('Default subtitles mode')
AV1_CODEC_RANKING = _l('AV1 Codec Ranking')
VP8_VP9_CODEC_RANKING = _l('VP8/VP9 Codec Ranking')
H264_CODEC_RANKING = _l('H.264 Codec Ranking')
USE_INTEGRATED_SOURCES = _l('Use integrated sources')
ROUTE_IMAGES = _l('Route images')
ENABLE_COMMENTS_JS = _l('Enable comments.js')
ENABLE_SPONSORBLOCK = _l('Enable SponsorBlock')
ENABLE_EMBED_PAGE = _l('Enable embed page')
# Setting names (auto-generated from setting keys)
RELATED_VIDEOS_MODE = _l('Related videos mode')
COMMENTS_MODE = _l('Comments mode')
ENABLE_COMMENT_AVATARS = _l('Enable comment avatars')
DEFAULT_COMMENT_SORTING = _l('Default comment sorting')
THEATER_MODE = _l('Theater mode')
AUTOPLAY_VIDEOS = _l('Autoplay videos')
DEFAULT_RESOLUTION = _l('Default resolution')
USE_VIDEO_PLAYER = _l('Use video player')
USE_VIDEO_DOWNLOAD = _l('Use video download')
PROXY_IMAGES = _l('Proxy images')
THEME = _l('Theme')
FONT = _l('Font')
LANGUAGE = _l('Language')
EMBED_PAGE_MODE = _l('Embed page mode')
# Common option values
OFF = _l('Off')
ON = _l('On')
DISABLED = _l('Disabled')
ENABLED = _l('Enabled')
ALWAYS_SHOWN = _l('Always shown')
SHOWN_BY_CLICKING_BUTTON = _l('Shown by clicking button')
NATIVE = _l('Native')
NATIVE_WITH_HOTKEYS = _l('Native with hotkeys')
PLYR = _l('Plyr')
# Theme options
LIGHT = _l('Light')
GRAY = _l('Gray')
DARK = _l('Dark')
# Font options
BROWSER_DEFAULT = _l('Browser default')
LIBERATION_SERIF = _l('Liberation Serif')
ARIAL = _l('Arial')
VERDANA = _l('Verdana')
TAHOMA = _l('Tahoma')
# Search and filter options
SORT_BY = _l('Sort by')
RELEVANCE = _l('Relevance')
UPLOAD_DATE = _l('Upload date')
VIEW_COUNT = _l('View count')
RATING = _l('Rating')
# Time filters
ANY = _l('Any')
LAST_HOUR = _l('Last hour')
TODAY = _l('Today')
THIS_WEEK = _l('This week')
THIS_MONTH = _l('This month')
THIS_YEAR = _l('This year')
# Content types
TYPE = _l('Type')
VIDEO = _l('Video')
CHANNEL = _l('Channel')
PLAYLIST = _l('Playlist')
MOVIE = _l('Movie')
SHOW = _l('Show')
# Duration filters
DURATION = _l('Duration')
SHORT_DURATION = _l('Short (< 4 minutes)')
LONG_DURATION = _l('Long (> 20 minutes)')
# Actions
SEARCH = _l('Search')
DOWNLOAD = _l('Download')
SUBSCRIBE = _l('Subscribe')
UNSUBSCRIBE = _l('Unsubscribe')
IMPORT = _l('Import')
EXPORT = _l('Export')
SAVE = _l('Save')
CHECK = _l('Check')
MUTE = _l('Mute')
UNMUTE = _l('Unmute')
# Common UI elements
OPTIONS = _l('Options')
SETTINGS = _l('Settings')
ERROR = _l('Error')
LOADING = _l('loading...')

View File

@@ -8,7 +8,7 @@ import json
import string
import gevent
import math
from flask import request
from flask import request, abort
import flask
@@ -107,7 +107,7 @@ def get_playlist_page():
util.prefix_urls(item)
util.add_extra_html_info(item)
if 'id' in item:
item['thumbnail'] = f"{settings.img_prefix}https://i.ytimg.com/vi/{item['id']}/hqdefault.jpg"
item['thumbnail'] = f"{settings.img_prefix}https://i.ytimg.com/vi/{item['id']}/hq720.jpg"
item['url'] += '&list=' + playlist_id
if item['index']:
@@ -115,7 +115,7 @@ def get_playlist_page():
video_count = yt_data_extract.deep_get(info, 'metadata', 'video_count')
if video_count is None:
video_count = 40
video_count = 1000
return flask.render_template(
'playlist.html',

View File

@@ -113,12 +113,12 @@ def read_protobuf(data):
length = read_varint(data)
value = data.read(length)
elif wire_type == 3:
end_bytes = encode_varint((field_number << 3) | 4)
end_bytes = varint_encode((field_number << 3) | 4)
value = read_group(data, end_bytes)
elif wire_type == 5:
value = data.read(4)
else:
raise Exception("Unknown wire type: " + str(wire_type) + ", Tag: " + bytes_to_hex(succinct_encode(tag)) + ", at position " + str(data.tell()))
raise Exception("Unknown wire type: " + str(wire_type) + " at position " + str(data.tell()))
yield (wire_type, field_number, value)

View File

@@ -97,6 +97,7 @@ import re
import time
import json
import os
import traceback
import pprint

View File

@@ -256,7 +256,8 @@ hr {
padding-top: 6px;
text-align: center;
white-space: nowrap;
border: none;
border: 1px solid;
border-color: var(--button-border);
border-radius: 0.2rem;
}

View File

@@ -1,20 +1,22 @@
:root {
--background: #212121;
--background: #121113;
--text: #FFFFFF;
--secondary-hover: #73828c;
--secondary-focus: #303030;
--secondary-inverse: #FFF;
--secondary-hover: #222222;
--secondary-focus: #121113;
--secondary-inverse: #FFFFFF;
--primary-background: #242424;
--secondary-background: #424242;
--thumb-background: #757575;
--secondary-background: #222222;
--thumb-background: #222222;
--link: #00B0FF;
--link-visited: #40C4FF;
--border-bg: #FFFFFF;
--buttom: #dcdcdb;
--buttom-text: #415462;
--button-border: #91918c;
--buttom-hover: #BBB;
--search-text: #FFF;
--time-background: #212121;
--time-text: #FFF;
--border-bg: #222222;
--border-bg-settings: #000000;
--border-bg-license: #000000;
--buttom: #121113;
--buttom-text: #FFFFFF;
--button-border: #222222;
--buttom-hover: #222222;
--search-text: #FFFFFF;
--time-background: #121113;
--time-text: #FFFFFF;
}

View File

@@ -1,19 +1,21 @@
:root {
--background: #2d3743;
--background: #2D3743;
--text: #FFFFFF;
--secondary-hover: #73828c;
--secondary-hover: #73828C;
--secondary-focus: rgba(115, 130, 140, 0.125);
--secondary-inverse: #FFFFFF;
--primary-background: #2d3743;
--primary-background: #2D3743;
--secondary-background: #102027;
--thumb-background: #35404D;
--link: #22aaff;
--link-visited: #7755ff;
--link: #22AAFF;
--link-visited: #7755FF;
--border-bg: #FFFFFF;
--buttom: #DCDCDC;
--buttom-text: #415462;
--button-border: #91918c;
--buttom-hover: #BBBBBB;
--border-bg-settings: #FFFFFF;
--border-bg-license: #FFFFFF;
--buttom: #2D3743;
--buttom-text: #FFFFFF;
--button-border: #102027;
--buttom-hover: #102027;
--search-text: #FFFFFF;
--time-background: #212121;
--time-text: #FFFFFF;

View File

@@ -20,6 +20,29 @@
// TODO: Call abort to cancel in-progress appends?
// Buffer sizes for different systems
const BUFFER_CONFIG = {
default: 50 * 10**6, // 50 megabytes
webOS: 20 * 10**6, // 20 megabytes WebOS (LG)
samsungTizen: 20 * 10**6, // 20 megabytes Samsung Tizen OS
androidTV: 30 * 10**6, // 30 megabytes Android TV
desktop: 50 * 10**6, // 50 megabytes PC/Mac
};
function detectSystem() {
const userAgent = navigator.userAgent.toLowerCase();
if (/webos|lg browser/i.test(userAgent)) {
return "webOS";
} else if (/tizen/i.test(userAgent)) {
return "samsungTizen";
} else if (/android tv|smart-tv/i.test(userAgent)) {
return "androidTV";
} else if (/firefox|chrome|safari|edge/i.test(userAgent)) {
return "desktop";
} else {
return "default";
}
}
function AVMerge(video, srcInfo, startTime){
this.audioSource = null;
@@ -164,6 +187,8 @@ AVMerge.prototype.printDebuggingInfo = function() {
}
function Stream(avMerge, source, startTime, avRatio) {
const selectedSystem = detectSystem();
let baseBufferTarget = BUFFER_CONFIG[selectedSystem] || BUFFER_CONFIG.default;
this.avMerge = avMerge;
this.video = avMerge.video;
this.url = source['url'];
@@ -173,10 +198,11 @@ function Stream(avMerge, source, startTime, avRatio) {
this.mimeCodec = source['mime_codec']
this.streamType = source['acodec'] ? 'audio' : 'video';
if (this.streamType == 'audio') {
this.bufferTarget = avRatio*50*10**6;
this.bufferTarget = avRatio * baseBufferTarget;
} else {
this.bufferTarget = 50*10**6; // 50 megabytes
this.bufferTarget = baseBufferTarget;
}
console.info(`Detected system: ${selectedSystem}. Applying bufferTarget of ${this.bufferTarget} bytes to ${this.streamType}.`);
this.initRange = source['init_range'];
this.indexRange = source['index_range'];

View File

@@ -114,3 +114,60 @@ function copyTextToClipboard(text) {
window.addEventListener('DOMContentLoaded', function() {
cur_track_idx = getDefaultTranscriptTrackIdx();
});
/**
* Thumbnail fallback handler
* Tries lower quality thumbnails when higher quality fails (404)
* Priority: hq720.jpg -> sddefault.jpg -> hqdefault.jpg -> mqdefault.jpg -> default.jpg
*/
function thumbnail_fallback(img) {
const src = img.src || img.dataset.src;
if (!src) return;
// Handle YouTube video thumbnails
if (src.includes('/i.ytimg.com/')) {
// Extract video ID from URL
const match = src.match(/\/vi\/([^/]+)/);
if (!match) return;
const videoId = match[1];
const imgPrefix = settings_img_prefix || '';
// Define fallback order (from highest to lowest quality)
const fallbacks = [
'hq720.jpg',
'sddefault.jpg',
'hqdefault.jpg',
'mqdefault.jpg',
'default.jpg'
];
// Find current quality and try next fallback
for (let i = 0; i < fallbacks.length; i++) {
if (src.includes(fallbacks[i])) {
// Try next quality
if (i < fallbacks.length - 1) {
const newSrc = imgPrefix + 'https://i.ytimg.com/vi/' + videoId + '/' + fallbacks[i + 1];
if (img.dataset.src) {
img.dataset.src = newSrc;
} else {
img.src = newSrc;
}
}
break;
}
}
}
// Handle YouTube channel avatars (ggpht.com)
else if (src.includes('ggpht.com') || src.includes('yt3.ggpht.com')) {
// Try to increase avatar size (s88 -> s240)
const newSrc = src.replace(/=s\d+-c-k/, '=s240-c-k-c0x00ffffff-no-rj');
if (newSrc !== src) {
if (img.dataset.src) {
img.dataset.src = newSrc;
} else {
img.src = newSrc;
}
}
}
}

View File

@@ -58,7 +58,7 @@
},
});
const player = new Plyr(document.getElementById('js-video-player'), {
const playerOptions = {
// Learning about autoplay permission https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Permissions-Policy/autoplay#syntax
autoplay: autoplayActive,
disableContextMenu: false,
@@ -117,5 +117,20 @@
tooltips: {
controls: true,
},
}
const player = new Plyr(document.getElementById('js-video-player'), playerOptions);
// disable double click to fullscreen
// https://github.com/sampotts/plyr/issues/1370#issuecomment-528966795
player.eventListeners.forEach(function(eventListener) {
if(eventListener.type === 'dblclick') {
eventListener.element.removeEventListener(eventListener.type, eventListener.callback, eventListener.options);
}
});
// Add .started property, true after the playback has been started
// Needed so controls won't be hidden before playback has started
player.started = false;
player.once('playing', function(){this.started = true});
})();

View File

@@ -5,8 +5,9 @@ function changeQuality(selection) {
let videoPaused = video.paused;
let videoSpeed = video.playbackRate;
let srcInfo;
if (avMerge)
if (avMerge && typeof avMerge.close === 'function') {
avMerge.close();
}
if (selection.type == 'uni'){
srcInfo = data['uni_sources'][selection.index];
video.src = srcInfo.url;

View File

@@ -181,7 +181,7 @@ label[for=options-toggle-cbox] {
.table td,.table th {
padding: 10px 10px;
border: 1px solid var(--secondary-background);
border: 1px solid var(--border-bg-license);
text-align: center;
}

View File

@@ -10,9 +10,11 @@
--link: #212121;
--link-visited: #808080;
--border-bg: #212121;
--buttom: #DCDCDC;
--border-bg-settings: #91918C;
--border-bg-license: #91918C;
--buttom: #FFFFFF;
--buttom-text: #212121;
--button-border: #91918c;
--button-border: #91918C;
--buttom-hover: #BBBBBB;
--search-text: #212121;
--time-background: #212121;

View File

@@ -37,3 +37,41 @@ e.g. Firefox playback speed options */
max-height: 320px;
overflow-y: auto;
}
/*
* Custom styles similar to youtube
*/
.plyr__controls {
display: flex;
justify-content: center;
}
.plyr__progress__container {
position: absolute;
bottom: 0;
width: 100%;
margin-bottom: -10px;
}
.plyr__controls .plyr__controls__item:first-child {
margin-left: 0;
margin-right: 0;
z-index: 5;
}
.plyr__controls .plyr__controls__item.plyr__volume {
margin-left: auto;
}
.plyr__controls .plyr__controls__item.plyr__progress__container {
padding-left: 10px;
padding-right: 10px;
}
.plyr__progress input[type="range"] {
margin-bottom: 50px;
}
/*
* End custom styles
*/

View File

@@ -155,7 +155,7 @@ label[for=options-toggle-cbox] {
}
.settings-form > h2 {
border-bottom: 2px solid var(--border-bg);
border-bottom: 2px solid var(--border-bg-settings);
padding-bottom: 0.5rem;
}

View File

@@ -128,6 +128,29 @@ header {
background-color: var(--buttom-hover);
}
.live-url-choices {
background-color: var(--thumb-background);
margin: 1rem 0;
padding: 1rem;
}
.playability-error {
position: relative;
box-sizing: border-box;
height: 30vh;
margin: 1rem 0;
}
.playability-error > span {
display: flex;
background-color: var(--thumb-background);
height: 100%;
object-fit: cover;
justify-content: center;
align-items: center;
text-align: center;
}
.playlist {
display: grid;
grid-gap: 4px;
@@ -622,6 +645,9 @@ figure.sc-video {
max-height: 80vh;
overflow-y: scroll;
}
.playability-error {
height: 60vh;
}
.playlist {
display: grid;
grid-gap: 1px;

View File

@@ -1089,12 +1089,12 @@ def serve_subscription_thumbnail(thumbnail):
f.close()
return flask.Response(image, mimetype='image/jpeg')
url = f"https://i.ytimg.com/vi/{video_id}/hqdefault.jpg"
url = f"https://i.ytimg.com/vi/{video_id}/hq720.jpg"
try:
image = util.fetch_url(url, report_text="Saved thumbnail: " + video_id)
except urllib.error.HTTPError as e:
print("Failed to download thumbnail for " + video_id + ": " + str(e))
abort(e.code)
flask.abort(e.code)
try:
f = open(thumbnail_path, 'wb')
except FileNotFoundError:

View File

@@ -26,6 +26,12 @@
// @license-end
</script>
{% endif %}
<script>
// @license magnet:?xt=urn:btih:0b31508aeb0634b347b8270c7bee4d411b5d4109&dn=agpl-3.0.txt AGPL-v3-or-Later
// Image prefix for thumbnails
let settings_img_prefix = "{{ settings.img_prefix or '' }}";
// @license-end
</script>
</head>
<body>
@@ -35,57 +41,57 @@
</nav>
<form class="form" id="site-search" action="/youtube.com/results">
<input type="search" name="search_query" class="search-box" value="{{ search_box_value }}"
{{ "autofocus" if (request.path in ("/", "/results") or error_message) else "" }} required placeholder="Type to search...">
<button type="submit" value="Search" class="search-button">Search</button>
{{ "autofocus" if (request.path in ("/", "/results") or error_message) else "" }} required placeholder="{{ _('Type to search...') }}">
<button type="submit" value="Search" class="search-button">{{ _('Search') }}</button>
<!-- options -->
<div class="dropdown">
<!-- hidden box -->
<input id="options-toggle-cbox" class="opt-box" type="checkbox">
<!-- end hidden box -->
<label class="dropdown-label" for="options-toggle-cbox">Options</label>
<label class="dropdown-label" for="options-toggle-cbox">{{ _('Options') }}</label>
<div class="dropdown-content">
<h3>Sort by</h3>
<h3>{{ _('Sort by') }}</h3>
<div class="option">
<input type="radio" id="sort_relevance" name="sort" value="0">
<label for="sort_relevance">Relevance</label>
<label for="sort_relevance">{{ _('Relevance') }}</label>
</div>
<div class="option">
<input type="radio" id="sort_upload_date" name="sort" value="2">
<label for="sort_upload_date">Upload date</label>
<label for="sort_upload_date">{{ _('Upload date') }}</label>
</div>
<div class="option">
<input type="radio" id="sort_view_count" name="sort" value="3">
<label for="sort_view_count">View count</label>
<label for="sort_view_count">{{ _('View count') }}</label>
</div>
<div class="option">
<input type="radio" id="sort_rating" name="sort" value="1">
<label for="sort_rating">Rating</label>
<label for="sort_rating">{{ _('Rating') }}</label>
</div>
<h3>Upload date</h3>
<h3>{{ _('Upload date') }}</h3>
<div class="option">
<input type="radio" id="time_any" name="time" value="0">
<label for="time_any">Any</label>
<label for="time_any">{{ _('Any') }}</label>
</div>
<div class="option">
<input type="radio" id="time_last_hour" name="time" value="1">
<label for="time_last_hour">Last hour</label>
<label for="time_last_hour">{{ _('Last hour') }}</label>
</div>
<div class="option">
<input type="radio" id="time_today" name="time" value="2">
<label for="time_today">Today</label>
<label for="time_today">{{ _('Today') }}</label>
</div>
<div class="option">
<input type="radio" id="time_this_week" name="time" value="3">
<label for="time_this_week">This week</label>
<label for="time_this_week">{{ _('This week') }}</label>
</div>
<div class="option">
<input type="radio" id="time_this_month" name="time" value="4">
<label for="time_this_month">This month</label>
<label for="time_this_month">{{ _('This month') }}</label>
</div>
<div class="option">
<input type="radio" id="time_this_year" name="time" value="5">
<label for="time_this_year">This year</label>
<label for="time_this_year">{{ _('This year') }}</label>
</div>
<h3>Type</h3>

View File

@@ -81,10 +81,10 @@
<!-- new-->
<div id="links-metadata">
{% if current_tab in ('videos', 'shorts', 'streams') %}
{% set sorts = [('1', 'views'), ('2', 'oldest'), ('3', 'newest'), ('4', 'newest - no shorts'),] %}
{% set sorts = [('3', 'newest'), ('4', 'newest - no shorts')] %}
<div id="number-of-results">{{ number_of_videos }} videos</div>
{% elif current_tab == 'playlists' %}
{% set sorts = [('2', 'oldest'), ('3', 'newest'), ('4', 'last video added')] %}
{% set sorts = [('3', 'newest'), ('4', 'last video added')] %}
{% if items %}
<h2 class="page-number">Page {{ page_number }}</h2>
{% else %}

View File

@@ -23,11 +23,11 @@
<a class="thumbnail-box" href="{{ info['url'] }}" title="{{ info['title'] }}">
<div class="thumbnail {% if info['type'] == 'channel' %} channel {% endif %}">
{% if lazy_load %}
<img class="thumbnail-img lazy" alt="&#x20;" data-src="{{ info['thumbnail'] }}">
<img class="thumbnail-img lazy" alt="&#x20;" data-src="{{ info['thumbnail'] }}" onerror="thumbnail_fallback(this)">
{% elif info['type'] == 'channel' %}
<img class="thumbnail-img channel" alt="&#x20;" src="{{ info['thumbnail'] }}">
<img class="thumbnail-img channel" alt="&#x20;" src="{{ info['thumbnail'] }}" onerror="thumbnail_fallback(this)">
{% else %}
<img class="thumbnail-img" alt="&#x20;" src="{{ info['thumbnail'] }}">
<img class="thumbnail-img" alt="&#x20;" src="{{ info['thumbnail'] }}" onerror="thumbnail_fallback(this)">
{% endif %}
{% if info['type'] != 'channel' %}

View File

@@ -31,11 +31,19 @@
<input type="number" id="{{ 'setting_' + setting_name }}" name="{{ setting_name }}" value="{{ value }}" step="1">
{% endif %}
{% elif setting_info['type'].__name__ == 'float' %}
<input type="number" id="{{ 'setting_' + setting_name }}" name="{{ setting_name }}" value="{{ value }}" step="0.01">
{% elif setting_info['type'].__name__ == 'str' %}
<input type="text" id="{{ 'setting_' + setting_name }}" name="{{ setting_name }}" value="{{ value }}">
{% if 'options' is in(setting_info) %}
<select id="{{ 'setting_' + setting_name }}" name="{{ setting_name }}">
{% for option in setting_info['options'] %}
<option value="{{ option[0] }}" {{ 'selected' if option[0] == value else '' }}>{{ option[1] }}</option>
{% endfor %}
</select>
{% else %}
<span>Error: Unknown setting type: setting_info['type'].__name__</span>
<input type="text" id="{{ 'setting_' + setting_name }}" name="{{ setting_name }}" value="{{ value }}">
{% endif %}
{% else %}
<span>Error: Unknown setting type: {{ setting_info['type'].__name__ }}</span>
{% endif %}
</li>
{% endif %}

View File

@@ -85,6 +85,16 @@
<option value='{"type": "pair", "index": {{ loop.index0}}}' {{ 'selected' if loop.index0 == pair_idx and using_pair_sources else '' }} >{{ src_pair['quality_string'] }}</option>
{% endfor %}
</select>
{% if audio_tracks and audio_tracks|length > 1 %}
<select id="audio-language-select" autocomplete="off" title="Audio language">
{% for track in audio_tracks %}
<option value="{{ track.get('track_id', track['language']) }}" {{ 'selected' if loop.index0 == 0 else '' }}>
🔊 {{ track['language_name'] }}{% if track.get('is_default') %} (Default){% endif %}
</option>
{% endfor %}
</select>
{% endif %}
{% endif %}
</div>
<input class="v-checkbox" name="video_info_list" value="{{ video_info }}" form="playlist-edit" type="checkbox">
@@ -246,6 +256,38 @@
let storyboard_url = {{ storyboard_url | tojson }};
// @license-end
</script>
<!-- Audio language selector handler -->
<script>
// @license magnet:?xt=urn:btih:0b31508aeb0634b347b8270c7bee4d411b5d4109&dn=agpl-3.0.txt AGPL-v3-or-Later
(function() {
'use strict';
const audioSelect = document.getElementById('audio-language-select');
const qualitySelect = document.getElementById('quality-select');
if (audioSelect && qualitySelect) {
audioSelect.addEventListener('change', function() {
const selectedAudio = this.value;
const selectedQuality = qualitySelect.value;
// Parse current quality selection
let qualityData;
try {
qualityData = JSON.parse(selectedQuality);
} catch(e) {
return;
}
// Reload video with new audio language
const currentUrl = new URL(window.location.href);
currentUrl.searchParams.set('audio_lang', selectedAudio);
window.location.href = currentUrl.toString();
});
}
}());
// @license-end
</script>
<script src="/youtube.com/static/js/common.js"></script>
<script src="/youtube.com/static/js/transcript-table.js"></script>
{% if settings.use_video_player == 2 %}

View File

@@ -1,4 +1,5 @@
from datetime import datetime
import logging
import settings
import socks
import sockshandler
@@ -18,6 +19,8 @@ import gevent.queue
import gevent.lock
import collections
import stem
logger = logging.getLogger(__name__)
import stem.control
import traceback
@@ -302,7 +305,23 @@ def fetch_url_response(url, headers=(), timeout=15, data=None,
def fetch_url(url, headers=(), timeout=15, report_text=None, data=None,
cookiejar_send=None, cookiejar_receive=None, use_tor=True,
debug_name=None):
while True:
"""
Fetch URL with exponential backoff retry logic for rate limiting.
Retries:
- 429 Too Many Requests: Exponential backoff (1s, 2s, 4s, 8s, 16s)
- 503 Service Unavailable: Exponential backoff
- 302 Redirect to Google Sorry: Treated as rate limit
Max retries: 5 attempts with exponential backoff
"""
import random
max_retries = 5
base_delay = 1.0 # Base delay in seconds
for attempt in range(max_retries):
try:
start_time = time.monotonic()
response, cleanup_func = fetch_url_response(
@@ -318,10 +337,11 @@ def fetch_url(url, headers=(), timeout=15, report_text=None, data=None,
cleanup_func(response) # release_connection for urllib3
content = decode_content(
content,
response.getheader('Content-Encoding', default='identity'))
response.headers.get('Content-Encoding', default='identity'))
if (settings.debugging_save_responses
and debug_name is not None and content):
and debug_name is not None
and content):
save_dir = os.path.join(settings.data_dir, 'debug')
if not os.path.exists(save_dir):
os.makedirs(save_dir)
@@ -329,6 +349,7 @@ def fetch_url(url, headers=(), timeout=15, report_text=None, data=None,
with open(os.path.join(save_dir, debug_name), 'wb') as f:
f.write(content)
# Check for rate limiting (429) or redirect to Google Sorry
if response.status == 429 or (
response.status == 302 and (response.getheader('Location') == url
or response.getheader('Location').startswith(
@@ -336,7 +357,7 @@ def fetch_url(url, headers=(), timeout=15, report_text=None, data=None,
)
)
):
print(response.status, response.reason, response.headers)
logger.info(f'Rate limit response: {response.status} {response.reason}')
ip = re.search(
br'IP address: ((?:[\da-f]*:)+[\da-f]+|(?:\d+\.)+\d+)',
content)
@@ -346,28 +367,88 @@ def fetch_url(url, headers=(), timeout=15, report_text=None, data=None,
response.getheader('Set-Cookie') or '')
ip = ip.group(1) if ip else None
# don't get new identity if we're not using Tor
if not use_tor:
# If this is the last attempt, raise error
if attempt >= max_retries - 1:
if not use_tor or not settings.route_tor:
logger.warning(f'YouTube returned 429 but Tor is not enabled. Consider enabling Tor routing.')
raise FetchError('429', reason=response.reason, ip=ip)
else:
# Tor is enabled but we've exhausted retries
logger.error(f'YouTube blocked request - Tor exit node overutilized after {max_retries} retries. Exit IP: {ip}')
raise FetchError('429', reason=response.reason, ip=ip,
error_message='Tor exit node overutilized after multiple retries')
print('Error: YouTube blocked the request because the Tor exit node is overutilized. Exit node IP address: %s' % ip)
# For Tor: get new identity immediately on 429
if use_tor and settings.route_tor:
logger.info(f'YouTube blocked request - Tor exit node overutilized. Exit IP: {ip}. Getting new identity...')
# get new identity
error = tor_manager.new_identity(start_time)
if error:
raise FetchError(
'429', reason=response.reason, ip=ip,
error_message='Automatic circuit change: ' + error)
else:
continue # retry now that we have new identity
continue # retry with new identity
elif response.status >= 400:
raise FetchError(str(response.status), reason=response.reason,
ip=None)
# For non-Tor: exponential backoff
delay = (base_delay * (2 ** attempt)) + random.uniform(0, 1)
logger.info(f'Rate limited (429). Waiting {delay:.1f}s before retry {attempt + 1}/{max_retries}...')
time.sleep(delay)
continue # retry
# Check for client errors (400, 404) - don't retry these
if response.status == 400:
logger.error(f'Bad Request (400) - Invalid parameters or URL: {url[:100]}')
raise FetchError('400', reason='Bad Request - Invalid parameters or URL format', ip=None)
if response.status == 404:
logger.warning(f'Not Found (404): {url[:100]}')
raise FetchError('404', reason='Not Found', ip=None)
# Check for other server errors (503, 502, 504)
if response.status in (502, 503, 504):
if attempt >= max_retries - 1:
logger.error(f'Server error {response.status} after {max_retries} retries')
raise FetchError(str(response.status), reason=response.reason, ip=None)
# Exponential backoff for server errors
delay = (base_delay * (2 ** attempt)) + random.uniform(0, 1)
logger.warning(f'Server error ({response.status}). Waiting {delay:.1f}s before retry {attempt + 1}/{max_retries}...')
time.sleep(delay)
continue
# Success - break out of retry loop
break
except urllib3.exceptions.MaxRetryError as e:
# If this is the last attempt, raise the error
if attempt >= max_retries - 1:
exception_cause = e.__context__.__context__
if (isinstance(exception_cause, socks.ProxyConnectionError)
and settings.route_tor):
msg = ('Failed to connect to Tor. Check that Tor is open and '
'that your internet connection is working.\n\n'
+ str(e))
logger.error(f'Tor connection failed: {msg}')
raise FetchError('502', reason='Bad Gateway',
error_message=msg)
elif isinstance(e.__context__,
urllib3.exceptions.NewConnectionError):
msg = 'Failed to establish a connection.\n\n' + str(e)
logger.error(f'Connection failed: {msg}')
raise FetchError(
'502', reason='Bad Gateway',
error_message=msg)
else:
raise
# Wait and retry
delay = (base_delay * (2 ** attempt)) + random.uniform(0, 1)
logger.warning(f'Connection error. Waiting {delay:.1f}s before retry {attempt + 1}/{max_retries}...')
time.sleep(delay)
if report_text:
print(report_text, ' Latency:', round(response_time - start_time, 3), ' Read time:', round(read_finish - response_time,3))
logger.info(f'{report_text} - Latency: {round(response_time - start_time, 3)}s - Read time: {round(read_finish - response_time, 3)}s')
return content
@@ -394,23 +475,22 @@ def head(url, use_tor=False, report_text=None, max_redirects=10):
round(time.monotonic() - start_time, 3))
return response
mobile_user_agent = 'Mozilla/5.0 (Linux; Android 14) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.6312.80 Mobile Safari/537.36'
mobile_user_agent = 'Mozilla/5.0 (Linux; Android 7.0; Redmi Note 4 Build/NRD90M) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Mobile Safari/537.36'
mobile_ua = (('User-Agent', mobile_user_agent),)
desktop_user_agent = 'Mozilla/5.0 (Windows NT 10.0; rv:124.0) Gecko/20100101 Firefox/124.0'
desktop_user_agent = 'Mozilla/5.0 (Windows NT 6.1; rv:52.0) Gecko/20100101 Firefox/52.0'
desktop_ua = (('User-Agent', desktop_user_agent),)
json_header = (('Content-Type', 'application/json'),)
desktop_xhr_headers = (
('Accept', '*/*'),
('Accept-Language', 'en-US,en;q=0.5'),
('X-YouTube-Client-Name', '1'),
('X-YouTube-Client-Version', '2.20240327.00.00'),
('X-YouTube-Client-Version', '2.20240304.00.00'),
) + desktop_ua
mobile_xhr_headers = (
('Accept', '*/*'),
('Accept-Language', 'en-US,en;q=0.5'),
('X-YouTube-Client-Name', '1'),
('X-YouTube-Client-Version', '2.20240328.08.00'),
('X-YouTube-Client-Name', '2'),
('X-YouTube-Client-Version', '2.20240304.08.00'),
) + mobile_ua
@@ -462,7 +542,7 @@ class RateLimitedQueue(gevent.queue.Queue):
def download_thumbnail(save_directory, video_id):
url = f"https://i.ytimg.com/vi/{video_id}/hqdefault.jpg"
url = f"https://i.ytimg.com/vi/{video_id}/hq720.jpg"
save_location = os.path.join(save_directory, video_id + ".jpg")
try:
thumbnail = fetch_url(url, report_text="Saved thumbnail: " + video_id)
@@ -502,9 +582,40 @@ def video_id(url):
return urllib.parse.parse_qs(url_parts.query)['v'][0]
# default, sddefault, mqdefault, hqdefault, hq720
def get_thumbnail_url(video_id):
return f"{settings.img_prefix}https://i.ytimg.com/vi/{video_id}/hqdefault.jpg"
def get_thumbnail_url(video_id, quality='hq720'):
"""Get thumbnail URL with fallback to lower quality if needed.
Args:
video_id: YouTube video ID
quality: Preferred quality ('maxres', 'hq720', 'sd', 'hq', 'mq', 'default')
Returns:
Tuple of (best_available_url, quality_used)
"""
# Quality priority order (highest to lowest)
quality_order = {
'maxres': ['maxresdefault.jpg', 'sddefault.jpg', 'hqdefault.jpg'],
'hq720': ['hq720.jpg', 'sddefault.jpg', 'hqdefault.jpg'],
'sd': ['sddefault.jpg', 'hqdefault.jpg'],
'hq': ['hqdefault.jpg', 'mqdefault.jpg'],
'mq': ['mqdefault.jpg', 'default.jpg'],
'default': ['default.jpg'],
}
qualities = quality_order.get(quality, quality_order['hq720'])
base_url = f"{settings.img_prefix}https://i.ytimg.com/vi/{video_id}/"
# For now, return the highest quality URL
# The browser will handle 404s gracefully with alt text
return base_url + qualities[0], qualities[0]
def get_best_thumbnail_url(video_id):
"""Get the best available thumbnail URL for a video.
Tries hq720 first (for HD videos), falls back to sddefault for SD videos.
"""
return get_thumbnail_url(video_id, quality='hq720')[0]
def seconds_to_timestamp(seconds):
@@ -538,6 +649,12 @@ def prefix_url(url):
if url is None:
return None
url = url.lstrip('/') # some urls have // before them, which has a special meaning
# Increase resolution for YouTube channel avatars
if url and ('ggpht.com' in url or 'yt3.ggpht.com' in url):
# Replace size parameter with higher resolution (s240 instead of s88)
url = re.sub(r'=s\d+-c-k', '=s240-c-k-c0x00ffffff-no-rj', url)
return '/' + url
@@ -667,25 +784,6 @@ def to_valid_filename(name):
# https://github.com/yt-dlp/yt-dlp/blob/master/yt_dlp/extractor/youtube.py#L72
INNERTUBE_CLIENTS = {
'android_music': {
'INNERTUBE_API_KEY': 'AIzaSyAOghZGza2MQSZkY_zfZ370N-PUdXEo8AI',
'INNERTUBE_CONTEXT': {
'client': {
'hl': 'en',
'gl': 'US',
'clientName': 'ANDROID_MUSIC',
'clientVersion': '6.44.54',
'osName': 'Android',
'osVersion': '14',
'androidSdkVersion': 34,
'platform': 'MOBILE',
'userAgent': 'com.google.android.apps.youtube.music/6.44.54 (Linux; U; Android 14; US) gzip'
}
},
'INNERTUBE_CONTEXT_CLIENT_NAME': 21,
'REQUIRE_JS_PLAYER': False
},
'android': {
'INNERTUBE_API_KEY': 'AIzaSyA8eiZmM1FaDVjRy-df2KTyQ_vz_yYM39w',
'INNERTUBE_CONTEXT': {
@@ -693,13 +791,40 @@ INNERTUBE_CLIENTS = {
'hl': 'en',
'gl': 'US',
'clientName': 'ANDROID',
'clientVersion': '19.12.36',
'clientVersion': '19.09.36',
'osName': 'Android',
'osVersion': '14',
'androidSdkVersion': 34,
'osVersion': '12',
'androidSdkVersion': 31,
'platform': 'MOBILE',
'userAgent': 'com.google.android.youtube/19.13.36 (Linux; U; Android 14; en_US; Google Pixel 6 Pro) gzip'
}
'userAgent': 'com.google.android.youtube/19.09.36 (Linux; U; Android 12; US) gzip'
},
# https://github.com/yt-dlp/yt-dlp/pull/575#issuecomment-887739287
#'thirdParty': {
# 'embedUrl': 'https://google.com', # Can be any valid URL
#}
},
'INNERTUBE_CONTEXT_CLIENT_NAME': 3,
'REQUIRE_JS_PLAYER': False,
},
'android-test-suite': {
'INNERTUBE_API_KEY': 'AIzaSyA8eiZmM1FaDVjRy-df2KTyQ_vz_yYM39w',
'INNERTUBE_CONTEXT': {
'client': {
'hl': 'en',
'gl': 'US',
'clientName': 'ANDROID_TESTSUITE',
'clientVersion': '1.9',
'osName': 'Android',
'osVersion': '12',
'androidSdkVersion': 31,
'platform': 'MOBILE',
'userAgent': 'com.google.android.youtube/1.9 (Linux; U; Android 12; US) gzip'
},
# https://github.com/yt-dlp/yt-dlp/pull/575#issuecomment-887739287
#'thirdParty': {
# 'embedUrl': 'https://google.com', # Can be any valid URL
#}
},
'INNERTUBE_CONTEXT_CLIENT_NAME': 3,
'REQUIRE_JS_PLAYER': False,
@@ -712,9 +837,9 @@ INNERTUBE_CLIENTS = {
'hl': 'en',
'gl': 'US',
'clientName': 'IOS',
'clientVersion': '19.12.3',
'clientVersion': '19.09.3',
'deviceModel': 'iPhone14,3',
'userAgent': 'com.google.ios.youtube/19.12.3 (iPhone14,3; U; CPU iOS 15_6 like Mac OS X)'
'userAgent': 'com.google.ios.youtube/19.09.3 (iPhone14,3; U; CPU iOS 15_6 like Mac OS X)'
}
},
'INNERTUBE_CONTEXT_CLIENT_NAME': 5,
@@ -748,14 +873,62 @@ INNERTUBE_CLIENTS = {
'INNERTUBE_CONTEXT': {
'client': {
'clientName': 'WEB',
'clientVersion': '2.20240327.00.00',
'clientVersion': '2.20220801.00.00',
'userAgent': desktop_user_agent,
}
},
'INNERTUBE_CONTEXT_CLIENT_NAME': 1
},
'android_vr': {
'INNERTUBE_API_KEY': 'AIzaSyA8eiZmM1FaDVjRy-df2KTyQ_vz_yYM39w',
'INNERTUBE_CONTEXT': {
'client': {
'clientName': 'ANDROID_VR',
'clientVersion': '1.60.19',
'deviceMake': 'Oculus',
'deviceModel': 'Quest 3',
'androidSdkVersion': 32,
'userAgent': 'com.google.android.apps.youtube.vr.oculus/1.60.19 (Linux; U; Android 12L; eureka-user Build/SQ3A.220605.009.A1) gzip',
'osName': 'Android',
'osVersion': '12L',
},
},
'INNERTUBE_CONTEXT_CLIENT_NAME': 28,
'REQUIRE_JS_PLAYER': False,
},
}
def get_visitor_data():
visitor_data = None
visitor_data_cache = os.path.join(settings.data_dir, 'visitorData.txt')
if not os.path.exists(settings.data_dir):
os.makedirs(settings.data_dir)
if os.path.isfile(visitor_data_cache):
with open(visitor_data_cache, 'r') as file:
print('Getting visitor_data from cache')
visitor_data = file.read()
max_age = 12*3600
file_age = time.time() - os.path.getmtime(visitor_data_cache)
if file_age > max_age:
print('visitor_data cache is too old. Removing file...')
os.remove(visitor_data_cache)
return visitor_data
print('Fetching youtube homepage to get visitor_data')
yt_homepage = 'https://www.youtube.com'
yt_resp = fetch_url(yt_homepage, headers={'User-Agent': mobile_user_agent}, report_text='Getting youtube homepage')
visitor_data_re = r'''"visitorData":\s*?"(.+?)"'''
visitor_data_match = re.search(visitor_data_re, yt_resp.decode())
if visitor_data_match:
visitor_data = visitor_data_match.group(1)
print(f'Got visitor_data: {len(visitor_data)}')
with open(visitor_data_cache, 'w') as file:
print('Saving visitor_data cache...')
file.write(visitor_data)
return visitor_data
else:
print('Unable to get visitor_data value')
return visitor_data
def call_youtube_api(client, api, data):
client_params = INNERTUBE_CLIENTS[client]
@@ -763,12 +936,17 @@ def call_youtube_api(client, api, data):
key = client_params['INNERTUBE_API_KEY']
host = client_params.get('INNERTUBE_HOST') or 'www.youtube.com'
user_agent = context['client'].get('userAgent') or mobile_user_agent
visitor_data = get_visitor_data()
url = 'https://' + host + '/youtubei/v1/' + api + '?key=' + key
if visitor_data:
context['client'].update({'visitorData': visitor_data})
data['context'] = context
data = json.dumps(data)
headers = (('Content-Type', 'application/json'),('User-Agent', user_agent))
if visitor_data:
headers = ( *headers, ('X-Goog-Visitor-Id', visitor_data ))
response = fetch_url(
url, data=data, headers=headers,
debug_name='youtubei_' + api + '_' + client,
@@ -779,6 +957,8 @@ def call_youtube_api(client, api, data):
def strip_non_ascii(string):
''' Returns the string without non ASCII characters'''
if string is None:
return ""
stripped = (c for c in string if 0 < ord(c) < 127)
return ''.join(stripped)

View File

@@ -1,3 +1,3 @@
from __future__ import unicode_literals
__version__ = '0.2.13'
__version__ = 'v0.4.1'

View File

@@ -6,6 +6,9 @@ import settings
from flask import request
import flask
import logging
logger = logging.getLogger(__name__)
import json
import gevent
@@ -343,7 +346,6 @@ def _add_to_error(info, key, additional_message):
def fetch_player_response(client, video_id):
return util.call_youtube_api(client, 'player', {
'videoId': video_id,
'params': 'CgIIAQ==',
})
@@ -368,32 +370,42 @@ def fetch_watch_page_info(video_id, playlist_id, index):
watch_page = watch_page.decode('utf-8')
return yt_data_extract.extract_watch_info_from_html(watch_page)
def extract_info(video_id, use_invidious, playlist_id=None, index=None):
primary_client = 'android_vr'
fallback_client = 'ios'
last_resort_client = 'tv_embedded'
tasks = (
# Get video metadata from here
gevent.spawn(fetch_watch_page_info, video_id, playlist_id, index),
gevent.spawn(fetch_player_response, 'android', video_id)
gevent.spawn(fetch_player_response, primary_client, video_id)
)
gevent.joinall(tasks)
util.check_gevent_exceptions(*tasks)
info, player_response = tasks[0].value, tasks[1].value
info = tasks[0].value or {}
player_response = tasks[1].value or {}
yt_data_extract.update_with_new_urls(info, player_response)
# Age restricted video, retry
if info['age_restricted'] or info['player_urls_missing']:
if info['age_restricted']:
print('Age restricted video, retrying')
else:
print('Player urls missing, retrying')
player_response = fetch_player_response('tv_embedded', video_id)
# Fallback to 'ios' if no valid URLs are found
if not info.get('formats') or info.get('player_urls_missing'):
print(f"No URLs found in '{primary_client}', attempting with '{fallback_client}'.")
player_response = fetch_player_response(fallback_client, video_id) or {}
yt_data_extract.update_with_new_urls(info, player_response)
# Final attempt with 'tv_embedded' if there are still no URLs
if not info.get('formats') or info.get('player_urls_missing'):
print(f"No URLs found in '{fallback_client}', attempting with '{last_resort_client}'")
player_response = fetch_player_response(last_resort_client, video_id) or {}
yt_data_extract.update_with_new_urls(info, player_response)
# signature decryption
if info.get('formats'):
decryption_error = decrypt_signatures(info, video_id)
if decryption_error:
decryption_error = 'Error decrypting url signatures: ' + decryption_error
info['playability_error'] = decryption_error
info['playability_error'] = 'Error decrypting url signatures: ' + decryption_error
# check if urls ready (non-live format) in former livestream
# urls not ready if all of them have no filesize
@@ -407,20 +419,20 @@ def extract_info(video_id, use_invidious, playlist_id=None, index=None):
# livestream urls
# sometimes only the livestream urls work soon after the livestream is over
if (info['hls_manifest_url']
and (info['live'] or not info['formats'] or not info['urls_ready'])
):
info['hls_formats'] = []
if info.get('hls_manifest_url') and (info.get('live') or not info.get('formats') or not info['urls_ready']):
try:
manifest = util.fetch_url(info['hls_manifest_url'],
debug_name='hls_manifest.m3u8',
report_text='Fetched hls manifest'
).decode('utf-8')
info['hls_formats'], err = yt_data_extract.extract_hls_formats(manifest)
if not err:
info['playability_error'] = None
for fmt in info['hls_formats']:
fmt['video_quality'] = video_quality_string(fmt)
else:
except Exception as e:
print(f"Error obteniendo HLS manifest: {e}")
info['hls_formats'] = []
# check for 403. Unnecessary for tor video routing b/c ip address is same
@@ -616,7 +628,12 @@ def get_watch_page(video_id=None):
# prefix urls, and other post-processing not handled by yt_data_extract
for item in info['related_videos']:
item['thumbnail'] = "https://i.ytimg.com/vi/{}/hqdefault.jpg".format(item['id']) # set HQ relateds thumbnail videos
# For playlists, use first_video_id for thumbnail, not playlist id
if item.get('type') == 'playlist' and item.get('first_video_id'):
item['thumbnail'] = "https://i.ytimg.com/vi/{}/hq720.jpg".format(item['first_video_id'])
elif item.get('type') == 'video':
item['thumbnail'] = "https://i.ytimg.com/vi/{}/hq720.jpg".format(item['id'])
# For other types, keep existing thumbnail or skip
util.prefix_urls(item)
util.add_extra_html_info(item)
for song in info['music_list']:
@@ -624,6 +641,9 @@ def get_watch_page(video_id=None):
if info['playlist']:
playlist_id = info['playlist']['id']
for item in info['playlist']['items']:
# Set high quality thumbnail for playlist videos
if item.get('type') == 'video' and item.get('id'):
item['thumbnail'] = "https://i.ytimg.com/vi/{}/hq720.jpg".format(item['id'])
util.prefix_urls(item)
util.add_extra_html_info(item)
if playlist_id:
@@ -650,12 +670,6 @@ def get_watch_page(video_id=None):
'/videoplayback',
'/videoplayback/name/' + filename)
if settings.gather_googlevideo_domains:
with open(os.path.join(settings.data_dir, 'googlevideo-domains.txt'), 'a+', encoding='utf-8') as f:
url = info['formats'][0]['url']
subdomain = url[0:url.find(".googlevideo.com")]
f.write(subdomain + "\n")
download_formats = []
for format in (info['formats'] + info['hls_formats']):
@@ -682,6 +696,30 @@ def get_watch_page(video_id=None):
pair_sources = source_info['pair_sources']
uni_idx, pair_idx = source_info['uni_idx'], source_info['pair_idx']
# Extract audio tracks using yt-dlp for multi-language support
audio_tracks = []
try:
from youtube import ytdlp_integration
logger.info(f'Extracting audio tracks for video: {video_id}')
ytdlp_info = ytdlp_integration.extract_video_info_ytdlp(video_id)
audio_tracks = ytdlp_info.get('audio_tracks', [])
if audio_tracks:
logger.info(f'✓ Found {len(audio_tracks)} audio tracks:')
for i, track in enumerate(audio_tracks[:10], 1): # Log first 10
logger.info(f' [{i}] {track["language_name"]} ({track["language"]}) - '
f'bitrate: {track.get("audio_bitrate", "N/A")}k, '
f'codec: {track.get("acodec", "N/A")}, '
f'format_id: {track.get("format_id", "N/A")}')
if len(audio_tracks) > 10:
logger.info(f' ... and {len(audio_tracks) - 10} more')
else:
logger.warning(f'No audio tracks found for video {video_id}')
except Exception as e:
logger.error(f'Failed to extract audio tracks: {e}', exc_info=True)
audio_tracks = []
pair_quality = yt_data_extract.deep_get(pair_sources, pair_idx, 'quality')
uni_quality = yt_data_extract.deep_get(uni_sources, uni_idx, 'quality')
@@ -805,7 +843,9 @@ def get_watch_page(video_id=None):
'playlist': info['playlist'],
'related': info['related_videos'],
'playability_error': info['playability_error'],
'audio_tracks': audio_tracks,
},
audio_tracks = audio_tracks,
font_family = youtube.font_choices[settings.font], # for embed page
**source_info,
using_pair_sources = using_pair_sources,
@@ -814,9 +854,17 @@ def get_watch_page(video_id=None):
@yt_app.route('/api/<path:dummy>')
def get_captions(dummy):
try:
result = util.fetch_url('https://www.youtube.com' + request.full_path)
result = result.replace(b"align:start position:0%", b"")
return result
except util.FetchError as e:
# Return empty captions gracefully instead of error page
logger.warning(f'Failed to fetch captions: {e}')
return flask.Response(b'WEBVTT\n\n', mimetype='text/vtt', status=200)
except Exception as e:
logger.error(f'Unexpected error fetching captions: {e}')
return flask.Response(b'WEBVTT\n\n', mimetype='text/vtt', status=200)
times_reg = re.compile(r'^\d\d:\d\d:\d\d\.\d\d\d --> \d\d:\d\d:\d\d\.\d\d\d.*$')
@@ -881,3 +929,18 @@ def get_transcript(caption_path):
return flask.Response(result.encode('utf-8'),
mimetype='text/plain;charset=UTF-8')
# ============================================================================
# yt-dlp Integration Routes
# ============================================================================
@yt_app.route('/ytl-api/video-with-audio/<video_id>')
def proxy_video_with_audio(video_id):
"""
Proxy para servir video con audio específico usando yt-dlp
"""
from youtube import ytdlp_proxy
audio_lang = request.args.get('lang', 'en')
max_quality = int(request.args.get('quality', 720))
return ytdlp_proxy.stream_video_with_audio(video_id, audio_lang, max_quality)

View File

@@ -226,6 +226,89 @@ def check_missing_keys(object, *key_sequences):
return None
def extract_lockup_view_model_info(item, additional_info={}):
"""Extract info from new lockupViewModel format (YouTube 2024+)"""
info = {'error': None}
content_type = item.get('contentType', '')
content_id = item.get('contentId', '')
# Extract title from metadata
metadata = item.get('metadata', {})
lockup_metadata = metadata.get('lockupMetadataViewModel', {})
title_data = lockup_metadata.get('title', {})
info['title'] = title_data.get('content', '')
# Determine type based on contentType
if 'PLAYLIST' in content_type:
info['type'] = 'playlist'
info['playlist_type'] = 'playlist'
info['id'] = content_id
info['video_count'] = None
info['first_video_id'] = None
# Try to get video count from metadata
metadata_rows = lockup_metadata.get('metadata', {})
for row in metadata_rows.get('contentMetadataViewModel', {}).get('metadataRows', []):
for part in row.get('metadataParts', []):
text = part.get('text', {}).get('content', '')
if 'video' in text.lower():
info['video_count'] = extract_int(text)
elif 'VIDEO' in content_type:
info['type'] = 'video'
info['id'] = content_id
info['view_count'] = None
info['approx_view_count'] = None
info['time_published'] = None
info['duration'] = None
# Extract duration/other info from metadata rows
metadata_rows = lockup_metadata.get('metadata', {})
for row in metadata_rows.get('contentMetadataViewModel', {}).get('metadataRows', []):
for part in row.get('metadataParts', []):
text = part.get('text', {}).get('content', '')
if 'view' in text.lower():
info['approx_view_count'] = extract_approx_int(text)
elif 'ago' in text.lower():
info['time_published'] = text
elif 'CHANNEL' in content_type:
info['type'] = 'channel'
info['id'] = content_id
info['approx_subscriber_count'] = None
else:
info['type'] = 'unsupported'
return info
# Extract thumbnail from contentImage
content_image = item.get('contentImage', {})
collection_thumb = content_image.get('collectionThumbnailViewModel', {})
primary_thumb = collection_thumb.get('primaryThumbnail', {})
thumb_vm = primary_thumb.get('thumbnailViewModel', {})
image_sources = thumb_vm.get('image', {}).get('sources', [])
if image_sources:
info['thumbnail'] = image_sources[0].get('url', '')
else:
info['thumbnail'] = ''
# Extract author info if available
info['author'] = None
info['author_id'] = None
info['author_url'] = None
# Try to get first video ID from inline player data
item_playback = item.get('itemPlayback', {})
inline_player = item_playback.get('inlinePlayerData', {})
on_select = inline_player.get('onSelect', {})
innertube_cmd = on_select.get('innertubeCommand', {})
watch_endpoint = innertube_cmd.get('watchEndpoint', {})
if watch_endpoint.get('videoId'):
info['first_video_id'] = watch_endpoint.get('videoId')
info.update(additional_info)
return info
def extract_item_info(item, additional_info={}):
if not item:
return {'error': 'No item given'}
@@ -243,6 +326,10 @@ def extract_item_info(item, additional_info={}):
info['type'] = 'unsupported'
return info
# Handle new lockupViewModel format (YouTube 2024+)
if type == 'lockupViewModel':
return extract_lockup_view_model_info(item, additional_info)
# type looks like e.g. 'compactVideoRenderer' or 'gridVideoRenderer'
# camelCase split, https://stackoverflow.com/a/37697078
type_parts = [s.lower() for s in re.sub(r'([A-Z][a-z]+)', r' \1', type).split()]
@@ -441,6 +528,9 @@ _item_types = {
'channelRenderer',
'compactChannelRenderer',
'gridChannelRenderer',
# New viewModel format (YouTube 2024+)
'lockupViewModel',
}
def _traverse_browse_renderer(renderer):

View File

@@ -229,7 +229,7 @@ def extract_playlist_metadata(polymer_json):
if metadata['first_video_id'] is None:
metadata['thumbnail'] = None
else:
metadata['thumbnail'] = f"https://i.ytimg.com/vi/{metadata['first_video_id']}/hqdefault.jpg"
metadata['thumbnail'] = f"https://i.ytimg.com/vi/{metadata['first_video_id']}/hq720.jpg"
metadata['video_count'] = extract_int(header.get('numVideosText'))
metadata['description'] = extract_str(header.get('descriptionText'), default='')

View File

@@ -0,0 +1,78 @@
#!/usr/bin/env python3
"""
yt-dlp integration wrapper for backward compatibility.
This module now uses the centralized ytdlp_service for all operations.
"""
import logging
from youtube.ytdlp_service import (
extract_video_info,
get_language_name,
clear_cache,
get_cache_info,
)
logger = logging.getLogger(__name__)
def extract_video_info_ytdlp(video_id):
"""
Extract video information using yt-dlp (with caching).
This is a wrapper around ytdlp_service.extract_video_info()
for backward compatibility.
Args:
video_id: YouTube video ID
Returns:
Dictionary with audio_tracks, formats, title, duration
"""
logger.debug(f'Extracting video info (legacy API): {video_id}')
info = extract_video_info(video_id)
# Convert to legacy format for backward compatibility
return {
'audio_tracks': info.get('audio_tracks', []),
'all_audio_formats': info.get('formats', []),
'formats': info.get('formats', []),
'title': info.get('title', ''),
'duration': info.get('duration', 0),
'error': info.get('error'),
}
def get_audio_formats_for_language(video_id, language='en'):
"""
Get available audio formats for a specific language.
Args:
video_id: YouTube video ID
language: Language code (default: 'en')
Returns:
List of audio format dicts
"""
info = extract_video_info_ytdlp(video_id)
if 'error' in info:
logger.warning(f'Cannot get audio formats: {info["error"]}')
return []
audio_formats = []
for track in info.get('audio_tracks', []):
if track['language'] == language:
audio_formats.append(track)
logger.debug(f'Found {len(audio_formats)} {language} audio formats')
return audio_formats
__all__ = [
'extract_video_info_ytdlp',
'get_audio_formats_for_language',
'get_language_name',
'clear_cache',
'get_cache_info',
]

99
youtube/ytdlp_proxy.py Normal file
View File

@@ -0,0 +1,99 @@
#!/usr/bin/env python3
"""
Proxy for serving videos with specific audio using yt-dlp.
This module provides streaming functionality for unified formats
with specific audio languages.
"""
import logging
from flask import Response, request, stream_with_context
import urllib.request
import urllib.error
from youtube.ytdlp_service import find_best_unified_format
logger = logging.getLogger(__name__)
def stream_video_with_audio(video_id: str, audio_language: str = 'en', max_quality: int = 720):
"""
Stream video with specific audio language.
Args:
video_id: YouTube video ID
audio_language: Preferred audio language (default: 'en')
max_quality: Maximum video height (default: 720)
Returns:
Flask Response with video stream, or 404 if not available
"""
logger.info(f'Stream request: {video_id} | audio={audio_language} | quality={max_quality}p')
# Find best unified format
best_format = find_best_unified_format(video_id, audio_language, max_quality)
if not best_format:
logger.info(f'No suitable unified format found, returning 404 to trigger fallback')
return Response('No suitable unified format available', status=404)
url = best_format.get('url')
if not url:
logger.error('Format found but no URL available')
return Response('Format URL not available', status=500)
logger.debug(f'Streaming from: {url[:80]}...')
# Stream the video
try:
req = urllib.request.Request(url)
req.add_header('User-Agent', 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36')
req.add_header('Accept', '*/*')
# Add Range header if client requests it
if 'Range' in request.headers:
req.add_header('Range', request.headers['Range'])
logger.debug(f'Range request: {request.headers["Range"]}')
resp = urllib.request.urlopen(req, timeout=60)
def generate():
"""Generator for streaming video chunks."""
try:
while True:
chunk = resp.read(65536) # 64KB chunks
if not chunk:
break
yield chunk
except Exception as e:
logger.error(f'Stream error: {e}')
raise
# Build response headers
response_headers = {
'Content-Type': resp.headers.get('Content-Type', 'video/mp4'),
'Access-Control-Allow-Origin': '*',
}
# Copy important headers
for header in ['Content-Length', 'Content-Range', 'Accept-Ranges']:
if header in resp.headers:
response_headers[header] = resp.headers[header]
status_code = resp.getcode()
logger.info(f'Streaming started: {status_code}')
return Response(
stream_with_context(generate()),
status=status_code,
headers=response_headers,
direct_passthrough=True
)
except urllib.error.HTTPError as e:
logger.error(f'HTTP error streaming: {e.code} {e.reason}')
return Response(f'Error: {e.code} {e.reason}', status=e.code)
except urllib.error.URLError as e:
logger.error(f'URL error streaming: {e.reason}')
return Response(f'Network error: {e.reason}', status=502)
except Exception as e:
logger.error(f'Streaming error: {e}', exc_info=True)
return Response(f'Error: {e}', status=500)

393
youtube/ytdlp_service.py Normal file
View File

@@ -0,0 +1,393 @@
#!/usr/bin/env python3
"""
Centralized yt-dlp integration with caching, logging, and error handling.
This module provides a clean interface for yt-dlp functionality:
- Multi-language audio track extraction
- Subtitle extraction
- Age-restricted video support
All yt-dlp usage should go through this module for consistency.
"""
import logging
from functools import lru_cache
from typing import Dict, List, Optional, Any
import yt_dlp
import settings
logger = logging.getLogger(__name__)
# Language name mapping
LANGUAGE_NAMES = {
'en': 'English',
'es': 'Español',
'fr': 'Français',
'de': 'Deutsch',
'it': 'Italiano',
'pt': 'Português',
'ru': 'Русский',
'ja': '日本語',
'ko': '한국어',
'zh': '中文',
'ar': 'العربية',
'hi': 'हिन्दी',
'und': 'Unknown',
'zxx': 'No linguistic content',
}
def get_language_name(lang_code: str) -> str:
"""Convert ISO 639-1/2 language code to readable name."""
if not lang_code:
return 'Unknown'
return LANGUAGE_NAMES.get(lang_code.lower(), lang_code.upper())
def _get_ytdlp_config() -> Dict[str, Any]:
"""Get yt-dlp configuration from settings."""
config = {
'quiet': True,
'no_warnings': True,
'extract_flat': False,
'format': 'best',
'skip_download': True,
'socket_timeout': 30,
'extractor_retries': 3,
'http_chunk_size': 10485760, # 10MB
}
# Configure Tor proxy if enabled
if settings.route_tor:
config['proxy'] = 'socks5://127.0.0.1:9150'
logger.debug('Tor proxy enabled for yt-dlp')
# Use cookies if available
import os
cookies_file = 'youtube_cookies.txt'
if os.path.exists(cookies_file):
config['cookiefile'] = cookies_file
logger.debug('Using cookies file for yt-dlp')
return config
@lru_cache(maxsize=128)
def extract_video_info(video_id: str) -> Dict[str, Any]:
"""
Extract video information using yt-dlp with caching.
Args:
video_id: YouTube video ID
Returns:
Dictionary with video information including audio tracks
Caching:
Results are cached to avoid repeated requests to YouTube.
Cache size is limited to prevent memory issues.
"""
# Check if yt-dlp is enabled
if not getattr(settings, 'ytdlp_enabled', True):
logger.debug('yt-dlp integration is disabled')
return {'error': 'yt-dlp disabled', 'audio_tracks': []}
url = f'https://www.youtube.com/watch?v={video_id}'
ydl_opts = _get_ytdlp_config()
try:
logger.debug(f'Extracting video info: {video_id}')
with yt_dlp.YoutubeDL(ydl_opts) as ydl:
info = ydl.extract_info(url, download=False)
if not info:
logger.warning(f'No info returned for video: {video_id}')
return {'error': 'No info returned', 'audio_tracks': []}
logger.info(f'Extracted {len(info.get("formats", []))} total formats')
# Extract audio tracks grouped by language
audio_tracks = _extract_audio_tracks(info)
return {
'video_id': video_id,
'title': info.get('title', ''),
'duration': info.get('duration', 0),
'audio_tracks': audio_tracks,
'formats': info.get('formats', []),
'subtitles': info.get('subtitles', {}),
'automatic_captions': info.get('automatic_captions', {}),
}
except yt_dlp.utils.DownloadError as e:
logger.error(f'yt-dlp download error for {video_id}: {e}')
return {'error': str(e), 'audio_tracks': []}
except Exception as e:
logger.error(f'yt-dlp extraction error for {video_id}: {e}', exc_info=True)
return {'error': str(e), 'audio_tracks': []}
def _extract_audio_tracks(info: Dict[str, Any]) -> List[Dict[str, Any]]:
"""
Extract audio tracks from video info, grouped by language.
Returns a list of unique audio tracks (one per language),
keeping the highest quality for each language.
"""
audio_by_language = {}
all_formats = info.get('formats', [])
logger.debug(f'Processing {len(all_formats)} formats to extract audio tracks')
for fmt in all_formats:
# Only audio-only formats
has_audio = fmt.get('acodec') and fmt.get('acodec') != 'none'
has_video = fmt.get('vcodec') and fmt.get('vcodec') != 'none'
if not has_audio or has_video:
continue
# Extract language information
lang = (
fmt.get('language') or
fmt.get('audio_language') or
fmt.get('lang') or
'und'
)
# Get language name
lang_name = (
fmt.get('language_name') or
fmt.get('lang_name') or
get_language_name(lang)
)
# Get bitrate
bitrate = fmt.get('abr') or fmt.get('tbr') or 0
# Create track info
track_info = {
'language': lang,
'language_name': lang_name,
'format_id': str(fmt.get('format_id', '')),
'itag': str(fmt.get('format_id', '')),
'ext': fmt.get('ext'),
'acodec': fmt.get('acodec'),
'audio_bitrate': int(bitrate) if bitrate else 0,
'audio_sample_rate': fmt.get('asr'),
'url': fmt.get('url'),
'filesize': fmt.get('filesize'),
}
# Keep best quality per language
lang_key = lang.lower()
if lang_key not in audio_by_language:
audio_by_language[lang_key] = track_info
logger.debug(f' Added {lang} ({lang_name}) - {bitrate}k')
else:
current_bitrate = audio_by_language[lang_key].get('audio_bitrate', 0)
if bitrate > current_bitrate:
logger.debug(f' Updated {lang} ({lang_name}): {current_bitrate}k → {bitrate}k')
audio_by_language[lang_key] = track_info
# Convert to list and sort
audio_tracks = list(audio_by_language.values())
# Sort: English first, then by bitrate (descending)
audio_tracks.sort(
key=lambda x: (
0 if x['language'] == 'en' else 1,
-x.get('audio_bitrate', 0)
)
)
logger.info(f'Extracted {len(audio_tracks)} unique audio languages')
for track in audio_tracks[:5]: # Log first 5
logger.info(f'{track["language_name"]} ({track["language"]}): {track["audio_bitrate"]}k')
return audio_tracks
def get_subtitle_url(video_id: str, lang: str = 'en') -> Optional[str]:
"""
Get subtitle URL for a specific language.
Args:
video_id: YouTube video ID
lang: Language code (default: 'en')
Returns:
URL to subtitle file, or None if not available
"""
info = extract_video_info(video_id)
if 'error' in info:
logger.warning(f'Cannot get subtitles: {info["error"]}')
return None
# Try manual subtitles first
subtitles = info.get('subtitles', {})
if lang in subtitles:
for sub in subtitles[lang]:
if sub.get('ext') == 'vtt':
logger.debug(f'Found manual {lang} subtitle')
return sub.get('url')
# Try automatic captions
auto_captions = info.get('automatic_captions', {})
if lang in auto_captions:
for sub in auto_captions[lang]:
if sub.get('ext') == 'vtt':
logger.debug(f'Found automatic {lang} subtitle')
return sub.get('url')
logger.debug(f'No {lang} subtitle found')
return None
def find_best_unified_format(
video_id: str,
audio_language: str = 'en',
max_quality: int = 720
) -> Optional[Dict[str, Any]]:
"""
Find best unified (video+audio) format for specific language and quality.
Args:
video_id: YouTube video ID
audio_language: Preferred audio language
max_quality: Maximum video height (e.g., 720, 1080)
Returns:
Format dict if found, None otherwise
"""
info = extract_video_info(video_id)
if 'error' in info or not info.get('formats'):
return None
# Quality thresholds (minimum acceptable height as % of requested)
thresholds = {
2160: 0.85,
1440: 0.80,
1080: 0.70,
720: 0.70,
480: 0.60,
360: 0.50,
}
# Get threshold for requested quality
threshold = 0.70
for q, t in thresholds.items():
if max_quality >= q:
threshold = t
break
min_height = int(max_quality * threshold)
logger.debug(f'Quality threshold: {threshold:.0%} = min {min_height}p for {max_quality}p')
candidates = []
audio_lang_lower = audio_language.lower()
for fmt in info['formats']:
# Must have both video and audio
has_video = fmt.get('vcodec') and fmt.get('vcodec') != 'none'
has_audio = fmt.get('acodec') and fmt.get('acodec') != 'none'
if not (has_video and has_audio):
continue
# Skip HLS/DASH formats
protocol = fmt.get('protocol', '')
format_id = str(fmt.get('format_id', ''))
if any(x in protocol.lower() for x in ['m3u8', 'hls', 'dash']):
continue
if format_id.startswith('9'): # HLS formats
continue
height = fmt.get('height', 0)
if height < min_height:
continue
# Language matching
lang = (
fmt.get('language') or
fmt.get('audio_language') or
'en'
).lower()
lang_match = (
lang == audio_lang_lower or
lang.startswith(audio_lang_lower[:2]) or
audio_lang_lower.startswith(lang[:2])
)
if not lang_match:
continue
# Calculate score
score = 0
# Language match bonus
if lang == audio_lang_lower:
score += 10000
elif lang.startswith(audio_lang_lower[:2]):
score += 8000
else:
score += 5000
# Quality score
quality_diff = abs(height - max_quality)
if height >= max_quality:
score += 3000 - quality_diff
else:
score += 2000 - quality_diff
# Protocol preference
if protocol in ('https', 'http'):
score += 500
# Format preference
if fmt.get('ext') == 'mp4':
score += 100
candidates.append({
'format': fmt,
'score': score,
'height': height,
'lang': lang,
})
if not candidates:
logger.debug(f'No unified format found for {max_quality}p + {audio_language}')
return None
# Sort by score and return best
candidates.sort(key=lambda x: x['score'], reverse=True)
best = candidates[0]
logger.info(
f'Selected unified format: {best["format"].get("format_id")} | '
f'{best["lang"]} | {best["height"]}p | score={best["score"]}'
)
return best['format']
def clear_cache():
"""Clear the video info cache."""
extract_video_info.cache_clear()
logger.info('yt-dlp cache cleared')
def get_cache_info() -> Dict[str, Any]:
"""Get cache statistics."""
cache_info = extract_video_info.cache_info()
return {
'hits': cache_info.hits,
'misses': cache_info.misses,
'size': cache_info.currsize,
'maxsize': cache_info.maxsize,
'hit_rate': cache_info.hits / (cache_info.hits + cache_info.misses) if (cache_info.hits + cache_info.misses) > 0 else 0,
}