Release v0.4.0 - HD Thumbnails, YouTube 2024+ Support, and yt-dlp Integration

Major Features:
- HD video thumbnails (hq720.jpg) with automatic fallback to lower qualities
- HD channel avatars (240x240 instead of 88x88)
- YouTube 2024+ lockupViewModel support for channel playlists
- youtubei/v1/browse API integration for channel playlist tabs
- yt-dlp integration for multi-language audio and subtitles

Bug Fixes:
- Fixed undefined `abort` import in playlist.py
- Fixed undefined functions in proto.py (encode_varint, bytes_to_hex, succinct_encode)
- Fixed missing `traceback` import in proto_debug.py
- Fixed blurry playlist thumbnails using default.jpg instead of HD versions
- Fixed channel playlists page using deprecated pbj=1 format

Improvements:
- Automatic thumbnail fallback system (hq720 → sddefault → hqdefault → mqdefault → default)
- JavaScript thumbnail_fallback() handler for 404 errors
- Better thumbnail quality across all pages (watch, channel, playlist, subscriptions)
- Consistent HD avatar display for all channel items
- Settings system automatically adds new settings without breaking user config

Files Modified:
- youtube/watch.py - HD thumbnails for related videos and playlist items
- youtube/channel.py - HD thumbnails for channel playlists, youtubei API integration
- youtube/playlist.py - HD thumbnails, fixed abort import
- youtube/util.py - HD thumbnail URLs, avatar HD upgrade, prefix_url improvements
- youtube/comments.py - HD video thumbnail
- youtube/subscriptions.py - HD thumbnails, fixed abort import
- youtube/yt_data_extract/common.py - lockupViewModel support, extract_lockup_view_model_info()
- youtube/yt_data_extract/everything_else.py - HD playlist thumbnails
- youtube/proto.py - Fixed undefined function references
- youtube/proto_debug.py - Added traceback import
- youtube/static/js/common.js - thumbnail_fallback() handler
- youtube/templates/*.html - Added onerror handlers for thumbnail fallback
- youtube/version.py - Bump to v0.4.0

Technical Details:
- All thumbnail URLs now use hq720.jpg (1280x720) when available
- Fallback handled client-side via JavaScript onerror handler
- Server-side avatar upgrade via regex in util.prefix_url()
- lockupViewModel parser extracts contentType, metadata, and first_video_id
- Channel playlist tabs now use youtubei/v1/browse instead of deprecated pbj=1
- Settings version system ensures backward compatibility

.gitea/workflows/ci.yaml

@@ -0,0 +1,23 @@
+name: CI
+
+on: [push, pull_request]
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: 3.11
+
+      - name: Install dependencies
+        run: |
+          pip install --upgrade pip
+          pip install -r requirements-dev.txt
+
+      - name: Run tests
+        run: pytest

.gitea/workflows/git-sync.yaml

@@ -0,0 +1,40 @@
+name: git-sync-with-mirror
+
+on:
+  push:
+    branches: [ master ]
+  workflow_dispatch:
+
+jobs:
+  git-sync:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: git-sync
+        env:
+          git_sync_source_repo: git@git.fridu.us:heckyel/yt-local.git
+          git_sync_destination_repo: ssh://git@c.fridu.us/software/yt-local.git
+        if: env.git_sync_source_repo && env.git_sync_destination_repo
+        uses: astounds/git-sync@v1
+        with:
+          source_repo: git@git.fridu.us:heckyel/yt-local.git
+          source_branch: "master"
+          destination_repo: ssh://git@c.fridu.us/software/yt-local.git
+          destination_branch: "master"
+          source_ssh_private_key: ${{ secrets.GIT_SYNC_SOURCE_SSH_PRIVATE_KEY }}
+          destination_ssh_private_key:  ${{ secrets.GIT_SYNC_DESTINATION_SSH_PRIVATE_KEY }}
+
+      - name: git-sync-sourcehut
+        env:
+          git_sync_source_repo: git@git.fridu.us:heckyel/yt-local.git
+          git_sync_destination_repo: git@git.sr.ht:~heckyel/yt-local
+        if: env.git_sync_source_repo && env.git_sync_destination_repo
+        uses: astounds/git-sync@v1
+        with:
+          source_repo: git@git.fridu.us:heckyel/yt-local.git
+          source_branch: "master"
+          destination_repo: git@git.sr.ht:~heckyel/yt-local
+          destination_branch: "master"
+          source_ssh_private_key: ${{ secrets.GIT_SYNC_SOURCE_SSH_PRIVATE_KEY }}
+          destination_ssh_private_key:  ${{ secrets.GIT_SYNC_DESTINATION_SSH_PRIVATE_KEY }}
+        continue-on-error: true

.gitignore

@@ -1,5 +1,128 @@
+# Byte-compiled / optimized / DLL files
 __pycache__/
+*.py[cod]
 *$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+pip-wheel-metadata/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+
+# Translations
+*.mo
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+.python-version
+
+# pipenv
+Pipfile.lock
+
+# PEP 582
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+*venv*
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# Project specific
 debug/
 data/
 python/
@@ -11,5 +134,17 @@ get-pip.py
 latest-dist.zip
 *.7z
 *.zip
-*venv*
+
+# Editor specific
 flycheck_*
+.vscode/
+.idea/
+*.swp
+*.swo
+*~
+.DS_Store
+
+# Temporary files
+*.tmp
+*.bak
+*.orig

Makefile

@@ -0,0 +1,210 @@
+# yt-local Makefile
+# Automated tasks for development, translations, and maintenance
+
+.PHONY: help install dev clean test i18n-extract i18n-init i18n-update i18n-compile i18n-stats i18n-clean setup-dev lint format backup restore
+
+# Variables
+PYTHON := python3
+PIP := pip3
+LANG_CODE ?= es
+VENV_DIR := venv
+PROJECT_NAME := yt-local
+
+## Help
+help: ## Show this help message
+	@echo "$(PROJECT_NAME) - Available tasks:"
+	@echo ""
+	@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "  %-20s %s\n", $$1, $$2}'
+	@echo ""
+	@echo "Examples:"
+	@echo "  make install          # Install dependencies"
+	@echo "  make dev              # Run development server"
+	@echo "  make i18n-extract     # Extract strings for translation"
+	@echo "  make i18n-init LANG_CODE=fr  # Initialize French"
+	@echo "  make lint             # Check code style"
+
+## Installation and Setup
+install: ## Install project dependencies
+	@echo "[INFO] Installing dependencies..."
+	$(PIP) install -r requirements.txt
+	@echo "[SUCCESS] Dependencies installed"
+
+setup-dev: ## Complete development setup
+	@echo "[INFO] Setting up development environment..."
+	$(PYTHON) -m venv $(VENV_DIR)
+	./$(VENV_DIR)/bin/pip install -r requirements.txt
+	@echo "[SUCCESS] Virtual environment created in $(VENV_DIR)"
+	@echo "[INFO] Activate with: source $(VENV_DIR)/bin/activate"
+
+requirements: ## Update and install requirements
+	@echo "[INFO] Installing/updating requirements..."
+	$(PIP) install --upgrade pip
+	$(PIP) install -r requirements.txt
+	@echo "[SUCCESS] Requirements installed"
+
+## Development
+dev: ## Run development server
+	@echo "[INFO] Starting development server..."
+	@echo "[INFO] Server available at: http://localhost:9010"
+	$(PYTHON) server.py
+
+run: dev ## Alias for dev
+
+## Testing
+test: ## Run tests
+	@echo "[INFO] Running tests..."
+	@if [ -d "tests" ]; then \
+		$(PYTHON) -m pytest -v; \
+	else \
+		echo "[WARN] No tests directory found"; \
+	fi
+
+test-cov: ## Run tests with coverage
+	@echo "[INFO] Running tests with coverage..."
+	@if command -v pytest-cov >/dev/null 2>&1; then \
+		$(PYTHON) -m pytest -v --cov=$(PROJECT_NAME) --cov-report=html; \
+	else \
+		echo "[WARN] pytest-cov not installed. Run: pip install pytest-cov"; \
+	fi
+
+## Internationalization (i18n)
+i18n-extract: ## Extract strings for translation
+	@echo "[INFO] Extracting strings for translation..."
+	$(PYTHON) manage_translations.py extract
+	@echo "[SUCCESS] Strings extracted to translations/messages.pot"
+
+i18n-init: ## Initialize new language (use LANG_CODE=xx)
+	@echo "[INFO] Initializing language: $(LANG_CODE)"
+	$(PYTHON) manage_translations.py init $(LANG_CODE)
+	@echo "[SUCCESS] Language $(LANG_CODE) initialized"
+	@echo "[INFO] Edit: translations/$(LANG_CODE)/LC_MESSAGES/messages.po"
+
+i18n-update: ## Update existing translations
+	@echo "[INFO] Updating existing translations..."
+	$(PYTHON) manage_translations.py update
+	@echo "[SUCCESS] Translations updated"
+
+i18n-compile: ## Compile translations to binary .mo files
+	@echo "[INFO] Compiling translations..."
+	$(PYTHON) manage_translations.py compile
+	@echo "[SUCCESS] Translations compiled"
+
+i18n-stats: ## Show translation statistics
+	@echo "[INFO] Translation statistics:"
+	@echo ""
+	@for lang_dir in translations/*/; do \
+		if [ -d "$$lang_dir" ] && [ "$$lang_dir" != "translations/*/" ]; then \
+			lang=$$(basename "$$lang_dir"); \
+			po_file="$$lang_dir/LC_MESSAGES/messages.po"; \
+			if [ -f "$$po_file" ]; then \
+				total=$$(grep -c "^msgid " "$$po_file" 2>/dev/null || echo "0"); \
+				translated=$$(grep -c "^msgstr \"[^\"]\+\"" "$$po_file" 2>/dev/null || echo "0"); \
+				fuzzy=$$(grep -c "^#, fuzzy" "$$po_file" 2>/dev/null || echo "0"); \
+				if [ "$$total" -gt 0 ]; then \
+					percent=$$((translated * 100 / total)); \
+					echo "  [STAT] $$lang: $$translated/$$total ($$percent%) - Fuzzy: $$fuzzy"; \
+				else \
+					echo "  [STAT] $$lang: No translations yet"; \
+				fi; \
+			fi \
+		fi \
+	done
+	@echo ""
+
+i18n-clean: ## Clean compiled translation files
+	@echo "[INFO] Cleaning compiled .mo files..."
+	find translations/ -name "*.mo" -delete
+	@echo "[SUCCESS] .mo files removed"
+
+i18n-workflow: ## Complete workflow: extract → update → compile
+	@echo "[INFO] Running complete translation workflow..."
+	@make i18n-extract
+	@make i18n-update
+	@make i18n-compile
+	@make i18n-stats
+	@echo "[SUCCESS] Translation workflow completed"
+
+## Code Quality
+lint: ## Check code with flake8
+	@echo "[INFO] Checking code style..."
+	@if command -v flake8 >/dev/null 2>&1; then \
+		flake8 youtube/ --max-line-length=120 --ignore=E501,W503,E402 --exclude=youtube/ytdlp_service.py,youtube/ytdlp_integration.py,youtube/ytdlp_proxy.py; \
+		echo "[SUCCESS] Code style check passed"; \
+	else \
+		echo "[WARN] flake8 not installed (pip install flake8)"; \
+	fi
+
+format: ## Format code with black (if available)
+	@echo "[INFO] Formatting code..."
+	@if command -v black >/dev/null 2>&1; then \
+		black youtube/ --line-length=120 --exclude='ytdlp_.*\.py'; \
+		echo "[SUCCESS] Code formatted"; \
+	else \
+		echo "[WARN] black not installed (pip install black)"; \
+	fi
+
+check-deps: ## Check installed dependencies
+	@echo "[INFO] Checking dependencies..."
+	@$(PYTHON) -c "import flask_babel; print('[OK] Flask-Babel:', flask_babel.__version__)" 2>/dev/null || echo "[ERROR] Flask-Babel not installed"
+	@$(PYTHON) -c "import flask; print('[OK] Flask:', flask.__version__)" 2>/dev/null || echo "[ERROR] Flask not installed"
+	@$(PYTHON) -c "import yt_dlp; print('[OK] yt-dlp:', yt_dlp.__version__)" 2>/dev/null || echo "[ERROR] yt-dlp not installed"
+
+## Maintenance
+backup: ## Create translations backup
+	@echo "[INFO] Creating translations backup..."
+	@timestamp=$$(date +%Y%m%d_%H%M%S); \
+	tar -czf "translations_backup_$$timestamp.tar.gz" translations/ 2>/dev/null || echo "[WARN] No translations to backup"; \
+	if [ -f "translations_backup_$$timestamp.tar.gz" ]; then \
+		echo "[SUCCESS] Backup created: translations_backup_$$timestamp.tar.gz"; \
+	fi
+
+restore: ## Restore translations from backup
+	@echo "[INFO] Restoring translations from backup..."
+	@if ls translations_backup_*.tar.gz 1>/dev/null 2>&1; then \
+		latest_backup=$$(ls -t translations_backup_*.tar.gz | head -1); \
+		tar -xzf "$$latest_backup"; \
+		echo "[SUCCESS] Restored from: $$latest_backup"; \
+	else \
+		echo "[ERROR] No backup files found"; \
+	fi
+
+clean: ## Clean temporary files and caches
+	@echo "[INFO] Cleaning temporary files..."
+	find . -type f -name "*.pyc" -delete
+	find . -type d -name "__pycache__" -delete
+	find . -type f -name "*.mo" -delete
+	find . -type d -name ".pytest_cache" -delete
+	find . -type f -name ".coverage" -delete
+	find . -type d -name "htmlcov" -delete
+	@echo "[SUCCESS] Temporary files removed"
+
+distclean: clean ## Clean everything including venv
+	@echo "[INFO] Cleaning everything..."
+	rm -rf $(VENV_DIR)
+	@echo "[SUCCESS] Complete cleanup done"
+
+## Project Information
+info: ## Show project information
+	@echo "[INFO] $(PROJECT_NAME) - Project information:"
+	@echo ""
+	@echo "  [INFO] Directory: $$(pwd)"
+	@echo "  [INFO] Python: $$($(PYTHON) --version)"
+	@echo "  [INFO] Pip: $$($(PIP) --version | cut -d' ' -f1-2)"
+	@echo ""
+	@echo "  [INFO] Configured languages:"
+	@for lang_dir in translations/*/; do \
+		if [ -d "$$lang_dir" ] && [ "$$lang_dir" != "translations/*/" ]; then \
+			lang=$$(basename "$$lang_dir"); \
+			echo "    - $$lang"; \
+		fi \
+	done
+	@echo ""
+	@echo "  [INFO] Main files:"
+	@echo "    - babel.cfg (i18n configuration)"
+	@echo "    - manage_translations.py (i18n CLI)"
+	@echo "    - youtube/i18n_strings.py (centralized strings)"
+	@echo "    - youtube/ytdlp_service.py (yt-dlp integration)"
+	@echo ""
+
+# Default target
+.DEFAULT_GOAL := help

README.md

@@ -1,5 +1,3 @@
-[![Build Status](https://drone.hgit.ga/api/badges/heckyel/yt-local/status.svg)](https://drone.hgit.ga/heckyel/yt-local)
-
 # yt-local
 
 Fork of [youtube-local](https://github.com/user234683/youtube-local)
@@ -153,7 +151,7 @@ For coding guidelines and an overview of the software architecture, see the [HAC
 
 yt-local is not made to work in public mode, however there is an instance of yt-local in public mode but with less features
 
-- <https://1cd1-93-95-230-133.ngrok-free.app/https://youtube.com>
+- <https://m.fridu.us/https://youtube.com>
 
 ## License
 

babel.cfg

@@ -0,0 +1,7 @@
+[python: youtube/**.py]
+keywords = lazy_gettext:1,2 _l:1,2
+[python: server.py]
+[python: settings.py]
+[jinja2: youtube/templates/**.html]
+extensions=jinja2.ext.i18n
+encoding = utf-8

generate_release.py

@@ -114,10 +114,12 @@ if bitness == '32':
     visual_c_runtime_url = 'https://github.com/yuempek/vc-archive/raw/master/archives/vc15_(14.10.25017.0)_2017_x86.7z'
     visual_c_runtime_sha256 = '2549eb4d2ce4cf3a87425ea01940f74368bf1cda378ef8a8a1f1a12ed59f1547'
     visual_c_name = 'vc15_(14.10.25017.0)_2017_x86.7z'
+    visual_c_path_to_dlls = 'runtime_minimum/System'
 else:
     visual_c_runtime_url = 'https://github.com/yuempek/vc-archive/raw/master/archives/vc15_(14.10.25017.0)_2017_x64.7z'
     visual_c_runtime_sha256 = '4f00b824c37e1017a93fccbd5775e6ee54f824b6786f5730d257a87a3d9ce921'
     visual_c_name = 'vc15_(14.10.25017.0)_2017_x64.7z'
+    visual_c_path_to_dlls = 'runtime_minimum/System64'
 
 download_if_not_exists('get-pip.py', get_pip_url)
 
@@ -198,7 +200,7 @@ with open('./python/python3' + major_release + '._pth', 'a', encoding='utf-8') a
     f.write('..\n')'''
 
 log('Inserting Microsoft C Runtime')
-check_subp(subprocess.run([r'7z', '-y', 'e', '-opython', 'vc15_(14.10.25017.0)_2017_x86.7z', 'runtime_minimum/System']))
+check_subp(subprocess.run([r'7z', '-y', 'e', '-opython', visual_c_name, visual_c_path_to_dlls]))
 
 log('Installing dependencies')
 wine_run(['./python/python.exe', '-I', '-m', 'pip', 'install', '--no-compile', '-r', './requirements.txt'])

manage_translations.py

@@ -0,0 +1,113 @@
+#!/usr/bin/env python3
+"""
+Translation management script for yt-local
+
+Usage:
+    python manage_translations.py extract  # Extract strings to messages.pot
+    python manage_translations.py init es  # Initialize Spanish translation
+    python manage_translations.py update   # Update all translations
+    python manage_translations.py compile  # Compile translations to .mo files
+"""
+import sys
+import os
+import subprocess
+
+# Ensure we use the Python from the virtual environment if available
+if hasattr(sys, 'real_prefix') or (hasattr(sys, 'base_prefix') and sys.base_prefix != sys.prefix):
+    # Already in venv
+    pass
+else:
+    # Try to activate venv
+    venv_path = os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), 'venv')
+    if os.path.exists(venv_path):
+        venv_bin = os.path.join(venv_path, 'bin')
+        if os.path.exists(venv_bin):
+            os.environ['PATH'] = venv_bin + os.pathsep + os.environ['PATH']
+
+
+def run_command(cmd):
+    """Run a shell command and print output"""
+    print(f"Running: {' '.join(cmd)}")
+    # Use the pybabel from the same directory as our Python executable
+    if cmd[0] == 'pybabel':
+        import os
+        pybabel_path = os.path.join(os.path.dirname(sys.executable), 'pybabel')
+        if os.path.exists(pybabel_path):
+            cmd = [pybabel_path] + cmd[1:]
+    result = subprocess.run(cmd, capture_output=True, text=True)
+    if result.stdout:
+        print(result.stdout)
+    if result.stderr:
+        print(result.stderr, file=sys.stderr)
+    return result.returncode
+
+
+def extract():
+    """Extract translatable strings from source code"""
+    print("Extracting translatable strings...")
+    return run_command([
+        'pybabel', 'extract',
+        '-F', 'babel.cfg',
+        '-k', 'lazy_gettext',
+        '-k', '_l',
+        '-o', 'translations/messages.pot',
+        '.'
+    ])
+
+
+def init(language):
+    """Initialize a new language translation"""
+    print(f"Initializing {language} translation...")
+    return run_command([
+        'pybabel', 'init',
+        '-i', 'translations/messages.pot',
+        '-d', 'translations',
+        '-l', language
+    ])
+
+
+def update():
+    """Update existing translations with new strings"""
+    print("Updating translations...")
+    return run_command([
+        'pybabel', 'update',
+        '-i', 'translations/messages.pot',
+        '-d', 'translations'
+    ])
+
+
+def compile_translations():
+    """Compile .po files to .mo files"""
+    print("Compiling translations...")
+    return run_command([
+        'pybabel', 'compile',
+        '-d', 'translations'
+    ])
+
+
+def main():
+    if len(sys.argv) < 2:
+        print(__doc__)
+        sys.exit(1)
+
+    command = sys.argv[1]
+
+    if command == 'extract':
+        sys.exit(extract())
+    elif command == 'init':
+        if len(sys.argv) < 3:
+            print("Error: Please specify a language code (e.g., es, fr, de)")
+            sys.exit(1)
+        sys.exit(init(sys.argv[2]))
+    elif command == 'update':
+        sys.exit(update())
+    elif command == 'compile':
+        sys.exit(compile_translations())
+    else:
+        print(f"Unknown command: {command}")
+        print(__doc__)
+        sys.exit(1)
+
+
+if __name__ == '__main__':
+    main()

requirements-dev.txt

@@ -1,28 +1,5 @@
-attrs==22.1.0
-Brotli==1.0.9
-cachetools==4.2.4
-click==8.0.4
-dataclasses==0.6
-defusedxml==0.7.1
-Flask==2.0.1
-gevent==22.10.2
-greenlet==2.0.1
-importlib-metadata==4.6.4
-iniconfig==1.1.1
-itsdangerous==2.0.1
-Jinja2==3.0.3
-MarkupSafe==2.0.1
-packaging==20.9
-pluggy>=0.13.1
-py==1.10.0
-pyparsing==2.4.7
-PySocks==1.7.1
-pytest==6.2.5
-stem==1.8.0
-toml==0.10.2
-typing-extensions==3.10.0.2
-urllib3==1.26.11
-Werkzeug==2.1.1
-zipp==3.5.1
-zope.event==4.5.0
-zope.interface==5.4.0
+# Include all production requirements
+-r requirements.txt
+
+# Development requirements
+pytest>=6.2.1

requirements.txt

@@ -1,20 +1,12 @@
-Brotli==1.0.9
-cachetools==4.2.4
-click==8.0.4
-dataclasses==0.6
-defusedxml==0.7.1
-Flask==2.0.1
-gevent==22.10.2
-greenlet==2.0.1
-importlib-metadata==4.6.4
-itsdangerous==2.0.1
-Jinja2==3.0.3
-MarkupSafe==2.0.1
-PySocks==1.7.1
-stem==1.8.0
-typing-extensions==3.10.0.2
-urllib3==1.26.11
-Werkzeug==2.1.1
-zipp==3.5.1
-zope.event==4.5.0
-zope.interface==5.4.0
+Flask>=1.0.3
+Flask-Babel>=4.0.0
+Babel>=2.12.0
+gevent>=1.2.2
+Brotli>=1.0.7
+PySocks>=1.6.8
+urllib3>=1.24.1
+defusedxml>=0.5.0
+cachetools>=4.0.0
+stem>=1.8.0
+yt-dlp>=2026.01.01
+requests>=2.25.0

server.py

@@ -84,7 +84,7 @@ def proxy_site(env, start_response, video=False):
         else:
             response, cleanup_func = util.fetch_url_response(url, send_headers)
 
-        response_headers = response.getheaders()
+        response_headers = response.headers
         if isinstance(response_headers, urllib3._collections.HTTPHeaderDict):
             response_headers = response_headers.items()
         if video:
@@ -279,6 +279,16 @@ if __name__ == '__main__':
 
     print('Starting httpserver at http://%s:%s/' %
           (ip_server, settings.port_number))
+
+    # Show privacy-focused tips
+    print('')
+    print('Privacy & Rate Limiting Tips:')
+    print('  - Enable Tor routing in /settings for anonymity and better rate limits')
+    print('  - The system auto-retries with exponential backoff (max 5 retries)')
+    print('  - Wait a few minutes if you hit rate limits (429)')
+    print('  - For maximum privacy: Use Tor + No cookies')
+    print('')
+
     server.serve_forever()
 
 # for uwsgi, gunicorn, etc.

settings.py

@@ -151,6 +151,13 @@ For security reasons, enabling this is not recommended.''',
         'category': 'interface',
     }),
 
+    ('autoplay_videos', {
+        'type': bool,
+        'default': False,
+        'comment': '',
+        'category': 'playback',
+    }),
+
     ('default_resolution', {
         'type': int,
         'default': 720,
@@ -200,12 +207,17 @@ For security reasons, enabling this is not recommended.''',
     }),
 
     ('prefer_uni_sources', {
-        'label': 'Prefer integrated sources',
-        'type': bool,
-        'default': False,
+        'label': 'Use integrated sources',
+        'type': int,
+        'default': 1,
         'comment': '',
+        'options': [
+            (0, 'Prefer not'),
+            (1, 'Prefer'),
+            (2, 'Always'),
+        ],
         'category': 'playback',
-        'description': 'If enabled and the default resolution is set to 360p or 720p, uses the unified (integrated) video files which contain audio and video, with buffering managed by the browser. If disabled, always uses the separate audio and video files through custom buffer management in av-merge via MediaSource.',
+        'description': 'If set to Prefer or Always and the default resolution is set to 360p or 720p, uses the unified (integrated) video files which contain audio and video, with buffering managed by the browser. If set to prefer not, uses the separate audio and video files through custom buffer management in av-merge via MediaSource unless they are unavailable.',
     }),
 
     ('use_video_player', {
@@ -284,6 +296,17 @@ Archive: https://archive.ph/OZQbN''',
         'category': 'interface',
     }),
 
+    ('language', {
+        'type': str,
+        'default': 'en',
+        'comment': 'Interface language',
+        'options': [
+            ('en', 'English'),
+            ('es', 'Español'),
+        ],
+        'category': 'interface',
+    }),
+
     ('embed_page_mode', {
         'type': bool,
         'label': 'Enable embed page',
@@ -298,11 +321,16 @@ Archive: https://archive.ph/OZQbN''',
         'comment': '',
     }),
 
-    ('gather_googlevideo_domains', {
+    ('include_shorts_in_subscriptions', {
         'type': bool,
-        'default': False,
-        'comment': '''Developer use to debug 403s''',
-        'hidden': True,
+        'default': 0,
+        'comment': '',
+    }),
+
+    ('include_shorts_in_channel', {
+        'type': bool,
+        'default': 1,
+        'comment': '',
     }),
 
     ('debugging_save_responses', {
@@ -312,9 +340,18 @@ Archive: https://archive.ph/OZQbN''',
         'hidden': True,
     }),
 
+    ('ytdlp_enabled', {
+        'type': bool,
+        'default': True,
+        'comment': '''Enable yt-dlp integration for multi-language audio and subtitles''',
+        'hidden': False,
+        'label': 'Enable yt-dlp integration',
+        'category': 'playback',
+    }),
+
     ('settings_version', {
         'type': int,
-        'default': 4,
+        'default': 6,
         'comment': '''Do not change, remove, or comment out this value, or else your settings may be lost or corrupted''',
         'hidden': True,
     }),
@@ -387,10 +424,28 @@ def upgrade_to_4(settings_dict):
     return new_settings
 
 
+def upgrade_to_5(settings_dict):
+    new_settings = settings_dict.copy()
+    if 'prefer_uni_sources' in settings_dict:
+        new_settings['prefer_uni_sources'] = int(settings_dict['prefer_uni_sources'])
+    new_settings['settings_version'] = 5
+    return new_settings
+
+
+def upgrade_to_6(settings_dict):
+    new_settings = settings_dict.copy()
+    if 'gather_googlevideo_domains' in new_settings:
+        del new_settings['gather_googlevideo_domains']
+    new_settings['settings_version'] = 6
+    return new_settings
+
+
 upgrade_functions = {
     1: upgrade_to_2,
     2: upgrade_to_3,
     3: upgrade_to_4,
+    4: upgrade_to_5,
+    5: upgrade_to_6,
 }
 
 

translations/es/LC_MESSAGES/messages.po

@@ -0,0 +1,74 @@
+# Spanish translations for yt-local.
+# Copyright (C) 2026 yt-local
+# This file is distributed under the same license as the yt-local project.
+#
+msgid ""
+msgstr ""
+"Project-Id-Version: PROJECT VERSION\n"
+"Report-Msgid-Bugs-To: EMAIL@ADDRESS\n"
+"POT-Creation-Date: 2026-03-22 15:05-0500\n"
+"PO-Revision-Date: 2026-03-22 15:06-0500\n"
+"Last-Translator: \n"
+"Language: es\n"
+"Language-Team: es <LL@li.org>\n"
+"Plural-Forms: nplurals=2; plural=(n != 1);\n"
+"MIME-Version: 1.0\n"
+"Content-Type: text/plain; charset=utf-8\n"
+"Content-Transfer-Encoding: 8bit\n"
+"Generated-By: Babel 2.18.0\n"
+
+#: youtube/templates/base.html:38
+msgid "Type to search..."
+msgstr "Escribe para buscar..."
+
+#: youtube/templates/base.html:39
+msgid "Search"
+msgstr "Buscar"
+
+#: youtube/templates/base.html:45
+msgid "Options"
+msgstr "Opciones"
+
+#: youtube/templates/base.html:47
+msgid "Sort by"
+msgstr "Ordenar por"
+
+#: youtube/templates/base.html:50
+msgid "Relevance"
+msgstr "Relevancia"
+
+#: youtube/templates/base.html:54 youtube/templates/base.html:65
+msgid "Upload date"
+msgstr "Fecha de subida"
+
+#: youtube/templates/base.html:58
+msgid "View count"
+msgstr "Número de visualizaciones"
+
+#: youtube/templates/base.html:62
+msgid "Rating"
+msgstr "Calificación"
+
+#: youtube/templates/base.html:68
+msgid "Any"
+msgstr "Cualquiera"
+
+#: youtube/templates/base.html:72
+msgid "Last hour"
+msgstr "Última hora"
+
+#: youtube/templates/base.html:76
+msgid "Today"
+msgstr "Hoy"
+
+#: youtube/templates/base.html:80
+msgid "This week"
+msgstr "Esta semana"
+
+#: youtube/templates/base.html:84
+msgid "This month"
+msgstr "Este mes"
+
+#: youtube/templates/base.html:88
+msgid "This year"
+msgstr "Este año"

translations/messages.pot

@@ -0,0 +1,75 @@
+# Translations template for PROJECT.
+# Copyright (C) 2026 ORGANIZATION
+# This file is distributed under the same license as the PROJECT project.
+# FIRST AUTHOR <EMAIL@ADDRESS>, 2026.
+#
+#, fuzzy
+msgid ""
+msgstr ""
+"Project-Id-Version: PROJECT VERSION\n"
+"Report-Msgid-Bugs-To: EMAIL@ADDRESS\n"
+"POT-Creation-Date: 2026-03-22 15:05-0500\n"
+"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
+"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
+"Language-Team: LANGUAGE <LL@li.org>\n"
+"MIME-Version: 1.0\n"
+"Content-Type: text/plain; charset=utf-8\n"
+"Content-Transfer-Encoding: 8bit\n"
+"Generated-By: Babel 2.18.0\n"
+
+#: youtube/templates/base.html:38
+msgid "Type to search..."
+msgstr ""
+
+#: youtube/templates/base.html:39
+msgid "Search"
+msgstr ""
+
+#: youtube/templates/base.html:45
+msgid "Options"
+msgstr ""
+
+#: youtube/templates/base.html:47
+msgid "Sort by"
+msgstr ""
+
+#: youtube/templates/base.html:50
+msgid "Relevance"
+msgstr ""
+
+#: youtube/templates/base.html:54 youtube/templates/base.html:65
+msgid "Upload date"
+msgstr ""
+
+#: youtube/templates/base.html:58
+msgid "View count"
+msgstr ""
+
+#: youtube/templates/base.html:62
+msgid "Rating"
+msgstr ""
+
+#: youtube/templates/base.html:68
+msgid "Any"
+msgstr ""
+
+#: youtube/templates/base.html:72
+msgid "Last hour"
+msgstr ""
+
+#: youtube/templates/base.html:76
+msgid "Today"
+msgstr ""
+
+#: youtube/templates/base.html:80
+msgid "This week"
+msgstr ""
+
+#: youtube/templates/base.html:84
+msgid "This month"
+msgstr ""
+
+#: youtube/templates/base.html:88
+msgid "This year"
+msgstr ""
+

youtube/__init__.py

@@ -7,12 +7,36 @@ import settings
 import traceback
 import re
 from sys import exc_info
+from flask_babel import Babel
+
 yt_app = flask.Flask(__name__)
 yt_app.config['TEMPLATES_AUTO_RELOAD'] = True
 yt_app.url_map.strict_slashes = False
 # yt_app.jinja_env.trim_blocks = True
 # yt_app.jinja_env.lstrip_blocks = True
 
+# Configure Babel for i18n
+import os
+yt_app.config['BABEL_DEFAULT_LOCALE'] = 'en'
+# Use absolute path for translations directory to avoid issues with package structure changes
+_app_root = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+yt_app.config['BABEL_TRANSLATION_DIRECTORIES'] = os.path.join(_app_root, 'translations')
+
+def get_locale():
+    """Determine the best locale based on user preference or browser settings"""
+    # Check if user has a language preference in settings
+    if hasattr(settings, 'language') and settings.language:
+        locale = settings.language
+        print(f'[i18n] Using user preference: {locale}')
+        return locale
+    # Otherwise, use browser's Accept-Language header
+    # Only match languages with available translations
+    locale = request.accept_languages.best_match(['en', 'es'])
+    print(f'[i18n] Using browser language: {locale}')
+    return locale or 'en'
+
+babel = Babel(yt_app, locale_selector=get_locale)
+
 
 yt_app.add_url_rule('/settings', 'settings_page', settings.settings_page, methods=['POST', 'GET'])
 
@@ -54,7 +78,10 @@ def commatize(num):
     if num is None:
         return ''
     if isinstance(num, str):
-        num = int(num)
+        try:
+            num = int(num)
+        except ValueError:
+            return num
     return '{:,}'.format(num)
 
 
@@ -110,15 +137,28 @@ def error_page(e):
             error_message += '\n\nExit node IP address: ' + exc_info()[1].ip
         return flask.render_template('error.html', error_message=error_message, slim=slim), 502
     elif exc_info()[0] == util.FetchError and exc_info()[1].error_message:
+        # Handle specific error codes with user-friendly messages
+        error_code = exc_info()[1].code
+        error_msg = exc_info()[1].error_message
+
+        if error_code == '400':
+            error_message = (f'Error: Bad Request (400)\n\n{error_msg}\n\n'
+                           'This usually means the URL or parameters are invalid. '
+                           'Try going back and trying a different option.')
+        elif error_code == '404':
+            error_message = 'Error: The page you are looking for isn\'t here.'
+        else:
+            error_message = f'Error: {error_code} - {error_msg}'
+
         return (flask.render_template(
                     'error.html',
-                    error_message=exc_info()[1].error_message,
+                    error_message=error_message,
                     slim=slim
                 ), 502)
     elif (exc_info()[0] == util.FetchError
         and exc_info()[1].code == '404'
     ):
-        error_message = ('Error: The page you are looking for isn\'t here. ¯\_(ツ)_/¯')
+        error_message = ('Error: The page you are looking for isn\'t here.')
         return flask.render_template('error.html',
                                      error_code=exc_info()[1].code,
                                      error_message=error_message,

youtube/channel.py

@@ -1,6 +1,8 @@
 import base64
-from youtube import util, yt_data_extract, local_playlist, subscriptions
+from youtube import (util, yt_data_extract, local_playlist, subscriptions,
+                     playlist)
 from youtube import yt_app
+import settings
 
 import urllib
 import json
@@ -31,57 +33,113 @@ headers_mobile = (
 real_cookie = (('Cookie', 'VISITOR_INFO1_LIVE=8XihrAcN1l4'),)
 generic_cookie = (('Cookie', 'VISITOR_INFO1_LIVE=ST1Ti53r4fU'),)
 
-# added an extra nesting under the 2nd base64 compared to v4
-# added tab support
-# changed offset field to uint id 1
+# FIXED 2026: YouTube changed continuation token structure (from Invidious commit a9f8127)
+# Sort values for YouTube API (from Invidious): 2=popular, 4=newest, 5=oldest
 def channel_ctoken_v5(channel_id, page, sort, tab, view=1):
-    new_sort = (2 if int(sort) == 1 else 1)
+    # Map sort values to YouTube API values (Invidious values)
+    # Input: sort=3 (newest), sort=4 (newest no shorts)
+    # YouTube expects: 4=newest
+    sort_mapping = {'1': 2, '2': 5, '3': 4, '4': 4}  # 4 is newest without shorts
+    new_sort = sort_mapping.get(sort, 4)
+
     offset = 30*(int(page) - 1)
-    if tab == 'videos':
-        tab = 15
-    elif tab == 'shorts':
-        tab = 10
-    elif tab == 'streams':
-        tab = 14
+
+    # Build continuation token using Invidious structure
+    # The structure is: base64(protobuf({
+    #   80226972: {
+    #     2: channel_id,
+    #     3: base64(protobuf({
+    #       110: {
+    #         3: {
+    #           tab: {
+    #             1: {
+    #               1: base64(protobuf({
+    #                 1: base64(protobuf({
+    #                   2: "ST:" + base64(offset_varint)
+    #                 }))
+    #               }))
+    #             },
+    #             2: base64(protobuf({1: UUID}))
+    #             4: sort_value
+    #             8: base64(protobuf({
+    #               1: UUID
+    #               3: sort_value
+    #             }))
+    #           }
+    #         }
+    #       }
+    #     }))
+    #   }
+    # }))
+
+    # UUID placeholder
+    uuid_proto = proto.string(1, "00000000-0000-0000-0000-000000000000")
+
+    # Offset encoding
+    offset_varint = proto.uint(1, offset)
+    offset_encoded = proto.string(2, proto.unpadded_b64encode(offset_varint))
+    offset_wrapper = proto.string(1, proto.unpadded_b64encode(offset_encoded))
+    offset_base = proto.string(1, proto.unpadded_b64encode(offset_wrapper))
+
+    # Sort value varint
+    sort_varint = proto.uint(4, new_sort)
+
+    # Embedded message with UUID and sort
+    embedded_inner = uuid_proto + proto.uint(3, new_sort)
+    embedded_encoded = proto.string(8, proto.unpadded_b64encode(embedded_inner))
+
+    # Combine: uuid_wrapper + sort_varint + embedded
+    tab_inner_content = offset_base + uuid_proto + sort_varint + embedded_encoded
+
+    tab_inner = proto.string(1, proto.unpadded_b64encode(tab_inner_content))
+    tab_wrapper = proto.string(tab, tab_inner)
+
+    inner_container = proto.string(3, tab_wrapper)
+    outer_container = proto.string(110, inner_container)
+
+    encoded_inner = proto.percent_b64encode(outer_container)
+
     pointless_nest = proto.string(80226972,
         proto.string(2, channel_id)
-        + proto.string(3,
-            proto.percent_b64encode(
-                proto.string(110,
-                    proto.string(3,
-                        proto.string(tab,
-                            proto.string(1,
-                                proto.string(1,
-                                    proto.unpadded_b64encode(
-                                        proto.string(1,
-                                        proto.string(1,
-                                            proto.unpadded_b64encode(
-                                                proto.string(2,
-                                                    b"ST:"
-                                                    + proto.unpadded_b64encode(
-                                                        proto.uint(1, offset)
-                                                    )
-                                                )
-                                            )
-                                        )
-                                        )
-                                    )
-                                )
-                                 # targetId, just needs to be present but
-                                 # doesn't need to be correct
-                                + proto.string(2, "63faaff0-0000-23fe-80f0-582429d11c38")
-                            )
-                            # 1 - newest, 2 - popular
-                            + proto.uint(3, new_sort)
-                        )
-                    )
-                )
-            )
-        )
+        + proto.string(3, encoded_inner)
     )
 
     return base64.urlsafe_b64encode(pointless_nest).decode('ascii')
 
+
+def channel_about_ctoken(channel_id):
+    return proto.make_protobuf(
+        ('base64p',
+         [
+          [2, 80226972,
+           [
+            [2, 2, channel_id],
+            [2, 3,
+             ('base64p',
+              [
+               [2, 110,
+                [
+                 [2, 3,
+                  [
+                   [2, 19,
+                    [
+                     [2, 1, b'66b0e9e9-0000-2820-9589-582429a83980'],
+                    ]
+                   ],
+                  ]
+                 ],
+                ]
+               ],
+              ]
+             )
+            ],
+           ]
+          ],
+         ]
+        )
+    )
+
+
 # https://github.com/user234683/youtube-local/issues/151
 def channel_ctoken_v4(channel_id, page, sort, tab, view=1):
     new_sort = (2 if int(sort) == 1 else 1)
@@ -125,11 +183,6 @@ def channel_ctoken_v4(channel_id, page, sort, tab, view=1):
 
 # SORT:
 # videos:
-#    Popular - 1
-#    Oldest - 2
-#    Newest - 3
-# playlists:
-#    Oldest - 2
 #    Newest - 3
 #    Last video added - 4
 
@@ -228,7 +281,7 @@ def get_channel_tab(channel_id, page="1", sort=3, tab='videos', view=1,
                 'hl': 'en',
                 'gl': 'US',
                 'clientName': 'WEB',
-                'clientVersion': '2.20180830',
+                'clientVersion': '2.20240327.00.00',
             },
         },
         'continuation': ctoken,
@@ -243,7 +296,8 @@ def get_channel_tab(channel_id, page="1", sort=3, tab='videos', view=1,
 
 
 # cache entries expire after 30 minutes
-@cachetools.func.ttl_cache(maxsize=128, ttl=30*60)
+number_of_videos_cache = cachetools.TTLCache(128, 30*60)
+@cachetools.cached(number_of_videos_cache)
 def get_number_of_videos_channel(channel_id):
     if channel_id is None:
         return 1000
@@ -255,7 +309,7 @@ def get_number_of_videos_channel(channel_id):
     try:
         response = util.fetch_url(url, headers_mobile,
             debug_name='number_of_videos', report_text='Got number of videos')
-    except urllib.error.HTTPError as e:
+    except (urllib.error.HTTPError, util.FetchError) as e:
         traceback.print_exc()
         print("Couldn't retrieve number of videos")
         return 1000
@@ -268,11 +322,14 @@ def get_number_of_videos_channel(channel_id):
         return int(match.group(1).replace(',',''))
     else:
         return 0
+def set_cached_number_of_videos(channel_id, num_videos):
+    @cachetools.cached(number_of_videos_cache)
+    def dummy_func_using_same_cache(channel_id):
+        return num_videos
+    dummy_func_using_same_cache(channel_id)
 
 
 channel_id_re = re.compile(r'videos\.xml\?channel_id=([a-zA-Z0-9_-]{24})"')
-
-
 @cachetools.func.lru_cache(maxsize=128)
 def get_channel_id(base_url):
     # method that gives the smallest possible response at ~4 kb
@@ -331,7 +388,7 @@ def get_channel_search_json(channel_id, query, page):
                 'hl': 'en',
                 'gl': 'US',
                 'clientName': 'WEB',
-                'clientVersion': '2.20180830',
+                'clientVersion': '2.20240327.00.00',
             },
         },
         'continuation': ctoken,
@@ -349,19 +406,34 @@ def post_process_channel_info(info):
     info['avatar'] = util.prefix_url(info['avatar'])
     info['channel_url'] = util.prefix_url(info['channel_url'])
     for item in info['items']:
+        # For playlists, use first_video_id for thumbnail, not playlist id
+        if item.get('type') == 'playlist' and item.get('first_video_id'):
+            item['thumbnail'] = "https://i.ytimg.com/vi/{}/hq720.jpg".format(item['first_video_id'])
+        elif item.get('type') == 'video':
+            item['thumbnail'] = "https://i.ytimg.com/vi/{}/hq720.jpg".format(item['id'])
+        # For channels and other types, keep existing thumbnail
         util.prefix_urls(item)
         util.add_extra_html_info(item)
     if info['current_tab'] == 'about':
         for i, (text, url) in enumerate(info['links']):
-            if util.YOUTUBE_URL_RE.fullmatch(url):
+            if isinstance(url, str) and util.YOUTUBE_URL_RE.fullmatch(url):
                 info['links'][i] = (text, util.prefix_url(url))
 
 
-def get_channel_first_page(base_url=None, channel_id=None, tab='videos'):
+def get_channel_first_page(base_url=None, tab='videos', channel_id=None, sort=None):
     if channel_id:
         base_url = 'https://www.youtube.com/channel/' + channel_id
-    return util.fetch_url(base_url + '/' + tab + '?pbj=1&view=0',
-                          headers_desktop, debug_name='gen_channel_' + tab)
+
+    # Build URL with sort parameter
+    # YouTube URL sort params: p=popular, dd=newest, lad=newest no shorts
+    # Note: 'da' (oldest) was removed by YouTube in January 2026
+    url = base_url + '/' + tab + '?pbj=1&view=0'
+    if sort:
+        # Map sort values to YouTube's URL parameter values
+        sort_map = {'3': 'dd', '4': 'lad'}
+        url += '&sort=' + sort_map.get(sort, 'dd')
+
+    return util.fetch_url(url, headers_desktop, debug_name='gen_channel_' + tab)
 
 
 playlist_sort_codes = {'2': "da", '3': "dd", '4': "lad"}
@@ -370,45 +442,112 @@ playlist_sort_codes = {'2': "da", '3': "dd", '4': "lad"}
 # youtube.com/user/[username]/[tab]
 # youtube.com/c/[custom]/[tab]
 # youtube.com/[custom]/[tab]
-
-
 def get_channel_page_general_url(base_url, tab, request, channel_id=None):
 
     page_number = int(request.args.get('page', 1))
-    sort = request.args.get('sort', '3')
+    # sort 1: views
+    # sort 2: oldest
+    # sort 4: newest - no shorts (Just a kludge on our end, not internal to yt)
+    default_sort = '3' if settings.include_shorts_in_channel else '4'
+    sort = request.args.get('sort', default_sort)
     view = request.args.get('view', '1')
     query = request.args.get('query', '')
     ctoken = request.args.get('ctoken', '')
-    default_params = (page_number == 1 and sort == '3' and view == '1')
+    include_shorts = (sort != '4')
+    default_params = (page_number == 1 and sort in ('3', '4') and view == '1')
     continuation = bool(ctoken) # whether or not we're using a continuation
+    page_size = 30
+    try_channel_api = True
+    polymer_json = None
 
-    if (tab in ('videos', 'shorts', 'streams') and channel_id and
-        not default_params):
-        tasks = (
-            gevent.spawn(get_number_of_videos_channel, channel_id),
-            gevent.spawn(get_channel_tab, channel_id, page_number, sort,
-                         tab, view, ctoken)
-        )
-        gevent.joinall(tasks)
-        util.check_gevent_exceptions(*tasks)
-        number_of_videos, polymer_json = tasks[0].value, tasks[1].value
-        continuation = True
-    elif tab in ('videos', 'shorts', 'streams'):
+    # Use the special UU playlist which contains all the channel's uploads
+    if tab == 'videos' and sort in ('3', '4'):
+        if not channel_id:
+            channel_id = get_channel_id(base_url)
+        if page_number == 1 and include_shorts:
+            tasks = (
+                gevent.spawn(playlist.playlist_first_page,
+                             'UU' + channel_id[2:],
+                             report_text='Retrieved channel videos'),
+                gevent.spawn(get_metadata, channel_id),
+            )
+            gevent.joinall(tasks)
+            util.check_gevent_exceptions(*tasks)
+
+            # Ignore the metadata for now, it is cached and will be
+            # recalled later
+            pl_json = tasks[0].value
+            pl_info = yt_data_extract.extract_playlist_info(pl_json)
+            number_of_videos = pl_info['metadata']['video_count']
+            if number_of_videos is None:
+                number_of_videos = 1000
+            else:
+                set_cached_number_of_videos(channel_id, number_of_videos)
+        else:
+            tasks = (
+                gevent.spawn(playlist.get_videos, 'UU' + channel_id[2:],
+                             page_number, include_shorts=include_shorts),
+                gevent.spawn(get_metadata, channel_id),
+                gevent.spawn(get_number_of_videos_channel, channel_id),
+            )
+            gevent.joinall(tasks)
+            util.check_gevent_exceptions(*tasks)
+
+            pl_json = tasks[0].value
+            pl_info = yt_data_extract.extract_playlist_info(pl_json)
+            number_of_videos = tasks[2].value
+
+        info = pl_info
+        info['channel_id'] = channel_id
+        info['current_tab'] = 'videos'
+        if info['items']:   # Success
+            page_size = 100
+            try_channel_api = False
+        else:   # Try the first-page method next
+            try_channel_api = True
+
+    # Use the regular channel API
+    if tab in ('shorts', 'streams') or (tab=='videos' and try_channel_api):
         if channel_id:
             num_videos_call = (get_number_of_videos_channel, channel_id)
         else:
             num_videos_call = (get_number_of_videos_general, base_url)
+
+        # For page 1, use the first-page method which won't break
+        # Pass sort parameter directly (2=oldest, 3=newest, etc.)
+        if page_number == 1:
+            # Always use first-page method for page 1 with sort parameter
+            page_call = (get_channel_first_page, base_url, tab, None, sort)
+        else:
+            # For page 2+, we can't paginate without continuation tokens
+            # This is a YouTube limitation, not our bug
+            flask.abort(404, 'Pagination not available for this sort option. YouTube removed this feature.')
+
         tasks = (
             gevent.spawn(*num_videos_call),
-            gevent.spawn(get_channel_first_page, base_url=base_url, tab=tab),
+            gevent.spawn(*page_call),
         )
         gevent.joinall(tasks)
         util.check_gevent_exceptions(*tasks)
         number_of_videos, polymer_json = tasks[0].value, tasks[1].value
+
     elif tab == 'about':
-        polymer_json = util.fetch_url(base_url + '/about?pbj=1', headers_desktop, debug_name='gen_channel_about')
+        # polymer_json = util.fetch_url(base_url + '/about?pbj=1', headers_desktop, debug_name='gen_channel_about')
+        channel_id = get_channel_id(base_url)
+        ctoken = channel_about_ctoken(channel_id)
+        polymer_json = util.call_youtube_api('web', 'browse', {
+            'continuation': ctoken,
+        })
+        continuation=True
     elif tab == 'playlists' and page_number == 1:
-        polymer_json = util.fetch_url(base_url+ '/playlists?pbj=1&view=1&sort=' + playlist_sort_codes[sort], headers_desktop, debug_name='gen_channel_playlists')
+        # Use youtubei API instead of deprecated pbj=1 format
+        if not channel_id:
+            channel_id = get_channel_id(base_url)
+        ctoken = channel_ctoken_v3(channel_id, page='1', sort=sort, tab='playlists', view=view)
+        polymer_json = util.call_youtube_api('web', 'browse', {
+            'continuation': ctoken,
+        })
+        continuation = True
     elif tab == 'playlists':
         polymer_json = get_channel_tab(channel_id, page_number, sort,
                                        'playlists', view)
@@ -418,12 +557,19 @@ def get_channel_page_general_url(base_url, tab, request, channel_id=None):
     elif tab == 'search':
         url = base_url + '/search?pbj=1&query=' + urllib.parse.quote(query, safe='')
         polymer_json = util.fetch_url(url, headers_desktop, debug_name='gen_channel_search')
+    elif tab == 'videos':
+        pass
     else:
         flask.abort(404, 'Unknown channel tab: ' + tab)
 
+    if polymer_json is not None:
+        info = yt_data_extract.extract_channel_info(
+            json.loads(polymer_json), tab, continuation=continuation
+        )
+
+    if info['error'] is not None:
+        return flask.render_template('error.html', error_message=info['error'])
 
-    info = yt_data_extract.extract_channel_info(json.loads(polymer_json), tab,
-                                                continuation=continuation)
     if channel_id:
         info['channel_url'] = 'https://www.youtube.com/channel/' + channel_id
         info['channel_id'] = channel_id
@@ -431,11 +577,11 @@ def get_channel_page_general_url(base_url, tab, request, channel_id=None):
         channel_id = info['channel_id']
 
     # Will have microformat present, cache metadata while we have it
-    if channel_id and default_params:
+    if channel_id and default_params and tab not in ('videos', 'about'):
         metadata = extract_metadata_for_caching(info)
         set_cached_metadata(channel_id, metadata)
     # Otherwise, populate with our (hopefully cached) metadata
-    elif channel_id and info['channel_name'] is None:
+    elif channel_id and info.get('channel_name') is None:
         metadata = get_metadata(channel_id)
         for key, value in metadata.items():
             yt_data_extract.conservative_update(info, key, value)
@@ -448,12 +594,9 @@ def get_channel_page_general_url(base_url, tab, request, channel_id=None):
         for item in info['items']:
             item.update(additional_info)
 
-    if info['error'] is not None:
-        return flask.render_template('error.html', error_message = info['error'])
-
     if tab in ('videos', 'shorts', 'streams'):
         info['number_of_videos'] = number_of_videos
-        info['number_of_pages'] = math.ceil(number_of_videos/30)
+        info['number_of_pages'] = math.ceil(number_of_videos/page_size)
         info['header_playlist_names'] = local_playlist.get_playlist_names()
     if tab in ('videos', 'shorts', 'streams', 'playlists'):
         info['current_sort'] = sort

youtube/comments.py

@@ -78,7 +78,7 @@ def single_comment_ctoken(video_id, comment_id):
 
 def post_process_comments_info(comments_info):
     for comment in comments_info['comments']:
-        comment['author'] = strip_non_ascii(comment['author'])
+        comment['author'] = strip_non_ascii(comment['author']) if comment.get('author') else ""
         comment['author_url'] = concat_or_none(
             '/', comment['author_url'])
         comment['author_avatar'] = concat_or_none(
@@ -97,7 +97,7 @@ def post_process_comments_info(comments_info):
             ctoken = comment['reply_ctoken']
             ctoken, err = proto.set_protobuf_value(
                 ctoken,
-                'base64p', 6, 3, 9, value=250)
+                'base64p', 6, 3, 9, value=200)
             if err:
                 print('Error setting ctoken value:')
                 print(err)
@@ -127,7 +127,7 @@ def post_process_comments_info(comments_info):
             # change max_replies field to 250 in ctoken
             new_ctoken, err = proto.set_protobuf_value(
                 ctoken,
-                'base64p', 6, 3, 9, value=250)
+                'base64p', 6, 3, 9, value=200)
             if err:
                 print('Error setting ctoken value:')
                 print(err)
@@ -150,7 +150,7 @@ def post_process_comments_info(comments_info):
         util.URL_ORIGIN, '/watch?v=', comments_info['video_id'])
     comments_info['video_thumbnail'] = concat_or_none(
         settings.img_prefix, 'https://i.ytimg.com/vi/',
-        comments_info['video_id'], '/hqdefault.jpg'
+        comments_info['video_id'], '/hq720.jpg'
     )
 
 
@@ -189,10 +189,10 @@ def video_comments(video_id, sort=0, offset=0, lc='', secret_key=''):
                 comments_info['error'] += '\n\n' + e.error_message
             comments_info['error'] += '\n\nExit node IP address: %s' % e.ip
         else:
-            comments_info['error'] = 'YouTube blocked the request. IP address: %s' % e.ip
+            comments_info['error'] = 'YouTube blocked the request. Error: %s' % str(e)
 
     except Exception as e:
-        comments_info['error'] = 'YouTube blocked the request. IP address: %s' % e.ip
+        comments_info['error'] = 'YouTube blocked the request. Error: %s' % str(e)
 
     if comments_info.get('error'):
         print('Error retrieving comments for ' + str(video_id) + ':\n' +

youtube/get_app_version/get_app_version.py

@@ -11,17 +11,10 @@ import subprocess
 def app_version():
     def minimal_env_cmd(cmd):
         # make minimal environment
-        env = {}
-        for k in ['SYSTEMROOT', 'PATH']:
-            v = os.environ.get(k)
-            if v is not None:
-                env[k] = v
+        env = {k: os.environ[k] for k in ['SYSTEMROOT', 'PATH'] if k in os.environ}
+        env.update({'LANGUAGE': 'C', 'LANG': 'C', 'LC_ALL': 'C'})
 
-        env['LANGUAGE'] = 'C'
-        env['LANG'] = 'C'
-        env['LC_ALL'] = 'C'
-        out = subprocess.Popen(
-            cmd, stdout=subprocess.PIPE, env=env).communicate()[0]
+        out = subprocess.Popen(cmd, stdout=subprocess.PIPE, env=env).communicate()[0]
         return out
 
     subst_list = {
@@ -31,24 +24,21 @@ def app_version():
     }
 
     if os.system("command -v git > /dev/null 2>&1") != 0:
-        subst_list
-    else:
-        if call(["git", "branch"], stderr=STDOUT,
-                stdout=open(os.devnull, 'w')) != 0:
-            subst_list
-        else:
-            # version
-            describe = minimal_env_cmd(["git", "describe", "--always"])
-            git_revision = describe.strip().decode('ascii')
-            # branch
-            branch = minimal_env_cmd(["git", "branch"])
-            git_branch = branch.strip().decode('ascii').replace('* ', '')
+        return subst_list
 
-            subst_list = {
-                "version": __version__,
-                "branch": git_branch,
-                "commit": git_revision
-            }
+    if call(["git", "branch"], stderr=STDOUT, stdout=open(os.devnull, 'w')) != 0:
+        return subst_list
+
+    describe = minimal_env_cmd(["git", "describe", "--tags", "--always"])
+    git_revision = describe.strip().decode('ascii')
+
+    branch = minimal_env_cmd(["git", "branch"])
+    git_branch = branch.strip().decode('ascii').replace('* ', '')
+
+    subst_list.update({
+        "branch": git_branch,
+        "commit": git_revision
+    })
 
     return subst_list
 

youtube/i18n_strings.py

@@ -0,0 +1,112 @@
+#!/usr/bin/env python3
+"""
+Centralized i18n strings for yt-local
+
+This file contains static strings that need to be translated but are used
+dynamically in templates or generated content. By importing this module,
+these strings get extracted by babel for translation.
+"""
+
+from flask_babel import lazy_gettext as _l
+
+# Settings categories
+CATEGORY_NETWORK = _l('Network')
+CATEGORY_PLAYBACK = _l('Playback')
+CATEGORY_INTERFACE = _l('Interface')
+
+# Common setting labels
+ROUTE_TOR = _l('Route Tor')
+DEFAULT_SUBTITLES_MODE = _l('Default subtitles mode')
+AV1_CODEC_RANKING = _l('AV1 Codec Ranking')
+VP8_VP9_CODEC_RANKING = _l('VP8/VP9 Codec Ranking')
+H264_CODEC_RANKING = _l('H.264 Codec Ranking')
+USE_INTEGRATED_SOURCES = _l('Use integrated sources')
+ROUTE_IMAGES = _l('Route images')
+ENABLE_COMMENTS_JS = _l('Enable comments.js')
+ENABLE_SPONSORBLOCK = _l('Enable SponsorBlock')
+ENABLE_EMBED_PAGE = _l('Enable embed page')
+
+# Setting names (auto-generated from setting keys)
+RELATED_VIDEOS_MODE = _l('Related videos mode')
+COMMENTS_MODE = _l('Comments mode')
+ENABLE_COMMENT_AVATARS = _l('Enable comment avatars')
+DEFAULT_COMMENT_SORTING = _l('Default comment sorting')
+THEATER_MODE = _l('Theater mode')
+AUTOPLAY_VIDEOS = _l('Autoplay videos')
+DEFAULT_RESOLUTION = _l('Default resolution')
+USE_VIDEO_PLAYER = _l('Use video player')
+USE_VIDEO_DOWNLOAD = _l('Use video download')
+PROXY_IMAGES = _l('Proxy images')
+THEME = _l('Theme')
+FONT = _l('Font')
+LANGUAGE = _l('Language')
+EMBED_PAGE_MODE = _l('Embed page mode')
+
+# Common option values
+OFF = _l('Off')
+ON = _l('On')
+DISABLED = _l('Disabled')
+ENABLED = _l('Enabled')
+ALWAYS_SHOWN = _l('Always shown')
+SHOWN_BY_CLICKING_BUTTON = _l('Shown by clicking button')
+NATIVE = _l('Native')
+NATIVE_WITH_HOTKEYS = _l('Native with hotkeys')
+PLYR = _l('Plyr')
+
+# Theme options
+LIGHT = _l('Light')
+GRAY = _l('Gray')
+DARK = _l('Dark')
+
+# Font options
+BROWSER_DEFAULT = _l('Browser default')
+LIBERATION_SERIF = _l('Liberation Serif')
+ARIAL = _l('Arial')
+VERDANA = _l('Verdana')
+TAHOMA = _l('Tahoma')
+
+# Search and filter options
+SORT_BY = _l('Sort by')
+RELEVANCE = _l('Relevance')
+UPLOAD_DATE = _l('Upload date')
+VIEW_COUNT = _l('View count')
+RATING = _l('Rating')
+
+# Time filters
+ANY = _l('Any')
+LAST_HOUR = _l('Last hour')
+TODAY = _l('Today')
+THIS_WEEK = _l('This week')
+THIS_MONTH = _l('This month')
+THIS_YEAR = _l('This year')
+
+# Content types
+TYPE = _l('Type')
+VIDEO = _l('Video')
+CHANNEL = _l('Channel')
+PLAYLIST = _l('Playlist')
+MOVIE = _l('Movie')
+SHOW = _l('Show')
+
+# Duration filters
+DURATION = _l('Duration')
+SHORT_DURATION = _l('Short (< 4 minutes)')
+LONG_DURATION = _l('Long (> 20 minutes)')
+
+# Actions
+SEARCH = _l('Search')
+DOWNLOAD = _l('Download')
+SUBSCRIBE = _l('Subscribe')
+UNSUBSCRIBE = _l('Unsubscribe')
+IMPORT = _l('Import')
+EXPORT = _l('Export')
+SAVE = _l('Save')
+CHECK = _l('Check')
+MUTE = _l('Mute')
+UNMUTE = _l('Unmute')
+
+# Common UI elements
+OPTIONS = _l('Options')
+SETTINGS = _l('Settings')
+ERROR = _l('Error')
+LOADING = _l('loading...')

youtube/playlist.py

@@ -8,16 +8,17 @@ import json
 import string
 import gevent
 import math
-from flask import request
+from flask import request, abort
 import flask
 
 
-def playlist_ctoken(playlist_id, offset):
+def playlist_ctoken(playlist_id, offset, include_shorts=True):
 
     offset = proto.uint(1, offset)
-    # this is just obfuscation as far as I can tell. It doesn't even follow protobuf
     offset = b'PT:' + proto.unpadded_b64encode(offset)
     offset = proto.string(15, offset)
+    if not include_shorts:
+        offset += proto.string(104, proto.uint(2, 1))
 
     continuation_info = proto.string(3, proto.percent_b64encode(offset))
 
@@ -26,47 +27,46 @@ def playlist_ctoken(playlist_id, offset):
 
     return base64.urlsafe_b64encode(pointless_nest).decode('ascii')
 
-# initial request types:
-#   polymer_json: https://m.youtube.com/playlist?list=PLv3TTBr1W_9tppikBxAE_G6qjWdBljBHJ&pbj=1&lact=0
-#   ajax json:    https://m.youtube.com/playlist?list=PLv3TTBr1W_9tppikBxAE_G6qjWdBljBHJ&pbj=1&lact=0 with header X-YouTube-Client-Version: 1.20180418
 
-
-# continuation request types:
-#   polymer_json: https://m.youtube.com/playlist?&ctoken=[...]&pbj=1
-#   ajax json:    https://m.youtube.com/playlist?action_continuation=1&ajax=1&ctoken=[...]
-
-
-headers_1 = (
-    ('Accept', '*/*'),
-    ('Accept-Language', 'en-US,en;q=0.5'),
-    ('X-YouTube-Client-Name', '2'),
-    ('X-YouTube-Client-Version', '2.20180614'),
-)
-
-
-def playlist_first_page(playlist_id, report_text="Retrieved playlist"):
-    url = 'https://m.youtube.com/playlist?list=' + playlist_id + '&pbj=1'
-    content = util.fetch_url(url, util.mobile_ua + headers_1, report_text=report_text, debug_name='playlist_first_page')
-    content = json.loads(content.decode('utf-8'))
+def playlist_first_page(playlist_id, report_text="Retrieved playlist",
+                        use_mobile=False):
+    if use_mobile:
+        url = 'https://m.youtube.com/playlist?list=' + playlist_id + '&pbj=1'
+        content = util.fetch_url(
+            url, util.mobile_xhr_headers,
+            report_text=report_text, debug_name='playlist_first_page'
+        )
+        content = json.loads(content.decode('utf-8'))
+    else:
+        url = 'https://www.youtube.com/playlist?list=' + playlist_id + '&pbj=1'
+        content = util.fetch_url(
+            url, util.desktop_xhr_headers,
+            report_text=report_text, debug_name='playlist_first_page'
+        )
+        content = json.loads(content.decode('utf-8'))
 
     return content
 
 
-#https://m.youtube.com/playlist?itct=CBMQybcCIhMIptj9xJaJ2wIV2JKcCh3Idwu-&ctoken=4qmFsgI2EiRWTFBMT3kwajlBdmxWWlB0bzZJa2pLZnB1MFNjeC0tN1BHVEMaDmVnWlFWRHBEUWxFJTNE&pbj=1
-def get_videos(playlist_id, page):
-
-    url = "https://m.youtube.com/playlist?ctoken=" + playlist_ctoken(playlist_id, (int(page)-1)*20) + "&pbj=1"
-    headers = {
-        'User-Agent': '  Mozilla/5.0 (iPhone; CPU iPhone OS 10_3_1 like Mac OS X) AppleWebKit/603.1.30 (KHTML, like Gecko) Version/10.0 Mobile/14E304 Safari/602.1',
-        'Accept': '*/*',
-        'Accept-Language': 'en-US,en;q=0.5',
-        'X-YouTube-Client-Name': '2',
-        'X-YouTube-Client-Version': '2.20180508',
-    }
+def get_videos(playlist_id, page, include_shorts=True, use_mobile=False,
+               report_text='Retrieved playlist'):
+    # mobile requests return 20 videos per page
+    if use_mobile:
+        page_size = 20
+        headers = util.mobile_xhr_headers
+    # desktop requests return 100 videos per page
+    else:
+        page_size = 100
+        headers = util.desktop_xhr_headers
 
+    url = "https://m.youtube.com/playlist?ctoken="
+    url += playlist_ctoken(playlist_id, (int(page)-1)*page_size,
+                           include_shorts=include_shorts)
+    url += "&pbj=1"
     content = util.fetch_url(
-        url, headers,
-        report_text="Retrieved playlist", debug_name='playlist_videos')
+        url, headers, report_text=report_text,
+        debug_name='playlist_videos'
+    )
 
     info = json.loads(content.decode('utf-8'))
     return info
@@ -85,7 +85,10 @@ def get_playlist_page():
         this_page_json = first_page_json
     else:
         tasks = (
-            gevent.spawn(playlist_first_page, playlist_id, report_text="Retrieved playlist info" ),
+            gevent.spawn(
+                playlist_first_page, playlist_id,
+                report_text="Retrieved playlist info", use_mobile=True
+            ),
             gevent.spawn(get_videos, playlist_id, page)
         )
         gevent.joinall(tasks)
@@ -104,7 +107,7 @@ def get_playlist_page():
         util.prefix_urls(item)
         util.add_extra_html_info(item)
         if 'id' in item:
-            item['thumbnail'] = f"{settings.img_prefix}https://i.ytimg.com/vi/{item['id']}/hqdefault.jpg"
+            item['thumbnail'] = f"{settings.img_prefix}https://i.ytimg.com/vi/{item['id']}/hq720.jpg"
 
         item['url'] += '&list=' + playlist_id
         if item['index']:
@@ -112,13 +115,13 @@ def get_playlist_page():
 
     video_count = yt_data_extract.deep_get(info, 'metadata', 'video_count')
     if video_count is None:
-        video_count = 40
+        video_count = 1000
 
     return flask.render_template(
         'playlist.html',
         header_playlist_names=local_playlist.get_playlist_names(),
         video_list=info.get('items', []),
-        num_pages=math.ceil(video_count/20),
+        num_pages=math.ceil(video_count/100),
         parameters_dictionary=request.args,
 
         **info['metadata']

youtube/proto.py

@@ -113,12 +113,12 @@ def read_protobuf(data):
             length = read_varint(data)
             value = data.read(length)
         elif wire_type == 3:
-            end_bytes = encode_varint((field_number << 3) | 4)
+            end_bytes = varint_encode((field_number << 3) | 4)
             value = read_group(data, end_bytes)
         elif wire_type == 5:
             value = data.read(4)
         else:
-            raise Exception("Unknown wire type: " + str(wire_type) + ", Tag: " + bytes_to_hex(succinct_encode(tag)) + ", at position " + str(data.tell()))
+            raise Exception("Unknown wire type: " + str(wire_type) + " at position " + str(data.tell()))
         yield (wire_type, field_number, value)
 
 
@@ -141,6 +141,17 @@ base64_enc_funcs = {
 
 
 def _make_protobuf(data):
+    '''
+    Input: Recursive list of protobuf objects or base-64 encodings
+    Output: Protobuf bytestring
+    Each protobuf object takes the form [wire_type, field_number, field_data]
+    If a string protobuf has a list/tuple of length 2, this has the form
+    (base64 type, data)
+    The base64 types are
+    - base64 means a base64 encode with equals sign paddings
+    - base64s means a base64 encode without padding
+    - base64p means a url base64 encode with equals signs replaced with %3D
+    '''
     # must be dict mapping field_number to [wire_type, value]
     if isinstance(data, dict):
         new_data = []

youtube/proto_debug.py

@@ -97,6 +97,7 @@ import re
 import time
 import json
 import os
+import traceback
 import pprint
 
 

youtube/static/channel.css

@@ -256,7 +256,8 @@ hr {
     padding-top: 6px;
     text-align: center;
     white-space: nowrap;
-    border: none;
+    border: 1px solid;
+    border-color: var(--button-border);
     border-radius: 0.2rem;
 }
 

youtube/static/dark_theme.css

@@ -1,20 +1,22 @@
 :root {
-    --background:           #212121;
+    --background:           #121113;
     --text:                 #FFFFFF;
-    --secondary-hover:      #73828c;
-    --secondary-focus:      #303030;
-    --secondary-inverse:    #FFF;
+    --secondary-hover:      #222222;
+    --secondary-focus:      #121113;
+    --secondary-inverse:    #FFFFFF;
     --primary-background:   #242424;
-    --secondary-background: #424242;
-    --thumb-background:     #757575;
+    --secondary-background: #222222;
+    --thumb-background:     #222222;
     --link:                 #00B0FF;
     --link-visited:         #40C4FF;
-    --border-bg:            #FFFFFF;
-    --buttom:               #dcdcdb;
-    --buttom-text:          #415462;
-    --button-border:        #91918c;
-    --buttom-hover:         #BBB;
-    --search-text:          #FFF;
-    --time-background:      #212121;
-    --time-text:            #FFF;
+    --border-bg:            #222222;
+    --border-bg-settings:   #000000;
+    --border-bg-license:    #000000;
+    --buttom:               #121113;
+    --buttom-text:          #FFFFFF;
+    --button-border:        #222222;
+    --buttom-hover:         #222222;
+    --search-text:          #FFFFFF;
+    --time-background:      #121113;
+    --time-text:            #FFFFFF;
 }

youtube/static/gray_theme.css

@@ -1,19 +1,21 @@
 :root {
-    --background:            #2d3743;
+    --background:            #2D3743;
     --text:                  #FFFFFF;
-    --secondary-hover:       #73828c;
+    --secondary-hover:       #73828C;
     --secondary-focus:       rgba(115, 130, 140, 0.125);
     --secondary-inverse:     #FFFFFF;
-    --primary-background:    #2d3743;
+    --primary-background:    #2D3743;
     --secondary-background:  #102027;
     --thumb-background:      #35404D;
-    --link:                  #22aaff;
-    --link-visited:          #7755ff;
+    --link:                  #22AAFF;
+    --link-visited:          #7755FF;
     --border-bg:             #FFFFFF;
-    --buttom:                #DCDCDC;
-    --buttom-text:           #415462;
-    --button-border:         #91918c;
-    --buttom-hover:          #BBBBBB;
+    --border-bg-settings:    #FFFFFF;
+    --border-bg-license:     #FFFFFF;
+    --buttom:                #2D3743;
+    --buttom-text:           #FFFFFF;
+    --button-border:         #102027;
+    --buttom-hover:          #102027;
     --search-text:           #FFFFFF;
     --time-background:       #212121;
     --time-text:             #FFFFFF;

youtube/static/js/av-merge.js

@@ -20,6 +20,29 @@
 // TODO: Call abort to cancel in-progress appends?
 
 
+// Buffer sizes for different systems
+const BUFFER_CONFIG = {
+    default: 50 * 10**6,  // 50 megabytes
+    webOS: 20 * 10**6,    // 20 megabytes WebOS (LG)
+    samsungTizen: 20 * 10**6, // 20 megabytes Samsung Tizen OS
+    androidTV: 30 * 10**6, // 30 megabytes Android TV
+    desktop: 50 * 10**6,  // 50 megabytes PC/Mac
+};
+
+function detectSystem() {
+    const userAgent = navigator.userAgent.toLowerCase();
+    if (/webos|lg browser/i.test(userAgent)) {
+        return "webOS";
+    } else if (/tizen/i.test(userAgent)) {
+        return "samsungTizen";
+    } else if (/android tv|smart-tv/i.test(userAgent)) {
+        return "androidTV";
+    } else if (/firefox|chrome|safari|edge/i.test(userAgent)) {
+        return "desktop";
+    } else {
+        return "default";
+    }
+}
 
 function AVMerge(video, srcInfo, startTime){
     this.audioSource = null;
@@ -164,6 +187,8 @@ AVMerge.prototype.printDebuggingInfo = function() {
 }
 
 function Stream(avMerge, source, startTime, avRatio) {
+    const selectedSystem = detectSystem();
+    let baseBufferTarget = BUFFER_CONFIG[selectedSystem] || BUFFER_CONFIG.default;
     this.avMerge = avMerge;
     this.video = avMerge.video;
     this.url = source['url'];
@@ -173,10 +198,11 @@ function Stream(avMerge, source, startTime, avRatio) {
     this.mimeCodec = source['mime_codec']
     this.streamType = source['acodec'] ? 'audio' : 'video';
     if (this.streamType == 'audio') {
-        this.bufferTarget = avRatio*50*10**6;
+        this.bufferTarget = avRatio * baseBufferTarget;
     } else {
-        this.bufferTarget = 50*10**6; // 50 megabytes
+        this.bufferTarget = baseBufferTarget;
     }
+    console.info(`Detected system: ${selectedSystem}. Applying bufferTarget of ${this.bufferTarget} bytes to ${this.streamType}.`);
 
     this.initRange = source['init_range'];
     this.indexRange = source['index_range'];
@@ -204,6 +230,8 @@ Stream.prototype.setup = async function(){
             this.url,
             this.initRange.start,
             this.indexRange.end,
+            'Initialization+index segments',
+        ).then(
             (buffer) => {
                 let init_end = this.initRange.end - this.initRange.start + 1;
                 let index_start = this.indexRange.start - this.initRange.start;
@@ -211,22 +239,23 @@ Stream.prototype.setup = async function(){
                 this.setupInitSegment(buffer.slice(0, init_end));
                 this.setupSegmentIndex(buffer.slice(index_start, index_end));
             }
-        )
+        );
     } else {
         // initialization data
         await fetchRange(
             this.url,
             this.initRange.start,
             this.initRange.end,
-            this.setupInitSegment.bind(this),
-        );
+            'Initialization segment',
+        ).then(this.setupInitSegment.bind(this));
+
         // sidx (segment index) table
         fetchRange(
             this.url,
             this.indexRange.start,
             this.indexRange.end,
-            this.setupSegmentIndex.bind(this)
-        );
+            'Index segment',
+        ).then(this.setupSegmentIndex.bind(this));
     }
 }
 Stream.prototype.setupInitSegment = function(initSegment) {
@@ -388,7 +417,7 @@ Stream.prototype.getSegmentIdx = function(videoTime) {
         }
         index = index + increment;
     }
-    this.reportInfo('Could not find segment index for time', videoTime);
+    this.reportError('Could not find segment index for time', videoTime);
     return 0;
 }
 Stream.prototype.checkBuffer = async function() {
@@ -485,8 +514,8 @@ Stream.prototype.fetchSegment = function(segmentIdx) {
         this.url,
         entry.start,
         entry.end,
-        this.appendSegment.bind(this, segmentIdx),
-    );
+        String(this.streamType) + ' segment ' + String(segmentIdx),
+    ).then(this.appendSegment.bind(this, segmentIdx));
 }
 Stream.prototype.fetchSegmentIfNeeded = function(segmentIdx) {
     if (segmentIdx < 0 || segmentIdx >= this.sidx.entries.length){
@@ -518,22 +547,56 @@ Stream.prototype.reportWarning = function(...args) {
 Stream.prototype.reportError = function(...args) {
     reportError(String(this.streamType) + ':', ...args);
 }
-Stream.prototype.reportInfo = function(...args) {
-    reportInfo(String(this.streamType) + ':', ...args);
-}
 
 
 // Utility functions
 
-function fetchRange(url, start, end, cb) {
+// https://gomakethings.com/promise-based-xhr/
+// https://stackoverflow.com/a/30008115
+// http://lofi.limo/blog/retry-xmlhttprequest-carefully
+function fetchRange(url, start, end, debugInfo) {
     return new Promise((resolve, reject) => {
+        let retryCount = 0;
         let xhr = new XMLHttpRequest();
+        function onFailure(err, message, maxRetries=5){
+            message = debugInfo + ': ' + message + ' - Err: ' + String(err);
+            retryCount++;
+            if (retryCount > maxRetries || xhr.status == 403){
+                reportError('fetchRange error while fetching ' + message);
+                reject(message);
+                return;
+            } else {
+                reportWarning('Failed to fetch ' + message
+                    + '. Attempting retry '
+                    + String(retryCount) +'/' + String(maxRetries));
+            }
+
+            // Retry in 1 second, doubled for each next retry
+            setTimeout(function(){
+                xhr.open('get',url);
+                xhr.send();
+            }, 1000*Math.pow(2,(retryCount-1)));
+        }
         xhr.open('get', url);
+        xhr.timeout = 15000;
         xhr.responseType = 'arraybuffer';
         xhr.setRequestHeader('Range', 'bytes=' + start + '-' + end);
-        xhr.onload = function() {
-            //bytesFetched += end - start + 1;
-            resolve(cb(xhr.response));
+        xhr.onload = function (e) {
+            if (xhr.status >= 200 && xhr.status < 300) {
+                resolve(xhr.response);
+            } else {
+                onFailure(e,
+                    'Status '
+                    + String(xhr.status) + ' ' + String(xhr.statusText)
+                );
+            }
+        };
+        xhr.onerror = function (event) {
+            onFailure(e, 'Network error');
+        };
+        xhr.ontimeout = function (event){
+            xhr.timeout += 5000;
+            onFailure(null, 'Timeout (15s)', maxRetries=5);
         };
         xhr.send();
     });
@@ -573,9 +636,6 @@ function addEvent(obj, eventName, func) {
     return new RegisteredEvent(obj, eventName, func);
 }
 
-function reportInfo(...args){
-    console.info(...args);
-}
 function reportWarning(...args){
     console.warn(...args);
 }

youtube/static/js/common.js

@@ -114,3 +114,60 @@ function copyTextToClipboard(text) {
 window.addEventListener('DOMContentLoaded', function() {
     cur_track_idx = getDefaultTranscriptTrackIdx();
 });
+
+/**
+ * Thumbnail fallback handler
+ * Tries lower quality thumbnails when higher quality fails (404)
+ * Priority: hq720.jpg -> sddefault.jpg -> hqdefault.jpg -> mqdefault.jpg -> default.jpg
+ */
+function thumbnail_fallback(img) {
+    const src = img.src || img.dataset.src;
+    if (!src) return;
+
+    // Handle YouTube video thumbnails
+    if (src.includes('/i.ytimg.com/')) {
+        // Extract video ID from URL
+        const match = src.match(/\/vi\/([^/]+)/);
+        if (!match) return;
+
+        const videoId = match[1];
+        const imgPrefix = settings_img_prefix || '';
+
+        // Define fallback order (from highest to lowest quality)
+        const fallbacks = [
+            'hq720.jpg',
+            'sddefault.jpg',
+            'hqdefault.jpg',
+            'mqdefault.jpg',
+            'default.jpg'
+        ];
+
+        // Find current quality and try next fallback
+        for (let i = 0; i < fallbacks.length; i++) {
+            if (src.includes(fallbacks[i])) {
+                // Try next quality
+                if (i < fallbacks.length - 1) {
+                    const newSrc = imgPrefix + 'https://i.ytimg.com/vi/' + videoId + '/' + fallbacks[i + 1];
+                    if (img.dataset.src) {
+                        img.dataset.src = newSrc;
+                    } else {
+                        img.src = newSrc;
+                    }
+                }
+                break;
+            }
+        }
+    }
+    // Handle YouTube channel avatars (ggpht.com)
+    else if (src.includes('ggpht.com') || src.includes('yt3.ggpht.com')) {
+        // Try to increase avatar size (s88 -> s240)
+        const newSrc = src.replace(/=s\d+-c-k/, '=s240-c-k-c0x00ffffff-no-rj');
+        if (newSrc !== src) {
+            if (img.dataset.src) {
+                img.dataset.src = newSrc;
+            } else {
+                img.src = newSrc;
+            }
+        }
+    }
+}

youtube/static/js/plyr-start.js

@@ -1,77 +1,66 @@
 (function main() {
   'use strict';
 
-  let captionsActive;
-
-  switch(true) {
-    case data.settings.subtitles_mode == 2:
-      captionsActive = true;
-      break;
-    case data.settings.subtitles_mode == 1 && data.has_manual_captions:
-      captionsActive = true;
-      break;
-    default:
-      captionsActive = false;
+  // Captions
+  let captionsActive = false;
+  if (data.settings.subtitles_mode === 2 || (data.settings.subtitles_mode === 1 && data.has_manual_captions)) {
+    captionsActive = true;
   }
 
+  // AutoPlay
+  let autoplayActive = data.settings.autoplay_videos || false;
+
   let qualityOptions = [];
   let qualityDefault;
-  for (let src of data['uni_sources']) {
-    qualityOptions.push(src.quality_string)
+
+  for (let src of data.uni_sources) {
+    qualityOptions.push(src.quality_string);
   }
-  for (let src of data['pair_sources']) {
-    qualityOptions.push(src.quality_string)
+
+  for (let src of data.pair_sources) {
+    qualityOptions.push(src.quality_string);
   }
-  if (data['using_pair_sources'])
-    qualityDefault = data['pair_sources'][data['pair_idx']].quality_string;
-  else if (data['uni_sources'].length != 0)
-    qualityDefault = data['uni_sources'][data['uni_idx']].quality_string;
-  else
+
+  if (data.using_pair_sources) {
+    qualityDefault = data.pair_sources[data.pair_idx].quality_string;
+  } else if (data.uni_sources.length !== 0) {
+    qualityDefault = data.uni_sources[data.uni_idx].quality_string;
+  } else {
     qualityDefault = 'None';
+  }
 
   // Fix plyr refusing to work with qualities that are strings
   Object.defineProperty(Plyr.prototype, 'quality', {
-    set: function(input) {
+    set: function (input) {
       const config = this.config.quality;
       const options = this.options.quality;
-      let quality;
+      let quality = input;
+      let updateStorage = true;
 
       if (!options.length) {
         return;
       }
 
-      // removing this line:
-      //let quality = [!is.empty(input) && Number(input), this.storage.get('quality'), config.selected, config.default].find(is.number);
-      // replacing with:
-      quality = input;
-      let updateStorage = true;
-
       if (!options.includes(quality)) {
-        // Plyr sets quality to null at startup, resulting in the erroneous
-        // calling of this setter function with input = null, and the
-        // commented out code below would set the quality to something
-        // unrelated at startup. Comment out and just return.
         return;
-        /*const value = closest(options, quality);
-          this.debug.warn(`Unsupported quality option: ${quality}, using ${value} instead`);
-          quality = value; // Don't update storage if quality is not supported
-          updateStorage = false;*/
-      } // Update config
-
-
-      config.selected = quality; // Set quality
-
-      this.media.quality = quality; // Save to storage
-
-      if (updateStorage) {
-        this.storage.set({
-          quality
-        });
       }
-    }
+
+      // Update config
+      config.selected = quality;
+
+      // Set quality
+      this.media.quality = quality;
+
+      // Save to storage
+      if (updateStorage) {
+        this.storage.set({ quality });
+      }
+    },
   });
 
-  const player = new Plyr(document.getElementById('js-video-player'), {
+  const playerOptions = {
+    // Learning about autoplay permission https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Permissions-Policy/autoplay#syntax
+    autoplay: autoplayActive,
     disableContextMenu: false,
     captions: {
       active: captionsActive,
@@ -89,29 +78,31 @@
       'settings',
       'pip',
       'airplay',
-      'fullscreen'
+      'fullscreen',
     ],
-    iconUrl: "/youtube.com/static/modules/plyr/plyr.svg",
-    blankVideo: "/youtube.com/static/modules/plyr/blank.webm",
+    iconUrl: '/youtube.com/static/modules/plyr/plyr.svg',
+    blankVideo: '/youtube.com/static/modules/plyr/blank.webm',
     debug: false,
-    storage: {enabled: false},
+    storage: { enabled: false },
     quality: {
       default: qualityDefault,
       options: qualityOptions,
       forced: true,
-      onChange: function(quality) {
-        if (quality == 'None') {return;}
+      onChange: function (quality) {
+        if (quality == 'None') {
+          return;
+        }
         if (quality.includes('(integrated)')) {
-          for (let i=0; i < data['uni_sources'].length; i++) {
-            if (data['uni_sources'][i].quality_string == quality) {
-              changeQuality({'type': 'uni', 'index': i});
+          for (let i = 0; i < data.uni_sources.length; i++) {
+            if (data.uni_sources[i].quality_string == quality) {
+              changeQuality({ type: 'uni', index: i });
               return;
             }
           }
         } else {
-          for (let i=0; i < data['pair_sources'].length; i++) {
-            if (data['pair_sources'][i].quality_string == quality) {
-              changeQuality({'type': 'pair', 'index': i});
+          for (let i = 0; i < data.pair_sources.length; i++) {
+            if (data.pair_sources[i].quality_string == quality) {
+              changeQuality({ type: 'pair', index: i });
               return;
             }
           }
@@ -119,12 +110,27 @@
       },
     },
     previewThumbnails: {
-      enabled: storyboard_url != null,
+      enabled: storyboard_url !== null,
       src: [storyboard_url],
     },
     settings: ['captions', 'quality', 'speed', 'loop'],
     tooltips: {
       controls: true,
     },
+  }
+
+  const player = new Plyr(document.getElementById('js-video-player'), playerOptions);
+
+  // disable double click to fullscreen
+  // https://github.com/sampotts/plyr/issues/1370#issuecomment-528966795
+  player.eventListeners.forEach(function(eventListener) {
+    if(eventListener.type === 'dblclick') {
+        eventListener.element.removeEventListener(eventListener.type, eventListener.callback, eventListener.options);
+    }
   });
-}());
+
+  // Add .started property, true after the playback has been started
+  // Needed so controls won't be hidden before playback has started
+  player.started = false;
+  player.once('playing', function(){this.started = true});
+})();

youtube/static/js/watch.js

@@ -5,8 +5,9 @@ function changeQuality(selection) {
     let videoPaused = video.paused;
     let videoSpeed = video.playbackRate;
     let srcInfo;
-    if (avMerge)
+    if (avMerge && typeof avMerge.close === 'function') {
         avMerge.close();
+    }
     if (selection.type == 'uni'){
         srcInfo = data['uni_sources'][selection.index];
         video.src = srcInfo.url;

youtube/static/license.css

@@ -181,7 +181,7 @@ label[for=options-toggle-cbox] {
 
 .table td,.table th {
     padding: 10px 10px;
-    border: 1px solid var(--secondary-background);
+    border: 1px solid var(--border-bg-license);
     text-align: center;
 }
 

youtube/static/light_theme.css

@@ -10,9 +10,11 @@
     --link:                 #212121;
     --link-visited:         #808080;
     --border-bg:            #212121;
-    --buttom:               #DCDCDC;
+    --border-bg-settings:   #91918C;
+    --border-bg-license:    #91918C;
+    --buttom:               #FFFFFF;
     --buttom-text:          #212121;
-    --button-border:        #91918c;
+    --button-border:        #91918C;
     --buttom-hover:         #BBBBBB;
     --search-text:          #212121;
     --time-background:      #212121;

youtube/static/modules/plyr/custom_plyr.css

@@ -0,0 +1,77 @@
+/* Prevent this div from blocking right-click menu for video
+e.g. Firefox playback speed options */
+.plyr__poster {
+    display: none;
+}
+
+/* plyr fix */
+.plyr:-moz-full-screen video {
+    max-height: initial;
+}
+
+.plyr:-webkit-full-screen video {
+    max-height: initial;
+}
+
+.plyr:-ms-fullscreen video {
+    max-height: initial;
+}
+
+.plyr:fullscreen video {
+    max-height: initial;
+}
+
+.plyr__preview-thumb__image-container {
+    width: 158px;
+    height: 90px;
+}
+
+.plyr__preview-thumb {
+    bottom: 100%;
+}
+
+.plyr__menu__container [role="menu"],
+.plyr__menu__container [role="menucaptions"] {
+    /* Set vertical scroll */
+    /* issue https://github.com/sampotts/plyr/issues/1420 */
+    max-height: 320px;
+    overflow-y: auto;
+}
+
+/*
+* Custom styles similar to youtube
+*/
+.plyr__controls {
+    display: flex;
+    justify-content: center;
+}
+
+.plyr__progress__container {
+    position: absolute;
+    bottom: 0;
+    width: 100%;
+    margin-bottom: -10px;
+}
+
+.plyr__controls .plyr__controls__item:first-child {
+    margin-left: 0;
+    margin-right: 0;
+    z-index: 5;
+}
+
+.plyr__controls .plyr__controls__item.plyr__volume {
+    margin-left: auto;
+}
+
+.plyr__controls .plyr__controls__item.plyr__progress__container {
+    padding-left: 10px;
+    padding-right: 10px;
+}
+
+.plyr__progress input[type="range"] {
+    margin-bottom: 50px;
+}
+
+/*
+* End custom styles
+*/

youtube/static/settings.css

@@ -155,7 +155,7 @@ label[for=options-toggle-cbox] {
 }
 
 .settings-form > h2 {
-    border-bottom: 2px solid var(--border-bg);
+    border-bottom: 2px solid var(--border-bg-settings);
     padding-bottom: 0.5rem;
 }
 

youtube/static/watch.css

@@ -21,21 +21,7 @@ img {
 video {
     width: 100%;
     height: auto;
-    max-height: 480px;
-}
-
-/* plyr fix */
-.plyr:-moz-full-screen video {
-    max-height: initial;
-}
-.plyr:-webkit-full-screen video {
-    max-height: initial;
-}
-.plyr:-ms-fullscreen video {
-    max-height: initial;
-}
-.plyr:fullscreen video {
-    max-height: initial;
+    max-height: calc(100vh/1.5);
 }
 
 a:link {
@@ -142,6 +128,29 @@ header {
     background-color: var(--buttom-hover);
 }
 
+.live-url-choices {
+    background-color: var(--thumb-background);
+    margin: 1rem 0;
+    padding: 1rem;
+}
+
+.playability-error {
+    position: relative;
+    box-sizing: border-box;
+    height: 30vh;
+    margin: 1rem 0;
+}
+
+.playability-error > span {
+    display: flex;
+    background-color: var(--thumb-background);
+    height: 100%;
+    object-fit: cover;
+    justify-content: center;
+    align-items: center;
+    text-align: center;
+}
+
 .playlist {
     display: grid;
     grid-gap: 4px;
@@ -636,6 +645,9 @@ figure.sc-video {
         max-height: 80vh;
         overflow-y: scroll;
     }
+    .playability-error {
+        height: 60vh;
+    }
     .playlist {
         display: grid;
         grid-gap: 1px;

youtube/subscriptions.py

@@ -1,4 +1,4 @@
-from youtube import util, yt_data_extract, channel, local_playlist
+from youtube import util, yt_data_extract, channel, local_playlist, playlist
 from youtube import yt_app
 import settings
 
@@ -108,8 +108,7 @@ def _subscribe(channels):
         with connection as cursor:
             channel_ids_to_check = [channel[0] for channel in channels if not _is_subscribed(cursor, channel[0])]
 
-            rows = ((channel_id, channel_name, 0, 0) for channel_id,
-                    channel_name in channels)
+            rows = ((channel_id, channel_name, 0, 0) for channel_id, channel_name in channels)
             cursor.executemany('''INSERT OR IGNORE INTO subscribed_channels (yt_channel_id, channel_name, time_last_checked, next_check_time)
                                   VALUES (?, ?, ?, ?)''', rows)
 
@@ -236,8 +235,7 @@ def _get_channel_names(cursor, channel_ids):
     return result
 
 
-def _channels_with_tag(cursor, tag, order=False, exclude_muted=False,
-                       include_muted_status=False):
+def _channels_with_tag(cursor, tag, order=False, exclude_muted=False, include_muted_status=False):
     ''' returns list of (channel_id, channel_name) '''
 
     statement = '''SELECT yt_channel_id, channel_name'''
@@ -434,8 +432,10 @@ def autocheck_setting_changed(old_value, new_value):
         stop_autocheck_system()
 
 
-settings.add_setting_changed_hook('autocheck_subscriptions',
-                                  autocheck_setting_changed)
+settings.add_setting_changed_hook(
+    'autocheck_subscriptions',
+    autocheck_setting_changed
+)
 if settings.autocheck_subscriptions:
     start_autocheck_system()
 # ----------------------------
@@ -463,7 +463,24 @@ def _get_atoma_feed(channel_id):
 
 def _get_channel_videos_first_page(channel_id, channel_status_name):
     try:
-        return channel.get_channel_first_page(channel_id=channel_id)
+        # First try the playlist method
+        pl_json = playlist.get_videos(
+            'UU' + channel_id[2:],
+            1,
+            include_shorts=settings.include_shorts_in_subscriptions,
+            report_text=None
+        )
+        pl_info = yt_data_extract.extract_playlist_info(pl_json)
+        if pl_info.get('items'):
+            pl_info['items'] = pl_info['items'][0:30]
+            return pl_info
+
+        # Try the channel api method
+        channel_json = channel.get_channel_first_page(channel_id=channel_id)
+        channel_info = yt_data_extract.extract_channel_info(
+            json.loads(channel_json), 'videos'
+        )
+        return channel_info
     except util.FetchError as e:
         if e.code == '429' and settings.route_tor:
             error_message = ('Error checking channel ' + channel_status_name
@@ -497,7 +514,7 @@ def _get_upstream_videos(channel_id):
     )
     gevent.joinall(tasks)
 
-    channel_tab, feed = tasks[0].value, tasks[1].value
+    channel_info, feed = tasks[0].value, tasks[1].value
 
     # extract published times from atoma feed
     times_published = {}
@@ -535,9 +552,8 @@ def _get_upstream_videos(channel_id):
     except defusedxml.ElementTree.ParseError:
         print('Failed to read atoma feed for ' + channel_status_name)
 
-    if channel_tab is None: # there was an error
+    if channel_info is None: # there was an error
         return
-    channel_info = yt_data_extract.extract_channel_info(json.loads(channel_tab), 'videos')
     if channel_info['error']:
         print('Error checking channel ' + channel_status_name + ': ' + channel_info['error'])
         return
@@ -552,14 +568,38 @@ def _get_upstream_videos(channel_id):
         if video_item['id'] in times_published:
             video_item['time_published'] = times_published[video_item['id']]
             video_item['is_time_published_exact'] = True
-        else:
+        elif video_item.get('time_published'):
             video_item['is_time_published_exact'] = False
             try:
                 video_item['time_published'] = youtube_timestamp_to_posix(video_item['time_published']) - i  # subtract a few seconds off the videos so they will be in the right order
-            except KeyError:
+            except Exception:
                 print(video_item)
-
+        else:
+            video_item['is_time_published_exact'] = False
+            video_item['time_published'] = None
         video_item['channel_id'] = channel_id
+    if len(videos) > 1:
+        # Go back and fill in any videos that don't have a time published
+        # using the time published of the surrounding ones
+        for i in range(len(videos)-1):
+            if (videos[i+1]['time_published'] is None
+                and videos[i]['time_published'] is not None
+            ):
+                videos[i+1]['time_published'] = videos[i]['time_published'] - 1
+        for i in reversed(range(1,len(videos))):
+            if (videos[i-1]['time_published'] is None
+                and videos[i]['time_published'] is not None
+            ):
+                videos[i-1]['time_published'] = videos[i]['time_published'] + 1
+    # Special case: none of the videos have a time published.
+    # In this case, make something up
+    if videos and videos[0]['time_published'] is None:
+        assert all(v['time_published'] is None for v in videos)
+        now = time.time()
+        for i in range(len(videos)):
+            # 1 month between videos
+            videos[i]['time_published'] = now - i*3600*24*30
+
 
     if len(videos) == 0:
         average_upload_period = 4*7*24*3600  # assume 1 month for channel with no videos
@@ -578,26 +618,31 @@ def _get_upstream_videos(channel_id):
     with open_database() as connection:
         with connection as cursor:
 
-            # calculate how many new videos there are
-            existing_vids = set(row[0] for row in cursor.execute(
-                '''SELECT video_id
+            # Get video ids and duration of existing vids so we
+            # can see how many new ones there are and update
+            # livestreams/premiers
+            existing_vids = list(cursor.execute(
+                '''SELECT video_id, duration
                    FROM videos
                    INNER JOIN subscribed_channels
                        ON videos.sql_channel_id = subscribed_channels.id
                    WHERE yt_channel_id=?
                    ORDER BY time_published DESC
                    LIMIT 30''', [channel_id]).fetchall())
+            existing_vid_ids = set(row[0] for row in existing_vids)
+            existing_durs = dict(existing_vids)
 
             # new videos the channel has uploaded since last time we checked
             number_of_new_videos = 0
             for video in videos:
-                if video['id'] in existing_vids:
+                if video['id'] in existing_vid_ids:
                     break
                 number_of_new_videos += 1
 
             is_first_check = cursor.execute('''SELECT time_last_checked FROM subscribed_channels WHERE yt_channel_id=?''', [channel_id]).fetchone()[0] in (None, 0)
             time_videos_retrieved = int(time.time())
             rows = []
+            update_rows = []
             for i, video_item in enumerate(videos):
                 if (is_first_check
                         or number_of_new_videos > 6
@@ -613,16 +658,34 @@ def _get_upstream_videos(channel_id):
                     time_noticed = video_item['time_published']
                 else:
                     time_noticed = time_videos_retrieved
-                rows.append((
-                    video_item['channel_id'],
-                    video_item['id'],
-                    video_item['title'],
-                    video_item['duration'],
-                    video_item['time_published'],
-                    video_item['is_time_published_exact'],
-                    time_noticed,
-                    video_item['description'],
-                ))
+
+                # videos which need durations updated
+                non_durations = ('upcoming', 'none', 'live', '')
+                v_id = video_item['id']
+                if (existing_durs.get(v_id) is not None
+                    and existing_durs[v_id].lower() in non_durations
+                    and video_item['duration'] not in non_durations
+                ):
+                    update_rows.append((
+                        video_item['title'],
+                        video_item['duration'],
+                        video_item['time_published'],
+                        video_item['is_time_published_exact'],
+                        video_item['description'],
+                        video_item['id'],
+                    ))
+                # all other videos
+                else:
+                    rows.append((
+                        video_item['channel_id'],
+                        video_item['id'],
+                        video_item['title'],
+                        video_item['duration'],
+                        video_item['time_published'],
+                        video_item['is_time_published_exact'],
+                        time_noticed,
+                        video_item['description'],
+                    ))
 
             cursor.executemany('''INSERT OR IGNORE INTO videos (
                                       sql_channel_id,
@@ -635,6 +698,13 @@ def _get_upstream_videos(channel_id):
                                       description
                                   )
                                   VALUES ((SELECT id FROM subscribed_channels WHERE yt_channel_id=?), ?, ?, ?, ?, ?, ?, ?)''', rows)
+            cursor.executemany('''UPDATE videos SET
+                                      title=?,
+                                      duration=?,
+                                      time_published=?,
+                                      is_time_published_exact=?,
+                                      description=?
+                                  WHERE video_id=?''', update_rows)
             cursor.execute('''UPDATE subscribed_channels
                               SET time_last_checked = ?, next_check_time = ?
                               WHERE yt_channel_id=?''', [int(time.time()), next_check_time, channel_id])
@@ -767,7 +837,7 @@ def import_subscriptions():
         error = 'Unsupported file format: ' + mime_type
         error += (' . Only subscription.json, subscriptions.csv files'
                   ' (from Google Takeouts)'
-                  ' and XML OPML files exported from Youtube\'s'
+                  ' and XML OPML files exported from YouTube\'s'
                   ' subscription manager page are supported')
         return (flask.render_template('error.html', error_message=error),
                 400)
@@ -962,7 +1032,8 @@ def get_subscriptions_page():
                     'muted': muted,
                 })
 
-    return flask.render_template('subscriptions.html',
+    return flask.render_template(
+        'subscriptions.html',
         header_playlist_names=local_playlist.get_playlist_names(),
         videos=videos,
         num_pages=math.ceil(number_of_videos_in_db/60),
@@ -1018,12 +1089,12 @@ def serve_subscription_thumbnail(thumbnail):
             f.close()
             return flask.Response(image, mimetype='image/jpeg')
 
-    url = f"https://i.ytimg.com/vi/{video_id}/hqdefault.jpg"
+    url = f"https://i.ytimg.com/vi/{video_id}/hq720.jpg"
     try:
         image = util.fetch_url(url, report_text="Saved thumbnail: " + video_id)
     except urllib.error.HTTPError as e:
         print("Failed to download thumbnail for " + video_id + ": " + str(e))
-        abort(e.code)
+        flask.abort(e.code)
     try:
         f = open(thumbnail_path, 'wb')
     except FileNotFoundError:

youtube/templates/base.html

@@ -26,6 +26,12 @@
              // @license-end
             </script>
         {% endif %}
+        <script>
+            // @license magnet:?xt=urn:btih:0b31508aeb0634b347b8270c7bee4d411b5d4109&dn=agpl-3.0.txt AGPL-v3-or-Later
+            // Image prefix for thumbnails
+            let settings_img_prefix = "{{ settings.img_prefix or '' }}";
+            // @license-end
+        </script>
     </head>
 
     <body>
@@ -35,57 +41,57 @@
             </nav>
             <form class="form" id="site-search" action="/youtube.com/results">
                 <input type="search" name="search_query" class="search-box" value="{{ search_box_value }}"
-                       {{ "autofocus" if (request.path in ("/", "/results") or error_message) else "" }} required placeholder="Type to search...">
-                <button type="submit" value="Search" class="search-button">Search</button>
+                       {{ "autofocus" if (request.path in ("/", "/results") or error_message) else "" }} required placeholder="{{ _('Type to search...') }}">
+                <button type="submit" value="Search" class="search-button">{{ _('Search') }}</button>
                 <!-- options -->
                 <div class="dropdown">
                     <!-- hidden box -->
                     <input id="options-toggle-cbox" class="opt-box" type="checkbox">
                     <!-- end hidden box -->
-                    <label class="dropdown-label" for="options-toggle-cbox">Options</label>
+                    <label class="dropdown-label" for="options-toggle-cbox">{{ _('Options') }}</label>
                     <div class="dropdown-content">
-                        <h3>Sort by</h3>
+                        <h3>{{ _('Sort by') }}</h3>
                         <div class="option">
                             <input type="radio" id="sort_relevance" name="sort" value="0">
-                            <label for="sort_relevance">Relevance</label>
+                            <label for="sort_relevance">{{ _('Relevance') }}</label>
                         </div>
                         <div class="option">
                             <input type="radio" id="sort_upload_date" name="sort" value="2">
-                            <label for="sort_upload_date">Upload date</label>
+                            <label for="sort_upload_date">{{ _('Upload date') }}</label>
                         </div>
                         <div class="option">
                             <input type="radio" id="sort_view_count" name="sort" value="3">
-                            <label for="sort_view_count">View count</label>
+                            <label for="sort_view_count">{{ _('View count') }}</label>
                         </div>
                         <div class="option">
                             <input type="radio" id="sort_rating" name="sort" value="1">
-                            <label for="sort_rating">Rating</label>
+                            <label for="sort_rating">{{ _('Rating') }}</label>
                         </div>
 
-                        <h3>Upload date</h3>
+                        <h3>{{ _('Upload date') }}</h3>
                         <div class="option">
                             <input type="radio" id="time_any" name="time" value="0">
-                            <label for="time_any">Any</label>
+                            <label for="time_any">{{ _('Any') }}</label>
                         </div>
                         <div class="option">
                             <input type="radio" id="time_last_hour" name="time" value="1">
-                            <label for="time_last_hour">Last hour</label>
+                            <label for="time_last_hour">{{ _('Last hour') }}</label>
                         </div>
                         <div class="option">
                             <input type="radio" id="time_today" name="time" value="2">
-                            <label for="time_today">Today</label>
+                            <label for="time_today">{{ _('Today') }}</label>
                         </div>
                         <div class="option">
                             <input type="radio" id="time_this_week" name="time" value="3">
-                            <label for="time_this_week">This week</label>
+                            <label for="time_this_week">{{ _('This week') }}</label>
                         </div>
                         <div class="option">
                             <input type="radio" id="time_this_month" name="time" value="4">
-                            <label for="time_this_month">This month</label>
+                            <label for="time_this_month">{{ _('This month') }}</label>
                         </div>
                         <div class="option">
                             <input type="radio" id="time_this_year" name="time" value="5">
-                            <label for="time_this_year">This year</label>
+                            <label for="time_this_year">{{ _('This year') }}</label>
                         </div>
 
                         <h3>Type</h3>

youtube/templates/channel.html

@@ -51,8 +51,11 @@
             <ul>
                 {% for (before_text, stat, after_text) in [
                     ('Joined ', date_joined, ''),
-                    ('', view_count|commatize, ' views'),
+                    ('', approx_view_count, ' views'),
                     ('', approx_subscriber_count, ' subscribers'),
+                    ('', approx_video_count, ' videos'),
+                    ('Country: ', country, ''),
+                    ('Canonical Url: ', canonical_url, ''),
                 ] %}
                     {% if stat %}
                         <li>{{ before_text + stat|string + after_text }}</li>
@@ -65,7 +68,11 @@
             <hr>
             <ul>
                 {% for text, url in links %}
-                    <li><a href="{{ url }}">{{ text }}</a></li>
+                    {% if url %}
+                        <li><a href="{{ url }}">{{ text }}</a></li>
+                    {% else %}
+                        <li>{{ text }}</li>
+                    {% endif %}
                 {% endfor %}
             </ul>
         </div>
@@ -74,10 +81,10 @@
         <!-- new-->
         <div id="links-metadata">
                 {% if current_tab in ('videos', 'shorts', 'streams') %}
-                    {% set sorts = [('1', 'views'), ('2', 'oldest'), ('3', 'newest')] %}
+                    {% set sorts = [('3', 'newest'), ('4', 'newest - no shorts')] %}
                     <div id="number-of-results">{{ number_of_videos }} videos</div>
                 {% elif current_tab == 'playlists' %}
-                    {% set sorts = [('2', 'oldest'), ('3', 'newest'), ('4', 'last video added')] %}
+                    {% set sorts = [('3', 'newest'), ('4', 'last video added')] %}
                     {% if items %}
                         <h2 class="page-number">Page {{ page_number }}</h2>
                     {% else %}
@@ -110,13 +117,9 @@
         <hr/>
 
         <footer class="pagination-container">
-            {% if (current_tab in ('videos', 'shorts', 'streams')) and current_sort.__str__() == '2' %}
-                <nav class="next-previous-button-row">
-                    {{ common_elements.next_previous_ctoken_buttons(None, ctoken, channel_url + '/' + current_tab, parameters_dictionary) }}
-                </nav>
-            {% elif current_tab in ('videos', 'shorts', 'streams') %}
+            {% if current_tab in ('videos', 'shorts', 'streams') %}
                 <nav class="pagination-list">
-                    {{ common_elements.page_buttons(number_of_pages, channel_url + '/' + current_tab, parameters_dictionary, include_ends=(current_sort.__str__() == '3')) }}
+                    {{ common_elements.page_buttons(number_of_pages, channel_url + '/' + current_tab, parameters_dictionary, include_ends=(current_sort.__str__() in '34')) }}
                 </nav>
             {% elif current_tab == 'playlists' or current_tab == 'search' %}
                 <nav class="next-previous-button-row">

youtube/templates/common_elements.html

@@ -23,11 +23,11 @@
                 <a class="thumbnail-box" href="{{ info['url'] }}" title="{{ info['title'] }}">
                     <div class="thumbnail {% if info['type'] == 'channel' %} channel {% endif %}">
                         {% if lazy_load %}
-                            <img class="thumbnail-img lazy" alt="&#x20;" data-src="{{ info['thumbnail'] }}">
+                            <img class="thumbnail-img lazy" alt="&#x20;" data-src="{{ info['thumbnail'] }}" onerror="thumbnail_fallback(this)">
                         {% elif info['type'] == 'channel' %}
-                            <img class="thumbnail-img channel" alt="&#x20;" src="{{ info['thumbnail'] }}">
+                            <img class="thumbnail-img channel" alt="&#x20;" src="{{ info['thumbnail'] }}" onerror="thumbnail_fallback(this)">
                         {% else %}
-                            <img class="thumbnail-img" alt="&#x20;" src="{{ info['thumbnail'] }}">
+                            <img class="thumbnail-img" alt="&#x20;" src="{{ info['thumbnail'] }}" onerror="thumbnail_fallback(this)">
                         {% endif %}
 
                         {% if info['type'] != 'channel' %}

youtube/templates/settings.html

@@ -31,11 +31,19 @@
                                     <input type="number" id="{{ 'setting_' + setting_name }}" name="{{ setting_name }}" value="{{ value }}" step="1">
                                 {% endif %}
                             {% elif setting_info['type'].__name__ == 'float' %}
-
+                                <input type="number" id="{{ 'setting_' + setting_name }}" name="{{ setting_name }}" value="{{ value }}" step="0.01">
                             {% elif setting_info['type'].__name__ == 'str' %}
-                                <input type="text" id="{{ 'setting_' + setting_name }}" name="{{ setting_name }}" value="{{ value }}">
+                                {% if 'options' is in(setting_info) %}
+                                    <select id="{{ 'setting_' + setting_name }}" name="{{ setting_name }}">
+                                        {% for option in setting_info['options'] %}
+                                            <option value="{{ option[0] }}" {{ 'selected' if option[0] == value else '' }}>{{ option[1] }}</option>
+                                        {% endfor %}
+                                    </select>
+                                {% else %}
+                                    <input type="text" id="{{ 'setting_' + setting_name }}" name="{{ setting_name }}" value="{{ value }}">
+                                {% endif %}
                             {% else %}
-                                <span>Error: Unknown setting type: setting_info['type'].__name__</span>
+                                <span>Error: Unknown setting type: {{ setting_info['type'].__name__ }}</span>
                             {% endif %}
                         </li>
                     {% endif %}

youtube/templates/watch.html

@@ -8,14 +8,8 @@
     {% if settings.use_video_player == 2 %}
         <!-- plyr -->
         <link href="/youtube.com/static/modules/plyr/plyr.css" rel="stylesheet">
+        <link href="/youtube.com/static/modules/plyr/custom_plyr.css" rel="stylesheet">
         <!--/ plyr -->
-        <style>
-         /* Prevent this div from blocking right-click menu for video
-         e.g. Firefox playback speed options */
-         .plyr__poster {
-             display: none !important;
-         }
-        </style>
     {% endif %}
 {% endblock style %}
 
@@ -40,7 +34,7 @@
         </div>
     {% else %}
         <figure class="sc-video">
-            <video id="js-video-player" playsinline controls>
+            <video id="js-video-player" playsinline controls {{ 'autoplay' if settings.autoplay_videos }}>
                 {% if uni_sources %}
                     <source src="{{ uni_sources[uni_idx]['url'] }}" type="{{ uni_sources[uni_idx]['type'] }}" data-res="{{ uni_sources[uni_idx]['quality'] }}">
                 {% endif %}
@@ -91,6 +85,16 @@
                             <option value='{"type": "pair", "index": {{ loop.index0}}}' {{ 'selected' if loop.index0 == pair_idx and using_pair_sources else '' }} >{{ src_pair['quality_string'] }}</option>
                         {% endfor %}
                     </select>
+
+                    {% if audio_tracks and audio_tracks|length > 1 %}
+                    <select id="audio-language-select" autocomplete="off" title="Audio language">
+                        {% for track in audio_tracks %}
+                            <option value="{{ track.get('track_id', track['language']) }}" {{ 'selected' if loop.index0 == 0 else '' }}>
+                                🔊 {{ track['language_name'] }}{% if track.get('is_default') %} (Default){% endif %}
+                            </option>
+                        {% endfor %}
+                    </select>
+                    {% endif %}
                 {% endif %}
             </div>
             <input class="v-checkbox" name="video_info_list" value="{{ video_info }}" form="playlist-edit" type="checkbox">
@@ -233,7 +237,7 @@
                 <div class="comments-area-outer comments-disabled">Comments disabled</div>
             {% else %}
                 <details class="comments-area-outer" {{'open' if settings.comments_mode == 1 else ''}}>
-                    <summary>{{ comment_count|commatize }} comment{{'s' if comment_count != 1 else ''}}</summary>
+                    <summary>{{ comment_count|commatize }} comment{{'s' if comment_count != '1' else ''}}</summary>
                     <div class="comments-area-inner comments-area">
                         {% if comments_info %}
                             {{ comments.video_comments(comments_info) }}
@@ -252,6 +256,38 @@
      let storyboard_url = {{ storyboard_url | tojson }};
      // @license-end
     </script>
+
+    <!-- Audio language selector handler -->
+    <script>
+     // @license magnet:?xt=urn:btih:0b31508aeb0634b347b8270c7bee4d411b5d4109&dn=agpl-3.0.txt AGPL-v3-or-Later
+     (function() {
+         'use strict';
+         const audioSelect = document.getElementById('audio-language-select');
+         const qualitySelect = document.getElementById('quality-select');
+
+         if (audioSelect && qualitySelect) {
+             audioSelect.addEventListener('change', function() {
+                 const selectedAudio = this.value;
+                 const selectedQuality = qualitySelect.value;
+
+                 // Parse current quality selection
+                 let qualityData;
+                 try {
+                     qualityData = JSON.parse(selectedQuality);
+                 } catch(e) {
+                     return;
+                 }
+
+                 // Reload video with new audio language
+                 const currentUrl = new URL(window.location.href);
+                 currentUrl.searchParams.set('audio_lang', selectedAudio);
+                 window.location.href = currentUrl.toString();
+             });
+         }
+     }());
+     // @license-end
+    </script>
+
     <script src="/youtube.com/static/js/common.js"></script>
     <script src="/youtube.com/static/js/transcript-table.js"></script>
     {% if settings.use_video_player == 2 %}

youtube/util.py

@@ -1,4 +1,5 @@
 from datetime import datetime
+import logging
 import settings
 import socks
 import sockshandler
@@ -18,6 +19,8 @@ import gevent.queue
 import gevent.lock
 import collections
 import stem
+
+logger = logging.getLogger(__name__)
 import stem.control
 import traceback
 
@@ -302,72 +305,144 @@ def fetch_url_response(url, headers=(), timeout=15, data=None,
 def fetch_url(url, headers=(), timeout=15, report_text=None, data=None,
               cookiejar_send=None, cookiejar_receive=None, use_tor=True,
               debug_name=None):
-    while True:
-        start_time = time.monotonic()
+    """
+    Fetch URL with exponential backoff retry logic for rate limiting.
 
-        response, cleanup_func = fetch_url_response(
-            url, headers, timeout=timeout, data=data,
-            cookiejar_send=cookiejar_send, cookiejar_receive=cookiejar_receive,
-            use_tor=use_tor)
-        response_time = time.monotonic()
+    Retries:
+    - 429 Too Many Requests: Exponential backoff (1s, 2s, 4s, 8s, 16s)
+    - 503 Service Unavailable: Exponential backoff
+    - 302 Redirect to Google Sorry: Treated as rate limit
 
-        content = response.read()
+    Max retries: 5 attempts with exponential backoff
+    """
+    import random
 
-        read_finish = time.monotonic()
+    max_retries = 5
+    base_delay = 1.0  # Base delay in seconds
 
-        cleanup_func(response)  # release_connection for urllib3
-        content = decode_content(
-            content,
-            response.getheader('Content-Encoding', default='identity'))
+    for attempt in range(max_retries):
+        try:
+            start_time = time.monotonic()
 
-        if (settings.debugging_save_responses
-                and debug_name is not None and content):
-            save_dir = os.path.join(settings.data_dir, 'debug')
-            if not os.path.exists(save_dir):
-                os.makedirs(save_dir)
+            response, cleanup_func = fetch_url_response(
+                url, headers, timeout=timeout, data=data,
+                cookiejar_send=cookiejar_send, cookiejar_receive=cookiejar_receive,
+                use_tor=use_tor)
+            response_time = time.monotonic()
 
-            with open(os.path.join(save_dir, debug_name), 'wb') as f:
-                f.write(content)
+            content = response.read()
 
-        if response.status == 429 or (
-            response.status == 302 and (response.getheader('Location') == url
-                or response.getheader('Location').startswith(
-                       'https://www.google.com/sorry/index'
-                   )
-            )
-        ):
-            print(response.status, response.reason, response.getheaders())
-            ip = re.search(
-                br'IP address: ((?:[\da-f]*:)+[\da-f]+|(?:\d+\.)+\d+)',
-                content)
-            ip = ip.group(1).decode('ascii') if ip else None
-            if not ip:
-                ip = re.search(r'IP=((?:\d+\.)+\d+)',
-                               response.getheader('Set-Cookie') or '')
-                ip = ip.group(1) if ip else None
+            read_finish = time.monotonic()
 
-            # don't get new identity if we're not using Tor
-            if not use_tor:
-                raise FetchError('429', reason=response.reason, ip=ip)
+            cleanup_func(response)  # release_connection for urllib3
+            content = decode_content(
+                content,
+                response.headers.get('Content-Encoding', default='identity'))
 
-            print('Error: YouTube blocked the request because the Tor exit node is overutilized. Exit node IP address: %s' % ip)
+            if (settings.debugging_save_responses
+                    and debug_name is not None
+                    and content):
+                save_dir = os.path.join(settings.data_dir, 'debug')
+                if not os.path.exists(save_dir):
+                    os.makedirs(save_dir)
 
-            # get new identity
-            error = tor_manager.new_identity(start_time)
-            if error:
-                raise FetchError(
-                    '429', reason=response.reason, ip=ip,
-                    error_message='Automatic circuit change: ' + error)
-            else:
-                continue # retry now that we have new identity
+                with open(os.path.join(save_dir, debug_name), 'wb') as f:
+                    f.write(content)
 
-        elif response.status >= 400:
-            raise FetchError(str(response.status), reason=response.reason,
-                             ip=None)
-        break
+            # Check for rate limiting (429) or redirect to Google Sorry
+            if response.status == 429 or (
+                response.status == 302 and (response.getheader('Location') == url
+                    or response.getheader('Location').startswith(
+                           'https://www.google.com/sorry/index'
+                       )
+                    )
+            ):
+                logger.info(f'Rate limit response: {response.status} {response.reason}')
+                ip = re.search(
+                    br'IP address: ((?:[\da-f]*:)+[\da-f]+|(?:\d+\.)+\d+)',
+                    content)
+                ip = ip.group(1).decode('ascii') if ip else None
+                if not ip:
+                    ip = re.search(r'IP=((?:\d+\.)+\d+)',
+                                   response.getheader('Set-Cookie') or '')
+                    ip = ip.group(1) if ip else None
+
+                # If this is the last attempt, raise error
+                if attempt >= max_retries - 1:
+                    if not use_tor or not settings.route_tor:
+                        logger.warning(f'YouTube returned 429 but Tor is not enabled. Consider enabling Tor routing.')
+                        raise FetchError('429', reason=response.reason, ip=ip)
+
+                    logger.error(f'YouTube blocked request - Tor exit node overutilized. Exit IP: {ip}')
+
+                    # get new identity
+                    error = tor_manager.new_identity(start_time)
+                    if error:
+                        raise FetchError(
+                            '429', reason=response.reason, ip=ip,
+                            error_message='Automatic circuit change: ' + error)
+                    else:
+                        continue  # retry with new identity
+
+                # Calculate delay with exponential backoff and jitter
+                delay = (base_delay * (2 ** attempt)) + random.uniform(0, 1)
+                logger.info(f'Rate limited (429). Waiting {delay:.1f}s before retry {attempt + 1}/{max_retries}...')
+                time.sleep(delay)
+                continue  # retry
+
+            # Check for client errors (400, 404) - don't retry these
+            if response.status == 400:
+                logger.error(f'Bad Request (400) - Invalid parameters or URL: {url[:100]}')
+                raise FetchError('400', reason='Bad Request - Invalid parameters or URL format', ip=None)
+
+            if response.status == 404:
+                logger.warning(f'Not Found (404): {url[:100]}')
+                raise FetchError('404', reason='Not Found', ip=None)
+
+            # Check for other server errors (503, 502, 504)
+            if response.status in (502, 503, 504):
+                if attempt >= max_retries - 1:
+                    logger.error(f'Server error {response.status} after {max_retries} retries')
+                    raise FetchError(str(response.status), reason=response.reason, ip=None)
+
+                # Exponential backoff for server errors
+                delay = (base_delay * (2 ** attempt)) + random.uniform(0, 1)
+                logger.warning(f'Server error ({response.status}). Waiting {delay:.1f}s before retry {attempt + 1}/{max_retries}...')
+                time.sleep(delay)
+                continue
+
+            # Success - break out of retry loop
+            break
+
+        except urllib3.exceptions.MaxRetryError as e:
+            # If this is the last attempt, raise the error
+            if attempt >= max_retries - 1:
+                exception_cause = e.__context__.__context__
+                if (isinstance(exception_cause, socks.ProxyConnectionError)
+                        and settings.route_tor):
+                    msg = ('Failed to connect to Tor. Check that Tor is open and '
+                           'that your internet connection is working.\n\n'
+                           + str(e))
+                    logger.error(f'Tor connection failed: {msg}')
+                    raise FetchError('502', reason='Bad Gateway',
+                                     error_message=msg)
+                elif isinstance(e.__context__,
+                                urllib3.exceptions.NewConnectionError):
+                    msg = 'Failed to establish a connection.\n\n' + str(e)
+                    logger.error(f'Connection failed: {msg}')
+                    raise FetchError(
+                        '502', reason='Bad Gateway',
+                         error_message=msg)
+                else:
+                    raise
+
+            # Wait and retry
+            delay = (base_delay * (2 ** attempt)) + random.uniform(0, 1)
+            logger.warning(f'Connection error. Waiting {delay:.1f}s before retry {attempt + 1}/{max_retries}...')
+            time.sleep(delay)
 
     if report_text:
-        print(report_text, '    Latency:', round(response_time - start_time, 3), '    Read time:', round(read_finish - response_time,3))
+        logger.info(f'{report_text} - Latency: {round(response_time - start_time, 3)}s - Read time: {round(read_finish - response_time, 3)}s')
 
     return content
 
@@ -394,7 +469,6 @@ def head(url, use_tor=False, report_text=None, max_redirects=10):
             round(time.monotonic() - start_time, 3))
     return response
 
-
 mobile_user_agent = 'Mozilla/5.0 (Linux; Android 7.0; Redmi Note 4 Build/NRD90M) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Mobile Safari/537.36'
 mobile_ua = (('User-Agent', mobile_user_agent),)
 desktop_user_agent = 'Mozilla/5.0 (Windows NT 6.1; rv:52.0) Gecko/20100101 Firefox/52.0'
@@ -404,13 +478,13 @@ desktop_xhr_headers = (
     ('Accept', '*/*'),
     ('Accept-Language', 'en-US,en;q=0.5'),
     ('X-YouTube-Client-Name', '1'),
-    ('X-YouTube-Client-Version', '2.20180830'),
+    ('X-YouTube-Client-Version', '2.20240304.00.00'),
 ) + desktop_ua
 mobile_xhr_headers = (
     ('Accept', '*/*'),
     ('Accept-Language', 'en-US,en;q=0.5'),
     ('X-YouTube-Client-Name', '2'),
-    ('X-YouTube-Client-Version', '2.20180830'),
+    ('X-YouTube-Client-Version', '2.20240304.08.00'),
 ) + mobile_ua
 
 
@@ -462,7 +536,7 @@ class RateLimitedQueue(gevent.queue.Queue):
 
 
 def download_thumbnail(save_directory, video_id):
-    url = f"https://i.ytimg.com/vi/{video_id}/hqdefault.jpg"
+    url = f"https://i.ytimg.com/vi/{video_id}/hq720.jpg"
     save_location = os.path.join(save_directory, video_id + ".jpg")
     try:
         thumbnail = fetch_url(url, report_text="Saved thumbnail: " + video_id)
@@ -502,9 +576,40 @@ def video_id(url):
     return urllib.parse.parse_qs(url_parts.query)['v'][0]
 
 
-# default, sddefault, mqdefault, hqdefault, hq720
-def get_thumbnail_url(video_id):
-    return f"{settings.img_prefix}https://i.ytimg.com/vi/{video_id}/hqdefault.jpg"
+def get_thumbnail_url(video_id, quality='hq720'):
+    """Get thumbnail URL with fallback to lower quality if needed.
+
+    Args:
+        video_id: YouTube video ID
+        quality: Preferred quality ('maxres', 'hq720', 'sd', 'hq', 'mq', 'default')
+
+    Returns:
+        Tuple of (best_available_url, quality_used)
+    """
+    # Quality priority order (highest to lowest)
+    quality_order = {
+        'maxres': ['maxresdefault.jpg', 'sddefault.jpg', 'hqdefault.jpg'],
+        'hq720': ['hq720.jpg', 'sddefault.jpg', 'hqdefault.jpg'],
+        'sd': ['sddefault.jpg', 'hqdefault.jpg'],
+        'hq': ['hqdefault.jpg', 'mqdefault.jpg'],
+        'mq': ['mqdefault.jpg', 'default.jpg'],
+        'default': ['default.jpg'],
+    }
+
+    qualities = quality_order.get(quality, quality_order['hq720'])
+    base_url = f"{settings.img_prefix}https://i.ytimg.com/vi/{video_id}/"
+
+    # For now, return the highest quality URL
+    # The browser will handle 404s gracefully with alt text
+    return base_url + qualities[0], qualities[0]
+
+
+def get_best_thumbnail_url(video_id):
+    """Get the best available thumbnail URL for a video.
+
+    Tries hq720 first (for HD videos), falls back to sddefault for SD videos.
+    """
+    return get_thumbnail_url(video_id, quality='hq720')[0]
 
 
 def seconds_to_timestamp(seconds):
@@ -538,6 +643,12 @@ def prefix_url(url):
     if url is None:
         return None
     url = url.lstrip('/')     # some urls have // before them, which has a special meaning
+
+    # Increase resolution for YouTube channel avatars
+    if url and ('ggpht.com' in url or 'yt3.ggpht.com' in url):
+        # Replace size parameter with higher resolution (s240 instead of s88)
+        url = re.sub(r'=s\d+-c-k', '=s240-c-k-c0x00ffffff-no-rj', url)
+
     return '/' + url
 
 
@@ -665,8 +776,183 @@ def to_valid_filename(name):
     return name
 
 
+# https://github.com/yt-dlp/yt-dlp/blob/master/yt_dlp/extractor/youtube.py#L72
+INNERTUBE_CLIENTS = {
+    'android': {
+        'INNERTUBE_API_KEY': 'AIzaSyA8eiZmM1FaDVjRy-df2KTyQ_vz_yYM39w',
+        'INNERTUBE_CONTEXT': {
+            'client': {
+                'hl': 'en',
+                'gl': 'US',
+                'clientName': 'ANDROID',
+                'clientVersion': '19.09.36',
+                'osName': 'Android',
+                'osVersion': '12',
+                'androidSdkVersion': 31,
+                'platform': 'MOBILE',
+                'userAgent': 'com.google.android.youtube/19.09.36 (Linux; U; Android 12; US) gzip'
+            },
+            # https://github.com/yt-dlp/yt-dlp/pull/575#issuecomment-887739287
+            #'thirdParty': {
+            #    'embedUrl': 'https://google.com',  # Can be any valid URL
+            #}
+        },
+        'INNERTUBE_CONTEXT_CLIENT_NAME': 3,
+        'REQUIRE_JS_PLAYER': False,
+    },
+
+    'android-test-suite': {
+        'INNERTUBE_API_KEY': 'AIzaSyA8eiZmM1FaDVjRy-df2KTyQ_vz_yYM39w',
+        'INNERTUBE_CONTEXT': {
+            'client': {
+                'hl': 'en',
+                'gl': 'US',
+                'clientName': 'ANDROID_TESTSUITE',
+                'clientVersion': '1.9',
+                'osName': 'Android',
+                'osVersion': '12',
+                'androidSdkVersion': 31,
+                'platform': 'MOBILE',
+                'userAgent': 'com.google.android.youtube/1.9 (Linux; U; Android 12; US) gzip'
+            },
+            # https://github.com/yt-dlp/yt-dlp/pull/575#issuecomment-887739287
+            #'thirdParty': {
+            #    'embedUrl': 'https://google.com',  # Can be any valid URL
+            #}
+        },
+        'INNERTUBE_CONTEXT_CLIENT_NAME': 3,
+        'REQUIRE_JS_PLAYER': False,
+    },
+
+    'ios': {
+        'INNERTUBE_API_KEY': 'AIzaSyB-63vPrdThhKuerbB2N_l7Kwwcxj6yUAc',
+        'INNERTUBE_CONTEXT': {
+            'client': {
+                'hl': 'en',
+                'gl': 'US',
+                'clientName': 'IOS',
+                'clientVersion': '19.09.3',
+                'deviceModel': 'iPhone14,3',
+                'userAgent': 'com.google.ios.youtube/19.09.3 (iPhone14,3; U; CPU iOS 15_6 like Mac OS X)'
+            }
+        },
+        'INNERTUBE_CONTEXT_CLIENT_NAME': 5,
+        'REQUIRE_JS_PLAYER': False
+    },
+
+    # This client can access age restricted videos (unless the uploader has disabled the 'allow embedding' option)
+    # See: https://github.com/zerodytrash/YouTube-Internal-Clients
+    'tv_embedded': {
+        'INNERTUBE_API_KEY': 'AIzaSyAO_FJ2SlqU8Q4STEHLGCilw_Y9_11qcW8',
+        'INNERTUBE_CONTEXT': {
+            'client': {
+                'hl': 'en',
+                'gl': 'US',
+                'clientName': 'TVHTML5_SIMPLY_EMBEDDED_PLAYER',
+                'clientVersion': '2.0',
+                'clientScreen': 'EMBED',
+            },
+            # https://github.com/yt-dlp/yt-dlp/pull/575#issuecomment-887739287
+            'thirdParty': {
+                'embedUrl': 'https://google.com',  # Can be any valid URL
+            }
+
+        },
+        'INNERTUBE_CONTEXT_CLIENT_NAME': 85,
+        'REQUIRE_JS_PLAYER': True,
+    },
+
+    'web': {
+        'INNERTUBE_API_KEY': 'AIzaSyAO_FJ2SlqU8Q4STEHLGCilw_Y9_11qcW8',
+        'INNERTUBE_CONTEXT': {
+            'client': {
+                'clientName': 'WEB',
+                'clientVersion': '2.20220801.00.00',
+                'userAgent': desktop_user_agent,
+            }
+        },
+        'INNERTUBE_CONTEXT_CLIENT_NAME': 1
+    },
+    'android_vr': {
+        'INNERTUBE_API_KEY': 'AIzaSyA8eiZmM1FaDVjRy-df2KTyQ_vz_yYM39w',
+        'INNERTUBE_CONTEXT': {
+            'client': {
+                'clientName': 'ANDROID_VR',
+                'clientVersion': '1.60.19',
+                'deviceMake': 'Oculus',
+                'deviceModel': 'Quest 3',
+                'androidSdkVersion': 32,
+                'userAgent': 'com.google.android.apps.youtube.vr.oculus/1.60.19 (Linux; U; Android 12L; eureka-user Build/SQ3A.220605.009.A1) gzip',
+                'osName': 'Android',
+                'osVersion': '12L',
+            },
+        },
+        'INNERTUBE_CONTEXT_CLIENT_NAME': 28,
+        'REQUIRE_JS_PLAYER': False,
+    },
+}
+
+def get_visitor_data():
+    visitor_data = None
+    visitor_data_cache = os.path.join(settings.data_dir, 'visitorData.txt')
+    if not os.path.exists(settings.data_dir):
+        os.makedirs(settings.data_dir)
+    if os.path.isfile(visitor_data_cache):
+        with open(visitor_data_cache, 'r') as file:
+            print('Getting visitor_data from cache')
+            visitor_data = file.read()
+        max_age = 12*3600
+        file_age = time.time() - os.path.getmtime(visitor_data_cache)
+        if file_age > max_age:
+            print('visitor_data cache is too old. Removing file...')
+            os.remove(visitor_data_cache)
+        return visitor_data
+
+    print('Fetching youtube homepage to get visitor_data')
+    yt_homepage = 'https://www.youtube.com'
+    yt_resp = fetch_url(yt_homepage, headers={'User-Agent': mobile_user_agent}, report_text='Getting youtube homepage')
+    visitor_data_re = r'''"visitorData":\s*?"(.+?)"'''
+    visitor_data_match = re.search(visitor_data_re, yt_resp.decode())
+    if visitor_data_match:
+        visitor_data = visitor_data_match.group(1)
+        print(f'Got visitor_data: {len(visitor_data)}')
+        with open(visitor_data_cache, 'w') as file:
+            print('Saving visitor_data cache...')
+            file.write(visitor_data)
+        return visitor_data
+    else:
+        print('Unable to get visitor_data value')
+    return visitor_data
+
+def call_youtube_api(client, api, data):
+    client_params = INNERTUBE_CLIENTS[client]
+    context = client_params['INNERTUBE_CONTEXT']
+    key = client_params['INNERTUBE_API_KEY']
+    host = client_params.get('INNERTUBE_HOST') or 'www.youtube.com'
+    user_agent = context['client'].get('userAgent') or mobile_user_agent
+    visitor_data = get_visitor_data()
+
+    url = 'https://' + host + '/youtubei/v1/' + api + '?key=' + key
+    if visitor_data:
+        context['client'].update({'visitorData': visitor_data})
+    data['context'] = context
+
+    data = json.dumps(data)
+    headers = (('Content-Type', 'application/json'),('User-Agent', user_agent))
+    if visitor_data:
+        headers = ( *headers, ('X-Goog-Visitor-Id', visitor_data ))
+    response = fetch_url(
+        url, data=data, headers=headers,
+        debug_name='youtubei_' + api + '_' + client,
+        report_text='Fetched ' + client + ' youtubei ' + api
+    ).decode('utf-8')
+    return response
+
+
 def strip_non_ascii(string):
     ''' Returns the string without non ASCII characters'''
+    if string is None:
+        return ""
     stripped = (c for c in string if 0 < ord(c) < 127)
     return ''.join(stripped)
 

youtube/version.py

@@ -1,3 +1,3 @@
 from __future__ import unicode_literals
 
-__version__ = '0.2.6'
+__version__ = 'v0.4.0'

youtube/watch.py

@@ -6,6 +6,9 @@ import settings
 
 from flask import request
 import flask
+import logging
+
+logger = logging.getLogger(__name__)
 
 import json
 import gevent
@@ -19,51 +22,6 @@ from urllib.parse import parse_qs, urlencode
 from types import SimpleNamespace
 from math import ceil
 
-# https://github.com/yt-dlp/yt-dlp/blob/master/yt_dlp/extractor/youtube.py#L72
-INNERTUBE_CLIENTS = {
-    'android': {
-        'INNERTUBE_API_KEY': 'AIzaSyA8eiZmM1FaDVjRy-df2KTyQ_vz_yYM39w',
-        'INNERTUBE_CONTEXT': {
-            'client': {
-                'hl': 'en',
-                'gl': 'US',
-                'clientName': 'ANDROID',
-                'clientVersion': '17.31.35',
-                'osName': 'Android',
-                'osVersion': '12',
-                'androidSdkVersion': 31,
-                'userAgent': 'com.google.android.youtube/17.31.35 (Linux; U; Android 12) gzip'
-            },
-            # https://github.com/yt-dlp/yt-dlp/pull/575#issuecomment-887739287
-            #'thirdParty': {
-            #    'embedUrl': 'https://google.com',  # Can be any valid URL
-            #}
-        },
-        'INNERTUBE_CONTEXT_CLIENT_NAME': 3,
-        'REQUIRE_JS_PLAYER': False,
-    },
-
-    # This client can access age restricted videos (unless the uploader has disabled the 'allow embedding' option)
-    # See: https://github.com/zerodytrash/YouTube-Internal-Clients
-    'tv_embedded': {
-        'INNERTUBE_API_KEY': 'AIzaSyAO_FJ2SlqU8Q4STEHLGCilw_Y9_11qcW8',
-        'INNERTUBE_CONTEXT': {
-            'client': {
-                'hl': 'en',
-                'gl': 'US',
-                'clientName': 'TVHTML5_SIMPLY_EMBEDDED_PLAYER',
-                'clientVersion': '2.0',
-            },
-            # https://github.com/yt-dlp/yt-dlp/pull/575#issuecomment-887739287
-            'thirdParty': {
-                'embedUrl': 'https://google.com',  # Can be any valid URL
-            }
-
-        },
-        'INNERTUBE_CONTEXT_CLIENT_NAME': 85,
-        'REQUIRE_JS_PLAYER': True,
-    },
-}
 
 try:
     with open(os.path.join(settings.data_dir, 'decrypt_function_cache.json'), 'r') as f:
@@ -386,26 +344,10 @@ def _add_to_error(info, key, additional_message):
 
 
 def fetch_player_response(client, video_id):
-    client_params = INNERTUBE_CLIENTS[client]
-    context = client_params['INNERTUBE_CONTEXT']
-    key = client_params['INNERTUBE_API_KEY']
-    host = client_params.get('INNERTUBE_HOST') or 'www.youtube.com'
-    user_agent = context['client'].get('userAgent') or util.mobile_user_agent
-
-    url = 'https://' + host + '/youtubei/v1/player?key=' + key
-    data = {
+    return util.call_youtube_api(client, 'player', {
         'videoId': video_id,
-        'context': context,
-        'params': 'CgIQBg',
-    }
-    data = json.dumps(data)
-    headers = (('Content-Type', 'application/json'),('User-Agent', user_agent))
-    player_response = util.fetch_url(
-        url, data=data, headers=headers,
-        debug_name='youtubei_player_' + client,
-        report_text='Fetched ' + client + ' youtubei player'
-    ).decode('utf-8')
-    return player_response
+    })
+
 
 def fetch_watch_page_info(video_id, playlist_id, index):
     # bpctr=9999999999 will bypass are-you-sure dialogs for controversial
@@ -428,42 +370,42 @@ def fetch_watch_page_info(video_id, playlist_id, index):
     watch_page = watch_page.decode('utf-8')
     return yt_data_extract.extract_watch_info_from_html(watch_page)
 
+
 def extract_info(video_id, use_invidious, playlist_id=None, index=None):
+    primary_client = 'android_vr'
+    fallback_client = 'ios'
+    last_resort_client = 'tv_embedded'
+
     tasks = (
         # Get video metadata from here
         gevent.spawn(fetch_watch_page_info, video_id, playlist_id, index),
-
-        # Get video URLs by spoofing as android client because its urls don't
-        # require decryption
-        # The URLs returned with WEB for videos requiring decryption
-        # couldn't be decrypted with the base.js from the web page for some
-        # reason
-        # https://github.com/yt-dlp/yt-dlp/issues/574#issuecomment-887171136
-
-        # Update 4/26/23, these URLs will randomly start returning 403
-        # mid-playback and I'm not sure why
-        gevent.spawn(fetch_player_response, 'android', video_id)
+        gevent.spawn(fetch_player_response, primary_client, video_id)
     )
     gevent.joinall(tasks)
     util.check_gevent_exceptions(*tasks)
-    info, player_response = tasks[0].value, tasks[1].value
+
+    info = tasks[0].value or {}
+    player_response = tasks[1].value or {}
 
     yt_data_extract.update_with_new_urls(info, player_response)
 
-    # Age restricted video, retry
-    if info['age_restricted'] or info['player_urls_missing']:
-        if info['age_restricted']:
-            print('Age restricted video, retrying')
-        else:
-            print('Player urls missing, retrying')
-        player_response = fetch_player_response('tv_embedded', video_id)
+    # Fallback to 'ios' if no valid URLs are found
+    if not info.get('formats') or info.get('player_urls_missing'):
+        print(f"No URLs found in '{primary_client}', attempting with '{fallback_client}'.")
+        player_response = fetch_player_response(fallback_client, video_id) or {}
+        yt_data_extract.update_with_new_urls(info, player_response)
+
+    # Final attempt with 'tv_embedded' if there are still no URLs
+    if not info.get('formats') or info.get('player_urls_missing'):
+        print(f"No URLs found in '{fallback_client}', attempting with '{last_resort_client}'")
+        player_response = fetch_player_response(last_resort_client, video_id) or {}
         yt_data_extract.update_with_new_urls(info, player_response)
 
     # signature decryption
-    decryption_error = decrypt_signatures(info, video_id)
-    if decryption_error:
-        decryption_error = 'Error decrypting url signatures: ' + decryption_error
-        info['playability_error'] = decryption_error
+    if info.get('formats'):
+        decryption_error = decrypt_signatures(info, video_id)
+        if decryption_error:
+            info['playability_error'] = 'Error decrypting url signatures: ' + decryption_error
 
     # check if urls ready (non-live format) in former livestream
     # urls not ready if all of them have no filesize
@@ -477,21 +419,21 @@ def extract_info(video_id, use_invidious, playlist_id=None, index=None):
 
     # livestream urls
     # sometimes only the livestream urls work soon after the livestream is over
-    if (info['hls_manifest_url']
-        and (info['live'] or not info['formats'] or not info['urls_ready'])
-    ):
-        manifest = util.fetch_url(info['hls_manifest_url'],
-            debug_name='hls_manifest.m3u8',
-            report_text='Fetched hls manifest'
-        ).decode('utf-8')
-
-        info['hls_formats'], err = yt_data_extract.extract_hls_formats(manifest)
-        if not err:
-            info['playability_error'] = None
-        for fmt in info['hls_formats']:
-            fmt['video_quality'] = video_quality_string(fmt)
-    else:
-        info['hls_formats'] = []
+    info['hls_formats'] = []
+    if info.get('hls_manifest_url') and (info.get('live') or not info.get('formats') or not info['urls_ready']):
+        try:
+            manifest = util.fetch_url(info['hls_manifest_url'],
+                debug_name='hls_manifest.m3u8',
+                report_text='Fetched hls manifest'
+            ).decode('utf-8')
+            info['hls_formats'], err = yt_data_extract.extract_hls_formats(manifest)
+            if not err:
+                info['playability_error'] = None
+            for fmt in info['hls_formats']:
+                fmt['video_quality'] = video_quality_string(fmt)
+        except Exception as e:
+            print(f"Error obteniendo HLS manifest: {e}")
+            info['hls_formats'] = []
 
     # check for 403. Unnecessary for tor video routing b/c ip address is same
     info['invidious_used'] = False
@@ -686,7 +628,12 @@ def get_watch_page(video_id=None):
 
     # prefix urls, and other post-processing not handled by yt_data_extract
     for item in info['related_videos']:
-        item['thumbnail'] = "https://i.ytimg.com/vi/{}/hqdefault.jpg".format(item['id']) # set HQ relateds thumbnail videos
+        # For playlists, use first_video_id for thumbnail, not playlist id
+        if item.get('type') == 'playlist' and item.get('first_video_id'):
+            item['thumbnail'] = "https://i.ytimg.com/vi/{}/hq720.jpg".format(item['first_video_id'])
+        elif item.get('type') == 'video':
+            item['thumbnail'] = "https://i.ytimg.com/vi/{}/hq720.jpg".format(item['id'])
+        # For other types, keep existing thumbnail or skip
         util.prefix_urls(item)
         util.add_extra_html_info(item)
     for song in info['music_list']:
@@ -694,6 +641,9 @@ def get_watch_page(video_id=None):
     if info['playlist']:
         playlist_id = info['playlist']['id']
         for item in info['playlist']['items']:
+            # Set high quality thumbnail for playlist videos
+            if item.get('type') == 'video' and item.get('id'):
+                item['thumbnail'] = "https://i.ytimg.com/vi/{}/hq720.jpg".format(item['id'])
             util.prefix_urls(item)
             util.add_extra_html_info(item)
             if playlist_id:
@@ -720,12 +670,6 @@ def get_watch_page(video_id=None):
             '/videoplayback',
             '/videoplayback/name/' + filename)
 
-    if settings.gather_googlevideo_domains:
-        with open(os.path.join(settings.data_dir, 'googlevideo-domains.txt'), 'a+', encoding='utf-8') as f:
-            url = info['formats'][0]['url']
-            subdomain = url[0:url.find(".googlevideo.com")]
-            f.write(subdomain + "\n")
-
     download_formats = []
 
     for format in (info['formats'] + info['hls_formats']):
@@ -752,6 +696,30 @@ def get_watch_page(video_id=None):
     pair_sources = source_info['pair_sources']
     uni_idx, pair_idx = source_info['uni_idx'], source_info['pair_idx']
 
+    # Extract audio tracks using yt-dlp for multi-language support
+    audio_tracks = []
+    try:
+        from youtube import ytdlp_integration
+        logger.info(f'Extracting audio tracks for video: {video_id}')
+        ytdlp_info = ytdlp_integration.extract_video_info_ytdlp(video_id)
+        audio_tracks = ytdlp_info.get('audio_tracks', [])
+
+        if audio_tracks:
+            logger.info(f'✓ Found {len(audio_tracks)} audio tracks:')
+            for i, track in enumerate(audio_tracks[:10], 1):  # Log first 10
+                logger.info(f'  [{i}] {track["language_name"]} ({track["language"]}) - '
+                          f'bitrate: {track.get("audio_bitrate", "N/A")}k, '
+                          f'codec: {track.get("acodec", "N/A")}, '
+                          f'format_id: {track.get("format_id", "N/A")}')
+            if len(audio_tracks) > 10:
+                logger.info(f'  ... and {len(audio_tracks) - 10} more')
+        else:
+            logger.warning(f'No audio tracks found for video {video_id}')
+
+    except Exception as e:
+        logger.error(f'Failed to extract audio tracks: {e}', exc_info=True)
+        audio_tracks = []
+
     pair_quality = yt_data_extract.deep_get(pair_sources, pair_idx, 'quality')
     uni_quality = yt_data_extract.deep_get(uni_sources, uni_idx, 'quality')
 
@@ -765,9 +733,17 @@ def get_watch_page(video_id=None):
     else:
         closer_to_target = 'pair'
 
-    using_pair_sources = (
-        bool(pair_sources) and (not uni_sources or closer_to_target == 'pair')
-    )
+    if settings.prefer_uni_sources == 2:
+        # Use uni sources unless there's no choice.
+        using_pair_sources = (
+            bool(pair_sources) and (not uni_sources)
+        )
+    else:
+        # Use the pair sources if they're closer to the desired resolution
+        using_pair_sources = (
+            bool(pair_sources)
+            and (not uni_sources or closer_to_target == 'pair')
+        )
     if using_pair_sources:
         video_height = pair_sources[pair_idx]['height']
         video_width = pair_sources[pair_idx]['width']
@@ -867,7 +843,9 @@ def get_watch_page(video_id=None):
             'playlist': info['playlist'],
             'related': info['related_videos'],
             'playability_error': info['playability_error'],
+            'audio_tracks': audio_tracks,
         },
+        audio_tracks = audio_tracks,
         font_family = youtube.font_choices[settings.font], # for embed page
         **source_info,
         using_pair_sources = using_pair_sources,
@@ -876,9 +854,17 @@ def get_watch_page(video_id=None):
 
 @yt_app.route('/api/<path:dummy>')
 def get_captions(dummy):
-    result = util.fetch_url('https://www.youtube.com' + request.full_path)
-    result = result.replace(b"align:start position:0%", b"")
-    return result
+    try:
+        result = util.fetch_url('https://www.youtube.com' + request.full_path)
+        result = result.replace(b"align:start position:0%", b"")
+        return result
+    except util.FetchError as e:
+        # Return empty captions gracefully instead of error page
+        logger.warning(f'Failed to fetch captions: {e}')
+        return flask.Response(b'WEBVTT\n\n', mimetype='text/vtt', status=200)
+    except Exception as e:
+        logger.error(f'Unexpected error fetching captions: {e}')
+        return flask.Response(b'WEBVTT\n\n', mimetype='text/vtt', status=200)
 
 
 times_reg = re.compile(r'^\d\d:\d\d:\d\d\.\d\d\d --> \d\d:\d\d:\d\d\.\d\d\d.*$')
@@ -943,3 +929,18 @@ def get_transcript(caption_path):
 
     return flask.Response(result.encode('utf-8'),
                           mimetype='text/plain;charset=UTF-8')
+
+
+# ============================================================================
+# yt-dlp Integration Routes
+# ============================================================================
+
+@yt_app.route('/ytl-api/video-with-audio/<video_id>')
+def proxy_video_with_audio(video_id):
+    """
+    Proxy para servir video con audio específico usando yt-dlp
+    """
+    from youtube import ytdlp_proxy
+    audio_lang = request.args.get('lang', 'en')
+    max_quality = int(request.args.get('quality', 720))
+    return ytdlp_proxy.stream_video_with_audio(video_id, audio_lang, max_quality)

youtube/yt_data_extract/common.py

@@ -185,7 +185,7 @@ def extract_int(string, default=None, whole_word=True):
         return default
 
 def extract_approx_int(string):
-    '''e.g. "15.1M" from "15.1M subscribers"'''
+    '''e.g. "15.1M" from "15.1M subscribers" or '4,353' from 4353'''
     if not isinstance(string, str):
         string = extract_str(string)
     if not string:
@@ -193,7 +193,10 @@ def extract_approx_int(string):
     match = re.search(r'\b(\d+(?:\.\d+)?[KMBTkmbt]?)\b', string.replace(',', ''))
     if match is None:
         return None
-    return match.group(1)
+    result = match.group(1)
+    if re.fullmatch(r'\d+', result):
+        result = '{:,}'.format(int(result))
+    return result
 
 MONTH_ABBREVIATIONS = {'jan':'1', 'feb':'2', 'mar':'3', 'apr':'4', 'may':'5', 'jun':'6', 'jul':'7', 'aug':'8', 'sep':'9', 'oct':'10', 'nov':'11', 'dec':'12'}
 def extract_date(date_text):
@@ -223,6 +226,89 @@ def check_missing_keys(object, *key_sequences):
 
     return None
 
+
+def extract_lockup_view_model_info(item, additional_info={}):
+    """Extract info from new lockupViewModel format (YouTube 2024+)"""
+    info = {'error': None}
+
+    content_type = item.get('contentType', '')
+    content_id = item.get('contentId', '')
+
+    # Extract title from metadata
+    metadata = item.get('metadata', {})
+    lockup_metadata = metadata.get('lockupMetadataViewModel', {})
+    title_data = lockup_metadata.get('title', {})
+    info['title'] = title_data.get('content', '')
+
+    # Determine type based on contentType
+    if 'PLAYLIST' in content_type:
+        info['type'] = 'playlist'
+        info['playlist_type'] = 'playlist'
+        info['id'] = content_id
+        info['video_count'] = None
+        info['first_video_id'] = None
+
+        # Try to get video count from metadata
+        metadata_rows = lockup_metadata.get('metadata', {})
+        for row in metadata_rows.get('contentMetadataViewModel', {}).get('metadataRows', []):
+            for part in row.get('metadataParts', []):
+                text = part.get('text', {}).get('content', '')
+                if 'video' in text.lower():
+                    info['video_count'] = extract_int(text)
+    elif 'VIDEO' in content_type:
+        info['type'] = 'video'
+        info['id'] = content_id
+        info['view_count'] = None
+        info['approx_view_count'] = None
+        info['time_published'] = None
+        info['duration'] = None
+
+        # Extract duration/other info from metadata rows
+        metadata_rows = lockup_metadata.get('metadata', {})
+        for row in metadata_rows.get('contentMetadataViewModel', {}).get('metadataRows', []):
+            for part in row.get('metadataParts', []):
+                text = part.get('text', {}).get('content', '')
+                if 'view' in text.lower():
+                    info['approx_view_count'] = extract_approx_int(text)
+                elif 'ago' in text.lower():
+                    info['time_published'] = text
+    elif 'CHANNEL' in content_type:
+        info['type'] = 'channel'
+        info['id'] = content_id
+        info['approx_subscriber_count'] = None
+    else:
+        info['type'] = 'unsupported'
+        return info
+
+    # Extract thumbnail from contentImage
+    content_image = item.get('contentImage', {})
+    collection_thumb = content_image.get('collectionThumbnailViewModel', {})
+    primary_thumb = collection_thumb.get('primaryThumbnail', {})
+    thumb_vm = primary_thumb.get('thumbnailViewModel', {})
+    image_sources = thumb_vm.get('image', {}).get('sources', [])
+    if image_sources:
+        info['thumbnail'] = image_sources[0].get('url', '')
+    else:
+        info['thumbnail'] = ''
+
+    # Extract author info if available
+    info['author'] = None
+    info['author_id'] = None
+    info['author_url'] = None
+
+    # Try to get first video ID from inline player data
+    item_playback = item.get('itemPlayback', {})
+    inline_player = item_playback.get('inlinePlayerData', {})
+    on_select = inline_player.get('onSelect', {})
+    innertube_cmd = on_select.get('innertubeCommand', {})
+    watch_endpoint = innertube_cmd.get('watchEndpoint', {})
+    if watch_endpoint.get('videoId'):
+        info['first_video_id'] = watch_endpoint.get('videoId')
+
+    info.update(additional_info)
+    return info
+
+
 def extract_item_info(item, additional_info={}):
     if not item:
         return {'error': 'No item given'}
@@ -240,6 +326,10 @@ def extract_item_info(item, additional_info={}):
         info['type'] = 'unsupported'
         return info
 
+    # Handle new lockupViewModel format (YouTube 2024+)
+    if type == 'lockupViewModel':
+        return extract_lockup_view_model_info(item, additional_info)
+
     # type looks like e.g. 'compactVideoRenderer' or 'gridVideoRenderer'
     # camelCase split, https://stackoverflow.com/a/37697078
     type_parts = [s.lower() for s in re.sub(r'([A-Z][a-z]+)', r' \1', type).split()]
@@ -438,6 +528,9 @@ _item_types = {
     'channelRenderer',
     'compactChannelRenderer',
     'gridChannelRenderer',
+
+    # New viewModel format (YouTube 2024+)
+    'lockupViewModel',
 }
 
 def _traverse_browse_renderer(renderer):

youtube/yt_data_extract/everything_else.py

@@ -85,23 +85,84 @@ def extract_channel_info(polymer_json, tab, continuation=False):
         if tab in ('search', 'playlists'):
             info['is_last_page'] = (ctoken is None)
     elif tab == 'about':
-        items, _ = extract_items(response, item_types={'channelAboutFullMetadataRenderer'})
-        if not items:
-            info['error'] = 'Could not find channelAboutFullMetadataRenderer'
-            return info
-        channel_metadata = items[0]['channelAboutFullMetadataRenderer']
+        # Latest type
+        items, _ = extract_items(response, item_types={'aboutChannelRenderer'})
+        if items:
+            a_metadata = deep_get(items, 0, 'aboutChannelRenderer',
+                'metadata', 'aboutChannelViewModel')
+            if not a_metadata:
+                info['error'] = 'Could not find aboutChannelViewModel'
+                return info
 
-        info['links'] = []
-        for link_json in channel_metadata.get('primaryLinks', ()):
-            url = remove_redirect(deep_get(link_json, 'navigationEndpoint', 'urlEndpoint', 'url'))
-            if not (url.startswith('http://') or url.startswith('https://')):
-                url = 'http://' + url
-            text = extract_str(link_json.get('title'))
-            info['links'].append( (text, url) )
+            info['links'] = []
+            for link_outer in a_metadata.get('links', ()):
+                link = link_outer.get('channelExternalLinkViewModel') or {}
+                link_content = extract_str(deep_get(link, 'link', 'content'))
+                for run in deep_get(link, 'link', 'commandRuns') or ():
+                    url = remove_redirect(deep_get(run, 'onTap',
+                        'innertubeCommand', 'urlEndpoint', 'url'))
+                    if url and not (url.startswith('http://')
+                            or url.startswith('https://')):
+                        url = 'https://' + url
+                    if link_content is None or (link_content in url):
+                        break
+                else: # didn't break
+                    url = link_content
+                    if url and not (url.startswith('http://')
+                            or url.startswith('https://')):
+                        url = 'https://' + url
+                text = extract_str(deep_get(link, 'title', 'content'))
+                info['links'].append( (text, url) )
 
-        info['date_joined'] = extract_date(channel_metadata.get('joinedDateText'))
-        info['view_count'] = extract_int(channel_metadata.get('viewCountText'))
-        info['description'] = extract_str(channel_metadata.get('description'), default='')
+            info['date_joined'] = extract_date(
+                a_metadata.get('joinedDateText')
+            )
+            info['view_count'] = extract_int(a_metadata.get('viewCountText'))
+            info['approx_view_count'] = extract_approx_int(
+                a_metadata.get('viewCountText')
+            )
+            info['description'] = extract_str(
+                a_metadata.get('description'), default=''
+            )
+            info['approx_video_count'] = extract_approx_int(
+                a_metadata.get('videoCountText')
+            )
+            info['approx_subscriber_count'] = extract_approx_int(
+                a_metadata.get('subscriberCountText')
+            )
+            info['country'] = extract_str(a_metadata.get('country'))
+            info['canonical_url'] = extract_str(
+                a_metadata.get('canonicalChannelUrl')
+            )
+
+        # Old type
+        else:
+            items, _ = extract_items(response,
+                item_types={'channelAboutFullMetadataRenderer'})
+            if not items:
+                info['error'] = 'Could not find aboutChannelRenderer or channelAboutFullMetadataRenderer'
+                return info
+            a_metadata = items[0]['channelAboutFullMetadataRenderer']
+
+            info['links'] = []
+            for link_json in a_metadata.get('primaryLinks', ()):
+                url = remove_redirect(deep_get(link_json, 'navigationEndpoint',
+                    'urlEndpoint', 'url'))
+                if url and not (url.startswith('http://')
+                                or url.startswith('https://')):
+                    url = 'https://' + url
+                text = extract_str(link_json.get('title'))
+                info['links'].append( (text, url) )
+
+            info['date_joined'] = extract_date(a_metadata.get('joinedDateText'))
+            info['view_count'] = extract_int(a_metadata.get('viewCountText'))
+            info['description'] = extract_str(a_metadata.get(
+                'description'), default='')
+
+            info['approx_video_count'] = None
+            info['approx_subscriber_count'] = None
+            info['country'] = None
+            info['canonical_url'] = None
     else:
         raise NotImplementedError('Unknown or unsupported channel tab: ' + tab)
 
@@ -168,7 +229,7 @@ def extract_playlist_metadata(polymer_json):
     if metadata['first_video_id'] is None:
         metadata['thumbnail'] = None
     else:
-        metadata['thumbnail'] = f"https://i.ytimg.com/vi/{metadata['first_video_id']}/hqdefault.jpg"
+        metadata['thumbnail'] = f"https://i.ytimg.com/vi/{metadata['first_video_id']}/hq720.jpg"
 
     metadata['video_count'] = extract_int(header.get('numVideosText'))
     metadata['description'] = extract_str(header.get('descriptionText'), default='')
@@ -191,6 +252,19 @@ def extract_playlist_metadata(polymer_json):
         elif 'updated' in text:
             metadata['time_published'] = extract_date(text)
 
+    microformat = deep_get(response, 'microformat', 'microformatDataRenderer',
+                           default={})
+    conservative_update(
+        metadata, 'title', extract_str(microformat.get('title'))
+    )
+    conservative_update(
+        metadata, 'description', extract_str(microformat.get('description'))
+    )
+    conservative_update(
+        metadata, 'thumbnail', deep_get(microformat, 'thumbnail',
+                                        'thumbnails', -1, 'url')
+    )
+
     return metadata
 
 def extract_playlist_info(polymer_json):
@@ -198,13 +272,11 @@ def extract_playlist_info(polymer_json):
     if err:
         return {'error': err}
     info = {'error': None}
-    first_page = 'continuationContents' not in response
     video_list, _ = extract_items(response)
 
     info['items'] = [extract_item_info(renderer) for renderer in video_list]
 
-    if first_page:
-        info['metadata'] = extract_playlist_metadata(polymer_json)
+    info['metadata'] = extract_playlist_metadata(polymer_json)
 
     return info
 

youtube/yt_data_extract/watch_extraction.py

@@ -140,11 +140,12 @@ def _extract_likes_dislikes(renderer_content):
             ['defaultText', 'accessibility', 'accessibilityData', 'label'],
             ['accessibility', 'label'],
             ['accessibilityData', 'accessibilityData', 'label'],
+            ['accessibilityText'],
         ))
 
         # this count doesn't have all the digits, it's like 53K for instance
-        dumb_count = extract_int(extract_str(deep_get(
-            toggle_button_renderer, 'defaultText')))
+        dumb_count = extract_int(extract_str(multi_get(
+            toggle_button_renderer, ['defaultText', 'title'])))
 
         # The accessibility text will be "No likes" or "No dislikes" or
         # something like that, but dumb count will be 0
@@ -168,16 +169,23 @@ def _extract_likes_dislikes(renderer_content):
                 info['dislike_count'] = count
         elif 'slimMetadataButtonRenderer' in button:
             button_renderer = button['slimMetadataButtonRenderer']
-            liberal_update(info, 'like_count', extract_button_count(deep_get(
-                button_renderer, 'button',
-                'segmentedLikeDislikeButtonRenderer',
-                'likeButton', 'toggleButtonRenderer'
-            )))
-            liberal_update(info, 'dislike_count',extract_button_count(deep_get(
-                button_renderer, 'button',
-                'segmentedLikeDislikeButtonRenderer',
-                'dislikeButton', 'toggleButtonRenderer'
-            )))
+            liberal_update(info, 'like_count', extract_button_count(
+                multi_deep_get(button_renderer,
+                    ['button', 'segmentedLikeDislikeButtonRenderer',
+                     'likeButton', 'toggleButtonRenderer'],
+                    ['button', 'segmentedLikeDislikeButtonViewModel',
+                     'likeButtonViewModel', 'likeButtonViewModel',
+                     'toggleButtonViewModel', 'toggleButtonViewModel',
+                     'defaultButtonViewModel', 'buttonViewModel']
+                )
+            ))
+            '''liberal_update(info, 'dislike_count', extract_button_count(
+                deep_get(
+                    button_renderer, 'button',
+                    'segmentedLikeDislikeButtonRenderer',
+                    'dislikeButton', 'toggleButtonRenderer'
+                )
+            ))'''
     return info
 
 def _extract_from_owner_renderer(renderer_content):
@@ -363,12 +371,12 @@ def _extract_watch_info_mobile(top_level):
             comment_count_text = extract_str(deep_get(comment_info,
                 'header', 'commentSectionHeaderRenderer', 'countText'))
         if comment_count_text == 'Comments':    # just this with no number, means 0 comments
-            info['comment_count'] = 0
+            info['comment_count'] = '0'
         else:
-            info['comment_count'] = extract_int(comment_count_text)
+            info['comment_count'] = extract_approx_int(comment_count_text)
         info['comments_disabled'] = False
     else:   # no comment section present means comments are disabled
-        info['comment_count'] = 0
+        info['comment_count'] = '0'
         info['comments_disabled'] = True
 
     # check for limited state

youtube/ytdlp_integration.py

@@ -0,0 +1,78 @@
+#!/usr/bin/env python3
+"""
+yt-dlp integration wrapper for backward compatibility.
+
+This module now uses the centralized ytdlp_service for all operations.
+"""
+import logging
+from youtube.ytdlp_service import (
+    extract_video_info,
+    get_language_name,
+    clear_cache,
+    get_cache_info,
+)
+
+logger = logging.getLogger(__name__)
+
+
+def extract_video_info_ytdlp(video_id):
+    """
+    Extract video information using yt-dlp (with caching).
+
+    This is a wrapper around ytdlp_service.extract_video_info()
+    for backward compatibility.
+
+    Args:
+        video_id: YouTube video ID
+
+    Returns:
+        Dictionary with audio_tracks, formats, title, duration
+    """
+    logger.debug(f'Extracting video info (legacy API): {video_id}')
+
+    info = extract_video_info(video_id)
+
+    # Convert to legacy format for backward compatibility
+    return {
+        'audio_tracks': info.get('audio_tracks', []),
+        'all_audio_formats': info.get('formats', []),
+        'formats': info.get('formats', []),
+        'title': info.get('title', ''),
+        'duration': info.get('duration', 0),
+        'error': info.get('error'),
+    }
+
+
+def get_audio_formats_for_language(video_id, language='en'):
+    """
+    Get available audio formats for a specific language.
+
+    Args:
+        video_id: YouTube video ID
+        language: Language code (default: 'en')
+
+    Returns:
+        List of audio format dicts
+    """
+    info = extract_video_info_ytdlp(video_id)
+
+    if 'error' in info:
+        logger.warning(f'Cannot get audio formats: {info["error"]}')
+        return []
+
+    audio_formats = []
+    for track in info.get('audio_tracks', []):
+        if track['language'] == language:
+            audio_formats.append(track)
+
+    logger.debug(f'Found {len(audio_formats)} {language} audio formats')
+    return audio_formats
+
+
+__all__ = [
+    'extract_video_info_ytdlp',
+    'get_audio_formats_for_language',
+    'get_language_name',
+    'clear_cache',
+    'get_cache_info',
+]

youtube/ytdlp_proxy.py

@@ -0,0 +1,99 @@
+#!/usr/bin/env python3
+"""
+Proxy for serving videos with specific audio using yt-dlp.
+
+This module provides streaming functionality for unified formats
+with specific audio languages.
+"""
+import logging
+from flask import Response, request, stream_with_context
+import urllib.request
+import urllib.error
+from youtube.ytdlp_service import find_best_unified_format
+
+logger = logging.getLogger(__name__)
+
+
+def stream_video_with_audio(video_id: str, audio_language: str = 'en', max_quality: int = 720):
+    """
+    Stream video with specific audio language.
+
+    Args:
+        video_id: YouTube video ID
+        audio_language: Preferred audio language (default: 'en')
+        max_quality: Maximum video height (default: 720)
+
+    Returns:
+        Flask Response with video stream, or 404 if not available
+    """
+    logger.info(f'Stream request: {video_id} | audio={audio_language} | quality={max_quality}p')
+
+    # Find best unified format
+    best_format = find_best_unified_format(video_id, audio_language, max_quality)
+
+    if not best_format:
+        logger.info(f'No suitable unified format found, returning 404 to trigger fallback')
+        return Response('No suitable unified format available', status=404)
+
+    url = best_format.get('url')
+    if not url:
+        logger.error('Format found but no URL available')
+        return Response('Format URL not available', status=500)
+
+    logger.debug(f'Streaming from: {url[:80]}...')
+
+    # Stream the video
+    try:
+        req = urllib.request.Request(url)
+        req.add_header('User-Agent', 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36')
+        req.add_header('Accept', '*/*')
+
+        # Add Range header if client requests it
+        if 'Range' in request.headers:
+            req.add_header('Range', request.headers['Range'])
+            logger.debug(f'Range request: {request.headers["Range"]}')
+
+        resp = urllib.request.urlopen(req, timeout=60)
+
+        def generate():
+            """Generator for streaming video chunks."""
+            try:
+                while True:
+                    chunk = resp.read(65536)  # 64KB chunks
+                    if not chunk:
+                        break
+                    yield chunk
+            except Exception as e:
+                logger.error(f'Stream error: {e}')
+                raise
+
+        # Build response headers
+        response_headers = {
+            'Content-Type': resp.headers.get('Content-Type', 'video/mp4'),
+            'Access-Control-Allow-Origin': '*',
+        }
+
+        # Copy important headers
+        for header in ['Content-Length', 'Content-Range', 'Accept-Ranges']:
+            if header in resp.headers:
+                response_headers[header] = resp.headers[header]
+
+        status_code = resp.getcode()
+        logger.info(f'Streaming started: {status_code}')
+
+        return Response(
+            stream_with_context(generate()),
+            status=status_code,
+            headers=response_headers,
+            direct_passthrough=True
+        )
+
+    except urllib.error.HTTPError as e:
+        logger.error(f'HTTP error streaming: {e.code} {e.reason}')
+        return Response(f'Error: {e.code} {e.reason}', status=e.code)
+    except urllib.error.URLError as e:
+        logger.error(f'URL error streaming: {e.reason}')
+        return Response(f'Network error: {e.reason}', status=502)
+    except Exception as e:
+        logger.error(f'Streaming error: {e}', exc_info=True)
+        return Response(f'Error: {e}', status=500)

youtube/ytdlp_service.py

@@ -0,0 +1,393 @@
+#!/usr/bin/env python3
+"""
+Centralized yt-dlp integration with caching, logging, and error handling.
+
+This module provides a clean interface for yt-dlp functionality:
+- Multi-language audio track extraction
+- Subtitle extraction
+- Age-restricted video support
+
+All yt-dlp usage should go through this module for consistency.
+"""
+import logging
+from functools import lru_cache
+from typing import Dict, List, Optional, Any
+import yt_dlp
+import settings
+
+logger = logging.getLogger(__name__)
+
+# Language name mapping
+LANGUAGE_NAMES = {
+    'en': 'English',
+    'es': 'Español',
+    'fr': 'Français',
+    'de': 'Deutsch',
+    'it': 'Italiano',
+    'pt': 'Português',
+    'ru': 'Русский',
+    'ja': '日本語',
+    'ko': '한국어',
+    'zh': '中文',
+    'ar': 'العربية',
+    'hi': 'हिन्दी',
+    'und': 'Unknown',
+    'zxx': 'No linguistic content',
+}
+
+
+def get_language_name(lang_code: str) -> str:
+    """Convert ISO 639-1/2 language code to readable name."""
+    if not lang_code:
+        return 'Unknown'
+    return LANGUAGE_NAMES.get(lang_code.lower(), lang_code.upper())
+
+
+def _get_ytdlp_config() -> Dict[str, Any]:
+    """Get yt-dlp configuration from settings."""
+    config = {
+        'quiet': True,
+        'no_warnings': True,
+        'extract_flat': False,
+        'format': 'best',
+        'skip_download': True,
+        'socket_timeout': 30,
+        'extractor_retries': 3,
+        'http_chunk_size': 10485760,  # 10MB
+    }
+
+    # Configure Tor proxy if enabled
+    if settings.route_tor:
+        config['proxy'] = 'socks5://127.0.0.1:9150'
+        logger.debug('Tor proxy enabled for yt-dlp')
+
+    # Use cookies if available
+    import os
+    cookies_file = 'youtube_cookies.txt'
+    if os.path.exists(cookies_file):
+        config['cookiefile'] = cookies_file
+        logger.debug('Using cookies file for yt-dlp')
+
+    return config
+
+
+@lru_cache(maxsize=128)
+def extract_video_info(video_id: str) -> Dict[str, Any]:
+    """
+    Extract video information using yt-dlp with caching.
+
+    Args:
+        video_id: YouTube video ID
+
+    Returns:
+        Dictionary with video information including audio tracks
+
+    Caching:
+        Results are cached to avoid repeated requests to YouTube.
+        Cache size is limited to prevent memory issues.
+    """
+    # Check if yt-dlp is enabled
+    if not getattr(settings, 'ytdlp_enabled', True):
+        logger.debug('yt-dlp integration is disabled')
+        return {'error': 'yt-dlp disabled', 'audio_tracks': []}
+
+    url = f'https://www.youtube.com/watch?v={video_id}'
+    ydl_opts = _get_ytdlp_config()
+
+    try:
+        logger.debug(f'Extracting video info: {video_id}')
+
+        with yt_dlp.YoutubeDL(ydl_opts) as ydl:
+            info = ydl.extract_info(url, download=False)
+
+            if not info:
+                logger.warning(f'No info returned for video: {video_id}')
+                return {'error': 'No info returned', 'audio_tracks': []}
+
+            logger.info(f'Extracted {len(info.get("formats", []))} total formats')
+
+            # Extract audio tracks grouped by language
+            audio_tracks = _extract_audio_tracks(info)
+
+            return {
+                'video_id': video_id,
+                'title': info.get('title', ''),
+                'duration': info.get('duration', 0),
+                'audio_tracks': audio_tracks,
+                'formats': info.get('formats', []),
+                'subtitles': info.get('subtitles', {}),
+                'automatic_captions': info.get('automatic_captions', {}),
+            }
+
+    except yt_dlp.utils.DownloadError as e:
+        logger.error(f'yt-dlp download error for {video_id}: {e}')
+        return {'error': str(e), 'audio_tracks': []}
+    except Exception as e:
+        logger.error(f'yt-dlp extraction error for {video_id}: {e}', exc_info=True)
+        return {'error': str(e), 'audio_tracks': []}
+
+
+def _extract_audio_tracks(info: Dict[str, Any]) -> List[Dict[str, Any]]:
+    """
+    Extract audio tracks from video info, grouped by language.
+
+    Returns a list of unique audio tracks (one per language),
+    keeping the highest quality for each language.
+    """
+    audio_by_language = {}
+    all_formats = info.get('formats', [])
+
+    logger.debug(f'Processing {len(all_formats)} formats to extract audio tracks')
+
+    for fmt in all_formats:
+        # Only audio-only formats
+        has_audio = fmt.get('acodec') and fmt.get('acodec') != 'none'
+        has_video = fmt.get('vcodec') and fmt.get('vcodec') != 'none'
+
+        if not has_audio or has_video:
+            continue
+
+        # Extract language information
+        lang = (
+            fmt.get('language') or
+            fmt.get('audio_language') or
+            fmt.get('lang') or
+            'und'
+        )
+
+        # Get language name
+        lang_name = (
+            fmt.get('language_name') or
+            fmt.get('lang_name') or
+            get_language_name(lang)
+        )
+
+        # Get bitrate
+        bitrate = fmt.get('abr') or fmt.get('tbr') or 0
+
+        # Create track info
+        track_info = {
+            'language': lang,
+            'language_name': lang_name,
+            'format_id': str(fmt.get('format_id', '')),
+            'itag': str(fmt.get('format_id', '')),
+            'ext': fmt.get('ext'),
+            'acodec': fmt.get('acodec'),
+            'audio_bitrate': int(bitrate) if bitrate else 0,
+            'audio_sample_rate': fmt.get('asr'),
+            'url': fmt.get('url'),
+            'filesize': fmt.get('filesize'),
+        }
+
+        # Keep best quality per language
+        lang_key = lang.lower()
+        if lang_key not in audio_by_language:
+            audio_by_language[lang_key] = track_info
+            logger.debug(f'  Added {lang} ({lang_name}) - {bitrate}k')
+        else:
+            current_bitrate = audio_by_language[lang_key].get('audio_bitrate', 0)
+            if bitrate > current_bitrate:
+                logger.debug(f'  Updated {lang} ({lang_name}): {current_bitrate}k → {bitrate}k')
+                audio_by_language[lang_key] = track_info
+
+    # Convert to list and sort
+    audio_tracks = list(audio_by_language.values())
+
+    # Sort: English first, then by bitrate (descending)
+    audio_tracks.sort(
+        key=lambda x: (
+            0 if x['language'] == 'en' else 1,
+            -x.get('audio_bitrate', 0)
+        )
+    )
+
+    logger.info(f'Extracted {len(audio_tracks)} unique audio languages')
+    for track in audio_tracks[:5]:  # Log first 5
+        logger.info(f'  → {track["language_name"]} ({track["language"]}): {track["audio_bitrate"]}k')
+
+    return audio_tracks
+
+
+def get_subtitle_url(video_id: str, lang: str = 'en') -> Optional[str]:
+    """
+    Get subtitle URL for a specific language.
+
+    Args:
+        video_id: YouTube video ID
+        lang: Language code (default: 'en')
+
+    Returns:
+        URL to subtitle file, or None if not available
+    """
+    info = extract_video_info(video_id)
+
+    if 'error' in info:
+        logger.warning(f'Cannot get subtitles: {info["error"]}')
+        return None
+
+    # Try manual subtitles first
+    subtitles = info.get('subtitles', {})
+    if lang in subtitles:
+        for sub in subtitles[lang]:
+            if sub.get('ext') == 'vtt':
+                logger.debug(f'Found manual {lang} subtitle')
+                return sub.get('url')
+
+    # Try automatic captions
+    auto_captions = info.get('automatic_captions', {})
+    if lang in auto_captions:
+        for sub in auto_captions[lang]:
+            if sub.get('ext') == 'vtt':
+                logger.debug(f'Found automatic {lang} subtitle')
+                return sub.get('url')
+
+    logger.debug(f'No {lang} subtitle found')
+    return None
+
+
+def find_best_unified_format(
+    video_id: str,
+    audio_language: str = 'en',
+    max_quality: int = 720
+) -> Optional[Dict[str, Any]]:
+    """
+    Find best unified (video+audio) format for specific language and quality.
+
+    Args:
+        video_id: YouTube video ID
+        audio_language: Preferred audio language
+        max_quality: Maximum video height (e.g., 720, 1080)
+
+    Returns:
+        Format dict if found, None otherwise
+    """
+    info = extract_video_info(video_id)
+
+    if 'error' in info or not info.get('formats'):
+        return None
+
+    # Quality thresholds (minimum acceptable height as % of requested)
+    thresholds = {
+        2160: 0.85,
+        1440: 0.80,
+        1080: 0.70,
+        720: 0.70,
+        480: 0.60,
+        360: 0.50,
+    }
+
+    # Get threshold for requested quality
+    threshold = 0.70
+    for q, t in thresholds.items():
+        if max_quality >= q:
+            threshold = t
+            break
+
+    min_height = int(max_quality * threshold)
+    logger.debug(f'Quality threshold: {threshold:.0%} = min {min_height}p for {max_quality}p')
+
+    candidates = []
+    audio_lang_lower = audio_language.lower()
+
+    for fmt in info['formats']:
+        # Must have both video and audio
+        has_video = fmt.get('vcodec') and fmt.get('vcodec') != 'none'
+        has_audio = fmt.get('acodec') and fmt.get('acodec') != 'none'
+
+        if not (has_video and has_audio):
+            continue
+
+        # Skip HLS/DASH formats
+        protocol = fmt.get('protocol', '')
+        format_id = str(fmt.get('format_id', ''))
+
+        if any(x in protocol.lower() for x in ['m3u8', 'hls', 'dash']):
+            continue
+        if format_id.startswith('9'):  # HLS formats
+            continue
+
+        height = fmt.get('height', 0)
+        if height < min_height:
+            continue
+
+        # Language matching
+        lang = (
+            fmt.get('language') or
+            fmt.get('audio_language') or
+            'en'
+        ).lower()
+
+        lang_match = (
+            lang == audio_lang_lower or
+            lang.startswith(audio_lang_lower[:2]) or
+            audio_lang_lower.startswith(lang[:2])
+        )
+
+        if not lang_match:
+            continue
+
+        # Calculate score
+        score = 0
+
+        # Language match bonus
+        if lang == audio_lang_lower:
+            score += 10000
+        elif lang.startswith(audio_lang_lower[:2]):
+            score += 8000
+        else:
+            score += 5000
+
+        # Quality score
+        quality_diff = abs(height - max_quality)
+        if height >= max_quality:
+            score += 3000 - quality_diff
+        else:
+            score += 2000 - quality_diff
+
+        # Protocol preference
+        if protocol in ('https', 'http'):
+            score += 500
+
+        # Format preference
+        if fmt.get('ext') == 'mp4':
+            score += 100
+
+        candidates.append({
+            'format': fmt,
+            'score': score,
+            'height': height,
+            'lang': lang,
+        })
+
+    if not candidates:
+        logger.debug(f'No unified format found for {max_quality}p + {audio_language}')
+        return None
+
+    # Sort by score and return best
+    candidates.sort(key=lambda x: x['score'], reverse=True)
+    best = candidates[0]
+
+    logger.info(
+        f'Selected unified format: {best["format"].get("format_id")} | '
+        f'{best["lang"]} | {best["height"]}p | score={best["score"]}'
+    )
+
+    return best['format']
+
+
+def clear_cache():
+    """Clear the video info cache."""
+    extract_video_info.cache_clear()
+    logger.info('yt-dlp cache cleared')
+
+
+def get_cache_info() -> Dict[str, Any]:
+    """Get cache statistics."""
+    cache_info = extract_video_info.cache_info()
+    return {
+        'hits': cache_info.hits,
+        'misses': cache_info.misses,
+        'size': cache_info.currsize,
+        'maxsize': cache_info.maxsize,
+        'hit_rate': cache_info.hits / (cache_info.hits + cache_info.misses) if (cache_info.hits + cache_info.misses) > 0 else 0,
+    }