feat: 镜像构建时打包本地修补的 yt-dlp wheel

This commit is contained in:
tigerenwork 2026-08-15 02:03:59 +08:00
parent 3702e9fb94
commit d0ae78e9de
8 changed files with 197 additions and 0 deletions

3
.gitignore vendored
View File

@ -55,6 +55,9 @@ __pycache__
# Downloaded media
/downloads
# Private yt-dlp wheel (built by build-ytdlp.sh and baked into the image)
/vendor/yt_dlp-*.whl
# Runtime data
/library
/metube-config

View File

@ -23,9 +23,29 @@ Ubuntu 22.04 x86_64, so images are cross-built with `docker buildx` for
buildx builder with amd64 support (OrbStack / Docker Desktop provide one by
default, e.g. via QEMU emulation).
## Private yt-dlp build
Until upstream yt-dlp fixes the extractor bug, the image bakes a wheel built
from a local patched checkout instead of the pinned PyPI release:
- `./build-ytdlp.sh` builds the wheel from `~/Dev/github/yt-dlp` (override the
path with `YTDLP_REPO=...`) into `vendor/yt_dlp-*.whl`.
- The Dockerfile force-reinstalls that wheel over the version pinned in
`uv.lock` when the wheel is present; otherwise it silently falls back to the
pinned release.
- The wheel is git-ignored, so it is not committed.
- `publish.sh` runs `./build-ytdlp.sh` automatically and aborts if the local
checkout is missing.
See **`YTDLP_PRIVATE_BUILD.md`** for full details, including a step-by-step
guide to switching back to the official build once the upstream fix lands.
## Build & push (manual)
```sh
# Optional: stage the private yt-dlp wheel (see "Private yt-dlp build")
./build-ytdlp.sh
docker buildx build --platform linux/amd64 \
--build-arg VERSION=1.22 \
-t 192.168.2.212:3000/tigeren/metube:1.22 \

View File

@ -34,6 +34,17 @@ COPY pyproject.toml uv.lock ./
RUN --mount=type=cache,target=/root/.cache/uv \
UV_PROJECT_ENVIRONMENT=/usr/local uv sync --frozen --no-dev --compile-bytecode --extra headless
# Private yt-dlp build: bake the locally-patched wheel (built by build-ytdlp.sh
# from a local yt-dlp checkout) over the pinned PyPI release until the upstream
# fix lands. The wheel is git-ignored; vendor/.gitkeep keeps the dir present so
# this COPY always succeeds and the RUN is a no-op when no wheel is staged.
COPY vendor ./vendor
RUN if [ -n "$(ls /app/vendor/yt_dlp-*.whl 2>/dev/null || true)" ]; then \
uv pip install --python /usr/local/bin/python --force-reinstall --no-deps /app/vendor/yt_dlp-*.whl; \
else \
echo "No private yt-dlp wheel in vendor/ - using pinned yt-dlp from uv.lock"; \
fi
# Use sed to strip carriage-return characters from the entrypoint script (in case building on Windows)
COPY docker-entrypoint.sh ./
RUN sed -i 's/\r$//g' docker-entrypoint.sh && chmod +x docker-entrypoint.sh

126
YTDLP_PRIVATE_BUILD.md Normal file
View File

@ -0,0 +1,126 @@
# Private yt-dlp build
MeTube uses [yt-dlp](https://github.com/yt-dlp/yt-dlp) as its core downloader,
pinned to the official PyPI release via `pyproject.toml` / `uv.lock`.
There is currently an extractor bug in the official release that is not yet
fixed upstream. Until it is, the Docker image bakes a wheel built from a local,
patched yt-dlp checkout (`~/Dev/github/yt-dlp`) instead of the pinned release.
This document explains how the override works, how to build/update the private
wheel, and — most importantly — **how to switch back to the official build once
the upstream fix lands**.
---
## How the override works
1. `./build-ytdlp.sh` builds a wheel from a local yt-dlp checkout and stages it
as `vendor/yt_dlp-*.whl` (the versioned filename is required by `uv`).
2. The `Dockerfile`, right after `uv sync`, force-reinstalls that wheel over the
version pinned in `uv.lock`:
```dockerfile
COPY vendor ./vendor
RUN if [ -n "$(ls /app/vendor/yt_dlp-*.whl 2>/dev/null || true)" ]; then \
uv pip install --python /usr/local/bin/python --force-reinstall --no-deps /app/vendor/yt_dlp-*.whl; \
else \
echo "No private yt-dlp wheel in vendor/ - using pinned yt-dlp from uv.lock"; \
fi
```
3. `publish.sh` runs `./build-ytdlp.sh` before every build, so published images
always carry the patched build (and the publish aborts loudly if the local
checkout is missing).
4. The wheel is **git-ignored** (`/vendor/yt_dlp-*.whl`). `vendor/.gitkeep` and
`vendor/.gitignore` are committed so the directory always exists for
`COPY vendor ./vendor`.
## Building / updating the private wheel
```sh
./build-ytdlp.sh # builds from ~/Dev/github/yt-dlp
YTDLP_REPO=/path/to/yt-dlp ./build-ytdlp.sh # custom checkout path
```
Rebuild whenever the local checkout changes, then rebuild the image:
```sh
docker compose up -d --build --force-recreate
```
The script fails with a clear error if the configured checkout is missing.
---
## Switching back to the official build (when upstream fixes it)
### 1. Remove the Dockerfile override
Delete the `COPY vendor` / `RUN uv pip install` block from `Dockerfile`
(added in the "Private yt-dlp build" section, right after `uv sync`).
### 2. Remove the build step from `publish.sh`
Delete the block:
```sh
# Bake the private yt-dlp build (see build-ytdlp.sh) into the image until the
# upstream fix lands. Fails loudly if the local checkout is missing.
./build-ytdlp.sh
```
### 3. Update the pinned version
Bump `yt-dlp` to the fixed release and re-resolve the lockfile:
```sh
uv add "yt-dlp[default,curl-cffi]@latest" # or pin the exact fixed version
uv lock
```
Commit the updated `pyproject.toml` and `uv.lock`.
### 4. Clean up the override artifacts
```sh
rm build-ytdlp.sh # no longer needed
rm -f vendor/yt_dlp-*.whl # delete any staged wheel
```
You may also remove the `vendor/` directory (`.gitkeep`, `.gitignore`) and the
`/vendor/yt_dlp-*.whl` entry from `.gitignore`.
### 5. Update the docs
Remove the "Private yt-dlp build" section from `DEPLOY.md` (and delete this file
if you no longer want the guide around).
### 6. Rebuild and verify
```sh
docker compose up -d --build --force-recreate
```
Confirm the installed version is the official release:
```sh
docker compose exec metube python3 -c "import yt_dlp; print(yt_dlp.version.__version__)"
```
### 7. Publish
When you next publish, `./publish.sh` will no longer stage a private wheel and
the image will use the official yt-dlp release.
---
## Verifying which build is running
The `/version` endpoint reports `METUBE_VERSION`; the yt-dlp version is logged
by the app at startup. To check inside a running container:
```sh
docker compose exec metube python3 -c \
"import yt_dlp, yt_dlp.extractor.spankbang as s; import inspect; print(yt_dlp.version.__version__, 'patched:', 'spankbang' in inspect.getsource(s).lower())"
```

31
build-ytdlp.sh Executable file
View File

@ -0,0 +1,31 @@
#!/usr/bin/env bash
# Build the private yt-dlp wheel (locally-patched build) and stage it for the
# Docker image.
#
# MeTube pins the official yt-dlp release via uv.lock. Until the upstream fix
# lands, a wheel built from a local yt-dlp checkout is baked into the image
# instead (see the Dockerfile). This script produces vendor/yt_dlp.whl.
#
# Usage:
# ./build-ytdlp.sh # uses ~/Dev/github/yt-dlp
# YTDLP_REPO=/path/to/yt-dlp ./build-ytdlp.sh
set -euo pipefail
cd "$(dirname "$0")"
YTDLP_REPO="${YTDLP_REPO:-$HOME/Dev/github/yt-dlp}"
OUT_DIR="$(pwd)/vendor"
mkdir -p "$OUT_DIR"
if [[ ! -d "$YTDLP_REPO" ]]; then
echo "Error: local yt-dlp checkout not found at $YTDLP_REPO" >&2
echo "Set YTDLP_REPO to your patched checkout, or drop the override (see DEPLOY.md)." >&2
exit 1
fi
echo "Building yt-dlp wheel from $YTDLP_REPO ..."
(cd "$YTDLP_REPO" && uv build --wheel -o "$OUT_DIR")
# Keep the original wheel filename (uv requires it to encode the version).
echo "Wheel ready:"
ls -1 "$OUT_DIR"/yt_dlp-*.whl

View File

@ -52,6 +52,10 @@ fi
echo "Publishing $IMAGE:$NEW_VERSION (current: $CURRENT_VERSION)"
# Bake the private yt-dlp build (see build-ytdlp.sh) into the image until the
# upstream fix lands. Fails loudly if the local checkout is missing.
./build-ytdlp.sh
# Record the new version as the current online version. BSD sed has no `\b`,
# so require a non-digit (or end of line) after the version to avoid matching
# e.g. 1.1 inside 1.10.

2
vendor/.gitignore vendored Normal file
View File

@ -0,0 +1,2 @@
*
!.gitkeep

0
vendor/.gitkeep vendored Normal file
View File