feat: 镜像构建时打包本地修补的 yt-dlp wheel
This commit is contained in:
parent
3702e9fb94
commit
d0ae78e9de
|
|
@ -55,6 +55,9 @@ __pycache__
|
|||
# Downloaded media
|
||||
/downloads
|
||||
|
||||
# Private yt-dlp wheel (built by build-ytdlp.sh and baked into the image)
|
||||
/vendor/yt_dlp-*.whl
|
||||
|
||||
# Runtime data
|
||||
/library
|
||||
/metube-config
|
||||
|
|
|
|||
20
DEPLOY.md
20
DEPLOY.md
|
|
@ -23,9 +23,29 @@ Ubuntu 22.04 x86_64, so images are cross-built with `docker buildx` for
|
|||
buildx builder with amd64 support (OrbStack / Docker Desktop provide one by
|
||||
default, e.g. via QEMU emulation).
|
||||
|
||||
## Private yt-dlp build
|
||||
|
||||
Until upstream yt-dlp fixes the extractor bug, the image bakes a wheel built
|
||||
from a local patched checkout instead of the pinned PyPI release:
|
||||
|
||||
- `./build-ytdlp.sh` builds the wheel from `~/Dev/github/yt-dlp` (override the
|
||||
path with `YTDLP_REPO=...`) into `vendor/yt_dlp-*.whl`.
|
||||
- The Dockerfile force-reinstalls that wheel over the version pinned in
|
||||
`uv.lock` when the wheel is present; otherwise it silently falls back to the
|
||||
pinned release.
|
||||
- The wheel is git-ignored, so it is not committed.
|
||||
- `publish.sh` runs `./build-ytdlp.sh` automatically and aborts if the local
|
||||
checkout is missing.
|
||||
|
||||
See **`YTDLP_PRIVATE_BUILD.md`** for full details, including a step-by-step
|
||||
guide to switching back to the official build once the upstream fix lands.
|
||||
|
||||
## Build & push (manual)
|
||||
|
||||
```sh
|
||||
# Optional: stage the private yt-dlp wheel (see "Private yt-dlp build")
|
||||
./build-ytdlp.sh
|
||||
|
||||
docker buildx build --platform linux/amd64 \
|
||||
--build-arg VERSION=1.22 \
|
||||
-t 192.168.2.212:3000/tigeren/metube:1.22 \
|
||||
|
|
|
|||
11
Dockerfile
11
Dockerfile
|
|
@ -34,6 +34,17 @@ COPY pyproject.toml uv.lock ./
|
|||
RUN --mount=type=cache,target=/root/.cache/uv \
|
||||
UV_PROJECT_ENVIRONMENT=/usr/local uv sync --frozen --no-dev --compile-bytecode --extra headless
|
||||
|
||||
# Private yt-dlp build: bake the locally-patched wheel (built by build-ytdlp.sh
|
||||
# from a local yt-dlp checkout) over the pinned PyPI release until the upstream
|
||||
# fix lands. The wheel is git-ignored; vendor/.gitkeep keeps the dir present so
|
||||
# this COPY always succeeds and the RUN is a no-op when no wheel is staged.
|
||||
COPY vendor ./vendor
|
||||
RUN if [ -n "$(ls /app/vendor/yt_dlp-*.whl 2>/dev/null || true)" ]; then \
|
||||
uv pip install --python /usr/local/bin/python --force-reinstall --no-deps /app/vendor/yt_dlp-*.whl; \
|
||||
else \
|
||||
echo "No private yt-dlp wheel in vendor/ - using pinned yt-dlp from uv.lock"; \
|
||||
fi
|
||||
|
||||
# Use sed to strip carriage-return characters from the entrypoint script (in case building on Windows)
|
||||
COPY docker-entrypoint.sh ./
|
||||
RUN sed -i 's/\r$//g' docker-entrypoint.sh && chmod +x docker-entrypoint.sh
|
||||
|
|
|
|||
|
|
@ -0,0 +1,126 @@
|
|||
# Private yt-dlp build
|
||||
|
||||
MeTube uses [yt-dlp](https://github.com/yt-dlp/yt-dlp) as its core downloader,
|
||||
pinned to the official PyPI release via `pyproject.toml` / `uv.lock`.
|
||||
|
||||
There is currently an extractor bug in the official release that is not yet
|
||||
fixed upstream. Until it is, the Docker image bakes a wheel built from a local,
|
||||
patched yt-dlp checkout (`~/Dev/github/yt-dlp`) instead of the pinned release.
|
||||
|
||||
This document explains how the override works, how to build/update the private
|
||||
wheel, and — most importantly — **how to switch back to the official build once
|
||||
the upstream fix lands**.
|
||||
|
||||
---
|
||||
|
||||
## How the override works
|
||||
|
||||
1. `./build-ytdlp.sh` builds a wheel from a local yt-dlp checkout and stages it
|
||||
as `vendor/yt_dlp-*.whl` (the versioned filename is required by `uv`).
|
||||
2. The `Dockerfile`, right after `uv sync`, force-reinstalls that wheel over the
|
||||
version pinned in `uv.lock`:
|
||||
|
||||
```dockerfile
|
||||
COPY vendor ./vendor
|
||||
RUN if [ -n "$(ls /app/vendor/yt_dlp-*.whl 2>/dev/null || true)" ]; then \
|
||||
uv pip install --python /usr/local/bin/python --force-reinstall --no-deps /app/vendor/yt_dlp-*.whl; \
|
||||
else \
|
||||
echo "No private yt-dlp wheel in vendor/ - using pinned yt-dlp from uv.lock"; \
|
||||
fi
|
||||
```
|
||||
|
||||
3. `publish.sh` runs `./build-ytdlp.sh` before every build, so published images
|
||||
always carry the patched build (and the publish aborts loudly if the local
|
||||
checkout is missing).
|
||||
4. The wheel is **git-ignored** (`/vendor/yt_dlp-*.whl`). `vendor/.gitkeep` and
|
||||
`vendor/.gitignore` are committed so the directory always exists for
|
||||
`COPY vendor ./vendor`.
|
||||
|
||||
## Building / updating the private wheel
|
||||
|
||||
```sh
|
||||
./build-ytdlp.sh # builds from ~/Dev/github/yt-dlp
|
||||
YTDLP_REPO=/path/to/yt-dlp ./build-ytdlp.sh # custom checkout path
|
||||
```
|
||||
|
||||
Rebuild whenever the local checkout changes, then rebuild the image:
|
||||
|
||||
```sh
|
||||
docker compose up -d --build --force-recreate
|
||||
```
|
||||
|
||||
The script fails with a clear error if the configured checkout is missing.
|
||||
|
||||
---
|
||||
|
||||
## Switching back to the official build (when upstream fixes it)
|
||||
|
||||
### 1. Remove the Dockerfile override
|
||||
|
||||
Delete the `COPY vendor` / `RUN uv pip install` block from `Dockerfile`
|
||||
(added in the "Private yt-dlp build" section, right after `uv sync`).
|
||||
|
||||
### 2. Remove the build step from `publish.sh`
|
||||
|
||||
Delete the block:
|
||||
|
||||
```sh
|
||||
# Bake the private yt-dlp build (see build-ytdlp.sh) into the image until the
|
||||
# upstream fix lands. Fails loudly if the local checkout is missing.
|
||||
./build-ytdlp.sh
|
||||
```
|
||||
|
||||
### 3. Update the pinned version
|
||||
|
||||
Bump `yt-dlp` to the fixed release and re-resolve the lockfile:
|
||||
|
||||
```sh
|
||||
uv add "yt-dlp[default,curl-cffi]@latest" # or pin the exact fixed version
|
||||
uv lock
|
||||
```
|
||||
|
||||
Commit the updated `pyproject.toml` and `uv.lock`.
|
||||
|
||||
### 4. Clean up the override artifacts
|
||||
|
||||
```sh
|
||||
rm build-ytdlp.sh # no longer needed
|
||||
rm -f vendor/yt_dlp-*.whl # delete any staged wheel
|
||||
```
|
||||
|
||||
You may also remove the `vendor/` directory (`.gitkeep`, `.gitignore`) and the
|
||||
`/vendor/yt_dlp-*.whl` entry from `.gitignore`.
|
||||
|
||||
### 5. Update the docs
|
||||
|
||||
Remove the "Private yt-dlp build" section from `DEPLOY.md` (and delete this file
|
||||
if you no longer want the guide around).
|
||||
|
||||
### 6. Rebuild and verify
|
||||
|
||||
```sh
|
||||
docker compose up -d --build --force-recreate
|
||||
```
|
||||
|
||||
Confirm the installed version is the official release:
|
||||
|
||||
```sh
|
||||
docker compose exec metube python3 -c "import yt_dlp; print(yt_dlp.version.__version__)"
|
||||
```
|
||||
|
||||
### 7. Publish
|
||||
|
||||
When you next publish, `./publish.sh` will no longer stage a private wheel and
|
||||
the image will use the official yt-dlp release.
|
||||
|
||||
---
|
||||
|
||||
## Verifying which build is running
|
||||
|
||||
The `/version` endpoint reports `METUBE_VERSION`; the yt-dlp version is logged
|
||||
by the app at startup. To check inside a running container:
|
||||
|
||||
```sh
|
||||
docker compose exec metube python3 -c \
|
||||
"import yt_dlp, yt_dlp.extractor.spankbang as s; import inspect; print(yt_dlp.version.__version__, 'patched:', 'spankbang' in inspect.getsource(s).lower())"
|
||||
```
|
||||
|
|
@ -0,0 +1,31 @@
|
|||
#!/usr/bin/env bash
|
||||
# Build the private yt-dlp wheel (locally-patched build) and stage it for the
|
||||
# Docker image.
|
||||
#
|
||||
# MeTube pins the official yt-dlp release via uv.lock. Until the upstream fix
|
||||
# lands, a wheel built from a local yt-dlp checkout is baked into the image
|
||||
# instead (see the Dockerfile). This script produces vendor/yt_dlp.whl.
|
||||
#
|
||||
# Usage:
|
||||
# ./build-ytdlp.sh # uses ~/Dev/github/yt-dlp
|
||||
# YTDLP_REPO=/path/to/yt-dlp ./build-ytdlp.sh
|
||||
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")"
|
||||
|
||||
YTDLP_REPO="${YTDLP_REPO:-$HOME/Dev/github/yt-dlp}"
|
||||
OUT_DIR="$(pwd)/vendor"
|
||||
mkdir -p "$OUT_DIR"
|
||||
|
||||
if [[ ! -d "$YTDLP_REPO" ]]; then
|
||||
echo "Error: local yt-dlp checkout not found at $YTDLP_REPO" >&2
|
||||
echo "Set YTDLP_REPO to your patched checkout, or drop the override (see DEPLOY.md)." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "Building yt-dlp wheel from $YTDLP_REPO ..."
|
||||
(cd "$YTDLP_REPO" && uv build --wheel -o "$OUT_DIR")
|
||||
|
||||
# Keep the original wheel filename (uv requires it to encode the version).
|
||||
echo "Wheel ready:"
|
||||
ls -1 "$OUT_DIR"/yt_dlp-*.whl
|
||||
|
|
@ -52,6 +52,10 @@ fi
|
|||
|
||||
echo "Publishing $IMAGE:$NEW_VERSION (current: $CURRENT_VERSION)"
|
||||
|
||||
# Bake the private yt-dlp build (see build-ytdlp.sh) into the image until the
|
||||
# upstream fix lands. Fails loudly if the local checkout is missing.
|
||||
./build-ytdlp.sh
|
||||
|
||||
# Record the new version as the current online version. BSD sed has no `\b`,
|
||||
# so require a non-digit (or end of line) after the version to avoid matching
|
||||
# e.g. 1.1 inside 1.10.
|
||||
|
|
|
|||
|
|
@ -0,0 +1,2 @@
|
|||
*
|
||||
!.gitkeep
|
||||
Loading…
Reference in New Issue