xorbitlab
/
nextav
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 1 Wiki Activity
nextav/docs/backlog/LIVE_TRANSCODE_REINTRODUCTI...

32 KiB
Raw Permalink Blame History

Live Transcoding Reintroduction — Comprehensive Implementation Proposal

Status: Backlog / Design Proposal Author: Engineering Target Branch: feature/live-transcode-v2 Related Docs:

  • docs/active/media-streaming/TRANSCODING_REMOVAL_DESIGN.md
  • docs/active/media-streaming/TRANSCODING_REMOVAL_SUMMARY.md
  • docs/archive/transcoding-legacy/JELLYFIN_TRANSCODING_ARCHITECTURE.md
  • docs/archive/transcoding-legacy/STASH-ANALYSIS-AND-SOLUTION-PLAN.md
  • docs/archive/transcoding-legacy/03-process-management-transcoding.md
  • docs/archive/transcoding-legacy/ANTI-JITTER-IMPLEMENTATION.md
  • docs/archive/transcoding-legacy/VIDEO-PLAYER-REPLACEMENT-PLAN.md

1. Why Bring Live Transcoding Back?

NextAV currently routes any non‑natively‑supported format (MKV/AVI/WMV/FLV/MOV/TS/etc.) to a Local Player Launcher (VLC/IINA/PotPlayer). This eliminated server CPU usage but came at three real costs:

Loss User Impact
Cross‑device playback (phone, tablet, TV browser) "I can't watch my MKVs on iPad"
Single‑click in‑browser playback for all formats Friction: install app, allow protocol, retry
Adaptive bitrate over slow networks Large files unplayable over LAN/WAN

The goal of v2 is to bring back YouTube/Jellyfin‑class live transcoding that does not repeat the historical pain points (process leaks, jitter, broken seek, duration corruption, runaway CPU).

2. Honest Postmortem of v1 (Why It Was Removed)

From the legacy docs and archive/transcoding-legacy/*, the v1 transcoder failed at six things at once:

2.1 Process Lifecycle

  • FFmpeg child processes orphaned on browser close / route change.
  • No central registry → could not enumerate or kill cleanly.
  • No client heartbeat → processes ran forever when client died silently.
  • kill('SIGKILL') raced with the HTTP stream close, leading to half-shut pipes and EPIPE/ECONNRESET.

2.2 Seeking

  • Old design transcoded from t=0 on every seek. A seek to 1h00m wasted 1h of CPU before producing one byte.
  • The Stash analysis (STASH-ANALYSIS-AND-SOLUTION-PLAN.md) correctly identified that the fix is -ss <seek> -i <file> (input seek) + restart, but it was never properly wired into the player.

2.3 Duration & Progress Jitter

  • The browser's HTMLMediaElement.duration reflects buffered duration when the upstream is a piped MP4 with frag_keyframe+empty_moov. The progress bar therefore reported absurd durations (e.g. "6s" for a 9‑min video) and jumped backwards as new fragments arrived.
  • ANTI-JITTER-IMPLEMENTATION.md patched the symptom in the UI, not the cause (lack of a known duration / HLS playlist with #EXT-X-ENDLIST).

2.4 Format Detection

  • Decision was made per request based on a JSON blob in media.codec_info. That blob was sometimes missing, sometimes wrong (the scanner only ran ffprobe once and never re‑probed), so the same file would sometimes stream direct, sometimes transcode.

2.5 Resource Contention

  • No concurrency cap. Three simultaneous 1080p libx264 -preset veryfast jobs would saturate any consumer CPU and starve thumbnail generation and the Next.js event loop.
  • No throttling: FFmpeg encoded as fast as possible and ran 60+ seconds ahead of the player, wasting CPU if the user closed the tab.

2.6 HLS Half-Implementation

  • src/app/api/stream/hls/... exists today and generates a fake playlist that points at byte‑range slices of the source file (ts-segmentation-service.ts). For real .ts Transport Streams this can work, but for arbitrary inputs (MKV/AVI) the resulting "segments" are not valid TS and players fail with bufferAppendError.
  • The hls-session-manager.ts heartbeat exists but nothing in the player calls it.

3. Design Principles for v2

  1. HLS-first, MP4-fragment second. All live transcoding produces an HLS playlist + MPEG-TS (or fMP4) segments on disk. The browser sees a finite playlist with a real #EXT-X-PLAYLIST-TYPE:VOD and #EXT-X-ENDLIST, so duration is correct from frame 1 and seeking works without restarts inside the encoded window.
  2. Stash-style seek = restart. Seeks outside the already-encoded window kill the FFmpeg job and restart with -ss <t> at the new position. The playlist is regenerated with a virtual timeline that maps to real timestamps via #EXT-X-DISCONTINUITY.
  3. One job per (videoId, profile). A Job is keyed by videoId + quality + audioTrack + subTrack. Re-requests of the same key attach to the existing job; different keys evict the older one for that videoId after a grace period.
  4. Heartbeats are mandatory. The client pings every 20s. Jobs without a ping for 60s (HLS) / 10s (progressive) are killed. This is Jellyfin's model and the only one proven to survive real users.
  5. Server-Sent Events for control. A single SSE channel per session reports ready, segment-available, transcode-error, transcode-progress, replacing polling.
  6. Hard concurrency cap. Default MAX_CONCURRENT_TRANSCODES=2 per node. Additional requests get HTTP 503 with Retry-After, plus a clear UI message.
  7. Direct streaming is always tried first. Live transcoding is a fallback, not the default. H.264/AAC MP4 over range requests stays the happy path.
  8. Disk is the cache. Every produced segment is written to /tmp/nextav-hls/<videoId>/<profile>/seg_%05d.ts. After the job is killed, segments live another 5 minutes so the user can rewind without re-encoding.
  9. Observability is shipped, not patched on. Every job emits structured logs + a /api/transcode/status snapshot suitable for a tiny admin page.
  10. Feature-flagged rollout. Live transcoding is behind ENABLE_LIVE_TRANSCODE=false until validated on a Linux box with both software and hardware encoders.

4. End-to-End Architecture

┌──────────────────────────────────────────────────────────────────────┐
│                              Browser                                  │
│  ArtPlayer + hls.js  ──► /api/transcode/start  (POST, returns job)    │
│        │                                                              │
│        │ loads m3u8                                                   │
│        ▼                                                              │
│  /api/transcode/[jobId]/master.m3u8 ──► VOD playlist (finite)         │
│  /api/transcode/[jobId]/seg/000.ts  ──► segment file (disk)           │
│        │                                                              │
│        │ every 20s                                                    │
│  /api/transcode/[jobId]/ping  (POST)                                  │
│        │                                                              │
│  /api/transcode/[jobId]  (DELETE) on unmount / close                  │
└──────────────────────────────────────────────────────────────────────┘
                                  │
                                  ▼
┌──────────────────────────────────────────────────────────────────────┐
│                          Next.js Server                               │
│                                                                       │
│   TranscodeOrchestrator (singleton, src/lib/transcode/orchestrator)   │
│     ├─ JobRegistry          (Map<jobId, TranscodeJob>)                │
│     ├─ HeartbeatManager     (kill-timers per job)                     │
│     ├─ ConcurrencyGate      (semaphore, MAX_CONCURRENT_TRANSCODES)    │
│     ├─ SegmentJanitor       (LRU eviction, disk-budget aware)         │
│     └─ HwCapabilityProbe    (one-time on boot, picks h264_videotoolbox│
│                              / h264_nvenc / h264_qsv / libx264)       │
│                                                                       │
│   TranscodeJob                                                        │
│     ├─ ffmpeg ChildProcess (spawned, attached to AbortController)     │
│     ├─ outDir (/tmp/nextav-hls/<videoId>/<profile>)                   │
│     ├─ playlistPath, segmentCount, lastSegmentMtime                   │
│     ├─ lastPing, isPaused (throttler input)                           │
│     └─ status: pending | encoding | ready | paused | failed | killed  │
│                                                                       │
│   FormatDecisionEngine (src/lib/transcode/decision.ts)                │
│     direct | hls-remux (no re-encode) | hls-transcode                 │
└──────────────────────────────────────────────────────────────────────┘

5. Data & API Surface

5.1 New / Modified Tables

No schema changes to user data, but a small operational table to survive process restarts:

CREATE TABLE IF NOT EXISTS transcode_jobs (
  job_id        TEXT PRIMARY KEY,
  media_id      INTEGER NOT NULL REFERENCES media(id) ON DELETE CASCADE,
  profile       TEXT NOT NULL,           -- e.g. '720p-h264-aac'
  out_dir       TEXT NOT NULL,
  start_ts      REAL NOT NULL DEFAULT 0, -- the -ss value
  status        TEXT NOT NULL,           -- pending|encoding|ready|paused|failed|killed
  pid           INTEGER,
  created_at    DATETIME DEFAULT CURRENT_TIMESTAMP,
  last_ping_at  DATETIME,
  error         TEXT
);
CREATE INDEX IF NOT EXISTS idx_transcode_jobs_media ON transcode_jobs(media_id);
CREATE INDEX IF NOT EXISTS idx_transcode_jobs_status ON transcode_jobs(status);

On boot, TranscodeOrchestrator.recover() reads status IN ('encoding','paused'), attempts process.kill(pid, 0) to see if the process is alive; if not, mark killed, schedule disk cleanup.

5.2 HTTP API

Method Path Purpose
POST /api/transcode/start Body: { mediaId, profile?, startTime? }. Returns { jobId, playlistUrl, etaSeconds, profile }. Idempotent on (mediaId, profile) within 5s.
GET /api/transcode/:jobId/master.m3u8 The VOD playlist. Regenerated on every request from disk state — always reflects what segments actually exist. Includes #EXT-X-PLAYLIST-TYPE:VOD and #EXT-X-ENDLIST once encoding finishes.
GET /api/transcode/:jobId/seg/:n.ts Streams a segment file. Sets Cache-Control: private, max-age=3600 and strong ETag.
POST /api/transcode/:jobId/ping Body: { position, isPaused }. Resets kill-timer, feeds throttler.
POST /api/transcode/:jobId/seek Body: { position }. If the target is inside the encoded window → 204 (client just seeks). If outside → orchestrator kills and restarts with -ss position, responds with a new jobId and the client reloads.
DELETE /api/transcode/:jobId Kills the job and schedules cleanup.
GET /api/transcode/:jobId/events SSE: ready, segment, progress, error, killed.
GET /api/transcode/status Admin/debug snapshot of JobRegistry.

5.3 Profiles

// src/lib/transcode/profiles.ts
export const PROFILES = {
  '1080p-h264-aac': { vCodec:'h264', vBitrate:'5000k', maxHeight:1080, aCodec:'aac', aBitrate:'192k' },
  '720p-h264-aac':  { vCodec:'h264', vBitrate:'2800k', maxHeight: 720, aCodec:'aac', aBitrate:'160k' },
  '480p-h264-aac':  { vCodec:'h264', vBitrate:'1200k', maxHeight: 480, aCodec:'aac', aBitrate:'128k' },
  'remux-copy':     { vCodec:'copy', aCodec:'copy' }, // for AVC+AAC in MKV — wrap into HLS without re-encoding
} as const;

The client picks an initial profile from the saved user preference; ABR is not in v2 scope (single‑variant playlist only). Quality switching = new job.

6. Format Decision Engine

// src/lib/transcode/decision.ts
type Decision =
  | { kind: 'direct' }                          // browser can play the file as-is
  | { kind: 'hls-remux';   profile: 'remux-copy' }   // codecs OK, container not
  | { kind: 'hls-transcode'; profile: '720p-h264-aac' | ... };

Rules, evaluated in order:

  1. If extension ∈ {mp4,m4v,webm,ogg,ogv} and videoCodec ∈ {h264,vp8,vp9,av1} and audioCodec ∈ {aac,opus,vorbis} → direct.
  2. If videoCodec ∈ {h264} and audioCodec ∈ {aac} but container is anything else (mkv, ts, mov) → hls-remux (-c copy, basically free CPU).
  3. If videoCodec ∈ {hevc,vp9,av1} and the requesting User-Agent indicates support (Safari/Chrome with hardware decode) → direct with a probe header (HTTP 200 + X-Probe: 1); on player error within 5s, client posts /api/transcode/start to fall back.
  4. Otherwise → hls-transcode with the user's preferred profile (default 720p).

The decision is computed fresh on every /api/transcode/start using a server-side ffprobe of the file (cached in media.codec_info with a TTL of 7 days and a codec_info_version integer so we can invalidate the whole cache atomically).

7. FFmpeg Command Templates

7.1 Transcode (the hot path)

ffmpeg \
  -hide_banner -loglevel error -nostats \
  -ss <START_TS>            \  # input seek for fast start
  -copyts                   \  # keep original timestamps for accurate playlist
  -i "<INPUT_PATH>"         \
  -map 0:v:0 -map 0:a:0?    \
  -c:v <VCODEC>             \  # h264_videotoolbox|h264_nvenc|h264_qsv|libx264
  -preset <PRESET>          \  # libx264: veryfast | hw: p4/quality
  -profile:v main -level 4.1 \
  -pix_fmt yuv420p          \
  -vf "scale=-2:'min(<MAX_H>,ih)':flags=lanczos" \
  -b:v <VBITRATE> -maxrate <VBITRATE> -bufsize <2*VBITRATE> \
  -force_key_frames "expr:gte(t,n_forced*4)" \
  -c:a aac -b:a <ABITRATE> -ac 2 \
  -f hls                    \
  -hls_time 4               \  # 4s segments — fast startup, good seek granularity
  -hls_list_size 0          \  # keep all segments in the playlist
  -hls_segment_type mpegts  \
  -hls_flags independent_segments+temp_file \
  -hls_segment_filename "<OUT_DIR>/seg_%05d.ts" \
  -start_number 0           \
  "<OUT_DIR>/index.m3u8"

Key choices and why:

  • -ss before -i (input seek): O(1) on indexed containers; correctly drops everything before the seek point.
  • -copyts + -start_at_zero are NOT used together — we keep original timestamps so when we discontinuity-merge after a seek-restart the player handles it via #EXT-X-DISCONTINUITY.
  • -force_key_frames expr:gte(t,n_forced*4) makes every segment independently decodable, which lets -hls_flags independent_segments work and gives clean seeks.
  • -hls_flags temp_file writes seg_00042.ts.tmp and renames atomically, so the playlist regenerator never sees a half-written segment.
  • We do not use -re — we want to encode as fast as the CPU allows, then throttle via stdin p/u once we are >60s ahead of playback (Jellyfin pattern).

7.2 Remux (free path for h264+aac in mkv/mov)

Same skeleton but -c:v copy -c:a copy -bsf:v h264_mp4toannexb (the bitstream filter is required to convert AVC's length-prefixed NAL units into Annex‑B for MPEG‑TS).

7.3 Hardware Acceleration

Probed once at boot by running ffmpeg -hide_banner -encoders | grep h264_. Selection priority:

  1. macOS dev: h264_videotoolbox (zero CPU, ships with macOS)
  2. Linux + NVIDIA: h264_nvenc
  3. Linux + Intel iGPU: h264_qsv (requires libmfx)
  4. Fallback everywhere: libx264 -preset veryfast

The choice is recorded in /api/transcode/status and surfaced in Settings → "Transcoding Backend: h264_videotoolbox (auto)".

8. Process Lifecycle (the part v1 got wrong)

client click play
   │
   ▼
POST /api/transcode/start
   │  ┌─────────────────────────────────────────────────────────┐
   │  │ ConcurrencyGate.acquire()  ── may 503 with Retry-After   │
   │  └─────────────────────────────────────────────────────────┘
   ▼
orchestrator.startJob(mediaId, profile, startTs)
   │
   ├─ pick existing job? (same key, alive, ping <30s) → return it
   ├─ ffprobe file (cached) → decide remux vs transcode
   ├─ mkdir <out_dir>
   ├─ spawn ffmpeg, attach { stdout: ignore, stderr: pipe-to-ring-buffer, stdio:[pipe,ignore,pipe] }
   ├─ insert into DB (status=encoding, pid)
   ├─ HeartbeatManager.armKillTimer(jobId, 60s)
   ├─ wait for first segment to appear (fs.watch on out_dir)
   │     ── on timeout 15s → kill + fail with TRANSCODE_STARTUP_TIMEOUT
   └─ return { jobId, playlistUrl: /api/transcode/<jobId>/master.m3u8 }

client loads master.m3u8 every ~segment
client posts /ping every 20s              ── resets kill-timer
client posts /seek when user drags        ── may trigger restart
client posts DELETE on close              ── immediate kill

orchestrator monitors:
  - ffmpeg exit code  → status=ready (0) | failed (non-zero, capture stderr ring)
  - stderr ring       → parse "frame=… time=00:01:23.45" for progress events
  - throttler tick (1Hz):
      if (encoder_position - last_ping_position) > 60s and !isPaused
        ffmpeg.stdin.write('p\n')   // pause
      else if was paused and gap < 30s
        ffmpeg.stdin.write('u\n')   // resume

8.1 Kill Is a State Machine, Not a Signal

async function killJob(jobId: string, reason: string) {
  const job = registry.get(jobId);
  if (!job || job.status === 'killed') return;
  job.status = 'killed';
  // 1. Try graceful: write 'q' to stdin (ffmpeg writes trailers, exits 0)
  try { job.proc.stdin.write('q\n'); } catch {}
  // 2. Wait up to 3s
  const exited = await waitForExit(job.proc, 3_000);
  // 3. SIGTERM, then 2s grace
  if (!exited) { job.proc.kill('SIGTERM'); await waitForExit(job.proc, 2_000); }
  // 4. SIGKILL
  if (!job.proc.killed) job.proc.kill('SIGKILL');
  // 5. DB + concurrency release + janitor schedule
  db.prepare('UPDATE transcode_jobs SET status=? WHERE job_id=?').run('killed', jobId);
  gate.release();
  janitor.scheduleCleanup(job.outDir, /* delayMs */ 5 * 60_000);
}

This is the three-layer pattern from archive/transcoding-legacy/03-process-management-transcoding.md (HTTP context → registry → forced kill), ported faithfully.

8.2 Seek Handling

// /api/transcode/[jobId]/seek
if (target >= job.encodedFromTs && target <= job.lastSegmentTs - 8) {
  // Inside the encoded window — the client already has segments. No action.
  return new Response(null, { status: 204 });
}
// Outside → restart. Old job dies; new job starts at target.
const newJob = await orchestrator.restart(job, /* startTs */ target);
return Response.json({ jobId: newJob.id, playlistUrl: newJob.playlistUrl });

9. Playlist Regeneration (eliminates duration jitter)

The legacy implementation streamed the playlist FFmpeg wrote directly. That playlist grew over time, which is why the browser reported a growing duration and the bar jumped.

v2 regenerates the playlist on every GET from the on-disk segment files plus the known total duration:

function buildPlaylist(job: TranscodeJob): string {
  const segs = fs.readdirSync(job.outDir)
    .filter(f => /^seg_\d{5}\.ts$/.test(f))
    .sort();
  const totalDuration = job.knownDuration; // from ffprobe, REAL duration

  const lines = [
    '#EXTM3U',
    '#EXT-X-VERSION:3',
    '#EXT-X-TARGETDURATION:5',
    '#EXT-X-MEDIA-SEQUENCE:0',
    '#EXT-X-PLAYLIST-TYPE:VOD',
  ];

  // Emit all real segments
  for (const seg of segs) lines.push(`#EXTINF:4.000,`, `seg/${seg}`);

  // If FFmpeg has not yet produced the tail, append a "gap" sentinel so the
  // browser knows total duration. We DO NOT emit #EXT-X-ENDLIST until the
  // encoder exits cleanly — that flips the playlist into final VOD mode.
  if (job.status === 'ready') lines.push('#EXT-X-ENDLIST');

  return lines.join('\n');
}

The trick: even before #EXT-X-ENDLIST is emitted, the player has a #EXT-X-PLAYLIST-TYPE:VOD hint and the real ffprobe duration is stored in the segment count expectation. We also send Content-Duration header for ArtPlayer to consume.

10. UI / Player Integration

10.1 Component Flow

unified-video-player.tsx:

detectVideoFormat(video)
   │
   ├─ direct  → ArtPlayer with src=/api/stream/direct/<id>
   ├─ hls-*   → POST /api/transcode/start
   │            ArtPlayer + hls.js with src=<playlistUrl>
   │            on artplayer "seek" event → POST /seek (may swap source)
   │            every 20s timer → POST /ping
   │            on unmount / onClose → DELETE
   └─ local-player → existing LocalPlayerLauncher (kept as final fallback
                     for formats we explicitly refuse, e.g. raw VOB / RealMedia)

10.2 UI States

State UI
requesting Spinner + "Preparing video…" + ETA from etaSeconds
encoding, no segments yet Same spinner; auto-timeout 15s → error UI
encoding, playing Hidden, but a tiny "Transcoding" badge in the player chrome (toggle in settings)
503 Retry-After Toast: "Server busy transcoding 2 other videos. Try again in 30s."
failed Error card with stderr snippet + button "Open in local player" (degrades to existing LocalPlayerLauncher)

10.3 Settings

New Settings → Playback section:

  • Transcoding: Off / Auto / Always
  • Preferred quality: 480p / 720p / 1080p / Source
  • Hardware acceleration: read-only label of detected backend
  • Show transcoding badge in player: toggle

Stored in localStorage and mirrored to user_preferences (table TBD or reuse existing settings store).

11. Concurrency, Throttling, Disk

11.1 Concurrency Gate

class Semaphore {
  constructor(public max: number) {}
  // acquire returns immediately or rejects with { code: 'BUSY', retryAfterSec }
}

MAX_CONCURRENT_TRANSCODES env var, default 2 for dev, configurable in Settings → Admin.

11.2 Throttling (the Jellyfin pattern)

A 1Hz ticker per active job:

encoderPosition = parse from stderr 'time=00:hh:mm:ss.ms'
playheadPosition = job.lastPingPosition
gap = encoderPosition - playheadPosition
if (gap > 60 && !job.paused)   send 'p\n'; job.paused = true
if (gap < 30 && job.paused)    send 'u\n'; job.paused = false

This caps wasted work at ~60s of encode beyond playback.

11.3 Disk Budget (Segment Janitor)

  • Per-job soft cap: 5 minutes * profile_bitrate ≈ 100 MB for 720p.
  • Global soft cap: TRANSCODE_DISK_BUDGET_MB, default 5000 MB.
  • LRU eviction of whole jobs (never partial — a half-deleted job breaks the playlist) when global cap is exceeded.
  • After job kill: keep outDir alive 5 minutes so a quick reopen reuses the cache; then rm -rf.

12. Failure Modes & Recovery

Failure Detection Response
FFmpeg won't start (binary missing) spawn ENOENT Hard 500 + log; surface in Settings "FFmpeg not found at "
First segment never appears 15s fs.watch timeout Kill job, return TRANSCODE_STARTUP_TIMEOUT, client falls back to local-player UI
FFmpeg crashes mid-stream exit event with code ≠ 0 Capture last 50 stderr lines, mark failed, push SSE error, client offers retry or local-player
Disk full ENOSPC in stderr Janitor force-cleanup, then either restart job or fail with explicit message
Client tab closed silently No ping for 60s Auto-kill via HeartbeatManager
Next.js process restart DB has status=encoding rows with dead PIDs recover() on boot marks them killed, cleans disks
Two clients open same video Same (mediaId, profile) key Both attach to one job; ping is per-session, kill only when all sessions are gone (refcount on job)
H.264 in MKV thought to be HEVC Bad codec_info cache decision.ts re-probes if codec_info is older than 7 days or absent

13. Cleanup of v1 Debris (must happen before v2)

The following files are dead code from v1 and must be deleted (not patched), so v2 starts from a clean slate:

  • src/lib/ffmpeg/process-registry.ts (will be replaced by src/lib/transcode/orchestrator.ts)
  • src/lib/hls-session-manager.ts
  • src/lib/ts-segmentation-service.ts
  • src/lib/hls-error-handler.ts
  • src/lib/ts-converter.ts
  • src/app/api/stream/hls/**
  • src/app/api/stream/[id]/transcode/** (will be replaced by /api/transcode/**)
  • src/app/api/videos/[id]/convert-ts/**
  • src/app/api/ffmpeg/status/** (replaced by /api/transcode/status)
  • src/lib/hooks/use-protected-duration.ts (no longer needed once the playlist reports correct duration)

The local-player-launcher UI and /api/external-stream/[id] stay — they remain the user-visible fallback when transcoding is disabled, fails, or is rate-limited.

14. New Files (proposed layout)

src/lib/transcode/
  orchestrator.ts        # JobRegistry, start/kill/recover, singleton
  job.ts                 # TranscodeJob class
  heartbeat.ts           # HeartbeatManager
  concurrency.ts         # Semaphore
  janitor.ts             # disk LRU cleanup
  throttler.ts           # 1Hz stdin p/u
  hw-probe.ts            # one-time encoder probe, cached
  profiles.ts            # PROFILES constant
  decision.ts            # FormatDecisionEngine
  ffmpeg-args.ts         # buildTranscodeArgs / buildRemuxArgs
  playlist.ts            # buildPlaylist (regenerates m3u8 from disk)
  ffprobe.ts             # cached probe, writes media.codec_info

src/app/api/transcode/
  start/route.ts                  # POST
  [jobId]/master.m3u8/route.ts    # GET
  [jobId]/seg/[name]/route.ts     # GET
  [jobId]/ping/route.ts           # POST
  [jobId]/seek/route.ts           # POST
  [jobId]/route.ts                # DELETE
  [jobId]/events/route.ts         # GET (SSE)
  status/route.ts                 # GET admin

src/components/
  unified-video-player.tsx        # MODIFIED: branches on decision
  transcode-status-badge.tsx      # NEW: tiny chrome badge

15. Phased Roadmap

Phase Scope Exit Criteria Risk
0. Cleanup Delete dead v1 code (§13), drop fluent-ffmpeg dep, keep only the ffmpeg binary for thumbnails. pnpm build green; existing direct-stream + local-player flow untouched. Low
1. Spike: orchestrator + remux only decision.ts for hls-remux only (h264+aac in mkv/mov). Full orchestrator, kill state machine, heartbeat, single-segment playlist regen. No transcoding yet — copy only. Play a 2 GB MKV (h264+aac) in Chrome/Safari/Firefox. Seek, pause, close. lsof shows no leaked FDs after 5 minutes. High (the lifecycle code is the hard part)
2. Software transcoding Add libx264 -preset veryfast path. 720p only. Throttler + concurrency gate. Play an AVI (mpeg4+mp3) end-to-end. Two concurrent jobs run, third gets 503. Kill -9 the server: on restart, dead-job rows are cleaned, no orphaned ffmpeg. High
3. Hardware encoder + profiles hw-probe.ts, 480p/720p/1080p, settings UI. macOS dev box uses h264_videotoolbox, CPU < 30% on 1080p. Linux box auto-picks h264_nvenc if available. Medium
4. Seek-restart + discontinuity /seek endpoint, restart-on-large-seek, #EXT-X-DISCONTINUITY in regenerated playlist. Seek to t=1h in a 2h MKV completes within 3s on hw encoder; progress bar shows correct global timeline. Medium
5. SSE + observability /events SSE channel, /status admin page, structured stderr capture. A dev panel at /admin/transcode shows live jobs, CPU%, ETA. Low
6. Polish + flags Feature flag default-on per env; user-facing error UI; docs. E2E run-book passes (see §17). Move this doc to docs/active/media-streaming/. Low

Estimated effort: 6 calendar weeks at one engineer, dominated by Phases 1 and 2.

16. Configuration (env)

ENABLE_LIVE_TRANSCODE=false           # master switch
FFMPEG_PATH=/usr/bin/ffmpeg
FFPROBE_PATH=/usr/bin/ffprobe
TRANSCODE_TMP_DIR=/tmp/nextav-hls
MAX_CONCURRENT_TRANSCODES=2
TRANSCODE_DISK_BUDGET_MB=5000
TRANSCODE_HEARTBEAT_TIMEOUT_MS=60000
TRANSCODE_STARTUP_TIMEOUT_MS=15000
TRANSCODE_HWACCEL=auto                # auto|videotoolbox|nvenc|qsv|none
TRANSCODE_DEFAULT_PROFILE=720p-h264-aac

17. Acceptance / Run-Book

Manual E2E (must all pass on macOS + Linux):

  1. Direct path untouched: play an MP4(H.264+AAC), confirm /api/stream/direct/<id> is used and no transcode rows appear.
  2. Remux happy path: play an MKV (H.264+AAC). Confirm hls-remux, CPU < 5%.
  3. Transcode happy path: play an AVI (MPEG-4 ASP). Confirm software encoder, CPU usage matches nproc/2 budget, video plays smoothly.
  4. HW encoder: on a Mac, confirm h264_videotoolbox is chosen, CPU < 30% on 1080p.
  5. Seek inside window: seek +30s, no new job spawned.
  6. Seek outside window: seek +30 minutes, old job dies within 3s, new job spawns, video resumes within 5s.
  7. Concurrency: open three transcoding videos simultaneously; third receives 503 + UI toast.
  8. Heartbeat death: close tab during transcoding, wait 90s, confirm process is killed (ps aux | grep ffmpeg empty) and segments are cleaned within 5 min.
  9. Crash recovery: kill -9 the Next.js server during transcoding, restart, confirm DB rows cleaned, disk cleaned, no orphan ffmpeg.
  10. Disk pressure: fill disk to 95%, request a transcode → fails with explicit ENOSPC message, not a hang.
  11. Player error fallback: corrupt a segment on disk while playing → SSE error event → UI offers "Open in local player" without page reload.
  12. Duration correctness: For a 9:00 video, the progress bar reports 9:00 from the first frame and never moves backward.

18. Risks & Open Questions

  • Q1: ABR is out of scope for v2. Should we plan a v3 with multi-variant playlists, or is single-variant "good enough" for a personal media library? Recommendation: defer; single-variant with manual quality switch covers 95% of the use case.
  • Q2: Subtitles. MKVs frequently carry SRT/SSA tracks. Initial proposal: extract on demand to WebVTT via ffmpeg -map 0:s:0 -c:s webvtt, expose as /api/transcode/<jobId>/sub/<idx>.vtt. Not in Phase 1–4; track as a follow-up.
  • Q3: Multi-audio tracks. Same story as subtitles — out of scope for v2, but the profile key already supports +audioTrack for future use.
  • R1: Process management is the historical failure point. Phase 1 must be ruthlessly tested with lsof, ps, and crash injection before any encoding work begins. If Phase 1 cannot demonstrate zero leaks under chaos testing, abort the whole effort and keep the local-player approach.
  • R2: Browser HLS behavior varies. Safari handles HLS natively; Chrome/Firefox need hls.js. The chosen MEDIA-SEQUENCE / DISCONTINUITY strategy must be validated on all three. Reserve 1 week of Phase 4 just for cross-browser bug triage.

19. Decision Required Before Coding

  1. Approve the HLS-on-disk approach (vs. the v1 piped-MP4 approach). Strongly recommended; everything else in this doc depends on it.
  2. Approve single-variant playlists for v2 (no ABR).
  3. Confirm the delete-first stance on legacy code (§13). If we keep v1 stubs around, the new code paths will be ambiguous and the old bugs will resurface.
  4. Confirm MAX_CONCURRENT_TRANSCODES=2 as the default for a personal/home server. Adjust upward only when the target deployment is a multi-user server.

Once these four are signed off, work can start with Phase 0 cleanup.