# Live Transcoding Reintroduction — Comprehensive Implementation Proposal

> Status: **Backlog / Design Proposal**
> Author: Engineering
> Target Branch: `feature/live-transcode-v2`
> Related Docs:
> - `docs/active/media-streaming/TRANSCODING_REMOVAL_DESIGN.md`
> - `docs/active/media-streaming/TRANSCODING_REMOVAL_SUMMARY.md`
> - `docs/archive/transcoding-legacy/JELLYFIN_TRANSCODING_ARCHITECTURE.md`
> - `docs/archive/transcoding-legacy/STASH-ANALYSIS-AND-SOLUTION-PLAN.md`
> - `docs/archive/transcoding-legacy/03-process-management-transcoding.md`
> - `docs/archive/transcoding-legacy/ANTI-JITTER-IMPLEMENTATION.md`
> - `docs/archive/transcoding-legacy/VIDEO-PLAYER-REPLACEMENT-PLAN.md`

---

## 1. Why Bring Live Transcoding Back?

NextAV currently routes any non‑natively‑supported format (MKV/AVI/WMV/FLV/MOV/TS/etc.) to a **Local Player Launcher** (VLC/IINA/PotPlayer). This eliminated server CPU usage but came at three real costs:

| Loss | User Impact |
|------|-------------|
| Cross‑device playback (phone, tablet, TV browser) | "I can't watch my MKVs on iPad" |
| Single‑click in‑browser playback for all formats | Friction: install app, allow protocol, retry |
| Adaptive bitrate over slow networks | Large files unplayable over LAN/WAN |

The goal of v2 is to bring back **YouTube/Jellyfin‑class live transcoding** that does **not** repeat the historical pain points (process leaks, jitter, broken seek, duration corruption, runaway CPU).

---

## 2. Honest Postmortem of v1 (Why It Was Removed)

From the legacy docs and `archive/transcoding-legacy/*`, the v1 transcoder failed at six things at once:

### 2.1 Process Lifecycle
- FFmpeg child processes orphaned on browser close / route change.
- No central registry → could not enumerate or kill cleanly.
- No client heartbeat → processes ran forever when client died silently.
- `kill('SIGKILL')` raced with the HTTP stream close, leading to half-shut pipes and `EPIPE`/`ECONNRESET`.

### 2.2 Seeking
- Old design transcoded **from t=0** on every seek. A seek to 1h00m wasted 1h of CPU before producing one byte.
- The Stash analysis (`STASH-ANALYSIS-AND-SOLUTION-PLAN.md`) correctly identified that the fix is `-ss <seek> -i <file>` (input seek) + restart, but it was never properly wired into the player.

### 2.3 Duration & Progress Jitter
- The browser's `HTMLMediaElement.duration` reflects **buffered** duration when the upstream is a piped MP4 with `frag_keyframe+empty_moov`. The progress bar therefore reported absurd durations (e.g. "6s" for a 9‑min video) and jumped backwards as new fragments arrived.
- `ANTI-JITTER-IMPLEMENTATION.md` patched the symptom in the UI, not the cause (lack of a known duration / HLS playlist with `#EXT-X-ENDLIST`).

### 2.4 Format Detection
- Decision was made per request based on a JSON blob in `media.codec_info`. That blob was sometimes missing, sometimes wrong (the scanner only ran ffprobe once and never re‑probed), so the same file would sometimes stream direct, sometimes transcode.

### 2.5 Resource Contention
- No concurrency cap. Three simultaneous 1080p `libx264 -preset veryfast` jobs would saturate any consumer CPU and starve thumbnail generation and the Next.js event loop.
- No throttling: FFmpeg encoded as fast as possible and ran 60+ seconds ahead of the player, wasting CPU if the user closed the tab.

### 2.6 HLS Half-Implementation
- `src/app/api/stream/hls/...` exists today and generates a **fake playlist** that points at byte‑range slices of the source file (`ts-segmentation-service.ts`). For real `.ts` Transport Streams this can work, but for arbitrary inputs (MKV/AVI) the resulting "segments" are not valid TS and players fail with `bufferAppendError`.
- The `hls-session-manager.ts` heartbeat exists but nothing in the player calls it.

---

## 3. Design Principles for v2

1. **HLS-first, MP4-fragment second.** All live transcoding produces an HLS playlist + MPEG-TS (or fMP4) segments on disk. The browser sees a finite playlist with a real `#EXT-X-PLAYLIST-TYPE:VOD` and `#EXT-X-ENDLIST`, so `duration` is correct from frame 1 and seeking works without restarts inside the encoded window.
2. **Stash-style seek = restart.** Seeks outside the already-encoded window kill the FFmpeg job and restart with `-ss <t>` at the new position. The playlist is regenerated with a virtual timeline that maps to real timestamps via `#EXT-X-DISCONTINUITY`.
3. **One job per (videoId, profile).** A Job is keyed by `videoId + quality + audioTrack + subTrack`. Re-requests of the same key attach to the existing job; different keys evict the older one for that videoId after a grace period.
4. **Heartbeats are mandatory.** The client pings every 20s. Jobs without a ping for 60s (HLS) / 10s (progressive) are killed. This is Jellyfin's model and the only one proven to survive real users.
5. **Server-Sent Events for control.** A single SSE channel per session reports `ready`, `segment-available`, `transcode-error`, `transcode-progress`, replacing polling.
6. **Hard concurrency cap.** Default `MAX_CONCURRENT_TRANSCODES=2` per node. Additional requests get HTTP 503 with `Retry-After`, plus a clear UI message.
7. **Direct streaming is always tried first.** Live transcoding is a fallback, not the default. H.264/AAC MP4 over range requests stays the happy path.
8. **Disk is the cache.** Every produced segment is written to `/tmp/nextav-hls/<videoId>/<profile>/seg_%05d.ts`. After the job is killed, segments live another 5 minutes so the user can rewind without re-encoding.
9. **Observability is shipped, not patched on.** Every job emits structured logs + a `/api/transcode/status` snapshot suitable for a tiny admin page.
10. **Feature-flagged rollout.** Live transcoding is behind `ENABLE_LIVE_TRANSCODE=false` until validated on a Linux box with both software and hardware encoders.

---

## 4. End-to-End Architecture

```
┌──────────────────────────────────────────────────────────────────────┐
│                              Browser                                  │
│  ArtPlayer + hls.js  ──► /api/transcode/start  (POST, returns job)    │
│        │                                                              │
│        │ loads m3u8                                                   │
│        ▼                                                              │
│  /api/transcode/[jobId]/master.m3u8 ──► VOD playlist (finite)         │
│  /api/transcode/[jobId]/seg/000.ts  ──► segment file (disk)           │
│        │                                                              │
│        │ every 20s                                                    │
│  /api/transcode/[jobId]/ping  (POST)                                  │
│        │                                                              │
│  /api/transcode/[jobId]  (DELETE) on unmount / close                  │
└──────────────────────────────────────────────────────────────────────┘
                                  │
                                  ▼
┌──────────────────────────────────────────────────────────────────────┐
│                          Next.js Server                               │
│                                                                       │
│   TranscodeOrchestrator (singleton, src/lib/transcode/orchestrator)   │
│     ├─ JobRegistry          (Map<jobId, TranscodeJob>)                │
│     ├─ HeartbeatManager     (kill-timers per job)                     │
│     ├─ ConcurrencyGate      (semaphore, MAX_CONCURRENT_TRANSCODES)    │
│     ├─ SegmentJanitor       (LRU eviction, disk-budget aware)         │
│     └─ HwCapabilityProbe    (one-time on boot, picks h264_videotoolbox│
│                              / h264_nvenc / h264_qsv / libx264)       │
│                                                                       │
│   TranscodeJob                                                        │
│     ├─ ffmpeg ChildProcess (spawned, attached to AbortController)     │
│     ├─ outDir (/tmp/nextav-hls/<videoId>/<profile>)                   │
│     ├─ playlistPath, segmentCount, lastSegmentMtime                   │
│     ├─ lastPing, isPaused (throttler input)                           │
│     └─ status: pending | encoding | ready | paused | failed | killed  │
│                                                                       │
│   FormatDecisionEngine (src/lib/transcode/decision.ts)                │
│     direct | hls-remux (no re-encode) | hls-transcode                 │
└──────────────────────────────────────────────────────────────────────┘
```

---

## 5. Data & API Surface

### 5.1 New / Modified Tables

No schema changes to user data, but a small operational table to survive process restarts:

```sql
CREATE TABLE IF NOT EXISTS transcode_jobs (
  job_id        TEXT PRIMARY KEY,
  media_id      INTEGER NOT NULL REFERENCES media(id) ON DELETE CASCADE,
  profile       TEXT NOT NULL,           -- e.g. '720p-h264-aac'
  out_dir       TEXT NOT NULL,
  start_ts      REAL NOT NULL DEFAULT 0, -- the -ss value
  status        TEXT NOT NULL,           -- pending|encoding|ready|paused|failed|killed
  pid           INTEGER,
  created_at    DATETIME DEFAULT CURRENT_TIMESTAMP,
  last_ping_at  DATETIME,
  error         TEXT
);
CREATE INDEX IF NOT EXISTS idx_transcode_jobs_media ON transcode_jobs(media_id);
CREATE INDEX IF NOT EXISTS idx_transcode_jobs_status ON transcode_jobs(status);
```

On boot, `TranscodeOrchestrator.recover()` reads `status IN ('encoding','paused')`, attempts `process.kill(pid, 0)` to see if the process is alive; if not, mark `killed`, schedule disk cleanup.

### 5.2 HTTP API

| Method | Path | Purpose |
|--------|------|---------|
| `POST` | `/api/transcode/start` | Body: `{ mediaId, profile?, startTime? }`. Returns `{ jobId, playlistUrl, etaSeconds, profile }`. Idempotent on `(mediaId, profile)` within 5s. |
| `GET`  | `/api/transcode/:jobId/master.m3u8` | The VOD playlist. Regenerated on every request from disk state — always reflects what segments actually exist. Includes `#EXT-X-PLAYLIST-TYPE:VOD` and `#EXT-X-ENDLIST` once encoding finishes. |
| `GET`  | `/api/transcode/:jobId/seg/:n.ts` | Streams a segment file. Sets `Cache-Control: private, max-age=3600` and strong ETag. |
| `POST` | `/api/transcode/:jobId/ping` | Body: `{ position, isPaused }`. Resets kill-timer, feeds throttler. |
| `POST` | `/api/transcode/:jobId/seek` | Body: `{ position }`. If the target is **inside** the encoded window → 204 (client just seeks). If **outside** → orchestrator kills and restarts with `-ss position`, responds with a **new** `jobId` and the client reloads. |
| `DELETE` | `/api/transcode/:jobId` | Kills the job and schedules cleanup. |
| `GET` | `/api/transcode/:jobId/events` | SSE: `ready`, `segment`, `progress`, `error`, `killed`. |
| `GET` | `/api/transcode/status` | Admin/debug snapshot of `JobRegistry`. |

### 5.3 Profiles

```ts
// src/lib/transcode/profiles.ts
export const PROFILES = {
  '1080p-h264-aac': { vCodec:'h264', vBitrate:'5000k', maxHeight:1080, aCodec:'aac', aBitrate:'192k' },
  '720p-h264-aac':  { vCodec:'h264', vBitrate:'2800k', maxHeight: 720, aCodec:'aac', aBitrate:'160k' },
  '480p-h264-aac':  { vCodec:'h264', vBitrate:'1200k', maxHeight: 480, aCodec:'aac', aBitrate:'128k' },
  'remux-copy':     { vCodec:'copy', aCodec:'copy' }, // for AVC+AAC in MKV — wrap into HLS without re-encoding
} as const;
```

The client picks an initial profile from the saved user preference; ABR is **not** in v2 scope (single‑variant playlist only). Quality switching = new job.

---

## 6. Format Decision Engine

```ts
// src/lib/transcode/decision.ts
type Decision =
  | { kind: 'direct' }                          // browser can play the file as-is
  | { kind: 'hls-remux';   profile: 'remux-copy' }   // codecs OK, container not
  | { kind: 'hls-transcode'; profile: '720p-h264-aac' | ... };
```

Rules, evaluated in order:

1. If `extension ∈ {mp4,m4v,webm,ogg,ogv}` **and** `videoCodec ∈ {h264,vp8,vp9,av1}` **and** `audioCodec ∈ {aac,opus,vorbis}` → `direct`.
2. If `videoCodec ∈ {h264}` **and** `audioCodec ∈ {aac}` but container is anything else (mkv, ts, mov) → `hls-remux` (`-c copy`, basically free CPU).
3. If `videoCodec ∈ {hevc,vp9,av1}` and the requesting User-Agent indicates support (Safari/Chrome with hardware decode) → `direct` with a probe header (HTTP 200 + `X-Probe: 1`); on player error within 5s, client posts `/api/transcode/start` to fall back.
4. Otherwise → `hls-transcode` with the user's preferred profile (default 720p).

The decision is computed **fresh** on every `/api/transcode/start` using a server-side ffprobe of the file (cached in `media.codec_info` with a TTL of 7 days and a `codec_info_version` integer so we can invalidate the whole cache atomically).

---

## 7. FFmpeg Command Templates

### 7.1 Transcode (the hot path)

```bash
ffmpeg \
  -hide_banner -loglevel error -nostats \
  -ss <START_TS>            \  # input seek for fast start
  -copyts                   \  # keep original timestamps for accurate playlist
  -i "<INPUT_PATH>"         \
  -map 0:v:0 -map 0:a:0?    \
  -c:v <VCODEC>             \  # h264_videotoolbox|h264_nvenc|h264_qsv|libx264
  -preset <PRESET>          \  # libx264: veryfast | hw: p4/quality
  -profile:v main -level 4.1 \
  -pix_fmt yuv420p          \
  -vf "scale=-2:'min(<MAX_H>,ih)':flags=lanczos" \
  -b:v <VBITRATE> -maxrate <VBITRATE> -bufsize <2*VBITRATE> \
  -force_key_frames "expr:gte(t,n_forced*4)" \
  -c:a aac -b:a <ABITRATE> -ac 2 \
  -f hls                    \
  -hls_time 4               \  # 4s segments — fast startup, good seek granularity
  -hls_list_size 0          \  # keep all segments in the playlist
  -hls_segment_type mpegts  \
  -hls_flags independent_segments+temp_file \
  -hls_segment_filename "<OUT_DIR>/seg_%05d.ts" \
  -start_number 0           \
  "<OUT_DIR>/index.m3u8"
```

Key choices and **why**:

- `-ss` **before** `-i` (input seek): O(1) on indexed containers; correctly drops everything before the seek point.
- `-copyts` + `-start_at_zero` are NOT used together — we keep original timestamps so when we discontinuity-merge after a seek-restart the player handles it via `#EXT-X-DISCONTINUITY`.
- `-force_key_frames expr:gte(t,n_forced*4)` makes every segment independently decodable, which lets `-hls_flags independent_segments` work and gives clean seeks.
- `-hls_flags temp_file` writes `seg_00042.ts.tmp` and renames atomically, so the playlist regenerator never sees a half-written segment.
- We do **not** use `-re` — we want to encode as fast as the CPU allows, then **throttle** via stdin `p`/`u` once we are >60s ahead of playback (Jellyfin pattern).

### 7.2 Remux (free path for h264+aac in mkv/mov)

Same skeleton but `-c:v copy -c:a copy -bsf:v h264_mp4toannexb` (the bitstream filter is required to convert AVC's length-prefixed NAL units into Annex‑B for MPEG‑TS).

### 7.3 Hardware Acceleration

Probed once at boot by running `ffmpeg -hide_banner -encoders | grep h264_`. Selection priority:

1. **macOS dev**: `h264_videotoolbox` (zero CPU, ships with macOS)
2. **Linux + NVIDIA**: `h264_nvenc`
3. **Linux + Intel iGPU**: `h264_qsv` (requires `libmfx`)
4. **Fallback everywhere**: `libx264 -preset veryfast`

The choice is recorded in `/api/transcode/status` and surfaced in Settings → "Transcoding Backend: h264_videotoolbox (auto)".

---

## 8. Process Lifecycle (the part v1 got wrong)

```
client click play
   │
   ▼
POST /api/transcode/start
   │  ┌─────────────────────────────────────────────────────────┐
   │  │ ConcurrencyGate.acquire()  ── may 503 with Retry-After   │
   │  └─────────────────────────────────────────────────────────┘
   ▼
orchestrator.startJob(mediaId, profile, startTs)
   │
   ├─ pick existing job? (same key, alive, ping <30s) → return it
   ├─ ffprobe file (cached) → decide remux vs transcode
   ├─ mkdir <out_dir>
   ├─ spawn ffmpeg, attach { stdout: ignore, stderr: pipe-to-ring-buffer, stdio:[pipe,ignore,pipe] }
   ├─ insert into DB (status=encoding, pid)
   ├─ HeartbeatManager.armKillTimer(jobId, 60s)
   ├─ wait for first segment to appear (fs.watch on out_dir)
   │     ── on timeout 15s → kill + fail with TRANSCODE_STARTUP_TIMEOUT
   └─ return { jobId, playlistUrl: /api/transcode/<jobId>/master.m3u8 }

client loads master.m3u8 every ~segment
client posts /ping every 20s              ── resets kill-timer
client posts /seek when user drags        ── may trigger restart
client posts DELETE on close              ── immediate kill

orchestrator monitors:
  - ffmpeg exit code  → status=ready (0) | failed (non-zero, capture stderr ring)
  - stderr ring       → parse "frame=… time=00:01:23.45" for progress events
  - throttler tick (1Hz):
      if (encoder_position - last_ping_position) > 60s and !isPaused
        ffmpeg.stdin.write('p\n')   // pause
      else if was paused and gap < 30s
        ffmpeg.stdin.write('u\n')   // resume
```

### 8.1 Kill Is a State Machine, Not a Signal

```ts
async function killJob(jobId: string, reason: string) {
  const job = registry.get(jobId);
  if (!job || job.status === 'killed') return;
  job.status = 'killed';
  // 1. Try graceful: write 'q' to stdin (ffmpeg writes trailers, exits 0)
  try { job.proc.stdin.write('q\n'); } catch {}
  // 2. Wait up to 3s
  const exited = await waitForExit(job.proc, 3_000);
  // 3. SIGTERM, then 2s grace
  if (!exited) { job.proc.kill('SIGTERM'); await waitForExit(job.proc, 2_000); }
  // 4. SIGKILL
  if (!job.proc.killed) job.proc.kill('SIGKILL');
  // 5. DB + concurrency release + janitor schedule
  db.prepare('UPDATE transcode_jobs SET status=? WHERE job_id=?').run('killed', jobId);
  gate.release();
  janitor.scheduleCleanup(job.outDir, /* delayMs */ 5 * 60_000);
}
```

This is the three-layer pattern from `archive/transcoding-legacy/03-process-management-transcoding.md` (HTTP context → registry → forced kill), ported faithfully.

### 8.2 Seek Handling

```ts
// /api/transcode/[jobId]/seek
if (target >= job.encodedFromTs && target <= job.lastSegmentTs - 8) {
  // Inside the encoded window — the client already has segments. No action.
  return new Response(null, { status: 204 });
}
// Outside → restart. Old job dies; new job starts at target.
const newJob = await orchestrator.restart(job, /* startTs */ target);
return Response.json({ jobId: newJob.id, playlistUrl: newJob.playlistUrl });
```

---

## 9. Playlist Regeneration (eliminates duration jitter)

The legacy implementation streamed the playlist FFmpeg wrote directly. That playlist grew over time, which is why the browser reported a growing duration and the bar jumped.

v2 regenerates the playlist on **every GET** from the on-disk segment files plus the known total duration:

```ts
function buildPlaylist(job: TranscodeJob): string {
  const segs = fs.readdirSync(job.outDir)
    .filter(f => /^seg_\d{5}\.ts$/.test(f))
    .sort();
  const totalDuration = job.knownDuration; // from ffprobe, REAL duration

  const lines = [
    '#EXTM3U',
    '#EXT-X-VERSION:3',
    '#EXT-X-TARGETDURATION:5',
    '#EXT-X-MEDIA-SEQUENCE:0',
    '#EXT-X-PLAYLIST-TYPE:VOD',
  ];

  // Emit all real segments
  for (const seg of segs) lines.push(`#EXTINF:4.000,`, `seg/${seg}`);

  // If FFmpeg has not yet produced the tail, append a "gap" sentinel so the
  // browser knows total duration. We DO NOT emit #EXT-X-ENDLIST until the
  // encoder exits cleanly — that flips the playlist into final VOD mode.
  if (job.status === 'ready') lines.push('#EXT-X-ENDLIST');

  return lines.join('\n');
}
```

The trick: even before `#EXT-X-ENDLIST` is emitted, the player has a `#EXT-X-PLAYLIST-TYPE:VOD` hint and the real ffprobe duration is stored in the segment count expectation. We also send `Content-Duration` header for ArtPlayer to consume.

---

## 10. UI / Player Integration

### 10.1 Component Flow

`unified-video-player.tsx`:

```
detectVideoFormat(video)
   │
   ├─ direct  → ArtPlayer with src=/api/stream/direct/<id>
   ├─ hls-*   → POST /api/transcode/start
   │            ArtPlayer + hls.js with src=<playlistUrl>
   │            on artplayer "seek" event → POST /seek (may swap source)
   │            every 20s timer → POST /ping
   │            on unmount / onClose → DELETE
   └─ local-player → existing LocalPlayerLauncher (kept as final fallback
                     for formats we explicitly refuse, e.g. raw VOB / RealMedia)
```

### 10.2 UI States

| State | UI |
|-------|-----|
| `requesting` | Spinner + "Preparing video…" + ETA from `etaSeconds` |
| `encoding`, no segments yet | Same spinner; auto-timeout 15s → error UI |
| `encoding`, playing | Hidden, but a tiny "Transcoding" badge in the player chrome (toggle in settings) |
| `503 Retry-After` | Toast: "Server busy transcoding 2 other videos. Try again in 30s." |
| `failed` | Error card with stderr snippet + button "Open in local player" (degrades to existing LocalPlayerLauncher) |

### 10.3 Settings

New Settings → Playback section:

- **Transcoding**: Off / Auto / Always
- **Preferred quality**: 480p / 720p / 1080p / Source
- **Hardware acceleration**: read-only label of detected backend
- **Show transcoding badge in player**: toggle

Stored in `localStorage` and mirrored to `user_preferences` (table TBD or reuse existing settings store).

---

## 11. Concurrency, Throttling, Disk

### 11.1 Concurrency Gate

```ts
class Semaphore {
  constructor(public max: number) {}
  // acquire returns immediately or rejects with { code: 'BUSY', retryAfterSec }
}
```

`MAX_CONCURRENT_TRANSCODES` env var, default 2 for dev, configurable in Settings → Admin.

### 11.2 Throttling (the Jellyfin pattern)

A 1Hz ticker per active job:

```
encoderPosition = parse from stderr 'time=00:hh:mm:ss.ms'
playheadPosition = job.lastPingPosition
gap = encoderPosition - playheadPosition
if (gap > 60 && !job.paused)   send 'p\n'; job.paused = true
if (gap < 30 && job.paused)    send 'u\n'; job.paused = false
```

This caps wasted work at ~60s of encode beyond playback.

### 11.3 Disk Budget (Segment Janitor)

- Per-job soft cap: `5 minutes * profile_bitrate` ≈ 100 MB for 720p.
- Global soft cap: `TRANSCODE_DISK_BUDGET_MB`, default 5000 MB.
- LRU eviction of **whole jobs** (never partial — a half-deleted job breaks the playlist) when global cap is exceeded.
- After job kill: keep `outDir` alive 5 minutes so a quick reopen reuses the cache; then `rm -rf`.

---

## 12. Failure Modes & Recovery

| Failure | Detection | Response |
|---------|-----------|----------|
| FFmpeg won't start (binary missing) | spawn `ENOENT` | Hard 500 + log; surface in Settings "FFmpeg not found at <path>" |
| First segment never appears | 15s fs.watch timeout | Kill job, return `TRANSCODE_STARTUP_TIMEOUT`, client falls back to local-player UI |
| FFmpeg crashes mid-stream | `exit` event with code ≠ 0 | Capture last 50 stderr lines, mark `failed`, push SSE error, client offers retry or local-player |
| Disk full | `ENOSPC` in stderr | Janitor force-cleanup, then either restart job or fail with explicit message |
| Client tab closed silently | No ping for 60s | Auto-kill via HeartbeatManager |
| Next.js process restart | DB has `status=encoding` rows with dead PIDs | `recover()` on boot marks them killed, cleans disks |
| Two clients open same video | Same `(mediaId, profile)` key | Both attach to one job; ping is per-session, kill only when **all** sessions are gone (refcount on job) |
| H.264 in MKV thought to be HEVC | Bad codec_info cache | `decision.ts` re-probes if codec_info is older than 7 days or absent |

---

## 13. Cleanup of v1 Debris (must happen before v2)

The following files are dead code from v1 and must be deleted (not patched), so v2 starts from a clean slate:

- `src/lib/ffmpeg/process-registry.ts` (will be replaced by `src/lib/transcode/orchestrator.ts`)
- `src/lib/hls-session-manager.ts`
- `src/lib/ts-segmentation-service.ts`
- `src/lib/hls-error-handler.ts`
- `src/lib/ts-converter.ts`
- `src/app/api/stream/hls/**`
- `src/app/api/stream/[id]/transcode/**` (will be replaced by `/api/transcode/**`)
- `src/app/api/videos/[id]/convert-ts/**`
- `src/app/api/ffmpeg/status/**` (replaced by `/api/transcode/status`)
- `src/lib/hooks/use-protected-duration.ts` (no longer needed once the playlist reports correct duration)

The `local-player-launcher` UI and `/api/external-stream/[id]` stay — they remain the user-visible fallback when transcoding is disabled, fails, or is rate-limited.

---

## 14. New Files (proposed layout)

```
src/lib/transcode/
  orchestrator.ts        # JobRegistry, start/kill/recover, singleton
  job.ts                 # TranscodeJob class
  heartbeat.ts           # HeartbeatManager
  concurrency.ts         # Semaphore
  janitor.ts             # disk LRU cleanup
  throttler.ts           # 1Hz stdin p/u
  hw-probe.ts            # one-time encoder probe, cached
  profiles.ts            # PROFILES constant
  decision.ts            # FormatDecisionEngine
  ffmpeg-args.ts         # buildTranscodeArgs / buildRemuxArgs
  playlist.ts            # buildPlaylist (regenerates m3u8 from disk)
  ffprobe.ts             # cached probe, writes media.codec_info

src/app/api/transcode/
  start/route.ts                  # POST
  [jobId]/master.m3u8/route.ts    # GET
  [jobId]/seg/[name]/route.ts     # GET
  [jobId]/ping/route.ts           # POST
  [jobId]/seek/route.ts           # POST
  [jobId]/route.ts                # DELETE
  [jobId]/events/route.ts         # GET (SSE)
  status/route.ts                 # GET admin

src/components/
  unified-video-player.tsx        # MODIFIED: branches on decision
  transcode-status-badge.tsx      # NEW: tiny chrome badge
```

---

## 15. Phased Roadmap

| Phase | Scope | Exit Criteria | Risk |
|-------|-------|--------------|------|
| **0. Cleanup** | Delete dead v1 code (§13), drop `fluent-ffmpeg` dep, keep only the `ffmpeg` binary for thumbnails. | `pnpm build` green; existing direct-stream + local-player flow untouched. | Low |
| **1. Spike: orchestrator + remux only** | `decision.ts` for `hls-remux` only (h264+aac in mkv/mov). Full orchestrator, kill state machine, heartbeat, single-segment playlist regen. No transcoding yet — copy only. | Play a 2 GB MKV (h264+aac) in Chrome/Safari/Firefox. Seek, pause, close. `lsof` shows no leaked FDs after 5 minutes. | **High** (the lifecycle code is the hard part) |
| **2. Software transcoding** | Add `libx264 -preset veryfast` path. 720p only. Throttler + concurrency gate. | Play an AVI (mpeg4+mp3) end-to-end. Two concurrent jobs run, third gets 503. Kill -9 the server: on restart, dead-job rows are cleaned, no orphaned ffmpeg. | High |
| **3. Hardware encoder + profiles** | `hw-probe.ts`, 480p/720p/1080p, settings UI. | macOS dev box uses `h264_videotoolbox`, CPU < 30% on 1080p. Linux box auto-picks `h264_nvenc` if available. | Medium |
| **4. Seek-restart + discontinuity** | `/seek` endpoint, restart-on-large-seek, `#EXT-X-DISCONTINUITY` in regenerated playlist. | Seek to t=1h in a 2h MKV completes within 3s on hw encoder; progress bar shows correct global timeline. | Medium |
| **5. SSE + observability** | `/events` SSE channel, `/status` admin page, structured stderr capture. | A dev panel at `/admin/transcode` shows live jobs, CPU%, ETA. | Low |
| **6. Polish + flags** | Feature flag default-on per env; user-facing error UI; docs. | E2E run-book passes (see §17). Move this doc to `docs/active/media-streaming/`. | Low |

Estimated effort: **6 calendar weeks** at one engineer, dominated by Phases 1 and 2.

---

## 16. Configuration (env)

```env
ENABLE_LIVE_TRANSCODE=false           # master switch
FFMPEG_PATH=/usr/bin/ffmpeg
FFPROBE_PATH=/usr/bin/ffprobe
TRANSCODE_TMP_DIR=/tmp/nextav-hls
MAX_CONCURRENT_TRANSCODES=2
TRANSCODE_DISK_BUDGET_MB=5000
TRANSCODE_HEARTBEAT_TIMEOUT_MS=60000
TRANSCODE_STARTUP_TIMEOUT_MS=15000
TRANSCODE_HWACCEL=auto                # auto|videotoolbox|nvenc|qsv|none
TRANSCODE_DEFAULT_PROFILE=720p-h264-aac
```

---

## 17. Acceptance / Run-Book

Manual E2E (must all pass on macOS + Linux):

1. **Direct path untouched**: play an MP4(H.264+AAC), confirm `/api/stream/direct/<id>` is used and no transcode rows appear.
2. **Remux happy path**: play an MKV (H.264+AAC). Confirm `hls-remux`, CPU < 5%.
3. **Transcode happy path**: play an AVI (MPEG-4 ASP). Confirm software encoder, CPU usage matches `nproc/2` budget, video plays smoothly.
4. **HW encoder**: on a Mac, confirm `h264_videotoolbox` is chosen, CPU < 30% on 1080p.
5. **Seek inside window**: seek +30s, no new job spawned.
6. **Seek outside window**: seek +30 minutes, old job dies within 3s, new job spawns, video resumes within 5s.
7. **Concurrency**: open three transcoding videos simultaneously; third receives 503 + UI toast.
8. **Heartbeat death**: close tab during transcoding, wait 90s, confirm process is killed (`ps aux | grep ffmpeg` empty) and segments are cleaned within 5 min.
9. **Crash recovery**: `kill -9` the Next.js server during transcoding, restart, confirm DB rows cleaned, disk cleaned, no orphan ffmpeg.
10. **Disk pressure**: fill disk to 95%, request a transcode → fails with explicit `ENOSPC` message, not a hang.
11. **Player error fallback**: corrupt a segment on disk while playing → SSE error event → UI offers "Open in local player" without page reload.
12. **Duration correctness**: For a 9:00 video, the progress bar reports 9:00 from the first frame and never moves backward.

---

## 18. Risks & Open Questions

- **Q1**: ABR is out of scope for v2. Should we plan a v3 with multi-variant playlists, or is single-variant "good enough" for a personal media library? *Recommendation: defer; single-variant with manual quality switch covers 95% of the use case.*
- **Q2**: Subtitles. MKVs frequently carry SRT/SSA tracks. Initial proposal: extract on demand to WebVTT via `ffmpeg -map 0:s:0 -c:s webvtt`, expose as `/api/transcode/<jobId>/sub/<idx>.vtt`. Not in Phase 1–4; track as a follow-up.
- **Q3**: Multi-audio tracks. Same story as subtitles — out of scope for v2, but the profile key already supports `+audioTrack` for future use.
- **R1**: Process management is **the** historical failure point. Phase 1 must be ruthlessly tested with `lsof`, `ps`, and crash injection before any encoding work begins. If Phase 1 cannot demonstrate zero leaks under chaos testing, **abort the whole effort** and keep the local-player approach.
- **R2**: Browser HLS behavior varies. Safari handles HLS natively; Chrome/Firefox need hls.js. The chosen `MEDIA-SEQUENCE` / `DISCONTINUITY` strategy must be validated on all three. Reserve 1 week of Phase 4 just for cross-browser bug triage.

---

## 19. Decision Required Before Coding

1. Approve the **HLS-on-disk** approach (vs. the v1 piped-MP4 approach). *Strongly recommended; everything else in this doc depends on it.*
2. Approve **single-variant playlists** for v2 (no ABR).
3. Confirm the **delete-first** stance on legacy code (§13). If we keep v1 stubs around, the new code paths will be ambiguous and the old bugs will resurface.
4. Confirm `MAX_CONCURRENT_TRANSCODES=2` as the default for a personal/home server. Adjust upward only when the target deployment is a multi-user server.

Once these four are signed off, work can start with Phase 0 cleanup.