1510 lines
48 KiB
Markdown
1510 lines
48 KiB
Markdown
# Jellyfin Transcoding Architecture Documentation
|
|
|
|
## Executive Summary
|
|
|
|
Jellyfin's transcoding system is a sophisticated media processing pipeline built around **FFmpeg** that provides real-time video and audio conversion for various client devices. This document details the core architecture, control flow, and implementation patterns valuable for building a similar system in Next.js.
|
|
|
|
**Key Findings:**
|
|
- **Engine**: FFmpeg managed through MediaEncoder class with hardware acceleration support
|
|
- **Job Management**: TranscodeManager orchestrates process lifecycle, throttling, and cleanup
|
|
- **Session Control**: Client ping system (10s/60s timeouts) with automatic kill timers
|
|
- **Resource Management**: Intelligent throttling based on playback position (+60s threshold)
|
|
- **Segment Management**: Automatic cleanup for HLS content >5 minutes duration
|
|
- **Seeking Strategy**: FFmpeg process restart required for large seeks due to linear encoding nature
|
|
|
|
## Key Implementation Insights
|
|
|
|
### 1. **FFmpeg Process Management Strategy**
|
|
|
|
**Core Principle**: Each transcoding job is a **linear, stateful FFmpeg process** that cannot seek to arbitrary positions after encoding starts. This fundamental limitation drives the entire architecture design.
|
|
|
|
**Implications**:
|
|
- **Large seeks require process restart**: When users seek >30 seconds, kill current job and start new one
|
|
- **Small seeks use player buffering**: Let HLS players handle <30 second seeks naturally
|
|
- **Process isolation**: Each session gets independent FFmpeg process for resource control
|
|
|
|
### 2. **Resource Management Philosophy**
|
|
|
|
**Throttling Strategy**: Prevent transcoding from running too far ahead of playback
|
|
- **Threshold**: Pause encoding when >60 seconds ahead of client consumption
|
|
- **Control Method**: Send FFmpeg stdin commands ('p' for pause, 'u' for resume)
|
|
- **Benefits**: Reduces CPU/disk usage, prevents unnecessary work
|
|
|
|
**Segment Cleanup Policy**: Automatic disk space management for HLS
|
|
- **Trigger Conditions**: Video content with duration >5 minutes
|
|
- **Retention**: Configurable number of segments to keep
|
|
- **Safety**: Retry logic with graceful error handling
|
|
|
|
### 3. **Session Lifecycle Management**
|
|
|
|
**Ping System Design**: Keep-alive mechanism prevents orphaned processes
|
|
- **Progressive streams**: 10-second timeout (fast response needed)
|
|
- **HLS streams**: 60-second timeout (more buffering tolerance)
|
|
- **Client responsibility**: Ping every 30-45 seconds during active playback
|
|
|
|
**Kill Timer Implementation**: Automatic cleanup for abandoned sessions
|
|
- **Grace periods**: Multiple timeout intervals before termination
|
|
- **Resource cleanup**: Process termination + file cleanup + stream closure
|
|
|
|
### 4. **FFmpeg Command Architecture**
|
|
|
|
**Standard HLS Command Structure** (per memory specifications):
|
|
```bash
|
|
ffmpeg [input_modifiers] -i input.mp4 [encoding_params] -f hls -hls_time 6 -hls_segment_filename "segment%d.ts" output.m3u8
|
|
```
|
|
|
|
**Essential Parameters**:
|
|
- **`-f hls`**: HLS output format
|
|
- **`-hls_time 6`**: 6-second segments for optimal seek performance
|
|
- **`-hls_segment_filename`**: Consistent naming pattern
|
|
- **`output.m3u8`**: Playlist file output
|
|
|
|
## Critical Design Decisions
|
|
|
|
### 1. **When to Restart vs Continue Transcoding**
|
|
|
|
**Process Restart Required**:
|
|
- Large seek operations (>30 seconds)
|
|
- Quality/resolution changes
|
|
- Audio track switching
|
|
- Subtitle track changes
|
|
|
|
**Continue Existing Process**:
|
|
- Small seeks (<30 seconds) - let player handle
|
|
- Pause/resume operations
|
|
- Client reconnections within timeout window
|
|
|
|
### 2. **Segment Duration Strategy**
|
|
|
|
**6-Second Standard**: Optimal balance between:
|
|
- **Seek performance**: Reasonable granularity for user seeking
|
|
- **Network efficiency**: Not too many small requests
|
|
- **Startup time**: Quick initial buffering
|
|
- **Storage overhead**: Manageable file count
|
|
|
|
### 3. **Hardware Acceleration Integration**
|
|
|
|
**MediaEncoder Responsibilities**:
|
|
- **Capability detection**: Probe available hardware encoders
|
|
- **Process execution**: Manage FFmpeg with hardware flags
|
|
- **Fallback handling**: Graceful degradation to software encoding
|
|
- **Performance monitoring**: Track encoding speeds and success rates
|
|
|
|
## Architecture Patterns for Next.js Implementation
|
|
|
|
### 1. **Core Classes Structure**
|
|
|
|
```typescript
|
|
// Main orchestrator (equivalent to TranscodeManager)
|
|
class TranscodingOrchestrator {
|
|
private activeJobs = new Map<string, TranscodingJob>();
|
|
private killTimers = new Map<string, NodeJS.Timeout>();
|
|
|
|
async startTranscoding(request: TranscodingRequest): Promise<TranscodingJob>
|
|
pingSession(sessionId: string, isUserPaused?: boolean): void
|
|
async killSession(sessionId: string): Promise<void>
|
|
}
|
|
|
|
// FFmpeg interface (equivalent to MediaEncoder)
|
|
class MediaProcessor {
|
|
private ffmpegPath: string;
|
|
private hardwareCapabilities: HardwareCapabilities;
|
|
|
|
async probe(inputPath: string): Promise<MediaInfo>
|
|
async startProcess(args: string[]): Promise<ChildProcess>
|
|
detectHardwareSupport(): HardwareCapabilities
|
|
}
|
|
|
|
// Individual job tracking (equivalent to TranscodingJob)
|
|
class TranscodingSession {
|
|
id: string;
|
|
process: ChildProcess;
|
|
lastPingDate: Date;
|
|
throttler?: TranscodingThrottler;
|
|
segmentCleaner?: SegmentCleaner;
|
|
|
|
startKillTimer(callback: () => void): void
|
|
stopKillTimer(): void
|
|
pause(): Promise<void>
|
|
resume(): Promise<void>
|
|
}
|
|
```
|
|
|
|
### 2. **API Endpoint Design**
|
|
|
|
```typescript
|
|
// Next.js API routes structure
|
|
|
|
// POST /api/transcoding/start
|
|
export async function POST(request: Request) {
|
|
const { itemId, startTimeTicks, quality } = await request.json();
|
|
const job = await orchestrator.startTranscoding({
|
|
itemId,
|
|
startTimeTicks,
|
|
quality,
|
|
sessionId: generateSessionId()
|
|
});
|
|
return Response.json({ sessionId: job.id, playlistUrl: job.playlistPath });
|
|
}
|
|
|
|
// POST /api/transcoding/[sessionId]/ping
|
|
export async function POST(request: Request, { params }: { params: { sessionId: string } }) {
|
|
const { isUserPaused } = await request.json();
|
|
orchestrator.pingSession(params.sessionId, isUserPaused);
|
|
return Response.json({ success: true });
|
|
}
|
|
|
|
// GET /api/hls/[sessionId]/[segmentId].ts
|
|
export async function GET(request: Request, { params }: { params: { sessionId: string, segmentId: string } }) {
|
|
const segmentPath = getSegmentPath(params.sessionId, params.segmentId);
|
|
const stream = fs.createReadStream(segmentPath);
|
|
return new Response(stream as any);
|
|
}
|
|
|
|
// DELETE /api/transcoding/[sessionId]
|
|
export async function DELETE(request: Request, { params }: { params: { sessionId: string } }) {
|
|
await orchestrator.killSession(params.sessionId);
|
|
return Response.json({ success: true });
|
|
}
|
|
```
|
|
|
|
### 3. **Client Integration Patterns**
|
|
|
|
```typescript
|
|
class AdaptiveMediaPlayer {
|
|
private sessionId: string;
|
|
private pingInterval: NodeJS.Timeout;
|
|
private lastPosition: number = 0;
|
|
|
|
async startPlayback(itemId: string, startPosition: number = 0) {
|
|
// Start transcoding session
|
|
const response = await fetch('/api/transcoding/start', {
|
|
method: 'POST',
|
|
body: JSON.stringify({ itemId, startTimeTicks: startPosition * 10000000 })
|
|
});
|
|
const { sessionId, playlistUrl } = await response.json();
|
|
|
|
this.sessionId = sessionId;
|
|
this.startPinging();
|
|
|
|
// Load HLS playlist
|
|
this.hls.loadSource(playlistUrl);
|
|
}
|
|
|
|
private startPinging() {
|
|
this.pingInterval = setInterval(() => {
|
|
fetch(`/api/transcoding/${this.sessionId}/ping`, {
|
|
method: 'POST',
|
|
body: JSON.stringify({ isUserPaused: this.videoElement.paused })
|
|
});
|
|
}, 30000); // Ping every 30 seconds
|
|
}
|
|
|
|
async seekTo(targetPosition: number) {
|
|
const seekDistance = Math.abs(targetPosition - this.lastPosition);
|
|
|
|
if (seekDistance > 30) {
|
|
// Large seek: restart transcoding
|
|
await this.stopPlayback();
|
|
await this.startPlayback(this.itemId, targetPosition);
|
|
} else {
|
|
// Small seek: let HLS player handle
|
|
this.videoElement.currentTime = targetPosition;
|
|
}
|
|
|
|
this.lastPosition = targetPosition;
|
|
}
|
|
|
|
async stopPlayback() {
|
|
if (this.pingInterval) {
|
|
clearInterval(this.pingInterval);
|
|
}
|
|
|
|
if (this.sessionId) {
|
|
await fetch(`/api/transcoding/${this.sessionId}`, { method: 'DELETE' });
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
### 4. **Configuration Management**
|
|
|
|
```typescript
|
|
// Environment configuration
|
|
interface TranscodingConfig {
|
|
ffmpegPath: string;
|
|
transcodingTempPath: string;
|
|
maxConcurrentStreams: number;
|
|
segmentDuration: number; // 6 seconds recommended
|
|
segmentKeepCount: number;
|
|
throttleThresholdSeconds: number; // 60 seconds recommended
|
|
pingTimeoutProgressive: number; // 10 seconds
|
|
pingTimeoutHls: number; // 60 seconds
|
|
hardwareAcceleration: 'auto' | 'nvidia' | 'intel' | 'amd' | 'none';
|
|
}
|
|
|
|
// Load from environment
|
|
const config: TranscodingConfig = {
|
|
ffmpegPath: process.env.FFMPEG_PATH || '/usr/bin/ffmpeg',
|
|
transcodingTempPath: process.env.TRANSCODING_TEMP_PATH || '/tmp/transcoding',
|
|
maxConcurrentStreams: parseInt(process.env.MAX_CONCURRENT_STREAMS || '3'),
|
|
segmentDuration: 6,
|
|
segmentKeepCount: parseInt(process.env.SEGMENT_KEEP_COUNT || '10'),
|
|
throttleThresholdSeconds: 60,
|
|
pingTimeoutProgressive: 10000,
|
|
pingTimeoutHls: 60000,
|
|
hardwareAcceleration: (process.env.HARDWARE_ACCEL as any) || 'auto'
|
|
};
|
|
```
|
|
|
|
## Production Deployment Considerations
|
|
|
|
### 1. **Performance Monitoring**
|
|
|
|
```typescript
|
|
interface TranscodingMetrics {
|
|
activeJobs: number;
|
|
averageStartupTime: number;
|
|
successRate: number;
|
|
cpuUsage: number;
|
|
memoryUsage: number;
|
|
diskIORate: number;
|
|
clientTimeouts: number;
|
|
}
|
|
|
|
class MetricsCollector {
|
|
collectMetrics(): TranscodingMetrics {
|
|
return {
|
|
activeJobs: this.orchestrator.getActiveJobCount(),
|
|
averageStartupTime: this.calculateAverageStartup(),
|
|
successRate: this.calculateSuccessRate(),
|
|
cpuUsage: process.cpuUsage().user / 1000000, // Convert to seconds
|
|
memoryUsage: process.memoryUsage().heapUsed / 1024 / 1024, // MB
|
|
diskIORate: this.calculateDiskIO(),
|
|
clientTimeouts: this.timeoutCounter
|
|
};
|
|
}
|
|
}
|
|
```
|
|
|
|
### 2. **Error Handling Strategy**
|
|
|
|
```typescript
|
|
class TranscodingErrorHandler {
|
|
async handleFFmpegFailure(job: TranscodingSession, error: Error) {
|
|
// Log error with context
|
|
logger.error('FFmpeg process failed', {
|
|
jobId: job.id,
|
|
inputPath: job.inputPath,
|
|
error: error.message,
|
|
exitCode: job.process.exitCode
|
|
});
|
|
|
|
// Attempt recovery strategies
|
|
if (this.isRetryableError(error)) {
|
|
return this.retryWithFallback(job);
|
|
}
|
|
|
|
// Clean up resources
|
|
await this.cleanupFailedJob(job);
|
|
|
|
// Notify client
|
|
this.notifyClientError(job.sessionId, error);
|
|
}
|
|
|
|
private async retryWithFallback(job: TranscodingSession): Promise<boolean> {
|
|
// Try software encoding if hardware failed
|
|
if (job.hardwareAcceleration && this.isHardwareError(job.lastError)) {
|
|
logger.info('Retrying with software encoding', { jobId: job.id });
|
|
return this.restartWithSoftwareEncoding(job);
|
|
}
|
|
|
|
// Try lower quality settings
|
|
if (job.retryCount < 2) {
|
|
logger.info('Retrying with reduced quality', { jobId: job.id });
|
|
return this.restartWithReducedQuality(job);
|
|
}
|
|
|
|
return false;
|
|
}
|
|
}
|
|
```
|
|
|
|
### 3. **Security Considerations**
|
|
|
|
```typescript
|
|
class SecurityValidator {
|
|
validateTranscodingRequest(request: TranscodingRequest): ValidationResult {
|
|
// Path traversal protection
|
|
if (this.containsPathTraversal(request.inputPath)) {
|
|
return { valid: false, error: 'Invalid input path' };
|
|
}
|
|
|
|
// File type validation
|
|
if (!this.isAllowedMediaType(request.inputPath)) {
|
|
return { valid: false, error: 'Unsupported media type' };
|
|
}
|
|
|
|
// Resource limits
|
|
if (this.exceedsResourceLimits(request)) {
|
|
return { valid: false, error: 'Resource limits exceeded' };
|
|
}
|
|
|
|
// Rate limiting per client
|
|
if (this.isRateLimited(request.clientId)) {
|
|
return { valid: false, error: 'Rate limit exceeded' };
|
|
}
|
|
|
|
return { valid: true };
|
|
}
|
|
|
|
private containsPathTraversal(path: string): boolean {
|
|
return path.includes('..') || path.includes('~') || path.startsWith('/');
|
|
}
|
|
|
|
private isAllowedMediaType(path: string): boolean {
|
|
const allowedExtensions = ['.mp4', '.mkv', '.avi', '.mov', '.m4v'];
|
|
return allowedExtensions.some(ext => path.toLowerCase().endsWith(ext));
|
|
}
|
|
}
|
|
```
|
|
|
|
### 4. **Scaling Strategies**
|
|
|
|
```typescript
|
|
// Horizontal scaling with Redis coordination
|
|
class DistributedTranscodingOrchestrator {
|
|
private redis: Redis;
|
|
private nodeId: string;
|
|
|
|
async startTranscoding(request: TranscodingRequest): Promise<TranscodingJob> {
|
|
// Check global capacity
|
|
const globalLoad = await this.getGlobalLoad();
|
|
if (globalLoad > 0.8) {
|
|
throw new Error('System at capacity');
|
|
}
|
|
|
|
// Assign to least loaded node
|
|
const targetNode = await this.findLeastLoadedNode();
|
|
if (targetNode === this.nodeId) {
|
|
return this.startLocalTranscoding(request);
|
|
} else {
|
|
return this.delegateToNode(targetNode, request);
|
|
}
|
|
}
|
|
|
|
private async getGlobalLoad(): Promise<number> {
|
|
const nodes = await this.redis.smembers('transcoding:nodes');
|
|
let totalJobs = 0;
|
|
let totalCapacity = 0;
|
|
|
|
for (const node of nodes) {
|
|
const nodeLoad = await this.redis.hgetall(`transcoding:node:${node}`);
|
|
totalJobs += parseInt(nodeLoad.activeJobs || '0');
|
|
totalCapacity += parseInt(nodeLoad.maxCapacity || '0');
|
|
}
|
|
|
|
return totalCapacity > 0 ? totalJobs / totalCapacity : 1;
|
|
}
|
|
}
|
|
```
|
|
|
|
## Summary of Key Learnings
|
|
|
|
### **Architecture Principles**
|
|
1. **Process Isolation**: Each transcoding session gets independent FFmpeg process
|
|
2. **Linear Encoding**: FFmpeg cannot seek backwards, requiring process restart for large seeks
|
|
3. **Resource Management**: Proactive throttling and cleanup prevent resource exhaustion
|
|
4. **Client Lifecycle**: Ping-based session management with automatic cleanup
|
|
|
|
### **Performance Optimizations**
|
|
1. **Segment Strategy**: 6-second HLS segments balance seek performance with efficiency
|
|
2. **Throttling Logic**: Pause encoding when >60 seconds ahead of playback
|
|
3. **Cleanup Policies**: Automatic segment removal for content >5 minutes
|
|
4. **Hardware Acceleration**: Graceful fallback from hardware to software encoding
|
|
|
|
### **Implementation Strategy**
|
|
1. **API Design**: RESTful endpoints for session management and HLS segment delivery
|
|
2. **Client Integration**: Adaptive seeking strategy based on seek distance
|
|
3. **Error Handling**: Comprehensive retry logic with fallback options
|
|
4. **Security**: Input validation, path traversal protection, rate limiting
|
|
|
|
### **Deployment Considerations**
|
|
1. **Monitoring**: Track active jobs, success rates, resource usage
|
|
2. **Scaling**: Horizontal scaling with Redis coordination
|
|
3. **Storage**: Temporary file management and cleanup
|
|
4. **Network**: CDN integration for segment delivery
|
|
|
|
This architecture provides a robust foundation for building production-grade media transcoding systems with proper resource management, client lifecycle handling, and performance optimization.
|
|
|
|
## Core Architecture
|
|
|
|
### 1. Main Components
|
|
|
|
- **Role**: Central orchestrator for all transcoding operations
|
|
- **Key Responsibilities**:
|
|
- FFmpeg process lifecycle management
|
|
- Job tracking and session management
|
|
- Resource cleanup and monitoring
|
|
- Client ping/keepalive handling
|
|
|
|
#### MediaEncoder (`MediaEncoder.cs`)
|
|
- **Role**: FFmpeg binary interface and process execution
|
|
- **Key Responsibilities**:
|
|
- FFmpeg path validation and capability detection
|
|
- Process creation and monitoring
|
|
- Hardware acceleration detection
|
|
- Encoder/decoder capability enumeration
|
|
|
|
#### TranscodingJob (`TranscodingJob.cs`)
|
|
- **Role**: Individual transcoding session state management
|
|
- **Key Responsibilities**:
|
|
- Process lifecycle tracking
|
|
- Resource usage monitoring (bytes, position, bitrate)
|
|
- Timer management for auto-cleanup
|
|
- Client connection state
|
|
|
|
### 2. Control Flow
|
|
|
|
```mermaid
|
|
graph TB
|
|
Client[Client Request] --> API[API Controller]
|
|
API --> StreamState[Create StreamState]
|
|
StreamState --> TranscodeManager[TranscodeManager.StartFfMpeg]
|
|
TranscodeManager --> FFmpeg[Launch FFmpeg Process]
|
|
FFmpeg --> Job[Create TranscodingJob]
|
|
Job --> Monitor[Job Monitoring]
|
|
Monitor --> Throttle[Throttling System]
|
|
Monitor --> Cleanup[Segment Cleanup]
|
|
Monitor --> Ping[Ping System]
|
|
Ping --> Timeout[Kill Timer]
|
|
```
|
|
|
|
#### Detailed Flow:
|
|
|
|
1. **Request Reception**: API controllers (`DynamicHlsController`, `VideosController`, `AudioController`) receive transcoding requests
|
|
2. **State Creation**: `StreamingHelpers.GetStreamingState()` analyzes media and creates `StreamState`
|
|
3. **Job Initialization**: `TranscodeManager.StartFfMpeg()` creates and starts FFmpeg process
|
|
4. **Process Monitoring**: `TranscodingJob` tracks process state and resource usage
|
|
5. **Client Management**: Ping system keeps sessions alive, kill timers handle disconnections
|
|
6. **Resource Management**: Throttling and segment cleaning optimize performance
|
|
7. **Cleanup**: Automatic cleanup when sessions end or timeout
|
|
|
|
## Key Systems
|
|
|
|
### 1. Ping System - Keep Transcoding Alive
|
|
|
|
**Purpose**: Prevents transcoding jobs from being killed when clients are actively consuming content.
|
|
|
|
**Implementation**:
|
|
```csharp
|
|
// PingTranscodingJob method in TranscodeManager
|
|
public void PingTranscodingJob(string playSessionId, bool? isUserPaused)
|
|
{
|
|
var jobs = _activeTranscodingJobs.Where(j =>
|
|
string.Equals(playSessionId, j.PlaySessionId, StringComparison.OrdinalIgnoreCase))
|
|
.ToList();
|
|
|
|
foreach (var job in jobs)
|
|
{
|
|
if (isUserPaused.HasValue)
|
|
{
|
|
job.IsUserPaused = isUserPaused.Value;
|
|
}
|
|
PingTimer(job, true);
|
|
}
|
|
}
|
|
|
|
private void PingTimer(TranscodingJob job, bool isProgressCheckIn)
|
|
{
|
|
if (job.HasExited)
|
|
{
|
|
job.StopKillTimer();
|
|
return;
|
|
}
|
|
|
|
// Different timeouts for different job types
|
|
var timerDuration = job.Type != TranscodingJobType.Progressive ? 60000 : 10000;
|
|
|
|
job.PingTimeout = timerDuration;
|
|
job.LastPingDate = DateTime.UtcNow;
|
|
|
|
job.StartKillTimer(OnTranscodeKillTimerStopped);
|
|
}
|
|
```
|
|
|
|
**Key Parameters**:
|
|
- **Progressive streams**: 10 second timeout (10000ms)
|
|
- **HLS streams**: 60 second timeout (60000ms)
|
|
- **Ping frequency**: Clients should ping every 30-45 seconds
|
|
- **Auto-ping**: Triggered on playback progress events
|
|
|
|
### 2. Kill Timers - Automatic Cleanup
|
|
|
|
**Purpose**: Automatically terminate abandoned transcoding jobs to free resources.
|
|
|
|
**Implementation**:
|
|
```csharp
|
|
private async void OnTranscodeKillTimerStopped(object? state)
|
|
{
|
|
var job = state as TranscodingJob;
|
|
if (!job.HasExited && job.Type != TranscodingJobType.Progressive)
|
|
{
|
|
var timeSinceLastPing = (DateTime.UtcNow - job.LastPingDate).TotalMilliseconds;
|
|
|
|
if (timeSinceLastPing < job.PingTimeout)
|
|
{
|
|
// Reset timer if ping is still fresh
|
|
job.StartKillTimer(OnTranscodeKillTimerStopped, job.PingTimeout);
|
|
return;
|
|
}
|
|
}
|
|
|
|
// Kill the job
|
|
await KillTranscodingJob(job, true, path => true).ConfigureAwait(false);
|
|
}
|
|
```
|
|
|
|
**Configuration**:
|
|
- **Grace period**: Jobs get multiple timeout intervals before termination
|
|
- **Progressive vs HLS**: Different timeout strategies
|
|
- **Resource cleanup**: Process termination, file cleanup, live stream closure
|
|
|
|
### 3. Throttling - Resource Management
|
|
|
|
**Purpose**: Controls transcoding speed to prevent resource waste and improve efficiency.
|
|
|
|
**Conditions for Throttling**:
|
|
```csharp
|
|
private static bool EnableThrottling(StreamState state)
|
|
=> state.InputProtocol == MediaProtocol.File
|
|
&& state.RunTimeTicks.HasValue
|
|
&& state.RunTimeTicks.Value >= TimeSpan.FromMinutes(5).Ticks
|
|
&& state.IsInputVideo
|
|
&& state.VideoType == VideoType.VideoFile;
|
|
```
|
|
|
|
**Throttling Logic**:
|
|
```csharp
|
|
private bool IsThrottleAllowed(TranscodingJob job, int thresholdSeconds)
|
|
{
|
|
var bytesDownloaded = job.BytesDownloaded;
|
|
var transcodingPositionTicks = job.TranscodingPositionTicks ?? 0;
|
|
var downloadPositionTicks = job.DownloadPositionTicks ?? 0;
|
|
|
|
var gapLengthInTicks = TimeSpan.FromSeconds(thresholdSeconds).Ticks;
|
|
|
|
if (downloadPositionTicks > 0 && transcodingPositionTicks > 0)
|
|
{
|
|
// HLS - time-based consideration
|
|
var gap = transcodingPositionTicks - downloadPositionTicks;
|
|
return gap > gapLengthInTicks;
|
|
}
|
|
|
|
// Progressive - byte-based consideration
|
|
// Calculate if transcoding is ahead enough to throttle
|
|
}
|
|
```
|
|
|
|
**Control Mechanism**:
|
|
- **Pause command**: Send 'p' (or 'c' for older FFmpeg) to stdin
|
|
- **Resume command**: Send 'u' (or newline for older FFmpeg) to stdin
|
|
- **Threshold**: Minimum 60 seconds ahead before throttling kicks in
|
|
|
|
### 4. Segment Cleaning - HLS Management
|
|
|
|
**Purpose**: Removes old HLS segments to prevent disk space exhaustion.
|
|
|
|
**Conditions**:
|
|
```csharp
|
|
private static bool EnableSegmentCleaning(StreamState state)
|
|
=> state.InputProtocol is MediaProtocol.File or MediaProtocol.Http
|
|
&& state.IsInputVideo
|
|
&& state.TranscodingType == TranscodingJobType.Hls
|
|
&& state.RunTimeTicks.HasValue
|
|
&& state.RunTimeTicks.Value >= TimeSpan.FromMinutes(5).Ticks;
|
|
```
|
|
|
|
**Implementation**:
|
|
- **Segment retention**: Keeps last N segments (configurable)
|
|
- **Cleanup frequency**: Runs periodically during transcoding
|
|
- **File patterns**: Removes `.ts` or `.mp4` segments and related files
|
|
- **Safety**: Includes retry logic and error handling
|
|
|
|
## FFmpeg Command Structure
|
|
|
|
### Complete Command Template
|
|
|
|
```bash
|
|
{inputModifier} {inputArgument} -map_metadata -1 -map_chapters -1 -threads {threads} {mapArgs} {videoArguments} {audioArguments} -copyts -avoid_negative_ts disabled -max_muxing_queue_size {maxMuxingQueueSize} -f hls -max_delay 5000000 -hls_time {segmentLength} -hls_segment_type {segmentFormat} -start_number {startNumber} -hls_segment_filename "{segmentPath}" {hlsArguments} -y "{outputPath}"
|
|
```
|
|
|
|
### Parameter Breakdown
|
|
|
|
#### Input Modifiers
|
|
```bash
|
|
-re # Read input at native framerate
|
|
-hwaccel cuda # Hardware acceleration (optional)
|
|
-fflags +genpts # Generate presentation timestamps
|
|
-analyzeduration 5000000 # Analysis duration for streams
|
|
-readrate 10 # Input read rate limit (for segment deletion)
|
|
```
|
|
|
|
#### Core Parameters
|
|
```bash
|
|
-threads 0 # Auto-detect thread count
|
|
-map_metadata -1 # Strip metadata
|
|
-map_chapters -1 # Strip chapters
|
|
-copyts # Copy timestamps
|
|
-avoid_negative_ts disabled # Handle negative timestamps
|
|
-max_muxing_queue_size 128 # Muxing queue size
|
|
```
|
|
|
|
#### HLS-Specific Parameters
|
|
```bash
|
|
-f hls # Output format
|
|
-hls_time 6 # Segment duration (seconds)
|
|
-hls_segment_type mpegts # Segment container (mpegts/fmp4)
|
|
-hls_playlist_type vod # Playlist type (vod/event)
|
|
-hls_list_size 0 # Keep all segments in playlist
|
|
-start_number 0 # Starting segment number
|
|
-hls_segment_filename "output%d.ts" # Segment naming pattern
|
|
-hls_base_url "hls/stream/" # Base URL for segments
|
|
```
|
|
|
|
#### Output Specification
|
|
```bash
|
|
"output.m3u8" # Output playlist file
|
|
```
|
|
|
|
### Video Encoding Parameters
|
|
|
|
#### Quality Control
|
|
```bash
|
|
-c:v libx264 # Video codec
|
|
-preset veryfast # Encoding speed/quality trade-off
|
|
-crf 23 # Constant rate factor (quality)
|
|
-maxrate 2000k # Maximum bitrate
|
|
-bufsize 4000k # Buffer size
|
|
```
|
|
|
|
#### Resolution and Framerate
|
|
```bash
|
|
-vf "scale=1920:1080" # Scale to specific resolution
|
|
-r 30 # Output framerate
|
|
-pix_fmt yuv420p # Pixel format
|
|
```
|
|
|
|
### Audio Encoding Parameters
|
|
|
|
#### Basic Audio
|
|
```bash
|
|
-c:a aac # Audio codec
|
|
-ab 128k # Audio bitrate
|
|
-ar 48000 # Sample rate
|
|
-ac 2 # Audio channels
|
|
```
|
|
|
|
#### Advanced Audio Processing
|
|
```bash
|
|
-af "volume=1.0" # Audio filters
|
|
-acodec copy # Copy audio stream
|
|
```
|
|
|
|
## Implementation Guide for Next.js
|
|
|
|
### 1. Core Architecture
|
|
|
|
```typescript
|
|
// Core interfaces
|
|
interface TranscodingJob {
|
|
id: string;
|
|
playSessionId: string;
|
|
process: ChildProcess;
|
|
lastPingDate: Date;
|
|
pingTimeout: number;
|
|
bytesDownloaded: number;
|
|
transcodingPositionTicks: number;
|
|
activeRequestCount: number;
|
|
isUserPaused: boolean;
|
|
hasExited: boolean;
|
|
}
|
|
|
|
interface StreamState {
|
|
outputFilePath: string;
|
|
segmentLength: number;
|
|
inputProtocol: string;
|
|
runTimeTicks: number;
|
|
isInputVideo: boolean;
|
|
transcodingType: 'hls' | 'progressive';
|
|
}
|
|
```
|
|
|
|
### 2. TranscodeManager Implementation
|
|
|
|
```typescript
|
|
class TranscodeManager {
|
|
private activeJobs = new Map<string, TranscodingJob>();
|
|
private killTimers = new Map<string, NodeJS.Timeout>();
|
|
|
|
async startFfmpeg(state: StreamState, playSessionId: string): Promise<TranscodingJob> {
|
|
const commandArgs = this.buildFfmpegCommand(state);
|
|
const process = spawn('ffmpeg', commandArgs);
|
|
|
|
const job: TranscodingJob = {
|
|
id: crypto.randomUUID(),
|
|
playSessionId,
|
|
process,
|
|
lastPingDate: new Date(),
|
|
pingTimeout: state.transcodingType === 'progressive' ? 10000 : 60000,
|
|
bytesDownloaded: 0,
|
|
transcodingPositionTicks: 0,
|
|
activeRequestCount: 1,
|
|
isUserPaused: false,
|
|
hasExited: false
|
|
};
|
|
|
|
this.activeJobs.set(playSessionId, job);
|
|
this.startKillTimer(job);
|
|
|
|
return job;
|
|
}
|
|
|
|
pingTranscodingJob(playSessionId: string, isUserPaused?: boolean) {
|
|
const job = this.activeJobs.get(playSessionId);
|
|
if (!job || job.hasExited) return;
|
|
|
|
if (isUserPaused !== undefined) {
|
|
job.isUserPaused = isUserPaused;
|
|
}
|
|
|
|
job.lastPingDate = new Date();
|
|
this.resetKillTimer(job);
|
|
}
|
|
|
|
private startKillTimer(job: TranscodingJob) {
|
|
this.clearKillTimer(job.playSessionId);
|
|
|
|
const timer = setTimeout(() => {
|
|
this.checkAndKillJob(job);
|
|
}, job.pingTimeout);
|
|
|
|
this.killTimers.set(job.playSessionId, timer);
|
|
}
|
|
|
|
private async checkAndKillJob(job: TranscodingJob) {
|
|
const timeSinceLastPing = Date.now() - job.lastPingDate.getTime();
|
|
|
|
if (timeSinceLastPing < job.pingTimeout) {
|
|
// Reset timer if ping is still fresh
|
|
this.startKillTimer(job);
|
|
return;
|
|
}
|
|
|
|
// Kill the job
|
|
await this.killTranscodingJob(job);
|
|
}
|
|
}
|
|
```
|
|
|
|
### 3. API Endpoints
|
|
|
|
```typescript
|
|
// Next.js API routes
|
|
// /api/transcoding/[playSessionId]/ping
|
|
export async function POST(request: Request, { params }: { params: { playSessionId: string } }) {
|
|
const { isUserPaused } = await request.json();
|
|
transcodeManager.pingTranscodingJob(params.playSessionId, isUserPaused);
|
|
return Response.json({ success: true });
|
|
}
|
|
|
|
// /api/hls/[...segments]
|
|
export async function GET(request: Request, { params }: { params: { segments: string[] } }) {
|
|
const [playSessionId, segmentFile] = params.segments;
|
|
|
|
if (segmentFile.endsWith('.m3u8')) {
|
|
// Return playlist
|
|
return new Response(playlist, {
|
|
headers: { 'Content-Type': 'application/vnd.apple.mpegurl' }
|
|
});
|
|
} else {
|
|
// Return segment file
|
|
const filePath = path.join(transcodingDir, segmentFile);
|
|
const fileStream = fs.createReadStream(filePath);
|
|
return new Response(fileStream as any);
|
|
}
|
|
}
|
|
```
|
|
|
|
### 4. Client Integration
|
|
|
|
```typescript
|
|
// Client-side ping implementation
|
|
class MediaPlayer {
|
|
private pingInterval: NodeJS.Timeout | null = null;
|
|
private playSessionId: string;
|
|
|
|
startPinging() {
|
|
this.pingInterval = setInterval(() => {
|
|
fetch(`/api/transcoding/${this.playSessionId}/ping`, {
|
|
method: 'POST',
|
|
headers: { 'Content-Type': 'application/json' },
|
|
body: JSON.stringify({ isUserPaused: this.isPaused })
|
|
});
|
|
}, 30000); // Ping every 30 seconds
|
|
}
|
|
|
|
stopPinging() {
|
|
if (this.pingInterval) {
|
|
clearInterval(this.pingInterval);
|
|
this.pingInterval = null;
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
## Seeking Implementation
|
|
|
|
Jellyfin implements sophisticated seeking mechanisms for both progressive and HLS transcoding, handling different scenarios and optimizing for performance.
|
|
|
|
### 1. Core Seeking Logic
|
|
|
|
**Central Method**: `GetFastSeekCommandLineParameter()` in `EncodingHelper.cs`
|
|
|
|
```csharp
|
|
public string GetFastSeekCommandLineParameter(EncodingJobInfo state, EncodingOptions options, string segmentContainer)
|
|
{
|
|
var time = state.BaseRequest.StartTimeTicks ?? 0;
|
|
var maxTime = state.RunTimeTicks ?? 0;
|
|
var seekParam = string.Empty;
|
|
|
|
if (time > 0)
|
|
{
|
|
// For direct streaming/remuxing, we seek at the exact position of the keyframe
|
|
// However, ffmpeg will seek to previous keyframe when the exact time is the input
|
|
// Workaround this by adding 0.5s offset to the seeking time to get the exact keyframe on most videos.
|
|
// This will help subtitle syncing.
|
|
var isHlsRemuxing = state.IsVideoRequest && state.TranscodingType is TranscodingJobType.Hls && IsCopyCodec(state.OutputVideoCodec);
|
|
var seekTick = isHlsRemuxing ? time + 5000000L : time;
|
|
|
|
// Seeking beyond EOF makes no sense in transcoding. Clamp the seekTick value to
|
|
// [0, RuntimeTicks - 5.0s], so that the muxer gets packets and avoid error codes.
|
|
if (maxTime > 0)
|
|
{
|
|
seekTick = Math.Clamp(seekTick, 0, Math.Max(maxTime - 50000000L, 0));
|
|
}
|
|
|
|
seekParam += string.Format(CultureInfo.InvariantCulture, "-ss {0}", _mediaEncoder.GetTimeParameter(seekTick));
|
|
|
|
if (state.IsVideoRequest)
|
|
{
|
|
// Add -noaccurate_seek for specific conditions
|
|
if (!string.Equals(state.InputContainer, "wtv", StringComparison.OrdinalIgnoreCase)
|
|
&& !string.Equals(segmentFormat, "ts", StringComparison.OrdinalIgnoreCase)
|
|
&& state.TranscodingType != TranscodingJobType.Progressive
|
|
&& !state.EnableBreakOnNonKeyFrames(outputVideoCodec)
|
|
&& (state.BaseRequest.StartTimeTicks ?? 0) > 0)
|
|
{
|
|
seekParam += " -noaccurate_seek";
|
|
}
|
|
}
|
|
}
|
|
|
|
return seekParam;
|
|
}
|
|
```
|
|
|
|
### 2. Seeking Types
|
|
|
|
#### A. **Input Seeking** (`-ss` before input)
|
|
- **Purpose**: Seek to position before decoding starts
|
|
- **Advantages**: Very fast, minimal CPU usage
|
|
- **Disadvantages**: Less accurate, seeks to nearest keyframe
|
|
- **Usage**: Primary method for initial positioning
|
|
|
|
#### B. **Output Seeking** (`-ss` after input)
|
|
- **Purpose**: Decode from beginning, then seek in output
|
|
- **Advantages**: Frame-accurate positioning
|
|
- **Disadvantages**: High CPU usage, slower startup
|
|
- **Usage**: When precision is critical
|
|
|
|
#### C. **Accurate vs Fast Seeking**
|
|
- **Fast seeking** (`-noaccurate_seek`): Seeks to nearest keyframe (default)
|
|
- **Accurate seeking**: Frame-precise but slower
|
|
- **Dynamic selection**: Based on container and transcoding type
|
|
|
|
### 3. HLS Segment-Based Seeking
|
|
|
|
#### **Segment URL Structure**
|
|
```
|
|
/hls/{itemId}/{playlistId}/{segmentId}.{container}?runtimeTicks={position}&actualSegmentLengthTicks={duration}
|
|
```
|
|
|
|
#### **Key Parameters**:
|
|
- **`runtimeTicks`**: Starting position of segment in media timeline
|
|
- **`actualSegmentLengthTicks`**: Precise duration of this specific segment
|
|
- **`segmentId`**: Sequential segment number (0-based)
|
|
|
|
#### **Segment Calculation**:
|
|
```csharp
|
|
// Equal-length segments
|
|
var segmentLengthTicks = TimeSpan.FromSeconds(segmentLength).Ticks;
|
|
var wholeSegments = runtimeTicks / segmentLengthTicks;
|
|
var remainingTicks = runtimeTicks % segmentLengthTicks;
|
|
|
|
// Keyframe-aware segments (optimal)
|
|
var result = new List<double>();
|
|
var desiredSegmentLengthTicks = TimeSpan.FromMilliseconds(desiredSegmentLengthMs).Ticks;
|
|
foreach (var keyframe in keyframeData.KeyframeTicks)
|
|
{
|
|
if (keyframe >= desiredCutTime)
|
|
{
|
|
var currentSegmentLength = keyframe - lastKeyframe;
|
|
result.Add(TimeSpan.FromTicks(currentSegmentLength).TotalSeconds);
|
|
lastKeyframe = keyframe;
|
|
desiredCutTime += desiredSegmentLengthTicks;
|
|
}
|
|
}
|
|
```
|
|
|
|
### 4. Seek Optimizations
|
|
|
|
#### **HLS Remuxing Offset**
|
|
```csharp
|
|
// Add 0.5s offset for HLS remuxing to hit exact keyframes
|
|
var isHlsRemuxing = state.IsVideoRequest && state.TranscodingType is TranscodingJobType.Hls && IsCopyCodec(state.OutputVideoCodec);
|
|
var seekTick = isHlsRemuxing ? time + 5000000L : time; // +0.5s
|
|
```
|
|
|
|
#### **EOF Protection**
|
|
```csharp
|
|
// Prevent seeking beyond end of file
|
|
if (maxTime > 0)
|
|
{
|
|
seekTick = Math.Clamp(seekTick, 0, Math.Max(maxTime - 50000000L, 0)); // -5s buffer
|
|
}
|
|
```
|
|
|
|
#### **Container-Specific Rules**
|
|
- **WTV containers**: Never use `-noaccurate_seek` (breaks seeking)
|
|
- **MPEGTS segments**: Disable `-noaccurate_seek` for client compatibility
|
|
- **fMP4 containers**: Require `-noaccurate_seek` for audio sync
|
|
|
|
### 5. Keyframe Extraction
|
|
|
|
**Purpose**: Generate precise segment boundaries aligned with keyframes
|
|
|
|
#### **FFprobe Method**:
|
|
```bash
|
|
ffprobe -loglevel error -skip_frame nokey -select_streams v:0 -show_entries packet=pts_time,flags -of csv=print_section=0 "input.mp4"
|
|
```
|
|
|
|
#### **Matroska Method**:
|
|
- Direct cue point extraction from container metadata
|
|
- Much faster than FFprobe for MKV files
|
|
- Reads cue tables for instant keyframe positions
|
|
|
|
### 6. Real-Time Seeking Scenarios
|
|
|
|
#### **Progressive Transcoding**
|
|
```typescript
|
|
// Client seeks to new position
|
|
const seekTo = (positionSeconds: number) => {
|
|
// Kill current transcoding job
|
|
await fetch(`/api/transcode/kill/${playSessionId}`, { method: 'POST' });
|
|
|
|
// Start new transcoding from seek position
|
|
const startTimeTicks = positionSeconds * 10000000; // Convert to ticks
|
|
const newUrl = `/api/videos/${itemId}/stream?startTimeTicks=${startTimeTicks}`;
|
|
|
|
// Update video source
|
|
videoElement.src = newUrl;
|
|
};
|
|
```
|
|
|
|
#### **HLS Seeking**
|
|
```typescript
|
|
// HLS seeking is handled by the player automatically
|
|
// Server generates segments with proper seek points
|
|
const hlsPlayer = new Hls();
|
|
hlsPlayer.loadSource('/api/videos/{itemId}/master.m3u8');
|
|
|
|
// Player handles seeking by requesting appropriate segments
|
|
// No need to restart transcoding jobs
|
|
```
|
|
|
|
### 7. External Media Handling
|
|
|
|
#### **External Subtitles**
|
|
```csharp
|
|
// Also seek external subtitle streams
|
|
var seekSubParam = GetFastSeekCommandLineParameter(state, options, segmentContainer);
|
|
if (!string.IsNullOrEmpty(seekSubParam))
|
|
{
|
|
arg.Append(' ').Append(seekSubParam);
|
|
}
|
|
arg.Append(" -i file:\"").Append(subtitlePath).Append('"');
|
|
```
|
|
|
|
#### **External Audio**
|
|
```csharp
|
|
// Seek external audio streams to match video
|
|
var seekAudioParam = GetFastSeekCommandLineParameter(state, options, segmentContainer);
|
|
if (!string.IsNullOrEmpty(seekAudioParam))
|
|
{
|
|
arg.Append(' ').Append(seekAudioParam);
|
|
}
|
|
arg.Append(" -i \"").Append(state.AudioStream.Path).Append('"');
|
|
```
|
|
|
|
### 8. Implementation Guide for Next.js
|
|
|
|
#### **Progressive Seeking**
|
|
```typescript
|
|
class ProgressiveTranscoder {
|
|
async seekTo(positionTicks: number): Promise<string> {
|
|
// Kill existing job
|
|
await this.killCurrentJob();
|
|
|
|
// Calculate seek parameters
|
|
const seekSeconds = positionTicks / 10000000;
|
|
const maxTime = this.mediaDuration;
|
|
|
|
// Clamp to safe bounds
|
|
const safeSeekTicks = Math.max(0, Math.min(positionTicks, maxTime - 50000000));
|
|
|
|
// Build FFmpeg command with seek
|
|
const args = [
|
|
'-ss', this.formatTime(safeSeekTicks),
|
|
'-i', this.inputPath,
|
|
'-c:v', 'libx264',
|
|
'-preset', 'veryfast',
|
|
// ... other encoding params
|
|
this.outputPath
|
|
];
|
|
|
|
return this.startTranscoding(args);
|
|
}
|
|
|
|
private formatTime(ticks: number): string {
|
|
const totalSeconds = ticks / 10000000;
|
|
const hours = Math.floor(totalSeconds / 3600);
|
|
const minutes = Math.floor((totalSeconds % 3600) / 60);
|
|
const seconds = totalSeconds % 60;
|
|
return `${hours}:${minutes.toString().padStart(2, '0')}:${seconds.toFixed(6).padStart(9, '0')}`;
|
|
}
|
|
}
|
|
```
|
|
|
|
#### **HLS Seeking**
|
|
```typescript
|
|
class HLSTranscoder {
|
|
generateSegmentUrl(segmentId: number, runtimeTicks: number, segmentDurationTicks: number): string {
|
|
const params = new URLSearchParams({
|
|
runtimeTicks: runtimeTicks.toString(),
|
|
actualSegmentLengthTicks: segmentDurationTicks.toString()
|
|
});
|
|
|
|
return `/api/hls/${this.itemId}/${this.playlistId}/${segmentId}.ts?${params}`;
|
|
}
|
|
|
|
async generateSegment(segmentId: number, runtimeTicks: number): Promise<Buffer> {
|
|
const seekSeconds = runtimeTicks / 10000000;
|
|
|
|
const args = [
|
|
'-ss', this.formatTime(runtimeTicks),
|
|
'-i', this.inputPath,
|
|
'-t', this.segmentDuration.toString(),
|
|
'-c:v', 'libx264',
|
|
'-preset', 'veryfast',
|
|
'-force_key_frames', `expr:gte(t,n_forced*${this.segmentDuration})`,
|
|
'-f', 'mpegts',
|
|
'-'
|
|
];
|
|
|
|
return this.executeFFmpeg(args);
|
|
}
|
|
}
|
|
```
|
|
|
|
### 9. Performance Considerations
|
|
|
|
#### **Seek Performance Tips**:
|
|
1. **Use input seeking** (`-ss` before `-i`) when possible
|
|
2. **Cache keyframe data** for containers that support it
|
|
3. **Implement seek debouncing** to prevent rapid job restarts
|
|
4. **Use appropriate segment duration** (6s recommended for seek performance)
|
|
5. **Pre-generate keyframe indexes** for frequently accessed content
|
|
|
|
#### **Client-Side Optimizations**:
|
|
```typescript
|
|
// Debounce seeking to prevent excessive requests
|
|
const debouncedSeek = debounce((position: number) => {
|
|
this.performSeek(position);
|
|
}, 300);
|
|
|
|
// Progressive seeking strategy
|
|
if (Math.abs(targetPosition - currentPosition) < 30) {
|
|
// Small seeks: let player buffer naturally
|
|
player.currentTime = targetPosition;
|
|
} else {
|
|
// Large seeks: restart transcoding
|
|
this.seekTo(targetPosition);
|
|
}
|
|
```
|
|
|
|
## Progress Tracking and Seeking During Transcoding
|
|
|
|
**The Challenge**: Unlike direct play where duration and seek positions are straightforward, transcoding creates a "streaming-like" scenario where the real duration is not immediately available and progress tracking becomes complex.
|
|
|
|
### 1. Core Progress Tracking Architecture
|
|
|
|
**Key Components**:
|
|
- **`TranscodingPositionTicks`**: Where FFmpeg transcoding has currently reached
|
|
- **`DownloadPositionTicks`**: Where the client has consumed content to
|
|
- **`CompletionPercentage`**: Calculated progress based on runtime vs current position
|
|
- **`RunTimeTicks`**: Total media duration from metadata
|
|
|
|
#### **Progress Calculation Logic**:
|
|
```csharp
|
|
// From JobLogger.ParseLogLine() - extracts progress from FFmpeg output
|
|
var totalMs = state.RunTimeTicks.HasValue
|
|
? TimeSpan.FromTicks(state.RunTimeTicks.Value).TotalMilliseconds
|
|
: 0;
|
|
|
|
var currentMs = /* parsed from FFmpeg time output */;
|
|
|
|
if (totalMs > 0)
|
|
{
|
|
percent = 100.0 * currentMs / totalMs;
|
|
transcodingPosition = TimeSpan.FromMilliseconds(currentMs);
|
|
}
|
|
```
|
|
|
|
### 2. Real-Time Progress Updates
|
|
|
|
#### **FFmpeg Output Parsing**:
|
|
```csharp
|
|
// JobLogger monitors FFmpeg stderr output for progress
|
|
private void ParseLogLine(string line, EncodingJobInfo state)
|
|
{
|
|
// Parse: frame= 123 fps= 25 q=28.0 size= 1024kB time=00:01:23.45 bitrate= 512.0kbits/s
|
|
// Extract: time value for current transcoding position
|
|
// Extract: size value for bytes transcoded
|
|
// Extract: bitrate for current encoding rate
|
|
}
|
|
```
|
|
|
|
#### **Progress Reporting Chain**:
|
|
```typescript
|
|
// 1. FFmpeg outputs progress to stderr
|
|
// 2. JobLogger.ParseLogLine() extracts values
|
|
// 3. TranscodeManager.ReportTranscodingProgress() updates job state
|
|
// 4. SessionManager.ReportTranscodingInfo() updates client session
|
|
// 5. TranscodingInfo DTO sent to client via WebSocket/API
|
|
|
|
interface TranscodingInfo {
|
|
CompletionPercentage?: number; // 0-100 progress
|
|
Bitrate?: number; // Current bitrate
|
|
Framerate?: number; // Current FPS
|
|
Width?: number; // Video width
|
|
Height?: number; // Video height
|
|
AudioCodec: string; // Audio codec in use
|
|
VideoCodec: string; // Video codec in use
|
|
Container: string; // Output container
|
|
}
|
|
```
|
|
|
|
### 3. Client Progress Bar Implementation
|
|
|
|
#### **Progressive Transcoding**:
|
|
```typescript
|
|
class ProgressiveTranscodingProgress {
|
|
private transcodingInfo: TranscodingInfo;
|
|
private mediaRunTimeTicks: number;
|
|
|
|
updateProgressBar(): void {
|
|
if (this.transcodingInfo?.CompletionPercentage) {
|
|
// Use transcoding percentage directly
|
|
const progress = this.transcodingInfo.CompletionPercentage / 100;
|
|
this.progressBar.value = progress;
|
|
|
|
// Estimate current playable duration
|
|
const availableDuration = this.mediaRunTimeTicks * progress;
|
|
this.updateSeekableRange(0, availableDuration);
|
|
}
|
|
}
|
|
|
|
handleSeek(targetPositionTicks: number): void {
|
|
const transcodedTicks = this.mediaRunTimeTicks * (this.transcodingInfo.CompletionPercentage / 100);
|
|
|
|
if (targetPositionTicks <= transcodedTicks) {
|
|
// Seek within transcoded content
|
|
this.player.currentTime = targetPositionTicks / 10000000;
|
|
} else {
|
|
// Restart transcoding from seek position
|
|
this.startTranscodingFromPosition(targetPositionTicks);
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
#### **HLS Transcoding**:
|
|
```typescript
|
|
class HLSTranscodingProgress {
|
|
private segmentDuration: number = 6; // seconds
|
|
private totalSegments: number;
|
|
|
|
calculateProgress(): ProgressInfo {
|
|
// HLS progress based on segment availability
|
|
const availableSegments = this.getAvailableSegmentCount();
|
|
const progress = availableSegments / this.totalSegments;
|
|
|
|
return {
|
|
percentage: progress * 100,
|
|
availableDuration: availableSegments * this.segmentDuration,
|
|
seekableEnd: availableSegments * this.segmentDuration
|
|
};
|
|
}
|
|
|
|
updateSegmentDownloadPosition(): void {
|
|
// Update DownloadPositionTicks when segments are consumed
|
|
const segmentEndTicks = this.currentRuntimeTicks + this.actualSegmentLengthTicks;
|
|
this.transcodingJob.DownloadPositionTicks = Math.max(
|
|
this.transcodingJob.DownloadPositionTicks ?? segmentEndTicks,
|
|
segmentEndTicks
|
|
);
|
|
}
|
|
}
|
|
```
|
|
|
|
### 4. Solving the "Real Duration is Not Real" Problem
|
|
|
|
#### **Duration Estimation Strategies**:
|
|
|
|
**1. Metadata-Based Duration**:
|
|
```typescript
|
|
// Use media file metadata as baseline
|
|
const estimatedDuration = mediaSource.RunTimeTicks;
|
|
if (estimatedDuration) {
|
|
this.totalDuration = estimatedDuration;
|
|
this.progressBar.max = estimatedDuration;
|
|
}
|
|
```
|
|
|
|
**2. Progressive Duration Discovery**:
|
|
```typescript
|
|
// Update duration as transcoding progresses
|
|
if (transcodingInfo.CompletionPercentage > 0) {
|
|
const currentTranscodedTicks = /* current position from transcoding */;
|
|
const estimatedTotal = currentTranscodedTicks / (transcodingInfo.CompletionPercentage / 100);
|
|
|
|
// Only update if estimate seems reliable (>10% transcoded)
|
|
if (transcodingInfo.CompletionPercentage > 10) {
|
|
this.totalDuration = estimatedTotal;
|
|
}
|
|
}
|
|
```
|
|
|
|
**3. HLS Segment-Based Calculation**:
|
|
```typescript
|
|
// For HLS, calculate from segment structure
|
|
const calculateHLSDuration = (segments: SegmentInfo[]): number => {
|
|
return segments.reduce((total, segment) => {
|
|
return total + segment.actualSegmentLengthTicks;
|
|
}, 0);
|
|
};
|
|
```
|
|
|
|
### 5. Advanced Progress Management
|
|
|
|
#### **Buffering and Availability**:
|
|
```typescript
|
|
class TranscodingBuffer {
|
|
private bufferAheadSeconds: number = 30;
|
|
|
|
isPositionAvailable(targetPositionTicks: number): boolean {
|
|
const transcodedTicks = this.getTranscodedPositionTicks();
|
|
return targetPositionTicks <= transcodedTicks;
|
|
}
|
|
|
|
calculateSeekableRange(): { start: number; end: number } {
|
|
return {
|
|
start: 0,
|
|
end: this.getTranscodedPositionTicks() - (this.bufferAheadSeconds * 10000000)
|
|
};
|
|
}
|
|
|
|
shouldThrottleTranscoding(): boolean {
|
|
const gap = this.transcodingPositionTicks - this.downloadPositionTicks;
|
|
const targetGap = this.bufferAheadSeconds * 10000000; // 30s in ticks
|
|
return gap > targetGap;
|
|
}
|
|
}
|
|
```
|
|
|
|
#### **Smooth Progress Updates**:
|
|
```typescript
|
|
class SmoothProgressUpdater {
|
|
private interpolationInterval: number = 1000; // 1 second
|
|
private lastKnownPosition: number = 0;
|
|
private lastUpdateTime: number = Date.now();
|
|
|
|
interpolateProgress(): number {
|
|
if (!this.isPlaying) return this.lastKnownPosition;
|
|
|
|
const now = Date.now();
|
|
const elapsed = now - this.lastUpdateTime;
|
|
const estimatedProgress = this.lastKnownPosition + elapsed;
|
|
|
|
// Don't exceed known transcoded position
|
|
const maxAvailable = this.getTranscodedPositionTicks();
|
|
return Math.min(estimatedProgress, maxAvailable);
|
|
}
|
|
}
|
|
```
|
|
|
|
### 6. Error Handling and Edge Cases
|
|
|
|
#### **Transcoding Failures**:
|
|
```typescript
|
|
class TranscodingErrorHandler {
|
|
handleTranscodingError(error: TranscodingError): void {
|
|
switch (error.type) {
|
|
case 'SEEK_BEYOND_DURATION':
|
|
// Clamp seek to valid range
|
|
this.seekTo(Math.min(this.targetPosition, this.maxAvailablePosition));
|
|
break;
|
|
|
|
case 'TRANSCODING_STALLED':
|
|
// Restart transcoding
|
|
this.restartTranscodingFromLastKnownPosition();
|
|
break;
|
|
|
|
case 'INVALID_DURATION':
|
|
// Fall back to live estimation
|
|
this.enableLiveDurationEstimation();
|
|
break;
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
#### **Network Issues**:
|
|
```typescript
|
|
class NetworkResilienceHandler {
|
|
private retryPolicy = {
|
|
maxRetries: 3,
|
|
backoffMs: [1000, 2000, 4000]
|
|
};
|
|
|
|
async handleProgressUpdateFailure(attempt: number): Promise<void> {
|
|
if (attempt < this.retryPolicy.maxRetries) {
|
|
await this.delay(this.retryPolicy.backoffMs[attempt]);
|
|
return this.fetchProgressUpdate();
|
|
} else {
|
|
// Switch to local time-based estimation
|
|
this.enableLocalProgressEstimation();
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
### 7. Next.js Implementation Guide
|
|
|
|
#### **Complete Progress Management**:
|
|
```typescript
|
|
class NextJSTranscodingProgressManager {
|
|
private wsConnection: WebSocket;
|
|
private progressUpdateInterval: NodeJS.Timeout;
|
|
|
|
constructor(private videoElement: HTMLVideoElement) {
|
|
this.setupWebSocketUpdates();
|
|
this.setupProgressInterpolation();
|
|
}
|
|
|
|
private setupWebSocketUpdates(): void {
|
|
this.wsConnection.onmessage = (event) => {
|
|
const message = JSON.parse(event.data);
|
|
|
|
if (message.MessageType === 'TranscodingInfo') {
|
|
this.updateTranscodingInfo(message.Data);
|
|
}
|
|
};
|
|
}
|
|
|
|
private updateTranscodingInfo(info: TranscodingInfo): void {
|
|
// Update progress bar
|
|
if (info.CompletionPercentage) {
|
|
this.progressBar.value = info.CompletionPercentage;
|
|
}
|
|
|
|
// Update seekable range
|
|
const seekableEnd = this.calculateSeekableEnd(info);
|
|
this.videoElement.setAttribute('data-seekable-end', seekableEnd.toString());
|
|
|
|
// Update duration if we have better estimate
|
|
this.updateDurationEstimate(info);
|
|
}
|
|
|
|
async handleSeek(targetSeconds: number): Promise<void> {
|
|
const targetTicks = targetSeconds * 10000000;
|
|
const transcodedTicks = this.getTranscodedPositionTicks();
|
|
|
|
if (targetTicks <= transcodedTicks) {
|
|
// Seek within available content
|
|
this.videoElement.currentTime = targetSeconds;
|
|
} else {
|
|
// Show loading state
|
|
this.showBufferingState();
|
|
|
|
// Request new transcoding position
|
|
await this.requestTranscodingFromPosition(targetTicks);
|
|
|
|
// Update video source
|
|
this.videoElement.src = this.generateStreamUrl(targetTicks);
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
## Important Configuration
|
|
|
|
### Environment Variables
|
|
```bash
|
|
FFMPEG_PATH=/usr/bin/ffmpeg
|
|
TRANSCODING_TEMP_PATH=/tmp/jellyfin/transcoding
|
|
MAX_CONCURRENT_STREAMS=3
|
|
SEGMENT_KEEP_SECONDS=300
|
|
THROTTLE_DELAY_SECONDS=60
|
|
```
|
|
|
|
### Performance Tuning
|
|
- **Thread count**: Auto-detect based on CPU cores
|
|
- **Buffer sizes**: Adjust based on available memory
|
|
- **Segment duration**: 6 seconds for good seek performance
|
|
- **Concurrent streams**: Limit based on system resources
|
|
|
|
### Security Considerations
|
|
- **Input validation**: Sanitize all file paths and parameters
|
|
- **Resource limits**: Prevent DOS through excessive transcoding
|
|
- **Access control**: Validate session ownership
|
|
- **File cleanup**: Remove orphaned files regularly
|
|
|
|
## Monitoring and Logging
|
|
|
|
### Key Metrics to Track
|
|
- Active transcoding jobs count
|
|
- Resource usage (CPU, memory, disk I/O)
|
|
- Average transcoding speed vs playback speed
|
|
- Client ping frequency and timeouts
|
|
- Segment cleanup efficiency
|
|
|
|
### Error Handling
|
|
- FFmpeg process failures
|
|
- Disk space exhaustion
|
|
- Network timeouts
|
|
- Invalid media files
|
|
- Hardware acceleration failures
|
|
|
|
This architecture provides a robust foundation for building a media transcoding system with proper resource management, client lifecycle handling, and performance optimization. |