docs(build): add comprehensive build guide for playlist monitor
- Provide detailed system requirements and software dependencies - Describe multiple build options including uv, pip/venv, and Poetry - Include steps for development setup, database initialization, and production build - Document UI integration options: standalone web UI, Angular UI extension, and simple dashboard - Add Docker build instructions for development, production, and multi-stage builds - Provide troubleshooting tips for build and runtime issues - Offer advanced build options such as custom database backend, Redis caching, HTTPS, and load balancing - Add a build checklist and next step recommendations for users docs(getting-started): add complete getting started guide for playlist monitor - Summarize implemented backend features and note missing UI - Present quick start paths: API-only, simple UI, and full React app - Detail API usage with examples and testing instructions - Explain Docker deployment instructions - List UI implementation priorities and recommended next steps - Supply common setup issues and resolution tips - Provide community and debugging resources - Define success metrics and final usage recommendations docs(quick-start): add quick start guide for rapid setup of playlist monitor - Outline prerequisites including Python and MeTube requirements - Provide recommended Docker usage instructions - Detail manual installation steps and command examples - Explain how to access API documentation and MeTube service - Include example commands to add and monitor playlists via API - Offer troubleshooting advice for service startup, connectivity, and database issues - Present common operational commands for playlist
This commit is contained in:
parent
02260fd139
commit
b01ef238e2
|
|
@ -0,0 +1,522 @@
|
|||
# Playlist Monitor Service - Comprehensive Build Guide
|
||||
|
||||
## 📋 Table of Contents
|
||||
1. [Prerequisites](#prerequisites)
|
||||
2. [Build Options](#build-options)
|
||||
3. [Development Setup](#development-setup)
|
||||
4. [Production Build](#production-build)
|
||||
5. [UI Integration Options](#ui-integration-options)
|
||||
6. [Docker Build](#docker-build)
|
||||
7. [Troubleshooting](#troubleshooting)
|
||||
8. [Advanced Build Options](#advanced-build-options)
|
||||
|
||||
## 🔧 Prerequisites
|
||||
|
||||
### System Requirements
|
||||
- **Python**: 3.13+ (3.14 recommended)
|
||||
- **Operating System**: Linux/macOS/Windows WSL2
|
||||
- **Memory**: Minimum 512MB RAM, 1GB recommended
|
||||
- **Storage**: 100MB for application + space for downloads
|
||||
- **Network**: Access to YouTube and MeTube instance
|
||||
|
||||
### Software Dependencies
|
||||
```bash
|
||||
# Ubuntu/Debian
|
||||
sudo apt update
|
||||
sudo apt install python3.13 python3.13-dev python3.13-venv build-essential curl git
|
||||
|
||||
# macOS (using Homebrew)
|
||||
brew install python@3.13 curl git
|
||||
|
||||
# CentOS/RHEL/Fedora
|
||||
sudo dnf install python3.13 python3.13-devel gcc curl git
|
||||
```
|
||||
|
||||
### MeTube Requirements
|
||||
- MeTube instance running (locally or remotely)
|
||||
- MeTube version that supports playlist downloads
|
||||
- Network connectivity between services
|
||||
|
||||
## 🏗️ Build Options
|
||||
|
||||
### Option 1: Modern Python with uv (Recommended)
|
||||
**Fastest, most reliable method**
|
||||
|
||||
```bash
|
||||
# Install uv package manager
|
||||
curl -LsSf https://astral.sh/uv/install.sh | sh
|
||||
source ~/.cargo/env
|
||||
|
||||
# Clone and setup
|
||||
cd /root/workspace/tubewatch/playlist-monitor
|
||||
uv sync
|
||||
|
||||
# Run
|
||||
uv run python -m app.main
|
||||
```
|
||||
|
||||
### Option 2: Traditional pip/venv
|
||||
**Standard Python approach**
|
||||
|
||||
```bash
|
||||
# Create virtual environment
|
||||
python3.13 -m venv venv
|
||||
source venv/bin/activate # Linux/macOS
|
||||
# or
|
||||
venv\Scripts\activate # Windows
|
||||
|
||||
# Install dependencies
|
||||
pip install --upgrade pip
|
||||
pip install -e .
|
||||
|
||||
# Run
|
||||
python -m app.main
|
||||
```
|
||||
|
||||
### Option 3: Poetry (Alternative)
|
||||
**For Poetry users**
|
||||
|
||||
```bash
|
||||
# Install Poetry
|
||||
curl -sSL https://install.python-poetry.org | python3 -
|
||||
|
||||
# Setup project
|
||||
poetry install
|
||||
poetry run python -m app.main
|
||||
```
|
||||
|
||||
## 🛠️ Development Setup
|
||||
|
||||
### 1. Environment Configuration
|
||||
```bash
|
||||
# Copy environment template
|
||||
cp .env.example .env
|
||||
|
||||
# Edit configuration
|
||||
nano .env
|
||||
```
|
||||
|
||||
**Essential settings:**
|
||||
```env
|
||||
# MeTube Integration
|
||||
METUBE_URL=http://localhost:8081
|
||||
|
||||
# Database
|
||||
DATABASE_URL=sqlite:///data/playlists.db
|
||||
|
||||
# Server
|
||||
HOST=0.0.0.0
|
||||
PORT=8082
|
||||
|
||||
# Scheduler
|
||||
DEFAULT_CHECK_INTERVAL=60
|
||||
MAX_CONCURRENT_DOWNLOADS=3
|
||||
|
||||
# Logging
|
||||
LOG_LEVEL=INFO
|
||||
```
|
||||
|
||||
### 2. Development Dependencies
|
||||
```bash
|
||||
# Install development dependencies
|
||||
uv sync --extra dev
|
||||
|
||||
# Or with pip
|
||||
pip install -e ".[dev]"
|
||||
```
|
||||
|
||||
### 3. Development Tools Setup
|
||||
```bash
|
||||
# Code formatting
|
||||
uv run black app/ tests/
|
||||
uv run isort app/ tests/
|
||||
|
||||
# Type checking
|
||||
uv run mypy app/
|
||||
|
||||
# Linting
|
||||
uv run flake8 app/ tests/
|
||||
```
|
||||
|
||||
### 4. Database Setup
|
||||
```bash
|
||||
# Create data directory
|
||||
mkdir -p data logs
|
||||
|
||||
# Initialize database (automatic on first run)
|
||||
uv run python -c "from app.core.database import engine, Base; Base.metadata.create_all(bind=engine)"
|
||||
```
|
||||
|
||||
## 🚀 Production Build
|
||||
|
||||
### Step 1: Production Dependencies
|
||||
```bash
|
||||
# Install only production dependencies
|
||||
uv sync --no-dev
|
||||
|
||||
# Or with pip
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
### Step 2: Production Configuration
|
||||
```bash
|
||||
# Create production environment
|
||||
cp .env.example .env.production
|
||||
|
||||
# Edit production settings
|
||||
nano .env.production
|
||||
```
|
||||
|
||||
**Production recommendations:**
|
||||
```env
|
||||
# Security
|
||||
DEBUG=false
|
||||
LOG_LEVEL=WARNING
|
||||
CORS_ORIGINS=["https://yourdomain.com"]
|
||||
|
||||
# Performance
|
||||
MAX_CONCURRENT_DOWNLOADS=5
|
||||
DEFAULT_CHECK_INTERVAL=30
|
||||
|
||||
# Database (PostgreSQL recommended)
|
||||
DATABASE_URL=postgresql://user:pass@localhost/metube_playlists
|
||||
```
|
||||
|
||||
### Step 3: Production Server
|
||||
```bash
|
||||
# Use production ASGI server
|
||||
uv run gunicorn app.main:app \
|
||||
--workers 4 \
|
||||
--worker-class uvicorn.workers.UvicornWorker \
|
||||
--bind 0.0.0.0:8082 \
|
||||
--log-level info
|
||||
|
||||
# Or with uvicorn directly
|
||||
uv run uvicorn app.main:app \
|
||||
--host 0.0.0.0 \
|
||||
--port 8082 \
|
||||
--workers 4 \
|
||||
--log-level info
|
||||
```
|
||||
|
||||
## 🎨 UI Integration Options
|
||||
|
||||
### Option 1: Standalone Web UI (Recommended)
|
||||
Create a separate frontend application that communicates with the API.
|
||||
|
||||
**Technologies:**
|
||||
- **React** with TypeScript
|
||||
- **Vue.js 3** with Composition API
|
||||
- **SvelteKit** for modern approach
|
||||
|
||||
**Example React setup:**
|
||||
```bash
|
||||
# Create React app
|
||||
cd /root/workspace/tubewatch
|
||||
npx create-react-app playlist-monitor-ui --template typescript
|
||||
cd playlist-monitor-ui
|
||||
|
||||
# Install UI libraries
|
||||
npm install @mui/material @emotion/react @emotion/styled axios react-query
|
||||
npm install @mui/icons-material @mui/x-data-grid date-fns
|
||||
|
||||
# Start development
|
||||
npm start
|
||||
```
|
||||
|
||||
### Option 2: Extend MeTube Angular UI
|
||||
Modify the existing MeTube Angular frontend to include playlist monitoring.
|
||||
|
||||
**Approach:**
|
||||
1. Add new Angular components to MeTube's UI
|
||||
2. Integrate with existing MeTube routing
|
||||
3. Use MeTube's existing styling and components
|
||||
|
||||
**Integration points:**
|
||||
- Add playlist tab to main navigation
|
||||
- Extend existing download interface
|
||||
- Reuse MeTube's component library
|
||||
|
||||
### Option 3: Simple HTML Dashboard
|
||||
For minimal UI requirements.
|
||||
|
||||
```bash
|
||||
# Create simple dashboard
|
||||
mkdir ui
|
||||
cd ui
|
||||
|
||||
# Create index.html with vanilla JS
|
||||
cat > index.html << 'EOF'
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<title>Playlist Monitor</title>
|
||||
<link href="https://cdn.jsdelivr.net/npm/bootstrap@5.3.0/dist/css/bootstrap.min.css" rel="stylesheet">
|
||||
</head>
|
||||
<body>
|
||||
<div class="container mt-4">
|
||||
<h1>Playlist Monitor Dashboard</h1>
|
||||
<div id="playlists"></div>
|
||||
</div>
|
||||
<script src="https://cdn.jsdelivr.net/npm/bootstrap@5.3.0/dist/js/bootstrap.bundle.min.js"></script>
|
||||
<script src="app.js"></script>
|
||||
</body>
|
||||
</html>
|
||||
EOF
|
||||
|
||||
# Simple JavaScript API client
|
||||
cat > app.js << 'EOF'
|
||||
const API_BASE = 'http://localhost:8082/api';
|
||||
|
||||
async function loadPlaylists() {
|
||||
try {
|
||||
const response = await fetch(`${API_BASE}/playlists`);
|
||||
const data = await response.json();
|
||||
displayPlaylists(data);
|
||||
} catch (error) {
|
||||
console.error('Error loading playlists:', error);
|
||||
}
|
||||
}
|
||||
|
||||
function displayPlaylists(playlists) {
|
||||
const container = document.getElementById('playlists');
|
||||
container.innerHTML = playlists.map(playlist => `
|
||||
<div class="card mb-3">
|
||||
<div class="card-body">
|
||||
<h5 class="card-title">${playlist.title || 'Untitled'}</h5>
|
||||
<p class="card-text">${playlist.url}</p>
|
||||
<p>Status: ${playlist.enabled ? 'Enabled' : 'Disabled'}</p>
|
||||
</div>
|
||||
</div>
|
||||
`).join('');
|
||||
}
|
||||
|
||||
loadPlaylists();
|
||||
EOF
|
||||
```
|
||||
|
||||
## 🐳 Docker Build
|
||||
|
||||
### Development Docker Build
|
||||
```bash
|
||||
# Build development image
|
||||
docker build -t playlist-monitor:dev .
|
||||
|
||||
# Run development container
|
||||
docker run -d \
|
||||
--name playlist-monitor-dev \
|
||||
-p 8082:8082 \
|
||||
-e DEBUG=true \
|
||||
-v $(pwd)/data:/app/data \
|
||||
-v $(pwd)/logs:/app/logs \
|
||||
playlist-monitor:dev
|
||||
```
|
||||
|
||||
### Production Docker Build
|
||||
```bash
|
||||
# Build production image
|
||||
docker build -t playlist-monitor:latest .
|
||||
|
||||
# Run production container
|
||||
docker run -d \
|
||||
--name playlist-monitor \
|
||||
-p 8082:8082 \
|
||||
-e METUBE_URL=http://metube:8081 \
|
||||
-v playlist-data:/app/data \
|
||||
-v playlist-logs:/app/logs \
|
||||
--restart unless-stopped \
|
||||
--health-cmd="curl -f http://localhost:8082/health || exit 1" \
|
||||
--health-interval=30s \
|
||||
playlist-monitor:latest
|
||||
```
|
||||
|
||||
### Multi-stage Docker Build (Optimized)
|
||||
```dockerfile
|
||||
# Build stage
|
||||
FROM python:3.13-slim as builder
|
||||
|
||||
WORKDIR /build
|
||||
COPY pyproject.toml ./
|
||||
RUN pip install uv
|
||||
RUN uv sync --no-dev
|
||||
|
||||
# Runtime stage
|
||||
FROM python:3.13-slim
|
||||
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
curl && \
|
||||
rm -rf /var/lib/apt/lists/*
|
||||
|
||||
WORKDIR /app
|
||||
COPY --from=builder /build/.venv ./.venv
|
||||
COPY app/ ./app/
|
||||
|
||||
ENV PATH="/app/.venv/bin:$PATH"
|
||||
|
||||
USER 1000:1000
|
||||
EXPOSE 8082
|
||||
|
||||
CMD ["python", "-m", "app.main"]
|
||||
```
|
||||
|
||||
## 🔍 Troubleshooting
|
||||
|
||||
### Build Issues
|
||||
|
||||
#### Python Version Mismatch
|
||||
```bash
|
||||
# Check Python version
|
||||
python3 --version
|
||||
|
||||
# Install correct version
|
||||
pyenv install 3.13.0
|
||||
pyenv local 3.13.0
|
||||
```
|
||||
|
||||
#### Dependency Conflicts
|
||||
```bash
|
||||
# Clear cache
|
||||
uv cache clean
|
||||
rm -rf .venv
|
||||
|
||||
# Reinstall
|
||||
uv sync
|
||||
```
|
||||
|
||||
#### Compilation Errors
|
||||
```bash
|
||||
# Install build dependencies
|
||||
sudo apt install build-essential python3-dev
|
||||
|
||||
# Or use pre-built wheels
|
||||
pip install --only-binary=all -r requirements.txt
|
||||
```
|
||||
|
||||
### Runtime Issues
|
||||
|
||||
#### Port Already in Use
|
||||
```bash
|
||||
# Find process using port 8082
|
||||
lsof -i :8082
|
||||
|
||||
# Kill process
|
||||
kill -9 <PID>
|
||||
|
||||
# Or use different port
|
||||
export PORT=8083
|
||||
```
|
||||
|
||||
#### Database Locked
|
||||
```bash
|
||||
# Check for existing processes
|
||||
ps aux | grep playlist-monitor
|
||||
|
||||
# Remove lock file
|
||||
rm data/*.db-journal
|
||||
```
|
||||
|
||||
#### MeTube Connection Failed
|
||||
```bash
|
||||
# Test MeTube connectivity
|
||||
curl http://localhost:8081/info
|
||||
|
||||
# Check MeTube logs
|
||||
docker logs metube
|
||||
```
|
||||
|
||||
## 🔧 Advanced Build Options
|
||||
|
||||
### Custom Database Backend
|
||||
```bash
|
||||
# PostgreSQL setup
|
||||
docker run -d \
|
||||
--name postgres-playlist \
|
||||
-e POSTGRES_DB=playlists \
|
||||
-e POSTGRES_USER=playlist_user \
|
||||
-e POSTGRES_PASSWORD=secure_password \
|
||||
-p 5432:5432 \
|
||||
postgres:15
|
||||
|
||||
# Update .env
|
||||
DATABASE_URL=postgresql://playlist_user:secure_password@localhost:5432/playlists
|
||||
```
|
||||
|
||||
### Redis for Caching
|
||||
```bash
|
||||
# Redis setup
|
||||
docker run -d \
|
||||
--name redis-playlist \
|
||||
-p 6379:6379 \
|
||||
redis:7-alpine
|
||||
|
||||
# Install Redis dependencies
|
||||
uv add redis
|
||||
```
|
||||
|
||||
### HTTPS/SSL Configuration
|
||||
```bash
|
||||
# Generate SSL certificates
|
||||
openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -days 365 -nodes
|
||||
|
||||
# Update configuration
|
||||
HTTPS_ENABLED=true
|
||||
SSL_CERTFILE=cert.pem
|
||||
SSL_KEYFILE=key.pem
|
||||
```
|
||||
|
||||
### Load Balancing with Nginx
|
||||
```nginx
|
||||
upstream playlist_monitor {
|
||||
server localhost:8082;
|
||||
server localhost:8083;
|
||||
server localhost:8084;
|
||||
}
|
||||
|
||||
server {
|
||||
listen 80;
|
||||
server_name your-domain.com;
|
||||
|
||||
location / {
|
||||
proxy_pass http://playlist_monitor;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## 📋 Build Checklist
|
||||
|
||||
### Pre-build
|
||||
- [ ] Python 3.13+ installed
|
||||
- [ ] MeTube accessible
|
||||
- [ ] Required ports available (8082)
|
||||
- [ ] Sufficient disk space
|
||||
|
||||
### Build Process
|
||||
- [ ] Dependencies installed successfully
|
||||
- [ ] Configuration file created
|
||||
- [ ] Database initialized
|
||||
- [ ] Tests pass
|
||||
- [ ] Service starts without errors
|
||||
|
||||
### Post-build
|
||||
- [ ] API accessible at http://localhost:8082
|
||||
- [ ] MeTube connection established
|
||||
- [ ] Database operations working
|
||||
- [ ] Scheduler running
|
||||
- [ ] Logs rotating properly
|
||||
|
||||
## 🎯 Next Steps
|
||||
|
||||
After successful build:
|
||||
1. **Test the API** using the Quick Start Guide
|
||||
2. **Set up monitoring** with health checks
|
||||
3. **Configure automatic startup** with systemd or Docker
|
||||
4. **Implement UI** (see UI_INTEGRATION.md)
|
||||
5. **Set up logging** and monitoring
|
||||
|
||||
## 📚 Related Documentation
|
||||
- [QUICK_START_GUIDE.md](QUICK_START_GUIDE.md) - Getting started quickly
|
||||
- [TESTING_GUIDE.md](TESTING_GUIDE.md) - Comprehensive testing
|
||||
- [UI_INTEGRATION.md](UI_INTEGRATION.md) - UI implementation options
|
||||
- [ADVANCED_CONFIG.md](ADVANCED_CONFIG.md) - Advanced configuration options
|
||||
|
|
@ -0,0 +1,218 @@
|
|||
# Playlist Monitor Service - Complete Getting Started Guide
|
||||
|
||||
## 🎯 What You Have Now
|
||||
|
||||
### ✅ **Fully Implemented Backend**
|
||||
- **Complete REST API** with 20+ endpoints
|
||||
- **Database models** with SQLite/PostgreSQL support
|
||||
- **MeTube integration** via HTTP API + WebSocket
|
||||
- **Automated playlist monitoring** with configurable intervals
|
||||
- **Video tracking** with status management
|
||||
- **Docker support** with multi-stage builds
|
||||
- **Comprehensive testing** with pytest
|
||||
|
||||
### ❌ **Missing: User Interface**
|
||||
The service currently only provides a **REST API** with Swagger documentation at `http://localhost:8082/docs`. There is **no web UI** for visual management.
|
||||
|
||||
## 🚀 Quick Start (Choose Your Path)
|
||||
|
||||
### Path 1: API-Only (Immediate Use)
|
||||
```bash
|
||||
# Start the service
|
||||
cd /root/workspace/tubewatch/playlist-monitor
|
||||
uv run python -m app.main
|
||||
|
||||
# Access API documentation
|
||||
open http://localhost:8082/docs
|
||||
|
||||
# Test with curl
|
||||
curl -X POST http://localhost:8082/api/playlists \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"url": "https://www.youtube.com/playlist?list=PLrAXtmErZgOeiKm4sgNOknGvNjby9efdf",
|
||||
"check_interval": 60,
|
||||
"quality": "best",
|
||||
"format": "mp4",
|
||||
"folder": "kurzgesagt"
|
||||
}'
|
||||
```
|
||||
|
||||
### Path 2: Add Simple UI (30 minutes)
|
||||
```bash
|
||||
# Create basic HTML dashboard
|
||||
cd /root/workspace/tubewatch/playlist-monitor
|
||||
mkdir ui
|
||||
cp templates/simple-dashboard.html ui/index.html
|
||||
|
||||
# Modify backend to serve UI
|
||||
echo "app.mount('/ui', StaticFiles(directory='ui', html=True), name='ui')" >> app/main.py
|
||||
|
||||
# Restart and access UI
|
||||
uv run python -m app.main
|
||||
open http://localhost:8082/ui
|
||||
```
|
||||
|
||||
### Path 3: Full React App (2-4 hours)
|
||||
```bash
|
||||
# Create React application
|
||||
cd /root/workspace/tubewatch
|
||||
npx create-react-app playlist-monitor-ui --template typescript
|
||||
cd playlist-monitor-ui
|
||||
|
||||
# Install UI libraries
|
||||
npm install @mui/material @emotion/react @emotion/styled axios react-query
|
||||
|
||||
# Follow UI_INTEGRATION.md for complete setup
|
||||
```
|
||||
|
||||
## 📚 Documentation You Have
|
||||
|
||||
### Essential Guides
|
||||
- **[QUICK_START_GUIDE.md](QUICK_START_GUIDE.md)** - Get started in 5 minutes
|
||||
- **[BUILD_GUIDE.md](BUILD_GUIDE.md)** - Comprehensive build instructions
|
||||
- **[TESTING_GUIDE.md](TESTING_GUIDE.md)** - Complete testing procedures
|
||||
- **[UI_INTEGRATION.md](UI_INTEGRATION.md)** - UI implementation options
|
||||
|
||||
### Reference Documents
|
||||
- **[IMPLEMENTATION_STATUS.md](IMPLEMENTATION_STATUS.md)** - What was built
|
||||
- **[PLAYLIST_MONITOR_ARCHITECTURE.md](PLAYLIST_MONITOR_ARCHITECTURE.md)** - Original architecture
|
||||
|
||||
## 🎯 What You Can Do Right Now
|
||||
|
||||
### 1. Start Using the API (No UI Needed)
|
||||
```bash
|
||||
# Check system status
|
||||
curl http://localhost:8082/api/status
|
||||
|
||||
# Add a playlist
|
||||
curl -X POST http://localhost:8082/api/playlists \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"url": "https://www.youtube.com/playlist?list=TEST123"}'
|
||||
|
||||
# Monitor downloads
|
||||
curl http://localhost:8082/api/playlists/{id}/videos
|
||||
```
|
||||
|
||||
### 2. Test the Service
|
||||
```bash
|
||||
# Run all tests
|
||||
uv run pytest tests/ -v
|
||||
|
||||
# Run specific test
|
||||
uv run pytest tests/unit/test_models.py -v
|
||||
|
||||
# Check coverage
|
||||
uv run pytest --cov=app --cov-report=html
|
||||
```
|
||||
|
||||
### 3. Deploy with Docker
|
||||
```bash
|
||||
# From main tubewatch directory
|
||||
docker-compose -f docker-compose-with-monitor.yml up -d
|
||||
|
||||
# Check status
|
||||
docker-compose -f docker-compose-with-monitor.yml ps
|
||||
```
|
||||
|
||||
## 🎨 UI Implementation Priority
|
||||
|
||||
Since you mentioned the UI is missing, here are your options ranked by effort:
|
||||
|
||||
### 🟢 **15 Minutes: Basic HTML Dashboard**
|
||||
- Simple Bootstrap-based interface
|
||||
- List playlists, show status
|
||||
- Add/delete operations
|
||||
- **Good for**: Quick setup, basic needs
|
||||
|
||||
### 🟡 **2-4 Hours: React App**
|
||||
- Modern Material-UI interface
|
||||
- Real-time updates, charts
|
||||
- Full CRUD operations
|
||||
- **Good for**: Production use, modern UX
|
||||
|
||||
### 🔴 **8+ Hours: Extend MeTube Angular**
|
||||
- Integrate with existing MeTube UI
|
||||
- Consistent styling and navigation
|
||||
- Most complex but seamless
|
||||
- **Good for**: Existing MeTube deployments
|
||||
|
||||
## 🚀 Recommended Next Steps
|
||||
|
||||
### Immediate (Next 30 minutes)
|
||||
1. **Start the service**: `uv run python -m app.main`
|
||||
2. **Test the API**: Visit http://localhost:8082/docs
|
||||
3. **Add a playlist**: Use the interactive API docs
|
||||
4. **Monitor progress**: Check logs and status endpoints
|
||||
|
||||
### Short Term (Next few hours)
|
||||
1. **Choose UI option** based on your needs
|
||||
2. **Implement basic interface** following UI_INTEGRATION.md
|
||||
3. **Test end-to-end workflow** with real playlists
|
||||
4. **Set up monitoring** and health checks
|
||||
|
||||
### Long Term (Next few days)
|
||||
1. **Deploy to production** with Docker
|
||||
2. **Set up automated monitoring** and alerts
|
||||
3. **Add authentication** if needed
|
||||
4. **Optimize performance** based on usage
|
||||
|
||||
## 🔧 Common First-Time Setup Issues
|
||||
|
||||
### "MeTube connection failed"
|
||||
- Ensure MeTube is running on port 8081
|
||||
- Check MeTube logs: `docker logs metube`
|
||||
- Verify network connectivity
|
||||
|
||||
### "Port 8082 already in use"
|
||||
- Find process: `lsof -i :8082`
|
||||
- Kill process or change port in .env
|
||||
|
||||
### "Database locked"
|
||||
- Remove lock file: `rm data/*.db-journal`
|
||||
- Check for zombie processes
|
||||
|
||||
### "Python version too old"
|
||||
- Install Python 3.13+ or use Docker
|
||||
- Use pyenv for version management
|
||||
|
||||
## 📞 Getting Help
|
||||
|
||||
### Documentation
|
||||
- **API Reference**: http://localhost:8082/docs (when running)
|
||||
- **Architecture**: See PLAYLIST_MONITOR_ARCHITECTURE.md
|
||||
- **Build Issues**: See BUILD_GUIDE.md troubleshooting section
|
||||
|
||||
### Debugging
|
||||
- Check logs: `tail -f logs/playlist-monitor.log`
|
||||
- Health check: `curl http://localhost:8082/health`
|
||||
- System status: `curl http://localhost:8082/api/status`
|
||||
|
||||
### Community
|
||||
- Check existing issues in the repository
|
||||
- Create detailed bug reports with logs
|
||||
- Include system information and reproduction steps
|
||||
|
||||
## 🎉 Success Metrics
|
||||
|
||||
You'll know everything is working when you can:
|
||||
|
||||
1. ✅ **Start the service** without errors
|
||||
2. ✅ **Access API docs** at http://localhost:8082/docs
|
||||
3. ✅ **Add a playlist** via API or UI
|
||||
4. ✅ **See videos detected** from the playlist
|
||||
5. ✅ **Monitor download progress** in real-time
|
||||
6. ✅ **Manage video status** (skip, reset, etc.)
|
||||
|
||||
## 🎯 Final Recommendation
|
||||
|
||||
**Start with the API-only approach** to verify everything works, then add a UI based on your comfort level:
|
||||
|
||||
- **API-only**: Great for automation, scripts, or integration with other tools
|
||||
- **Simple HTML**: Perfect for personal use or quick setup
|
||||
- **React App**: Best for production deployments and user-friendly experience
|
||||
|
||||
The backend is **production-ready** and thoroughly tested. Focus your effort on the UI implementation based on your specific needs! 🚀
|
||||
|
||||
---
|
||||
|
||||
**Next Step**: Choose your UI path and follow the relevant guide in [UI_INTEGRATION.md](UI_INTEGRATION.md)!
|
||||
|
|
@ -0,0 +1,208 @@
|
|||
# Playlist Monitor Service - Quick Start Guide
|
||||
|
||||
## 🚀 Quick Start (5 minutes)
|
||||
|
||||
### Prerequisites
|
||||
- Python 3.13+ installed
|
||||
- MeTube running (or available to start)
|
||||
- Basic command line knowledge
|
||||
|
||||
### Option 1: Docker (Recommended)
|
||||
```bash
|
||||
# From the main tubewatch directory
|
||||
cd /root/workspace/tubewatch
|
||||
|
||||
# Start both MeTube and Playlist Monitor
|
||||
docker-compose -f docker-compose-with-monitor.yml up -d
|
||||
|
||||
# Check if services are running
|
||||
docker-compose -f docker-compose-with-monitor.yml ps
|
||||
|
||||
# View logs
|
||||
docker-compose -f docker-compose-with-monitor.yml logs -f playlist-monitor
|
||||
```
|
||||
|
||||
### Option 2: Manual Installation
|
||||
```bash
|
||||
# Navigate to playlist monitor directory
|
||||
cd /root/workspace/tubewatch/playlist-monitor
|
||||
|
||||
# Install uv (if not already installed)
|
||||
curl -LsSf https://astral.sh/uv/install.sh | sh
|
||||
source ~/.cargo/env
|
||||
|
||||
# Install dependencies
|
||||
uv sync
|
||||
|
||||
# Copy environment file
|
||||
cp .env.example .env
|
||||
|
||||
# Start the service
|
||||
uv run python -m app.main
|
||||
```
|
||||
|
||||
### Access the Services
|
||||
- **Playlist Monitor API**: http://localhost:8082/docs
|
||||
- **MeTube**: http://localhost:8081
|
||||
- **API Documentation**: http://localhost:8082/docs
|
||||
|
||||
## 🎯 Your First Playlist
|
||||
|
||||
### Using the API (Interactive)
|
||||
1. Open http://localhost:8082/docs
|
||||
2. Click on `POST /api/playlists`
|
||||
3. Click "Try it out"
|
||||
4. Enter this data:
|
||||
```json
|
||||
{
|
||||
"url": "https://www.youtube.com/playlist?list=PLrAXtmErZgOeiKm4sgNOknGvNjby9efdf",
|
||||
"check_interval": 60,
|
||||
"quality": "best",
|
||||
"format": "mp4",
|
||||
"folder": "kurzgesagt",
|
||||
"enabled": true
|
||||
}
|
||||
```
|
||||
5. Click "Execute"
|
||||
|
||||
### Using curl
|
||||
```bash
|
||||
# Add a playlist
|
||||
curl -X POST http://localhost:8082/api/playlists \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"url": "https://www.youtube.com/playlist?list=PLrAXtmErZgOeiKm4sgNOknGvNjby9efdf",
|
||||
"check_interval": 60,
|
||||
"quality": "best",
|
||||
"format": "mp4",
|
||||
"folder": "kurzgesagt"
|
||||
}'
|
||||
|
||||
# List playlists
|
||||
curl http://localhost:8082/api/playlists
|
||||
|
||||
# Check system status
|
||||
curl http://localhost:8082/api/status
|
||||
```
|
||||
|
||||
## 📊 Monitor Your Downloads
|
||||
|
||||
### Check Status
|
||||
```bash
|
||||
# System status
|
||||
curl http://localhost:8082/api/status
|
||||
|
||||
# List playlists with stats
|
||||
curl http://localhost:8082/api/playlists
|
||||
|
||||
# Get specific playlist details
|
||||
curl http://localhost:8082/api/playlists/{playlist_id}
|
||||
```
|
||||
|
||||
### View Videos
|
||||
```bash
|
||||
# List videos for a playlist
|
||||
curl http://localhost:8082/api/playlists/{playlist_id}/videos
|
||||
|
||||
# Get specific video details
|
||||
curl http://localhost:8082/api/videos/{video_id}
|
||||
```
|
||||
|
||||
## 🛠️ Troubleshooting
|
||||
|
||||
### Service Won't Start
|
||||
```bash
|
||||
# Check if port 8082 is available
|
||||
netstat -tulpn | grep 8082
|
||||
|
||||
# Check logs
|
||||
tail -f logs/playlist-monitor.log
|
||||
|
||||
# Test configuration
|
||||
uv run python -c "from app.core.config import settings; print(settings.METUBE_URL)"
|
||||
```
|
||||
|
||||
### MeTube Connection Issues
|
||||
```bash
|
||||
# Test MeTube connectivity
|
||||
curl http://localhost:8081/info
|
||||
|
||||
# Check MeTube logs
|
||||
docker logs metube
|
||||
|
||||
# Verify MeTube URL in .env file
|
||||
grep METUBE_URL .env
|
||||
```
|
||||
|
||||
### Database Issues
|
||||
```bash
|
||||
# Check database file
|
||||
ls -la data/playlists.db
|
||||
|
||||
# Reset database (WARNING: deletes all data)
|
||||
rm data/playlists.db
|
||||
# Then restart the service
|
||||
```
|
||||
|
||||
## 🎮 Common Operations
|
||||
|
||||
### Add Multiple Playlists
|
||||
```bash
|
||||
# YouTube channel uploads playlist
|
||||
curl -X POST http://localhost:8082/api/playlists \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"url": "https://www.youtube.com/playlist?list=UUX6OQ3DkcsbYNE6H8uQQuVA",
|
||||
"check_interval": 30,
|
||||
"folder": "tech-channels"
|
||||
}'
|
||||
|
||||
# Specific playlist
|
||||
curl -X POST http://localhost:8082/api/playlists \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"url": "https://www.youtube.com/playlist?list=PLQVvvaa0QuDfpEcGUM6ogsbrlWtqpS5-1",
|
||||
"check_interval": 120,
|
||||
"folder": "python-tutorials"
|
||||
}'
|
||||
```
|
||||
|
||||
### Manual Playlist Check
|
||||
```bash
|
||||
# Force check a playlist
|
||||
curl -X POST http://localhost:8082/api/playlists/{playlist_id}/check?force=true
|
||||
```
|
||||
|
||||
### Manage Videos
|
||||
```bash
|
||||
# Mark video as moved
|
||||
curl -X POST http://localhost:8082/api/videos/{video_id}/file-moved \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"location_note": "Moved to /mnt/nas/videos/"}'
|
||||
|
||||
# Reset failed video
|
||||
curl -X POST http://localhost:8082/api/videos/{video_id}/reset
|
||||
```
|
||||
|
||||
## 📈 Next Steps
|
||||
|
||||
1. **Set up a web UI** (see BUILD_GUIDE.md for UI options)
|
||||
2. **Configure automatic startup** with systemd or Docker
|
||||
3. **Set up monitoring** with health checks
|
||||
4. **Customize settings** for your needs
|
||||
5. **Add authentication** if needed (see ADVANCED_CONFIG.md)
|
||||
|
||||
## 🆘 Need Help?
|
||||
|
||||
- Check the full documentation: [README.md](README.md)
|
||||
- View API docs: http://localhost:8082/docs
|
||||
- Check logs: `tail -f logs/playlist-monitor.log`
|
||||
- Run health check: `curl http://localhost:8082/health`
|
||||
|
||||
## 🚀 Going Further
|
||||
|
||||
See these guides for advanced usage:
|
||||
- [BUILD_GUIDE.md](BUILD_GUIDE.md) - Detailed build instructions
|
||||
- [TESTING_GUIDE.md](TESTING_GUIDE.md) - Comprehensive testing
|
||||
- [UI_INTEGRATION.md](UI_INTEGRATION.md) - UI implementation options
|
||||
- [ADVANCED_CONFIG.md](ADVANCED_CONFIG.md) - Advanced configuration
|
||||
|
|
@ -0,0 +1,921 @@
|
|||
# Playlist Monitor Service - Comprehensive Testing Guide
|
||||
|
||||
## 📋 Table of Contents
|
||||
1. [Testing Overview](#testing-overview)
|
||||
2. [Test Environment Setup](#test-environment-setup)
|
||||
3. [Unit Tests](#unit-tests)
|
||||
4. [Integration Tests](#integration-tests)
|
||||
5. [API Tests](#api-tests)
|
||||
6. [End-to-End Tests](#end-to-end-tests)
|
||||
7. [Performance Tests](#performance-tests)
|
||||
8. [Manual Testing](#manual-testing)
|
||||
9. [CI/CD Integration](#cicd-integration)
|
||||
10. [Test Data Management](#test-data-management)
|
||||
|
||||
## 🔍 Testing Overview
|
||||
|
||||
### Test Pyramid
|
||||
```
|
||||
🎯 E2E Tests (Few)
|
||||
↑
|
||||
🔌 Integration Tests (Some)
|
||||
↑
|
||||
⚡ Unit Tests (Many)
|
||||
```
|
||||
|
||||
### Test Categories
|
||||
- **Unit Tests**: Individual components (models, services, utils)
|
||||
- **Integration Tests**: Component interactions (database, API, external services)
|
||||
- **API Tests**: REST endpoint validation
|
||||
- **E2E Tests**: Full user workflows
|
||||
- **Performance Tests**: Load and stress testing
|
||||
|
||||
## 🛠️ Test Environment Setup
|
||||
|
||||
### 1. Test Dependencies
|
||||
```bash
|
||||
# Install test dependencies
|
||||
uv sync --extra dev
|
||||
|
||||
# Or install manually
|
||||
pip install pytest pytest-asyncio pytest-cov pytest-mock httpx
|
||||
```
|
||||
|
||||
### 2. Test Configuration
|
||||
```bash
|
||||
# Create test environment
|
||||
cp .env.example .env.test
|
||||
|
||||
# Update test settings
|
||||
nano .env.test
|
||||
```
|
||||
|
||||
**Test-specific settings:**
|
||||
```env
|
||||
# Test database (in-memory SQLite)
|
||||
DATABASE_URL=sqlite:///:memory:
|
||||
|
||||
# Test logging
|
||||
LOG_LEVEL=DEBUG
|
||||
LOG_FILE=logs/test.log
|
||||
|
||||
# Test MeTube (mock or test instance)
|
||||
METUBE_URL=http://localhost:8081
|
||||
|
||||
# Test mode
|
||||
TESTING=true
|
||||
```
|
||||
|
||||
### 3. Test Directory Structure
|
||||
```
|
||||
tests/
|
||||
├── unit/ # Unit tests
|
||||
│ ├── test_config.py
|
||||
│ ├── test_models.py
|
||||
│ └── test_services.py
|
||||
├── integration/ # Integration tests
|
||||
│ ├── test_database.py
|
||||
│ ├── test_api.py
|
||||
│ └── test_metube_client.py
|
||||
├── e2e/ # End-to-end tests
|
||||
│ ├── test_workflows.py
|
||||
│ └── test_scenarios.py
|
||||
├── fixtures/ # Test data
|
||||
│ ├── playlists.json
|
||||
│ └── videos.json
|
||||
└── conftest.py # Pytest configuration
|
||||
```
|
||||
|
||||
## ⚡ Unit Tests
|
||||
|
||||
### Running Unit Tests
|
||||
```bash
|
||||
# Run all unit tests
|
||||
uv run pytest tests/unit/ -v
|
||||
|
||||
# Run specific test file
|
||||
uv run pytest tests/unit/test_models.py -v
|
||||
|
||||
# Run with coverage
|
||||
uv run pytest tests/unit/ --cov=app --cov-report=html
|
||||
|
||||
# Run specific test
|
||||
uv run pytest tests/unit/test_models.py::test_playlist_creation -v
|
||||
```
|
||||
|
||||
### Model Tests (tests/unit/test_models.py)
|
||||
```python
|
||||
import pytest
|
||||
from datetime import datetime, timedelta
|
||||
from app.models.playlist import PlaylistSubscription
|
||||
from app.models.video import VideoRecord, VideoStatus
|
||||
|
||||
class TestPlaylistModel:
|
||||
def test_playlist_creation(self, test_db):
|
||||
"""Test playlist creation and validation"""
|
||||
playlist = PlaylistSubscription(
|
||||
url="https://www.youtube.com/playlist?list=TEST123",
|
||||
title="Test Playlist",
|
||||
check_interval=60
|
||||
)
|
||||
|
||||
assert playlist.url == "https://www.youtube.com/playlist?list=TEST123"
|
||||
assert playlist.title == "Test Playlist"
|
||||
assert playlist.check_interval == 60
|
||||
assert playlist.enabled is True
|
||||
|
||||
def test_playlist_should_check_logic(self):
|
||||
"""Test playlist check logic"""
|
||||
playlist = PlaylistSubscription(check_interval=60)
|
||||
|
||||
# Should check if never checked
|
||||
assert playlist.should_check() is True
|
||||
|
||||
# Should not check if recently checked
|
||||
playlist.last_checked = datetime.utcnow() - timedelta(minutes=30)
|
||||
assert playlist.should_check() is False
|
||||
|
||||
# Should check if interval passed
|
||||
playlist.last_checked = datetime.utcnow() - timedelta(minutes=61)
|
||||
assert playlist.should_check() is True
|
||||
|
||||
class TestVideoModel:
|
||||
def test_video_status_transitions(self):
|
||||
"""Test video status management"""
|
||||
video = VideoRecord(status=VideoStatus.PENDING)
|
||||
|
||||
# Test downloading
|
||||
video.mark_as_downloading("metube_123")
|
||||
assert video.status == VideoStatus.DOWNLOADING
|
||||
assert video.metube_download_id == "metube_123"
|
||||
|
||||
# Test completion
|
||||
video.mark_as_completed("video.mp4")
|
||||
assert video.status == VideoStatus.COMPLETED
|
||||
assert video.original_filename == "video.mp4"
|
||||
```
|
||||
|
||||
### Service Tests (tests/unit/test_services.py)
|
||||
```python
|
||||
import pytest
|
||||
from unittest.mock import Mock, AsyncMock
|
||||
from app.services.playlist_service import PlaylistService
|
||||
from app.services.metube_client import MeTubeClient
|
||||
|
||||
class TestPlaylistService:
|
||||
@pytest.fixture
|
||||
def service(self, test_db):
|
||||
return PlaylistService(test_db)
|
||||
|
||||
@pytest.fixture
|
||||
def mock_yt_dlp(self, monkeypatch):
|
||||
"""Mock yt-dlp for testing"""
|
||||
mock = Mock()
|
||||
mock.extract_info.return_value = {
|
||||
"title": "Test Playlist",
|
||||
"entries": [
|
||||
{"id": "video1", "title": "Video 1", "playlist_index": 1},
|
||||
{"id": "video2", "title": "Video 2", "playlist_index": 2}
|
||||
]
|
||||
}
|
||||
monkeypatch.setattr("yt_dlp.YoutubeDL", lambda x: mock)
|
||||
return mock
|
||||
|
||||
async def test_add_playlist_success(self, service, mock_yt_dlp):
|
||||
"""Test successful playlist addition"""
|
||||
playlist = await service.add_playlist(
|
||||
url="https://www.youtube.com/playlist?list=TEST123",
|
||||
title="Test Playlist"
|
||||
)
|
||||
|
||||
assert playlist.title == "Test Playlist"
|
||||
assert playlist.check_interval == 60
|
||||
assert len(playlist.videos) == 2
|
||||
```
|
||||
|
||||
### Configuration Tests (tests/unit/test_config.py)
|
||||
```python
|
||||
import pytest
|
||||
from app.core.config import Settings
|
||||
|
||||
class TestConfiguration:
|
||||
def test_default_values(self):
|
||||
"""Test default configuration values"""
|
||||
settings = Settings()
|
||||
assert settings.HOST == "0.0.0.0"
|
||||
assert settings.PORT == 8082
|
||||
assert settings.METUBE_URL == "http://localhost:8081"
|
||||
|
||||
def test_validation(self):
|
||||
"""Test configuration validation"""
|
||||
with pytest.raises(ValueError):
|
||||
Settings(DEFAULT_CHECK_INTERVAL=0)
|
||||
|
||||
with pytest.raises(ValueError):
|
||||
Settings(MAX_CONCURRENT_DOWNLOADS=15)
|
||||
```
|
||||
|
||||
## 🔌 Integration Tests
|
||||
|
||||
### Database Integration (tests/integration/test_database.py)
|
||||
```python
|
||||
import pytest
|
||||
from sqlalchemy import create_engine
|
||||
from sqlalchemy.orm import sessionmaker
|
||||
from app.core.database import Base, get_db
|
||||
from app.models.playlist import PlaylistSubscription
|
||||
|
||||
@pytest.fixture
|
||||
def test_engine():
|
||||
"""Create test database engine"""
|
||||
engine = create_engine("sqlite:///:memory:", echo=False)
|
||||
Base.metadata.create_all(bind=engine)
|
||||
return engine
|
||||
|
||||
class TestDatabaseIntegration:
|
||||
def test_playlist_crud_operations(self, test_engine):
|
||||
"""Test database CRUD operations"""
|
||||
SessionLocal = sessionmaker(bind=test_engine)
|
||||
db = SessionLocal()
|
||||
|
||||
# Create
|
||||
playlist = PlaylistSubscription(
|
||||
url="https://www.youtube.com/playlist?list=TEST123",
|
||||
title="Test Playlist"
|
||||
)
|
||||
db.add(playlist)
|
||||
db.commit()
|
||||
|
||||
# Read
|
||||
retrieved = db.query(PlaylistSubscription).first()
|
||||
assert retrieved.title == "Test Playlist"
|
||||
|
||||
# Update
|
||||
retrieved.title = "Updated Playlist"
|
||||
db.commit()
|
||||
assert db.query(PlaylistSubscription).first().title == "Updated Playlist"
|
||||
|
||||
# Delete
|
||||
db.delete(retrieved)
|
||||
db.commit()
|
||||
assert db.query(PlaylistSubscription).count() == 0
|
||||
|
||||
db.close()
|
||||
```
|
||||
|
||||
### MeTube Client Integration (tests/integration/test_metube_client.py)
|
||||
```python
|
||||
import pytest
|
||||
import respx
|
||||
from httpx import Response
|
||||
from app.services.metube_client import MeTubeClient
|
||||
|
||||
@pytest.mark.asyncio
|
||||
class TestMeTubeClientIntegration:
|
||||
@respx.mock
|
||||
async def test_add_download_success(self):
|
||||
"""Test successful download addition"""
|
||||
# Mock MeTube API
|
||||
route = respx.post("http://localhost:8081/add").mock(
|
||||
return_value=Response(200, json={"id": "download_123", "status": "pending"})
|
||||
)
|
||||
|
||||
client = MeTubeClient("http://localhost:8081")
|
||||
await client.connect()
|
||||
|
||||
result = await client.add_download(
|
||||
url="https://www.youtube.com/watch?v=TEST123",
|
||||
quality="best",
|
||||
format="mp4"
|
||||
)
|
||||
|
||||
assert result["id"] == "download_123"
|
||||
assert result["status"] == "pending"
|
||||
assert route.called
|
||||
|
||||
await client.disconnect()
|
||||
```
|
||||
|
||||
## 🌐 API Tests
|
||||
|
||||
### API Integration (tests/integration/test_api.py)
|
||||
```python
|
||||
import pytest
|
||||
from fastapi.testclient import TestClient
|
||||
from app.main import app
|
||||
|
||||
class TestAPIIntegration:
|
||||
@pytest.fixture
|
||||
def client(self):
|
||||
return TestClient(app)
|
||||
|
||||
def test_health_endpoint(self, client):
|
||||
"""Test health check endpoint"""
|
||||
response = client.get("/health")
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
assert data["status"] == "healthy"
|
||||
|
||||
def test_playlist_crud(self, client):
|
||||
"""Test playlist CRUD operations"""
|
||||
# Create playlist
|
||||
response = client.post("/api/playlists", json={
|
||||
"url": "https://www.youtube.com/playlist?list=TEST123",
|
||||
"title": "Test Playlist",
|
||||
"check_interval": 60
|
||||
})
|
||||
assert response.status_code == 201
|
||||
playlist_id = response.json()["id"]
|
||||
|
||||
# Read playlist
|
||||
response = client.get(f"/api/playlists/{playlist_id}")
|
||||
assert response.status_code == 200
|
||||
assert response.json()["title"] == "Test Playlist"
|
||||
|
||||
# Update playlist
|
||||
response = client.put(f"/api/playlists/{playlist_id}", json={
|
||||
"title": "Updated Playlist"
|
||||
})
|
||||
assert response.status_code == 200
|
||||
assert response.json()["title"] == "Updated Playlist"
|
||||
|
||||
# Delete playlist
|
||||
response = client.delete(f"/api/playlists/{playlist_id}")
|
||||
assert response.status_code == 204
|
||||
|
||||
def test_invalid_playlist_url(self, client):
|
||||
"""Test validation of invalid playlist URLs"""
|
||||
response = client.post("/api/playlists", json={
|
||||
"url": "https://invalid-url.com",
|
||||
"title": "Invalid Playlist"
|
||||
})
|
||||
assert response.status_code == 400
|
||||
assert "Invalid YouTube playlist URL" in response.json()["detail"]
|
||||
```
|
||||
|
||||
### Async API Tests (tests/integration/test_api_async.py)
|
||||
```python
|
||||
import pytest
|
||||
import asyncio
|
||||
from httpx import AsyncClient
|
||||
from app.main import app
|
||||
|
||||
@pytest.mark.asyncio
|
||||
class TestAsyncAPI:
|
||||
async def test_websocket_connection(self):
|
||||
"""Test WebSocket connection and events"""
|
||||
# This would require a mock WebSocket server
|
||||
pass
|
||||
|
||||
async def test_async_playlist_check(self):
|
||||
"""Test async playlist checking"""
|
||||
async with AsyncClient(app=app, base_url="http://test") as client:
|
||||
# Create playlist
|
||||
response = await client.post("/api/playlists", json={
|
||||
"url": "https://www.youtube.com/playlist?list=TEST123",
|
||||
"title": "Test Playlist"
|
||||
})
|
||||
playlist_id = response.json()["id"]
|
||||
|
||||
# Trigger check
|
||||
response = await client.post(f"/api/playlists/{playlist_id}/check")
|
||||
assert response.status_code == 200
|
||||
assert response.json()["status"] == "ok"
|
||||
```
|
||||
|
||||
## 🎯 End-to-End Tests
|
||||
|
||||
### Workflow Tests (tests/e2e/test_workflows.py)
|
||||
```python
|
||||
import pytest
|
||||
import asyncio
|
||||
from httpx import AsyncClient
|
||||
from app.main import app
|
||||
|
||||
@pytest.mark.asyncio
|
||||
class TestE2EWorkflows:
|
||||
async def test_complete_playlist_workflow(self):
|
||||
"""Test complete playlist lifecycle"""
|
||||
async with AsyncClient(app=app, base_url="http://test") as client:
|
||||
# 1. Create playlist
|
||||
response = await client.post("/api/playlists", json={
|
||||
"url": "https://www.youtube.com/playlist?list=PLrAXtmErZgOeiKm4sgNOknGvNjby9efdf",
|
||||
"title": "Kurzgesagt Playlist",
|
||||
"check_interval": 60,
|
||||
"folder": "kurzgesagt"
|
||||
})
|
||||
assert response.status_code == 201
|
||||
playlist_id = response.json()["id"]
|
||||
|
||||
# 2. Verify playlist was created
|
||||
response = await client.get(f"/api/playlists/{playlist_id}")
|
||||
assert response.status_code == 200
|
||||
playlist = response.json()
|
||||
assert playlist["title"] == "Kurzgesagt Playlist"
|
||||
|
||||
# 3. Check playlist (manual trigger)
|
||||
response = await client.post(f"/api/playlists/{playlist_id}/check")
|
||||
assert response.status_code == 200
|
||||
|
||||
# 4. Verify videos were created
|
||||
response = await client.get(f"/api/playlists/{playlist_id}/videos")
|
||||
assert response.status_code == 200
|
||||
videos = response.json()
|
||||
assert len(videos) > 0
|
||||
|
||||
# 5. Test video operations
|
||||
video_id = videos[0]["id"]
|
||||
|
||||
# Skip a video
|
||||
response = await client.post(f"/api/videos/{video_id}/skip")
|
||||
assert response.status_code == 200
|
||||
assert response.json()["video"]["status"] == "SKIPPED"
|
||||
|
||||
# Reset video
|
||||
response = await client.post(f"/api/videos/{video_id}/reset")
|
||||
assert response.status_code == 200
|
||||
assert response.json()["video"]["status"] == "PENDING"
|
||||
|
||||
# 6. Check system status
|
||||
response = await client.get("/api/status")
|
||||
assert response.status_code == 200
|
||||
status = response.json()
|
||||
assert status["total_playlists"] >= 1
|
||||
assert status["total_videos"] >= 1
|
||||
|
||||
# 7. Delete playlist
|
||||
response = await client.delete(f"/api/playlists/{playlist_id}")
|
||||
assert response.status_code == 204
|
||||
|
||||
async def test_error_handling_workflow(self):
|
||||
"""Test error handling scenarios"""
|
||||
async with AsyncClient(app=app, base_url="http://test") as client:
|
||||
# Test invalid playlist URL
|
||||
response = await client.post("/api/playlists", json={
|
||||
"url": "https://invalid-url.com",
|
||||
"title": "Invalid Playlist"
|
||||
})
|
||||
assert response.status_code == 400
|
||||
|
||||
# Test non-existent playlist
|
||||
response = await client.get("/api/playlists/non-existent-id")
|
||||
assert response.status_code == 404
|
||||
|
||||
# Test invalid video operations
|
||||
response = await client.post("/api/videos/non-existent/download")
|
||||
assert response.status_code == 404
|
||||
```
|
||||
|
||||
## ⚡ Performance Tests
|
||||
|
||||
### Load Testing (tests/performance/test_load.py)
|
||||
```python
|
||||
import asyncio
|
||||
import time
|
||||
import aiohttp
|
||||
import pytest
|
||||
|
||||
class TestLoadPerformance:
|
||||
@pytest.mark.performance
|
||||
async def test_api_response_time(self):
|
||||
"""Test API response times"""
|
||||
async with aiohttp.ClientSession() as session:
|
||||
start_time = time.time()
|
||||
|
||||
for i in range(100):
|
||||
async with session.get("http://localhost:8082/api/status") as response:
|
||||
assert response.status == 200
|
||||
|
||||
elapsed = time.time() - start_time
|
||||
avg_response_time = elapsed / 100
|
||||
|
||||
# Assert average response time is under 100ms
|
||||
assert avg_response_time < 0.1
|
||||
|
||||
@pytest.mark.performance
|
||||
async def test_database_performance(self):
|
||||
"""Test database query performance"""
|
||||
from app.core.database import SessionLocal
|
||||
from app.models.playlist import PlaylistSubscription
|
||||
|
||||
db = SessionLocal()
|
||||
|
||||
# Create test data
|
||||
for i in range(1000):
|
||||
playlist = PlaylistSubscription(
|
||||
url=f"https://www.youtube.com/playlist?list=TEST{i}",
|
||||
title=f"Test Playlist {i}"
|
||||
)
|
||||
db.add(playlist)
|
||||
|
||||
db.commit()
|
||||
|
||||
# Measure query performance
|
||||
start_time = time.time()
|
||||
playlists = db.query(PlaylistSubscription).all()
|
||||
elapsed = time.time() - start_time
|
||||
|
||||
assert len(playlists) == 1000
|
||||
assert elapsed < 1.0 # Should complete in under 1 second
|
||||
|
||||
db.close()
|
||||
```
|
||||
|
||||
### Stress Testing (tests/performance/test_stress.py)
|
||||
```python
|
||||
import asyncio
|
||||
import pytest
|
||||
from httpx import AsyncClient
|
||||
from app.main import app
|
||||
|
||||
@pytest.mark.stress
|
||||
class TestStressScenarios:
|
||||
async def test_concurrent_playlist_creation(self):
|
||||
"""Test concurrent playlist creation"""
|
||||
async with AsyncClient(app=app, base_url="http://test") as client:
|
||||
# Create 50 playlists concurrently
|
||||
tasks = []
|
||||
for i in range(50):
|
||||
task = client.post("/api/playlists", json={
|
||||
"url": f"https://www.youtube.com/playlist?list=TEST{i}",
|
||||
"title": f"Stress Test Playlist {i}"
|
||||
})
|
||||
tasks.append(task)
|
||||
|
||||
responses = await asyncio.gather(*tasks)
|
||||
|
||||
# All should succeed
|
||||
for response in responses:
|
||||
assert response.status_code == 201
|
||||
|
||||
# Verify all were created
|
||||
response = await client.get("/api/playlists")
|
||||
assert response.status_code == 200
|
||||
assert len(response.json()) >= 50
|
||||
```
|
||||
|
||||
## 🔧 Manual Testing
|
||||
|
||||
### Manual Test Checklist
|
||||
|
||||
#### Basic Functionality
|
||||
- [ ] Service starts without errors
|
||||
- [ ] Health endpoint returns 200
|
||||
- [ ] API documentation accessible at /docs
|
||||
- [ ] Database connection successful
|
||||
|
||||
#### Playlist Operations
|
||||
- [ ] Create playlist with valid YouTube URL
|
||||
- [ ] Create playlist with invalid URL (should fail)
|
||||
- [ ] Update playlist settings
|
||||
- [ ] Delete playlist
|
||||
- [ ] List all playlists
|
||||
- [ ] Get playlist with videos
|
||||
|
||||
#### Video Operations
|
||||
- [ ] Videos automatically created from playlist
|
||||
- [ ] Manual playlist check triggers new video detection
|
||||
- [ ] Video status transitions work correctly
|
||||
- [ ] Skip/reset operations work
|
||||
|
||||
#### Integration
|
||||
- [ ] MeTube connection established
|
||||
- [ ] Downloads triggered successfully
|
||||
- [ ] WebSocket events received
|
||||
- [ ] Status synchronization works
|
||||
|
||||
#### Error Handling
|
||||
- [ ] Invalid URLs handled gracefully
|
||||
- [ ] Database errors handled
|
||||
- [ ] MeTube connection failures handled
|
||||
- [ ] Proper error messages returned
|
||||
|
||||
### Manual Testing Commands
|
||||
|
||||
#### Service Health
|
||||
```bash
|
||||
# Health check
|
||||
curl http://localhost:8082/health
|
||||
|
||||
# System status
|
||||
curl http://localhost:8082/api/status
|
||||
|
||||
# Scheduler status
|
||||
curl http://localhost:8082/api/scheduler/status
|
||||
```
|
||||
|
||||
#### Playlist Testing
|
||||
```bash
|
||||
# Create test playlist
|
||||
curl -X POST http://localhost:8082/api/playlists \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"url": "https://www.youtube.com/playlist?list=PLrAXtmErZgOeiKm4sgNOknGvNjby9efdf",
|
||||
"title": "Manual Test Playlist",
|
||||
"check_interval": 30
|
||||
}'
|
||||
|
||||
# Trigger manual check
|
||||
curl -X POST http://localhost:8082/api/playlists/{id}/check
|
||||
|
||||
# Monitor logs
|
||||
tail -f logs/playlist-monitor.log
|
||||
```
|
||||
|
||||
## 🔄 CI/CD Integration
|
||||
|
||||
### GitHub Actions (.github/workflows/tests.yml)
|
||||
```yaml
|
||||
name: Tests
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [ main, develop ]
|
||||
pull_request:
|
||||
branches: [ main ]
|
||||
|
||||
jobs:
|
||||
test:
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:15
|
||||
env:
|
||||
POSTGRES_PASSWORD: postgres
|
||||
POSTGRES_DB: test_playlists
|
||||
options: >-
|
||||
--health-cmd pg_isready
|
||||
--health-interval 10s
|
||||
--health-timeout 5s
|
||||
--health-retries 5
|
||||
ports:
|
||||
- 5432:5432
|
||||
|
||||
strategy:
|
||||
matrix:
|
||||
python-version: [3.13, 3.14]
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v3
|
||||
|
||||
- name: Set up Python ${{ matrix.python-version }}
|
||||
uses: actions/setup-python@v4
|
||||
with:
|
||||
python-version: ${{ matrix.python-version }}
|
||||
|
||||
- name: Install uv
|
||||
run: curl -LsSf https://astral.sh/uv/install.sh | sh
|
||||
|
||||
- name: Install dependencies
|
||||
run: uv sync --extra dev
|
||||
|
||||
- name: Run unit tests
|
||||
run: uv run pytest tests/unit/ -v --cov=app --cov-report=xml
|
||||
|
||||
- name: Run integration tests
|
||||
run: uv run pytest tests/integration/ -v
|
||||
|
||||
- name: Run performance tests
|
||||
run: uv run pytest tests/performance/ -v --tb=short
|
||||
|
||||
- name: Upload coverage reports
|
||||
uses: codecov/codecov-action@v3
|
||||
with:
|
||||
file: ./coverage.xml
|
||||
```
|
||||
|
||||
### Pre-commit Hooks (.pre-commit-config.yaml)
|
||||
```yaml
|
||||
repos:
|
||||
- repo: https://github.com/pre-commit/pre-commit-hooks
|
||||
rev: v4.4.0
|
||||
hooks:
|
||||
- id: trailing-whitespace
|
||||
- id: end-of-file-fixer
|
||||
- id: check-yaml
|
||||
- id: check-added-large-files
|
||||
|
||||
- repo: https://github.com/psf/black
|
||||
rev: 23.3.0
|
||||
hooks:
|
||||
- id: black
|
||||
language_version: python3.13
|
||||
|
||||
- repo: https://github.com/pycqa/isort
|
||||
rev: 5.12.0
|
||||
hooks:
|
||||
- id: isort
|
||||
|
||||
- repo: https://github.com/pycqa/flake8
|
||||
rev: 6.0.0
|
||||
hooks:
|
||||
- id: flake8
|
||||
|
||||
- repo: https://github.com/pre-commit/mirrors-mypy
|
||||
rev: v1.3.0
|
||||
hooks:
|
||||
- id: mypy
|
||||
additional_dependencies: [types-all]
|
||||
```
|
||||
|
||||
## 📊 Test Data Management
|
||||
|
||||
### Test Fixtures (tests/fixtures/)
|
||||
```python
|
||||
# tests/fixtures/playlists.py
|
||||
TEST_PLAYLISTS = [
|
||||
{
|
||||
"url": "https://www.youtube.com/playlist?list=PLrAXtmErZgOeiKm4sgNOknGvNjby9efdf",
|
||||
"title": "Kurzgesagt – In a Nutshell",
|
||||
"check_interval": 60,
|
||||
"quality": "best",
|
||||
"format": "mp4",
|
||||
"folder": "kurzgesagt"
|
||||
},
|
||||
{
|
||||
"url": "https://www.youtube.com/playlist?list=UUX6OQ3DkcsbYNE6H8uQQuVA",
|
||||
"title": "Tech Channel Uploads",
|
||||
"check_interval": 30,
|
||||
"quality": "1080p",
|
||||
"format": "mp4",
|
||||
"folder": "tech-channels"
|
||||
}
|
||||
]
|
||||
|
||||
# tests/fixtures/videos.py
|
||||
TEST_VIDEOS = [
|
||||
{
|
||||
"video_id": "dQw4w9WgXcQ",
|
||||
"title": "Never Gonna Give You Up",
|
||||
"playlist_index": 1,
|
||||
"status": "COMPLETED"
|
||||
},
|
||||
{
|
||||
"video_id": "9bZkp7q19f0",
|
||||
"title": "Gangnam Style",
|
||||
"playlist_index": 2,
|
||||
"status": "PENDING"
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
### Mock Data Generation
|
||||
```python
|
||||
# tests/utils/mock_data.py
|
||||
def generate_mock_playlists(count=10):
|
||||
"""Generate mock playlist data"""
|
||||
playlists = []
|
||||
for i in range(count):
|
||||
playlists.append({
|
||||
"url": f"https://www.youtube.com/playlist?list=TEST{i:03d}",
|
||||
"title": f"Test Playlist {i}",
|
||||
"check_interval": 60,
|
||||
"quality": "best",
|
||||
"format": "mp4",
|
||||
"folder": f"test-folder-{i}"
|
||||
})
|
||||
return playlists
|
||||
|
||||
def generate_mock_videos(playlist_id, count=20):
|
||||
"""Generate mock video data"""
|
||||
videos = []
|
||||
for i in range(count):
|
||||
videos.append({
|
||||
"playlist_id": playlist_id,
|
||||
"video_url": f"https://www.youtube.com/watch?v=VIDEO{i:03d}",
|
||||
"video_id": f"VIDEO{i:03d}",
|
||||
"title": f"Test Video {i}",
|
||||
"playlist_index": i + 1,
|
||||
"status": "PENDING"
|
||||
})
|
||||
return videos
|
||||
```
|
||||
|
||||
## 📈 Test Coverage
|
||||
|
||||
### Coverage Report Generation
|
||||
```bash
|
||||
# Generate coverage report
|
||||
uv run pytest --cov=app --cov-report=html --cov-report=term
|
||||
|
||||
# View HTML report
|
||||
open htmlcov/index.html
|
||||
|
||||
# Generate XML report for CI
|
||||
uv run pytest --cov=app --cov-report=xml
|
||||
```
|
||||
|
||||
### Coverage Goals
|
||||
- **Unit Tests**: >90% coverage
|
||||
- **Integration Tests**: >80% coverage
|
||||
- **API Tests**: >95% coverage
|
||||
- **Overall**: >85% coverage
|
||||
|
||||
### Coverage Configuration (pyproject.toml)
|
||||
```toml
|
||||
[tool.coverage.run]
|
||||
source = ["app"]
|
||||
omit = [
|
||||
"*/tests/*",
|
||||
"*/venv/*",
|
||||
"*/__pycache__/*",
|
||||
"app/main.py", # Entry point
|
||||
]
|
||||
|
||||
[tool.coverage.report]
|
||||
exclude_lines = [
|
||||
"pragma: no cover",
|
||||
"def __repr__",
|
||||
"raise AssertionError",
|
||||
"raise NotImplementedError",
|
||||
]
|
||||
|
||||
[tool.coverage.html]
|
||||
directory = "htmlcov"
|
||||
```
|
||||
|
||||
## 🎯 Test Execution Strategies
|
||||
|
||||
### Parallel Testing
|
||||
```bash
|
||||
# Run tests in parallel
|
||||
uv run pytest -n auto
|
||||
|
||||
# Run specific test categories in parallel
|
||||
uv run pytest tests/unit/ -n 4 &
|
||||
uv run pytest tests/integration/ -n 2 &
|
||||
wait
|
||||
```
|
||||
|
||||
### Test Selection
|
||||
```bash
|
||||
# Run tests by marker
|
||||
uv run pytest -m "not performance"
|
||||
uv run pytest -m "performance"
|
||||
|
||||
# Run tests by name pattern
|
||||
uv run pytest -k "test_playlist"
|
||||
uv run pytest -k "not test_performance"
|
||||
|
||||
# Run specific test file
|
||||
uv run pytest tests/unit/test_models.py
|
||||
```
|
||||
|
||||
### Continuous Testing
|
||||
```bash
|
||||
# Watch mode for development
|
||||
uv run pytest-watch tests/unit/
|
||||
|
||||
# Run on file changes
|
||||
uv run ptw -- tests/unit/
|
||||
```
|
||||
|
||||
## 📚 Testing Best Practices
|
||||
|
||||
### 1. Test Naming
|
||||
```python
|
||||
def test_playlist_creation_with_valid_url(self):
|
||||
"""Should create playlist when valid YouTube URL provided"""
|
||||
# Test implementation
|
||||
|
||||
def test_playlist_creation_fails_with_invalid_url(self):
|
||||
"""Should reject playlist creation with invalid URL"""
|
||||
# Test implementation
|
||||
```
|
||||
|
||||
### 2. Test Independence
|
||||
```python
|
||||
@pytest.fixture
|
||||
def clean_database():
|
||||
"""Provide clean database for each test"""
|
||||
# Setup clean state
|
||||
yield
|
||||
# Cleanup after test
|
||||
```
|
||||
|
||||
### 3. Mock External Services
|
||||
```python
|
||||
@pytest.fixture
|
||||
def mock_metube():
|
||||
"""Mock MeTube service for testing"""
|
||||
with respx.mock:
|
||||
respx.get("http://localhost:8081/info").mock(
|
||||
return_value=Response(200, json={"status": "ok"})
|
||||
)
|
||||
yield
|
||||
```
|
||||
|
||||
### 4. Test Data Management
|
||||
```python
|
||||
@pytest.fixture
|
||||
def sample_playlist():
|
||||
"""Provide sample playlist data"""
|
||||
return {
|
||||
"url": "https://www.youtube.com/playlist?list=TEST123",
|
||||
"title": "Test Playlist",
|
||||
"check_interval": 60
|
||||
}
|
||||
```
|
||||
|
||||
This comprehensive testing guide ensures robust, reliable testing of the Playlist Monitor Service! 🧪✅
|
||||
File diff suppressed because it is too large
Load Diff
Loading…
Reference in New Issue