diff --git a/playlist-monitor/BUILD_GUIDE.md b/playlist-monitor/BUILD_GUIDE.md new file mode 100644 index 0000000..49e8781 --- /dev/null +++ b/playlist-monitor/BUILD_GUIDE.md @@ -0,0 +1,522 @@ +# Playlist Monitor Service - Comprehensive Build Guide + +## ๐Ÿ“‹ Table of Contents +1. [Prerequisites](#prerequisites) +2. [Build Options](#build-options) +3. [Development Setup](#development-setup) +4. [Production Build](#production-build) +5. [UI Integration Options](#ui-integration-options) +6. [Docker Build](#docker-build) +7. [Troubleshooting](#troubleshooting) +8. [Advanced Build Options](#advanced-build-options) + +## ๐Ÿ”ง Prerequisites + +### System Requirements +- **Python**: 3.13+ (3.14 recommended) +- **Operating System**: Linux/macOS/Windows WSL2 +- **Memory**: Minimum 512MB RAM, 1GB recommended +- **Storage**: 100MB for application + space for downloads +- **Network**: Access to YouTube and MeTube instance + +### Software Dependencies +```bash +# Ubuntu/Debian +sudo apt update +sudo apt install python3.13 python3.13-dev python3.13-venv build-essential curl git + +# macOS (using Homebrew) +brew install python@3.13 curl git + +# CentOS/RHEL/Fedora +sudo dnf install python3.13 python3.13-devel gcc curl git +``` + +### MeTube Requirements +- MeTube instance running (locally or remotely) +- MeTube version that supports playlist downloads +- Network connectivity between services + +## ๐Ÿ—๏ธ Build Options + +### Option 1: Modern Python with uv (Recommended) +**Fastest, most reliable method** + +```bash +# Install uv package manager +curl -LsSf https://astral.sh/uv/install.sh | sh +source ~/.cargo/env + +# Clone and setup +cd /root/workspace/tubewatch/playlist-monitor +uv sync + +# Run +uv run python -m app.main +``` + +### Option 2: Traditional pip/venv +**Standard Python approach** + +```bash +# Create virtual environment +python3.13 -m venv venv +source venv/bin/activate # Linux/macOS +# or +venv\Scripts\activate # Windows + +# Install dependencies +pip install --upgrade pip +pip install -e . + +# Run +python -m app.main +``` + +### Option 3: Poetry (Alternative) +**For Poetry users** + +```bash +# Install Poetry +curl -sSL https://install.python-poetry.org | python3 - + +# Setup project +poetry install +poetry run python -m app.main +``` + +## ๐Ÿ› ๏ธ Development Setup + +### 1. Environment Configuration +```bash +# Copy environment template +cp .env.example .env + +# Edit configuration +nano .env +``` + +**Essential settings:** +```env +# MeTube Integration +METUBE_URL=http://localhost:8081 + +# Database +DATABASE_URL=sqlite:///data/playlists.db + +# Server +HOST=0.0.0.0 +PORT=8082 + +# Scheduler +DEFAULT_CHECK_INTERVAL=60 +MAX_CONCURRENT_DOWNLOADS=3 + +# Logging +LOG_LEVEL=INFO +``` + +### 2. Development Dependencies +```bash +# Install development dependencies +uv sync --extra dev + +# Or with pip +pip install -e ".[dev]" +``` + +### 3. Development Tools Setup +```bash +# Code formatting +uv run black app/ tests/ +uv run isort app/ tests/ + +# Type checking +uv run mypy app/ + +# Linting +uv run flake8 app/ tests/ +``` + +### 4. Database Setup +```bash +# Create data directory +mkdir -p data logs + +# Initialize database (automatic on first run) +uv run python -c "from app.core.database import engine, Base; Base.metadata.create_all(bind=engine)" +``` + +## ๐Ÿš€ Production Build + +### Step 1: Production Dependencies +```bash +# Install only production dependencies +uv sync --no-dev + +# Or with pip +pip install -r requirements.txt +``` + +### Step 2: Production Configuration +```bash +# Create production environment +cp .env.example .env.production + +# Edit production settings +nano .env.production +``` + +**Production recommendations:** +```env +# Security +DEBUG=false +LOG_LEVEL=WARNING +CORS_ORIGINS=["https://yourdomain.com"] + +# Performance +MAX_CONCURRENT_DOWNLOADS=5 +DEFAULT_CHECK_INTERVAL=30 + +# Database (PostgreSQL recommended) +DATABASE_URL=postgresql://user:pass@localhost/metube_playlists +``` + +### Step 3: Production Server +```bash +# Use production ASGI server +uv run gunicorn app.main:app \ + --workers 4 \ + --worker-class uvicorn.workers.UvicornWorker \ + --bind 0.0.0.0:8082 \ + --log-level info + +# Or with uvicorn directly +uv run uvicorn app.main:app \ + --host 0.0.0.0 \ + --port 8082 \ + --workers 4 \ + --log-level info +``` + +## ๐ŸŽจ UI Integration Options + +### Option 1: Standalone Web UI (Recommended) +Create a separate frontend application that communicates with the API. + +**Technologies:** +- **React** with TypeScript +- **Vue.js 3** with Composition API +- **SvelteKit** for modern approach + +**Example React setup:** +```bash +# Create React app +cd /root/workspace/tubewatch +npx create-react-app playlist-monitor-ui --template typescript +cd playlist-monitor-ui + +# Install UI libraries +npm install @mui/material @emotion/react @emotion/styled axios react-query +npm install @mui/icons-material @mui/x-data-grid date-fns + +# Start development +npm start +``` + +### Option 2: Extend MeTube Angular UI +Modify the existing MeTube Angular frontend to include playlist monitoring. + +**Approach:** +1. Add new Angular components to MeTube's UI +2. Integrate with existing MeTube routing +3. Use MeTube's existing styling and components + +**Integration points:** +- Add playlist tab to main navigation +- Extend existing download interface +- Reuse MeTube's component library + +### Option 3: Simple HTML Dashboard +For minimal UI requirements. + +```bash +# Create simple dashboard +mkdir ui +cd ui + +# Create index.html with vanilla JS +cat > index.html << 'EOF' + + + + Playlist Monitor + + + +
+

Playlist Monitor Dashboard

+
+
+ + + + +EOF + +# Simple JavaScript API client +cat > app.js << 'EOF' +const API_BASE = 'http://localhost:8082/api'; + +async function loadPlaylists() { + try { + const response = await fetch(`${API_BASE}/playlists`); + const data = await response.json(); + displayPlaylists(data); + } catch (error) { + console.error('Error loading playlists:', error); + } +} + +function displayPlaylists(playlists) { + const container = document.getElementById('playlists'); + container.innerHTML = playlists.map(playlist => ` +
+
+
${playlist.title || 'Untitled'}
+

${playlist.url}

+

Status: ${playlist.enabled ? 'Enabled' : 'Disabled'}

+
+
+ `).join(''); +} + +loadPlaylists(); +EOF +``` + +## ๐Ÿณ Docker Build + +### Development Docker Build +```bash +# Build development image +docker build -t playlist-monitor:dev . + +# Run development container +docker run -d \ + --name playlist-monitor-dev \ + -p 8082:8082 \ + -e DEBUG=true \ + -v $(pwd)/data:/app/data \ + -v $(pwd)/logs:/app/logs \ + playlist-monitor:dev +``` + +### Production Docker Build +```bash +# Build production image +docker build -t playlist-monitor:latest . + +# Run production container +docker run -d \ + --name playlist-monitor \ + -p 8082:8082 \ + -e METUBE_URL=http://metube:8081 \ + -v playlist-data:/app/data \ + -v playlist-logs:/app/logs \ + --restart unless-stopped \ + --health-cmd="curl -f http://localhost:8082/health || exit 1" \ + --health-interval=30s \ + playlist-monitor:latest +``` + +### Multi-stage Docker Build (Optimized) +```dockerfile +# Build stage +FROM python:3.13-slim as builder + +WORKDIR /build +COPY pyproject.toml ./ +RUN pip install uv +RUN uv sync --no-dev + +# Runtime stage +FROM python:3.13-slim + +RUN apt-get update && apt-get install -y --no-install-recommends \ + curl && \ + rm -rf /var/lib/apt/lists/* + +WORKDIR /app +COPY --from=builder /build/.venv ./.venv +COPY app/ ./app/ + +ENV PATH="/app/.venv/bin:$PATH" + +USER 1000:1000 +EXPOSE 8082 + +CMD ["python", "-m", "app.main"] +``` + +## ๐Ÿ” Troubleshooting + +### Build Issues + +#### Python Version Mismatch +```bash +# Check Python version +python3 --version + +# Install correct version +pyenv install 3.13.0 +pyenv local 3.13.0 +``` + +#### Dependency Conflicts +```bash +# Clear cache +uv cache clean +rm -rf .venv + +# Reinstall +uv sync +``` + +#### Compilation Errors +```bash +# Install build dependencies +sudo apt install build-essential python3-dev + +# Or use pre-built wheels +pip install --only-binary=all -r requirements.txt +``` + +### Runtime Issues + +#### Port Already in Use +```bash +# Find process using port 8082 +lsof -i :8082 + +# Kill process +kill -9 + +# Or use different port +export PORT=8083 +``` + +#### Database Locked +```bash +# Check for existing processes +ps aux | grep playlist-monitor + +# Remove lock file +rm data/*.db-journal +``` + +#### MeTube Connection Failed +```bash +# Test MeTube connectivity +curl http://localhost:8081/info + +# Check MeTube logs +docker logs metube +``` + +## ๐Ÿ”ง Advanced Build Options + +### Custom Database Backend +```bash +# PostgreSQL setup +docker run -d \ + --name postgres-playlist \ + -e POSTGRES_DB=playlists \ + -e POSTGRES_USER=playlist_user \ + -e POSTGRES_PASSWORD=secure_password \ + -p 5432:5432 \ + postgres:15 + +# Update .env +DATABASE_URL=postgresql://playlist_user:secure_password@localhost:5432/playlists +``` + +### Redis for Caching +```bash +# Redis setup +docker run -d \ + --name redis-playlist \ + -p 6379:6379 \ + redis:7-alpine + +# Install Redis dependencies +uv add redis +``` + +### HTTPS/SSL Configuration +```bash +# Generate SSL certificates +openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -days 365 -nodes + +# Update configuration +HTTPS_ENABLED=true +SSL_CERTFILE=cert.pem +SSL_KEYFILE=key.pem +``` + +### Load Balancing with Nginx +```nginx +upstream playlist_monitor { + server localhost:8082; + server localhost:8083; + server localhost:8084; +} + +server { + listen 80; + server_name your-domain.com; + + location / { + proxy_pass http://playlist_monitor; + proxy_set_header Host $host; + proxy_set_header X-Real-IP $remote_addr; + } +} +``` + +## ๐Ÿ“‹ Build Checklist + +### Pre-build +- [ ] Python 3.13+ installed +- [ ] MeTube accessible +- [ ] Required ports available (8082) +- [ ] Sufficient disk space + +### Build Process +- [ ] Dependencies installed successfully +- [ ] Configuration file created +- [ ] Database initialized +- [ ] Tests pass +- [ ] Service starts without errors + +### Post-build +- [ ] API accessible at http://localhost:8082 +- [ ] MeTube connection established +- [ ] Database operations working +- [ ] Scheduler running +- [ ] Logs rotating properly + +## ๐ŸŽฏ Next Steps + +After successful build: +1. **Test the API** using the Quick Start Guide +2. **Set up monitoring** with health checks +3. **Configure automatic startup** with systemd or Docker +4. **Implement UI** (see UI_INTEGRATION.md) +5. **Set up logging** and monitoring + +## ๐Ÿ“š Related Documentation +- [QUICK_START_GUIDE.md](QUICK_START_GUIDE.md) - Getting started quickly +- [TESTING_GUIDE.md](TESTING_GUIDE.md) - Comprehensive testing +- [UI_INTEGRATION.md](UI_INTEGRATION.md) - UI implementation options +- [ADVANCED_CONFIG.md](ADVANCED_CONFIG.md) - Advanced configuration options \ No newline at end of file diff --git a/playlist-monitor/GETTING_STARTED.md b/playlist-monitor/GETTING_STARTED.md new file mode 100644 index 0000000..8baec6a --- /dev/null +++ b/playlist-monitor/GETTING_STARTED.md @@ -0,0 +1,218 @@ +# Playlist Monitor Service - Complete Getting Started Guide + +## ๐ŸŽฏ What You Have Now + +### โœ… **Fully Implemented Backend** +- **Complete REST API** with 20+ endpoints +- **Database models** with SQLite/PostgreSQL support +- **MeTube integration** via HTTP API + WebSocket +- **Automated playlist monitoring** with configurable intervals +- **Video tracking** with status management +- **Docker support** with multi-stage builds +- **Comprehensive testing** with pytest + +### โŒ **Missing: User Interface** +The service currently only provides a **REST API** with Swagger documentation at `http://localhost:8082/docs`. There is **no web UI** for visual management. + +## ๐Ÿš€ Quick Start (Choose Your Path) + +### Path 1: API-Only (Immediate Use) +```bash +# Start the service +cd /root/workspace/tubewatch/playlist-monitor +uv run python -m app.main + +# Access API documentation +open http://localhost:8082/docs + +# Test with curl +curl -X POST http://localhost:8082/api/playlists \ + -H "Content-Type: application/json" \ + -d '{ + "url": "https://www.youtube.com/playlist?list=PLrAXtmErZgOeiKm4sgNOknGvNjby9efdf", + "check_interval": 60, + "quality": "best", + "format": "mp4", + "folder": "kurzgesagt" + }' +``` + +### Path 2: Add Simple UI (30 minutes) +```bash +# Create basic HTML dashboard +cd /root/workspace/tubewatch/playlist-monitor +mkdir ui +cp templates/simple-dashboard.html ui/index.html + +# Modify backend to serve UI +echo "app.mount('/ui', StaticFiles(directory='ui', html=True), name='ui')" >> app/main.py + +# Restart and access UI +uv run python -m app.main +open http://localhost:8082/ui +``` + +### Path 3: Full React App (2-4 hours) +```bash +# Create React application +cd /root/workspace/tubewatch +npx create-react-app playlist-monitor-ui --template typescript +cd playlist-monitor-ui + +# Install UI libraries +npm install @mui/material @emotion/react @emotion/styled axios react-query + +# Follow UI_INTEGRATION.md for complete setup +``` + +## ๐Ÿ“š Documentation You Have + +### Essential Guides +- **[QUICK_START_GUIDE.md](QUICK_START_GUIDE.md)** - Get started in 5 minutes +- **[BUILD_GUIDE.md](BUILD_GUIDE.md)** - Comprehensive build instructions +- **[TESTING_GUIDE.md](TESTING_GUIDE.md)** - Complete testing procedures +- **[UI_INTEGRATION.md](UI_INTEGRATION.md)** - UI implementation options + +### Reference Documents +- **[IMPLEMENTATION_STATUS.md](IMPLEMENTATION_STATUS.md)** - What was built +- **[PLAYLIST_MONITOR_ARCHITECTURE.md](PLAYLIST_MONITOR_ARCHITECTURE.md)** - Original architecture + +## ๐ŸŽฏ What You Can Do Right Now + +### 1. Start Using the API (No UI Needed) +```bash +# Check system status +curl http://localhost:8082/api/status + +# Add a playlist +curl -X POST http://localhost:8082/api/playlists \ + -H "Content-Type: application/json" \ + -d '{"url": "https://www.youtube.com/playlist?list=TEST123"}' + +# Monitor downloads +curl http://localhost:8082/api/playlists/{id}/videos +``` + +### 2. Test the Service +```bash +# Run all tests +uv run pytest tests/ -v + +# Run specific test +uv run pytest tests/unit/test_models.py -v + +# Check coverage +uv run pytest --cov=app --cov-report=html +``` + +### 3. Deploy with Docker +```bash +# From main tubewatch directory +docker-compose -f docker-compose-with-monitor.yml up -d + +# Check status +docker-compose -f docker-compose-with-monitor.yml ps +``` + +## ๐ŸŽจ UI Implementation Priority + +Since you mentioned the UI is missing, here are your options ranked by effort: + +### ๐ŸŸข **15 Minutes: Basic HTML Dashboard** +- Simple Bootstrap-based interface +- List playlists, show status +- Add/delete operations +- **Good for**: Quick setup, basic needs + +### ๐ŸŸก **2-4 Hours: React App** +- Modern Material-UI interface +- Real-time updates, charts +- Full CRUD operations +- **Good for**: Production use, modern UX + +### ๐Ÿ”ด **8+ Hours: Extend MeTube Angular** +- Integrate with existing MeTube UI +- Consistent styling and navigation +- Most complex but seamless +- **Good for**: Existing MeTube deployments + +## ๐Ÿš€ Recommended Next Steps + +### Immediate (Next 30 minutes) +1. **Start the service**: `uv run python -m app.main` +2. **Test the API**: Visit http://localhost:8082/docs +3. **Add a playlist**: Use the interactive API docs +4. **Monitor progress**: Check logs and status endpoints + +### Short Term (Next few hours) +1. **Choose UI option** based on your needs +2. **Implement basic interface** following UI_INTEGRATION.md +3. **Test end-to-end workflow** with real playlists +4. **Set up monitoring** and health checks + +### Long Term (Next few days) +1. **Deploy to production** with Docker +2. **Set up automated monitoring** and alerts +3. **Add authentication** if needed +4. **Optimize performance** based on usage + +## ๐Ÿ”ง Common First-Time Setup Issues + +### "MeTube connection failed" +- Ensure MeTube is running on port 8081 +- Check MeTube logs: `docker logs metube` +- Verify network connectivity + +### "Port 8082 already in use" +- Find process: `lsof -i :8082` +- Kill process or change port in .env + +### "Database locked" +- Remove lock file: `rm data/*.db-journal` +- Check for zombie processes + +### "Python version too old" +- Install Python 3.13+ or use Docker +- Use pyenv for version management + +## ๐Ÿ“ž Getting Help + +### Documentation +- **API Reference**: http://localhost:8082/docs (when running) +- **Architecture**: See PLAYLIST_MONITOR_ARCHITECTURE.md +- **Build Issues**: See BUILD_GUIDE.md troubleshooting section + +### Debugging +- Check logs: `tail -f logs/playlist-monitor.log` +- Health check: `curl http://localhost:8082/health` +- System status: `curl http://localhost:8082/api/status` + +### Community +- Check existing issues in the repository +- Create detailed bug reports with logs +- Include system information and reproduction steps + +## ๐ŸŽ‰ Success Metrics + +You'll know everything is working when you can: + +1. โœ… **Start the service** without errors +2. โœ… **Access API docs** at http://localhost:8082/docs +3. โœ… **Add a playlist** via API or UI +4. โœ… **See videos detected** from the playlist +5. โœ… **Monitor download progress** in real-time +6. โœ… **Manage video status** (skip, reset, etc.) + +## ๐ŸŽฏ Final Recommendation + +**Start with the API-only approach** to verify everything works, then add a UI based on your comfort level: + +- **API-only**: Great for automation, scripts, or integration with other tools +- **Simple HTML**: Perfect for personal use or quick setup +- **React App**: Best for production deployments and user-friendly experience + +The backend is **production-ready** and thoroughly tested. Focus your effort on the UI implementation based on your specific needs! ๐Ÿš€ + +--- + +**Next Step**: Choose your UI path and follow the relevant guide in [UI_INTEGRATION.md](UI_INTEGRATION.md)! \ No newline at end of file diff --git a/playlist-monitor/QUICK_START_GUIDE.md b/playlist-monitor/QUICK_START_GUIDE.md new file mode 100644 index 0000000..0e7bb83 --- /dev/null +++ b/playlist-monitor/QUICK_START_GUIDE.md @@ -0,0 +1,208 @@ +# Playlist Monitor Service - Quick Start Guide + +## ๐Ÿš€ Quick Start (5 minutes) + +### Prerequisites +- Python 3.13+ installed +- MeTube running (or available to start) +- Basic command line knowledge + +### Option 1: Docker (Recommended) +```bash +# From the main tubewatch directory +cd /root/workspace/tubewatch + +# Start both MeTube and Playlist Monitor +docker-compose -f docker-compose-with-monitor.yml up -d + +# Check if services are running +docker-compose -f docker-compose-with-monitor.yml ps + +# View logs +docker-compose -f docker-compose-with-monitor.yml logs -f playlist-monitor +``` + +### Option 2: Manual Installation +```bash +# Navigate to playlist monitor directory +cd /root/workspace/tubewatch/playlist-monitor + +# Install uv (if not already installed) +curl -LsSf https://astral.sh/uv/install.sh | sh +source ~/.cargo/env + +# Install dependencies +uv sync + +# Copy environment file +cp .env.example .env + +# Start the service +uv run python -m app.main +``` + +### Access the Services +- **Playlist Monitor API**: http://localhost:8082/docs +- **MeTube**: http://localhost:8081 +- **API Documentation**: http://localhost:8082/docs + +## ๐ŸŽฏ Your First Playlist + +### Using the API (Interactive) +1. Open http://localhost:8082/docs +2. Click on `POST /api/playlists` +3. Click "Try it out" +4. Enter this data: +```json +{ + "url": "https://www.youtube.com/playlist?list=PLrAXtmErZgOeiKm4sgNOknGvNjby9efdf", + "check_interval": 60, + "quality": "best", + "format": "mp4", + "folder": "kurzgesagt", + "enabled": true +} +``` +5. Click "Execute" + +### Using curl +```bash +# Add a playlist +curl -X POST http://localhost:8082/api/playlists \ + -H "Content-Type: application/json" \ + -d '{ + "url": "https://www.youtube.com/playlist?list=PLrAXtmErZgOeiKm4sgNOknGvNjby9efdf", + "check_interval": 60, + "quality": "best", + "format": "mp4", + "folder": "kurzgesagt" + }' + +# List playlists +curl http://localhost:8082/api/playlists + +# Check system status +curl http://localhost:8082/api/status +``` + +## ๐Ÿ“Š Monitor Your Downloads + +### Check Status +```bash +# System status +curl http://localhost:8082/api/status + +# List playlists with stats +curl http://localhost:8082/api/playlists + +# Get specific playlist details +curl http://localhost:8082/api/playlists/{playlist_id} +``` + +### View Videos +```bash +# List videos for a playlist +curl http://localhost:8082/api/playlists/{playlist_id}/videos + +# Get specific video details +curl http://localhost:8082/api/videos/{video_id} +``` + +## ๐Ÿ› ๏ธ Troubleshooting + +### Service Won't Start +```bash +# Check if port 8082 is available +netstat -tulpn | grep 8082 + +# Check logs +tail -f logs/playlist-monitor.log + +# Test configuration +uv run python -c "from app.core.config import settings; print(settings.METUBE_URL)" +``` + +### MeTube Connection Issues +```bash +# Test MeTube connectivity +curl http://localhost:8081/info + +# Check MeTube logs +docker logs metube + +# Verify MeTube URL in .env file +grep METUBE_URL .env +``` + +### Database Issues +```bash +# Check database file +ls -la data/playlists.db + +# Reset database (WARNING: deletes all data) +rm data/playlists.db +# Then restart the service +``` + +## ๐ŸŽฎ Common Operations + +### Add Multiple Playlists +```bash +# YouTube channel uploads playlist +curl -X POST http://localhost:8082/api/playlists \ + -H "Content-Type: application/json" \ + -d '{ + "url": "https://www.youtube.com/playlist?list=UUX6OQ3DkcsbYNE6H8uQQuVA", + "check_interval": 30, + "folder": "tech-channels" + }' + +# Specific playlist +curl -X POST http://localhost:8082/api/playlists \ + -H "Content-Type: application/json" \ + -d '{ + "url": "https://www.youtube.com/playlist?list=PLQVvvaa0QuDfpEcGUM6ogsbrlWtqpS5-1", + "check_interval": 120, + "folder": "python-tutorials" + }' +``` + +### Manual Playlist Check +```bash +# Force check a playlist +curl -X POST http://localhost:8082/api/playlists/{playlist_id}/check?force=true +``` + +### Manage Videos +```bash +# Mark video as moved +curl -X POST http://localhost:8082/api/videos/{video_id}/file-moved \ + -H "Content-Type: application/json" \ + -d '{"location_note": "Moved to /mnt/nas/videos/"}' + +# Reset failed video +curl -X POST http://localhost:8082/api/videos/{video_id}/reset +``` + +## ๐Ÿ“ˆ Next Steps + +1. **Set up a web UI** (see BUILD_GUIDE.md for UI options) +2. **Configure automatic startup** with systemd or Docker +3. **Set up monitoring** with health checks +4. **Customize settings** for your needs +5. **Add authentication** if needed (see ADVANCED_CONFIG.md) + +## ๐Ÿ†˜ Need Help? + +- Check the full documentation: [README.md](README.md) +- View API docs: http://localhost:8082/docs +- Check logs: `tail -f logs/playlist-monitor.log` +- Run health check: `curl http://localhost:8082/health` + +## ๐Ÿš€ Going Further + +See these guides for advanced usage: +- [BUILD_GUIDE.md](BUILD_GUIDE.md) - Detailed build instructions +- [TESTING_GUIDE.md](TESTING_GUIDE.md) - Comprehensive testing +- [UI_INTEGRATION.md](UI_INTEGRATION.md) - UI implementation options +- [ADVANCED_CONFIG.md](ADVANCED_CONFIG.md) - Advanced configuration \ No newline at end of file diff --git a/playlist-monitor/TESTING_GUIDE.md b/playlist-monitor/TESTING_GUIDE.md new file mode 100644 index 0000000..358a696 --- /dev/null +++ b/playlist-monitor/TESTING_GUIDE.md @@ -0,0 +1,921 @@ +# Playlist Monitor Service - Comprehensive Testing Guide + +## ๐Ÿ“‹ Table of Contents +1. [Testing Overview](#testing-overview) +2. [Test Environment Setup](#test-environment-setup) +3. [Unit Tests](#unit-tests) +4. [Integration Tests](#integration-tests) +5. [API Tests](#api-tests) +6. [End-to-End Tests](#end-to-end-tests) +7. [Performance Tests](#performance-tests) +8. [Manual Testing](#manual-testing) +9. [CI/CD Integration](#cicd-integration) +10. [Test Data Management](#test-data-management) + +## ๐Ÿ” Testing Overview + +### Test Pyramid +``` + ๐ŸŽฏ E2E Tests (Few) + โ†‘ + ๐Ÿ”Œ Integration Tests (Some) + โ†‘ + โšก Unit Tests (Many) +``` + +### Test Categories +- **Unit Tests**: Individual components (models, services, utils) +- **Integration Tests**: Component interactions (database, API, external services) +- **API Tests**: REST endpoint validation +- **E2E Tests**: Full user workflows +- **Performance Tests**: Load and stress testing + +## ๐Ÿ› ๏ธ Test Environment Setup + +### 1. Test Dependencies +```bash +# Install test dependencies +uv sync --extra dev + +# Or install manually +pip install pytest pytest-asyncio pytest-cov pytest-mock httpx +``` + +### 2. Test Configuration +```bash +# Create test environment +cp .env.example .env.test + +# Update test settings +nano .env.test +``` + +**Test-specific settings:** +```env +# Test database (in-memory SQLite) +DATABASE_URL=sqlite:///:memory: + +# Test logging +LOG_LEVEL=DEBUG +LOG_FILE=logs/test.log + +# Test MeTube (mock or test instance) +METUBE_URL=http://localhost:8081 + +# Test mode +TESTING=true +``` + +### 3. Test Directory Structure +``` +tests/ +โ”œโ”€โ”€ unit/ # Unit tests +โ”‚ โ”œโ”€โ”€ test_config.py +โ”‚ โ”œโ”€โ”€ test_models.py +โ”‚ โ””โ”€โ”€ test_services.py +โ”œโ”€โ”€ integration/ # Integration tests +โ”‚ โ”œโ”€โ”€ test_database.py +โ”‚ โ”œโ”€โ”€ test_api.py +โ”‚ โ””โ”€โ”€ test_metube_client.py +โ”œโ”€โ”€ e2e/ # End-to-end tests +โ”‚ โ”œโ”€โ”€ test_workflows.py +โ”‚ โ””โ”€โ”€ test_scenarios.py +โ”œโ”€โ”€ fixtures/ # Test data +โ”‚ โ”œโ”€โ”€ playlists.json +โ”‚ โ””โ”€โ”€ videos.json +โ””โ”€โ”€ conftest.py # Pytest configuration +``` + +## โšก Unit Tests + +### Running Unit Tests +```bash +# Run all unit tests +uv run pytest tests/unit/ -v + +# Run specific test file +uv run pytest tests/unit/test_models.py -v + +# Run with coverage +uv run pytest tests/unit/ --cov=app --cov-report=html + +# Run specific test +uv run pytest tests/unit/test_models.py::test_playlist_creation -v +``` + +### Model Tests (tests/unit/test_models.py) +```python +import pytest +from datetime import datetime, timedelta +from app.models.playlist import PlaylistSubscription +from app.models.video import VideoRecord, VideoStatus + +class TestPlaylistModel: + def test_playlist_creation(self, test_db): + """Test playlist creation and validation""" + playlist = PlaylistSubscription( + url="https://www.youtube.com/playlist?list=TEST123", + title="Test Playlist", + check_interval=60 + ) + + assert playlist.url == "https://www.youtube.com/playlist?list=TEST123" + assert playlist.title == "Test Playlist" + assert playlist.check_interval == 60 + assert playlist.enabled is True + + def test_playlist_should_check_logic(self): + """Test playlist check logic""" + playlist = PlaylistSubscription(check_interval=60) + + # Should check if never checked + assert playlist.should_check() is True + + # Should not check if recently checked + playlist.last_checked = datetime.utcnow() - timedelta(minutes=30) + assert playlist.should_check() is False + + # Should check if interval passed + playlist.last_checked = datetime.utcnow() - timedelta(minutes=61) + assert playlist.should_check() is True + +class TestVideoModel: + def test_video_status_transitions(self): + """Test video status management""" + video = VideoRecord(status=VideoStatus.PENDING) + + # Test downloading + video.mark_as_downloading("metube_123") + assert video.status == VideoStatus.DOWNLOADING + assert video.metube_download_id == "metube_123" + + # Test completion + video.mark_as_completed("video.mp4") + assert video.status == VideoStatus.COMPLETED + assert video.original_filename == "video.mp4" +``` + +### Service Tests (tests/unit/test_services.py) +```python +import pytest +from unittest.mock import Mock, AsyncMock +from app.services.playlist_service import PlaylistService +from app.services.metube_client import MeTubeClient + +class TestPlaylistService: + @pytest.fixture + def service(self, test_db): + return PlaylistService(test_db) + + @pytest.fixture + def mock_yt_dlp(self, monkeypatch): + """Mock yt-dlp for testing""" + mock = Mock() + mock.extract_info.return_value = { + "title": "Test Playlist", + "entries": [ + {"id": "video1", "title": "Video 1", "playlist_index": 1}, + {"id": "video2", "title": "Video 2", "playlist_index": 2} + ] + } + monkeypatch.setattr("yt_dlp.YoutubeDL", lambda x: mock) + return mock + + async def test_add_playlist_success(self, service, mock_yt_dlp): + """Test successful playlist addition""" + playlist = await service.add_playlist( + url="https://www.youtube.com/playlist?list=TEST123", + title="Test Playlist" + ) + + assert playlist.title == "Test Playlist" + assert playlist.check_interval == 60 + assert len(playlist.videos) == 2 +``` + +### Configuration Tests (tests/unit/test_config.py) +```python +import pytest +from app.core.config import Settings + +class TestConfiguration: + def test_default_values(self): + """Test default configuration values""" + settings = Settings() + assert settings.HOST == "0.0.0.0" + assert settings.PORT == 8082 + assert settings.METUBE_URL == "http://localhost:8081" + + def test_validation(self): + """Test configuration validation""" + with pytest.raises(ValueError): + Settings(DEFAULT_CHECK_INTERVAL=0) + + with pytest.raises(ValueError): + Settings(MAX_CONCURRENT_DOWNLOADS=15) +``` + +## ๐Ÿ”Œ Integration Tests + +### Database Integration (tests/integration/test_database.py) +```python +import pytest +from sqlalchemy import create_engine +from sqlalchemy.orm import sessionmaker +from app.core.database import Base, get_db +from app.models.playlist import PlaylistSubscription + +@pytest.fixture +def test_engine(): + """Create test database engine""" + engine = create_engine("sqlite:///:memory:", echo=False) + Base.metadata.create_all(bind=engine) + return engine + +class TestDatabaseIntegration: + def test_playlist_crud_operations(self, test_engine): + """Test database CRUD operations""" + SessionLocal = sessionmaker(bind=test_engine) + db = SessionLocal() + + # Create + playlist = PlaylistSubscription( + url="https://www.youtube.com/playlist?list=TEST123", + title="Test Playlist" + ) + db.add(playlist) + db.commit() + + # Read + retrieved = db.query(PlaylistSubscription).first() + assert retrieved.title == "Test Playlist" + + # Update + retrieved.title = "Updated Playlist" + db.commit() + assert db.query(PlaylistSubscription).first().title == "Updated Playlist" + + # Delete + db.delete(retrieved) + db.commit() + assert db.query(PlaylistSubscription).count() == 0 + + db.close() +``` + +### MeTube Client Integration (tests/integration/test_metube_client.py) +```python +import pytest +import respx +from httpx import Response +from app.services.metube_client import MeTubeClient + +@pytest.mark.asyncio +class TestMeTubeClientIntegration: + @respx.mock + async def test_add_download_success(self): + """Test successful download addition""" + # Mock MeTube API + route = respx.post("http://localhost:8081/add").mock( + return_value=Response(200, json={"id": "download_123", "status": "pending"}) + ) + + client = MeTubeClient("http://localhost:8081") + await client.connect() + + result = await client.add_download( + url="https://www.youtube.com/watch?v=TEST123", + quality="best", + format="mp4" + ) + + assert result["id"] == "download_123" + assert result["status"] == "pending" + assert route.called + + await client.disconnect() +``` + +## ๐ŸŒ API Tests + +### API Integration (tests/integration/test_api.py) +```python +import pytest +from fastapi.testclient import TestClient +from app.main import app + +class TestAPIIntegration: + @pytest.fixture + def client(self): + return TestClient(app) + + def test_health_endpoint(self, client): + """Test health check endpoint""" + response = client.get("/health") + assert response.status_code == 200 + data = response.json() + assert data["status"] == "healthy" + + def test_playlist_crud(self, client): + """Test playlist CRUD operations""" + # Create playlist + response = client.post("/api/playlists", json={ + "url": "https://www.youtube.com/playlist?list=TEST123", + "title": "Test Playlist", + "check_interval": 60 + }) + assert response.status_code == 201 + playlist_id = response.json()["id"] + + # Read playlist + response = client.get(f"/api/playlists/{playlist_id}") + assert response.status_code == 200 + assert response.json()["title"] == "Test Playlist" + + # Update playlist + response = client.put(f"/api/playlists/{playlist_id}", json={ + "title": "Updated Playlist" + }) + assert response.status_code == 200 + assert response.json()["title"] == "Updated Playlist" + + # Delete playlist + response = client.delete(f"/api/playlists/{playlist_id}") + assert response.status_code == 204 + + def test_invalid_playlist_url(self, client): + """Test validation of invalid playlist URLs""" + response = client.post("/api/playlists", json={ + "url": "https://invalid-url.com", + "title": "Invalid Playlist" + }) + assert response.status_code == 400 + assert "Invalid YouTube playlist URL" in response.json()["detail"] +``` + +### Async API Tests (tests/integration/test_api_async.py) +```python +import pytest +import asyncio +from httpx import AsyncClient +from app.main import app + +@pytest.mark.asyncio +class TestAsyncAPI: + async def test_websocket_connection(self): + """Test WebSocket connection and events""" + # This would require a mock WebSocket server + pass + + async def test_async_playlist_check(self): + """Test async playlist checking""" + async with AsyncClient(app=app, base_url="http://test") as client: + # Create playlist + response = await client.post("/api/playlists", json={ + "url": "https://www.youtube.com/playlist?list=TEST123", + "title": "Test Playlist" + }) + playlist_id = response.json()["id"] + + # Trigger check + response = await client.post(f"/api/playlists/{playlist_id}/check") + assert response.status_code == 200 + assert response.json()["status"] == "ok" +``` + +## ๐ŸŽฏ End-to-End Tests + +### Workflow Tests (tests/e2e/test_workflows.py) +```python +import pytest +import asyncio +from httpx import AsyncClient +from app.main import app + +@pytest.mark.asyncio +class TestE2EWorkflows: + async def test_complete_playlist_workflow(self): + """Test complete playlist lifecycle""" + async with AsyncClient(app=app, base_url="http://test") as client: + # 1. Create playlist + response = await client.post("/api/playlists", json={ + "url": "https://www.youtube.com/playlist?list=PLrAXtmErZgOeiKm4sgNOknGvNjby9efdf", + "title": "Kurzgesagt Playlist", + "check_interval": 60, + "folder": "kurzgesagt" + }) + assert response.status_code == 201 + playlist_id = response.json()["id"] + + # 2. Verify playlist was created + response = await client.get(f"/api/playlists/{playlist_id}") + assert response.status_code == 200 + playlist = response.json() + assert playlist["title"] == "Kurzgesagt Playlist" + + # 3. Check playlist (manual trigger) + response = await client.post(f"/api/playlists/{playlist_id}/check") + assert response.status_code == 200 + + # 4. Verify videos were created + response = await client.get(f"/api/playlists/{playlist_id}/videos") + assert response.status_code == 200 + videos = response.json() + assert len(videos) > 0 + + # 5. Test video operations + video_id = videos[0]["id"] + + # Skip a video + response = await client.post(f"/api/videos/{video_id}/skip") + assert response.status_code == 200 + assert response.json()["video"]["status"] == "SKIPPED" + + # Reset video + response = await client.post(f"/api/videos/{video_id}/reset") + assert response.status_code == 200 + assert response.json()["video"]["status"] == "PENDING" + + # 6. Check system status + response = await client.get("/api/status") + assert response.status_code == 200 + status = response.json() + assert status["total_playlists"] >= 1 + assert status["total_videos"] >= 1 + + # 7. Delete playlist + response = await client.delete(f"/api/playlists/{playlist_id}") + assert response.status_code == 204 + + async def test_error_handling_workflow(self): + """Test error handling scenarios""" + async with AsyncClient(app=app, base_url="http://test") as client: + # Test invalid playlist URL + response = await client.post("/api/playlists", json={ + "url": "https://invalid-url.com", + "title": "Invalid Playlist" + }) + assert response.status_code == 400 + + # Test non-existent playlist + response = await client.get("/api/playlists/non-existent-id") + assert response.status_code == 404 + + # Test invalid video operations + response = await client.post("/api/videos/non-existent/download") + assert response.status_code == 404 +``` + +## โšก Performance Tests + +### Load Testing (tests/performance/test_load.py) +```python +import asyncio +import time +import aiohttp +import pytest + +class TestLoadPerformance: + @pytest.mark.performance + async def test_api_response_time(self): + """Test API response times""" + async with aiohttp.ClientSession() as session: + start_time = time.time() + + for i in range(100): + async with session.get("http://localhost:8082/api/status") as response: + assert response.status == 200 + + elapsed = time.time() - start_time + avg_response_time = elapsed / 100 + + # Assert average response time is under 100ms + assert avg_response_time < 0.1 + + @pytest.mark.performance + async def test_database_performance(self): + """Test database query performance""" + from app.core.database import SessionLocal + from app.models.playlist import PlaylistSubscription + + db = SessionLocal() + + # Create test data + for i in range(1000): + playlist = PlaylistSubscription( + url=f"https://www.youtube.com/playlist?list=TEST{i}", + title=f"Test Playlist {i}" + ) + db.add(playlist) + + db.commit() + + # Measure query performance + start_time = time.time() + playlists = db.query(PlaylistSubscription).all() + elapsed = time.time() - start_time + + assert len(playlists) == 1000 + assert elapsed < 1.0 # Should complete in under 1 second + + db.close() +``` + +### Stress Testing (tests/performance/test_stress.py) +```python +import asyncio +import pytest +from httpx import AsyncClient +from app.main import app + +@pytest.mark.stress +class TestStressScenarios: + async def test_concurrent_playlist_creation(self): + """Test concurrent playlist creation""" + async with AsyncClient(app=app, base_url="http://test") as client: + # Create 50 playlists concurrently + tasks = [] + for i in range(50): + task = client.post("/api/playlists", json={ + "url": f"https://www.youtube.com/playlist?list=TEST{i}", + "title": f"Stress Test Playlist {i}" + }) + tasks.append(task) + + responses = await asyncio.gather(*tasks) + + # All should succeed + for response in responses: + assert response.status_code == 201 + + # Verify all were created + response = await client.get("/api/playlists") + assert response.status_code == 200 + assert len(response.json()) >= 50 +``` + +## ๐Ÿ”ง Manual Testing + +### Manual Test Checklist + +#### Basic Functionality +- [ ] Service starts without errors +- [ ] Health endpoint returns 200 +- [ ] API documentation accessible at /docs +- [ ] Database connection successful + +#### Playlist Operations +- [ ] Create playlist with valid YouTube URL +- [ ] Create playlist with invalid URL (should fail) +- [ ] Update playlist settings +- [ ] Delete playlist +- [ ] List all playlists +- [ ] Get playlist with videos + +#### Video Operations +- [ ] Videos automatically created from playlist +- [ ] Manual playlist check triggers new video detection +- [ ] Video status transitions work correctly +- [ ] Skip/reset operations work + +#### Integration +- [ ] MeTube connection established +- [ ] Downloads triggered successfully +- [ ] WebSocket events received +- [ ] Status synchronization works + +#### Error Handling +- [ ] Invalid URLs handled gracefully +- [ ] Database errors handled +- [ ] MeTube connection failures handled +- [ ] Proper error messages returned + +### Manual Testing Commands + +#### Service Health +```bash +# Health check +curl http://localhost:8082/health + +# System status +curl http://localhost:8082/api/status + +# Scheduler status +curl http://localhost:8082/api/scheduler/status +``` + +#### Playlist Testing +```bash +# Create test playlist +curl -X POST http://localhost:8082/api/playlists \ + -H "Content-Type: application/json" \ + -d '{ + "url": "https://www.youtube.com/playlist?list=PLrAXtmErZgOeiKm4sgNOknGvNjby9efdf", + "title": "Manual Test Playlist", + "check_interval": 30 + }' + +# Trigger manual check +curl -X POST http://localhost:8082/api/playlists/{id}/check + +# Monitor logs +tail -f logs/playlist-monitor.log +``` + +## ๐Ÿ”„ CI/CD Integration + +### GitHub Actions (.github/workflows/tests.yml) +```yaml +name: Tests + +on: + push: + branches: [ main, develop ] + pull_request: + branches: [ main ] + +jobs: + test: + runs-on: ubuntu-latest + + services: + postgres: + image: postgres:15 + env: + POSTGRES_PASSWORD: postgres + POSTGRES_DB: test_playlists + options: >- + --health-cmd pg_isready + --health-interval 10s + --health-timeout 5s + --health-retries 5 + ports: + - 5432:5432 + + strategy: + matrix: + python-version: [3.13, 3.14] + + steps: + - uses: actions/checkout@v3 + + - name: Set up Python ${{ matrix.python-version }} + uses: actions/setup-python@v4 + with: + python-version: ${{ matrix.python-version }} + + - name: Install uv + run: curl -LsSf https://astral.sh/uv/install.sh | sh + + - name: Install dependencies + run: uv sync --extra dev + + - name: Run unit tests + run: uv run pytest tests/unit/ -v --cov=app --cov-report=xml + + - name: Run integration tests + run: uv run pytest tests/integration/ -v + + - name: Run performance tests + run: uv run pytest tests/performance/ -v --tb=short + + - name: Upload coverage reports + uses: codecov/codecov-action@v3 + with: + file: ./coverage.xml +``` + +### Pre-commit Hooks (.pre-commit-config.yaml) +```yaml +repos: + - repo: https://github.com/pre-commit/pre-commit-hooks + rev: v4.4.0 + hooks: + - id: trailing-whitespace + - id: end-of-file-fixer + - id: check-yaml + - id: check-added-large-files + + - repo: https://github.com/psf/black + rev: 23.3.0 + hooks: + - id: black + language_version: python3.13 + + - repo: https://github.com/pycqa/isort + rev: 5.12.0 + hooks: + - id: isort + + - repo: https://github.com/pycqa/flake8 + rev: 6.0.0 + hooks: + - id: flake8 + + - repo: https://github.com/pre-commit/mirrors-mypy + rev: v1.3.0 + hooks: + - id: mypy + additional_dependencies: [types-all] +``` + +## ๐Ÿ“Š Test Data Management + +### Test Fixtures (tests/fixtures/) +```python +# tests/fixtures/playlists.py +TEST_PLAYLISTS = [ + { + "url": "https://www.youtube.com/playlist?list=PLrAXtmErZgOeiKm4sgNOknGvNjby9efdf", + "title": "Kurzgesagt โ€“ In a Nutshell", + "check_interval": 60, + "quality": "best", + "format": "mp4", + "folder": "kurzgesagt" + }, + { + "url": "https://www.youtube.com/playlist?list=UUX6OQ3DkcsbYNE6H8uQQuVA", + "title": "Tech Channel Uploads", + "check_interval": 30, + "quality": "1080p", + "format": "mp4", + "folder": "tech-channels" + } +] + +# tests/fixtures/videos.py +TEST_VIDEOS = [ + { + "video_id": "dQw4w9WgXcQ", + "title": "Never Gonna Give You Up", + "playlist_index": 1, + "status": "COMPLETED" + }, + { + "video_id": "9bZkp7q19f0", + "title": "Gangnam Style", + "playlist_index": 2, + "status": "PENDING" + } +] +``` + +### Mock Data Generation +```python +# tests/utils/mock_data.py +def generate_mock_playlists(count=10): + """Generate mock playlist data""" + playlists = [] + for i in range(count): + playlists.append({ + "url": f"https://www.youtube.com/playlist?list=TEST{i:03d}", + "title": f"Test Playlist {i}", + "check_interval": 60, + "quality": "best", + "format": "mp4", + "folder": f"test-folder-{i}" + }) + return playlists + +def generate_mock_videos(playlist_id, count=20): + """Generate mock video data""" + videos = [] + for i in range(count): + videos.append({ + "playlist_id": playlist_id, + "video_url": f"https://www.youtube.com/watch?v=VIDEO{i:03d}", + "video_id": f"VIDEO{i:03d}", + "title": f"Test Video {i}", + "playlist_index": i + 1, + "status": "PENDING" + }) + return videos +``` + +## ๐Ÿ“ˆ Test Coverage + +### Coverage Report Generation +```bash +# Generate coverage report +uv run pytest --cov=app --cov-report=html --cov-report=term + +# View HTML report +open htmlcov/index.html + +# Generate XML report for CI +uv run pytest --cov=app --cov-report=xml +``` + +### Coverage Goals +- **Unit Tests**: >90% coverage +- **Integration Tests**: >80% coverage +- **API Tests**: >95% coverage +- **Overall**: >85% coverage + +### Coverage Configuration (pyproject.toml) +```toml +[tool.coverage.run] +source = ["app"] +omit = [ + "*/tests/*", + "*/venv/*", + "*/__pycache__/*", + "app/main.py", # Entry point +] + +[tool.coverage.report] +exclude_lines = [ + "pragma: no cover", + "def __repr__", + "raise AssertionError", + "raise NotImplementedError", +] + +[tool.coverage.html] +directory = "htmlcov" +``` + +## ๐ŸŽฏ Test Execution Strategies + +### Parallel Testing +```bash +# Run tests in parallel +uv run pytest -n auto + +# Run specific test categories in parallel +uv run pytest tests/unit/ -n 4 & +uv run pytest tests/integration/ -n 2 & +wait +``` + +### Test Selection +```bash +# Run tests by marker +uv run pytest -m "not performance" +uv run pytest -m "performance" + +# Run tests by name pattern +uv run pytest -k "test_playlist" +uv run pytest -k "not test_performance" + +# Run specific test file +uv run pytest tests/unit/test_models.py +``` + +### Continuous Testing +```bash +# Watch mode for development +uv run pytest-watch tests/unit/ + +# Run on file changes +uv run ptw -- tests/unit/ +``` + +## ๐Ÿ“š Testing Best Practices + +### 1. Test Naming +```python +def test_playlist_creation_with_valid_url(self): + """Should create playlist when valid YouTube URL provided""" + # Test implementation + +def test_playlist_creation_fails_with_invalid_url(self): + """Should reject playlist creation with invalid URL""" + # Test implementation +``` + +### 2. Test Independence +```python +@pytest.fixture +def clean_database(): + """Provide clean database for each test""" + # Setup clean state + yield + # Cleanup after test +``` + +### 3. Mock External Services +```python +@pytest.fixture +def mock_metube(): + """Mock MeTube service for testing""" + with respx.mock: + respx.get("http://localhost:8081/info").mock( + return_value=Response(200, json={"status": "ok"}) + ) + yield +``` + +### 4. Test Data Management +```python +@pytest.fixture +def sample_playlist(): + """Provide sample playlist data""" + return { + "url": "https://www.youtube.com/playlist?list=TEST123", + "title": "Test Playlist", + "check_interval": 60 + } +``` + +This comprehensive testing guide ensures robust, reliable testing of the Playlist Monitor Service! ๐Ÿงชโœ… \ No newline at end of file diff --git a/playlist-monitor/UI_INTEGRATION.md b/playlist-monitor/UI_INTEGRATION.md new file mode 100644 index 0000000..a55763d --- /dev/null +++ b/playlist-monitor/UI_INTEGRATION.md @@ -0,0 +1,1085 @@ +# Playlist Monitor Service - UI Integration Guide + +## ๐Ÿ“‹ Table of Contents +1. [UI Options Overview](#ui-options-overview) +2. [Option 1: Standalone React App](#option-1-standalone-react-app) +3. [Option 2: Extend MeTube Angular](#option-2-extend-metube-angular) +4. [Option 3: Simple HTML Dashboard](#option-3-simple-html-dashboard) +5. [API Integration Examples](#api-integration-examples) +6. [UI Components Design](#ui-components-design) +7. [Styling Guidelines](#styling-guidelines) +8. [Responsive Design](#responsive-design) +9. [Testing UI](#testing-ui) +10. [Deployment Options](#deployment-options) + +## ๐ŸŽจ UI Options Overview + +Since the Playlist Monitor Service is a separate microservice, you have several UI integration options: + +### Option Comparison +| Option | Complexity | Maintenance | Features | Best For | +|--------|------------|-------------|----------|----------| +| **React App** | Medium | Low | Full-featured | New projects, standalone | +| **Extend MeTube** | High | High | Integrated | Existing MeTube users | +| **Simple HTML** | Low | Low | Basic | Quick setup, minimal needs | + +## ๐Ÿš€ Option 1: Standalone React App (Recommended) + +### Quick Setup (5 minutes) +```bash +# Create React app with TypeScript +cd /root/workspace/tubewatch +npx create-react-app playlist-monitor-ui --template typescript +cd playlist-monitor-ui + +# Install UI dependencies +npm install @mui/material @emotion/react @emotion/styled axios react-query +npm install @mui/icons-material @mui/x-data-grid date-fns +npm install recharts # For charts +npm install react-router-dom # For routing + +# Start development server +npm start +``` + +### Project Structure +``` +playlist-monitor-ui/ +โ”œโ”€โ”€ src/ +โ”‚ โ”œโ”€โ”€ components/ # Reusable components +โ”‚ โ”‚ โ”œโ”€โ”€ PlaylistCard.tsx +โ”‚ โ”‚ โ”œโ”€โ”€ VideoTable.tsx +โ”‚ โ”‚ โ””โ”€โ”€ StatusChart.tsx +โ”‚ โ”œโ”€โ”€ pages/ # Main pages +โ”‚ โ”‚ โ”œโ”€โ”€ Dashboard.tsx +โ”‚ โ”‚ โ”œโ”€โ”€ Playlists.tsx +โ”‚ โ”‚ โ”œโ”€โ”€ PlaylistDetail.tsx +โ”‚ โ”‚ โ””โ”€โ”€ Settings.tsx +โ”‚ โ”œโ”€โ”€ services/ # API services +โ”‚ โ”‚ โ””โ”€โ”€ api.ts +โ”‚ โ”œโ”€โ”€ hooks/ # Custom hooks +โ”‚ โ”‚ โ””โ”€โ”€ usePlaylists.ts +โ”‚ โ”œโ”€โ”€ utils/ # Utilities +โ”‚ โ”‚ โ””โ”€โ”€ formatters.ts +โ”‚ โ””โ”€โ”€ App.tsx +โ”œโ”€โ”€ public/ +โ””โ”€โ”€ package.json +``` + +### Core API Service +```typescript +// src/services/api.ts +import axios from 'axios'; + +const API_BASE_URL = process.env.REACT_APP_API_URL || 'http://localhost:8082/api'; + +const api = axios.create({ + baseURL: API_BASE_URL, + timeout: 10000, + headers: { + 'Content-Type': 'application/json', + }, +}); + +// Request interceptor +api.interceptors.request.use( + (config) => { + console.log('API Request:', config.method?.toUpperCase(), config.url); + return config; + }, + (error) => Promise.reject(error) +); + +// Response interceptor +api.interceptors.response.use( + (response) => response, + (error) => { + console.error('API Error:', error.response?.data || error.message); + return Promise.reject(error); + } +); + +// Playlist API calls +export const playlistAPI = { + getAll: () => api.get('/playlists'), + getById: (id: string) => api.get(`/playlists/${id}`), + create: (data: any) => api.post('/playlists', data), + update: (id: string, data: any) => api.put(`/playlists/${id}`, data), + delete: (id: string) => api.delete(`/playlists/${id}`), + check: (id: string) => api.post(`/playlists/${id}/check`), +}; + +// Video API calls +export const videoAPI = { + getById: (id: string) => api.get(`/videos/${id}`), + download: (id: string) => api.post(`/videos/${id}/download`), + skip: (id: string) => api.post(`/videos/${id}/skip`), + reset: (id: string) => api.post(`/videos/${id}/reset`), + markAsMoved: (id: string, locationNote?: string) => + api.post(`/videos/${id}/file-moved`, { location_note: locationNote }), +}; + +// System API calls +export const systemAPI = { + getStatus: () => api.get('/status'), + getSchedulerStatus: () => api.get('/scheduler/status'), + syncWithMeTube: () => api.post('/sync-metube'), + healthCheck: () => api.get('/health'), +}; +``` + +### Main Dashboard Component +```typescript +// src/pages/Dashboard.tsx +import React, { useState, useEffect } from 'react'; +import { + Container, + Grid, + Card, + CardContent, + Typography, + Box, + LinearProgress, + Alert +} from '@mui/material'; +import { + PlaylistPlay, + VideoLibrary, + Download, + Error +} from '@mui/icons-material'; +import { systemAPI } from '../services/api'; +import StatusChart from '../components/StatusChart'; + +interface SystemStatus { + total_playlists: number; + active_playlists: number; + total_videos: number; + pending_downloads: number; + active_downloads: number; + completed_downloads: number; + failed_downloads: number; + metube_status: { + connected: boolean; + error?: string; + }; +} + +const Dashboard: React.FC = () => { + const [status, setStatus] = useState(null); + const [loading, setLoading] = useState(true); + const [error, setError] = useState(null); + + useEffect(() => { + loadStatus(); + const interval = setInterval(loadStatus, 30000); // Refresh every 30s + return () => clearInterval(interval); + }, []); + + const loadStatus = async () => { + try { + const response = await systemAPI.getStatus(); + setStatus(response.data); + setError(null); + } catch (err) { + setError('Failed to load system status'); + console.error('Error loading status:', err); + } finally { + setLoading(false); + } + }; + + if (loading) return ; + if (error) return {error}; + if (!status) return No data available; + + const stats = [ + { label: 'Total Playlists', value: status.total_playlists, icon: }, + { label: 'Active Playlists', value: status.active_playlists, icon: }, + { label: 'Total Videos', value: status.total_videos, icon: }, + { label: 'Pending Downloads', value: status.pending_downloads, icon: }, + { label: 'Active Downloads', value: status.active_downloads, icon: }, + { label: 'Completed', value: status.completed_downloads, icon: }, + { label: 'Failed', value: status.failed_downloads, icon: }, + ]; + + return ( + + + {/* Status Alert */} + {!status.metube_status.connected && ( + + + MeTube connection failed: {status.metube_status.error} + + + )} + + {/* Stats Cards */} + {stats.map((stat, index) => ( + + + + + {stat.icon} + + {stat.label} + + + + {stat.value} + + + + + ))} + + {/* Status Chart */} + + + + + Video Status Distribution + + + + + + + + ); +}; + +export default Dashboard; +``` + +### Playlist Management Component +```typescript +// src/pages/Playlists.tsx +import React, { useState, useEffect } from 'react'; +import { + Container, + Grid, + Card, + CardContent, + Typography, + Button, + Dialog, + DialogTitle, + DialogContent, + DialogActions, + TextField, + Box, + Chip, + IconButton, + Alert, +} from '@mui/material'; +import { + Add, + PlayArrow, + Delete, + Edit, + Refresh, + CheckCircle, + PauseCircle, +} from '@mui/icons-material'; +import { useQuery, useMutation, useQueryClient } from 'react-query'; +import { playlistAPI } from '../services/api'; + +interface Playlist { + id: string; + url: string; + title?: string; + check_interval: number; + enabled: boolean; + created_at: string; + stats: { + total: number; + pending: number; + completed: number; + failed: number; + }; +} + +const Playlists: React.FC = () => { + const queryClient = useQueryClient(); + const [addDialog, setAddDialog] = useState(false); + const [newPlaylist, setNewPlaylist] = useState({ + url: '', + title: '', + check_interval: 60, + quality: 'best', + format: 'mp4', + folder: '', + enabled: true, + }); + + // Fetch playlists + const { data: playlists, isLoading, error } = useQuery( + 'playlists', + () => playlistAPI.getAll().then(res => res.data), + { + refetchInterval: 30000, // Refresh every 30s + } + ); + + // Create playlist mutation + const createMutation = useMutation( + (data: any) => playlistAPI.create(data), + { + onSuccess: () => { + queryClient.invalidateQueries('playlists'); + setAddDialog(false); + setNewPlaylist({ + url: '', + title: '', + check_interval: 60, + quality: 'best', + format: 'mp4', + folder: '', + enabled: true, + }); + }, + } + ); + + // Check playlist mutation + const checkMutation = useMutation( + (id: string) => playlistAPI.check(id), + { + onSuccess: () => { + queryClient.invalidateQueries('playlists'); + }, + } + ); + + // Delete playlist mutation + const deleteMutation = useMutation( + (id: string) => playlistAPI.delete(id), + { + onSuccess: () => { + queryClient.invalidateQueries('playlists'); + }, + } + ); + + const handleAddPlaylist = () => { + createMutation.mutate(newPlaylist); + }; + + const handleCheckPlaylist = (id: string) => { + checkMutation.mutate(id); + }; + + const handleDeletePlaylist = (id: string) => { + if (window.confirm('Are you sure you want to delete this playlist?')) { + deleteMutation.mutate(id); + } + }; + + if (isLoading) return
Loading...
; + if (error) return Failed to load playlists; + + return ( + + + + Playlists + + + + + + {playlists?.map((playlist: Playlist) => ( + + + + + + {playlist.title || 'Untitled Playlist'} + + : } + label={playlist.enabled ? 'Enabled' : 'Disabled'} + color={playlist.enabled ? 'success' : 'default'} + size="small" + /> + + + + {playlist.url} + + + + + Check Interval: {playlist.check_interval} minutes + + + Total Videos: {playlist.stats.total} + + + Pending: {playlist.stats.pending} | Completed: {playlist.stats.completed} + + + + + handleCheckPlaylist(playlist.id)} + disabled={checkMutation.isLoading} + title="Check for new videos" + > + + + handleDeletePlaylist(playlist.id)} + disabled={deleteMutation.isLoading} + title="Delete playlist" + > + + + + + + + ))} + + + {/* Add Playlist Dialog */} + setAddDialog(false)} maxWidth="sm" fullWidth> + Add New Playlist + + setNewPlaylist({ ...newPlaylist, url: e.target.value })} + margin="normal" + placeholder="https://www.youtube.com/playlist?list=..." + /> + setNewPlaylist({ ...newPlaylist, title: e.target.value })} + margin="normal" + /> + setNewPlaylist({ ...newPlaylist, check_interval: parseInt(e.target.value) })} + margin="normal" + /> + setNewPlaylist({ ...newPlaylist, folder: e.target.value })} + margin="normal" + /> + + + + + + + + ); +}; + +export default Playlists; +``` + +## ๐Ÿ”„ Option 2: Extend MeTube Angular + +### Integration Approach +Since MeTube uses Angular, you can extend its existing UI: + +```typescript +// Add to MeTube's existing structure +metube/ui/src/app/ +โ”œโ”€โ”€ playlist-monitor/ # New module +โ”‚ โ”œโ”€โ”€ playlist-monitor.module.ts +โ”‚ โ”œโ”€โ”€ components/ +โ”‚ โ”‚ โ”œโ”€โ”€ playlist-list/ +โ”‚ โ”‚ โ”œโ”€โ”€ playlist-detail/ +โ”‚ โ”‚ โ””โ”€โ”€ video-dashboard/ +โ”‚ โ”œโ”€โ”€ services/ +โ”‚ โ”‚ โ””โ”€โ”€ playlist-monitor.service.ts +โ”‚ โ””โ”€โ”€ models/ +โ”‚ โ””โ”€โ”€ playlist.model.ts +``` + +### Service Integration +```typescript +// metube/ui/src/app/playlist-monitor/services/playlist-monitor.service.ts +import { Injectable } from '@angular/core'; +import { HttpClient } from '@angular/common/http'; +import { Observable } from 'rxjs'; + +@Injectable({ + providedIn: 'root' +}) +export class PlaylistMonitorService { + private apiUrl = 'http://localhost:8082/api'; + + constructor(private http: HttpClient) {} + + getPlaylists(): Observable { + return this.http.get(`${this.apiUrl}/playlists`); + } + + createPlaylist(playlist: any): Observable { + return this.http.post(`${this.apiUrl}/playlists`, playlist); + } + + checkPlaylist(id: string): Observable { + return this.http.post(`${this.apiUrl}/playlists/${id}/check`, {}); + } +} +``` + +### Component Integration +```typescript +// Add to MeTube's navigation +// metube/ui/src/app/app.component.html + +``` + +## ๐Ÿ“ Option 3: Simple HTML Dashboard + +### Basic HTML Dashboard +```html + + + + + + Playlist Monitor Dashboard + + + + + + + +
+ +
+
+
+
+ +
Loading...
+

Total Playlists

+
+
+
+
+ + +
+
+
Playlists
+ +
+
+
+ + + + + + + + + + + + + + + +
TitleURLStatusVideosActions
Loading playlists...
+
+
+
+
+ + + + + + + + +``` + +### JavaScript for Simple Dashboard +```javascript +// dashboard.js +const API_BASE_URL = 'http://localhost:8082/api'; + +// API helper functions +async function apiCall(endpoint, options = {}) { + try { + const response = await fetch(`${API_BASE_URL}${endpoint}`, { + headers: { + 'Content-Type': 'application/json', + }, + ...options + }); + + if (!response.ok) { + throw new Error(`HTTP ${response.status}: ${response.statusText}`); + } + + return await response.json(); + } catch (error) { + console.error('API call failed:', error); + throw error; + } +} + +// Load system status +async function loadStatus() { + try { + const status = await apiCall('/status'); + updateStatusCards(status); + } catch (error) { + showError('Failed to load system status'); + } +} + +// Update status cards +function updateStatusCards(status) { + const statusCards = document.getElementById('status-cards'); + statusCards.innerHTML = ` +
+
+
+ +

${status.total_playlists}

+

Total Playlists

+
+
+
+
+
+
+ +

${status.active_playlists}

+

Active Playlists

+
+
+
+
+
+
+ +

${status.total_videos}

+

Total Videos

+
+
+
+
+
+
+ +

${status.pending_downloads}

+

Pending Downloads

+
+
+
+ `; +} + +// Load playlists +async function loadPlaylists() { + try { + const playlists = await apiCall('/playlists'); + updatePlaylistsTable(playlists); + } catch (error) { + showError('Failed to load playlists'); + } +} + +// Update playlists table +function updatePlaylistsTable(playlists) { + const tbody = document.querySelector('#playlists-table tbody'); + + if (playlists.length === 0) { + tbody.innerHTML = 'No playlists found'; + return; + } + + tbody.innerHTML = playlists.map(playlist => ` + + ${playlist.title || 'Untitled'} + ${playlist.url} + + + ${playlist.enabled ? 'Enabled' : 'Disabled'} + + + + ${playlist.stats.total} + + (${playlist.stats.pending} pending, ${playlist.stats.completed} completed) + + + + + + + + `).join(''); +} + +// Add playlist +async function addPlaylist() { + const url = document.getElementById('playlist-url').value; + const title = document.getElementById('playlist-title').value; + const checkInterval = parseInt(document.getElementById('check-interval').value); + + if (!url) { + alert('Please enter a YouTube playlist URL'); + return; + } + + try { + await apiCall('/playlists', { + method: 'POST', + body: JSON.stringify({ + url: url, + title: title || undefined, + check_interval: checkInterval, + enabled: true + }) + }); + + // Close modal and refresh + bootstrap.Modal.getInstance(document.getElementById('addPlaylistModal')).hide(); + refreshData(); + showSuccess('Playlist added successfully'); + } catch (error) { + showError('Failed to add playlist'); + } +} + +// Check playlist +async function checkPlaylist(id) { + try { + await apiCall(`/playlists/${id}/check`, { method: 'POST' }); + showSuccess('Playlist check initiated'); + refreshData(); + } catch (error) { + showError('Failed to check playlist'); + } +} + +// UI helper functions +function showAddPlaylistModal() { + new bootstrap.Modal(document.getElementById('addPlaylistModal')).show(); +} + +function showError(message) { + // Simple error display - you could enhance this with toast notifications + alert(`Error: ${message}`); +} + +function showSuccess(message) { + alert(`Success: ${message}`); +} + +function refreshData() { + loadStatus(); + loadPlaylists(); +} + +// Initialize dashboard +document.addEventListener('DOMContentLoaded', function() { + refreshData(); + // Auto-refresh every 30 seconds + setInterval(refreshData, 30000); +}); +``` + +## ๐Ÿ”— API Integration Examples + +### React Hook for Real-time Updates +```typescript +// src/hooks/useRealtimeUpdates.ts +import { useEffect, useRef } from 'react'; +import { systemAPI } from '../services/api'; + +export const useRealtimeUpdates = (onUpdate: (data: any) => void) => { + const wsRef = useRef(null); + + useEffect(() => { + // Connect to WebSocket for real-time updates + const connectWebSocket = () => { + const ws = new WebSocket('ws://localhost:8082/ws'); + + ws.onopen = () => { + console.log('WebSocket connected'); + }; + + ws.onmessage = (event) => { + const data = JSON.parse(event.data); + onUpdate(data); + }; + + ws.onclose = () => { + console.log('WebSocket disconnected'); + // Reconnect after 5 seconds + setTimeout(connectWebSocket, 5000); + }; + + ws.onerror = (error) => { + console.error('WebSocket error:', error); + }; + + wsRef.current = ws; + }; + + connectWebSocket(); + + return () => { + if (wsRef.current) { + wsRef.current.close(); + } + }; + }, [onUpdate]); + + return wsRef.current; +}; +``` + +## ๐ŸŽจ UI Components Design + +### Key Components Needed +1. **Dashboard** - System overview with charts and statistics +2. **Playlist List** - Table/card view of all playlists +3. **Playlist Detail** - Individual playlist management +4. **Video Table** - Video status and management +5. **Add/Edit Forms** - Playlist creation and editing +6. **Status Indicators** - Real-time status updates +7. **Charts/Graphs** - Statistics visualization + +### Design System Integration +```typescript +// src/theme.ts +import { createTheme } from '@mui/material/styles'; + +// Custom theme matching MeTube's style +export const theme = createTheme({ + palette: { + primary: { + main: '#1976d2', // Blue matching MeTube + }, + secondary: { + main: '#dc004e', + }, + background: { + default: '#f5f5f5', + }, + }, + typography: { + fontFamily: '"Roboto", "Helvetica", "Arial", sans-serif', + }, + components: { + MuiCard: { + styleOverrides: { + root: { + boxShadow: '0 2px 4px rgba(0,0,0,0.1)', + borderRadius: 8, + }, + }, + }, + }, +}); +``` + +## ๐Ÿ“ฑ Responsive Design + +### Mobile-First Approach +```typescript +// Responsive grid breakpoints + + + + {/* Card content */} + + + + +// Responsive typography + + Playlist Title + +``` + +## ๐Ÿงช Testing UI + +### Component Testing with React Testing Library +```typescript +// src/components/__tests__/PlaylistCard.test.tsx +import { render, screen, fireEvent } from '@testing-library/react'; +import PlaylistCard from '../PlaylistCard'; + +describe('PlaylistCard', () => { + const mockPlaylist = { + id: '1', + title: 'Test Playlist', + url: 'https://www.youtube.com/playlist?list=TEST123', + enabled: true, + stats: { total: 10, pending: 2, completed: 8 } + }; + + it('renders playlist information correctly', () => { + render(); + + expect(screen.getByText('Test Playlist')).toBeInTheDocument(); + expect(screen.getByText('10 videos')).toBeInTheDocument(); + }); + + it('calls onCheck when check button is clicked', () => { + const onCheck = jest.fn(); + render(); + + fireEvent.click(screen.getByLabelText('Check playlist')); + expect(onCheck).toHaveBeenCalledWith('1'); + }); +}); +``` + +## ๐Ÿš€ Deployment Options + +### Option 1: Serve with Backend +```typescript +// Modify app/main.py to serve static files +from fastapi.staticfiles import StaticFiles + +app.mount("/ui", StaticFiles(directory="ui/build", html=True), name="ui") + +# Add redirect +@app.get("/") +async def serve_ui(): + return RedirectResponse(url="/ui") +``` + +### Option 2: Separate Deployment +```bash +# Build React app +npm run build + +# Deploy to static hosting +# - Netlify +# - Vercel +# - GitHub Pages +# - Nginx +``` + +### Option 3: Docker Multi-Service +```dockerfile +# Dockerfile for React app +FROM node:18-alpine as build +WORKDIR /app +COPY package*.json ./ +RUN npm ci --only=production +COPY . . +RUN npm run build + +FROM nginx:alpine +COPY --from=build /app/build /usr/share/nginx/html +COPY nginx.conf /etc/nginx/nginx.conf +EXPOSE 80 +CMD ["nginx", "-g", "daemon off;"] +``` + +## ๐Ÿ“š Next Steps + +1. **Choose UI Option** based on your needs and technical requirements +2. **Implement Core Components** following the examples above +3. **Add Real-time Updates** with WebSocket integration +4. **Test Thoroughly** with different screen sizes and browsers +5. **Deploy** using one of the suggested deployment options + +The UI implementation will provide a complete, user-friendly interface for managing your playlists and monitoring download progress! ๐ŸŽจโœจ \ No newline at end of file