87 lines
4.4 KiB
Markdown
87 lines
4.4 KiB
Markdown
Project Description:
|
|
This is a nextjs project, basically a youtube like video sites.
|
|
the tailwindcss is v3 version. must ensure all the css related tailwind sytling code should comply with the v3 standard
|
|
|
|
Feature requirement:
|
|
1. Has a youtube like UI
|
|
2. has a concept of media library. each library is a path that mounted in the /mnt
|
|
3. has the ability to scan the media libaries
|
|
4. the quntitiy of media files can be vast, tens of thousands videos
|
|
5. the media can work with photos and videos
|
|
6. user can manage(add/remove) the media libraries.
|
|
7. there should be a database(sqlite) to save the media informations
|
|
8. there should be at least two tables in the database, one to keep the video informations, such as path, size, thumbnails, title, etc(not limit to these); one to keep the media libraries. videos should be linked to the library
|
|
9. thumbnail generate feature
|
|
10. bookmark feature: the video can be bookmarked
|
|
11. star feature: the video can be stared as one to five stars
|
|
|
|
|
|
UI:
|
|
1. two main areas, left is the left sidebar, right is the main video area
|
|
2. side bar can be expanded and collapsed
|
|
3. sidebar should have these sections: Settings, Videos, Photos, Folder Viewer
|
|
4. Settings section: open manage page in the main area, allow user to add media library path; delete library.
|
|
5. Videos section: open video cards page in the main area, each card has thumbnail pic displayed. the card lower part display video path, size.
|
|
5.1. the video card has 2 part, 80% of the upper area shows the thumbnail of the video, 20% of the lower part shows the video path, size in GB/MB, depends on the actual size of the video.
|
|
6. photo section: TBD
|
|
7. folder viewer: list libraries in the side bar folder viewer section. once one of the folder is selected, display the folder sturcture in the main area. the vdieo or photo display thunbnail and information as the video card would do. for the folder, display the folder icon, which can be entered in
|
|
8. the video card can be clicked, once clicked, a poped up video player will be displayed. it can be closed, fast forward, expand to full screen, etc.
|
|
9. can bookmark/un-bookmark the video, can star the video
|
|
|
|
Performance Optimization Plan
|
|
|
|
Phase 1: Critical API Pagination (Immediate Fix)
|
|
Implement database pagination across all list APIs
|
|
Add limit/offset parameters to /api/videos, /api/photos, /api/bookmarks
|
|
Add server-side filtering and sorting
|
|
Implement cursor-based pagination for better performance
|
|
Add database indexes for pagination queries
|
|
Create compound indexes for (type, created_at) queries
|
|
Add path-based indexes for folder-viewer queries
|
|
Optimize bookmark/star count queries
|
|
|
|
Phase 2: Frontend Memory Optimization
|
|
Implement virtual scrolling for large lists
|
|
Use react-window or react-virtualized for video/photo grids
|
|
Add infinite scroll with intersection observer
|
|
Implement client-side caching with LRU eviction
|
|
Add progressive loading strategies
|
|
Lazy load thumbnails as they come into view
|
|
Implement skeleton loaders during data fetching
|
|
Add debounced search with server-side filtering
|
|
|
|
Phase 3: File System Scanning Optimization
|
|
Parallel processing implementation
|
|
Use worker threads for thumbnail generation
|
|
Implement batch processing for database inserts
|
|
Add progress reporting with WebSocket/SSE
|
|
Smart scanning strategies
|
|
Implement incremental scanning (only new/changed files)
|
|
Add file watching for real-time updates
|
|
Use streaming file discovery instead of loading all paths
|
|
|
|
Phase 4: Database Performance
|
|
Connection pooling and optimization
|
|
Implement better-sqlite3 connection pooling
|
|
Add prepared statement caching
|
|
Implement batch operations for inserts/updates
|
|
Advanced indexing strategy
|
|
Full-text search indexes for title/path searching
|
|
Composite indexes for common query patterns
|
|
Materialized views for aggregated data (ratings, bookmarks)
|
|
|
|
Phase 5: Caching & CDN Strategy
|
|
Multi-level caching
|
|
Redis for API response caching
|
|
Browser caching with ETags
|
|
CDN for thumbnail delivery
|
|
Thumbnail optimization
|
|
WebP format with fallbacks
|
|
Multiple thumbnail sizes for different viewports
|
|
Lazy generation with background processing
|
|
Implementation Priority
|
|
P0 (Critical) : API pagination + frontend virtual scrolling
|
|
P1 (High) : Database indexes + connection optimization
|
|
P2 (Medium) : File scanning improvements
|
|
P3 (Low) : Advanced caching + CDN
|
|
This plan addresses all identified bottlenecks systematically, starting with the most critical issues that would cause immediate system failure with large datasets. |