Initial commit: TiDB local development environment with DM
- Minimal TiDB setup in standalone mode (no PD/TiKV needed) - TiDB Data Migration (DM) for continuous sync from test env - Automated initialization and configuration - Docker Compose v2 support - Health checks and proper service dependencies - DataGrip/MySQL client connectivity - Comprehensive documentation and helper scripts Features: - Auto-sync data from test environment to local TiDB - Multiple tables support via environment variables - Easy sync control (start/stop/pause/resume) - Connection testing and status monitoring - Troubleshooting guide Scripts: - start.sh: Quick start environment - status.sh: Check sync status - sync-control.sh: Manage sync tasks - test-connection.sh: Verify TiDB connectivity - check-docker.sh: Verify Docker setup Documentation: - README.md: Main setup guide - SYNC_GUIDE.md: Data synchronization guide - DATAGRIP_SETUP.md: Database client setup - DOCKER_COMPOSE_V2.md: Migration guide - TROUBLESHOOTING.md: Common issues and solutions
This commit is contained in:
commit
1938d26462
|
|
@ -0,0 +1,9 @@
|
||||||
|
# Test TiDB connection
|
||||||
|
TEST_DB_HOST=your-test-tidb-host
|
||||||
|
TEST_DB_PORT=4000
|
||||||
|
TEST_DB_USER=root
|
||||||
|
TEST_DB_PASSWORD=your-password
|
||||||
|
|
||||||
|
# Database and tables to sync
|
||||||
|
DATABASE_NAME=your_database
|
||||||
|
TABLES="table1,table2,table3"
|
||||||
|
|
@ -0,0 +1,11 @@
|
||||||
|
# Environment variables with credentials
|
||||||
|
.env
|
||||||
|
|
||||||
|
# Docker volumes (if you're using local bind mounts)
|
||||||
|
data/
|
||||||
|
|
||||||
|
# Logs
|
||||||
|
*.log
|
||||||
|
|
||||||
|
# macOS
|
||||||
|
.DS_Store
|
||||||
|
|
@ -0,0 +1,182 @@
|
||||||
|
# DataGrip / MySQL Client Setup
|
||||||
|
|
||||||
|
## TiDB Connection Details
|
||||||
|
|
||||||
|
Your local TiDB instance is ready for DataGrip or any MySQL-compatible client!
|
||||||
|
|
||||||
|
### Connection Settings
|
||||||
|
|
||||||
|
| Setting | Value |
|
||||||
|
|---------|-------|
|
||||||
|
| **Host** | `127.0.0.1` or `localhost` |
|
||||||
|
| **Port** | `4000` |
|
||||||
|
| **User** | `root` |
|
||||||
|
| **Password** | _(leave empty)_ |
|
||||||
|
| **Database** | _(optional, or specify your synced database)_ |
|
||||||
|
|
||||||
|
## DataGrip Configuration
|
||||||
|
|
||||||
|
### 1. Create New Data Source
|
||||||
|
|
||||||
|
1. Open DataGrip
|
||||||
|
2. Click **+** (New) → **Data Source** → **MySQL**
|
||||||
|
3. Enter connection details:
|
||||||
|
- **Host:** `127.0.0.1`
|
||||||
|
- **Port:** `4000`
|
||||||
|
- **Authentication:** User & Password
|
||||||
|
- **User:** `root`
|
||||||
|
- **Password:** _(leave empty)_
|
||||||
|
- **Database:** `your_database` (or leave empty to see all databases)
|
||||||
|
|
||||||
|
### 2. Driver Configuration
|
||||||
|
|
||||||
|
- DataGrip will automatically use the MySQL driver
|
||||||
|
- TiDB is MySQL protocol-compatible, so no special driver needed
|
||||||
|
- If prompted, download the MySQL JDBC driver
|
||||||
|
|
||||||
|
### 3. Test Connection
|
||||||
|
|
||||||
|
Click **Test Connection** to verify:
|
||||||
|
- ✅ Should show "Successful" if TiDB is running
|
||||||
|
- ❌ If failed, ensure TiDB is running: `docker ps | grep tidb`
|
||||||
|
|
||||||
|
### 4. Advanced Settings (Optional)
|
||||||
|
|
||||||
|
**URL Template:**
|
||||||
|
```
|
||||||
|
jdbc:mysql://127.0.0.1:4000
|
||||||
|
```
|
||||||
|
|
||||||
|
**Connection Properties (optional):**
|
||||||
|
- `useSSL=false` (for local development)
|
||||||
|
- `serverTimezone=UTC`
|
||||||
|
- `allowPublicKeyRetrieval=true`
|
||||||
|
|
||||||
|
Full URL example:
|
||||||
|
```
|
||||||
|
jdbc:mysql://127.0.0.1:4000?useSSL=false&serverTimezone=UTC
|
||||||
|
```
|
||||||
|
|
||||||
|
## Other MySQL Clients
|
||||||
|
|
||||||
|
### MySQL Command Line
|
||||||
|
```bash
|
||||||
|
mysql -h 127.0.0.1 -P 4000 -u root
|
||||||
|
```
|
||||||
|
|
||||||
|
### MySQL Workbench
|
||||||
|
- Connection Method: Standard TCP/IP
|
||||||
|
- Hostname: `127.0.0.1`
|
||||||
|
- Port: `4000`
|
||||||
|
- Username: `root`
|
||||||
|
- Password: _(leave empty)_
|
||||||
|
|
||||||
|
### DBeaver
|
||||||
|
- Database: MySQL
|
||||||
|
- Server Host: `127.0.0.1`
|
||||||
|
- Port: `4000`
|
||||||
|
- Username: `root`
|
||||||
|
- Password: _(leave empty)_
|
||||||
|
|
||||||
|
### TablePlus
|
||||||
|
- Connection Type: MySQL
|
||||||
|
- Host: `127.0.0.1`
|
||||||
|
- Port: `4000`
|
||||||
|
- User: `root`
|
||||||
|
- Password: _(leave empty)_
|
||||||
|
|
||||||
|
## Verifying Your Synced Data
|
||||||
|
|
||||||
|
Once connected, run these queries to check your synced data:
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- Show all databases
|
||||||
|
SHOW DATABASES;
|
||||||
|
|
||||||
|
-- Switch to your synced database
|
||||||
|
USE your_database;
|
||||||
|
|
||||||
|
-- Show tables
|
||||||
|
SHOW TABLES;
|
||||||
|
|
||||||
|
-- Check table data
|
||||||
|
SELECT * FROM table1 LIMIT 10;
|
||||||
|
|
||||||
|
-- Check table structure
|
||||||
|
DESCRIBE table1;
|
||||||
|
|
||||||
|
-- Show table row count
|
||||||
|
SELECT COUNT(*) FROM table1;
|
||||||
|
```
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
### Connection Refused
|
||||||
|
```bash
|
||||||
|
# Check if TiDB is running
|
||||||
|
docker ps | grep tidb
|
||||||
|
|
||||||
|
# Check TiDB logs
|
||||||
|
docker logs tidb
|
||||||
|
|
||||||
|
# Restart if needed
|
||||||
|
docker compose restart tidb
|
||||||
|
```
|
||||||
|
|
||||||
|
### Can't See Synced Data
|
||||||
|
```bash
|
||||||
|
# Check DM sync status
|
||||||
|
./status.sh
|
||||||
|
|
||||||
|
# Or manually:
|
||||||
|
docker exec dm-master /dmctl --master-addr=dm-master:8261 query-status test-to-local
|
||||||
|
```
|
||||||
|
|
||||||
|
### Port Already in Use
|
||||||
|
If port 4000 is already taken, you can change it in `docker-compose.yml`:
|
||||||
|
```yaml
|
||||||
|
services:
|
||||||
|
tidb:
|
||||||
|
ports:
|
||||||
|
- "14000:4000" # Change host port to 14000
|
||||||
|
```
|
||||||
|
Then connect to `127.0.0.1:14000` instead.
|
||||||
|
|
||||||
|
## Quick Verification
|
||||||
|
|
||||||
|
After starting your environment and connecting with DataGrip:
|
||||||
|
|
||||||
|
1. **Check if sync is working:**
|
||||||
|
```sql
|
||||||
|
-- In DataGrip, run:
|
||||||
|
SELECT DATABASE();
|
||||||
|
SHOW DATABASES LIKE 'your_database';
|
||||||
|
```
|
||||||
|
|
||||||
|
2. **Compare record counts:**
|
||||||
|
```sql
|
||||||
|
-- In DataGrip (local TiDB)
|
||||||
|
SELECT COUNT(*) FROM your_database.table1;
|
||||||
|
|
||||||
|
-- Compare with test environment to verify sync
|
||||||
|
```
|
||||||
|
|
||||||
|
3. **Check data freshness:**
|
||||||
|
```sql
|
||||||
|
-- If your table has timestamps
|
||||||
|
SELECT MAX(updated_at) FROM your_database.table1;
|
||||||
|
```
|
||||||
|
|
||||||
|
## Tips for DataGrip
|
||||||
|
|
||||||
|
- **Auto-sync schema:** Right-click database → Synchronize to refresh
|
||||||
|
- **Multiple connections:** You can add both test and local TiDB for comparison
|
||||||
|
- **Query console:** Use separate consoles for different environments
|
||||||
|
- **Data comparison:** Use DataGrip's compare feature to verify sync accuracy
|
||||||
|
|
||||||
|
## Security Note
|
||||||
|
|
||||||
|
⚠️ This setup uses no password for local development convenience. For production:
|
||||||
|
- Set a strong password for TiDB
|
||||||
|
- Enable SSL/TLS
|
||||||
|
- Use proper network isolation
|
||||||
|
|
@ -0,0 +1,97 @@
|
||||||
|
# Docker Compose v2 Migration Guide
|
||||||
|
|
||||||
|
This project uses **Docker Compose v2** (the newer version integrated into Docker CLI).
|
||||||
|
|
||||||
|
## What's the Difference?
|
||||||
|
|
||||||
|
| Version | Command | Status |
|
||||||
|
|---------|---------|--------|
|
||||||
|
| v1 | `docker-compose` | Legacy (standalone binary) |
|
||||||
|
| v2 | `docker compose` | Current (built into Docker) |
|
||||||
|
|
||||||
|
**Key change:** The command changed from `docker-compose` (with hyphen) to `docker compose` (with space).
|
||||||
|
|
||||||
|
## Check Your Version
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Check if you have Docker Compose v2
|
||||||
|
docker compose version
|
||||||
|
|
||||||
|
# Check if you have old v1
|
||||||
|
docker-compose --version
|
||||||
|
```
|
||||||
|
|
||||||
|
## If You Have v1 Only
|
||||||
|
|
||||||
|
### Option 1: Upgrade to Docker Compose v2 (Recommended)
|
||||||
|
|
||||||
|
**macOS with OrbStack:**
|
||||||
|
```bash
|
||||||
|
# OrbStack includes Docker Compose v2 by default
|
||||||
|
# Just make sure OrbStack is up to date
|
||||||
|
orb update
|
||||||
|
```
|
||||||
|
|
||||||
|
**macOS with Docker Desktop:**
|
||||||
|
```bash
|
||||||
|
# Update Docker Desktop to latest version
|
||||||
|
# Docker Compose v2 is included since Docker Desktop 3.4.0
|
||||||
|
```
|
||||||
|
|
||||||
|
**Manual Installation:**
|
||||||
|
```bash
|
||||||
|
# Install as Docker CLI plugin
|
||||||
|
mkdir -p ~/.docker/cli-plugins
|
||||||
|
curl -SL https://github.com/docker/compose/releases/latest/download/docker-compose-darwin-aarch64 -o ~/.docker/cli-plugins/docker-compose
|
||||||
|
chmod +x ~/.docker/cli-plugins/docker-compose
|
||||||
|
```
|
||||||
|
|
||||||
|
### Option 2: Create an Alias (Quick Fix)
|
||||||
|
|
||||||
|
If you can't upgrade immediately, create an alias:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Add to your ~/.zshrc or ~/.bashrc
|
||||||
|
echo "alias docker-compose='docker compose'" >> ~/.zshrc
|
||||||
|
source ~/.zshrc
|
||||||
|
```
|
||||||
|
|
||||||
|
### Option 3: Use Docker Compose v1 Syntax (Not Recommended)
|
||||||
|
|
||||||
|
Manually replace all `docker compose` commands with `docker-compose` in:
|
||||||
|
- `start.sh`
|
||||||
|
- `README.md`
|
||||||
|
- `DATAGRIP_SETUP.md`
|
||||||
|
|
||||||
|
## Why We Use v2
|
||||||
|
|
||||||
|
1. **Built-in**: No separate installation needed
|
||||||
|
2. **Better Performance**: Faster and more efficient
|
||||||
|
3. **Active Development**: v1 is in maintenance mode only
|
||||||
|
4. **Same YAML**: Your `docker-compose.yml` works with both versions
|
||||||
|
|
||||||
|
## Verify Your Setup
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./check-docker.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
This will check your Docker and Docker Compose installation.
|
||||||
|
|
||||||
|
## Common Issues
|
||||||
|
|
||||||
|
### "docker: 'compose' is not a docker command"
|
||||||
|
|
||||||
|
You have Docker but not Docker Compose v2. Solutions:
|
||||||
|
- Update Docker Desktop
|
||||||
|
- Install Docker Compose v2 plugin (see above)
|
||||||
|
- Use the alias workaround
|
||||||
|
|
||||||
|
### Both v1 and v2 installed
|
||||||
|
|
||||||
|
No problem! Both can coexist. Just use `docker compose` (v2) for this project.
|
||||||
|
|
||||||
|
## References
|
||||||
|
|
||||||
|
- [Docker Compose v2 Documentation](https://docs.docker.com/compose/cli-command/)
|
||||||
|
- [Migration Guide](https://docs.docker.com/compose/migrate/)
|
||||||
|
|
@ -0,0 +1,257 @@
|
||||||
|
# TiDB Local Development Environment
|
||||||
|
|
||||||
|
A minimal TiDB instance with Data Migration (DM) for syncing data from test environments.
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
```
|
||||||
|
┌─────────────────────────────────────────────────────────────┐
|
||||||
|
│ Your macOS (OrbStack) │
|
||||||
|
│ │
|
||||||
|
│ ┌──────────────┐ ┌──────────────┐ │
|
||||||
|
│ │ DataGrip │──────▶│ TiDB │ │
|
||||||
|
│ │ (port 4000) │ │ (Standalone) │ │
|
||||||
|
│ └──────────────┘ └───────▲───────┘ │
|
||||||
|
│ │ │
|
||||||
|
│ ┌───────┴────────┐ │
|
||||||
|
│ │ DM Worker │ │
|
||||||
|
│ │ (Sync Engine) │ │
|
||||||
|
│ └───────▲────────┘ │
|
||||||
|
│ │ │
|
||||||
|
│ ┌───────┴────────┐ │
|
||||||
|
│ │ DM Master │ │
|
||||||
|
│ │ (Orchestrator) │ │
|
||||||
|
│ └───────▲────────┘ │
|
||||||
|
│ │ │
|
||||||
|
└────────────────────────────────┼─────────────────────────────┘
|
||||||
|
│
|
||||||
|
(Continuous Sync)
|
||||||
|
│
|
||||||
|
┌────────────▼─────────────┐
|
||||||
|
│ Test TiDB Instance │
|
||||||
|
│ (Remote Environment) │
|
||||||
|
└──────────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
**Components:**
|
||||||
|
- **TiDB (Standalone Mode)**: Runs with embedded storage (unistore), no separate PD/TiKV needed
|
||||||
|
- **DM Master**: Manages data migration tasks
|
||||||
|
- **DM Worker**: Executes the actual data synchronization from test to local
|
||||||
|
- **DataGrip/MySQL Clients**: Connect directly to TiDB on port 4000
|
||||||
|
|
||||||
|
## Quick Reference
|
||||||
|
|
||||||
|
### Connection Info
|
||||||
|
| Service | Host | Port | User | Password |
|
||||||
|
|---------|------|------|------|----------|
|
||||||
|
| TiDB | `127.0.0.1` | `4000` | `root` | _(empty)_ |
|
||||||
|
| DM Master | `127.0.0.1` | `8261` | - | - |
|
||||||
|
|
||||||
|
### Useful Commands
|
||||||
|
```bash
|
||||||
|
# Start environment (auto-starts sync)
|
||||||
|
./start.sh
|
||||||
|
|
||||||
|
# Test connection
|
||||||
|
./test-connection.sh
|
||||||
|
|
||||||
|
# Check sync status
|
||||||
|
./status.sh
|
||||||
|
# or use the sync control script:
|
||||||
|
./sync-control.sh status
|
||||||
|
|
||||||
|
# Control sync task
|
||||||
|
./sync-control.sh [start|stop|pause|resume|restart|reinit]
|
||||||
|
|
||||||
|
# Connect with MySQL client
|
||||||
|
mysql -h 127.0.0.1 -P 4000 -u root
|
||||||
|
|
||||||
|
# View logs
|
||||||
|
docker compose logs -f
|
||||||
|
|
||||||
|
# Stop environment
|
||||||
|
docker compose down
|
||||||
|
```
|
||||||
|
|
||||||
|
For DataGrip/DBeaver setup, see [DATAGRIP_SETUP.md](DATAGRIP_SETUP.md)
|
||||||
|
|
||||||
|
## Prerequisites
|
||||||
|
|
||||||
|
- macOS with OrbStack (or Docker Desktop)
|
||||||
|
- Docker Compose v2 (command: `docker compose`, not `docker-compose`)
|
||||||
|
- Access to test TiDB instance
|
||||||
|
|
||||||
|
**Check your setup:**
|
||||||
|
```bash
|
||||||
|
./check-docker.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
**Note:** If you have Docker Compose v1 only, see [DOCKER_COMPOSE_V2.md](DOCKER_COMPOSE_V2.md) for migration options.
|
||||||
|
|
||||||
|
## Configuration
|
||||||
|
|
||||||
|
1. **Copy and edit `.env` file**:
|
||||||
|
```bash
|
||||||
|
cp .env.example .env
|
||||||
|
# Edit .env with your test database credentials
|
||||||
|
```
|
||||||
|
|
||||||
|
Required variables:
|
||||||
|
- `TEST_DB_HOST`: Your test TiDB host
|
||||||
|
- `TEST_DB_PORT`: Test TiDB port (default: 4000)
|
||||||
|
- `TEST_DB_USER`: Test database username
|
||||||
|
- `TEST_DB_PASSWORD`: Test database password
|
||||||
|
- `DATABASE_NAME`: Database to sync
|
||||||
|
- `TABLES`: Comma-separated list of tables (e.g., "table1,table2,table3")
|
||||||
|
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
### How the Sync Works
|
||||||
|
|
||||||
|
The data synchronization is **automatically configured and started** when you run `./start.sh`:
|
||||||
|
|
||||||
|
1. **Automatic Setup** (recommended):
|
||||||
|
- The `dm-init` container runs [`scripts/init-dm.sh`](scripts/init-dm.sh)
|
||||||
|
- It generates the actual DM task config from your `.env` variables
|
||||||
|
- The sync task is automatically started
|
||||||
|
- No manual intervention needed!
|
||||||
|
|
||||||
|
2. **Manual Control** (optional):
|
||||||
|
- Use [`./sync-control.sh`](sync-control.sh) for easy management
|
||||||
|
- Or use `dmctl` commands directly (see Manual DM Operations below)
|
||||||
|
|
||||||
|
**Note:** [`configs/task.yaml`](configs/task.yaml) is just a template. The real task config is generated dynamically at runtime.
|
||||||
|
|
||||||
|
**For detailed sync operations, see [SYNC_GUIDE.md](SYNC_GUIDE.md)**
|
||||||
|
|
||||||
|
### Start the environment
|
||||||
|
```bash
|
||||||
|
docker compose up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
This will:
|
||||||
|
1. Start TiDB in standalone mode
|
||||||
|
2. Start DM master and worker
|
||||||
|
3. Automatically configure the data source and sync task
|
||||||
|
4. Begin syncing data from test to local
|
||||||
|
|
||||||
|
### Check sync status
|
||||||
|
```bash
|
||||||
|
docker exec dm-master /dmctl --master-addr=dm-master:8261 query-status test-to-local
|
||||||
|
```
|
||||||
|
|
||||||
|
### Connect to local TiDB
|
||||||
|
|
||||||
|
**Command Line:**
|
||||||
|
```bash
|
||||||
|
mysql -h 127.0.0.1 -P 4000 -u root
|
||||||
|
```
|
||||||
|
|
||||||
|
**DataGrip / MySQL Workbench / DBeaver:**
|
||||||
|
- Host: `127.0.0.1`
|
||||||
|
- Port: `4000`
|
||||||
|
- User: `root`
|
||||||
|
- Password: _(leave empty)_
|
||||||
|
|
||||||
|
See [DATAGRIP_SETUP.md](DATAGRIP_SETUP.md) for detailed client setup instructions.
|
||||||
|
|
||||||
|
### View logs
|
||||||
|
```bash
|
||||||
|
# All services
|
||||||
|
docker compose logs -f
|
||||||
|
|
||||||
|
# Specific service
|
||||||
|
docker compose logs -f tidb
|
||||||
|
docker compose logs -f dm-worker
|
||||||
|
```
|
||||||
|
|
||||||
|
### Stop the environment
|
||||||
|
```bash
|
||||||
|
docker compose down
|
||||||
|
```
|
||||||
|
|
||||||
|
### Reset everything (including data)
|
||||||
|
```bash
|
||||||
|
docker compose down -v
|
||||||
|
```
|
||||||
|
|
||||||
|
## Manual DM Operations
|
||||||
|
|
||||||
|
### Check source configuration
|
||||||
|
```bash
|
||||||
|
docker exec dm-master /dmctl --master-addr=dm-master:8261 operate-source show
|
||||||
|
```
|
||||||
|
|
||||||
|
### Stop sync task
|
||||||
|
```bash
|
||||||
|
docker exec dm-master /dmctl --master-addr=dm-master:8261 stop-task test-to-local
|
||||||
|
```
|
||||||
|
|
||||||
|
### Start sync task
|
||||||
|
```bash
|
||||||
|
docker exec dm-master /dmctl --master-addr=dm-master:8261 start-task /configs/task.yaml
|
||||||
|
```
|
||||||
|
|
||||||
|
### Pause sync task
|
||||||
|
```bash
|
||||||
|
docker exec dm-master /dmctl --master-addr=dm-master:8261 pause-task test-to-local
|
||||||
|
```
|
||||||
|
|
||||||
|
### Resume sync task
|
||||||
|
```bash
|
||||||
|
docker exec dm-master /dmctl --master-addr=dm-master:8261 resume-task test-to-local
|
||||||
|
```
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
### Common Issues
|
||||||
|
|
||||||
|
For detailed troubleshooting, see [TROUBLESHOOTING.md](TROUBLESHOOTING.md)
|
||||||
|
|
||||||
|
### Quick Checks
|
||||||
|
|
||||||
|
#### TiDB health check failing
|
||||||
|
```bash
|
||||||
|
# Check if TiDB is healthy
|
||||||
|
docker ps | grep tidb
|
||||||
|
|
||||||
|
# Should show: (healthy)
|
||||||
|
# If not, check logs:
|
||||||
|
docker logs tidb
|
||||||
|
```
|
||||||
|
|
||||||
|
#### DM task fails to start
|
||||||
|
- Check if test database is accessible from container
|
||||||
|
- Verify credentials in `.env`
|
||||||
|
- Check logs: `docker-compose logs dm-worker`
|
||||||
|
|
||||||
|
### Tables not syncing
|
||||||
|
- Ensure tables exist in source database
|
||||||
|
- Verify table names in `TABLES` variable
|
||||||
|
- Check task status for specific errors
|
||||||
|
|
||||||
|
### TiDB connection issues
|
||||||
|
- Verify TiDB is running: `docker ps | grep tidb`
|
||||||
|
- Check health: `docker exec tidb mysql -h 127.0.0.1 -P 4000 -u root -e "SELECT 1"`
|
||||||
|
|
||||||
|
### Re-initialize DM configuration
|
||||||
|
```bash
|
||||||
|
docker compose up -d dm-init
|
||||||
|
```
|
||||||
|
|
||||||
|
## Resource Usage
|
||||||
|
|
||||||
|
Default resource limits (suitable for local development):
|
||||||
|
- TiDB: 2 CPU, 2GB RAM
|
||||||
|
- DM Worker: 1 CPU, 1GB RAM
|
||||||
|
- DM Master: 0.5 CPU, 512MB RAM
|
||||||
|
|
||||||
|
Adjust in `docker-compose.yml` if needed.
|
||||||
|
|
||||||
|
## Notes
|
||||||
|
|
||||||
|
- **Docker Compose v2**: This project uses `docker compose` (v2 syntax). If you have v1, either upgrade or create an alias: `alias docker-compose='docker compose'`
|
||||||
|
- **Standalone Mode**: TiDB runs without distributed storage, suitable for development only
|
||||||
|
- **Data Persistence**: Data is stored in Docker volumes, persists across restarts
|
||||||
|
- **Sync Mode**: Configured for full + incremental sync ("all" mode)
|
||||||
|
- **OrbStack DNS**: Uses `.orb.local` hostnames for container networking
|
||||||
|
|
@ -0,0 +1,354 @@
|
||||||
|
# Data Sync Guide
|
||||||
|
|
||||||
|
## How Sync Works
|
||||||
|
|
||||||
|
Your TiDB Data Migration (DM) setup continuously syncs data from your test environment to the local TiDB instance.
|
||||||
|
|
||||||
|
```
|
||||||
|
Test TiDB ──────┐
|
||||||
|
│
|
||||||
|
(DM reads changes)
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
DM Worker
|
||||||
|
│
|
||||||
|
(Applies to local)
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
Local TiDB
|
||||||
|
```
|
||||||
|
|
||||||
|
## Automatic Sync Setup
|
||||||
|
|
||||||
|
When you run `./start.sh`, the sync is **automatically configured and started**:
|
||||||
|
|
||||||
|
1. ✅ Reads your `.env` file for credentials and table list
|
||||||
|
2. ✅ Generates the DM task configuration
|
||||||
|
3. ✅ Configures the source connection (test TiDB)
|
||||||
|
4. ✅ Starts the sync task
|
||||||
|
5. ✅ Begins syncing data (full + incremental)
|
||||||
|
|
||||||
|
**You don't need to do anything manually!**
|
||||||
|
|
||||||
|
## Sync Modes
|
||||||
|
|
||||||
|
The sync is configured with `task-mode: "all"`:
|
||||||
|
- **Full sync**: Initial copy of all existing data
|
||||||
|
- **Incremental sync**: Continuous replication of changes (INSERT, UPDATE, DELETE)
|
||||||
|
|
||||||
|
## Managing Sync
|
||||||
|
|
||||||
|
### Easy Way (Recommended)
|
||||||
|
|
||||||
|
Use the [`sync-control.sh`](sync-control.sh) script:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Check sync status
|
||||||
|
./sync-control.sh status
|
||||||
|
|
||||||
|
# Stop sync
|
||||||
|
./sync-control.sh stop
|
||||||
|
|
||||||
|
# Start sync
|
||||||
|
./sync-control.sh start
|
||||||
|
|
||||||
|
# Pause sync (temporarily)
|
||||||
|
./sync-control.sh pause
|
||||||
|
|
||||||
|
# Resume sync
|
||||||
|
./sync-control.sh resume
|
||||||
|
|
||||||
|
# Restart sync (stop + start)
|
||||||
|
./sync-control.sh restart
|
||||||
|
|
||||||
|
# Re-initialize configuration
|
||||||
|
./sync-control.sh reinit
|
||||||
|
```
|
||||||
|
|
||||||
|
### Advanced Way (dmctl)
|
||||||
|
|
||||||
|
Use `dmctl` directly:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Check status
|
||||||
|
docker exec dm-master /dmctl --master-addr=dm-master:8261 query-status test-to-local
|
||||||
|
|
||||||
|
# Stop task
|
||||||
|
docker exec dm-master /dmctl --master-addr=dm-master:8261 stop-task test-to-local
|
||||||
|
|
||||||
|
# Start task
|
||||||
|
docker exec dm-master /dmctl --master-addr=dm-master:8261 start-task test-to-local
|
||||||
|
|
||||||
|
# Pause task
|
||||||
|
docker exec dm-master /dmctl --master-addr=dm-master:8261 pause-task test-to-local
|
||||||
|
|
||||||
|
# Resume task
|
||||||
|
docker exec dm-master /dmctl --master-addr=dm-master:8261 resume-task test-to-local
|
||||||
|
```
|
||||||
|
|
||||||
|
## Checking Sync Status
|
||||||
|
|
||||||
|
### Quick Check
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./status.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
This shows:
|
||||||
|
- Source configuration
|
||||||
|
- Task status (running, paused, stopped)
|
||||||
|
- Current sync position
|
||||||
|
- Error messages (if any)
|
||||||
|
- Local databases
|
||||||
|
|
||||||
|
### Detailed Status
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./sync-control.sh status
|
||||||
|
```
|
||||||
|
|
||||||
|
### Verify Data Sync
|
||||||
|
|
||||||
|
Connect to local TiDB and verify:
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- Connect
|
||||||
|
mysql -h 127.0.0.1 -P 4000 -u root
|
||||||
|
|
||||||
|
-- Check databases
|
||||||
|
SHOW DATABASES;
|
||||||
|
|
||||||
|
-- Switch to your database
|
||||||
|
USE your_database;
|
||||||
|
|
||||||
|
-- Check tables
|
||||||
|
SHOW TABLES;
|
||||||
|
|
||||||
|
-- Verify row count
|
||||||
|
SELECT COUNT(*) FROM table1;
|
||||||
|
|
||||||
|
-- Compare with source (if you have access)
|
||||||
|
-- Run the same query on test environment
|
||||||
|
```
|
||||||
|
|
||||||
|
## Configuration Files
|
||||||
|
|
||||||
|
### Environment Variables (`.env`)
|
||||||
|
|
||||||
|
This is where you configure what to sync:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Source database
|
||||||
|
TEST_DB_HOST=your-test-tidb-host
|
||||||
|
TEST_DB_PORT=4000
|
||||||
|
TEST_DB_USER=root
|
||||||
|
TEST_DB_PASSWORD=your-password
|
||||||
|
|
||||||
|
# What to sync
|
||||||
|
DATABASE_NAME=your_database
|
||||||
|
TABLES="table1,table2,table3"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Task Template (`configs/task.yaml`)
|
||||||
|
|
||||||
|
This is just a **template for reference**. The actual task config is generated by [`scripts/init-dm.sh`](scripts/init-dm.sh).
|
||||||
|
|
||||||
|
### Source Config (`configs/source.yaml`)
|
||||||
|
|
||||||
|
Template for source database connection. Also generated dynamically.
|
||||||
|
|
||||||
|
## Common Scenarios
|
||||||
|
|
||||||
|
### Adding/Removing Tables
|
||||||
|
|
||||||
|
1. Edit `.env` and update `TABLES` variable:
|
||||||
|
```bash
|
||||||
|
TABLES="table1,table2,table3,new_table4"
|
||||||
|
```
|
||||||
|
|
||||||
|
2. Re-initialize:
|
||||||
|
```bash
|
||||||
|
./sync-control.sh reinit
|
||||||
|
```
|
||||||
|
|
||||||
|
### Changing Source Database
|
||||||
|
|
||||||
|
1. Edit `.env` with new credentials
|
||||||
|
2. Restart everything:
|
||||||
|
```bash
|
||||||
|
docker compose down
|
||||||
|
./start.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
### Resetting Sync (Fresh Start)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Stop and remove everything
|
||||||
|
docker compose down -v
|
||||||
|
|
||||||
|
# Start fresh
|
||||||
|
./start.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
### Pausing Sync Temporarily
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Pause (without stopping containers)
|
||||||
|
./sync-control.sh pause
|
||||||
|
|
||||||
|
# Resume when ready
|
||||||
|
./sync-control.sh resume
|
||||||
|
```
|
||||||
|
|
||||||
|
## Monitoring
|
||||||
|
|
||||||
|
### View Logs
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# All services
|
||||||
|
docker compose logs -f
|
||||||
|
|
||||||
|
# DM Worker only
|
||||||
|
docker compose logs -f dm-worker
|
||||||
|
|
||||||
|
# DM Master only
|
||||||
|
docker compose logs -f dm-master
|
||||||
|
|
||||||
|
# Init script logs
|
||||||
|
docker logs dm-init
|
||||||
|
```
|
||||||
|
|
||||||
|
### Check DM Master Status
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker exec dm-master /dmctl --master-addr=dm-master:8261 operate-source show
|
||||||
|
```
|
||||||
|
|
||||||
|
### Check DM Worker Status
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker ps | grep dm-worker
|
||||||
|
```
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
### Sync Not Starting
|
||||||
|
|
||||||
|
**Check init logs:**
|
||||||
|
```bash
|
||||||
|
docker logs dm-init
|
||||||
|
```
|
||||||
|
|
||||||
|
**Common issues:**
|
||||||
|
- Wrong credentials in `.env`
|
||||||
|
- Test database not accessible
|
||||||
|
- Tables don't exist in source
|
||||||
|
|
||||||
|
**Solution:**
|
||||||
|
```bash
|
||||||
|
# Fix .env, then:
|
||||||
|
./sync-control.sh reinit
|
||||||
|
```
|
||||||
|
|
||||||
|
### Sync Stopped with Errors
|
||||||
|
|
||||||
|
**Check error message:**
|
||||||
|
```bash
|
||||||
|
./sync-control.sh status
|
||||||
|
```
|
||||||
|
|
||||||
|
**Common errors:**
|
||||||
|
- Network connectivity issues
|
||||||
|
- Permission denied on source
|
||||||
|
- Table schema mismatch
|
||||||
|
|
||||||
|
**Solution:**
|
||||||
|
```bash
|
||||||
|
# Fix the underlying issue, then:
|
||||||
|
./sync-control.sh restart
|
||||||
|
```
|
||||||
|
|
||||||
|
### Data Not Syncing
|
||||||
|
|
||||||
|
**Verify task is running:**
|
||||||
|
```bash
|
||||||
|
./sync-control.sh status
|
||||||
|
```
|
||||||
|
|
||||||
|
**Check if tables exist:**
|
||||||
|
```bash
|
||||||
|
# On source
|
||||||
|
mysql -h $TEST_DB_HOST -P 4000 -u root -p -e "SHOW TABLES FROM your_database;"
|
||||||
|
|
||||||
|
# On local
|
||||||
|
mysql -h 127.0.0.1 -P 4000 -u root -e "SHOW TABLES FROM your_database;"
|
||||||
|
```
|
||||||
|
|
||||||
|
**Compare row counts:**
|
||||||
|
```bash
|
||||||
|
# Create a verification script
|
||||||
|
mysql -h 127.0.0.1 -P 4000 -u root -e "SELECT COUNT(*) FROM your_database.table1;"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Performance Tuning
|
||||||
|
|
||||||
|
### Adjust Sync Speed
|
||||||
|
|
||||||
|
Edit `docker-compose.yml`:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
dm-worker:
|
||||||
|
deploy:
|
||||||
|
resources:
|
||||||
|
limits:
|
||||||
|
cpus: '2' # Increase CPU
|
||||||
|
memory: 2G # Increase memory
|
||||||
|
```
|
||||||
|
|
||||||
|
Then restart:
|
||||||
|
```bash
|
||||||
|
docker compose down
|
||||||
|
docker compose up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
### Monitor Resource Usage
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker stats
|
||||||
|
```
|
||||||
|
|
||||||
|
## Best Practices
|
||||||
|
|
||||||
|
1. **Always check status** after starting: `./status.sh`
|
||||||
|
2. **Monitor logs** during initial sync: `docker compose logs -f dm-worker`
|
||||||
|
3. **Verify data** in local TiDB after sync completes
|
||||||
|
4. **Use pause/resume** instead of stop/start for temporary halts
|
||||||
|
5. **Keep `.env` secure** - it contains credentials
|
||||||
|
6. **Test connectivity** before sync: `./test-connection.sh`
|
||||||
|
|
||||||
|
## FAQ
|
||||||
|
|
||||||
|
**Q: Is the sync real-time?**
|
||||||
|
A: Near real-time. Changes are replicated with minimal delay (usually seconds).
|
||||||
|
|
||||||
|
**Q: What happens if my laptop sleeps?**
|
||||||
|
A: Sync will resume automatically when containers restart.
|
||||||
|
|
||||||
|
**Q: Can I sync from multiple sources?**
|
||||||
|
A: Yes, but requires manual DM configuration. This setup is for single source.
|
||||||
|
|
||||||
|
**Q: Does it sync schema changes?**
|
||||||
|
A: Yes, DDL statements are replicated (CREATE, ALTER, DROP).
|
||||||
|
|
||||||
|
**Q: Can I sync to a different database name locally?**
|
||||||
|
A: Requires custom task configuration. Default syncs to same database name.
|
||||||
|
|
||||||
|
**Q: How do I exclude certain tables?**
|
||||||
|
A: Remove them from `TABLES` in `.env` and run `./sync-control.sh reinit`.
|
||||||
|
|
||||||
|
## See Also
|
||||||
|
|
||||||
|
- [README.md](README.md) - Main documentation
|
||||||
|
- [DATAGRIP_SETUP.md](DATAGRIP_SETUP.md) - Connect with GUI clients
|
||||||
|
- [scripts/init-dm.sh](scripts/init-dm.sh) - Initialization script
|
||||||
|
- [TiDB DM Documentation](https://docs.pingcap.com/tidb-data-migration/stable)
|
||||||
|
|
@ -0,0 +1,419 @@
|
||||||
|
# Troubleshooting Guide
|
||||||
|
|
||||||
|
## Common Issues and Solutions
|
||||||
|
|
||||||
|
### 1. TiDB Health Check Failing
|
||||||
|
|
||||||
|
#### Symptom
|
||||||
|
```
|
||||||
|
dependency failed to start: container tidb is unhealthy
|
||||||
|
```
|
||||||
|
|
||||||
|
Even though you can connect to TiDB from your host machine, Docker health check fails.
|
||||||
|
|
||||||
|
#### Root Cause
|
||||||
|
The original health check tried to use `mysql` command inside the TiDB container:
|
||||||
|
```yaml
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD", "mysql", "-h", "127.0.0.1", "-P", "4000", "-u", "root", "-e", "SELECT 1"]
|
||||||
|
```
|
||||||
|
|
||||||
|
The TiDB Docker image doesn't include the MySQL client binary, so this check always failed.
|
||||||
|
|
||||||
|
#### Solution ✅
|
||||||
|
Use TiDB's built-in HTTP status endpoint instead:
|
||||||
|
```yaml
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD", "wget", "-q", "-O-", "http://127.0.0.1:10080/status"]
|
||||||
|
interval: 10s
|
||||||
|
timeout: 5s
|
||||||
|
retries: 3
|
||||||
|
start_period: 10s
|
||||||
|
```
|
||||||
|
|
||||||
|
**Why this works:**
|
||||||
|
- TiDB exposes a status endpoint on port 10080
|
||||||
|
- `wget` is available in the container
|
||||||
|
- Returns HTTP 200 when TiDB is ready
|
||||||
|
- `start_period` gives TiDB time to initialize before health checks begin
|
||||||
|
|
||||||
|
### 2. Docker Compose Version Warning
|
||||||
|
|
||||||
|
#### Symptom
|
||||||
|
```
|
||||||
|
WARN[0000] version is obsolete, it will be ignored
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Solution ✅
|
||||||
|
Remove the `version` field from `docker-compose.yml`. Modern Docker Compose (v2) doesn't need it.
|
||||||
|
|
||||||
|
**Before:**
|
||||||
|
```yaml
|
||||||
|
version: '3.8'
|
||||||
|
services:
|
||||||
|
...
|
||||||
|
```
|
||||||
|
|
||||||
|
**After:**
|
||||||
|
```yaml
|
||||||
|
services:
|
||||||
|
...
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3. Service Dependencies Not Starting in Order
|
||||||
|
|
||||||
|
#### Symptom
|
||||||
|
Services fail because dependencies aren't ready yet.
|
||||||
|
|
||||||
|
#### Solution ✅
|
||||||
|
Use proper health checks and dependency conditions:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
dm-worker:
|
||||||
|
depends_on:
|
||||||
|
tidb:
|
||||||
|
condition: service_healthy
|
||||||
|
dm-master:
|
||||||
|
condition: service_healthy
|
||||||
|
```
|
||||||
|
|
||||||
|
**Important:**
|
||||||
|
- Each dependency must have a working health check
|
||||||
|
- `start_period` prevents false negatives during startup
|
||||||
|
|
||||||
|
### 4. dm-init Fails to Start
|
||||||
|
|
||||||
|
#### Symptom
|
||||||
|
```
|
||||||
|
Error: dm-init exits immediately
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Check:
|
||||||
|
```bash
|
||||||
|
docker logs dm-init
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Common Causes:
|
||||||
|
|
||||||
|
**a) .env not configured:**
|
||||||
|
```bash
|
||||||
|
# Check if .env exists and has real values
|
||||||
|
cat .env
|
||||||
|
```
|
||||||
|
|
||||||
|
**Solution:**
|
||||||
|
```bash
|
||||||
|
# Copy template and edit
|
||||||
|
cp .env.example .env
|
||||||
|
vim .env
|
||||||
|
```
|
||||||
|
|
||||||
|
**b) Test database not reachable:**
|
||||||
|
```bash
|
||||||
|
# Test from dm-init container
|
||||||
|
docker run --rm --network tidb-network pingcap/dm:latest \
|
||||||
|
sh -c "wget -q -O- http://tidb:10080/status"
|
||||||
|
```
|
||||||
|
|
||||||
|
**c) Script syntax error:**
|
||||||
|
```bash
|
||||||
|
# Check init script
|
||||||
|
sh -n scripts/init-dm.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5. Containers Keep Restarting
|
||||||
|
|
||||||
|
#### Check Status:
|
||||||
|
```bash
|
||||||
|
docker ps -a
|
||||||
|
docker logs <container_name>
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Common Issues:
|
||||||
|
|
||||||
|
**a) Port already in use:**
|
||||||
|
```
|
||||||
|
Error: bind: address already in use
|
||||||
|
```
|
||||||
|
|
||||||
|
**Solution:** Change ports in `docker-compose.yml`:
|
||||||
|
```yaml
|
||||||
|
ports:
|
||||||
|
- "14000:4000" # Changed from 4000:4000
|
||||||
|
```
|
||||||
|
|
||||||
|
**b) Out of memory:**
|
||||||
|
```
|
||||||
|
Error: OOM killed
|
||||||
|
```
|
||||||
|
|
||||||
|
**Solution:** Increase memory limits or free up system resources.
|
||||||
|
|
||||||
|
**c) Permission issues:**
|
||||||
|
```
|
||||||
|
Error: permission denied
|
||||||
|
```
|
||||||
|
|
||||||
|
**Solution:** Check volume permissions or run:
|
||||||
|
```bash
|
||||||
|
docker compose down -v # Remove volumes
|
||||||
|
docker compose up -d # Recreate
|
||||||
|
```
|
||||||
|
|
||||||
|
### 6. Sync Task Not Running
|
||||||
|
|
||||||
|
#### Check Status:
|
||||||
|
```bash
|
||||||
|
./status.sh
|
||||||
|
# or
|
||||||
|
./sync-control.sh status
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Common Issues:
|
||||||
|
|
||||||
|
**a) Task not created:**
|
||||||
|
```bash
|
||||||
|
# Check if source is configured
|
||||||
|
docker exec dm-master /dmctl --master-addr=dm-master:8261 operate-source show
|
||||||
|
```
|
||||||
|
|
||||||
|
**Solution:**
|
||||||
|
```bash
|
||||||
|
./sync-control.sh reinit
|
||||||
|
```
|
||||||
|
|
||||||
|
**b) Wrong credentials:**
|
||||||
|
Check logs:
|
||||||
|
```bash
|
||||||
|
docker logs dm-worker
|
||||||
|
```
|
||||||
|
|
||||||
|
Fix `.env` and reinit:
|
||||||
|
```bash
|
||||||
|
vim .env
|
||||||
|
./sync-control.sh reinit
|
||||||
|
```
|
||||||
|
|
||||||
|
**c) Table doesn't exist:**
|
||||||
|
Verify tables exist on source database:
|
||||||
|
```bash
|
||||||
|
# Connect to test DB and check
|
||||||
|
SHOW TABLES FROM your_database;
|
||||||
|
```
|
||||||
|
|
||||||
|
### 7. Connection Refused to TiDB
|
||||||
|
|
||||||
|
#### Symptom
|
||||||
|
```
|
||||||
|
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (61)
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Checks:
|
||||||
|
|
||||||
|
**a) Is TiDB running?**
|
||||||
|
```bash
|
||||||
|
docker ps | grep tidb
|
||||||
|
```
|
||||||
|
|
||||||
|
**b) Is it healthy?**
|
||||||
|
```bash
|
||||||
|
docker ps
|
||||||
|
# Look for "(healthy)" status
|
||||||
|
```
|
||||||
|
|
||||||
|
**c) Is port exposed?**
|
||||||
|
```bash
|
||||||
|
docker port tidb
|
||||||
|
# Should show: 4000/tcp -> 0.0.0.0:4000
|
||||||
|
```
|
||||||
|
|
||||||
|
**d) Test from inside container:**
|
||||||
|
```bash
|
||||||
|
docker exec tidb wget -q -O- http://127.0.0.1:10080/status
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Solutions:
|
||||||
|
|
||||||
|
**If container not running:**
|
||||||
|
```bash
|
||||||
|
docker compose up -d tidb
|
||||||
|
docker logs tidb
|
||||||
|
```
|
||||||
|
|
||||||
|
**If unhealthy:**
|
||||||
|
```bash
|
||||||
|
# Wait for health check
|
||||||
|
sleep 15
|
||||||
|
docker ps
|
||||||
|
|
||||||
|
# If still unhealthy, check logs
|
||||||
|
docker logs tidb
|
||||||
|
```
|
||||||
|
|
||||||
|
**If port not exposed:**
|
||||||
|
```bash
|
||||||
|
# Recreate container
|
||||||
|
docker compose down
|
||||||
|
docker compose up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
### 8. Data Not Syncing
|
||||||
|
|
||||||
|
#### Verify sync is running:
|
||||||
|
```bash
|
||||||
|
./sync-control.sh status
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Check sync lag:
|
||||||
|
Look for "syncer" section in status output.
|
||||||
|
|
||||||
|
#### Common Issues:
|
||||||
|
|
||||||
|
**a) Sync paused:**
|
||||||
|
```bash
|
||||||
|
./sync-control.sh resume
|
||||||
|
```
|
||||||
|
|
||||||
|
**b) Sync stopped with error:**
|
||||||
|
```bash
|
||||||
|
# Check error in status output
|
||||||
|
./sync-control.sh status
|
||||||
|
|
||||||
|
# Fix the issue, then restart
|
||||||
|
./sync-control.sh restart
|
||||||
|
```
|
||||||
|
|
||||||
|
**c) Network issues:**
|
||||||
|
```bash
|
||||||
|
# Test connectivity from dm-worker to source
|
||||||
|
docker exec dm-worker ping -c 3 your-test-db-host
|
||||||
|
```
|
||||||
|
|
||||||
|
**d) Binlog not enabled on source:**
|
||||||
|
Source database must have binlog enabled for incremental sync.
|
||||||
|
|
||||||
|
### 9. Slow Sync Performance
|
||||||
|
|
||||||
|
#### Check resource usage:
|
||||||
|
```bash
|
||||||
|
docker stats
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Solutions:
|
||||||
|
|
||||||
|
**a) Increase worker resources:**
|
||||||
|
Edit `docker-compose.yml`:
|
||||||
|
```yaml
|
||||||
|
dm-worker:
|
||||||
|
deploy:
|
||||||
|
resources:
|
||||||
|
limits:
|
||||||
|
cpus: '2'
|
||||||
|
memory: 2G
|
||||||
|
```
|
||||||
|
|
||||||
|
**b) Optimize batch size:**
|
||||||
|
See [TiDB DM Documentation](https://docs.pingcap.com/tidb-data-migration/stable/tune-configuration) for advanced tuning.
|
||||||
|
|
||||||
|
### 10. Docker Compose v1 vs v2 Issues
|
||||||
|
|
||||||
|
#### Symptom
|
||||||
|
```
|
||||||
|
docker: 'compose' is not a docker command
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Solution
|
||||||
|
See [DOCKER_COMPOSE_V2.md](DOCKER_COMPOSE_V2.md) for:
|
||||||
|
- Upgrading to v2
|
||||||
|
- Creating an alias
|
||||||
|
- Compatibility mode
|
||||||
|
|
||||||
|
## Diagnostic Commands
|
||||||
|
|
||||||
|
### Check everything at once:
|
||||||
|
```bash
|
||||||
|
# Service status
|
||||||
|
docker compose ps
|
||||||
|
|
||||||
|
# Health checks
|
||||||
|
docker ps
|
||||||
|
|
||||||
|
# Logs (all services)
|
||||||
|
docker compose logs --tail=50
|
||||||
|
|
||||||
|
# Logs (specific service)
|
||||||
|
docker compose logs --tail=50 tidb
|
||||||
|
|
||||||
|
# Resource usage
|
||||||
|
docker stats --no-stream
|
||||||
|
|
||||||
|
# Network connectivity
|
||||||
|
docker network inspect tidb-network
|
||||||
|
```
|
||||||
|
|
||||||
|
### Test connectivity:
|
||||||
|
```bash
|
||||||
|
# From host to TiDB
|
||||||
|
mysql -h 127.0.0.1 -P 4000 -u root -e "SELECT 1"
|
||||||
|
|
||||||
|
# From host (HTTP)
|
||||||
|
curl http://127.0.0.1:10080/status
|
||||||
|
|
||||||
|
# From container to TiDB
|
||||||
|
docker run --rm --network tidb-network pingcap/dm:latest \
|
||||||
|
wget -q -O- http://tidb:10080/status
|
||||||
|
```
|
||||||
|
|
||||||
|
### Reset everything:
|
||||||
|
```bash
|
||||||
|
# Stop and remove everything (including data)
|
||||||
|
docker compose down -v
|
||||||
|
|
||||||
|
# Start fresh
|
||||||
|
./start.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
## Getting Help
|
||||||
|
|
||||||
|
### Collect diagnostic information:
|
||||||
|
```bash
|
||||||
|
# Create a diagnostic report
|
||||||
|
echo "=== Docker Version ===" > diagnostic.txt
|
||||||
|
docker --version >> diagnostic.txt
|
||||||
|
docker compose version >> diagnostic.txt
|
||||||
|
|
||||||
|
echo -e "\n=== Container Status ===" >> diagnostic.txt
|
||||||
|
docker ps -a >> diagnostic.txt
|
||||||
|
|
||||||
|
echo -e "\n=== TiDB Logs ===" >> diagnostic.txt
|
||||||
|
docker logs tidb --tail=50 >> diagnostic.txt 2>&1
|
||||||
|
|
||||||
|
echo -e "\n=== DM Worker Logs ===" >> diagnostic.txt
|
||||||
|
docker logs dm-worker --tail=50 >> diagnostic.txt 2>&1
|
||||||
|
|
||||||
|
echo -e "\n=== DM Init Logs ===" >> diagnostic.txt
|
||||||
|
docker logs dm-init >> diagnostic.txt 2>&1
|
||||||
|
|
||||||
|
echo -e "\n=== Network Info ===" >> diagnostic.txt
|
||||||
|
docker network inspect tidb-network >> diagnostic.txt
|
||||||
|
|
||||||
|
echo "Report saved to diagnostic.txt"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Useful resources:
|
||||||
|
- [TiDB Documentation](https://docs.pingcap.com/tidb/stable)
|
||||||
|
- [TiDB DM Documentation](https://docs.pingcap.com/tidb-data-migration/stable)
|
||||||
|
- Project documentation:
|
||||||
|
- [README.md](README.md)
|
||||||
|
- [SYNC_GUIDE.md](SYNC_GUIDE.md)
|
||||||
|
- [DATAGRIP_SETUP.md](DATAGRIP_SETUP.md)
|
||||||
|
- [DOCKER_COMPOSE_V2.md](DOCKER_COMPOSE_V2.md)
|
||||||
|
|
||||||
|
## Still Having Issues?
|
||||||
|
|
||||||
|
If none of these solutions work:
|
||||||
|
|
||||||
|
1. Check logs: `docker compose logs`
|
||||||
|
2. Create diagnostic report (see above)
|
||||||
|
3. Check if it's a known issue in TiDB/DM GitHub issues
|
||||||
|
4. Verify your environment meets prerequisites (see [README.md](README.md))
|
||||||
|
|
@ -0,0 +1,45 @@
|
||||||
|
#!/bin/bash
|
||||||
|
|
||||||
|
echo "🐳 Checking Docker setup..."
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
# Check if Docker is installed
|
||||||
|
if ! command -v docker &> /dev/null; then
|
||||||
|
echo "❌ Docker is not installed"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "✅ Docker is installed"
|
||||||
|
docker --version
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
# Check if Docker Compose v2 is available
|
||||||
|
if docker compose version &> /dev/null; then
|
||||||
|
echo "✅ Docker Compose v2 is available"
|
||||||
|
docker compose version
|
||||||
|
echo ""
|
||||||
|
echo "👍 Your setup is compatible with this project!"
|
||||||
|
elif command -v docker-compose &> /dev/null; then
|
||||||
|
echo "⚠️ Only Docker Compose v1 (docker-compose) is available"
|
||||||
|
docker-compose --version
|
||||||
|
echo ""
|
||||||
|
echo "💡 Recommendation: Upgrade to Docker Compose v2 (comes with Docker Desktop)"
|
||||||
|
echo " Or create an alias: alias docker-compose='docker compose'"
|
||||||
|
else
|
||||||
|
echo "❌ Docker Compose is not available"
|
||||||
|
echo ""
|
||||||
|
echo "Please install Docker Compose:"
|
||||||
|
echo " - Docker Desktop (includes Compose v2): https://www.docker.com/products/docker-desktop"
|
||||||
|
echo " - Standalone: https://docs.docker.com/compose/install/"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Check if OrbStack is being used
|
||||||
|
if command -v orb &> /dev/null; then
|
||||||
|
echo ""
|
||||||
|
echo "🚀 OrbStack detected!"
|
||||||
|
orb version
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo ""
|
||||||
|
echo "Ready to start! Run: ./start.sh"
|
||||||
|
|
@ -0,0 +1,8 @@
|
||||||
|
source-id: "test-tidb"
|
||||||
|
enable-gtid: false
|
||||||
|
enable-relay: false
|
||||||
|
from:
|
||||||
|
host: "${TEST_DB_HOST}"
|
||||||
|
port: ${TEST_DB_PORT}
|
||||||
|
user: "${TEST_DB_USER}"
|
||||||
|
password: "${TEST_DB_PASSWORD}"
|
||||||
|
|
@ -0,0 +1,52 @@
|
||||||
|
# ==============================================================================
|
||||||
|
# DM Task Configuration Template
|
||||||
|
# ==============================================================================
|
||||||
|
#
|
||||||
|
# This file is a TEMPLATE and is NOT used directly by DM.
|
||||||
|
#
|
||||||
|
# The actual task configuration is dynamically generated by:
|
||||||
|
# scripts/init-dm.sh
|
||||||
|
#
|
||||||
|
# The script reads environment variables from .env and creates the real task.yaml
|
||||||
|
# with your specific database name and table list.
|
||||||
|
#
|
||||||
|
# HOW TO RUN THE SYNC:
|
||||||
|
# --------------------
|
||||||
|
# 1. Configure .env file with your credentials
|
||||||
|
# 2. Run: ./start.sh (auto-generates and starts sync)
|
||||||
|
# 3. Check status: ./status.sh or ./sync-control.sh status
|
||||||
|
#
|
||||||
|
# For detailed guide, see: SYNC_GUIDE.md
|
||||||
|
#
|
||||||
|
# MANUAL CONTROL:
|
||||||
|
# ---------------
|
||||||
|
# - Start: ./sync-control.sh start
|
||||||
|
# - Stop: ./sync-control.sh stop
|
||||||
|
# - Pause: ./sync-control.sh pause
|
||||||
|
# - Resume: ./sync-control.sh resume
|
||||||
|
# - Restart: ./sync-control.sh restart
|
||||||
|
# - Reinit: ./sync-control.sh reinit
|
||||||
|
#
|
||||||
|
# ==============================================================================
|
||||||
|
|
||||||
|
# Template structure (for reference):
|
||||||
|
# name: "test-to-local"
|
||||||
|
# task-mode: "all" # full + incremental sync
|
||||||
|
# target-database:
|
||||||
|
# host: "tidb"
|
||||||
|
# port: 4000
|
||||||
|
# user: "root"
|
||||||
|
# password: ""
|
||||||
|
#
|
||||||
|
# mysql-instances:
|
||||||
|
# - source-id: "test-tidb"
|
||||||
|
# block-allow-list: "sync-tables"
|
||||||
|
#
|
||||||
|
# block-allow-list:
|
||||||
|
# sync-tables:
|
||||||
|
# do-dbs: ["${DATABASE_NAME}"]
|
||||||
|
# do-tables:
|
||||||
|
# - db-name: "${DATABASE_NAME}"
|
||||||
|
# tbl-name: "table1"
|
||||||
|
# - db-name: "${DATABASE_NAME}"
|
||||||
|
# tbl-name: "table2"
|
||||||
|
|
@ -0,0 +1,128 @@
|
||||||
|
# OrbStack optimized settings
|
||||||
|
x-common-settings: &common
|
||||||
|
restart: unless-stopped
|
||||||
|
logging:
|
||||||
|
driver: "json-file"
|
||||||
|
options:
|
||||||
|
max-size: "10m"
|
||||||
|
max-file: "3"
|
||||||
|
|
||||||
|
services:
|
||||||
|
tidb:
|
||||||
|
<<: *common
|
||||||
|
image: pingcap/tidb:latest
|
||||||
|
container_name: tidb
|
||||||
|
hostname: tidb.orb.local # OrbStack DNS
|
||||||
|
command:
|
||||||
|
- tidb-server
|
||||||
|
- --store=unistore
|
||||||
|
- --path=/tmp/tidb
|
||||||
|
- --advertise-address=tidb
|
||||||
|
ports:
|
||||||
|
- "4000:4000"
|
||||||
|
- "10080:10080"
|
||||||
|
volumes:
|
||||||
|
- tidb_data:/tmp/tidb
|
||||||
|
environment:
|
||||||
|
- TIDB_ENABLE_TELEMETRY=false
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD", "wget", "-q", "-O-", "http://127.0.0.1:10080/status"]
|
||||||
|
interval: 10s
|
||||||
|
timeout: 5s
|
||||||
|
retries: 3
|
||||||
|
start_period: 10s
|
||||||
|
deploy:
|
||||||
|
resources:
|
||||||
|
limits:
|
||||||
|
cpus: '2'
|
||||||
|
memory: 2G
|
||||||
|
reservations:
|
||||||
|
cpus: '1'
|
||||||
|
memory: 1G
|
||||||
|
|
||||||
|
dm-master:
|
||||||
|
<<: *common
|
||||||
|
image: pingcap/dm:latest
|
||||||
|
container_name: dm-master
|
||||||
|
hostname: dm-master.orb.local
|
||||||
|
command:
|
||||||
|
- /dm-master
|
||||||
|
- --master-addr=:8261
|
||||||
|
- --advertise-addr=dm-master:8261
|
||||||
|
ports:
|
||||||
|
- "8261:8261"
|
||||||
|
volumes:
|
||||||
|
- dm_master_data:/data
|
||||||
|
- ./configs:/configs:ro
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD", "wget", "-q", "-O-", "http://127.0.0.1:8261/status"]
|
||||||
|
interval: 10s
|
||||||
|
timeout: 5s
|
||||||
|
retries: 3
|
||||||
|
start_period: 5s
|
||||||
|
deploy:
|
||||||
|
resources:
|
||||||
|
limits:
|
||||||
|
cpus: '0.5'
|
||||||
|
memory: 512M
|
||||||
|
|
||||||
|
dm-worker:
|
||||||
|
<<: *common
|
||||||
|
image: pingcap/dm:latest
|
||||||
|
container_name: dm-worker
|
||||||
|
hostname: dm-worker.orb.local
|
||||||
|
command:
|
||||||
|
- /dm-worker
|
||||||
|
- --worker-addr=:8262
|
||||||
|
- --advertise-addr=dm-worker:8262
|
||||||
|
- --join=dm-master:8261
|
||||||
|
volumes:
|
||||||
|
- dm_worker_data:/data
|
||||||
|
- ./configs:/configs:ro
|
||||||
|
depends_on:
|
||||||
|
tidb:
|
||||||
|
condition: service_healthy
|
||||||
|
dm-master:
|
||||||
|
condition: service_healthy
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD", "wget", "-q", "-O-", "http://127.0.0.1:8262/status"]
|
||||||
|
interval: 10s
|
||||||
|
timeout: 5s
|
||||||
|
retries: 3
|
||||||
|
start_period: 5s
|
||||||
|
deploy:
|
||||||
|
resources:
|
||||||
|
limits:
|
||||||
|
cpus: '1'
|
||||||
|
memory: 1G
|
||||||
|
|
||||||
|
dm-init:
|
||||||
|
image: pingcap/dm:latest
|
||||||
|
container_name: dm-init
|
||||||
|
volumes:
|
||||||
|
- ./configs:/configs:ro
|
||||||
|
- ./scripts:/scripts:ro
|
||||||
|
environment:
|
||||||
|
- TEST_DB_HOST=${TEST_DB_HOST}
|
||||||
|
- TEST_DB_PORT=${TEST_DB_PORT}
|
||||||
|
- TEST_DB_USER=${TEST_DB_USER}
|
||||||
|
- TEST_DB_PASSWORD=${TEST_DB_PASSWORD}
|
||||||
|
- DATABASE_NAME=${DATABASE_NAME}
|
||||||
|
- TABLES=${TABLES}
|
||||||
|
depends_on:
|
||||||
|
dm-worker:
|
||||||
|
condition: service_healthy
|
||||||
|
command: ["/bin/sh", "/scripts/init-dm.sh"]
|
||||||
|
restart: "no"
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
tidb_data:
|
||||||
|
driver: local
|
||||||
|
dm_master_data:
|
||||||
|
driver: local
|
||||||
|
dm_worker_data:
|
||||||
|
driver: local
|
||||||
|
|
||||||
|
networks:
|
||||||
|
default:
|
||||||
|
name: tidb-network
|
||||||
|
|
@ -0,0 +1,50 @@
|
||||||
|
#!/bin/sh
|
||||||
|
set -e
|
||||||
|
|
||||||
|
echo "Waiting for DM master to be ready..."
|
||||||
|
sleep 5
|
||||||
|
|
||||||
|
# Substitute environment variables in source.yaml
|
||||||
|
cat /configs/source.yaml | \
|
||||||
|
sed "s/\${TEST_DB_HOST}/$TEST_DB_HOST/g" | \
|
||||||
|
sed "s/\${TEST_DB_PORT}/$TEST_DB_PORT/g" | \
|
||||||
|
sed "s/\${TEST_DB_USER}/$TEST_DB_USER/g" | \
|
||||||
|
sed "s/\${TEST_DB_PASSWORD}/$TEST_DB_PASSWORD/g" \
|
||||||
|
> /tmp/source.yaml
|
||||||
|
|
||||||
|
echo "Creating DM source configuration..."
|
||||||
|
/dmctl --master-addr=dm-master:8261 operate-source create /tmp/source.yaml || true
|
||||||
|
|
||||||
|
# Generate task.yaml with multiple tables
|
||||||
|
echo "name: \"test-to-local\"
|
||||||
|
task-mode: \"all\"
|
||||||
|
target-database:
|
||||||
|
host: \"tidb\"
|
||||||
|
port: 4000
|
||||||
|
user: \"root\"
|
||||||
|
password: \"\"
|
||||||
|
|
||||||
|
mysql-instances:
|
||||||
|
- source-id: \"test-tidb\"
|
||||||
|
block-allow-list: \"sync-tables\"
|
||||||
|
|
||||||
|
block-allow-list:
|
||||||
|
sync-tables:
|
||||||
|
do-dbs: [\"$DATABASE_NAME\"]
|
||||||
|
do-tables:" > /tmp/task.yaml
|
||||||
|
|
||||||
|
# Add each table from TABLES variable
|
||||||
|
IFS=',' read -ra TABLE_ARRAY <<< "$TABLES"
|
||||||
|
for table in "${TABLE_ARRAY[@]}"; do
|
||||||
|
table=$(echo "$table" | xargs) # trim whitespace
|
||||||
|
echo " - db-name: \"$DATABASE_NAME\"
|
||||||
|
tbl-name: \"$table\"" >> /tmp/task.yaml
|
||||||
|
done
|
||||||
|
|
||||||
|
echo "Starting DM sync task..."
|
||||||
|
/dmctl --master-addr=dm-master:8261 start-task /tmp/task.yaml || true
|
||||||
|
|
||||||
|
echo "Checking task status..."
|
||||||
|
/dmctl --master-addr=dm-master:8261 query-status test-to-local
|
||||||
|
|
||||||
|
echo "DM initialization complete!"
|
||||||
|
|
@ -0,0 +1,48 @@
|
||||||
|
#!/bin/bash
|
||||||
|
set -e
|
||||||
|
|
||||||
|
echo "🚀 Starting TiDB Local Environment..."
|
||||||
|
|
||||||
|
# Check if .env exists
|
||||||
|
if [ ! -f .env ]; then
|
||||||
|
echo "⚠️ .env file not found!"
|
||||||
|
echo "📝 Creating .env from template..."
|
||||||
|
cp .env.example .env
|
||||||
|
echo "✏️ Please edit .env file with your test database credentials"
|
||||||
|
echo " Then run this script again."
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Source .env to check if configured
|
||||||
|
source .env
|
||||||
|
|
||||||
|
if [ "$TEST_DB_HOST" = "your-test-tidb-host" ]; then
|
||||||
|
echo "⚠️ .env file needs configuration!"
|
||||||
|
echo "✏️ Please edit .env file with your test database credentials"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "🐳 Starting Docker containers..."
|
||||||
|
docker compose up -d
|
||||||
|
|
||||||
|
echo ""
|
||||||
|
echo "⏳ Waiting for services to be ready..."
|
||||||
|
sleep 10
|
||||||
|
|
||||||
|
echo ""
|
||||||
|
echo "✅ Environment started successfully!"
|
||||||
|
echo ""
|
||||||
|
echo "📊 Connection Info:"
|
||||||
|
echo " TiDB: mysql -h 127.0.0.1 -P 4000 -u root"
|
||||||
|
echo " DataGrip: Host: 127.0.0.1, Port: 4000, User: root, Password: (empty)"
|
||||||
|
echo " DM Master: http://localhost:8261"
|
||||||
|
echo ""
|
||||||
|
echo "🔍 Useful commands:"
|
||||||
|
echo " Test connection: ./test-connection.sh"
|
||||||
|
echo " Check sync status: ./status.sh"
|
||||||
|
echo " Control sync: ./sync-control.sh [start|stop|pause|resume|restart]"
|
||||||
|
echo " View logs: docker compose logs -f"
|
||||||
|
echo " Stop environment: docker compose down"
|
||||||
|
echo ""
|
||||||
|
echo "📖 For DataGrip setup: see DATAGRIP_SETUP.md"
|
||||||
|
echo ""
|
||||||
|
|
@ -0,0 +1,22 @@
|
||||||
|
#!/bin/bash
|
||||||
|
|
||||||
|
echo "🔍 Checking DM Sync Status..."
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
# Check if containers are running
|
||||||
|
if ! docker ps | grep -q dm-master; then
|
||||||
|
echo "❌ DM Master is not running. Start the environment first:"
|
||||||
|
echo " ./start.sh"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "📡 Source Configuration:"
|
||||||
|
docker exec dm-master /dmctl --master-addr=dm-master:8261 operate-source show
|
||||||
|
|
||||||
|
echo ""
|
||||||
|
echo "📊 Task Status:"
|
||||||
|
docker exec dm-master /dmctl --master-addr=dm-master:8261 query-status test-to-local
|
||||||
|
|
||||||
|
echo ""
|
||||||
|
echo "💾 Local TiDB Databases:"
|
||||||
|
docker exec tidb mysql -h 127.0.0.1 -P 4000 -u root -e "SHOW DATABASES;"
|
||||||
|
|
@ -0,0 +1,74 @@
|
||||||
|
#!/bin/bash
|
||||||
|
|
||||||
|
TASK_NAME="test-to-local"
|
||||||
|
DMCTL="docker exec dm-master /dmctl --master-addr=dm-master:8261"
|
||||||
|
|
||||||
|
show_usage() {
|
||||||
|
echo "Usage: ./sync-control.sh [command]"
|
||||||
|
echo ""
|
||||||
|
echo "Commands:"
|
||||||
|
echo " status - Show sync task status"
|
||||||
|
echo " start - Start the sync task"
|
||||||
|
echo " stop - Stop the sync task"
|
||||||
|
echo " pause - Pause the sync task"
|
||||||
|
echo " resume - Resume the sync task"
|
||||||
|
echo " restart - Restart the sync task (stop + start)"
|
||||||
|
echo " reinit - Re-initialize DM configuration"
|
||||||
|
echo ""
|
||||||
|
}
|
||||||
|
|
||||||
|
check_dm() {
|
||||||
|
if ! docker ps | grep -q dm-master; then
|
||||||
|
echo "❌ DM Master is not running"
|
||||||
|
echo " Start with: ./start.sh"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
case "$1" in
|
||||||
|
status)
|
||||||
|
check_dm
|
||||||
|
echo "📊 Checking sync status for task: $TASK_NAME"
|
||||||
|
echo ""
|
||||||
|
$DMCTL query-status $TASK_NAME
|
||||||
|
;;
|
||||||
|
start)
|
||||||
|
check_dm
|
||||||
|
echo "▶️ Starting sync task: $TASK_NAME"
|
||||||
|
$DMCTL start-task $TASK_NAME
|
||||||
|
;;
|
||||||
|
stop)
|
||||||
|
check_dm
|
||||||
|
echo "⏹️ Stopping sync task: $TASK_NAME"
|
||||||
|
$DMCTL stop-task $TASK_NAME
|
||||||
|
;;
|
||||||
|
pause)
|
||||||
|
check_dm
|
||||||
|
echo "⏸️ Pausing sync task: $TASK_NAME"
|
||||||
|
$DMCTL pause-task $TASK_NAME
|
||||||
|
;;
|
||||||
|
resume)
|
||||||
|
check_dm
|
||||||
|
echo "▶️ Resuming sync task: $TASK_NAME"
|
||||||
|
$DMCTL resume-task $TASK_NAME
|
||||||
|
;;
|
||||||
|
restart)
|
||||||
|
check_dm
|
||||||
|
echo "🔄 Restarting sync task: $TASK_NAME"
|
||||||
|
$DMCTL stop-task $TASK_NAME
|
||||||
|
sleep 2
|
||||||
|
$DMCTL start-task $TASK_NAME
|
||||||
|
;;
|
||||||
|
reinit)
|
||||||
|
echo "🔄 Re-initializing DM configuration..."
|
||||||
|
docker compose up -d dm-init
|
||||||
|
echo ""
|
||||||
|
echo "⏳ Waiting for initialization..."
|
||||||
|
sleep 5
|
||||||
|
docker logs dm-init
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
show_usage
|
||||||
|
exit 1
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
|
@ -0,0 +1,36 @@
|
||||||
|
#!/bin/bash
|
||||||
|
|
||||||
|
echo "🔌 Testing TiDB Connection for DataGrip..."
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
# Check if TiDB container is running
|
||||||
|
if ! docker ps | grep -q tidb; then
|
||||||
|
echo "❌ TiDB container is not running"
|
||||||
|
echo " Start it with: ./start.sh"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "✅ TiDB container is running"
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
# Test connection
|
||||||
|
echo "🧪 Testing MySQL connection on 127.0.0.1:4000..."
|
||||||
|
if docker exec tidb mysql -h 127.0.0.1 -P 4000 -u root -e "SELECT VERSION();" 2>/dev/null; then
|
||||||
|
echo ""
|
||||||
|
echo "✅ Connection successful!"
|
||||||
|
echo ""
|
||||||
|
echo "📊 Available databases:"
|
||||||
|
docker exec tidb mysql -h 127.0.0.1 -P 4000 -u root -e "SHOW DATABASES;" 2>/dev/null
|
||||||
|
echo ""
|
||||||
|
echo "🎯 DataGrip Connection Settings:"
|
||||||
|
echo " Host: 127.0.0.1"
|
||||||
|
echo " Port: 4000"
|
||||||
|
echo " User: root"
|
||||||
|
echo " Password: (leave empty)"
|
||||||
|
echo ""
|
||||||
|
echo "📖 See DATAGRIP_SETUP.md for detailed instructions"
|
||||||
|
else
|
||||||
|
echo ""
|
||||||
|
echo "❌ Connection failed"
|
||||||
|
echo " Check TiDB logs: docker logs tidb"
|
||||||
|
fi
|
||||||
Loading…
Reference in New Issue