# TiDB Cloud to Local TiDB Migration Guide

This guide provides officially recommended approaches for migrating data from TiDB Cloud to a local TiDB instance, since TiDB Data Migration (DM) cannot be used with TiDB Cloud Serverless.

## Why DM Doesn't Work with TiDB Cloud

TiDB Data Migration (DM) fails with TiDB Cloud because:

1. **No MySQL binlog support** - TiDB Cloud Serverless doesn't expose binlog in the traditional MySQL way
2. **binlog_format is STATEMENT** - DM requires ROW format
3. **TiDB explicitly not supported as upstream** - DM is designed for MySQL/MariaDB → TiDB, not TiDB → TiDB

## Approach 1: Console Export + SQL Import (Simplest)

### Export from TiDB Cloud

1. **Using TiDB Cloud Console**:
   - Navigate to your cluster in the TiDB Cloud Console
   - Go to Data > Import
   - Click "Export Data to" > "Local File"
   - Select databases/tables to export
   - Choose format (SQL recommended for small datasets)
   - Click "Export"

2. **Using TiDB Cloud CLI**:
   ```bash
   # Create export task
   ticloud serverless export create -c <cluster-id>
   
   # Download exported data
   ticloud serverless export download -c <cluster-id> -e <export-id>
   ```

### Import to Local TiDB

```bash
# Import SQL file
mysql -h 127.0.0.1 -P 4000 -u root < exported_data.sql

# Or for CSV files
mysql -h 127.0.0.1 -P 4000 -u root -e "
LOAD DATA LOCAL INFILE 'table_data.csv' 
INTO TABLE your_table 
FIELDS TERMINATED BY ',' 
ENCLOSED BY '\"' 
LINES TERMINATED BY '\n'
IGNORE 1 ROWS;"
```

## Approach 2: Dumpling + TiDB Lightning (For Larger Datasets)

### Prerequisites

Install TiDB tools:
```bash
# Install TiUP
curl --proto '=https' --tlsv1.2 -sSf https://tiup-mirrors.pingcap.com/install.sh | sh
source ~/.bash_profile

# Install tools
tiup install dumpling tidb-lightning
```

### Export with Dumpling

```bash
# Export data from TiDB Cloud
dumpling \
  -u {TEST_DB_USER} \
  -p {TEST_DB_PASSWORD} \
  -P {TEST_DB_PORT} \
  -h {TEST_DB_HOST} \
  -o /tmp/tidb-export \
  --filetype sql \
  -r 20000 \
  -F 256MiB
```

### Import with TiDB Lightning

1. **Create configuration file** (`lightning.toml`):
   ```toml
   [lightning]
   level = "info"
   file = "tidb-lightning.log"
   
   [tikv-importer]
   backend = "local"
   sorted-kv-dir = "/tmp/sorted-kv-dir"
   
   [mydumper]
   data-source-dir = "/tmp/tidb-export"
   no-schema = false
   
   [tidb]
   host = "127.0.0.1"
   port = 4000
   user = "root"
   password = ""
   status-port = 10080
   pd-addr = "127.0.0.1:2379"
   ```

2. **Run TiDB Lightning**:
   ```bash
   tidb-lightning -config lightning.toml
   ```

## Approach 3: Periodic Sync Script

Create a script for periodic data sync:

### Export Script (`export-cloud.sh`)
```bash
#!/bin/bash

# Source .env file
source .env

# Export data using mysqldump (built-in tool)
mysqldump \
  -h $TEST_DB_HOST \
  -P $TEST_DB_PORT \
  -u $TEST_DB_USER \
  -p$TEST_DB_PASSWORD \
  --single-transaction \
  --routines \
  --triggers \
  $DATABASE_NAME \
  $TABLES > /tmp/cloud-export.sql

echo "Export completed: /tmp/cloud-export.sql"
```

### Import Script (`import-local.sh`)
```bash
#!/bin/bash

# Import to local TiDB
mysql -h 127.0.0.1 -P 4000 -u root < /tmp/cloud-export.sql

echo "Import completed to local TiDB"
```

### Combined Sync Script (`sync-data.sh`)
```bash
#!/bin/bash

echo "🔄 Syncing data from TiDB Cloud to local TiDB..."

# Export from cloud
./export-cloud.sh

# Import to local
./import-local.sh

echo "✅ Sync completed!"
```

## Approach 4: Application-Level Sync (For Continuous Updates)

For real-time sync, implement in your application:

```python
# Example Python script for selective sync
import mysql.connector

# Connect to both databases
cloud_db = mysql.connector.connect(
    host="gateway01.ap-northeast-1.prod.aws.tidbcloud.com",
    port=4000,
    user="3mmwxY44wQF4L6P.root",
    password="JQ8wbPsXfx7xJOR5",
    database="workflow_local"
)

local_db = mysql.connector.connect(
    host="127.0.0.1",
    port=4000,
    user="root",
    password="",
    database="workflow_local"
)

# Sync specific tables
def sync_table(table_name):
    # Get data from cloud
    cloud_cursor = cloud_db.cursor()
    cloud_cursor.execute(f"SELECT * FROM {table_name}")
    rows = cloud_cursor.fetchall()
    
    # Clear and insert into local
    local_cursor = local_db.cursor()
    local_cursor.execute(f"DELETE FROM {table_name}")
    
    if rows:
        placeholders = ','.join(['%s'] * len(rows[0]))
        local_cursor.executemany(
            f"INSERT INTO {table_name} VALUES ({placeholders})", 
            rows
        )
    
    local_db.commit()
    print(f"Synced {len(rows)} rows to {table_name}")

# Sync your tables
sync_table("plans")
```

## Recommended Solution for Your Setup

For development purposes, I recommend:

1. **Use Approach 1** (Console Export + SQL Import) for simplicity
2. **Create helper scripts** for periodic sync
3. **Consider application-level sync** for real-time needs

### Quick Setup

Create these helper scripts in your project:

```bash
# Make scripts executable
chmod +x sync-data.sh export-cloud.sh import-local.sh

# Run sync
./sync-data.sh
```

## Limitations and Considerations

### TiDB Cloud Serverless Limitations
- No traditional MySQL binlog access
- Limited to export/import methods
- No direct replication support in most plans

### Performance Considerations
- Full table exports can be slow for large datasets
- Network bandwidth affects sync speed
- Consider incremental exports for large tables

### Security Notes
- Store credentials securely (use .env file)
- Use TLS connections when possible
- Rotate credentials regularly

## Troubleshooting

### Connection Issues
```bash
# Test connection to TiDB Cloud
mysql -h $TEST_DB_HOST -P $TEST_DB_PORT -u $TEST_DB_USER -p

# Test connection to local TiDB
mysql -h 127.0.0.1 -P 4000 -u root
```

### Export Errors
- Ensure user has SELECT privileges
- Check network connectivity
- Verify table existence

### Import Errors
- Check schema compatibility
- Ensure sufficient disk space
- Verify TiDB is running

## References

- [TiDB Cloud Export Documentation](https://docs.pingcap.com/tidbcloud/serverless-export/)
- [TiDB Migration Tools Overview](https://docs.pingcap.com/tidb/stable/migration-tools)
- [Dumpling Documentation](https://docs.pingcap.com/tidb/stable/dumpling-overview)
- [TiDB Lightning Documentation](https://docs.pingcap.com/tidb/stable/tidb-lightning-overview)

For production use cases, contact TiDB Cloud Support to discuss available replication options for your specific plan.