167 lines
4.4 KiB
Markdown
167 lines
4.4 KiB
Markdown
# Grok Client Implementation
|
|
|
|
This implementation provides a client for interacting with the Grok-3 model via OpenRouter's API, similar to the existing Ollama client.
|
|
|
|
## Features
|
|
|
|
- **Async Interface**: Compatible with the existing async architecture
|
|
- **Health Checks**: Built-in connectivity testing
|
|
- **Think Tag Stripping**: Optional removal of `<think></think>` tags from responses
|
|
- **Configurable**: Easy configuration through the existing config system
|
|
- **Error Handling**: Comprehensive error handling and logging
|
|
|
|
## Setup
|
|
|
|
### 1. Install Dependencies
|
|
|
|
The required dependencies are already included in `requirements.txt`:
|
|
|
|
```bash
|
|
pip install -r requirements.txt
|
|
```
|
|
|
|
### 2. Configure OpenRouter API Key
|
|
|
|
You need to obtain an API key from [OpenRouter](https://openrouter.ai/). Then configure it in one of two ways:
|
|
|
|
#### Option A: Using config.json (Recommended)
|
|
|
|
Copy the example configuration and update it with your API key:
|
|
|
|
```bash
|
|
cp config/config.json.example config/config.json
|
|
```
|
|
|
|
Edit `config/config.json` and set your OpenRouter API key:
|
|
|
|
```json
|
|
{
|
|
"openrouter_api_key": "your_actual_api_key_here",
|
|
"openrouter_site_url": "https://your-site.com",
|
|
"openrouter_site_name": "Your Site Name"
|
|
}
|
|
```
|
|
|
|
#### Option B: Direct Configuration
|
|
|
|
Modify the default values in `config.py`:
|
|
|
|
```python
|
|
openrouter_api_key: str = "your_actual_api_key_here"
|
|
```
|
|
|
|
### 3. Test the Implementation
|
|
|
|
Run the test script to verify everything is working:
|
|
|
|
```bash
|
|
python test_grok_client.py
|
|
```
|
|
|
|
## Usage
|
|
|
|
### Basic Usage
|
|
|
|
```python
|
|
import asyncio
|
|
from config import Config
|
|
from grok_client import GrokClient
|
|
|
|
async def main():
|
|
config = Config()
|
|
|
|
async with GrokClient(config) as client:
|
|
# Check if the service is available
|
|
if await client.check_health():
|
|
# Generate a response
|
|
response = await client.generate_response("What is the meaning of life?")
|
|
print(response)
|
|
|
|
asyncio.run(main())
|
|
```
|
|
|
|
### Integration with Existing Code
|
|
|
|
The Grok client follows the same interface as the Ollama client, so you can easily swap between them:
|
|
|
|
```python
|
|
# Use Grok (default)
|
|
from grok_client import GrokClient
|
|
client = GrokClient(config)
|
|
|
|
# Or use Ollama (legacy)
|
|
from ollama_client import OllamaClient
|
|
client = OllamaClient(config)
|
|
|
|
# Both have the same interface
|
|
response = await client.generate_response("Your prompt here")
|
|
```
|
|
|
|
## Configuration Options
|
|
|
|
| Option | Description | Default |
|
|
|--------|-------------|---------|
|
|
| `openrouter_api_key` | Your OpenRouter API key | `""` |
|
|
| `openrouter_base_url` | OpenRouter API base URL | `"https://openrouter.ai/api/v1"` |
|
|
| `openrouter_model` | Model to use | `"x-ai/grok-3"` |
|
|
| `openrouter_site_url` | Your site URL for rankings | `""` |
|
|
| `openrouter_site_name` | Your site name for rankings | `""` |
|
|
|
|
## API Reference
|
|
|
|
### GrokClient
|
|
|
|
#### `__init__(config: Config)`
|
|
Initialize the Grok client with configuration.
|
|
|
|
#### `async generate_response(prompt: str, strip_think_tags: bool = True) -> Optional[str]`
|
|
Generate a response from Grok-3 for the given prompt.
|
|
|
|
- **prompt**: The input prompt for the model
|
|
- **strip_think_tags**: If True, removes `<think></think>` tags from the response
|
|
- **Returns**: The generated response text or None if failed
|
|
|
|
#### `async check_health() -> bool`
|
|
Check if OpenRouter service is available.
|
|
|
|
- **Returns**: True if the service is healthy, False otherwise
|
|
|
|
## Error Handling
|
|
|
|
The client includes comprehensive error handling:
|
|
|
|
- **API Key Missing**: Warns if no API key is configured
|
|
- **Network Errors**: Logs and handles connection issues
|
|
- **API Errors**: Handles OpenRouter API errors gracefully
|
|
- **Response Processing**: Safely processes and strips think tags
|
|
|
|
## Logging
|
|
|
|
The client uses the standard Python logging module. Set the log level to see detailed information:
|
|
|
|
```python
|
|
import logging
|
|
logging.basicConfig(level=logging.INFO)
|
|
```
|
|
|
|
## Troubleshooting
|
|
|
|
### Common Issues
|
|
|
|
1. **"OpenRouter API key not configured"**
|
|
- Make sure you've set the `openrouter_api_key` in your configuration
|
|
|
|
2. **"Grok client health check failed"**
|
|
- Check your internet connection
|
|
- Verify your API key is correct
|
|
- Ensure OpenRouter service is available
|
|
|
|
3. **Import errors**
|
|
- Make sure you've installed the requirements: `pip install -r requirements.txt`
|
|
|
|
### Getting Help
|
|
|
|
- Check the OpenRouter documentation: https://openrouter.ai/docs
|
|
- Verify your API key at: https://openrouter.ai/keys
|
|
- Review the test script (`test_grok_client.py`) for usage examples
|