xorbitlab
/
dom
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
dom/GROK_CLIENT.md

4.4 KiB
Raw Blame History

Grok Client Implementation

This implementation provides a client for interacting with the Grok-3 model via OpenRouter's API, similar to the existing Ollama client.

Features

  • Async Interface: Compatible with the existing async architecture
  • Health Checks: Built-in connectivity testing
  • Think Tag Stripping: Optional removal of <think></think> tags from responses
  • Configurable: Easy configuration through the existing config system
  • Error Handling: Comprehensive error handling and logging

Setup

1. Install Dependencies

The required dependencies are already included in requirements.txt:

pip install -r requirements.txt

2. Configure OpenRouter API Key

You need to obtain an API key from OpenRouter. Then configure it in one of two ways:

Option A: Using config.json (Recommended)

Copy the example configuration and update it with your API key:

cp config/config.json.example config/config.json

Edit config/config.json and set your OpenRouter API key:

{
  "openrouter_api_key": "your_actual_api_key_here",
  "openrouter_site_url": "https://your-site.com",
  "openrouter_site_name": "Your Site Name"
}

Option B: Direct Configuration

Modify the default values in config.py:

openrouter_api_key: str = "your_actual_api_key_here"

3. Test the Implementation

Run the test script to verify everything is working:

python test_grok_client.py

Usage

Basic Usage

import asyncio
from config import Config
from grok_client import GrokClient

async def main():
    config = Config()
    
    async with GrokClient(config) as client:
        # Check if the service is available
        if await client.check_health():
            # Generate a response
            response = await client.generate_response("What is the meaning of life?")
            print(response)

asyncio.run(main())

Integration with Existing Code

The Grok client follows the same interface as the Ollama client, so you can easily swap between them:

# Use Grok (default)
from grok_client import GrokClient
client = GrokClient(config)

# Or use Ollama (legacy)
from ollama_client import OllamaClient
client = OllamaClient(config)

# Both have the same interface
response = await client.generate_response("Your prompt here")

Configuration Options

Option Description Default
openrouter_api_key Your OpenRouter API key ""
openrouter_base_url OpenRouter API base URL "https://openrouter.ai/api/v1"
openrouter_model Model to use "x-ai/grok-3"
openrouter_site_url Your site URL for rankings ""
openrouter_site_name Your site name for rankings ""

API Reference

GrokClient

__init__(config: Config)

Initialize the Grok client with configuration.

async generate_response(prompt: str, strip_think_tags: bool = True) -> Optional[str]

Generate a response from Grok-3 for the given prompt.

  • prompt: The input prompt for the model
  • strip_think_tags: If True, removes <think></think> tags from the response
  • Returns: The generated response text or None if failed

async check_health() -> bool

Check if OpenRouter service is available.

  • Returns: True if the service is healthy, False otherwise

Error Handling

The client includes comprehensive error handling:

  • API Key Missing: Warns if no API key is configured
  • Network Errors: Logs and handles connection issues
  • API Errors: Handles OpenRouter API errors gracefully
  • Response Processing: Safely processes and strips think tags

Logging

The client uses the standard Python logging module. Set the log level to see detailed information:

import logging
logging.basicConfig(level=logging.INFO)

Troubleshooting

Common Issues

  1. "OpenRouter API key not configured"

    • Make sure you've set the openrouter_api_key in your configuration

  2. "Grok client health check failed"

    • Check your internet connection
    • Verify your API key is correct
    • Ensure OpenRouter service is available

  3. Import errors

    • Make sure you've installed the requirements: pip install -r requirements.txt

Getting Help