AI Chat Service CLI Testing Tool Documentation

Overview

The cli_test.py is a comprehensive command-line interface (CLI) tool for testing and interacting with the AI Chat Service JSON-RPC API. It provides an interactive chat interface, conversation management capabilities, and debugging features.

Features

Interactive Chat Interface: Real-time conversation with AI using prompt-toolkit
Conversation Management: Create, join, delete, and end conversations
Auto-Creation: Automatically create conversations when sending first message
Timeout Handling: User inactivity timeout with system prompts
Flexible Connection: Support for HTTP/HTTPS with custom hosts, ports, and paths
Verbose Logging: Detailed request/response logging for debugging
Error Handling: Comprehensive error detection and user-friendly messages
Multiple Operation Modes: Interactive chat, single message, or conversation management

Installation & Requirements

Dependencies

pip install aiohttp prompt-toolkit

Required Python Packages

aiohttp - Async HTTP client for API communication
prompt-toolkit - Interactive command-line interface
argparse - Command-line argument parsing (built-in)
asyncio - Asynchronous programming (built-in)
json - JSON handling (built-in)
uuid - UUID generation (built-in)
urllib.parse - URL parsing (built-in)
logging - Logging functionality (built-in)

Command-Line Arguments

Required Arguments

Argument	Type	Description
`--project-id`	string	Project ID for authentication (required)
`--token`	string	Authentication token (required)
`--convo-id`	string	Conversation ID to create, join, or manage (required)

Optional Arguments

Argument	Type	Default	Description
`--config-url`	string	None	URL to fetch YAML configuration for new conversations
`--reinit`	flag	False	Force reinitialize conversation even if it exists
`--delete`	flag	False	Delete the specified conversation
`--end`	flag	False	End the specified conversation
`--timeout`	integer	None	User inactivity timeout in milliseconds
`--conversation-timeout`	integer	3600	Conversation timeout in seconds
`--verbose`	flag	False	Enable verbose logging for debugging
`--chat-first`	string	None	Send message directly instead of creating conversation
`--url`	string	None	Full service URL (overrides host/port/proto/path)
`--host`	string	localhost	Service hostname
`--port`	integer	8080	Service port number
`--proto`	string	http	Protocol (http or https)
`--path`	string	""	URL path portion

Usage Examples

1. Interactive Chat Session

Start an interactive chat session with a new conversation:

python cli_test.py \
  --project-id "my-project" \
  --token "my-auth-token" \
  --convo-id "chat-session-001" \
  --config-url "https://example.com/chatbot-config.yaml"

Output:

create_result: {'jsonrpc': '2.0', 'result': {'status': 'created', 'id': 'chat-session-001', 'initial_message': 'Hello! How can I help you today?'}, 'id': '...'}
New conversation: chat-session-001

AI: Hello! How can I help you today?
You: What's the weather like?
AI: I'd be happy to help you with the weather! Could you please tell me which city or location you'd like to know the weather for?
You: exit
Exiting...

2. Join Existing Conversation

Join an existing conversation without reinitializing:

python cli_test.py \
  --project-id "my-project" \
  --token "my-auth-token" \
  --convo-id "existing-conversation-123"

Output:

create_result: {'jsonrpc': '2.0', 'result': {'status': 'exists', 'id': 'existing-conversation-123'}, 'id': '...'}
Rejoined existing conversation: existing-conversation-123
You: Continue our previous discussion
AI: Of course! Let me pick up where we left off...

3. Reinitialize Existing Conversation

Force reinitialize an existing conversation:

python cli_test.py \
  --project-id "my-project" \
  --token "my-auth-token" \
  --convo-id "existing-conversation-123" \
  --config-url "https://example.com/new-config.yaml" \
  --reinit

Output:

create_result: {'jsonrpc': '2.0', 'result': {'status': 'reinitialized', 'id': 'existing-conversation-123', 'initial_message': 'Welcome back! I've been reset with new capabilities.'}, 'id': '...'}
Reinitialized conversation: existing-conversation-123

AI: Welcome back! I've been reset with new capabilities.
You: What can you do now?

4. Send Single Message (Auto-Create)

Send a single message and automatically create conversation if needed:

python cli_test.py \
  --project-id "my-project" \
  --token "my-auth-token" \
  --convo-id "quick-question-001" \
  --config-url "https://example.com/config.yaml" \
  --chat-first "What's 2+2?"

Output:

AI: 2 + 2 equals 4.

5. Delete Conversation

Delete a conversation permanently:

python cli_test.py \
  --project-id "my-project" \
  --token "my-auth-token" \
  --convo-id "conversation-to-delete" \
  --delete

Output:

Delete result: {'status': 'deleted', 'id': 'conversation-to-delete'}

6. End Conversation

Mark a conversation as ended (moves to analytics):

python cli_test.py \
  --project-id "my-project" \
  --token "my-auth-token" \
  --convo-id "conversation-to-end" \
  --end

Output:

End conversation result: {'status': 'ended', 'id': 'conversation-to-end'}

7. Connect to Remote Service

Connect to a remote AI service with HTTPS:

python cli_test.py \
  --project-id "prod-project" \
  --token "prod-token" \
  --convo-id "remote-chat-001" \
  --config-url "https://config.example.com/prod-config.yaml" \
  --url "https://ai-service.example.com:443/api/v1"

8. Custom Host and Port

Connect using custom host and port:

python cli_test.py \
  --project-id "dev-project" \
  --token "dev-token" \
  --convo-id "dev-chat-001" \
  --config-url "http://localhost:3000/config.yaml" \
  --host "192.168.1.100" \
  --port "9090" \
  --proto "https"

9. Enable Verbose Logging

Enable detailed logging for debugging:

python cli_test.py \
  --project-id "debug-project" \
  --token "debug-token" \
  --convo-id "debug-session" \
  --config-url "https://example.com/config.yaml" \
  --verbose

Output with verbose logging:

INFO:__main__:Sending request to http://localhost:8080: method=create_conversation, params={'project_id': 'debug-project', 'token': 'debug-token', 'id': 'debug-session', 'config_url': 'https://example.com/config.yaml', 'reinit': False}
INFO:__main__:Received response: {'jsonrpc': '2.0', 'result': {'status': 'created', 'id': 'debug-session', 'initial_message': 'Hello!'}, 'id': '...'}

10. User Inactivity Timeout

Set user inactivity timeout (sends system message to AI):

python cli_test.py \
  --project-id "my-project" \
  --token "my-auth-token" \
  --convo-id "timeout-test" \
  --config-url "https://example.com/config.yaml" \
  --timeout 30000  # 30 seconds

Behavior: - If user doesn't respond within 30 seconds, sends system message: "The user has not responded, try to get their attention" - AI can then send a follow-up message to re-engage the user

Interactive Commands

While in interactive mode, you can use these commands:

Command	Description
`exit`	Exit the chat session
`quit`	Exit the chat session
`bye`	Exit the chat session
`Ctrl+C`	Force exit
`Ctrl+D`	EOF exit

Configuration Examples

Basic Configuration URL

Your config_url should point to a YAML file like this:

# Basic chatbot configuration
SWAIG:
  defaults:
    web_hook_url: "https://api.example.com/webhook"
  functions:
    - function: "get_weather"
      description: "Get current weather information"
      parameters:
        type: "object"
        properties:
          location:
            type: "string"
            description: "City or location name"

prompt:
  text: "You are a helpful assistant."
  temperature: 0.7

params:
  conversation_timeout: 3600
  sliding_window: 50

Advanced Configuration with Multiple Functions

# Advanced configuration with multiple SWAIG functions
SWAIG:
  defaults:
    web_hook_url: "https://api.example.com/swaig"
  functions:
    - function: "get_weather"
      description: "Get current weather for a location"
      parameters:
        type: "object"
        properties:
          location:
            type: "string"
            description: "City name"
        required: ["location"]

    - function: "search_web"
      description: "Search the web for information"
      parameters:
        type: "object"
        properties:
          query:
            type: "string"
            description: "Search query"
        required: ["query"]

prompt:
  text: |
    You are an intelligent assistant with access to weather and web search.
    Always be helpful and provide accurate information.
  temperature: 0.8

params:
  conversation_timeout: 7200
  sliding_window: 100

Error Handling

The CLI tool provides comprehensive error handling:

Common Errors and Solutions

1. Connection Refused

Error: Connection refused

Solution: Check if the AI service is running and accessible at the specified host/port.

2. Authentication Error

Error: Missing required key: project_id

Solution: Verify your --project-id and --token are correct.

3. Configuration Error

Error: Config fetch error

Solution: Ensure your --config-url is accessible and returns valid YAML.

4. Conversation Not Found

Error: The conversation was not found. It may have been deleted or timed out.

Solution: The conversation may have expired. Create a new one or check the conversation ID.

5. Invalid JSON Response

Error: Unexpected response format from server

Solution: Check service logs and ensure the API is responding correctly.

Advanced Usage Patterns

1. Automated Testing Script

Create a script for automated testing:

#!/bin/bash

PROJECT_ID="test-project"
TOKEN="test-token"
CONFIG_URL="https://example.com/test-config.yaml"

# Test conversation creation
python cli_test.py \
  --project-id "$PROJECT_ID" \
  --token "$TOKEN" \
  --convo-id "auto-test-$(date +%s)" \
  --config-url "$CONFIG_URL" \
  --chat-first "Hello, this is an automated test"

# Test conversation deletion
python cli_test.py \
  --project-id "$PROJECT_ID" \
  --token "$TOKEN" \
  --convo-id "auto-test-$(date +%s)" \
  --delete

2. Load Testing

Test multiple concurrent conversations:

#!/bin/bash

for i in {1..10}; do
  python cli_test.py \
    --project-id "load-test" \
    --token "load-token" \
    --convo-id "load-test-$i" \
    --config-url "https://example.com/config.yaml" \
    --chat-first "Load test message $i" &
done

wait  # Wait for all background processes

3. Environment-Based Configuration

Use environment variables for common settings:

#!/bin/bash

export AI_PROJECT_ID="my-project"
export AI_TOKEN="my-token"
export AI_CONFIG_URL="https://example.com/config.yaml"
export AI_HOST="localhost"
export AI_PORT="8080"

python cli_test.py \
  --project-id "$AI_PROJECT_ID" \
  --token "$AI_TOKEN" \
  --convo-id "env-test-001" \
  --config-url "$AI_CONFIG_URL" \
  --host "$AI_HOST" \
  --port "$AI_PORT"

Debugging and Troubleshooting

Enable Verbose Logging

Always use --verbose when debugging:

python cli_test.py \
  --project-id "debug" \
  --token "debug" \
  --convo-id "debug-001" \
  --config-url "https://example.com/config.yaml" \
  --verbose

Check Service Logs

Monitor the AI service logs while testing:

# In another terminal
sudo docker-compose logs -f web

Test Basic Connectivity

Test if the service is reachable:

curl -X POST http://localhost:8080/ \
  -H "Content-Type: application/json" \
  -d '{"jsonrpc":"2.0","method":"create_conversation","params":{"project_id":"test","token":"test","id":"test","config_url":"https://example.com/config.yaml"},"id":"1"}'

Validate Configuration URL

Test if your configuration URL is accessible:

curl -v "https://example.com/config.yaml"

Performance Considerations

Connection Reuse

The CLI tool uses aiohttp.ClientSession for efficient connection reuse during a session.

Memory Usage

For long interactive sessions, the tool maintains minimal memory footprint by not storing conversation history locally.

Timeout Settings

User timeout: Configurable via --timeout (milliseconds)
Conversation timeout: Configurable via --conversation-timeout (seconds)
HTTP timeout: Built into aiohttp client (30 seconds default)

Integration Examples

1. CI/CD Pipeline Integration

# .github/workflows/test-ai-service.yml
name: Test AI Service
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Setup Python
        uses: actions/setup-python@v2
        with:
          python-version: '3.9'
      - name: Install dependencies
        run: pip install aiohttp prompt-toolkit
      - name: Test AI Service
        run: |
          python cli_test.py \
            --project-id "${{ secrets.AI_PROJECT_ID }}" \
            --token "${{ secrets.AI_TOKEN }}" \
            --convo-id "ci-test-${{ github.run_id }}" \
            --config-url "${{ secrets.AI_CONFIG_URL }}" \
            --chat-first "CI/CD test message"

2. Monitoring Script

#!/usr/bin/env python3
import subprocess
import time
import json

def health_check():
    try:
        result = subprocess.run([
            'python', 'cli_test.py',
            '--project-id', 'health-check',
            '--token', 'health-token',
            '--convo-id', f'health-{int(time.time())}',
            '--config-url', 'https://example.com/config.yaml',
            '--chat-first', 'Health check'
        ], capture_output=True, text=True, timeout=30)

        if result.returncode == 0:
            print("✅ AI Service is healthy")
            return True
        else:
            print(f"❌ AI Service health check failed: {result.stderr}")
            return False
    except subprocess.TimeoutExpired:
        print("❌ AI Service health check timed out")
        return False

if __name__ == "__main__":
    health_check()

Security Considerations

Token Management

Never hardcode tokens in scripts
Use environment variables or secure credential stores
Rotate tokens regularly

Network Security

Use HTTPS in production environments
Validate SSL certificates
Consider VPN or private networks for sensitive deployments

Logging Security

Avoid logging sensitive data when using --verbose
Sanitize logs before sharing for debugging

Troubleshooting Guide

Issue: "ModuleNotFoundError: No module named 'prompt_toolkit'"

Solution:

pip install prompt-toolkit

Issue: "Connection refused" or "Connection timeout"

Solutions: 1. Check if the AI service is running: docker-compose ps 2. Verify the host and port: --host localhost --port 8080 3. Test basic connectivity: curl http://localhost:8080/

Issue: "Config fetch error"

Solutions: 1. Verify the config URL is accessible: curl https://example.com/config.yaml 2. Check YAML syntax validity 3. Ensure proper authentication if the config URL requires it

Issue: Interactive mode not working properly

Solutions: 1. Ensure you're using a proper terminal (not IDE console) 2. Try running with python -u cli_test.py for unbuffered output 3. Check if prompt-toolkit is properly installed

Issue: Verbose logging not showing

Solution: Ensure you're using the --verbose flag and check that logging is configured correctly.

Support and Contributing

Getting Help

Check this documentation first
Enable verbose logging for detailed error information
Check the AI service logs: docker-compose logs web
Verify your configuration YAML syntax and accessibility

Reporting Issues

When reporting issues, include: - Full command line used - Error messages (with --verbose enabled) - Configuration YAML (sanitized) - AI service logs (if accessible) - Environment details (Python version, OS, etc.)

Contributing

The CLI tool is designed to be extensible. Common areas for contribution: - Additional command-line options - Enhanced error handling - New output formats (JSON, CSV, etc.) - Integration with other tools - Performance improvements