zhukang 6143abc83b docs: add comprehensive documentation

- Add contributing guidelines to README
- Add detailed architecture documentation
- Add text analysis task documentation
- Include usage examples and best practices

2025-01-14 21:05:33 +08:00

3.1 KiB

Raw Blame History

Architecture Overview

Core Components

TaskExecutor

The TaskExecutor is the base class for all task implementations. It provides:

Task Step Management
- Sequential step execution
- State tracking
- Checkpoint creation
- Error handling
LLM Integration
- Asynchronous API calls
- Retry mechanisms
- Response validation

Configuration System

Config Loader
- YAML configuration files
- Environment variable support
- Configuration validation
Secure Configuration
- Encrypted storage for sensitive data
- Key management
- Secure API key handling

Task Implementation

Step Definition

Each task is defined as a series of steps:

self.task_steps = [
    {
        "id": "step_id",
        "name": "Step Name",
        "required_info": ["required_data"],
        "instruction": "Step instruction for LLM"
    }
]

Step Handlers

Step handlers are implemented as async methods:

async def handle_step_id(self, step_input: dict) -> dict:
    # 1. Process input
    processed_data = self.preprocess(step_input)
    
    # 2. Call LLM if needed
    llm_response = await self.llm_call(
        instruction=step_input["instruction"],
        context=processed_data
    )
    
    # 3. Process response
    result = self.postprocess(llm_response)
    
    return {"result": result}

Execution Flow

Initialization

executor = TaskExecutor(llm_model="model_name")

Task Setup
```
executor.task_steps = [...]
```

Execution

result = await executor.execute(input_data)

Step Processing
- Validate input
- Execute step handler
- Create checkpoint
- Handle errors
- Move to next step
Completion
- Return final result
- Clean up resources

Error Handling

Retry Mechanism
- API call retries with exponential backoff
- Configurable retry limits
Error Types
- TaskExecutionError: General execution errors
- StepValidationError: Input validation failures
- LLMError: LLM API related errors
Recovery
- Checkpoint-based recovery
- State restoration
- Partial results handling

Best Practices

Task Design
- Keep steps atomic and focused
- Clear step instructions
- Proper input validation
- Comprehensive error handling
LLM Usage
- Clear and specific prompts
- Response validation
- Handle token limits
- Consider cost and latency
Testing
- Unit tests for each step
- Integration tests for full flow
- Mock LLM calls in tests
- Test error scenarios
Security
- Secure API key handling
- Input sanitization
- Output validation
- Access control

Extension Points

Custom Step Handlers
- Implement custom logic
- Add new capabilities
- Integrate external services
LLM Providers
- Support multiple providers
- Custom response parsing
- Provider-specific optimizations
Monitoring & Logging
- Custom metrics
- Logging handlers
- Performance monitoring

3.1 KiB Raw Blame History