433 lines
15 KiB
Markdown
433 lines
15 KiB
Markdown
# Centralized AI Calls Specification
|
|
|
|
## Overview
|
|
|
|
This document specifies the centralized AI call system for PowerOn, replacing the current distributed AI call patterns with a unified approach that provides better optimization, consistency, and maintainability.
|
|
|
|
## Current State Analysis
|
|
|
|
### Existing AI Call Patterns
|
|
|
|
The current codebase uses multiple AI call patterns across different modules:
|
|
|
|
1. **ServiceCenter** - Basic/Advanced text and image processing
|
|
2. **HandlingTasks** - Task planning, action planning, result validation
|
|
3. **MethodOutlook** - Email composition with document context
|
|
4. **MethodAi** - General AI processing with document context
|
|
5. **MethodDocument** - HTML report generation and document conversion
|
|
|
|
### Current Implementation Issues
|
|
|
|
- **Inconsistent Model Selection**: Different modules use different AI models without optimization
|
|
- **Redundant Document Processing**: Each module implements its own document extraction logic
|
|
- **No Centralized Optimization**: No unified approach to content compression or failover
|
|
- **Maintenance Overhead**: Changes to AI logic require updates across multiple modules
|
|
|
|
## Centralized AI Call System
|
|
|
|
### Core Interface
|
|
|
|
```python
|
|
async def callAi(
|
|
prompt: str,
|
|
documents: List[ChatDocument] = None,
|
|
options: Dict[str, Any] = None
|
|
) -> CentralizedAiResponse
|
|
```
|
|
|
|
### Options Specification
|
|
|
|
```python
|
|
options = {
|
|
"process_type": "text" | "image", # Content type to process
|
|
"operation_type": "generate_plan" | "summarize_data" | "extract_content" | "analyse_content" | "generate_content" | "email_composition" | "report_generation" | "task_planning" | "result_validation",
|
|
"priority": "speed" | "quality" | "cost" | "balanced", # Model selection priority
|
|
"compress_prompt": bool, # Allow prompt compression
|
|
"compress_documents": bool, # Allow document compression
|
|
"process_documents_individually": bool, # Process documents separately
|
|
"max_cost": float, # Maximum cost limit
|
|
"max_processing_time": int, # Max processing time in seconds
|
|
"max_size_bytes": int, # Maximum result size
|
|
"result_format_requested": str, # Desired output format
|
|
"include_metadata": bool, # Include document metadata
|
|
"processing_mode": "basic" | "advanced" | "detailed" # Processing depth
|
|
}
|
|
```
|
|
|
|
### Response Format
|
|
|
|
The centralized AI call returns a standardized `CentralizedAiResponse` object:
|
|
|
|
```python
|
|
class CentralizedAiResponse(BaseModel):
|
|
aiResults: List[AiResult] = Field(default_factory=list)
|
|
success: bool = Field(description="Whether the AI call was successful")
|
|
error: Optional[str] = Field(None, description="Error message if the call failed")
|
|
|
|
class AiResult(BaseModel):
|
|
filename: str = Field(description="Name of the result document")
|
|
mimetype: str = Field(description="MIME type of the result document")
|
|
content: str = Field(description="Content of the result document")
|
|
```
|
|
|
|
## Current AI Calls Mapping
|
|
|
|
### 1. ServiceCenter AI Calls
|
|
|
|
#### Functions to DELETE from ServiceCenter
|
|
```python
|
|
# DELETE these functions from ServiceCenter:
|
|
async def callAiTextBasic(self, prompt: str, context: str = None) -> str
|
|
async def callAiTextAdvanced(self, prompt: str, context: str = None) -> str
|
|
async def callAiImageBasic(self, prompt: str, imageData: str, mimeType: str) -> str
|
|
async def callAiImageAdvanced(self, prompt: str, imageData: str, mimeType: str) -> str
|
|
async def extractContentFromDocument(self, prompt: str, document: ChatDocument) -> ExtractedContent
|
|
```
|
|
|
|
#### Direct Centralized Replacement - All calls go directly to interfaceAiCalls
|
|
```python
|
|
# Basic text processing - Direct call with speed priority
|
|
response = await self.service.interfaceAiCalls.callAi(
|
|
prompt=prompt,
|
|
documents=None,
|
|
options={
|
|
"process_type": "text",
|
|
"operation_type": "generate_content",
|
|
"priority": "speed",
|
|
"compress_prompt": True,
|
|
"compress_documents": False,
|
|
"max_cost": 0.01
|
|
}
|
|
)
|
|
# Access result: response.aiResults, response.success, response.error
|
|
|
|
# Advanced text processing - Direct call with quality priority
|
|
response = await self.service.interfaceAiCalls.callAi(
|
|
prompt=prompt,
|
|
documents=None,
|
|
options={
|
|
"process_type": "text",
|
|
"operation_type": "generate_content",
|
|
"priority": "quality",
|
|
"compress_prompt": False,
|
|
"compress_documents": False,
|
|
"max_cost": 0.05
|
|
}
|
|
)
|
|
# Access result: response.aiResults, response.success, response.error
|
|
|
|
# Image processing - Direct call with image documents
|
|
image_doc = ChatDocument(fileData=imageData, fileName="image", mimeType=mimeType)
|
|
response = await self.service.interfaceAiCalls.callAi(
|
|
prompt=prompt,
|
|
documents=[image_doc],
|
|
options={
|
|
"process_type": "image",
|
|
"operation_type": "analyse_content",
|
|
"priority": "balanced",
|
|
"compress_documents": True,
|
|
"max_cost": 0.03
|
|
}
|
|
)
|
|
# Access result: response.aiResults, response.success, response.error
|
|
```
|
|
|
|
### 2. HandlingTasks AI Calls
|
|
|
|
#### Current Implementation (TO BE REMOVED)
|
|
```python
|
|
# Task planning
|
|
prompt = await self.service.callAiTextAdvanced(task_planning_prompt)
|
|
|
|
# Action planning
|
|
prompt = await self.service.callAiTextAdvanced(action_prompt)
|
|
|
|
# Result validation
|
|
response = await self.service.callAiTextAdvanced(prompt)
|
|
```
|
|
|
|
#### Direct Centralized Replacement
|
|
```python
|
|
# Task planning - Direct call with detailed processing
|
|
response = await self.service.interfaceAiCalls.callAi(
|
|
prompt=task_planning_prompt,
|
|
documents=None,
|
|
options={
|
|
"process_type": "text",
|
|
"operation_type": "generate_plan",
|
|
"priority": "quality",
|
|
"compress_prompt": False,
|
|
"compress_documents": False,
|
|
"processing_mode": "detailed",
|
|
"max_cost": 0.10,
|
|
"max_processing_time": 60
|
|
}
|
|
)
|
|
# Access result: response.aiResults[0].content for the plan text
|
|
|
|
# Action planning - Direct call with detailed processing
|
|
response = await self.service.interfaceAiCalls.callAi(
|
|
prompt=action_prompt,
|
|
documents=None,
|
|
options={
|
|
"process_type": "text",
|
|
"operation_type": "generate_plan",
|
|
"priority": "quality",
|
|
"compress_prompt": False,
|
|
"compress_documents": False,
|
|
"processing_mode": "detailed",
|
|
"max_cost": 0.10,
|
|
"max_processing_time": 60
|
|
}
|
|
)
|
|
# Access result: response.aiResults[0].content for the action plan
|
|
|
|
# Result validation - Direct call with analysis focus
|
|
response = await self.service.interfaceAiCalls.callAi(
|
|
prompt=prompt,
|
|
documents=None,
|
|
options={
|
|
"process_type": "text",
|
|
"operation_type": "analyse_content",
|
|
"priority": "balanced",
|
|
"compress_prompt": True,
|
|
"compress_documents": False,
|
|
"processing_mode": "advanced",
|
|
"max_cost": 0.05,
|
|
"max_processing_time": 30
|
|
}
|
|
)
|
|
# Access result: response.aiResults[0].content for the validation result
|
|
```
|
|
|
|
### 3. MethodOutlook AI Calls
|
|
|
|
#### Current Implementation (TO BE REMOVED)
|
|
```python
|
|
# Document extraction
|
|
extracted_content = await self.service.extractContentFromDocument(
|
|
prompt="Extract readable text content for email composition",
|
|
document=doc
|
|
)
|
|
|
|
# Email composition
|
|
composed_email = await self.service.callAiTextAdvanced(ai_prompt)
|
|
```
|
|
|
|
#### Direct Centralized Replacement
|
|
```python
|
|
# Email composition with document context - Direct call
|
|
response = await self.service.interfaceAiCalls.callAi(
|
|
prompt=ai_prompt,
|
|
documents=documents,
|
|
options={
|
|
"process_type": "text",
|
|
"operation_type": "email_composition",
|
|
"priority": "speed",
|
|
"compress_prompt": True,
|
|
"compress_documents": True,
|
|
"process_documents_individually": False,
|
|
"result_format_requested": "email",
|
|
"include_metadata": True,
|
|
"max_cost": 0.02,
|
|
"max_processing_time": 15
|
|
}
|
|
)
|
|
# Access result: response.aiResults[0].content for the email content
|
|
# response.aiResults[0].mimetype will contain the MIME type of the delivered content
|
|
```
|
|
|
|
### 4. MethodAi AI Calls
|
|
|
|
#### Current Implementation (TO BE REMOVED)
|
|
```python
|
|
# Document extraction
|
|
extracted_content = await self.service.extractContentFromDocument(
|
|
prompt=extraction_prompt,
|
|
document=doc
|
|
)
|
|
|
|
# AI processing
|
|
if processingMode in ["advanced", "detailed"]:
|
|
result = await self.service.callAiTextAdvanced(enhanced_prompt, context)
|
|
else:
|
|
result = await self.service.callAiTextBasic(enhanced_prompt, context)
|
|
```
|
|
|
|
#### Direct Centralized Replacement
|
|
```python
|
|
# AI processing with document context - Direct call
|
|
response = await self.service.interfaceAiCalls.callAi(
|
|
prompt=enhanced_prompt,
|
|
documents=documents,
|
|
options={
|
|
"process_type": "text",
|
|
"operation_type": "generate_content",
|
|
"priority": "quality" if processingMode in ["advanced", "detailed"] else "speed",
|
|
"compress_prompt": processingMode != "detailed",
|
|
"compress_documents": True,
|
|
"process_documents_individually": True,
|
|
"processing_mode": processingMode,
|
|
"result_format_requested": output_format,
|
|
"include_metadata": includeMetadata,
|
|
"max_cost": 0.05 if processingMode in ["advanced", "detailed"] else 0.02,
|
|
"max_processing_time": 45 if processingMode in ["advanced", "detailed"] else 20
|
|
}
|
|
)
|
|
# Access result: response.aiResults for all generated documents
|
|
# Each aiResult.mimetype will contain the MIME type of the delivered content
|
|
```
|
|
|
|
### 5. MethodDocument AI Calls
|
|
|
|
#### Current Implementation (TO BE REMOVED)
|
|
```python
|
|
# Document extraction for HTML report
|
|
extracted_content = await self.service.extractContentFromDocument(
|
|
prompt="Extract readable text content for HTML report generation",
|
|
document=doc
|
|
)
|
|
|
|
# HTML report generation
|
|
aiReport = await self.service.callAiTextBasic(aiPrompt, combinedContent)
|
|
```
|
|
|
|
#### Direct Centralized Replacement
|
|
```python
|
|
# HTML report generation with document context - Direct call
|
|
response = await self.service.interfaceAiCalls.callAi(
|
|
prompt=aiPrompt,
|
|
documents=documents,
|
|
options={
|
|
"process_type": "text",
|
|
"operation_type": "report_generation",
|
|
"priority": "quality",
|
|
"compress_prompt": False,
|
|
"compress_documents": True,
|
|
"process_documents_individually": True,
|
|
"result_format_requested": "html",
|
|
"include_metadata": includeMetadata,
|
|
"processing_mode": "detailed",
|
|
"max_cost": 0.08,
|
|
"max_processing_time": 90
|
|
}
|
|
)
|
|
# Access result: response.aiResults[0].content for the HTML report
|
|
# response.aiResults[0].mimetype will contain the MIME type of the delivered content
|
|
```
|
|
|
|
## Document Processing Integration
|
|
|
|
### Centralized Document Extraction
|
|
|
|
The centralized system integrates with `DocumentExtraction.processFileData()` for all document processing:
|
|
|
|
```python
|
|
# Document extraction is handled automatically by the centralized callAi method
|
|
# No need for manual extractContentFromDocument calls
|
|
```
|
|
|
|
### Document Processing Options
|
|
|
|
- **Individual Processing**: Each document processed separately with its own context
|
|
- **Batch Processing**: All documents processed together for comprehensive analysis
|
|
- **Metadata Inclusion**: Optional inclusion of file metadata (size, type, etc.)
|
|
- **Content Compression**: Automatic compression for large documents
|
|
|
|
## Technical Benefits
|
|
|
|
- **Unified Model Selection**: Automatic selection based on content size, operation type, and priority
|
|
- **Consistent Document Processing**: Single point of document extraction logic
|
|
- **Content Compression**: Automatic compression for large prompts and documents
|
|
- **Centralized Error Handling**: Consistent error handling and logging across all AI operations
|
|
|
|
## Migration Strategy
|
|
|
|
### Phase 1: Direct Call Implementation
|
|
- Replace all `callAiTextBasic/Advanced` calls with direct `interfaceAiCalls.callAi()` calls
|
|
- Remove intermediate wrapper functions
|
|
- Each AI call gets specific options tailored to its needs
|
|
|
|
### Phase 2: DELETE Legacy Methods from ServiceCenter
|
|
- **DELETE** `callAiTextBasic`, `callAiTextAdvanced`, `callAiImageBasic`, `callAiImageAdvanced` methods from ServiceCenter
|
|
- **DELETE** `extractContentFromDocument` method from ServiceCenter (handled automatically by centralized system)
|
|
- Update all modules to use direct `self.service.interfaceAiCalls.callAi()` calls
|
|
|
|
### Phase 3: Optimize and Test
|
|
- Fine-tune options for each specific use case
|
|
- Add comprehensive testing for all AI call scenarios
|
|
- Monitor performance and cost optimization
|
|
|
|
### Phase 4: Documentation and Cleanup
|
|
- Update all documentation to reflect direct call approach
|
|
- Remove any remaining legacy code references
|
|
- Finalize centralized AI call patterns
|
|
|
|
## Configuration Examples
|
|
|
|
### High-Quality Document Analysis
|
|
```python
|
|
options = {
|
|
"process_type": "text",
|
|
"operation_type": "analyse_content",
|
|
"priority": "quality",
|
|
"compress_prompt": False,
|
|
"compress_documents": False,
|
|
"process_documents_individually": True,
|
|
"include_metadata": True,
|
|
"processing_mode": "detailed"
|
|
}
|
|
# Response: response.aiResults[0].content contains analysis
|
|
```
|
|
|
|
### Fast Email Composition
|
|
```python
|
|
options = {
|
|
"process_type": "text",
|
|
"operation_type": "email_composition",
|
|
"priority": "speed",
|
|
"compress_prompt": True,
|
|
"compress_documents": True,
|
|
"result_format_requested": "email",
|
|
"max_cost": 0.10
|
|
}
|
|
# Response: response.aiResults[0].content contains email
|
|
# response.aiResults[0].mimetype shows the MIME type of delivered content
|
|
```
|
|
|
|
### Cost-Optimized Report Generation
|
|
```python
|
|
options = {
|
|
"process_type": "text",
|
|
"operation_type": "report_generation",
|
|
"priority": "cost",
|
|
"compress_prompt": True,
|
|
"compress_documents": True,
|
|
"result_format_requested": "html",
|
|
"max_cost": 0.05,
|
|
"max_processing_time": 30
|
|
}
|
|
# Response: response.aiResults[0].content contains HTML report
|
|
# response.aiResults[0].mimetype shows the MIME type of delivered content
|
|
```
|
|
|
|
## Implementation Requirements
|
|
|
|
### **Direct Interface Usage**
|
|
All AI calls must use:
|
|
```python
|
|
await self.service.interfaceAiCalls.callAi(prompt, documents, options)
|
|
```
|
|
|
|
### **Functions to Remove**
|
|
- `self.service.callAiTextBasic()`
|
|
- `self.service.callAiTextAdvanced()`
|
|
- `self.service.callAiImageBasic()`
|
|
- `self.service.callAiImageAdvanced()`
|
|
- `self.service.extractContentFromDocument()`
|
|
|
|
### **Configuration per Use Case**
|
|
- **Task Planning**: Quality priority, detailed processing, 60s timeout
|
|
- **Email Composition**: Speed priority, compressed content, 15s timeout
|
|
- **Report Generation**: Quality priority, detailed processing, 90s timeout
|
|
- **Image Analysis**: Balanced priority, compressed documents
|