gateway/AI_ENGINE_MIGRATION_PLAN.md

AI Engine Migration Plan

Overview

This document outlines the migration strategy from the current AI call system to the new Smart AI Engine architecture.

Benefits of the New Architecture

1. Separation of Concerns

• Applications no longer need to worry about content size limits
• Centralized AI model selection and failover
• Intelligent content reduction strategies

2. Improved Reliability

• Automatic handling of "content too large" errors
• Multiple fallback strategies
• Model-specific optimization

3. Better Performance

• Optimal model selection based on content characteristics
• Intelligent chunking and processing strategies
• Reduced API costs through smart model selection

4. Enhanced Maintainability

• Single point of AI logic
• Easy to add new models and strategies
• Consistent error handling

Migration Phases

Phase 1: Infrastructure Setup (Week 1-2)

1. Create AI Engine Interface

   • ✅ interfaceAiEngine.py - Core interfaces and data structures
   • ✅ aiEngine.py - Smart AI Engine implementation
   • ✅ serviceCenter_ai_engine.py - ServiceCenter integration

2. Update Dependencies

   • Add new imports to existing modules
   • Update configuration for AI model selection
   • Add logging for AI engine operations

Phase 2: ServiceCenter Integration (Week 3)

1. Update ServiceCenter Class

   # Add to ServiceCenter.__init__
   self.ai_engine = ServiceCenterAIEngine(self)
   
   # Replace existing AI methods
   async def callAiTextAdvanced(self, prompt: str, context: str = None) -> str:
       return await self.ai_engine.callAiTextAdvanced(prompt, context)
   
   async def callAiTextBasic(self, prompt: str, context: str = None) -> str:
       return await self.ai_engine.callAiTextBasic(prompt, context)
   
   async def extractContentFromDocument(self, prompt: str, document: ChatDocument) -> str:
       return await self.ai_engine.extractContentFromDocument(prompt, document)
   
   async def summarizeChat(self, messages: List[ChatMessage]) -> str:
       return await self.ai_engine.summarizeChat(messages)
   

2. Add New Document-Aware Methods

   async def callAiWithDocuments(
       self,
       prompt: str,
       documents: List[ChatDocument] = None,
       operation_type: str = "general"
   ) -> str:
       return await self.ai_engine.callAiWithDocuments(
           prompt, documents, operation_type=operation_type
       )
   

Phase 3: Method Updates (Week 4-5)

1. Update MethodWeb.py

   # Before
   web_scrape_result = await web_interface.scrape(web_scrape_request)
   
   # After - no changes needed, but can be enhanced
   # The AI engine will automatically handle large content
   

2. Update MethodDocument.py

   # Before
   formatted_content = await self.service.callAiTextBasic(ai_prompt, content)
   
   # After
   formatted_content = await self.service.callAiForReportGeneration(
       prompt=ai_prompt,
       documents=chat_documents
   )
   

3. Update MethodAi.py

   # Before
   result = await self.service.callAiTextAdvanced(enhanced_prompt, context)
   
   # After
   result = await self.service.callAiWithDocuments(
       prompt=enhanced_prompt,
       documents=document_list,
       operation_type="ai_processing"
   )
   

4. Update MethodOutlook.py

   # Before
   composed_email = await self.service.interfaceAiCalls.callAiTextAdvanced(ai_prompt)
   
   # After
   composed_email = await self.service.callAiForEmailComposition(
       prompt=ai_prompt,
       documents=attached_documents
   )
   

Phase 4: Task Handling Updates (Week 6)

1. Update handlingTasks.py

   # Before
   prompt = await self.service.callAiTextAdvanced(task_planning_prompt)
   
   # After
   prompt = await self.service.callAiForTaskPlanning(
       prompt=task_planning_prompt,
       documents=available_documents,
       context=workflow_context
   )
   

2. Update promptFactory.py

   # Before
   messageSummary = await service.summarizeChat(context.workflow.messages)
   
   # After - no changes needed, method signature stays the same
   # But internally uses the new AI engine
   

Phase 5: Testing and Optimization (Week 7-8)

1. Unit Tests

   • Test AI engine with various content sizes
   • Test fallback strategies
   • Test model selection logic

2. Integration Tests

   • Test with real documents of various sizes
   • Test error scenarios
   • Test performance improvements

3. Performance Monitoring

   • Monitor AI call success rates
   • Monitor processing times
   • Monitor cost savings

Code Changes Required

1. ServiceCenter Updates

# Add to ServiceCenter.__init__
from modules.chat.serviceCenter_ai_engine import ServiceCenterAIEngine
self.ai_engine_wrapper = ServiceCenterAIEngine(self)

# Update existing methods to use AI engine
async def callAiTextAdvanced(self, prompt: str, context: str = None) -> str:
    return await self.ai_engine_wrapper.callAiTextAdvanced(prompt, context)

async def callAiTextBasic(self, prompt: str, context: str = None) -> str:
    return await self.ai_engine_wrapper.callAiTextBasic(prompt, context)

async def extractContentFromDocument(self, prompt: str, document: ChatDocument) -> str:
    return await self.ai_engine_wrapper.extractContentFromDocument(prompt, document)

async def summarizeChat(self, messages: List[ChatMessage]) -> str:
    return await self.ai_engine_wrapper.summarizeChat(messages)

2. Method Updates (Optional Enhancements)

# Enhanced method calls with document awareness
async def process_documents_with_ai(self, prompt: str, documents: List[ChatDocument]):
    return await self.service.callAiWithDocuments(
        prompt=prompt,
        documents=documents,
        operation_type="document_processing"
    )

3. Configuration Updates

# Add to config.ini
[AI_ENGINE]
DEFAULT_MODEL=anthropic_claude
FALLBACK_MODEL=openai_gpt35
MAX_CONTENT_SIZE=100000
ENABLE_CONTENT_REDUCTION=true
CONTENT_REDUCTION_THRESHOLD=0.8

Backward Compatibility

1. Method Signatures

• All existing method signatures remain unchanged
• Internal implementation uses new AI engine
• No breaking changes for existing code

2. Error Handling

• Same error types and messages
• Enhanced error recovery with fallback strategies
• Better error reporting with processing details

3. Performance

• Same or better performance
• Automatic optimization based on content
• Reduced API costs through smart model selection

Risk Mitigation

1. Gradual Rollout

• Deploy with feature flags
• A/B testing with subset of users
• Rollback capability

2. Monitoring

• Comprehensive logging of AI engine operations
• Performance metrics tracking
• Error rate monitoring

3. Fallback Strategy

• Keep original AI call methods as backup
• Automatic fallback to original methods on errors
• Manual override capability

Expected Benefits

1. Immediate Benefits

• Elimination of "content too large" errors
• Better handling of large documents
• Improved user experience

2. Long-term Benefits

• Easier addition of new AI models
• Better cost optimization
• Enhanced content processing capabilities
• Improved system reliability

3. Developer Benefits

• Simplified AI integration
• No need to worry about content size limits
• Consistent AI behavior across the system
• Better debugging and monitoring

Success Metrics

1. Error Reduction

• 90% reduction in "content too large" errors
• 50% reduction in AI call failures
• 95% success rate for document processing

2. Performance Improvement

• 20% faster processing for large documents
• 30% reduction in API costs
• 50% reduction in retry attempts

3. User Experience

• Faster response times
• More reliable document processing
• Better content extraction quality

Conclusion

The new AI Engine architecture provides a robust, scalable solution for handling AI calls with large content. The migration can be done gradually with full backward compatibility, ensuring minimal risk while providing significant benefits in reliability, performance, and maintainability.