gateway/notes/methodbased_specification.md
2025-06-10 18:19:33 +02:00

34 KiB

Chat System Process Flow Specification

1. System Overview

1.1 Core Components

  • WorkflowManager: Orchestrates the overall workflow process
  • ChatManager: Manages chat interactions and task execution
  • ServiceContainer: Central state and context management
  • AgentTask: Core data object for task execution

1.2 Service Container Structure

from enum import Enum
from typing import Dict, List, Optional, Any, Literal
from datetime import datetime, UTC
from pydantic import BaseModel, Field

class TaskStatus(str, Enum):
    PENDING = "pending"
    SUCCESS = "success"
    FAILED = "failed"
    RETRY = "retry"
    TIMEOUT = "timeout"
    ROLLBACK = "rollback"

class ActionStatus(str, Enum):
    PENDING = "pending"
    SUCCESS = "success"
    FAILED = "failed"
    RETRY = "retry"
    TIMEOUT = "timeout"
    SKIPPED = "skipped"
    DEPENDENCY_FAILED = "dependency_failed"

class AuthSource(str, Enum):
    LOCAL = "local"
    MSFT = "msft"
    GOOGLE = "google"
    # Add more auth sources as needed

class MethodParameter(BaseModel):
    """Model for method parameters"""
    name: str
    type: str
    required: bool
    validation: Optional[callable] = None
    description: str

class MethodResult(BaseModel):
    """Model for method results"""
    success: bool
    data: Dict[str, Any]
    metadata: Dict[str, Any]
    validation: List[str]

class MethodBase:
    """Base class for all methods"""
    
    def __init__(self, service):
        self.service = service
        self.name: str
        self.description: str
        self.auth_source: AuthSource = AuthSource.LOCAL  # Default to local auth
        
    @property
    def actions(self) -> Dict[str, Dict[str, Any]]:
        """Available actions and their parameters"""
        raise NotImplementedError
        
    async def execute(self, action: str, parameters: Dict[str, Any], auth_data: Optional[Dict[str, Any]] = None) -> MethodResult:
        """Execute method action with authentication data"""
        raise NotImplementedError
        
    async def validate_parameters(self, action: str, parameters: Dict[str, Any]) -> bool:
        """Validate action parameters"""
        if action not in self.actions:
            return False
            
        action_def = self.actions[action]
        required_params = {k for k, v in action_def['parameters'].items() if v['required']}
        return all(param in parameters for param in required_params)
        
    async def rollback(self, action: str, parameters: Dict[str, Any], auth_data: Optional[Dict[str, Any]] = None) -> None:
        """Rollback action if needed"""
        pass

class Action(BaseModel):
    """Action model with validation"""
    method: str
    action: str
    parameters: Dict[str, Any]
    retryCount: int = 0
    retryMax: int
    status: ActionStatus = ActionStatus.PENDING
    timeout: Optional[int] = None
    dependencies: List[str] = []
    rollback_on_failure: bool = False
    auth_source: Optional[AuthSource] = None  # Auth source for this action
    
    class Config:
        use_enum_values = True

class AgentTask(BaseModel):
    """Task model with validation"""
    id: str
    workflowId: str
    status: TaskStatus = TaskStatus.PENDING
    userInput: str
    dataList: List[Dict[str, str]]  # List of available connections
    actionList: List[Action]
    chatHistory: str
    taskHistory: str
    previousTaskFeedback: Optional[str]
    thisTaskFeedback: Optional[str]
    result: Optional[Dict[str, Any]]
    documentsInput: List[Dict]
    documentsOutput: List[Dict]
    startedAt: str
    finishedAt: Optional[str]
    error: Optional[str]
    dependencies: List[str] = []
    requiredOutputs: List[str] = []
    
    class Config:
        use_enum_values = True
        
    def get_auth_data(self, auth_source: AuthSource) -> Optional[Dict[str, Any]]:
        """Get authentication data for the specified source"""
        return next(
            (conn for conn in self.dataList if conn.get('source') == auth_source),
            None
        )
        
    def get_action_by_id(self, action_id: str) -> Optional[Action]:
        """Get action by its ID (method:action)"""
        return next((a for a in self.actionList if f"{a.method}:{a.action}" == action_id), None)
        
    def can_execute_action(self, action: Action) -> bool:
        """Check if action can be executed based on dependencies and auth"""
        # Check dependencies
        if action.dependencies:
            if not all(
                self.get_action_by_id(dep).status == ActionStatus.SUCCESS
                for dep in action.dependencies
            ):
                return False
                
        # Check authentication
        if action.auth_source and action.auth_source != AuthSource.LOCAL:
            if not self.get_auth_data(action.auth_source):
                return False
                
        return True
        
    def is_complete(self) -> bool:
        """Check if all actions are complete"""
        return all(a.status in [ActionStatus.SUCCESS, ActionStatus.SKIPPED] 
                  for a in self.actionList)
        
    def has_failed(self) -> bool:
        """Check if any action has failed"""
        return any(a.status == ActionStatus.FAILED for a in self.actionList)

class ServiceContainer:
    """Service container with improved state management"""
    
    def __init__(self):
        self.state = {
            'status': TaskStatus.PENDING,
            'retryCount': 0,
            'retryMax': 3,
            'timeout': 300,  # 5 minutes
            'lastError': None,
            'lastErrorTime': None
        }
        self.methods: Dict[str, MethodBase] = {}
        self.tasks: Dict[str, AgentTask] = {}
        self.promptManager = AIPromptManager()
        self.taskStateManager = TaskStateManager()
        self.documentProcessor = DocumentProcessor()
        
    async def execute_task(self, task: AgentTask) -> None:
        """Execute task with improved error handling and timeout"""
        try:
            # Check for timeout
            if (datetime.now(UTC) - datetime.fromisoformat(task.startedAt)).seconds > self.state['timeout']:
                task.status = TaskStatus.TIMEOUT
                return
                
            # Execute actions
            for action in task.actionList:
                if not task.can_execute_action(action):
                    if not task.get_auth_data(action.auth_source):
                        action.status = ActionStatus.FAILED
                        task.error = f"Missing authentication for {action.auth_source}"
                    else:
                        action.status = ActionStatus.DEPENDENCY_FAILED
                    continue
                    
                try:
                    # Get method
                    method = self.methods.get(action.method)
                    if not method:
                        raise ValueError(f"Unknown method: {action.method}")
                        
                    # Validate parameters
                    if not await method.validate_parameters(action.action, action.parameters):
                        raise ValueError(f"Invalid parameters for {action.method}:{action.action}")
                        
                    # Get auth data if needed
                    auth_data = None
                    if action.auth_source and action.auth_source != AuthSource.LOCAL:
                        auth_data = task.get_auth_data(action.auth_source)
                        if not auth_data:
                            raise ValueError(f"Missing authentication data for {action.auth_source}")
                        
                    # Execute with timeout
                    result = await asyncio.wait_for(
                        method.execute(action.action, action.parameters, auth_data),
                        timeout=action.timeout or 60
                    )
                    
                    if result.success:
                        action.status = ActionStatus.SUCCESS
                    else:
                        if self._should_retry(result.data.get('error')):
                            action.retryCount += 1
                            if action.retryCount > action.retryMax:
                                action.status = ActionStatus.FAILED
                                if action.rollback_on_failure:
                                    await method.rollback(action.action, action.parameters, auth_data)
                            else:
                                action.status = ActionStatus.RETRY
                        else:
                            action.status = ActionStatus.FAILED
                            if action.rollback_on_failure:
                                await method.rollback(action.action, action.parameters, auth_data)
                                
                except asyncio.TimeoutError:
                    action.status = ActionStatus.TIMEOUT
                except Exception as e:
                    action.status = ActionStatus.FAILED
                    if action.rollback_on_failure:
                        await method.rollback(action.action, action.parameters, auth_data)
                        
            # Update task status
            if task.has_failed():
                task.status = TaskStatus.FAILED
            elif task.is_complete():
                task.status = TaskStatus.SUCCESS
                task.finishedAt = datetime.now(UTC).isoformat()
                
        except Exception as e:
            task.status = TaskStatus.FAILED
            task.error = str(e)

class AIPromptManager:
    """Manages AI prompts and response validation"""
    
    def generatePrompt(self, context: Dict[str, Any], examples: List[Dict]) -> str:
        """Generate a context-aware prompt with few-shot examples"""
        prompt = (
            f"Task: {context['task']}\n"
            f"Document: {context['document']['name']} ({context['document']['type']})\n"
            "Examples:\n"
        )
        for ex in examples:
            prompt += f"- {ex['input']} => {ex['output']}\n"
        prompt += "Extract the most relevant information for the task above."
        return prompt

    def validateResponse(self, response: str, schema: Dict) -> bool:
        """Validate AI response against a schema"""
        import jsonschema
        try:
            jsonschema.validate(instance=response, schema=schema)
            return True
        except jsonschema.ValidationError:
            return False

class TaskStateManager:
    """Manages task state and retry tracking"""
    
    def __init__(self):
        self.taskStates = {}

    def trackState(self, task: AgentTask):
        """Track task state"""
        self.taskStates[task.id] = {
            "status": task.status,
            "retryState": getattr(task, "retryState", {}),
            "history": getattr(task, "history", [])
        }

    def canRetry(self, task: AgentTask, method: str) -> bool:
        """Check if task can be retried"""
        retryState = self.taskStates[task.id].get("retryState", {})
        return retryState.get(method, 0) < getattr(task, "retryMax", 3)

class DocumentContext(BaseModel):
    """Model for document context"""
    id: str
    extractionHistory: List[Dict]
    relevantSections: List[str]
    processingStatus: Dict[str, str]

class DocumentProcessor:
    """Processes documents with context awareness"""
    
    def process_with_context(self, doc: Dict, context: DocumentContext) -> Dict:
        """Process document with context"""
        extracted = {}
        for section in context.relevantSections:
            extracted[section] = doc.get(section)
        return extracted

    def track_extraction(self, doc: Dict, extraction: Dict):
        """Track document extraction"""
        if 'extractionHistory' not in doc:
            doc['extractionHistory'] = []
        doc['extractionHistory'].append(extraction)

class ErrorRecovery(BaseModel):
    """Model for error recovery strategies"""
    strategy: str  # e.g., "retry", "fallback", "skip"
    fallbackActions: List[str]
    contextPreservation: bool

### 1.3 Method-Based Module Structure
```python
# Example: methodSharepoint.py
class MethodSharepoint:
    """SharePoint method implementation"""
    
    def __init__(self, service):
        self.service = service
        self.name = "sharepoint"
        self.description = "Search and process SharePoint documents"
        self.auth_source = AuthSource.MSFT  # Requires Microsoft authentication
        
    @property
    def actions(self) -> Dict[str, Dict[str, Any]]:
        """Available actions and their parameters"""
        return {
            "search": {
                "description": "Search SharePoint documents",
                "retryMax": 3,
                "timeout": 30,
                "parameters": {
                    "query": {"type": "string", "required": True},
                    "site": {"type": "string", "required": False},
                    "folder": {"type": "string", "required": False},
                    "maxResults": {"type": "number", "required": False}
                }
            }
        }
    
    async def execute(self, action: str, parameters: Dict[str, Any], auth_data: Optional[Dict[str, Any]] = None) -> Dict[str, Any]:
        """Execute SharePoint method"""
        if not auth_data:
            return {"success": False, "error": "Missing Microsoft authentication"}
            
        if action == "search":
            return await self._searchDocuments(parameters, auth_data)
        return {"success": False, "error": f"Unknown action: {action}"}
    
    async def _searchDocuments(self, parameters: Dict[str, Any], auth_data: Dict[str, Any]) -> Dict[str, Any]:
        """Search SharePoint documents"""
        # Implementation using existing SharePoint agent functionality
        pass

# Example: methodOutlook.py
class MethodOutlook:
    """Outlook method implementation"""
    
    def __init__(self, service):
        self.service = service
        self.name = "outlook"
        self.description = "Handle Outlook email operations"
        
    @property
    def actions(self) -> Dict[str, Dict[str, Any]]:
        """Available actions and their parameters"""
        return {
            "readMails": {
                "description": "Read emails from specified folder",
                "retryMax": 2,  # Action-specific retry limit
                "parameters": {
                    "folder": {"type": "string", "required": False},
                    "unreadOnly": {"type": "boolean", "required": False},
                    "fromAddress": {"type": "string", "required": False},
                    "maxResults": {"type": "number", "required": False}
                }
            },
            "sendMail": {
                "description": "Send an email",
                "retryMax": 1,  # Action-specific retry limit
                "parameters": {
                    "to": {"type": "array", "items": "string", "required": True},
                    "subject": {"type": "string", "required": True},
                    "body": {"type": "string", "required": True},
                    "attachments": {"type": "array", "items": "FileRef", "required": False}
                }
            }
        }
    
    async def execute(self, action: str, parameters: Dict[str, Any]) -> Dict[str, Any]:
        """Execute Outlook method"""
        if action == "readMails":
            return await self._readMails(parameters)
        elif action == "sendMail":
            return await self._sendMail(parameters)
        return {"success": False, "error": f"Unknown action: {action}"}

1.4 Key Data Objects

class ChatWorkflow:
    id: str
    mandateId: str
    status: str
    name: Optional[str]
    currentRound: int
    lastActivity: str
    startedAt: str
    logs: List[ChatLog]
    messages: List[ChatMessage]
    stats: Optional[ChatStat]
    tasks: List[Task]

class AgentTask:
    id: str
    workflowId: str
    status: str  # pending, success, failed, retry
    userInput: str  # AI-processed summary
    dataList: List[Dict[str, str]]  # User connections
    actionList: List[Dict[str, Any]]  # Actions to execute, e.g.:
    # [
    #     {
    #         "method": "sharepoint",
    #         "action": "search",
    #         "parameters": {
    #             "query": "offerings",
    #             "site": "valueon"
    #         },
    #         "retryCount": 0,
    #         "retryMax": 3,
    #         "status": "pending"  # pending, success, failed, retry
    #     },
    #     {
    #         "method": "outlook",
    #         "action": "sendMail",
    #         "parameters": {
    #             "to": ["user@example.com"],
    #             "subject": "Offer Summary",
    #             "body": "..."
    #         },
    #         "retryCount": 0,
    #         "retryMax": 1,
    #         "status": "pending"
    #     }
    # ]
    chatHistory: str  # Summary of previous messages
    taskHistory: str  # Summary of previous tasks
    previousTaskFeedback: Optional[str]
    thisTaskFeedback: Optional[str]
    result: Optional[ChatMessage]
    documentsInput: List[Dict]
    documentsOutput: List[Dict]
    startedAt: str
    finishedAt: Optional[str]
    error: Optional[str]
    dependencies: List[str] = []     # Task dependencies
    requiredOutputs: List[str] = []  # Required outputs from dependencies

2. Process Flow

2.1 Initialization Phase

graph TD
    A[User Input] --> B[WorkflowManager.workflowProcess]
    B --> C[ChatManager.initialize]
    C --> D[Create ServiceContainer]
    D --> E[Create Initial Task]
  1. WorkflowManager.workflowProcess

    • Receives user input and workflow
    • Initializes chat manager
    • Starts task processing loop
  2. ChatManager.initialize

    • Creates ServiceContainer with all required components
    • Initializes service interfaces
    • Sets up task and state management

2.2 Task Creation Phase

  1. Create Initial Task
    def createInitialTask(self, userInput: UserInputRequest) -> AgentTask:
        # 1. Get available methods and their actions
        available_methods = self._getAvailableMethods()
        method_catalog = {
            method.name: {
                "description": method.description,
                "actions": method.actions
            }
            for method in available_methods
        }
    
        # 2. Process user input with AI including document analysis
        processedInput = await self.service.model['callAiBasic'](
            f"""Analyze user request and documents:
            User Prompt: {userInput.prompt}
            Documents: {userInput.listFileId}
    
            Available Methods:
            {json.dumps(method_catalog, indent=2)}
    
            Please provide:
            1. Main objective
            2. Required actions (using available methods and their actions)
            3. Required data sources
            4. Document processing requirements
            5. Expected output format
    
            Format your response as JSON:
            {{
                "objective": "string",
                "actions": [
                    {{
                        "method": "string",
                        "action": "string",
                        "parameters": {{
                            "param1": "value1",
                            "param2": "value2"
                        }}
                    }}
                ],
                "dataSources": ["string"],
                "documentRequirements": ["string"],
                "outputFormat": "string"
            }}
            """
        )
    
        # 3. Create task with processed input and initialize action states
        actions = []
        for action in processedInput['actions']:
            method = next(m for m in available_methods if m.name == action['method'])
            action_info = method.actions[action['action']]
            actions.append({
                **action,
                "retryCount": 0,
                "retryMax": action_info['retryMax'],
                "status": "pending"
            })
    
        task = AgentTask(
            workflowId=self.service.workflow.id,
            userInput=processedInput,
            dataList=self.service.context['dataConnections'],
            actionList=actions,
            chatHistory=await self.workflowSummarize(userInput),
            startedAt=datetime.now(UTC).isoformat()
        )
    
        # 4. Store in service
        self.service.tasks['current'] = task
        return task
    

2.3 Task Execution Phase

  1. Execute Task
    async def executeTask(self, task: AgentTask) -> None:
        """Execute task actions in sequence"""
        for action in task.actionList:
            if action['status'] == 'pending':
                try:
                    # Get method instance
                    method = self.service.methods[action['method']]
    
                    # Execute action
                    result = await method.execute(
                        action['action'],
                        action['parameters']
                    )
    
                    if result['success']:
                        action['status'] = 'success'
                    else:
                        if self._shouldRetry(result['error']):
                            action['retryCount'] += 1
                            if action['retryCount'] > action['retryMax']:
                                action['status'] = 'failed'
                                task.status = 'failed'
                                task.error = "Maximum retries exceeded"
                            else:
                                action['status'] = 'retry'
                                task.status = 'retry'
                        else:
                            action['status'] = 'failed'
                            task.status = 'failed'
                            task.error = result['error']
    
                except Exception as e:
                    action['status'] = 'failed'
                    task.status = 'failed'
                    task.error = str(e)
    
                # Update task status based on action status
                if action['status'] == 'failed':
                    break
    
        # Mark task as complete if all actions succeeded
        if all(a['status'] == 'success' for a in task.actionList):
            task.status = 'success'
            task.finishedAt = datetime.now(UTC).isoformat()
    

2.4 Task Analysis Phase

  1. Define Next Task
    def defineNextTask(self, currentTask: AgentTask) -> Optional[AgentTask]:
        try:
            # 1. Analyze current task results using basic AI
            analysis = await self.service.model['callAiBasic'](
                f"""Analyze task results and determine next steps:
                Previous Feedback: {currentTask.previousTaskFeedback}
                Current Feedback: {currentTask.thisTaskFeedback}
                User Input: {currentTask.userInput}
                Current Documents: {currentTask.documentsOutput}
    
                Please provide:
                1. Task completion status
                2. Next required actions
                3. Required documents
                4. Method recommendations
    
                Format your response as JSON:
                {{
                    "isComplete": boolean,
                    "nextActions": ["string"],
                    "requiredDocuments": ["string"],
                    "recommendedMethods": ["string"]
                }}
                """
            )
    
            # 2. Parse and validate AI response
            analysis_data = json.loads(analysis)
    
            # 3. Determine if next task needed
            if not analysis_data["isComplete"]:
                # 4. Create next task
                nextTask = self._createNextTask(currentTask, analysis_data)
                self.service.tasks['previous'] = currentTask
                self.service.tasks['current'] = nextTask
                return nextTask
    
            return None
    
        except Exception as e:
            logger.error(f"Error defining next task: {str(e)}")
            return None
    

3. Method Integration

3.1 Method Registration

def _registerMethods(self):
    """Register available methods in service container"""
    self.service.methods = {
        "sharepoint": MethodSharepoint(self.service),
        "outlook": MethodOutlook(self.service),
        "web": MethodWeb(self.service),
        "document": MethodDocument(self.service)
    }

3.2 Method Execution

def _executeMethod(self, method: str, parameters: Dict[str, Any]) -> Dict[str, Any]:
    """Execute a method with parameters"""
    try:
        # Get method implementation
        method_impl = self.service.methods.get(method)
        if not method_impl:
            return {"success": False, "error": f"Unknown method: {method}"}
        
        # Execute method
        return await method_impl.execute(parameters)
        
    except Exception as e:
        return {"success": False, "error": str(e)}

4. Error Handling

4.1 Error Types

  1. AI Errors

    • Model unavailable
    • Invalid response
    • Timeout
  2. Method Errors

    • Invalid method
    • Execution failure
    • Resource unavailable
  3. Task Errors

    • Invalid state
    • Missing data
    • Timeout

4.2 Retry Logic

def _shouldRetry(self, error: str) -> bool:
    """Determine if error is retryable"""
    retryable_errors = [
        "AI down",
        "Document not found",
        "Content extraction failed"
    ]
    return any(err in error for err in retryable_errors)

def _shouldCreateNextTask(self, analysis: Dict[str, Any]) -> bool:
    """Determine if next task is needed based on AI analysis"""
    return not analysis.get("isComplete", True)

5. AI Integration Points

5.1 User Input Processing

async def _processUserInput(self, input: str, documents: List[str]) -> str:
    """Process user input including document analysis"""
    context = {
        "task": "Process user input",
        "document": {"name": "User Input", "type": "text"}
    }
    examples = [
        {"input": "Search documents", "output": "Extract relevant information"}
    ]
    prompt = self.service.promptManager.generatePrompt(context, examples)
    
    return await self.service.model['callAiBasic'](
        f"""Analyze user request and documents:
        User Input: {input}
        Documents: {documents}
        
        {prompt}
        
        Please provide:
        1. Main objective
        2. Required actions
        3. Required data sources
        4. Document processing requirements
        5. Expected output format
        
        Format your response as JSON:
        {{
            "objective": "string",
            "actions": ["string"],
            "dataSources": ["string"],
            "documentRequirements": ["string"],
            "outputFormat": "string"
        }}
        """
    )

5.2 Task Analysis

async def _analyzeTaskResults(self, task: AgentTask) -> str:
    """Analyze task results and determine next steps"""
    context = {
        "task": "Analyze task results",
        "document": {"name": "Task Results", "type": "json"}
    }
    examples = [
        {"input": "Task completed", "output": "Generate next steps"}
    ]
    prompt = self.service.promptManager.generatePrompt(context, examples)
    
    return await self.service.model['callAiBasic'](
        f"""Analyze task results and determine next steps:
        Task Input: {task.userInput}
        Previous Feedback: {task.previousTaskFeedback}
        Current Feedback: {task.thisTaskFeedback}
        Current Documents: {task.documentsOutput}
        
        {prompt}
        
        Please provide:
        1. Task completion status
        2. Next required actions
        3. Required documents
        4. Method recommendations
        
        Format your response as JSON:
        {{
            "isComplete": boolean,
            "nextActions": ["string"],
            "requiredDocuments": ["string"],
            "recommendedMethods": ["string"]
        }}
        """
    )

5.3 Result Processing

async def _processTaskResults(self, task: AgentTask) -> str:
    """Process task results and generate feedback"""
    context = {
        "task": "Process task results",
        "document": {"name": "Task Results", "type": "json"}
    }
    examples = [
        {"input": "Task results", "output": "Generate summary"}
    ]
    prompt = self.service.promptManager.generatePrompt(context, examples)
    
    return await self.service.model['callAiBasic'](
        f"""Process task results and generate feedback:
        Task Input: {task.userInput}
        Method Results: {task.result}
        Generated Documents: {task.documentsOutput}
        
        {prompt}
        
        Please provide:
        1. Summary of completed actions
        2. Generated document descriptions
        3. Next steps or completion status
        
        Format your response as JSON:
        {{
            "summary": "string",
            "documents": ["string"],
            "nextSteps": ["string"]
        }}
        """
    )

6. File Structure and Implementation Plan

6.1 File Structure

gateway/
├── modules/
│   ├── workflow/
│   │   ├── managerWorkflow.py      # Workflow management and state machine
│   │   ├── managerChat.py          # Chat management and AI response validation
│   │   ├── managerPrompt.py        # AI prompt generation and management
│   │   ├── methodBase.py           # Base method class with result validation
│   │   ├── managerDocument.py      # Document operations management
│   │   └── processorDocument.py    # Document content extraction
│   │
│   ├── agents/                     # To be refactored into methods
│   │   ├── agentSharepoint.py      → methods/methodSharepoint.py
│   │   ├── agentOutlook.py         → methods/methodOutlook.py
│   │   ├── agentWebcrawler.py      → methods/methodWeb.py
│   │   ├── agentDocument.py        → methods/methodDocument.py
│   │   └── agentCoder.py          → methods/methodCoder.py
│   │
│   ├── methods/                    # New directory for method implementations
│   │   ├── methodSharepoint.py     # SharePoint operations
│   │   ├── methodOutlook.py        # Outlook operations
│   │   ├── methodWeb.py           # Web operations
│   │   ├── methodDocument.py       # Document operations
│   │   ├── methodCoder.py         # Code generation operations
│   │   └── methodPowerpoint.py     # PowerPoint operations
│   │
│   └── interfaces/
│       ├── serviceChatModel.py     # Chat system models and enums
│       └── serviceAppModel.py      # Application models including UserConnection

6.2 Implementation Plan

Phase 1: Core Structure Setup

  1. File Renaming and Organization

    • Rename manager files to follow manager*.py pattern
    • Move document processor to processorDocument.py
    • Create new methods directory
  2. Model Updates

    • Update serviceChatModel.py with new enums and models
    • Integrate UserConnection from serviceAppModel.py
    • Update validation logic in respective modules

Phase 2: Method Migration

  1. Base Method Implementation

    • Implement methodBase.py with core functionality
    • Add method result validation
    • Set up authentication handling
  2. Agent to Method Conversion

    • Convert each agent to its method implementation
    • Migrate functionality while maintaining existing behavior
    • Add method-specific validation
  3. New Method Implementation

    • Implement methodPowerpoint.py
    • Add PowerPoint-specific operations
    • Integrate with document processing

Phase 3: Manager Updates

  1. Chat Manager Enhancement

    • Integrate AI response validation
    • Update service container structure
    • Improve error handling
  2. Document Manager Integration

    • Update document operations for new method structure
    • Enhance content extraction capabilities
    • Improve file handling
  3. Workflow Manager Updates

    • Update state machine for method-based approach
    • Improve task management
    • Enhance error recovery

Phase 4: Testing and Validation

  1. Unit Testing

    • Test each method implementation
    • Validate error handling
    • Verify authentication flow
  2. Integration Testing

    • Test method interactions
    • Validate document processing
    • Verify workflow execution
  3. Performance Testing

    • Measure response times
    • Validate resource usage
    • Test concurrent operations

Phase 5: Documentation and Cleanup

  1. Documentation

    • Update API documentation
    • Document method implementations
    • Add usage examples
  2. Code Cleanup

    • Remove deprecated code
    • Clean up old agent files
    • Optimize imports
  3. Final Review

    • Code review
    • Security audit
    • Performance optimization

6.3 Migration Strategy

  1. Incremental Migration

    • Migrate one agent at a time
    • Maintain backward compatibility
    • Use feature flags for gradual rollout
  2. Testing Strategy

    • Unit tests for each method
    • Integration tests for workflows
    • End-to-end tests for complete scenarios
  3. Rollback Plan

    • Keep old agent implementations until stable
    • Maintain version control
    • Document rollback procedures

6.4 Success Criteria

  1. Functionality

    • All existing features working
    • New method-based structure operational
    • Improved error handling
  2. Performance

    • Equal or better response times
    • Reduced resource usage
    • Improved scalability
  3. Maintainability

    • Clear code structure
    • Comprehensive documentation
    • Easy to extend
  4. Security

    • Proper authentication handling
    • Secure data processing
    • Access control implementation