refactored connection token management and document handling

GOOGLE_OAUTH_SETUP.md

@@ -1,114 +0,0 @@
-# Google OAuth 2.0 Setup Guide for PowerOn
-
-## Overview
-This guide explains how to set up Google OAuth 2.0 authentication for the PowerOn application.
-
-## Prerequisites
-- A Google account
-- Access to Google Cloud Console (https://console.cloud.google.com/)
-
-## Step 1: Create a Google Cloud Project
-
-1. Go to [Google Cloud Console](https://console.cloud.google.com/)
-2. Click on the project dropdown at the top of the page
-3. Click "New Project"
-4. Enter a project name (e.g., "PowerOn OAuth")
-5. Click "Create"
-
-## Step 2: Enable Google+ API
-
-1. In your new project, go to "APIs & Services" > "Library"
-2. Search for "Google+ API" or "Google Identity"
-3. Click on "Google+ API" and click "Enable"
-
-## Step 3: Create OAuth 2.0 Credentials
-
-1. Go to "APIs & Services" > "Credentials"
-2. Click "Create Credentials" > "OAuth client ID"
-3. If prompted, configure the OAuth consent screen first:
-   - Choose "External" user type
-   - Fill in the required fields (App name, User support email, Developer contact information)
-   - Add scopes: `https://www.googleapis.com/auth/userinfo.profile`, `https://www.googleapis.com/auth/userinfo.email`
-   - Add test users if needed
-   - Click "Save and Continue" through all sections
-
-4. Back to creating OAuth client ID:
-   - Application type: "Web application"
-   - Name: "PowerOn Web Client"
-   - Authorized redirect URIs: Add your redirect URI
-     - For development: `http://localhost:8000/api/google/auth/callback`
-     - For production: `https://yourdomain.com/api/google/auth/callback`
-
-5. Click "Create"
-6. **Important**: Copy the Client ID and Client Secret - you'll need these for the next step
-
-## Step 4: Configure PowerOn Application
-
-1. Open your environment file (`gateway/env_dev.env` for development)
-2. Replace the placeholder values with your actual Google OAuth credentials:
-
-```env
-# Google OAuth Configuration
-Service_GOOGLE_CLIENT_ID = your-actual-client-id-from-google-console
-Service_GOOGLE_CLIENT_SECRET = your-actual-client-secret-from-google-console
-Service_GOOGLE_REDIRECT_URI = http://localhost:8000/api/google/auth/callback
-```
-
-3. Save the file
-4. Restart your PowerOn gateway server
-
-## Step 5: Test the Configuration
-
-1. Start your PowerOn application
-2. Go to the Connections module
-3. Click "Connect Google"
-4. You should be redirected to Google's OAuth consent screen
-5. After authorization, you should be redirected back to PowerOn
-
-## Troubleshooting
-
-### Common Issues
-
-#### 1. "Missing required parameter: redirect_uri"
-- **Cause**: Google OAuth client is not properly configured with the redirect URI
-- **Solution**: Ensure the redirect URI in Google Cloud Console exactly matches your application's callback URL
-
-#### 2. "Invalid client" error
-- **Cause**: Client ID or Client Secret is incorrect
-- **Solution**: Double-check the credentials in your environment file
-
-#### 3. "Redirect URI mismatch" error
-- **Cause**: The redirect URI in your OAuth request doesn't match what's configured in Google Cloud Console
-- **Solution**: Ensure both URIs are identical (including protocol, domain, port, and path)
-
-### Debug Steps
-
-1. Check the PowerOn gateway logs for OAuth configuration details
-2. Verify environment variables are loaded correctly
-3. Ensure the Google OAuth client is configured for "Web application" type
-4. Check that the redirect URI includes the full path: `/api/google/auth/callback`
-
-## Security Notes
-
-- **Never commit** your Google OAuth credentials to version control
-- Use environment variables or secure configuration management
-- Regularly rotate your client secrets
-- Monitor OAuth usage in Google Cloud Console
-
-## Production Considerations
-
-For production deployment:
-
-1. Use HTTPS for all OAuth redirects
-2. Configure proper domain verification in Google Cloud Console
-3. Set up monitoring and alerting for OAuth usage
-4. Consider implementing additional security measures like PKCE (Proof Key for Code Exchange)
-
-## Support
-
-If you continue to experience issues:
-
-1. Check the PowerOn gateway logs for detailed error messages
-2. Verify your Google OAuth configuration in Google Cloud Console
-3. Test with a simple OAuth flow to isolate the issue
-4. Ensure your Google Cloud project has billing enabled (required for some APIs)

modules/chat/documents/documentGeneration.py

@@ -21,12 +21,34 @@ class DocumentGenerator:
         Returns a list of processed document dictionaries.
         """
         try:
-            documents = action_result.data.get("documents", [])
+            # Read documents from the standard documents field (not data.documents)
+            documents = action_result.documents if hasattr(action_result, 'documents') else []
+            
+            if not documents:
+                logger.info(f"No documents found in action_result.documents for {action.execMethod}.{action.execAction}")
+                return []
+            
+            logger.info(f"Processing {len(documents)} documents from action_result.documents")
+            
+            # Check if documents are references (strings starting with "docItem:") or actual document objects
+            if documents and isinstance(documents[0], str) and documents[0].startswith("docItem:"):
+                # These are document references, resolve them to actual documents
+                logger.info(f"Resolving {len(documents)} document references to actual documents")
+                try:
+                    actual_documents = self.service.getChatDocumentsFromDocumentList(documents)
+                    logger.info(f"Resolved {len(actual_documents)} actual documents from references")
+                    documents = actual_documents
+                except Exception as e:
+                    logger.error(f"Error resolving document references: {str(e)}")
+                    return []
+            
             processed_documents = []
             for doc in documents:
                 processed_doc = self.processSingleDocument(doc, action)
                 if processed_doc:
                     processed_documents.append(processed_doc)
+            
+            logger.info(f"Successfully processed {len(processed_documents)} documents")
             return processed_documents
         except Exception as e:
             logger.error(f"Error processing action result documents: {str(e)}")
@@ -61,6 +83,35 @@ class DocumentGenerator:
                     'content': getattr(doc, 'content', ''),
                     'document': doc
                 }
+            elif hasattr(doc, 'documentName') and doc.documentName:
+                # ActionDocument object with documentName attribute
+                base_filename = doc.documentName
+                mime_type = getattr(doc, 'mimeType', 'application/octet-stream')
+                content = getattr(doc, 'documentData', '')
+                
+                # Add result label to filename for ActionDocument objects
+                if hasattr(action, 'execResultLabel') and action.execResultLabel:
+                    result_label = action.execResultLabel.strip()
+                    if result_label:
+                        # Check if filename already starts with resultLabel to avoid duplication
+                        if not base_filename.startswith(f"{result_label}-"):
+                            base_filename = f"{result_label}-{base_filename}"
+                            logger.info(f"Added resultLabel '{result_label}' as prefix to ActionDocument filename: {base_filename}")
+                        else:
+                            logger.info(f"ActionDocument filename already has resultLabel prefix: {base_filename}")
+                
+                # Calculate file size from actual content
+                fileSize = len(str(content)) if content else 0
+                
+                logger.info(f"Processed ActionDocument: {base_filename}, content length: {len(str(content))}, mimeType: {mime_type}")
+                
+                return {
+                    'filename': base_filename,
+                    'fileSize': fileSize,
+                    'mimeType': mime_type,
+                    'content': content,
+                    'document': doc
+                }
             elif isinstance(doc, dict):
                 # Dictionary format document - handle both 'documentName' and 'filename' keys
                 base_filename = doc.get('documentName', doc.get('filename', ''))
@@ -159,7 +210,7 @@ class DocumentGenerator:
         """
         try:
             logger.info(f"Creating documents from action result for {action.execMethod}.{action.execAction}")
-            logger.info(f"Action result data keys: {list(action_result.data.keys())}")
+            logger.info(f"Action result documents count: {len(action_result.documents) if action_result.documents else 0}")
             
             processed_docs = self.processActionResultDocuments(action_result, action, workflow)
             logger.info(f"Processed {len(processed_docs)} documents")

modules/chat/handling/executionState.py

@@ -33,8 +33,19 @@ class TaskExecutionState:
         """Get available results from successful actions"""
         results = []
         for action in self.successful_actions:
-            if action.data and action.data.get('result'):
+            if action.documents:
-                results.append(action.data['result'])
+                # Extract text content from documents
+                for doc in action.documents:
+                    if hasattr(doc, 'documentData'):
+                        if isinstance(doc.documentData, dict):
+                            result_text = doc.documentData.get("result", "")
+                        elif isinstance(doc.documentData, str):
+                            result_text = doc.documentData
+                        else:
+                            result_text = str(doc.documentData)
+                        
+                        if result_text and result_text.strip():
+                            results.append(result_text)
         return results
 
     def shouldRetryTask(self) -> bool:

modules/chat/handling/handlingTasks.py

@@ -474,8 +474,16 @@ class HandlingTasks:
                 step_result={
                     'successful_actions': sum(1 for result in action_results if result.success),
                     'total_actions': len(action_results),
-                    'results': [result.data.get('result', '') for result in action_results if result.success],
+                    'results': [self._extractResultText(result) for result in action_results if result.success],
-                    'errors': [result.error for result in action_results if not result.success]
+                    'errors': [result.error for result in action_results if not result.success],
+                    'documents': [
+                        {
+                            'action_index': i,
+                            'documents_count': len(result.documents) if hasattr(result, 'documents') and result.documents else 0,
+                            'documents': result.documents if hasattr(result, 'documents') and result.documents else []
+                        }
+                        for i, result in enumerate(action_results)
+                    ]
                 }
             )
             # Check workflow status before calling AI service
@@ -624,9 +632,17 @@ class HandlingTasks:
             if result.success:
                 created_documents = self.documentGenerator.createDocumentsFromActionResult(result, action, workflow)
                 action.setSuccess()
-                action.result = result.data.get("result", "")
+                # Extract result text from documents if available, otherwise use empty string
-                action.execResultLabel = result_label
+                action.result = ""
-                await self.createActionMessage(action, result, workflow, result_label, created_documents, task_step, task_index)
+                if result.documents and len(result.documents) > 0:
+                    # Try to get text content from the first document
+                    first_doc = result.documents[0]
+                    if hasattr(first_doc, 'documentData') and isinstance(first_doc.documentData, dict):
+                        action.result = first_doc.documentData.get("result", "")
+                    elif hasattr(first_doc, 'documentData') and isinstance(first_doc.documentData, str):
+                        action.result = first_doc.documentData
+                action.execResultLabel = result.resultLabel or result_label
+                await self.createActionMessage(action, result, workflow, result.resultLabel or result_label, created_documents, task_step, task_index)
                 
                 # Log action results
                 logger.info(f"✓ Action completed successfully")
@@ -689,38 +705,20 @@ class HandlingTasks:
                         "type": "error"
                     })
             
-            # Extract document filenames for the ActionResult
-            document_filenames = []
-            for doc in created_documents:
-                if hasattr(doc, 'filename'):
-                    document_filenames.append(doc.filename)
-                elif isinstance(doc, dict) and 'filename' in doc:
-                    document_filenames.append(doc['filename'])
-            
-            # Also include the original documents from the service result for validation
-            original_documents = result.data.get("documents", [])
-            
             # Log action summary
             logger.info(f"=== TASK {task_num} ACTION {action_num} COMPLETED ===")
             
+            # Preserve the original documents field from the method result
+            # This ensures the standard document format is maintained
+            original_documents = result.documents if hasattr(result, 'documents') else []
+            
+            # Extract result text from documents if available
+            result_text = self._extractResultText(result)
+            
             return ActionResult(
                 success=result.success,
-                data={
+                documents=original_documents,  # Preserve original documents field from method result
-                    "result": result.data.get("result", ""),
+                resultLabel=result.resultLabel or result_label,
-                    "documents": created_documents,  # Include actual document objects in data
-                    "actionId": action.id,
-                    "actionMethod": action.execMethod,
-                    "actionName": action.execAction,
-                    "resultLabel": result_label
-                },
-                documents=document_filenames,  # Keep as filenames for the documents field
-                metadata={
-                    "actionId": action.id,
-                    "actionMethod": action.execMethod,
-                    "actionName": action.execAction,
-                    "resultLabel": result_label
-                },
-                validation={},
                 error=result.error or ""
             )
         except Exception as e:
@@ -728,18 +726,8 @@ class HandlingTasks:
             action.setError(str(e))
             return ActionResult(
                 success=False,
-                data={
+                documents=[],  # Empty documents for error case
-                    "actionId": action.id,
+                resultLabel=result_label,
-                    "actionMethod": action.execMethod,
-                    "actionName": action.execAction,
-                    "documents": []
-                },
-                metadata={
-                    "actionId": action.id,
-                    "actionMethod": action.execMethod,
-                    "actionName": action.execAction
-                },
-                validation={},
                 error=str(e)
             )
 
@@ -901,4 +889,18 @@ class HandlingTasks:
             return True
         except Exception as e:
             logger.error(f"Error validating actions: {str(e)}")
-            return False 
+            return False
+
+    def _extractResultText(self, result: ActionResult) -> str:
+        """Extract result text from ActionResult documents"""
+        if not result.success or not result.documents:
+            return ""
+        
+        # Try to get text content from the first document
+        first_doc = result.documents[0]
+        if hasattr(first_doc, 'documentData') and isinstance(first_doc.documentData, dict):
+            return first_doc.documentData.get("result", "")
+        elif hasattr(first_doc, 'documentData') and isinstance(first_doc.documentData, str):
+            return first_doc.documentData
+        else:
+            return "" 

modules/chat/handling/promptFactory.py

@@ -5,6 +5,9 @@ import json
 import logging
 from typing import Any, Dict
 
+# Set up logger
+logger = logging.getLogger(__name__)
+
 # Prompt creation helpers extracted from managerChat.py
 
 def createTaskPlanningPrompt(context: Dict[str, Any]) -> str:
@@ -45,21 +48,21 @@ REQUIRED JSON STRUCTURE:
 }}
 
 EXAMPLES OF GOOD TASK OBJECTIVES:
-- \"Extract key information from documents for email preparation\"
+- \"Analyze documents and extract key insights for business communication\"
-- \"Draft professional email incorporating analyzed information\"
+- \"Create professional business communication incorporating analyzed information\"
-- \"Send email using specified email account\"
+- \"Execute business communication using specified channels\"
-- \"Store email draft and confirmation in system\"
+- \"Document and store all business communication outcomes\"
 
 EXAMPLES OF GOOD SUCCESS CRITERIA:
-- \"Document analysis completed with key points identified\"
+- \"Key insights extracted and ready for business use\"
-- \"Email draft created with professional tone and clear structure\"
+- \"Professional communication created with clear business value\"
-- \"Email successfully sent with delivery confirmation\"
+- \"Business communication successfully delivered\"
-- \"All outputs properly stored and accessible for future use\"
+- \"All outcomes properly documented and accessible\"
 
 EXAMPLES OF BAD TASK OBJECTIVES:
-- \"Open and read the PDF file\" (too granular)
+- \"Read the PDF file\" (too granular - should be \"Analyze document content\")
-- \"Identify table structure\" (technical detail)
+- \"Convert data to CSV\" (implementation detail - should be \"Structure data for analysis\")
-- \"Convert data to CSV format\" (implementation detail)
+- \"Send email\" (too specific - should be \"Deliver business communication\")
 
 NOTE: Respond with ONLY the JSON object. Do not include any explanatory text."""
 
@@ -73,6 +76,7 @@ async def createActionDefinitionPrompt(context, service) -> str:
     retry_count = context.retry_count or 0
     previous_action_results = context.previous_action_results or []
     previous_review_result = context.previous_review_result
+    previous_handover = getattr(context, 'previous_handover', None)
     methodList = service.getMethodsList()
     method_actions = {}
     for sig in methodList:
@@ -106,10 +110,17 @@ RETRY CONTEXT (Attempt {retry_count}):
 Previous action results that failed or were incomplete:
 """
         for i, result in enumerate(previous_action_results):
-            retry_context += f"- Action {i+1}: {result.actionMethod or 'unknown'}.{result.actionName or 'unknown'}\n"
+            retry_context += f"- Action {i+1}: ActionResult\n"
             retry_context += f"  Status: {result.success and 'success' or 'failed'}\n"
             retry_context += f"  Error: {result.error or 'None'}\n"
-            retry_context += f"  Result: {(result.data.get('result', '') if result.data else '')[:100]}...\n"
+            # Check if result has documents and show document info
+            if hasattr(result, 'documents') and result.documents:
+                doc_info = f"Documents: {len(result.documents)} document(s)"
+                if result.documents[0].documentName:
+                    doc_info += f" - {result.documents[0].documentName}"
+                retry_context += f"  {doc_info}\n"
+            else:
+                retry_context += f"  Documents: None\n"
         if previous_review_result:
             retry_context += f"""
 Previous review feedback:
@@ -169,6 +180,16 @@ SUCCESS CRITERIA: {success_criteria_str}
 CONTEXT - Chat History:
 {messageSummary}
 
+WORKFLOW CONTEXT - Previous Messages Summary:
+The following summarizes key information from previous workflow interactions to provide context for continued workflows:
+- Previous user inputs and their outcomes
+- Key decisions and findings from earlier tasks
+- Document processing results and insights
+- User preferences and requirements established
+- Any constraints or limitations identified
+
+This context helps ensure your actions build upon previous work and maintain consistency with the overall workflow objectives.
+
 AVAILABLE METHODS AND ACTIONS (with signatures):
 {available_methods_str}
 
@@ -191,7 +212,12 @@ DOCUMENT REFERENCE EXAMPLES:
 - Inventing message IDs instead of using actual document labels
 
 PREVIOUS RESULTS: {previous_results_str}
-IMPROVEMENTS NEEDED: {improvements_str}{retry_context}
+IMPROVEMENTS NEEDED: {improvements_str}
+
+PREVIOUS TASK HANDOVER CONTEXT:
+{previous_handover.workflowSummary if previous_handover and previous_handover.workflowSummary else 'No previous task handover available'}
+
+{retry_context}
 
 ACTION GENERATION PRINCIPLES:
 - Create meaningful actions per task step
@@ -206,6 +232,13 @@ ACTION GENERATION PRINCIPLES:
 - Address specific issues mentioned in previous review feedback
 - When specifying expectedDocumentFormats, ensure AI prompts explicitly request pure data without markdown formatting
 
+DOCUMENT ROUTING GUIDANCE:
+- Each action should produce documents with a clear resultLabel for routing
+- Use consistent naming: "task{{task_id}}_action{{action_number}}_{{descriptive_label}}"
+- Ensure document flow: Action A produces documents that Action B can consume
+- Document labels should be descriptive of content, not just "results" or "output"
+- Consider what subsequent actions will need and structure outputs accordingly
+
 INSTRUCTIONS:
 - Generate actions to accomplish this task step using available documents, connections, and previous results
 - Use docItem for single documents and docList labels for groups of documents as shown in AVAILABLE DOCUMENTS
@@ -382,46 +415,46 @@ async def createResultReviewPrompt(review_context) -> str:
     for action_result in (review_context.action_results or []):
         documents_metadata = []
         
-        # FIX: Look for documents in the correct place - action_result.data.documents contains actual document objects
+        # Get document information from step_result.documents
-        # action_result.documents only contains document references (strings)
+        action_index = len(step_result_serializable['action_results'])
-        documents_to_check = action_result.data.get("documents", [])
+        step_documents = step_result.get('documents', [])
         
-        for doc in documents_to_check:
+        logger.debug(f"Processing action {action_index}: step_documents count = {len(step_documents)}")
-            if hasattr(doc, 'filename'):
+        
+        if action_index < len(step_documents):
+            # Use the document information from step_result
+            step_doc_info = step_documents[action_index]
+            documents_count = step_doc_info.get('documents_count', 0)
+            documents_list = step_doc_info.get('documents', [])
+            
+            logger.debug(f"Action {action_index}: documents_count = {documents_count}, documents_list length = {len(documents_list)}")
+            
+            # Process the actual documents
+            for doc in documents_list:
+                # These are ActionDocument objects from ActionResult.documents
                 documents_metadata.append({
-                    'filename': doc.filename,
+                    'filename': doc.documentName or 'unknown',
-                    'fileSize': getattr(doc, 'fileSize', 0),
+                    'fileSize': len(str(doc.documentData or '')),
                     'mimeType': getattr(doc, 'mimeType', 'unknown')
                 })
-            elif isinstance(doc, dict):
+        else:
-                documents_metadata.append({
+            logger.warning(f"Action {action_index}: No step_documents info found - this should not happen with the new architecture")
-                    'filename': doc.get('filename', 'unknown'),
+            # No fallback - if step_result.documents is missing, we have a bug
-                    'fileSize': doc.get('fileSize', 0),
-                    'mimeType': doc.get('mimeType', 'unknown')
-                })
-            elif isinstance(doc, str):
-                # Handle case where documents are just filenames
-                documents_metadata.append({
-                    'filename': doc,
-                    'fileSize': 0,
-                    'mimeType': 'unknown'
-                })
         
         serializable_action_result = {
             'status': 'completed' if action_result.success else 'failed',
-            'result_summary': action_result.data.get('result', '')[:200] + '...' if len(action_result.data.get('result', '')) > 200 else action_result.data.get('result', ''),
+            'result_summary': action_result.resultLabel or 'Action completed successfully',
             'error': action_result.error,
-            'resultLabel': action_result.data.get('resultLabel', ''),
+            'resultLabel': action_result.resultLabel or '',
             'documents_count': len(documents_metadata),
             'documents_metadata': documents_metadata,
-            'actionId': action_result.actionId,
-            'actionMethod': action_result.actionMethod,
-            'actionName': action_result.actionName,
             'success_indicator': (
-                'documents' if len(documents_metadata) > 0 else
+                'documents' if len(documents_metadata) > 0 else 'none'
-                'text_result' if action_result.data.get('result', '').strip() else 'none'
             )
         }
+        
+        logger.debug(f"Action {action_index}: Final documents_count = {len(documents_metadata)}")
+        
         step_result_serializable['action_results'].append(serializable_action_result)
     step_result_json = json.dumps(step_result_serializable, indent=2, ensure_ascii=False)
     success_criteria_str = ', '.join(task_step.success_criteria or [])

modules/chat/methodBase.py

@@ -3,7 +3,7 @@ from typing import Dict, List, Optional, Any, Literal
 from datetime import datetime, UTC
 from pydantic import BaseModel, Field
 import logging
-from modules.interfaces.interfaceChatModel import ActionResult
+
 from functools import wraps
 import inspect
 
@@ -41,7 +41,7 @@ class MethodBase:
                     sig = inspect.signature(attr)
                     params = {}
                     for param_name, param in sig.parameters.items():
-                        if param_name not in ['self', 'parameters', 'authData']:
+                        if param_name not in ['self', 'parameters']:
                             param_type = param.annotation if param.annotation != param.empty else Any
                             params[param_name] = {
                                 'type': param_type,
@@ -130,18 +130,6 @@ class MethodBase:
                         descriptions[lastParam] += " " + line
         return descriptions, types
     
-    def _validateDocumentListParameter(self, parameters: Dict[str, Any], paramName: str = "documentList") -> bool:
-        """Validate that documentList parameter is a list of strings"""
-        if paramName not in parameters:
-            return False
-        
-        value = parameters[paramName]
-        if not isinstance(value, list):
-            return False
-        
-        # Check that all items in the list are strings
-        return all(isinstance(item, str) for item in value)
-    
     def _extractMainDescription(self, docstring: str) -> str:
         """Extract main description from docstring"""
         if not docstring:
@@ -167,109 +155,4 @@ class MethodBase:
         elif hasattr(type_annotation, '_name'):
             return type_annotation._name
         else:
-            return str(type_annotation)
+            return str(type_annotation)
-        
-    async def execute(self, action: str, parameters: Dict[str, Any], authData: Optional[Dict[str, Any]] = None) -> ActionResult:
-        """
-        Execute method action with authentication data
-        
-        Args:
-            action: The action to execute
-            parameters: Action parameters
-            authData: Authentication data
-            
-        Returns:
-            ActionResult containing execution results
-            
-        Raises:
-            ValueError: If action is not supported
-            RuntimeError: If authentication fails
-        """
-        try:
-            # Validate action
-            if action not in self.actions:
-                raise ValueError(f"Unsupported action: {action}")
-                
-            # Validate parameters
-            if not await self.validateParameters(action, parameters):
-                return self._createResult(
-                    success=False,
-                    data={},
-                    error="Invalid parameters"
-                )
-                
-            # Validate authentication
-            if not self._validateAuth(authData):
-                return self._createResult(
-                    success=False,
-                    data={},
-                    error="Authentication failed"
-                )
-                
-            # Execute action
-            return await self._executeAction(action, parameters, authData)
-            
-        except Exception as e:
-            self.logger.error(f"Error executing action {action}: {str(e)}")
-            return self._createResult(
-                success=False,
-                data={},
-                error=str(e)
-            )
-    
-    async def _executeAction(self, action: str, parameters: Dict[str, Any], authData: Optional[Dict[str, Any]] = None) -> ActionResult:
-        """Execute specific action - to be implemented by subclasses"""
-        raise NotImplementedError
-        
-    async def validateParameters(self, action: str, parameters: Dict[str, Any]) -> bool:
-        """Validate action parameters"""
-        try:
-            if action not in self.actions:
-                return False
-                
-            actionDef = self.actions[action]
-            requiredParams = {k for k, v in actionDef['parameters'].items() if v['required']}
-            
-            # Check required parameters
-            if not all(param in parameters for param in requiredParams):
-                return False
-            
-            # Validate documentList parameter if present
-            if "documentList" in parameters:
-                if not self._validateDocumentListParameter(parameters, "documentList"):
-                    self.logger.error("documentList parameter must be a list of strings")
-                    return False
-            
-            return True
-            
-        except Exception as e:
-            self.logger.error(f"Error validating parameters: {str(e)}")
-            return False
-        
-    async def rollback(self, action: str, parameters: Dict[str, Any], authData: Optional[Dict[str, Any]] = None) -> None:
-        """Rollback action if needed"""
-        try:
-            await self._rollbackAction(action, parameters, authData)
-        except Exception as e:
-            self.logger.error(f"Error rolling back action {action}: {str(e)}")
-            raise
-
-    async def _rollbackAction(self, action: str, parameters: Dict[str, Any], authData: Optional[Dict[str, Any]] = None) -> None:
-        """Rollback specific action - to be implemented by subclasses"""
-        pass
-
-    def _createResult(self, success: bool, data: Dict[str, Any], metadata: Optional[Dict[str, Any]] = None, error: Optional[str] = None) -> ActionResult:
-        """Create a method result"""
-        return ActionResult(
-            success=success,
-            data=data,
-            metadata=metadata or {},
-            validation={},
-            error=error
-        )
-
-    def _addValidationMessage(self, result: ActionResult, message: str) -> None:
-        """Add a validation message to the result"""
-        if 'messages' not in result.validation:
-            result.validation['messages'] = []
-        result.validation['messages'].append(message) 

modules/chat/serviceCenter.py

@@ -68,13 +68,13 @@ class ServiceCenter:
                                 # Discover actions from public methods
                                 actions = {}
                                 for methodName, method in inspect.getmembers(type(methodInstance), predicate=inspect.iscoroutinefunction):
-                                    if not methodName.startswith('_') and methodName not in ['execute', 'validateParameters']:
+                                    if not methodName.startswith('_'):
                                         # Bind the method to the instance
                                         bound_method = method.__get__(methodInstance, type(methodInstance))
                                         sig = inspect.signature(method)
                                         params = {}
                                         for paramName, param in sig.parameters.items():
-                                            if paramName not in ['self', 'authData']:
+                                            if paramName not in ['self']:
                                                 # Get parameter type
                                                 paramType = param.annotation if param.annotation != param.empty else Any
                                                 
@@ -719,9 +719,8 @@ Please provide a clear summary of this message."""
                 documentId=document.id
             )
             
-            # Update objectId to match document ID
+            # Note: ExtractedContent model only has 'id' and 'contents' fields
-            extractedContent.objectId = document.id
+            # No need to set objectId or objectType as they don't exist in the model
-            extractedContent.objectType = "ChatDocument"
             
             return extractedContent
             

modules/interfaces/interfaceAppModel.py

@@ -191,6 +191,7 @@ class Token(BaseModel, ModelMixin):
     id: Optional[str] = None
     userId: str
     authority: AuthAuthority
+    connectionId: Optional[str] = Field(None, description="ID of the connection this token belongs to")
     tokenAccess: str
     tokenType: str = "bearer"
     expiresAt: float
@@ -208,6 +209,7 @@ register_model_labels(
         "id": {"en": "ID", "fr": "ID"},
         "userId": {"en": "User ID", "fr": "ID utilisateur"},
         "authority": {"en": "Authority", "fr": "Autorité"},
+        "connectionId": {"en": "Connection ID", "fr": "ID de connexion"},
         "tokenAccess": {"en": "Access Token", "fr": "Jeton d'accès"},
         "tokenType": {"en": "Token Type", "fr": "Type de jeton"},
         "expiresAt": {"en": "Expires At", "fr": "Expire le"},

modules/interfaces/interfaceAppObjects.py

@@ -756,6 +756,7 @@ class AppObjects:
                 
             # Convert to dict and ensure all fields are properly set
             token_dict = token.dict()
+            # Ensure userId is set to current user (this might override the token's userId)
             token_dict["userId"] = self.currentUser.id
             
             # Convert datetime objects to ISO format strings
@@ -776,8 +777,8 @@ class AppObjects:
             logger.error(f"Error saving token: {str(e)}")
             raise
 
-    def getToken(self, authority: str) -> Optional[Token]:
+    def getToken(self, authority: str, auto_refresh: bool = True) -> Optional[Token]:
-        """Get the latest valid token for the current user and authority"""
+        """Get the latest valid token for the current user and authority, optionally auto-refresh if expired"""
         try:
             # Get tokens for this user and authority
             tokens = self.db.getRecordset("tokens", recordFilter={
@@ -794,8 +795,28 @@ class AppObjects:
             
             # Check if token is expired
             if latest_token.expiresAt and latest_token.expiresAt < datetime.now().timestamp():
-                logger.warning(f"Token for {authority} is expired (expiresAt: {latest_token.expiresAt})")
+                if auto_refresh:
-                return None  # Don't return expired tokens
+                    logger.info(f"Token for {authority} is expired, attempting refresh...")
+                    
+                    # Import TokenManager here to avoid circular imports
+                    from modules.security.tokenManager import TokenManager
+                    token_manager = TokenManager()
+                    
+                    # Try to refresh the token
+                    refreshed_token = token_manager.refresh_token(latest_token)
+                    if refreshed_token:
+                        # Save the new token and delete the old one
+                        self.saveToken(refreshed_token)
+                        self.deleteToken(authority)
+                        
+                        logger.info(f"Successfully refreshed token for {authority}")
+                        return refreshed_token
+                    else:
+                        logger.warning(f"Failed to refresh expired token for {authority}")
+                        return None
+                else:
+                    logger.warning(f"Token for {authority} is expired (expiresAt: {latest_token.expiresAt})")
+                    return None
             
             return latest_token
             
@@ -803,6 +824,53 @@ class AppObjects:
             logger.error(f"Error getting token: {str(e)}")
             return None
 
+    def getTokenForConnection(self, connectionId: str, auto_refresh: bool = True) -> Optional[Token]:
+        """Get the token for a specific connection, optionally auto-refresh if expired"""
+        try:
+            # Get token for this specific connection
+            tokens = self.db.getRecordset("tokens", recordFilter={
+                "connectionId": connectionId
+            })
+            
+            if not tokens:
+                logger.warning(f"No token found for connection: {connectionId}")
+                return None
+                
+            # Sort by creation date and get the latest
+            tokens.sort(key=lambda x: x.get("createdAt", ""), reverse=True)
+            latest_token = Token(**tokens[0])
+            
+            # Check if token is expired
+            if latest_token.expiresAt and latest_token.expiresAt < datetime.now().timestamp():
+                if auto_refresh:
+                    logger.info(f"Token for connection {connectionId} is expired, attempting refresh...")
+                    
+                    # Import TokenManager here to avoid circular imports
+                    from modules.security.tokenManager import TokenManager
+                    token_manager = TokenManager()
+                    
+                    # Try to refresh the token
+                    refreshed_token = token_manager.refresh_token(latest_token)
+                    if refreshed_token:
+                        # Save the new token and delete the old one
+                        self.saveToken(refreshed_token)
+                        self.deleteTokenByConnectionId(connectionId)
+                        
+                        logger.info(f"Successfully refreshed token for connection {connectionId}")
+                        return refreshed_token
+                    else:
+                        logger.warning(f"Failed to refresh expired token for connection {connectionId}")
+                        return None
+                else:
+                    logger.warning(f"Token for connection {connectionId} is expired (expiresAt: {latest_token.expiresAt})")
+                    return None
+            
+            return latest_token
+            
+        except Exception as e:
+            logger.error(f"Error getting token for connection {connectionId}: {str(e)}")
+            return None
+
     def deleteToken(self, authority: str) -> None:
         """Delete all tokens for the current user and authority"""
         try:
@@ -823,6 +891,44 @@ class AppObjects:
             logger.error(f"Error deleting token: {str(e)}")
             raise
 
+    def deleteTokenByConnectionId(self, connectionId: str) -> None:
+        """Delete all tokens for a specific connection"""
+        try:
+            # Get tokens to delete
+            tokens = self.db.getRecordset("tokens", recordFilter={
+                "connectionId": connectionId
+            })
+            
+            # Delete each token
+            for token in tokens:
+                self.db.recordDelete("tokens", token["id"])
+            
+            # Clear cache to ensure fresh data
+            self._clearTableCache("tokens")
+            
+        except Exception as e:
+            logger.error(f"Error deleting token for connection {connectionId}: {str(e)}")
+            raise
+
+    def logout(self) -> None:
+        """Logout current user - clear user context and tokens"""
+        try:
+            # Clear user context
+            self.currentUser = None
+            self.userId = None
+            self.mandateId = None
+            self.access = None
+            
+            # Clear database context
+            if hasattr(self, 'db'):
+                self.db.updateContext("")
+                
+            logger.info("User logged out successfully")
+            
+        except Exception as e:
+            logger.error(f"Error during logout: {str(e)}")
+            raise
+
 # Public Methods
 
 def getInterface(currentUser: User) -> AppObjects:

modules/interfaces/interfaceChatModel.py

@@ -12,73 +12,50 @@ from modules.shared.attributeUtils import register_model_labels, ModelMixin
 
 # ===== Method Models =====
 
+class ActionDocument(BaseModel, ModelMixin):
+    """Clear document structure for action results"""
+    documentName: str = Field(description="Name of the document")
+    documentData: Any = Field(description="Content/data of the document")
+    mimeType: str = Field(description="MIME type of the document")
+
+# Register labels for ActionDocument
+register_model_labels(
+    "ActionDocument",
+    {"en": "Action Document", "fr": "Document d'action"},
+    {
+        "documentName": {"en": "Document Name", "fr": "Nom du document"},
+        "documentData": {"en": "Document Data", "fr": "Données du document"},
+        "mimeType": {"en": "MIME Type", "fr": "Type MIME"}
+    }
+)
+
 class ActionResult(BaseModel, ModelMixin):
-    """Unified model for action results with workflow state management"""
+    """Clean action result with documents as primary output"""
-    # Core result fields
+    # Core result
-    success: bool = Field(description="Whether the method execution was successful")
+    success: bool = Field(description="Whether execution succeeded")
-    data: Dict[str, Any] = Field(description="Result data")
+    error: Optional[str] = Field(None, description="Error message if failed")
-    metadata: Dict[str, Any] = Field(default_factory=dict, description="Additional metadata")
-    error: Optional[str] = Field(None, description="Error message if any")
     
-    # Action identification
+    # Primary output - documents
-    actionId: Optional[str] = Field(None, description="ID of the action that produced this result")
+    documents: List[ActionDocument] = Field(default_factory=list, description="Document outputs")
-    actionMethod: Optional[str] = Field(None, description="Method of the action that produced this result")
+    resultLabel: Optional[str] = Field(None, description="Label for document routing")
-    actionName: Optional[str] = Field(None, description="Name of the action that produced this result")
-    
-    # Document handling
-    documents: List[str] = Field(default_factory=list, description="List of document references")
-    resultLabel: Optional[str] = Field(None, description="Label for the result")
-    
-    # Validation and workflow state
-    validation: Dict[str, Any] = Field(default_factory=dict, description="Validation information")
-    is_retry: bool = Field(default=False, description="Whether this is a retry attempt")
-    previous_error: Optional[str] = Field(None, description="Previous error message for retries")
-    applied_improvements: List[str] = Field(default_factory=list, description="Improvements applied for retry")
     
     @classmethod
-    def success(cls, documents: List[str] = None, resultLabel: str = None, data: Dict[str, Any] = None, 
+    def success(cls, documents: List[ActionDocument] = None, resultLabel: str = None) -> 'ActionResult':
-                actionId: str = None, actionMethod: str = None, actionName: str = None) -> 'ActionResult':
         """Create a successful action result"""
         return cls(
             success=True,
-            data=data or {},
             documents=documents or [],
-            resultLabel=resultLabel,
+            resultLabel=resultLabel
-            actionId=actionId,
-            actionMethod=actionMethod,
-            actionName=actionName
         )
     
     @classmethod
-    def failure(cls, error: str, data: Dict[str, Any] = None, 
+    def failure(cls, error: str, documents: List[ActionDocument] = None, resultLabel: str = None) -> 'ActionResult':
-                actionId: str = None, actionMethod: str = None, actionName: str = None) -> 'ActionResult':
         """Create a failed action result"""
         return cls(
             success=False,
-            data=data or {},
+            documents=documents or [],
             error=error,
-            actionId=actionId,
+            resultLabel=resultLabel
-            actionMethod=actionMethod,
-            actionName=actionName
-        )
-    
-    @classmethod
-    def retry(cls, previous_result: 'ActionResult', improvements: List[str] = None) -> 'ActionResult':
-        """Create a retry action result based on a previous result"""
-        return cls(
-            success=previous_result.success,
-            data=previous_result.data,
-            metadata=previous_result.metadata,
-            validation=previous_result.validation,
-            error=previous_result.error,
-            documents=previous_result.documents,
-            resultLabel=previous_result.resultLabel,
-            actionId=previous_result.actionId,
-            actionMethod=previous_result.actionMethod,
-            actionName=previous_result.actionName,
-            is_retry=True,
-            previous_error=previous_result.error,
-            applied_improvements=improvements or []
         )
 
 # Register labels for ActionResult
@@ -87,18 +64,9 @@ register_model_labels(
     {"en": "Action Result", "fr": "Résultat de l'action"},
     {
         "success": {"en": "Success", "fr": "Succès"},
-        "data": {"en": "Data", "fr": "Données"},
-        "metadata": {"en": "Metadata", "fr": "Métadonnées"},
-        "validation": {"en": "Validation", "fr": "Validation"},
         "error": {"en": "Error", "fr": "Erreur"},
         "documents": {"en": "Documents", "fr": "Documents"},
-        "resultLabel": {"en": "Result Label", "fr": "Étiquette du résultat"},
+        "resultLabel": {"en": "Result Label", "fr": "Étiquette du résultat"}
-        "actionId": {"en": "Action ID", "fr": "ID de l'action"},
-        "actionMethod": {"en": "Action Method", "fr": "Méthode de l'action"},
-        "actionName": {"en": "Action Name", "fr": "Nom de l'action"},
-        "is_retry": {"en": "Is Retry", "fr": "Est une nouvelle tentative"},
-        "previous_error": {"en": "Previous Error", "fr": "Erreur précédente"},
-        "applied_improvements": {"en": "Applied Improvements", "fr": "Améliorations appliquées"}
     }
 )
 
@@ -536,21 +504,102 @@ class TaskStep(BaseModel, ModelMixin):
     success_criteria: Optional[list[str]] = []
     estimated_complexity: Optional[str] = None
 
+class TaskHandover(BaseModel, ModelMixin):
+    """Structured handover between workflow phases and tasks"""
+    taskId: str = Field(description="Target task ID")
+    sourceTask: Optional[str] = Field(None, description="Source task ID")
+    
+    # Document handovers
+    inputDocuments: List[DocumentExchange] = Field(default_factory=list, description="Available input documents")
+    outputDocuments: List[DocumentExchange] = Field(default_factory=list, description="Produced output documents")
+    
+    # Context and state
+    context: Dict[str, Any] = Field(default_factory=dict, description="Task context")
+    previousResults: List[str] = Field(default_factory=list, description="Previous result summaries")
+    improvements: List[str] = Field(default_factory=list, description="Improvement suggestions")
+    
+    # Workflow context
+    workflowSummary: Optional[str] = Field(None, description="Summarized workflow context")
+    messageHistory: List[str] = Field(default_factory=list, description="Key message summaries")
+    
+    # Metadata
+    timestamp: datetime = Field(default_factory=lambda: datetime.now(UTC), description="When the handover was created")
+    handoverType: str = Field(default="task", description="Type of handover: task, phase, or workflow")
+    
+    def addInputDocument(self, documentExchange: DocumentExchange) -> None:
+        """Add an input document exchange"""
+        self.inputDocuments.append(documentExchange)
+    
+    def addOutputDocument(self, documentExchange: DocumentExchange) -> None:
+        """Add an output document exchange"""
+        self.outputDocuments.append(documentExchange)
+    
+    def getDocumentsForAction(self, actionId: str) -> List[DocumentExchange]:
+        """Get all document exchanges relevant for a specific action"""
+        relevant = []
+        for doc_exchange in self.inputDocuments + self.outputDocuments:
+            if doc_exchange.isForAction(actionId):
+                relevant.append(doc_exchange)
+        return relevant
+
+# Register labels for TaskHandover
+register_model_labels(
+    "TaskHandover",
+    {"en": "Task Handover", "fr": "Transfert de tâche"},
+    {
+        "taskId": {"en": "Task ID", "fr": "ID de la tâche"},
+        "sourceTask": {"en": "Source Task", "fr": "Tâche source"},
+        "inputDocuments": {"en": "Input Documents", "fr": "Documents d'entrée"},
+        "outputDocuments": {"en": "Output Documents", "fr": "Documents de sortie"},
+        "context": {"en": "Context", "fr": "Contexte"},
+        "previousResults": {"en": "Previous Results", "fr": "Résultats précédents"},
+        "improvements": {"en": "Improvements", "fr": "Améliorations"},
+        "workflowSummary": {"en": "Workflow Summary", "fr": "Résumé du workflow"},
+        "messageHistory": {"en": "Message History", "fr": "Historique des messages"},
+        "timestamp": {"en": "Timestamp", "fr": "Horodatage"},
+        "handoverType": {"en": "Handover Type", "fr": "Type de transfert"}
+    }
+)
 
 class TaskContext(BaseModel, ModelMixin):
     task_step: TaskStep
     workflow: Optional['ChatWorkflow'] = None
     workflow_id: Optional[str] = None
+    
+    # Available resources
     available_documents: Optional[list[str]] = []
+    available_connections: Optional[list[str]] = []
+    
+    # Previous execution state
     previous_results: Optional[list[str]] = []
+    previous_handover: Optional[TaskHandover] = None
+    
+    # Current execution state
     improvements: Optional[list[str]] = []
     retry_count: Optional[int] = 0
     previous_action_results: Optional[list] = []
     previous_review_result: Optional[dict] = None
     is_regeneration: Optional[bool] = False
+    
+    # Failure analysis
     failure_patterns: Optional[list[str]] = []
     failed_actions: Optional[list] = []
     successful_actions: Optional[list] = []
+    
+    def getDocumentReferences(self) -> List[str]:
+        """Get all available document references"""
+        docs = self.available_documents or []
+        if self.previous_handover:
+            for doc_exchange in self.previous_handover.inputDocuments:
+                docs.extend(doc_exchange.documents)
+        return list(set(docs))  # Remove duplicates
+    
+    def addImprovement(self, improvement: str) -> None:
+        """Add an improvement suggestion"""
+        if improvement not in (self.improvements or []):
+            if self.improvements is None:
+                self.improvements = []
+            self.improvements.append(improvement)
 
 class ReviewContext(BaseModel, ModelMixin):
     task_step: TaskStep
@@ -582,3 +631,5 @@ class WorkflowResult(BaseModel, ModelMixin):
     final_results_count: int
     error: Optional[str] = None
     phase: Optional[str] = None
+
+

modules/methods/EXCLUDED_methodCoder.py

@@ -1,327 +0,0 @@
-from typing import Dict, Any, Optional, List
-import logging
-import uuid
-from datetime import datetime, UTC
-
-from modules.chat.methodBase import MethodBase, ActionResult, action
-
-logger = logging.getLogger(__name__)
-
-class MethodCoder(MethodBase):
-    """Coder method implementation for code operations"""
-    
-    def __init__(self, serviceCenter: Any):
-        """Initialize the coder method"""
-        super().__init__(serviceCenter)
-        self.name = "coder"
-        self.description = "Handle code operations like analysis, generation, and refactoring"
-
-    @action
-    async def analyze(self, parameters: Dict[str, Any]) -> ActionResult:
-        """
-        Analyze code quality and structure
-        
-        Parameters:
-            documentList (str): Reference to the document list to analyze
-            aiPrompt (str): AI prompt for code analysis
-            language (str, optional): Programming language (default: "python")
-            checks (List[str], optional): Types of checks to perform (default: ["complexity", "style", "security"])
-        """
-        try:
-            documentList = parameters.get("documentList")
-            aiPrompt = parameters.get("aiPrompt")
-            language = parameters.get("language", "python")
-            checks = parameters.get("checks", ["complexity", "style", "security"])
-            
-            if not documentList:
-                return self._createResult(
-                    success=False,
-                    data={},
-                    error="Document list reference is required"
-                )
-            
-            if not aiPrompt:
-                return self._createResult(
-                    success=False,
-                    data={},
-                    error="AI prompt is required"
-                )
-            
-            # Handle new document list format (list of strings)
-            chatDocuments = self.service.getChatDocumentsFromDocumentList(documentList)
-            if not chatDocuments:
-                return self._createResult(
-                    success=False,
-                    data={},
-                    error="No documents found for the provided reference"
-                )
-            
-            # Process each document individually
-            all_code_content = []
-            
-            for chatDocument in chatDocuments:
-                fileId = chatDocument.fileId
-                code = self.service.getFileData(fileId)
-                file_info = self.service.getFileInfo(fileId)
-                
-                if not code:
-                    logger.warning(f"Code file is empty for fileId: {fileId}")
-                    continue
-                
-                # Use AI prompt to extract relevant code content
-                extracted_content = await self.service.extractContentFromFileData(
-                    prompt=aiPrompt,
-                    fileData=code,
-                    filename=file_info.get('name', 'code'),
-                    mimeType=file_info.get('mimeType', 'text/plain'),
-                    base64Encoded=False
-                )
-                
-                all_code_content.append(extracted_content)
-            
-            if not all_code_content:
-                return self._createResult(
-                    success=False,
-                    data={},
-                    error="No code content could be extracted from any documents"
-                )
-            
-            # Extract text content from ExtractedContent objects
-            text_contents = []
-            for content_obj in all_code_content:
-                if hasattr(content_obj, 'contents') and content_obj.contents:
-                    # Extract text from ContentItem objects
-                    for content_item in content_obj.contents:
-                        if hasattr(content_item, 'data') and content_item.data:
-                            text_contents.append(content_item.data)
-                elif isinstance(content_obj, str):
-                    text_contents.append(content_obj)
-                else:
-                    # Fallback: convert to string representation
-                    text_contents.append(str(content_obj))
-            
-            # Combine all extracted text content for analysis
-            combined_content = "\n\n--- CODE SEPARATOR ---\n\n".join(text_contents)
-            
-            # Create analysis prompt
-            analysis_prompt = f"""
-            Analyze this {language} code for quality, structure, and potential issues.
-            
-            Code to analyze:
-            {combined_content}
-            
-            Please check for:
-            {', '.join(checks)}
-            
-            Provide a detailed analysis including:
-            1. Code quality assessment
-            2. Potential issues and improvements
-            3. Security considerations
-            4. Performance optimizations
-            5. Best practices compliance
-            6. Summary of findings across all documents
-            """
-            
-            # Use AI service for analysis
-            analysis_result = await self.service.interfaceAiCalls.callAiTextAdvanced(analysis_prompt)
-            
-            # Create result data
-            result_data = {
-                "documentCount": len(chatDocuments),
-                "language": language,
-                "checks": checks,
-                "analysis": analysis_result,
-                "timestamp": datetime.now(UTC).isoformat()
-            }
-            
-            return self._createResult(
-                success=True,
-                data={
-                    "documents": [
-                        {
-                            "documentName": f"code_analysis_{datetime.now(UTC).strftime('%Y%m%d_%H%M%S')}.json",
-                            "documentData": result_data
-                        }
-                    ]
-                }
-            )
-            
-        except Exception as e:
-            logger.error(f"Error analyzing code: {str(e)}")
-            return self._createResult(
-                success=False,
-                data={},
-                error=str(e)
-            )
-    
-    @action
-    async def generate(self, parameters: Dict[str, Any]) -> ActionResult:
-        """
-        Generate code based on requirements
-        
-        Parameters:
-            requirements (str): Requirements for the code to generate
-            language (str, optional): Programming language (default: "python")
-            template (str, optional): Template or pattern to follow
-        """
-        try:
-            requirements = parameters.get("requirements")
-            language = parameters.get("language", "python")
-            template = parameters.get("template")
-            
-            if not requirements:
-                return self._createResult(
-                    success=False,
-                    data={},
-                    error="Requirements are required"
-                )
-            
-            # Create generation prompt
-            generation_prompt = f"""
-            Generate {language} code based on the following requirements:
-            
-            Requirements:
-            {requirements}
-            
-            {f'Template to follow: {template}' if template else ''}
-            
-            Please provide:
-            1. Complete, working code
-            2. Clear comments and documentation
-            3. Error handling where appropriate
-            4. Best practices implementation
-            """
-            
-            # Use AI service for code generation
-            generated_code = await self.service.interfaceAiCalls.callAiTextAdvanced(generation_prompt)
-            
-            # Create result data
-            result_data = {
-                "language": language,
-                "requirements": requirements,
-                "code": generated_code,
-                "timestamp": datetime.now(UTC).isoformat()
-            }
-            
-            return self._createResult(
-                success=True,
-                data={
-                    "documentName": f"generated_code_{datetime.now(UTC).strftime('%Y%m%d_%H%M%S')}.{language}",
-                    "documentData": result_data
-                }
-            )
-            
-        except Exception as e:
-            logger.error(f"Error generating code: {str(e)}")
-            return self._createResult(
-                success=False,
-                data={},
-                error=str(e)
-            )
-    
-    @action
-    async def refactor(self, parameters: Dict[str, Any]) -> ActionResult:
-        """
-        Refactor code for better quality
-        
-        Parameters:
-            documentList (str): Reference to the document list to refactor
-            aiImprovementPrompt (str): AI prompt for code improvements
-            language (str, optional): Programming language (default: "python")
-        """
-        try:
-            documentList = parameters.get("documentList")
-            aiImprovementPrompt = parameters.get("aiImprovementPrompt")
-            language = parameters.get("language", "python")
-            
-            if not documentList:
-                return self._createResult(
-                    success=False,
-                    data={},
-                    error="Document list reference is required"
-                )
-            
-            if not aiImprovementPrompt:
-                return self._createResult(
-                    success=False,
-                    data={},
-                    error="AI improvement prompt is required"
-                )
-            
-            # Handle new document list format (list of strings)
-            chatDocuments = self.service.getChatDocumentsFromDocumentList(documentList)
-            if not chatDocuments:
-                return self._createResult(
-                    success=False,
-                    data={},
-                    error="No documents found for the provided reference"
-                )
-            
-            # Process each document individually
-            refactored_results = []
-            
-            for chatDocument in chatDocuments:
-                fileId = chatDocument.fileId
-                code = self.service.getFileData(fileId)
-                file_info = self.service.getFileInfo(fileId)
-                
-                if not code:
-                    logger.warning(f"Code file is empty for fileId: {fileId}")
-                    continue
-                
-                # Create refactoring prompt for this specific document
-                refactor_prompt = f"""
-                Refactor this {language} code based on the following improvement requirements:
-                
-                Improvement requirements:
-                {aiImprovementPrompt}
-                
-                Original code:
-                {code}
-                
-                Please provide:
-                1. Refactored code with improvements
-                2. Explanation of changes made
-                3. Benefits of the refactoring
-                4. Any potential trade-offs
-                """
-                
-                # Use AI service for refactoring
-                refactored_code = await self.service.interfaceAiCalls.callAiTextAdvanced(refactor_prompt)
-                
-                refactored_results.append({
-                    "original_file": file_info.get('name', 'unknown'),
-                    "original_code": code,
-                    "refactored_code": refactored_code
-                })
-            
-            if not refactored_results:
-                return self._createResult(
-                    success=False,
-                    data={},
-                    error="No code could be refactored from any documents"
-                )
-            
-            # Create result data
-            result_data = {
-                "documentCount": len(chatDocuments),
-                "language": language,
-                "refactored_results": refactored_results,
-                "timestamp": datetime.now(UTC).isoformat()
-            }
-            
-            return self._createResult(
-                success=True,
-                data={
-                    "documentName": f"refactored_code_{datetime.now(UTC).strftime('%Y%m%d_%H%M%S')}.{language}",
-                    "documentData": result_data
-                }
-            )
-            
-        except Exception as e:
-            logger.error(f"Error refactoring code: {str(e)}")
-            return self._createResult(
-                success=False,
-                data={},
-                error=str(e)
-            ) 

modules/methods/methodAi.py

@@ -5,10 +5,10 @@ Handles direct AI calls for any type of task.
 
 import logging
 from typing import Dict, Any, List, Optional
-import uuid
 from datetime import datetime, UTC
 
-from modules.chat.methodBase import MethodBase, ActionResult, action
+from modules.chat.methodBase import MethodBase, action
+from modules.interfaces.interfaceChatModel import ActionResult
 
 logger = logging.getLogger(__name__)
 
@@ -43,9 +43,7 @@ class MethodAi(MethodBase):
             customInstructions = parameters.get("customInstructions", "")
             
             if not aiPrompt:
-                return self._createResult(
+                return ActionResult.failure(
-                    success=False,
-                    data={},
                     error="AI prompt is required"
                 )
             
@@ -56,27 +54,79 @@ class MethodAi(MethodBase):
                 if chatDocuments:
                     context_parts = []
                     for doc in chatDocuments:
-                        fileId = doc.fileId
+                        file_info = self.service.getFileInfo(doc.fileId)
-                        file_data = self.service.getFileData(fileId)
-                        file_info = self.service.getFileInfo(fileId)
                         
-                        if file_data:
+                        try:
-                            try:
+                            # Use the document content extraction service with the specific AI prompt context
-                                # Try to decode as text for context
+                            # This tells the extraction engine exactly what and how to extract
-                                content = file_data.decode('utf-8')
+                            extraction_prompt = f"""
-                                metadata_info = ""
+                            Extract content from this document for AI processing context.
-                                if file_info and includeMetadata:
+                            
-                                    metadata_info = f" (Size: {file_info.get('fileSize', 'unknown')}, Type: {file_info.get('mimeType', 'unknown')})"
+                            AI Task: {aiPrompt}
+                            Processing Mode: {processingMode}
+                            Expected Output: {output_extension.upper()} format
+                            
+                            Requirements:
+                            1. Extract the most relevant text content that would be useful for the AI task
+                            2. Focus on content that directly relates to: {aiPrompt}
+                            3. Include key information, data, and insights that the AI needs
+                            4. Provide clean, readable text without formatting artifacts
+                            
+                            Document: {doc.filename}
+                            """
+                            
+                            logger.debug(f"Extracting content from {doc.filename} with task-specific prompt: {extraction_prompt[:100]}...")
+                            
+                            extracted_content = await self.service.extractContentFromDocument(
+                                prompt=extraction_prompt.strip(), 
+                                document=doc
+                            )
+                            
+                            if extracted_content and extracted_content.contents:
+                                # Get the first content item's data
+                                content = ""
+                                for content_item in extracted_content.contents:
+                                    if hasattr(content_item, 'data') and content_item.data:
+                                        content += content_item.data + " "
+
                                 
-                                # Adjust context length based on processing mode
+                                if content.strip():
-                                max_length = 5000 if processingMode == "detailed" else 3000 if processingMode == "advanced" else 2000
+                                    metadata_info = ""
-                                context_parts.append(f"Document: {doc.filename}{metadata_info}\nContent:\n{content[:max_length]}...")
+                                    if file_info and includeMetadata:
-                            except UnicodeDecodeError:
+                                        metadata_info = f" (Size: {file_info.get('fileSize', 'unknown')}, Type: {file_info.get('mimeType', 'unknown')})"
-                                context_parts.append(f"Document: {doc.filename} [Binary content]")
+                                    
+                                    # Adjust context length based on processing mode and AI task relevance
+                                    base_length = 5000 if processingMode == "detailed" else 3000 if processingMode == "advanced" else 2000
+                                    
+                                    # For detailed mode, include more context
+                                    if processingMode == "detailed":
+                                        context_parts.append(f"Document: {doc.filename}{metadata_info}\nRelevance to AI Task: This document contains content directly related to '{aiPrompt[:100]}...'\nContent:\n{content[:base_length]}...")
+                                    else:
+                                        context_parts.append(f"Document: {doc.filename}{metadata_info}\nContent:\n{content[:base_length]}...")
+                                else:
+                                    context_parts.append(f"Document: {doc.filename} [No readable text content - binary file]")
+                            else:
+                                context_parts.append(f"Document: {doc.filename} [No readable text content - binary file]")
+                                
+                        except Exception as extract_error:
+                            context_parts.append(f"Document: {doc.filename} [Could not extract content - binary file]")
                     
                     if context_parts:
-                        context = "\n\n".join(context_parts)
+                        # Add a summary header to help the AI understand the context
-                        logger.info(f"Included {len(chatDocuments)} documents in AI context")
+                        context_header = f"""
+                        === DOCUMENT CONTEXT FOR AI PROCESSING ===
+                        AI Task: {aiPrompt[:100]}...
+                        Processing Mode: {processingMode}
+                        Expected Output Format: {output_extension.upper()}
+                        Total Documents: {len(chatDocuments)}
+                        
+                        The following documents contain content relevant to your task. 
+                        Use this information to provide the most accurate and helpful response.
+                        ================================================
+                        """
+                        
+                        context = context_header + "\n\n" + "\n\n".join(context_parts)
+                        logger.info(f"Included {len(chatDocuments)} documents in AI context with task-specific extraction")
             
             # Determine output format
             output_extension = ".txt"  # Default
@@ -126,39 +176,23 @@ class MethodAi(MethodBase):
             timestamp = datetime.now(UTC).strftime('%Y%m%d_%H%M%S')
             filename = f"ai_{processingMode}_{timestamp}{output_extension}"
             
-            # Create document through service (but don't add to workflow - let calling layer handle that)
+
-            document = self.service.createDocument(
-                fileName=filename,
-                mimeType=output_mime_type,
-                content=result,
-                base64encoded=False
-            )
             
-            return self._createResult(
+            # Return result in the standard ActionResult format
-                success=True,
+            return ActionResult.success(
-                data={
+                documents=[{
-                    "result": result,
+                    "documentName": filename,
-                    "filename": filename,
+                    "documentData": {
-                    "documentId": document.id if hasattr(document, 'id') else None,
+                        "result": result,
-                    "processedDocuments": len(documentList) if documentList else 0,
+                        "filename": filename,
-                    "processingMode": processingMode,
+                        "processedDocuments": len(documentList) if documentList else 0
-                    "document": document  # Include the created document in the result data
+                    },
-                },
+                    "mimeType": output_mime_type
-                metadata={
+                }]
-                    "method": "ai.process",
-                    "promptLength": len(aiPrompt),
-                    "contextLength": len(context),
-                    "outputFormat": output_extension,
-                    "includeMetadata": includeMetadata,
-                    "processingMode": processingMode,
-                    "hasCustomInstructions": bool(customInstructions)
-                }
             )
             
         except Exception as e:
-            logger.error(f"Error in ai.process: {str(e)}")
+            logger.error(f"Error in AI processing: {str(e)}")
-            return self._createResult(
+            return ActionResult.failure(
-                success=False,
+                error=str(e)
-                data={},
-                error=f"AI processing failed: {str(e)}"
             )

modules/methods/methodDocument.py

@@ -7,7 +7,8 @@ import logging
 from typing import Dict, Any, List, Optional
 from datetime import datetime, UTC
 
-from modules.chat.methodBase import MethodBase, ActionResult, action
+from modules.chat.methodBase import MethodBase, action
+from modules.interfaces.interfaceChatModel import ActionResult
 
 logger = logging.getLogger(__name__)
 
@@ -23,13 +24,13 @@ class MethodDocument(MethodBase):
     @action
     async def extract(self, parameters: Dict[str, Any]) -> ActionResult:
         """
-        Extract specific content from document with AI prompt and return it in the specified format.
+        Extract content from any document using AI prompt.
         
         Parameters:
-            documentList (str): Reference to the document list to extract content from
+            documentList (str): Document list reference
-            aiPrompt (str): AI prompt for content extraction
+            aiPrompt (str): AI prompt for extraction
-            expectedDocumentFormats (list, optional): Expected document formats with extension, mimeType, description
+            expectedDocumentFormats (list, optional): Output formats
-            includeMetadata (bool, optional): Whether to include metadata (default: True)
+            includeMetadata (bool, optional): Include metadata (default: True)
         """
         try:
             documentList = parameters.get("documentList")
@@ -38,24 +39,18 @@ class MethodDocument(MethodBase):
             includeMetadata = parameters.get("includeMetadata", True)
             
             if not documentList:
-                return self._createResult(
+                return ActionResult.failure(
-                    success=False,
-                    data={},
                     error="Document list reference is required"
                 )
             
             if not aiPrompt:
-                return self._createResult(
+                return ActionResult.failure(
-                    success=False,
-                    data={},
                     error="AI prompt is required"
                 )
             
             chatDocuments = self.service.getChatDocumentsFromDocumentList(documentList)
             if not chatDocuments:
-                return self._createResult(
+                return ActionResult.failure(
-                    success=False,
-                    data={},
                     error="No documents found for the provided reference"
                 )
             
@@ -64,31 +59,30 @@ class MethodDocument(MethodBase):
             file_infos = []
             
             for chatDocument in chatDocuments:
-                fileId = chatDocument.fileId
+                file_info = self.service.getFileInfo(chatDocument.fileId)
-                file_data = self.service.getFileData(fileId)
-                file_info = self.service.getFileInfo(fileId)
                 
-                if not file_data:
+                try:
-                    logger.warning(f"File not found or empty for fileId: {fileId}")
+                    # Use the document content extraction service with the specific AI prompt
+                    # This handles all document types (text, binary, image, etc.) intelligently
+                    extracted_content = await self.service.extractContentFromDocument(
+                        prompt=aiPrompt,
+                        document=chatDocument
+                    )
+                    
+                    if extracted_content and extracted_content.contents:
+                        all_extracted_content.append(extracted_content)
+                        if includeMetadata:
+                            file_infos.append(file_info)
+                        logger.info(f"Successfully extracted content from {chatDocument.filename}")
+                    else:
+                        logger.warning(f"No content extracted from {chatDocument.filename}")
+                        
+                except Exception as e:
+                    logger.error(f"Error extracting content from {chatDocument.filename}: {str(e)}")
                     continue
-                
-                extracted_content = await self.service.extractContentFromFileData(
-                    prompt=aiPrompt,
-                    fileData=file_data,
-                    filename=file_info.get('name', 'document'),
-                    mimeType=file_info.get('mimeType', 'application/octet-stream'),
-                    base64Encoded=False,
-                    documentId=chatDocument.id
-                )
-                
-                all_extracted_content.append(extracted_content)
-                if includeMetadata:
-                    file_infos.append(file_info)
             
             if not all_extracted_content:
-                return self._createResult(
+                return ActionResult.failure(
-                    success=False,
-                    data={},
                     error="No content could be extracted from any documents"
                 )
             
@@ -164,31 +158,25 @@ class MethodDocument(MethodBase):
                     "mimeType": final_mime_type
                 })
             
-            return self._createResult(
+            return ActionResult.success(
-                success=True,
+                documents=output_documents
-                data={
-                    "documents": output_documents
-                }
             )
         except Exception as e:
             logger.error(f"Error extracting content: {str(e)}")
-            return self._createResult(
+            return ActionResult.failure(
-                success=False,
-                data={},
                 error=str(e)
             )
 
     @action
     async def generate(self, parameters: Dict[str, Any]) -> ActionResult:
         """
-        Generate documents in specific formats from document references.
+        Convert TEXT-ONLY documents to target formats (NO AI usage).
-        This action automatically extracts content from documents and converts it to the specified format.
         
         Parameters:
-            documentList (list): List of document references to extract content from
+            documentList (list): TEXT-ONLY documents only
-            expectedDocumentFormats (list): Expected document formats with extension, mimeType, description
+            expectedDocumentFormats (list): Target formats
-            originalDocuments (list, optional): List of original document names
+            originalDocuments (list, optional): Original names
-            includeMetadata (bool, optional): Whether to include metadata (default: True)
+            includeMetadata (bool, optional): Include metadata (default: True)
         """
         try:
             document_list = parameters.get("documentList", [])
@@ -197,16 +185,12 @@ class MethodDocument(MethodBase):
             include_metadata = parameters.get("includeMetadata", True)
             
             if not document_list:
-                return self._createResult(
+                return ActionResult.failure(
-                    success=False,
-                    data={},
                     error="Document list is required for generation"
                 )
             
             if not expected_document_formats or len(expected_document_formats) == 0:
-                return self._createResult(
+                return ActionResult.failure(
-                    success=False,
-                    data={},
                     error="Expected document formats specification is required"
                 )
             
@@ -215,9 +199,7 @@ class MethodDocument(MethodBase):
             logger.info(f"Found {len(chat_documents)} chat documents")
             
             if not chat_documents:
-                return self._createResult(
+                return ActionResult.failure(
-                    success=False,
-                    data={},
                     error="No documents found for the provided documentList reference"
                 )
             
@@ -229,28 +211,42 @@ class MethodDocument(MethodBase):
             output_documents = []
             
             for i, chat_document in enumerate(chat_documents):
-                # Extract content from this document
+                # Extract content from this document directly - NO AI, just read the data as-is
-                # ChatDocument is just a reference, so we need to get file data using fileId
+                # This ensures we get the original text content for format conversion
                 content = ""
                 if hasattr(chat_document, 'fileId') and chat_document.fileId:
-                    # Need to get file data
+                    try:
-                    file_data = self.service.getFileData(chat_document.fileId)
+                        # Get file data directly without AI processing
-                    if file_data:
+                        file_data = self.service.getFileData(chat_document.fileId)
-                        if isinstance(file_data, bytes):
+                        if file_data:
-                            content = file_data.decode('utf-8', errors='ignore')
+                            # Check if it's text data and convert to string
+                            if isinstance(file_data, bytes):
+                                try:
+                                    # Try to decode as UTF-8 to check if it's text
+                                    content = file_data.decode('utf-8')
+                                    logger.info(f"Document {i+1} ({chat_document.filename}): Successfully decoded as UTF-8 text")
+                                except UnicodeDecodeError:
+                                    logger.info(f"Document {i+1} ({chat_document.filename}): Binary data, not text - skipping")
+                                    continue
+                            else:
+                                # Already a string
+                                content = str(file_data)
+                                logger.info(f"Document {i+1} ({chat_document.filename}): Already text data")
                         else:
-                            content = str(file_data)
+                            logger.warning(f"Document {i+1} ({chat_document.filename}): No file data found")
-                    else:
+                            continue
-                        logger.warning(f"Could not get file data for document {i+1}, skipping")
+                        
+                        if not content.strip():
+                            logger.info(f"Document {i+1} ({chat_document.filename}): Empty text content, skipping")
+                            continue
+                            
+                    except Exception as e:
+                        logger.warning(f"Error reading document {i+1} ({chat_document.filename}): {str(e)}")
                         continue
                 else:
                     logger.warning(f"Document {i+1} has no fileId, skipping")
                     continue
                 
-                if not content:
-                    logger.warning(f"Could not extract content from document {i+1}, skipping")
-                    continue
-                
                 logger.info(f"Extracted content from document {i+1}: {len(content)} characters")
                 
                 # Get the expected format for this document (or use default)
@@ -300,23 +296,16 @@ class MethodDocument(MethodBase):
                 })
             
             if not output_documents:
-                return self._createResult(
+                return ActionResult.failure(
-                    success=False,
-                    data={},
                     error="No documents could be generated"
                 )
             
-            return self._createResult(
+            return ActionResult.success(
-                success=True,
+                documents=output_documents
-                data={
-                    "documents": output_documents
-                }
             )
         except Exception as e:
             logger.error(f"Error generating document: {str(e)}")
-            return self._createResult(
+            return ActionResult.failure(
-                success=False,
-                data={},
                 error=str(e)
             )
 
@@ -502,37 +491,40 @@ class MethodDocument(MethodBase):
     @action
     async def generateReport(self, parameters: Dict[str, Any]) -> ActionResult:
         """
-        Generate a comprehensive, professional HTML report from multiple documents, consolidating and summarizing all findings using AI.
+        Generate HTML report from multiple documents using AI.
         
         Parameters:
-            documentList (str): Reference to the document list to create the report from
+            documentList (str): Document list reference
-            title (str, optional): Title for the report (default: "Summary Report")
+            prompt (str): AI prompt for report generation
-            includeMetadata (bool, optional): Whether to include metadata (default: True)
+            title (str, optional): Report title (default: "Summary Report")
+            includeMetadata (bool, optional): Include metadata (default: True)
         """
         try:
             documentList = parameters.get("documentList")
+            prompt = parameters.get("prompt")
             title = parameters.get("title", "Summary Report")
             includeMetadata = parameters.get("includeMetadata", True)
             
             if not documentList:
-                return self._createResult(
+                return ActionResult.failure(
-                    success=False,
-                    data={},
                     error="Document list reference is required"
                 )
             
+            if not prompt:
+                return ActionResult.failure(
+                    error="Prompt is required to specify what kind of report to generate"
+                )
+            
             chatDocuments = self.service.getChatDocumentsFromDocumentList(documentList)
             logger.info(f"Retrieved {len(chatDocuments)} chat documents for report generation")
             
             if not chatDocuments:
-                return self._createResult(
+                return ActionResult.failure(
-                    success=False,
-                    data={},
                     error="No documents found for the provided reference"
                 )
             
             # Generate HTML report
-            html_content = await self._generateHtmlReport(chatDocuments, title, includeMetadata)
+            html_content = await self._generateHtmlReport(chatDocuments, title, includeMetadata, prompt)
             
             # Create output filename
             timestamp = datetime.now(UTC).strftime('%Y%m%d_%H%M%S')
@@ -547,25 +539,20 @@ class MethodDocument(MethodBase):
             
             logger.info(f"Generated HTML report: {output_filename} with {len(html_content)} characters")
             
-            return self._createResult(
+            return ActionResult.success(
-                success=True,
+                documents=[{
-                data={
+                    "documentName": output_filename,
-                    "documents": [{
+                    "documentData": result_data,
-                        "documentName": output_filename,
+                    "mimeType": "text/html"
-                        "documentData": result_data,
+                }]
-                        "mimeType": "text/html"
-                    }]
-                }
             )
         except Exception as e:
             logger.error(f"Error generating report: {str(e)}")
-            return self._createResult(
+            return ActionResult.failure(
-                success=False,
-                data={},
                 error=str(e)
             )
 
-    async def _generateHtmlReport(self, chatDocuments: List[Any], title: str, includeMetadata: bool) -> str:
+    async def _generateHtmlReport(self, chatDocuments: List[Any], title: str, includeMetadata: bool, prompt: str) -> str:
         """
         Generate a comprehensive HTML report using AI from all input documents.
         """
@@ -578,28 +565,35 @@ class MethodDocument(MethodBase):
                 content = ""
                 logger.info(f"Processing document: type={type(doc)}")
                 
-                # Get actual file content using the fileId reference
+                # Get actual file content using the document content extraction service
                 try:
-                    file_data = self.service.getFileData(doc.fileId)
+                    extracted_content = await self.service.extractContentFromDocument(
-                    if file_data:
+                        prompt="Extract readable text content for HTML report generation", 
-                        # Convert bytes to string
+                        document=doc
-                        if isinstance(file_data, bytes):
+                    )
-                            content = file_data.decode('utf-8')
+                    
+                    if extracted_content and extracted_content.contents:
+                        # Get the first content item's data
+                        for content_item in extracted_content.contents:
+                            if hasattr(content_item, 'data') and content_item.data:
+                                content += content_item.data + " "
+                        
+                        if content.strip():
+                            logger.info(f"  Retrieved content from file: {len(content)} characters")
                         else:
-                            content = str(file_data)
+                            logger.info(f"  No readable text content found (binary file)")
-                        logger.info(f"  Retrieved content from file: {len(content)} characters")
                     else:
-                        logger.warning(f"  No file data found for fileId: {doc.fileId}")
+                        logger.info(f"  No content extracted (binary file)")
                 except Exception as e:
-                    logger.error(f"  Error retrieving file data: {str(e)}")
+                    logger.info(f"  Could not extract content (binary file): {str(e)}")
                 
                 # Skip empty documents
-                if content:
+                if content and content.strip():
                     validDocuments.append(doc)
                     allContent.append(f"Document: {doc.filename}\n{content}\n")
                     logger.info(f"  Added document to valid documents list")
                 else:
-                    logger.warning(f"  Skipping document with no content")
+                    logger.info(f"  Skipping document with no readable text content")
             
             if not validDocuments:
                 # If no valid documents, create a simple report
@@ -610,14 +604,14 @@ class MethodDocument(MethodBase):
                 html.append("</body></html>")
                 return '\n'.join(html)
             
-            # Create AI prompt for comprehensive report generation
+            # Create AI prompt for comprehensive report generation using user's prompt
             combinedContent = "\n\n".join(allContent)
             aiPrompt = f"""
-            Create a comprehensive, well-structured HTML report based on the following documents and content.
+            {prompt}
             
             Report Title: {title}
             
-            Requirements:
+            Additional Requirements:
             1. Create a professional, well-formatted HTML report
             2. Include an executive summary at the beginning
             3. Organize information logically with clear sections
@@ -629,17 +623,17 @@ class MethodDocument(MethodBase):
             Document Content:
             {combinedContent}
             
-            Generate a complete HTML report that integrates all the information into a cohesive, professional document.
+            Generate a complete HTML report that addresses the user's specific requirements and integrates all the information into a cohesive, professional document.
             """
             
             # Call AI to generate the report
             logger.info(f"Generating AI report for {len(validDocuments)} documents")
             aiReport = await self.service.callAiTextBasic(aiPrompt, combinedContent)
             
-            # If AI call fails, fall back to basic HTML
+            # If AI call fails, return error - AI is crucial for report generation
             if not aiReport or aiReport.strip() == "":
-                logger.warning("AI report generation failed, using fallback HTML")
+                logger.error("AI report generation failed - AI is crucial for this action")
-                return self._generateFallbackHtmlReport(validDocuments, title, includeMetadata)
+                raise Exception("AI report generation failed - AI is required for report generation")
             
             # Clean up the AI response and ensure it's valid HTML
             if not aiReport.strip().startswith('<html'):
@@ -665,66 +659,6 @@ class MethodDocument(MethodBase):
                 
         except Exception as e:
             logger.error(f"Error generating AI report: {str(e)}")
-            # Fall back to basic HTML report
+            # Re-raise the error - AI is crucial for report generation
-            return self._generateFallbackHtmlReport(chatDocuments, title, includeMetadata)
+            raise
-    
+    
-    def _generateFallbackHtmlReport(self, chatDocuments: List[Any], title: str, includeMetadata: bool) -> str:
-        """
-        Generate a basic HTML report as fallback when AI generation fails.
-        """
-        html = ["<html><head><meta charset='utf-8'><title>" + title + "</title></head><body>"]
-        
-        # Check if any document content already contains a title/header
-        has_title = False
-        for doc in chatDocuments:
-            if hasattr(doc, 'fileId') and doc.fileId:
-                try:
-                    file_data = self.service.getFileData(doc.fileId)
-                    if file_data:
-                        content = file_data.decode('utf-8') if isinstance(file_data, bytes) else str(file_data)
-                        if any(title.lower() in content.lower() for title in [title, "outlook", "report", "status"]):
-                            has_title = True
-                            break
-                except Exception:
-                    continue
-        
-        # Only add the title if no document content already has one
-        if not has_title:
-            html.append(f"<h1>{title}</h1>")
-        
-        html.append(f"<p><b>Generated:</b> {datetime.now(UTC).strftime('%Y-%m-%d %H:%M:%S UTC')}</p>")
-        html.append(f"<p><b>Total Documents:</b> {len(chatDocuments)}</p>")
-        
-        for i, doc in enumerate(chatDocuments, 1):
-            html.append(f"<h2>Document {i}: {doc.filename}</h2>")
-            
-            if includeMetadata:
-                html.append("<ul>")
-                html.append(f"<li><b>ID:</b> {doc.id}</li>")
-                html.append(f"<li><b>File ID:</b> {doc.fileId}</li>")
-                html.append(f"<li><b>Filename:</b> {doc.filename}</li>")
-                if hasattr(doc, 'createdAt'):
-                    html.append(f"<li><b>Created:</b> {doc.createdAt}</li>")
-                html.append("</ul>")
-            
-            # Add document content if available
-            content = ""
-            if hasattr(doc, 'fileId') and doc.fileId:
-                # ChatDocument is just a reference, so we need to get file data using fileId
-                try:
-                    file_data = self.service.getFileData(doc.fileId)
-                    if file_data:
-                        if isinstance(file_data, bytes):
-                            content = file_data.decode('utf-8')
-                        else:
-                            content = str(file_data)
-                except Exception as e:
-                    logger.warning(f"Could not retrieve content for document {doc.filename}: {str(e)}")
-            
-            if content:
-                html.append(f"<div style='white-space:pre-wrap; border:1px solid #ccc; padding:0.5em; margin-bottom:1em; background-color:#f9f9f9;'>{content}</div>")
-            else:
-                html.append("<p><em>No content available</em></p>")
-        
-        html.append("</body></html>")
-        return '\n'.join(html)

modules/methods/methodSharepoint.py

@@ -1,18 +1,19 @@
 """
-SharePoint method module.
+SharePoint operations method module.
-Handles SharePoint operations using the SharePoint service.
+Handles SharePoint document operations using the SharePoint service.
 """
 
 import logging
 from typing import Dict, Any, List, Optional
 from datetime import datetime, UTC
 import json
-import uuid
+import base64
+from urllib.parse import urlparse
 import aiohttp
 import asyncio
-from urllib.parse import urlparse
 
-from modules.chat.methodBase import MethodBase, ActionResult, action
+from modules.chat.methodBase import MethodBase, action
+from modules.interfaces.interfaceChatModel import ActionResult
 
 logger = logging.getLogger(__name__)
 
@@ -41,10 +42,10 @@ class MethodSharepoint(MethodBase):
                 logger.warning(f"Connection {userConnection.id} status is not active/pending: {userConnection.status.value}")
                 return None
             
-            # Get the corresponding token for this user and authority
+            # Get the token for this specific connection
-            token = self.service.interfaceApp.getToken(userConnection.authority.value)
+            token = self.service.interfaceApp.getTokenForConnection(userConnection.id)
             if not token:
-                logger.warning(f"No token found for user {userConnection.userId} and authority {userConnection.authority.value}")
+                logger.warning(f"No token found for connection {userConnection.id}")
                 return None
             
             # Check if token is expired
@@ -178,37 +179,21 @@ class MethodSharepoint(MethodBase):
             expectedDocumentFormats = parameters.get("expectedDocumentFormats", [])
             
             if not connectionReference or not siteUrl or not query:
-                return self._createResult(
+                return ActionResult.failure(error="Connection reference, site URL, and query are required")
-                    success=False,
-                    data={},
-                    error="Connection reference, site URL, and query are required"
-                )
             
             connection = self._getMicrosoftConnection(connectionReference)
             if not connection:
-                return self._createResult(
+                return ActionResult.failure(error="No valid Microsoft connection found for the provided connection reference")
-                    success=False,
-                    data={},
-                    error="No valid Microsoft connection found for the provided connection reference"
-                )
             
             # Parse site URL to get hostname and site path
             site_info = self._parseSiteUrl(siteUrl)
             if not site_info["hostname"] or not site_info["sitePath"]:
-                return self._createResult(
+                return ActionResult.failure(error=f"Invalid SharePoint site URL: {siteUrl}")
-                    success=False,
-                    data={},
-                    error=f"Invalid SharePoint site URL: {siteUrl}"
-                )
             
             # Get site ID
             site_id = await self._getSiteId(connection["accessToken"], site_info["hostname"], site_info["sitePath"])
             if not site_id:
-                return self._createResult(
+                return ActionResult.failure(error="Failed to get SharePoint site ID")
-                    success=False,
-                    data={},
-                    error="Failed to get SharePoint site ID"
-                )
             
             try:
                 # Use Microsoft Graph search API
@@ -219,11 +204,7 @@ class MethodSharepoint(MethodBase):
                 search_result = await self._makeGraphApiCall(connection["accessToken"], endpoint)
                 
                 if "error" in search_result:
-                    return self._createResult(
+                    return ActionResult.failure(error=f"Search failed: {search_result['error']}")
-                        success=False,
-                        data={},
-                        error=f"Search failed: {search_result['error']}"
-                    )
                 
                 # Process search results
                 items = search_result.get("value", [])
@@ -279,11 +260,7 @@ class MethodSharepoint(MethodBase):
                 
             except Exception as e:
                 logger.error(f"Error searching SharePoint: {str(e)}")
-                return self._createResult(
+                return ActionResult.failure(error=str(e))
-                    success=False,
-                    data={},
-                    error=str(e)
-                )
             
             # Determine output format based on expected formats
             output_extension = ".json"  # Default
@@ -298,26 +275,21 @@ class MethodSharepoint(MethodBase):
             else:
                 logger.info("No expected format specified, using default .json format")
             
-            return self._createResult(
+            return ActionResult(
                 success=True,
-                data={
+                documents=[
-                    "documents": [
+                    {
-                        {
+                        "documentName": f"sharepoint_find_path_{datetime.now(UTC).strftime('%Y%m%d_%H%M%S')}{output_extension}",
-                            "documentName": f"sharepoint_find_path_{datetime.now(UTC).strftime('%Y%m%d_%H%M%S')}{output_extension}",
+                        "documentData": result_data,
-                            "documentData": result_data,
+                        "mimeType": output_mime_type
-                            "mimeType": output_mime_type
+                    }
-                        }
+                ],
-                    ]
+                resultLabel="sharepoint_find_path"
-                }
             )
             
         except Exception as e:
             logger.error(f"Error finding document path: {str(e)}")
-            return self._createResult(
+            return ActionResult.failure(error=str(e))
-                success=False,
-                data={},
-                error=str(e)
-            )
     
     @action
     async def readDocument(self, parameters: Dict[str, Any]) -> ActionResult:
@@ -341,11 +313,7 @@ class MethodSharepoint(MethodBase):
             expectedDocumentFormats = parameters.get("expectedDocumentFormats", [])
             
             if not documentList or not connectionReference or not siteUrl or not documentPaths:
-                return self._createResult(
+                return ActionResult.failure(error="Document list reference, connection reference, site URL, and document paths are required")
-                    success=False,
-                    data={},
-                    error="Document list reference, connection reference, site URL, and document paths are required"
-                )
             
             # Get documents from reference - ensure documentList is a list, not a string
             if isinstance(documentList, str):
@@ -365,37 +333,21 @@ class MethodSharepoint(MethodBase):
                 logger.info(f"Created {len(chatDocuments)} mock documents for testing")
             
             if not chatDocuments:
-                return self._createResult(
+                return ActionResult.failure(error="No documents found for the provided reference")
-                    success=False,
-                    data={},
-                    error="No documents found for the provided reference"
-                )
             
             connection = self._getMicrosoftConnection(connectionReference)
             if not connection:
-                return self._createResult(
+                return ActionResult.failure(error="No valid Microsoft connection found for the provided connection reference")
-                    success=False,
-                    data={},
-                    error="No valid Microsoft connection found for the provided connection reference"
-                )
             
             # Parse site URL to get hostname and site path
             site_info = self._parseSiteUrl(siteUrl)
             if not site_info["hostname"] or not site_info["sitePath"]:
-                return self._createResult(
+                return ActionResult.failure(error=f"Invalid SharePoint site URL: {siteUrl}")
-                    success=False,
-                    data={},
-                    error=f"Invalid SharePoint site URL: {siteUrl}"
-                )
             
             # Get site ID
             site_id = await self._getSiteId(connection["accessToken"], site_info["hostname"], site_info["sitePath"])
             if not site_id:
-                return self._createResult(
+                return ActionResult.failure(error="Failed to get SharePoint site ID")
-                    success=False,
-                    data={},
-                    error="Failed to get SharePoint site ID"
-                )
             
             # Process each document path
             read_results = []
@@ -514,23 +466,21 @@ class MethodSharepoint(MethodBase):
             else:
                 logger.info("No expected format specified, using default .json format")
             
-            return self._createResult(
+            return ActionResult(
                 success=True,
-                data={
+                documents=[
-                    "documents": [
+                    {
-                        {
+                        "documentName": f"sharepoint_documents_{datetime.now(UTC).strftime('%Y%m%d_%H%M%S')}{output_extension}",
-                            "documentName": f"sharepoint_documents_{datetime.now(UTC).strftime('%Y%m%d_%H%M%S')}{output_extension}",
+                        "documentData": result_data,
-                            "documentData": result_data,
+                        "mimeType": output_mime_type
-                            "mimeType": output_mime_type
+                    }
-                        }
+                ],
-                    ]
+                resultLabel="sharepoint_documents"
-                }
             )
         except Exception as e:
             logger.error(f"Error reading SharePoint documents: {str(e)}")
-            return self._createResult(
+            return ActionResult(
                 success=False,
-                data={},
                 error=str(e)
             )
     
@@ -556,49 +506,29 @@ class MethodSharepoint(MethodBase):
             expectedDocumentFormats = parameters.get("expectedDocumentFormats", [])
             
             if not connectionReference or not siteUrl or not documentPaths or not documentList or not fileNames:
-                return self._createResult(
+                return ActionResult.failure(error="Connection reference, site URL, document paths, document list, and file names are required")
-                    success=False,
-                    data={},
-                    error="Connection reference, site URL, document paths, document list, and file names are required"
-                )
             
             # Get Microsoft connection
             connection = self._getMicrosoftConnection(connectionReference)
             if not connection:
-                return self._createResult(
+                return ActionResult.failure(error="No valid Microsoft connection found for the provided connection reference")
-                    success=False,
-                    data={},
-                    error="No valid Microsoft connection found for the provided connection reference"
-                )
             
             # Get documents from reference - ensure documentList is a list, not a string
             if isinstance(documentList, str):
                 documentList = [documentList]  # Convert string to list
             chatDocuments = self.service.getChatDocumentsFromDocumentList(documentList)
             if not chatDocuments:
-                return self._createResult(
+                return ActionResult.failure(error="No documents found for the provided reference")
-                    success=False,
-                    data={},
-                    error="No documents found for the provided reference"
-                )
             
             # Parse site URL to get hostname and site path
             site_info = self._parseSiteUrl(siteUrl)
             if not site_info["hostname"] or not site_info["sitePath"]:
-                return self._createResult(
+                return ActionResult.failure(error=f"Invalid SharePoint site URL: {siteUrl}")
-                    success=False,
-                    data={},
-                    error=f"Invalid SharePoint site URL: {siteUrl}"
-                )
             
             # Get site ID
             site_id = await self._getSiteId(connection["accessToken"], site_info["hostname"], site_info["sitePath"])
             if not site_id:
-                return self._createResult(
+                return ActionResult.failure(error="Failed to get SharePoint site ID")
-                    success=False,
-                    data={},
-                    error="Failed to get SharePoint site ID"
-                )
             
             # Process each document upload
             upload_results = []
@@ -714,24 +644,22 @@ class MethodSharepoint(MethodBase):
             else:
                 logger.info("No expected format specified, using default .json format")
             
-            return self._createResult(
+            return ActionResult(
                 success=True,
-                data={
+                documents=[
-                    "documents": [
+                    {
-                        {
+                        "documentName": f"sharepoint_upload_{datetime.now(UTC).strftime('%Y%m%d_%H%M%S')}{output_extension}",
-                            "documentName": f"sharepoint_upload_{datetime.now(UTC).strftime('%Y%m%d_%H%M%S')}{output_extension}",
+                        "documentData": result_data,
-                            "documentData": result_data,
+                        "mimeType": output_mime_type
-                            "mimeType": output_mime_type
+                    }
-                        }
+                ],
-                    ]
+                resultLabel="sharepoint_upload"
-                }
             )
             
         except Exception as e:
             logger.error(f"Error uploading to SharePoint: {str(e)}")
-            return self._createResult(
+            return ActionResult(
                 success=False,
-                data={},
                 error=str(e)
             )
     
@@ -755,20 +683,12 @@ class MethodSharepoint(MethodBase):
             expectedDocumentFormats = parameters.get("expectedDocumentFormats", [])
             
             if not connectionReference or not siteUrl or not folderPaths:
-                return self._createResult(
+                return ActionResult.failure(error="Connection reference, site URL, and folder paths are required")
-                    success=False,
-                    data={},
-                    error="Connection reference, site URL, and folder paths are required"
-                )
             
             # Get Microsoft connection
             connection = self._getMicrosoftConnection(connectionReference)
             if not connection:
-                return self._createResult(
+                return ActionResult.failure(error="No valid Microsoft connection found for the provided connection reference")
-                    success=False,
-                    data={},
-                    error="No valid Microsoft connection found for the provided connection reference"
-                )
             
             logger.info(f"Starting SharePoint listDocuments for site: {siteUrl}")
             logger.debug(f"Connection ID: {connection['id']}")
@@ -780,11 +700,7 @@ class MethodSharepoint(MethodBase):
             
             if not site_info["hostname"] or not site_info["sitePath"]:
                 logger.error(f"Failed to parse site URL: {siteUrl}")
-                return self._createResult(
+                return ActionResult.failure(error=f"Invalid SharePoint site URL: {siteUrl}")
-                    success=False,
-                    data={},
-                    error=f"Invalid SharePoint site URL: {siteUrl}"
-                )
             
             # Get site ID
             logger.info(f"Getting site ID for hostname: {site_info['hostname']}, path: {site_info['sitePath']}")
@@ -792,11 +708,7 @@ class MethodSharepoint(MethodBase):
             logger.info(f"Site ID result: {site_id}")
             
             if not site_id:
-                return self._createResult(
+                return ActionResult.failure(error="Failed to get SharePoint site ID")
-                    success=False,
-                    data={},
-                    error="Failed to get SharePoint site ID"
-                )
             
             # Process each folder path
             list_results = []
@@ -941,23 +853,21 @@ class MethodSharepoint(MethodBase):
             else:
                 logger.info("No expected format specified, using default .json format")
             
-            return self._createResult(
+            return ActionResult(
                 success=True,
-                data={
+                documents=[
-                    "documents": [
+                    {
-                        {
+                        "documentName": f"sharepoint_document_list_{datetime.now(UTC).strftime('%Y%m%d_%H%M%S')}{output_extension}",
-                            "documentName": f"sharepoint_document_list_{datetime.now(UTC).strftime('%Y%m%d_%H%M%S')}{output_extension}",
+                        "documentData": result_data,
-                            "documentData": result_data,
+                        "mimeType": output_mime_type
-                            "mimeType": output_mime_type
+                    }
-                        }
+                ],
-                    ]
+                resultLabel="sharepoint_document_list"
-                }
             )
             
         except Exception as e:
             logger.error(f"Error listing SharePoint documents: {str(e)}")
-            return self._createResult(
+            return ActionResult(
                 success=False,
-                data={},
                 error=str(e)
             ) 

modules/methods/methodWeb.py

@@ -1,17 +1,22 @@
 """
-Web method module.
+Web operations method module.
-Handles web operations using the web service.
+Handles web scraping, crawling, and search operations.
 """
 
 import logging
+import requests
+import json
+import re
+import copy
 from typing import Dict, Any, List, Optional
 from datetime import datetime, UTC
-import requests
+from urllib.parse import urlparse, urljoin
-from bs4 import BeautifulSoup
 import time
-import uuid
+import random
-import json
+from bs4 import BeautifulSoup
-import copy
+import os
+
+# Selenium imports for JavaScript-heavy pages
 from selenium import webdriver
 from selenium.webdriver.chrome.options import Options
 from selenium.common.exceptions import WebDriverException
@@ -19,7 +24,8 @@ from selenium.webdriver.common.by import By
 from selenium.webdriver.support.ui import WebDriverWait
 from selenium.webdriver.support import expected_conditions as EC
 
-from modules.chat.methodBase import MethodBase, ActionResult, action
+from modules.chat.methodBase import MethodBase, action
+from modules.interfaces.interfaceChatModel import ActionResult
 from modules.shared.configuration import APP_CONFIG
 
 logger = logging.getLogger(__name__)
@@ -491,18 +497,10 @@ class MethodWeb(MethodBase):
             expectedDocumentFormats = parameters.get("expectedDocumentFormats", [])
             
             if not query:
-                return self._createResult(
+                return ActionResult.failure(error="Search query is required")
-                    success=False,
-                    data={},
-                    error="Search query is required"
-                )
             
             if not self.srcApikey:
-                return self._createResult(
+                return ActionResult.failure(error="SerpAPI key not configured")
-                    success=False,
-                    data={},
-                    error="SerpAPI key not configured"
-                )
             
             userLanguage = "en"
             if hasattr(self.service, 'user') and hasattr(self.service.user, 'language'):
@@ -555,24 +553,22 @@ class MethodWeb(MethodBase):
                 "timestamp": datetime.now(UTC).isoformat()
             }
             
-            return self._createResult(
+            return ActionResult(
                 success=True,
-                data={
+                documents=[
-                    "documents": [
+                    {
-                        {
+                        "documentName": f"web_search_{datetime.now(UTC).strftime('%Y%m%d_%H%M%S')}{output_extension}",
-                            "documentName": f"web_search_{datetime.now(UTC).strftime('%Y%m%d_%H%M%S')}{output_extension}",
+                        "documentData": result_data,
-                            "documentData": result_data,
+                        "mimeType": output_mime_type
-                            "mimeType": output_mime_type
+                    }
-                        }
+                ],
-                    ]
+                resultLabel="web_search_results"
-                }
             )
             
         except Exception as e:
             logger.error(f"Error searching web: {str(e)}")
-            return self._createResult(
+            return ActionResult(
                 success=False,
-                data={},
                 error=str(e)
             )
 
@@ -614,11 +610,7 @@ class MethodWeb(MethodBase):
             expectedDocumentFormats = parameters.get("expectedDocumentFormats", [])
             
             if not document:
-                return self._createResult(
+                return ActionResult.failure(error="No document with URL list provided.")
-                    success=False,
-                    data={},
-                    error="No document with URL list provided."
-                )
             
             # Read the document content
             with open(document, "r", encoding="utf-8") as f:
@@ -630,11 +622,7 @@ class MethodWeb(MethodBase):
             urls = [u.strip() for u in urls if u.strip()]
             
             if not urls:
-                return self._createResult(
+                return ActionResult.failure(error="No valid URLs provided in the document.")
-                    success=False,
-                    data={},
-                    error="No valid URLs provided in the document."
-                )
             
             crawl_results = []
             for url in urls:
@@ -701,24 +689,22 @@ class MethodWeb(MethodBase):
                 "timestamp": datetime.now(UTC).isoformat()
             }
             
-            return self._createResult(
+            return ActionResult(
                 success=True,
-                data={
+                documents=[
-                    "documents": [
+                    {
-                        {
+                        "documentName": f"web_crawl_{datetime.now(UTC).strftime('%Y%m%d_%H%M%S')}{output_extension}",
-                            "documentName": f"web_crawl_{datetime.now(UTC).strftime('%Y%m%d_%H%M%S')}{output_extension}",
+                        "documentData": result_data,
-                            "documentData": result_data,
+                        "mimeType": output_mime_type
-                            "mimeType": output_mime_type
+                    }
-                        }
+                ],
-                    ]
+                resultLabel="web_crawl_results"
-                }
             )
             
         except Exception as e:
             logger.error(f"Error crawling web pages: {str(e)}")
-            return self._createResult(
+            return ActionResult(
                 success=False,
-                data={},
                 error=str(e)
             )
     
@@ -740,18 +726,16 @@ class MethodWeb(MethodBase):
             expectedDocumentFormats = parameters.get("expectedDocumentFormats", [])
             
             if not url or not selectors:
-                return self._createResult(
+                return ActionResult(
                     success=False,
-                    data={},
                     error="URL and selectors are required"
                 )
             
             # Read the URL
             soup = self._readUrl(url)
             if not soup:
-                return self._createResult(
+                return ActionResult(
                     success=False,
-                    data={},
                     error="Failed to read URL"
                 )
             
@@ -810,24 +794,22 @@ class MethodWeb(MethodBase):
             else:
                 logger.info(f"No expected format specified, using format parameter: {format}")
             
-            return self._createResult(
+            return ActionResult(
                 success=True,
-                data={
+                documents=[
-                    "documents": [
+                    {
-                        {
+                        "documentName": f"web_scrape_{datetime.now(UTC).strftime('%Y%m%d_%H%M%S')}{output_extension}",
-                            "documentName": f"web_scrape_{datetime.now(UTC).strftime('%Y%m%d_%H%M%S')}{output_extension}",
+                        "documentData": result_data,
-                            "documentData": result_data,
+                        "mimeType": output_mime_type
-                            "mimeType": output_mime_type
+                    }
-                        }
+                ],
-                    ]
+                resultLabel="web_scrape_results"
-                }
             )
             
         except Exception as e:
             logger.error(f"Error scraping web page: {str(e)}")
-            return self._createResult(
+            return ActionResult(
                 success=False,
-                data={},
                 error=str(e)
             )
     

modules/routes/routeSecurityGoogle.py

@@ -339,6 +339,7 @@ async def auth_callback(code: str, state: str, request: Request) -> HTMLResponse
                 token = Token(
                     userId=user.id,  # Use local user's ID
                     authority=AuthAuthority.GOOGLE,
+                    connectionId=connection_id,  # Link token to this specific connection
                     tokenAccess=token_response["access_token"],
                     tokenRefresh=token_response.get("refresh_token", ""),
                     tokenType=token_response.get("token_type", "bearer"),
@@ -457,4 +458,87 @@ async def logout(
         raise HTTPException(
             status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
             detail=f"Failed to logout: {str(e)}"
+        )
+
+@router.post("/refresh")
+@limiter.limit("10/minute")
+async def refresh_token(
+    request: Request,
+    currentUser: User = Depends(getCurrentUser)
+) -> Dict[str, Any]:
+    """Refresh Google OAuth token for current user"""
+    try:
+        appInterface = getInterface(currentUser)
+        
+        # Find Google connection for this user
+        logger.debug(f"Looking for Google connection for user {currentUser.id}")
+        
+        connections = appInterface.getUserConnections(currentUser.id)
+        google_connection = None
+        
+        for conn in connections:
+            if conn.authority == AuthAuthority.GOOGLE:
+                google_connection = conn
+                break
+        
+        if not google_connection:
+            raise HTTPException(
+                status_code=status.HTTP_404_NOT_FOUND,
+                detail="No Google connection found for current user"
+            )
+        
+        logger.debug(f"Found Google connection: {google_connection.id}, status={google_connection.status}")
+        
+        # Get the token for this specific connection using the new method
+        current_token = appInterface.getTokenForConnection(google_connection.id, auto_refresh=False)
+        
+        if not current_token:
+            raise HTTPException(
+                status_code=status.HTTP_404_NOT_FOUND,
+                detail="No Google token found for this connection"
+            )
+        
+        logger.debug(f"Found Google token: expires={current_token.expiresAt}, refresh_token_exists={bool(current_token.tokenRefresh)}")
+        
+        # Always attempt refresh (as per your requirement)
+        from modules.security.tokenManager import TokenManager
+        token_manager = TokenManager()
+        
+        refreshed_token = token_manager.refresh_token(current_token)
+        if refreshed_token:
+            # Save the new token and delete the old one
+            appInterface.saveToken(refreshed_token)
+            appInterface.deleteTokenByConnectionId(google_connection.id)
+            
+            # Update the connection's expiration time
+            google_connection.expiresAt = datetime.fromtimestamp(refreshed_token.expiresAt)
+            google_connection.lastChecked = datetime.now()
+            google_connection.status = ConnectionStatus.ACTIVE
+            
+            # Save updated connection
+            appInterface.db.recordModify("connections", google_connection.id, google_connection.to_dict())
+            
+            # Calculate time until expiration
+            import time
+            current_time = time.time()
+            expires_in = int(refreshed_token.expiresAt - current_time)
+            
+            return {
+                "message": "Token refreshed successfully",
+                "expires_at": refreshed_token.expiresAt,
+                "expires_in_seconds": expires_in
+            }
+        else:
+            raise HTTPException(
+                status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
+                detail="Failed to refresh token"
+            )
+            
+    except HTTPException:
+        raise
+    except Exception as e:
+        logger.error(f"Error refreshing Google token: {str(e)}")
+        raise HTTPException(
+            status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
+            detail=f"Failed to refresh token: {str(e)}"
         ) 

modules/routes/routeSecurityMsft.py

@@ -318,16 +318,21 @@ async def auth_callback(code: str, state: str, request: Request) -> HTMLResponse
                 rootInterface.db.clearTableCache("connections")
                 
                 # Save token
+                logger.info(f"Creating token for connection {connection_id}")
                 token = Token(
                     userId=user.id,  # Use local user's ID
                     authority=AuthAuthority.MSFT,
+                    connectionId=connection_id,  # Link token to this specific connection
                     tokenAccess=token_response["access_token"],
                     tokenRefresh=token_response.get("refresh_token", ""),
                     tokenType=token_response.get("token_type", "bearer"),
                     expiresAt=datetime.now().timestamp() + token_response.get("expires_in", 0),
                     createdAt=datetime.now()
                 )
+                
+                logger.info(f"Saving token with connectionId: {token.connectionId}")
                 interface.saveToken(token)
+                logger.info(f"Token saved successfully for connection {connection_id}")
                 
                 # Return success page with connection data
                 return HTMLResponse(
@@ -358,7 +363,7 @@ async def auth_callback(code: str, state: str, request: Request) -> HTMLResponse
                     """
                 )
             except Exception as e:
-                logger.error(f"Error updating connection: {str(e)}", exc_info=True)
+                logger.error(f"Error updating connection or saving token: {str(e)}", exc_info=True)
                 return HTMLResponse(
                     content=f"""
                     <html>
@@ -368,7 +373,7 @@ async def auth_callback(code: str, state: str, request: Request) -> HTMLResponse
                                 if (window.opener) {{
                                     window.opener.postMessage({{
                                         type: 'msft_connection_error',
-                                        error: 'Failed to update connection: {str(e)}'
+                                        error: 'Failed to update connection or save token: {str(e)}'
                                     }}, '*');
                                     // Wait for message to be sent before closing
                                     setTimeout(() => window.close(), 1000);
@@ -440,3 +445,86 @@ async def logout(
             status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
             detail=f"Failed to logout: {str(e)}"
         )
+
+@router.post("/refresh")
+@limiter.limit("10/minute")
+async def refresh_token(
+    request: Request,
+    currentUser: User = Depends(getCurrentUser)
+) -> Dict[str, Any]:
+    """Refresh Microsoft OAuth token for current user"""
+    try:
+        appInterface = getInterface(currentUser)
+        
+        # Find Microsoft connection for this user
+        logger.debug(f"Looking for Microsoft connection for user {currentUser.id}")
+        
+        connections = appInterface.getUserConnections(currentUser.id)
+        msft_connection = None
+        
+        for conn in connections:
+            if conn.authority == AuthAuthority.MSFT:
+                msft_connection = conn
+                break
+        
+        if not msft_connection:
+            raise HTTPException(
+                status_code=status.HTTP_404_NOT_FOUND,
+                detail="No Microsoft connection found for current user"
+            )
+        
+        logger.debug(f"Found Microsoft connection: {msft_connection.id}, status={msft_connection.status}")
+        
+        # Get the token for this specific connection using the new method
+        current_token = appInterface.getTokenForConnection(msft_connection.id, auto_refresh=False)
+        
+        if not current_token:
+            raise HTTPException(
+                status_code=status.HTTP_404_NOT_FOUND,
+                detail="No Microsoft token found for this connection"
+            )
+        
+        logger.debug(f"Found Microsoft token: expires={current_token.expiresAt}, refresh_token_exists={bool(current_token.tokenRefresh)}")
+        
+        # Always attempt refresh (as per your requirement)
+        from modules.security.tokenManager import TokenManager
+        token_manager = TokenManager()
+        
+        refreshed_token = token_manager.refresh_token(current_token)
+        if refreshed_token:
+            # Save the new token and delete the old one
+            appInterface.saveToken(refreshed_token)
+            appInterface.deleteTokenByConnectionId(msft_connection.id)
+            
+            # Update the connection's expiration time
+            msft_connection.expiresAt = datetime.fromtimestamp(refreshed_token.expiresAt)
+            msft_connection.lastChecked = datetime.now()
+            msft_connection.status = ConnectionStatus.ACTIVE
+            
+            # Save updated connection
+            appInterface.db.recordModify("connections", msft_connection.id, msft_connection.to_dict())
+            
+            # Calculate time until expiration
+            import time
+            current_time = time.time()
+            expires_in = int(refreshed_token.expiresAt - current_time)
+            
+            return {
+                "message": "Token refreshed successfully",
+                "expires_at": refreshed_token.expiresAt,
+                "expires_in_seconds": expires_in
+            }
+        else:
+            raise HTTPException(
+                status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
+                detail="Failed to refresh token"
+            )
+            
+    except HTTPException:
+        raise
+    except Exception as e:
+        logger.error(f"Error refreshing Microsoft token: {str(e)}")
+        raise HTTPException(
+            status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
+            detail=f"Failed to refresh token: {str(e)}"
+        )

modules/security/tokenManager.py

@@ -0,0 +1,170 @@
+"""
+Token Manager Service
+Handles all token operations including automatic refresh for backend services.
+"""
+
+import logging
+import httpx
+from datetime import datetime
+from typing import Optional, Dict, Any
+
+from modules.interfaces.interfaceAppModel import Token, AuthAuthority
+from modules.shared.configuration import APP_CONFIG
+
+logger = logging.getLogger(__name__)
+
+class TokenManager:
+    """Centralized token management service"""
+    
+    def __init__(self):
+        # Microsoft OAuth configuration
+        self.msft_client_id = APP_CONFIG.get("Service_MSFT_CLIENT_ID")
+        self.msft_client_secret = APP_CONFIG.get("Service_MSFT_CLIENT_SECRET")
+        self.msft_tenant_id = APP_CONFIG.get("Service_MSFT_TENANT_ID", "common")
+        
+        # Google OAuth configuration
+        self.google_client_id = APP_CONFIG.get("Service_GOOGLE_CLIENT_ID")
+        self.google_client_secret = APP_CONFIG.get("Service_GOOGLE_CLIENT_SECRET")
+    
+    def refresh_microsoft_token(self, refresh_token: str, user_id: str, old_token: Token) -> Optional[Token]:
+        """Refresh Microsoft OAuth token using refresh token"""
+        try:
+            if not self.msft_client_id or not self.msft_client_secret:
+                logger.error("Microsoft OAuth configuration not found")
+                return None
+            
+            # Microsoft token refresh endpoint
+            token_url = f"https://login.microsoftonline.com/{self.msft_tenant_id}/oauth2/v2.0/token"
+            
+            # Prepare refresh request
+            data = {
+                "client_id": self.msft_client_id,
+                "client_secret": self.msft_client_secret,
+                "grant_type": "refresh_token",
+                "refresh_token": refresh_token,
+                "scope": "Mail.ReadWrite Mail.Send Mail.ReadWrite.Shared User.Read"
+            }
+            
+            # Make refresh request
+            with httpx.Client(timeout=30.0) as client:
+                response = client.post(token_url, data=data)
+                
+                if response.status_code == 200:
+                    token_data = response.json()
+                    
+                    # Create new token
+                    new_token = Token(
+                        userId=user_id,
+                        authority=AuthAuthority.MSFT,
+                        connectionId=old_token.connectionId,  # Preserve connection ID
+                        tokenAccess=token_data["access_token"],
+                        tokenRefresh=token_data.get("refresh_token", refresh_token),  # Keep old refresh token if new one not provided
+                        tokenType=token_data.get("token_type", "bearer"),
+                        expiresAt=datetime.now().timestamp() + token_data.get("expires_in", 3600),
+                        createdAt=datetime.now()
+                    )
+                    
+                    logger.info(f"Successfully refreshed Microsoft token for user {user_id}")
+                    return new_token
+                else:
+                    logger.error(f"Failed to refresh Microsoft token: {response.status_code} - {response.text}")
+                    return None
+                    
+        except Exception as e:
+            logger.error(f"Error refreshing Microsoft token: {str(e)}")
+            return None
+    
+    def refresh_google_token(self, refresh_token: str, user_id: str, old_token: Token) -> Optional[Token]:
+        """Refresh Google OAuth token using refresh token"""
+        try:
+            if not self.google_client_id or not self.google_client_secret:
+                logger.error("Google OAuth configuration not found")
+                return None
+            
+            # Google token refresh endpoint
+            token_url = "https://oauth2.googleapis.com/token"
+            
+            # Prepare refresh request
+            data = {
+                "client_id": self.google_client_id,
+                "client_secret": self.google_client_secret,
+                "grant_type": "refresh_token",
+                "refresh_token": refresh_token,
+                "scope": "https://www.googleapis.com/auth/gmail.readonly https://www.googleapis.com/auth/userinfo.profile https://www.googleapis.com/auth/userinfo.email openid"
+            }
+            
+            # Make refresh request
+            with httpx.Client(timeout=30.0) as client:
+                response = client.post(token_url, data=data)
+                
+                if response.status_code == 200:
+                    token_data = response.json()
+                    
+                    # Create new token
+                    new_token = Token(
+                        userId=user_id,
+                        authority=AuthAuthority.GOOGLE,
+                        connectionId=old_token.connectionId,  # Preserve connection ID
+                        tokenAccess=token_data["access_token"],
+                        tokenRefresh=refresh_token,  # Google doesn't always provide new refresh token
+                        tokenType=token_data.get("token_type", "bearer"),
+                        expiresAt=datetime.now().timestamp() + token_data.get("expires_in", 3600),
+                        createdAt=datetime.now()
+                    )
+                    
+                    logger.info(f"Successfully refreshed Google token for user {user_id}")
+                    return new_token
+                else:
+                    logger.error(f"Failed to refresh Google token: {response.status_code} - {response.text}")
+                    return None
+                    
+        except Exception as e:
+            logger.error(f"Error refreshing Google token: {str(e)}")
+            return None
+    
+    def refresh_token(self, old_token: Token) -> Optional[Token]:
+        """Refresh an expired token using the appropriate OAuth service"""
+        try:
+            if not old_token.tokenRefresh:
+                logger.warning(f"No refresh token available for {old_token.authority}")
+                return None
+            
+            # Route to appropriate refresh method
+            if old_token.authority == AuthAuthority.MSFT:
+                return self.refresh_microsoft_token(old_token.tokenRefresh, old_token.userId, old_token)
+            elif old_token.authority == AuthAuthority.GOOGLE:
+                return self.refresh_google_token(old_token.tokenRefresh, old_token.userId, old_token)
+            else:
+                logger.warning(f"Unknown authority for token refresh: {old_token.authority}")
+                return None
+                
+        except Exception as e:
+            logger.error(f"Error refreshing token: {str(e)}")
+            return None
+    
+    def is_token_expired(self, token: Token) -> bool:
+        """Check if a token is expired"""
+        if not token.expiresAt:
+            return False
+        return datetime.now().timestamp() > token.expiresAt
+    
+    def get_token_status(self, token: Token) -> Dict[str, Any]:
+        """Get comprehensive token status information"""
+        current_time = datetime.now().timestamp()
+        
+        if not token.expiresAt:
+            return {
+                "status": "valid",
+                "expires_at": None,
+                "expires_in_seconds": None,
+                "expires_soon": False
+            }
+        
+        expires_in = int(token.expiresAt - current_time)
+        
+        return {
+            "status": "expired" if expires_in <= 0 else "valid",
+            "expires_at": token.expiresAt,
+            "expires_in_seconds": expires_in,
+            "expires_soon": expires_in <= 3600  # 1 hour
+        }

notes/changelog.txt

@@ -1,12 +1,19 @@
 
 TODO
+
+- check history --> tasks?
+- model reference diagram for all models. who uses who? --> to see the basic building blocks
+
+- automatischer token refresh msft und google integrieren
+
+- check method outlook: alles
+- check method sharepoint: alles
+- check method webcrawler: alles
+- check method google: alles
+- check zusammenfassung von 10 dokumenten >10 MB
+- test case bewerbung
+
 - neutralizer to activate AND put back placeholders to the returned data
-- referenceHandling and authentication for connections in the method actions
-- check methods
-- test for workflow backend with userdata
-- prompt for task definition to fix
-- method definition list directly based on functions
-- neutralizer to put back placeholders to the returned data after ai
 
 
 

notes/readme.md

@@ -91,6 +91,7 @@ Für größere Installationen die JSON-basierte Datenbank ersetzen durch:
 git remote set-url origin https://valueon@github.com/valueonag/gateway
 git remote set-url origin https://valueon@github.com/valueonag/frontend_agents
 git remote set-url origin https://valueon@github.com/valueonag/wiki
+git remote set-url origin https://valueon@github.com/valueonag/customer-svbe
 
 ### git delete workflow runs (cleanup)