PowerOn/gateway
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

handovers

Browse source
This commit is contained in:
ValueOn AG 2025-07-08 01:14:27 +02:00
parent 1ba107b4fd
commit 8d8b800859
22 changed files with 2693 additions and 6000 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

277
ENHANCED_AI_AGENT_RECOMMENDATIONS.md
View file

@ -1,277 +0,0 @@
# Enhanced AI Agent System Recommendations
## Overview
This document provides comprehensive recommendations for building a stable, robust, and perfect AI agent system with clear handovers and optimal user request processing.
## 1. **Enhanced Error Recovery & Resilience**
### ✅ **Implemented Features:**
- **Circuit Breaker Pattern**: Prevents cascading failures when AI services are down
- **Exponential Backoff Retry**: Intelligent retry with increasing delays
- **Timeout Handling**: Prevents hanging operations
- **Fallback Mechanisms**: Graceful degradation when AI fails
- **Alternative Approach Generation**: Tries different methods when original fails
### 🔄 **Additional Recommendations:**
#### A. **State Persistence & Recovery**
```python
# Add checkpoint system for long-running workflows
class WorkflowCheckpoint:
def save_checkpoint(self, workflow_id: str, task_step: int, state: Dict):
# Save current state to database
pass
def restore_checkpoint(self, workflow_id: str) -> Dict:
# Restore from last checkpoint
pass
```
#### B. **Graceful Degradation**
```python
# Implement multiple AI providers with fallback
class MultiProviderAIService:
def __init__(self):
self.providers = [
OpenAIService(),
AnthropicService(),
LocalLLMService() # Fallback
]
async def call_with_fallback(self, prompt: str) -> str:
for provider in self.providers:
try:
return await provider.call(prompt)
except Exception:
continue
raise Exception("All AI providers failed")
```
## 2. **Intelligent Task Planning & Execution**
### ✅ **Current Implementation:**
- **Task Planning**: AI analyzes request and creates logical task steps
- **Handover Review**: Validates each step before proceeding
- **Dynamic Action Generation**: Creates actions based on current context
### 🔄 **Enhanced Recommendations:**
#### A. **Dependency Graph Management**
```python
class TaskDependencyGraph:
def __init__(self):
self.nodes = {} # task_id -> task_info
self.edges = {} # task_id -> [dependencies]
def add_task(self, task_id: str, dependencies: List[str]):
self.nodes[task_id] = {"status": "pending"}
self.edges[task_id] = dependencies
def get_ready_tasks(self) -> List[str]:
# Return tasks with all dependencies completed
pass
def detect_cycles(self) -> bool:
# Detect circular dependencies
pass
```
#### B. **Parallel Task Execution**
```python
async def execute_parallel_tasks(self, independent_tasks: List[Dict]) -> List[Dict]:
"""Execute independent tasks in parallel for better performance"""
tasks = []
for task_step in independent_tasks:
task = asyncio.create_task(self._executeTaskStep(task_step))
tasks.append(task)
results = await asyncio.gather(*tasks, return_exceptions=True)
return results
```
## 3. **Advanced Quality Assurance**
### 🔄 **Quality Metrics & Validation:**
#### A. **Multi-Dimensional Quality Assessment**
```python
class QualityAssessor:
def assess_quality(self, result: Dict, criteria: Dict) -> QualityScore:
return QualityScore(
completeness=self._assess_completeness(result, criteria),
accuracy=self._assess_accuracy(result, criteria),
relevance=self._assess_relevance(result, criteria),
coherence=self._assess_coherence(result, criteria)
)
```
#### B. **Continuous Learning & Improvement**
```python
class LearningSystem:
def record_execution(self, task: Dict, result: Dict, quality_score: float):
"""Record execution for learning"""
pass
def suggest_improvements(self, task_type: str) -> List[str]:
"""Suggest improvements based on historical data"""
pass
```
## 4. **Enhanced Document & Context Management**
### 🔄 **Smart Document Processing:**
#### A. **Document Understanding & Classification**
```python
class DocumentProcessor:
def classify_document(self, content: str) -> DocumentType:
"""Classify document type for better processing"""
pass
def extract_key_information(self, document: Document) -> Dict:
"""Extract key information for context"""
pass
```
#### B. **Context-Aware Processing**
```python
class ContextManager:
def __init__(self):
self.context_stack = []
self.document_cache = {}
def add_context(self, context: Dict):
"""Add context for current processing"""
self.context_stack.append(context)
def get_relevant_context(self, task: Dict) -> Dict:
"""Get relevant context for specific task"""
pass
```
## 5. **Advanced Handover Mechanisms**
### 🔄 **Intelligent Handover System:**
#### A. **Handover Validation Engine**
```python
class HandoverValidator:
def validate_handover(self, from_task: Dict, to_task: Dict, data: Dict) -> ValidationResult:
"""Validate data handover between tasks"""
return ValidationResult(
is_valid=self._check_data_completeness(data, to_task),
missing_data=self._identify_missing_data(data, to_task),
quality_issues=self._identify_quality_issues(data),
suggestions=self._generate_handover_suggestions(data, to_task)
)
```
## 6. **Monitoring & Observability**
### 🔄 **Comprehensive Monitoring:**
#### A. **Real-Time Metrics**
```python
class MetricsCollector:
def __init__(self):
self.metrics = {
'task_execution_time': [],
'ai_call_latency': [],
'success_rate': [],
'error_rate': [],
'quality_scores': []
}
def record_metric(self, metric_name: str, value: float):
"""Record metric for monitoring"""
pass
def get_health_score(self) -> float:
"""Calculate overall system health score"""
pass
```
## 7. **Security & Privacy**
### 🔄 **Enhanced Security Measures:**
#### A. **Data Sanitization**
```python
class DataSanitizer:
def sanitize_input(self, user_input: str) -> str:
"""Sanitize user input for security"""
pass
def validate_documents(self, documents: List[Document]) -> bool:
"""Validate documents for security risks"""
pass
```
## 8. **Performance Optimization**
### 🔄 **Performance Enhancements:**
#### A. **Caching Strategy**
```python
class CacheManager:
def __init__(self):
self.document_cache = {}
self.ai_response_cache = {}
self.task_result_cache = {}
def get_cached_result(self, key: str) -> Optional[Dict]:
"""Get cached result if available"""
pass
def cache_result(self, key: str, result: Dict, ttl: int = 3600):
"""Cache result with TTL"""
pass
```
## 9. **Testing & Validation**
### 🔄 **Comprehensive Testing:**
#### A. **Automated Testing Framework**
```python
class TestFramework:
def test_task_planning(self, scenarios: List[Dict]):
"""Test task planning with various scenarios"""
pass
def test_handover_validation(self, test_cases: List[Dict]):
"""Test handover validation logic"""
pass
```
## 10. **Implementation Priority**
### **Phase 1 (Critical - Implement First):**
1. ✅ Circuit Breaker Pattern
2. ✅ Retry Mechanisms
3. ✅ Fallback Systems
4. 🔄 Enhanced Error Handling
### **Phase 2 (Important - Implement Next):**
1. 🔄 Parallel Task Execution
2. 🔄 Advanced Quality Assessment
3. 🔄 Smart Document Processing
4. 🔄 Comprehensive Monitoring
### **Phase 3 (Enhancement - Future):**
1. 🔄 Learning & Optimization
2. 🔄 Advanced Security
3. 🔄 Performance Optimization
4. 🔄 Advanced Testing
## Conclusion
The enhanced AI agent system provides:
- **Robustness**: Multiple layers of error recovery and fallback mechanisms
- **Intelligence**: Smart task planning and dynamic action generation
- **Quality**: Comprehensive validation and quality assessment
- **Observability**: Full monitoring and alerting capabilities
- **Scalability**: Resource management and performance optimization
- **Security**: Data protection and access control
This system will process user requests in a near-perfect way with clear handovers, comprehensive error handling, and continuous improvement capabilities.

345
modules/methods/methodCoder.py
View file

@ -1,29 +1,98 @@
from typing import Dict, Any, Optional
import logging
import uuid
from datetime import datetime, UTC
from modules.workflow.methodBase import MethodBase, ActionResult, action
logger = logging.getLogger(__name__)
class CoderService:
"""Service for code analysis, generation, and refactoring operations"""
class MethodCoder(MethodBase):
"""Coder method implementation for code operations"""
def __init__(self, serviceContainer: Any):
self.serviceContainer = serviceContainer
super().__init__(serviceContainer)
self.name = "coder"
self.description = "Handle code operations like analysis and generation"
async def analyzeCode(self, code: str, language: str = "python", checks: list = None) -> Dict[str, Any]:
"""Analyze code quality and structure"""
if checks is None:
checks = ["complexity", "style", "security"]
@action
async def analyze(self, parameters: Dict[str, Any]) -> ActionResult:
"""
Analyze code quality and structure
Parameters:
documentList (str): Reference to the document list to analyze
aiPrompt (str): AI prompt for code analysis
language (str, optional): Programming language (default: "python")
checks (List[str], optional): Types of checks to perform (default: ["complexity", "style", "security"])
"""
try:
documentList = parameters.get("documentList")
aiPrompt = parameters.get("aiPrompt")
language = parameters.get("language", "python")
checks = parameters.get("checks", ["complexity", "style", "security"])
if not documentList:
return self._createResult(
success=False,
data={},
error="Document list reference is required"
)
if not aiPrompt:
return self._createResult(
success=False,
data={},
error="AI prompt is required"
)
chatDocuments = self.serviceContainer.getChatDocumentsFromDocumentReference(documentList)
if not chatDocuments:
return self._createResult(
success=False,
data={},
error="No documents found for the provided reference"
)
# Extract content from all documents
all_code_content = []
for chatDocument in chatDocuments:
fileId = chatDocument.fileId
code = self.serviceContainer.getFileData(fileId)
file_info = self.serviceContainer.getFileInfo(fileId)
if not code:
logger.warning(f"Code file is empty for fileId: {fileId}")
continue
# Use AI prompt to extract relevant code content
extracted_content = await self.serviceContainer.extractContentFromFileData(
prompt=aiPrompt,
fileData=code,
filename=file_info.get('name', 'code'),
mimeType=file_info.get('mimeType', 'text/plain'),
base64Encoded=False
)
all_code_content.append(extracted_content)
if not all_code_content:
return self._createResult(
success=False,
data={},
error="No code content could be extracted from any documents"
)
# Combine all code content for analysis
combined_code = "\n\n--- CODE SEPARATOR ---\n\n".join(all_code_content)
# Create analysis prompt
analysis_prompt = f"""
Analyze this {language} code for quality, structure, and potential issues.
Code to analyze:
{code}
{combined_code}
Please check for:
{', '.join(checks)}
@ -34,144 +103,27 @@ class CoderService:
3. Security considerations
4. Performance optimizations
5. Best practices compliance
6. Summary of findings across all documents
"""
# Use AI service for analysis
analysis_result = await self.serviceContainer.interfaceAiCalls.callAiTextAdvanced(analysis_prompt)
return {
# Create result data
result_data = {
"documentCount": len(chatDocuments),
"language": language,
"checks": checks,
"analysis": analysis_result,
"timestamp": datetime.now(UTC).isoformat()
}
except Exception as e:
logger.error(f"Error analyzing code: {str(e)}")
return {
"error": str(e),
"language": language,
"checks": checks
}
async def generateCode(self, requirements: str, language: str = "python", template: str = None) -> Dict[str, Any]:
"""Generate code based on requirements"""
try:
# Create generation prompt
generation_prompt = f"""
Generate {language} code based on the following requirements:
Requirements:
{requirements}
{f'Template to follow: {template}' if template else ''}
Please provide:
1. Complete, working code
2. Clear comments and documentation
3. Error handling where appropriate
4. Best practices implementation
"""
# Use AI service for code generation
generated_code = await self.serviceContainer.interfaceAiCalls.callAiTextAdvanced(generation_prompt)
return {
"language": language,
"requirements": requirements,
"code": generated_code,
"timestamp": datetime.now(UTC).isoformat()
}
except Exception as e:
logger.error(f"Error generating code: {str(e)}")
return {
"error": str(e),
"language": language,
"requirements": requirements
}
async def refactorCode(self, code: str, language: str = "python", improvements: list = None) -> Dict[str, Any]:
"""Refactor code for better quality"""
if improvements is None:
improvements = ["style", "complexity"]
try:
# Create refactoring prompt
refactor_prompt = f"""
Refactor this {language} code to improve:
{', '.join(improvements)}
Original code:
{code}
Please provide:
1. Refactored code with improvements
2. Explanation of changes made
3. Benefits of the refactoring
4. Any potential trade-offs
"""
# Use AI service for refactoring
refactored_code = await self.serviceContainer.interfaceAiCalls.callAiTextAdvanced(refactor_prompt)
return {
"language": language,
"improvements": improvements,
"original_code": code,
"refactored_code": refactored_code,
"timestamp": datetime.now(UTC).isoformat()
}
except Exception as e:
logger.error(f"Error refactoring code: {str(e)}")
return {
"error": str(e),
"language": language,
"improvements": improvements
}
class MethodCoder(MethodBase):
"""Coder method implementation for code operations"""
def __init__(self, serviceContainer: Any):
super().__init__(serviceContainer)
self.name = "coder"
self.description = "Handle code operations like analysis and generation"
self.coderService = CoderService(serviceContainer)
@action
async def analyze(self, parameters: Dict[str, Any]) -> ActionResult:
"""
Analyze code quality and structure
Parameters:
code (str): The code to analyze
language (str, optional): Programming language (default: "python")
checks (List[str], optional): Types of checks to perform (default: ["complexity", "style", "security"])
"""
try:
code = parameters.get("code")
language = parameters.get("language", "python")
checks = parameters.get("checks", ["complexity", "style", "security"])
if not code:
return self._createResult(
success=False,
data={},
error="Code is required"
)
# Analyze code
results = await self.coderService.analyzeCode(
code=code,
language=language,
checks=checks
)
return self._createResult(
success=True,
data=results
data={
"documentName": f"code_analysis_{datetime.now(UTC).strftime('%Y%m%d_%H%M%S')}.json",
"documentData": result_data
}
)
except Exception as e:
@ -204,16 +156,39 @@ class MethodCoder(MethodBase):
error="Requirements are required"
)
# Generate code
code = await self.coderService.generateCode(
requirements=requirements,
language=language,
template=template
)
# Create generation prompt
generation_prompt = f"""
Generate {language} code based on the following requirements:
Requirements:
{requirements}
{f'Template to follow: {template}' if template else ''}
Please provide:
1. Complete, working code
2. Clear comments and documentation
3. Error handling where appropriate
4. Best practices implementation
"""
# Use AI service for code generation
generated_code = await self.serviceContainer.interfaceAiCalls.callAiTextAdvanced(generation_prompt)
# Create result data
result_data = {
"language": language,
"requirements": requirements,
"code": generated_code,
"timestamp": datetime.now(UTC).isoformat()
}
return self._createResult(
success=True,
data=code
data={
"documentName": f"generated_code_{datetime.now(UTC).strftime('%Y%m%d_%H%M%S')}.{language}",
"documentData": result_data
}
)
except Exception as e:
@ -230,32 +205,96 @@ class MethodCoder(MethodBase):
Refactor code for better quality
Parameters:
code (str): The code to refactor
documentList (str): Reference to the document list to refactor
aiImprovementPrompt (str): AI prompt for code improvements
language (str, optional): Programming language (default: "python")
improvements (List[str], optional): Types of improvements to make (default: ["style", "complexity"])
"""
try:
code = parameters.get("code")
documentList = parameters.get("documentList")
aiImprovementPrompt = parameters.get("aiImprovementPrompt")
language = parameters.get("language", "python")
improvements = parameters.get("improvements", ["style", "complexity"])
if not code:
if not documentList:
return self._createResult(
success=False,
data={},
error="Code is required"
error="Document list reference is required"
)
# Refactor code
results = await self.coderService.refactorCode(
code=code,
language=language,
improvements=improvements
)
if not aiImprovementPrompt:
return self._createResult(
success=False,
data={},
error="AI improvement prompt is required"
)
chatDocuments = self.serviceContainer.getChatDocumentsFromDocumentReference(documentList)
if not chatDocuments:
return self._createResult(
success=False,
data={},
error="No documents found for the provided reference"
)
# Process each document individually
refactored_results = []
for chatDocument in chatDocuments:
fileId = chatDocument.fileId
code = self.serviceContainer.getFileData(fileId)
file_info = self.serviceContainer.getFileInfo(fileId)
if not code:
logger.warning(f"Code file is empty for fileId: {fileId}")
continue
# Create refactoring prompt for this specific document
refactor_prompt = f"""
Refactor this {language} code based on the following improvement requirements:
Improvement requirements:
{aiImprovementPrompt}
Original code:
{code}
Please provide:
1. Refactored code with improvements
2. Explanation of changes made
3. Benefits of the refactoring
4. Any potential trade-offs
"""
# Use AI service for refactoring
refactored_code = await self.serviceContainer.interfaceAiCalls.callAiTextAdvanced(refactor_prompt)
refactored_results.append({
"original_file": file_info.get('name', 'unknown'),
"original_code": code,
"refactored_code": refactored_code
})
if not refactored_results:
return self._createResult(
success=False,
data={},
error="No code could be refactored from any documents"
)
# Create result data
result_data = {
"documentCount": len(chatDocuments),
"language": language,
"refactored_results": refactored_results,
"timestamp": datetime.now(UTC).isoformat()
}
return self._createResult(
success=True,
data=results
data={
"documentName": f"refactored_code_{datetime.now(UTC).strftime('%Y%m%d_%H%M%S')}.{language}",
"documentData": result_data
}
)
except Exception as e:

429
modules/methods/methodDocument.py
View file

@ -5,171 +5,14 @@ Handles document operations using the document service.
import logging
from typing import Dict, Any, List, Optional
import uuid
from datetime import datetime, UTC
from modules.workflow.managerDocument import DocumentManager
from modules.workflow.methodBase import MethodBase, ActionResult, action
logger = logging.getLogger(__name__)
class DocumentService:
"""Service for document content extraction, analysis, and summarization"""
def __init__(self, serviceContainer: Any):
self.serviceContainer = serviceContainer
async def extractContent(self, fileId: str, format: str = "text", includeMetadata: bool = True) -> Dict[str, Any]:
"""Extract content from document using prompt-based extraction"""
try:
# Get file data
file_data = self.serviceContainer.getFileData(fileId)
file_info = self.serviceContainer.getFileInfo(fileId)
if not file_data:
return {
"error": "File not found or empty",
"fileId": fileId
}
# Create extraction prompt based on format
extraction_prompt = f"""
Extract and structure the content from this document.
File information:
- Name: {file_info.get('name', 'Unknown')}
- Type: {file_info.get('mimeType', 'Unknown')}
- Size: {len(file_data)} bytes
Please extract:
1. Main content and key information
2. Structured data if present (tables, lists, etc.)
3. Important facts and figures
4. Key insights and takeaways
Format the output as: {format}
Include metadata: {includeMetadata}
"""
# Use the new direct file data extraction method
extracted_content = await self.serviceContainer.extractContentFromFileData(
prompt=extraction_prompt,
fileData=file_data,
filename=file_info.get('name', 'document'),
mimeType=file_info.get('mimeType', 'application/octet-stream'),
base64Encoded=False
)
result = {
"fileId": fileId,
"format": format,
"content": extracted_content,
"fileInfo": file_info if includeMetadata else None
}
return result
except Exception as e:
logger.error(f"Error extracting content: {str(e)}")
return {
"error": str(e),
"fileId": fileId
}
async def analyzeContent(self, fileId: str, analysis: list = None) -> Dict[str, Any]:
"""Analyze document content for entities, topics, and sentiment"""
if analysis is None:
analysis = ["entities", "topics", "sentiment"]
try:
# First extract content
content_result = await self.extractContent(fileId, "text", True)
if "error" in content_result:
return content_result
content = content_result.get("content", "")
# Create analysis prompt
analysis_prompt = f"""
Analyze this document content for the following aspects:
{', '.join(analysis)}
Document content:
{content[:5000]} # Limit content length
Please provide a detailed analysis including:
1. Key entities (people, organizations, locations, dates)
2. Main topics and themes
3. Sentiment analysis (positive, negative, neutral)
4. Key insights and patterns
5. Important relationships between entities
6. Document structure and organization
"""
# Use AI service for analysis
analysis_result = await self.serviceContainer.interfaceAiCalls.callAiTextAdvanced(analysis_prompt)
return {
"fileId": fileId,
"analysis": analysis,
"results": analysis_result,
"content": content_result
}
except Exception as e:
logger.error(f"Error analyzing content: {str(e)}")
return {
"error": str(e),
"fileId": fileId,
"analysis": analysis
}
async def summarizeContent(self, fileId: str, maxLength: int = 200, format: str = "text") -> Dict[str, Any]:
"""Summarize document content"""
try:
# First extract content
content_result = await self.extractContent(fileId, "text", False)
if "error" in content_result:
return content_result
content = content_result.get("content", "")
# Create summarization prompt
summary_prompt = f"""
Create a comprehensive summary of this document content.
Document content:
{content[:8000]} # Limit content length
Requirements:
- Maximum length: {maxLength} words
- Format: {format}
- Include key points and main ideas
- Maintain accuracy and completeness
- Use clear, professional language
- Highlight important insights and conclusions
"""
# Use AI service for summarization
summary = await self.serviceContainer.interfaceAiCalls.callAiTextAdvanced(summary_prompt)
return {
"fileId": fileId,
"maxLength": maxLength,
"format": format,
"summary": summary,
"wordCount": len(summary.split()),
"originalContent": content_result
}
except Exception as e:
logger.error(f"Error summarizing content: {str(e)}")
return {
"error": str(e),
"fileId": fileId,
"maxLength": maxLength
}
class MethodDocument(MethodBase):
"""Document method implementation for document operations"""
@ -178,7 +21,6 @@ class MethodDocument(MethodBase):
super().__init__(serviceContainer)
self.name = "document"
self.description = "Handle document operations like extraction and analysis"
self.documentService = DocumentService(serviceContainer)
self.documentManager = DocumentManager(serviceContainer)
@action
@ -187,34 +29,89 @@ class MethodDocument(MethodBase):
Extract content from document
Parameters:
fileId (str): The ID of the document to extract content from
documentList (str): Reference to the document list to extract content from
aiPrompt (str): AI prompt for content extraction
format (str, optional): Output format (default: "text")
includeMetadata (bool, optional): Whether to include metadata (default: True)
"""
try:
fileId = parameters.get("fileId")
documentList = parameters.get("documentList")
aiPrompt = parameters.get("aiPrompt")
format = parameters.get("format", "text")
includeMetadata = parameters.get("includeMetadata", True)
if not fileId:
if not documentList:
return self._createResult(
success=False,
data={},
error="File ID is required"
error="Document list reference is required"
)
# Extract content
content = await self.documentService.extractContent(
fileId=fileId,
format=format,
includeMetadata=includeMetadata
)
if not aiPrompt:
return self._createResult(
success=False,
data={},
error="AI prompt is required"
)
chatDocuments = self.serviceContainer.getChatDocumentsFromDocumentReference(documentList)
if not chatDocuments:
return self._createResult(
success=False,
data={},
error="No documents found for the provided reference"
)
# Extract content from all documents
all_extracted_content = []
file_infos = []
for chatDocument in chatDocuments:
fileId = chatDocument.fileId
file_data = self.serviceContainer.getFileData(fileId)
file_info = self.serviceContainer.getFileInfo(fileId)
if not file_data:
logger.warning(f"File not found or empty for fileId: {fileId}")
continue
extracted_content = await self.serviceContainer.extractContentFromFileData(
prompt=aiPrompt,
fileData=file_data,
filename=file_info.get('name', 'document'),
mimeType=file_info.get('mimeType', 'application/octet-stream'),
base64Encoded=False
)
all_extracted_content.append(extracted_content)
if includeMetadata:
file_infos.append(file_info)
if not all_extracted_content:
return self._createResult(
success=False,
data={},
error="No content could be extracted from any documents"
)
# Combine all extracted content
combined_content = "\n\n--- DOCUMENT SEPARATOR ---\n\n".join(all_extracted_content)
result_data = {
"documentCount": len(chatDocuments),
"format": format,
"content": combined_content,
"fileInfos": file_infos if includeMetadata else None,
"timestamp": datetime.now(UTC).isoformat()
}
return self._createResult(
success=True,
data=content
data={
"documentName": f"extracted_content_{datetime.now(UTC).strftime('%Y%m%d_%H%M%S')}.txt",
"documentData": result_data
}
)
except Exception as e:
logger.error(f"Error extracting content: {str(e)}")
return self._createResult(
@ -229,31 +126,102 @@ class MethodDocument(MethodBase):
Analyze document content
Parameters:
fileId (str): The ID of the document to analyze
documentList (str): Reference to the document list to analyze
aiPrompt (str): AI prompt for content analysis
analysis (List[str], optional): Types of analysis to perform (default: ["entities", "topics", "sentiment"])
"""
try:
fileId = parameters.get("fileId")
documentList = parameters.get("documentList")
aiPrompt = parameters.get("aiPrompt")
analysis = parameters.get("analysis", ["entities", "topics", "sentiment"])
if not fileId:
if not documentList:
return self._createResult(
success=False,
data={},
error="File ID is required"
error="Document list reference is required"
)
# Analyze content
results = await self.documentService.analyzeContent(
fileId=fileId,
analysis=analysis
)
if not aiPrompt:
return self._createResult(
success=False,
data={},
error="AI prompt is required"
)
chatDocuments = self.serviceContainer.getChatDocumentsFromDocumentReference(documentList)
if not chatDocuments:
return self._createResult(
success=False,
data={},
error="No documents found for the provided reference"
)
# Extract content from all documents
all_extracted_content = []
for chatDocument in chatDocuments:
fileId = chatDocument.fileId
file_data = self.serviceContainer.getFileData(fileId)
file_info = self.serviceContainer.getFileInfo(fileId)
if not file_data:
logger.warning(f"File not found or empty for fileId: {fileId}")
continue
extracted_content = await self.serviceContainer.extractContentFromFileData(
prompt=aiPrompt,
fileData=file_data,
filename=file_info.get('name', 'document'),
mimeType=file_info.get('mimeType', 'application/octet-stream'),
base64Encoded=False
)
all_extracted_content.append(extracted_content)
if not all_extracted_content:
return self._createResult(
success=False,
data={},
error="No content could be extracted from any documents"
)
# Combine all extracted content for analysis
combined_content = "\n\n--- DOCUMENT SEPARATOR ---\n\n".join(all_extracted_content)
analysis_prompt = f"""
Analyze this document content for the following aspects:
{', '.join(analysis)}
Document content:
{combined_content[:8000]} # Limit content length
Please provide a detailed analysis including:
1. Key entities (people, organizations, locations, dates)
2. Main topics and themes
3. Sentiment analysis (positive, negative, neutral)
4. Key insights and patterns
5. Important relationships between entities
6. Document structure and organization
"""
analysis_result = await self.serviceContainer.interfaceAiCalls.callAiTextAdvanced(analysis_prompt)
result_data = {
"documentCount": len(chatDocuments),
"analysis": analysis,
"results": analysis_result,
"content": combined_content,
"timestamp": datetime.now(UTC).isoformat()
}
return self._createResult(
success=True,
data=results
data={
"documentName": f"document_analysis_{datetime.now(UTC).strftime('%Y%m%d_%H%M%S')}.json",
"documentData": result_data
}
)
except Exception as e:
logger.error(f"Error analyzing content: {str(e)}")
return self._createResult(
@ -268,34 +236,105 @@ class MethodDocument(MethodBase):
Summarize document content
Parameters:
fileId (str): The ID of the document to summarize
documentList (str): Reference to the document list to summarize
aiPrompt (str): AI prompt for content extraction
maxLength (int, optional): Maximum length of summary in words (default: 200)
format (str, optional): Output format (default: "text")
"""
try:
fileId = parameters.get("fileId")
documentList = parameters.get("documentList")
aiPrompt = parameters.get("aiPrompt")
maxLength = parameters.get("maxLength", 200)
format = parameters.get("format", "text")
if not fileId:
if not documentList:
return self._createResult(
success=False,
data={},
error="File ID is required"
error="Document list reference is required"
)
# Summarize content
summary = await self.documentService.summarizeContent(
fileId=fileId,
maxLength=maxLength,
format=format
)
if not aiPrompt:
return self._createResult(
success=False,
data={},
error="AI prompt is required"
)
chatDocuments = self.serviceContainer.getChatDocumentsFromDocumentReference(documentList)
if not chatDocuments:
return self._createResult(
success=False,
data={},
error="No documents found for the provided reference"
)
# Extract content from all documents
all_extracted_content = []
for chatDocument in chatDocuments:
fileId = chatDocument.fileId
file_data = self.serviceContainer.getFileData(fileId)
file_info = self.serviceContainer.getFileInfo(fileId)
if not file_data:
logger.warning(f"File not found or empty for fileId: {fileId}")
continue
extracted_content = await self.serviceContainer.extractContentFromFileData(
prompt=aiPrompt,
fileData=file_data,
filename=file_info.get('name', 'document'),
mimeType=file_info.get('mimeType', 'application/octet-stream'),
base64Encoded=False
)
all_extracted_content.append(extracted_content)
if not all_extracted_content:
return self._createResult(
success=False,
data={},
error="No content could be extracted from any documents"
)
# Combine all extracted content for summarization
combined_content = "\n\n--- DOCUMENT SEPARATOR ---\n\n".join(all_extracted_content)
summary_prompt = f"""
Create a comprehensive summary of this document content.
Document content:
{combined_content[:8000]} # Limit content length
Requirements:
- Maximum length: {maxLength} words
- Format: {format}
- Include key points and main ideas
- Maintain accuracy and completeness
- Use clear, professional language
- Highlight important insights and conclusions
"""
summary = await self.serviceContainer.interfaceAiCalls.callAiTextAdvanced(summary_prompt)
result_data = {
"documentCount": len(chatDocuments),
"maxLength": maxLength,
"format": format,
"summary": summary,
"wordCount": len(summary.split()),
"originalContent": combined_content,
"timestamp": datetime.now(UTC).isoformat()
}
return self._createResult(
success=True,
data=summary
data={
"documentName": f"document_summary_{datetime.now(UTC).strftime('%Y%m%d_%H%M%S')}.txt",
"documentData": result_data
}
)
except Exception as e:
logger.error(f"Error summarizing content: {str(e)}")
return self._createResult(

461
modules/methods/methodExcel.py
View file

@ -1,461 +0,0 @@
"""
Excel method module.
Handles Excel operations using the Excel service.
"""
import logging
from typing import Dict, Any, List, Optional
from datetime import datetime, UTC
import json
import base64
from modules.workflow.methodBase import MethodBase, ActionResult, action
logger = logging.getLogger(__name__)
class ExcelService:
"""Service for Microsoft Excel operations using Graph API"""
def __init__(self, serviceContainer: Any):
self.serviceContainer = serviceContainer
def _getMicrosoftConnection(self, connectionReference: str) -> Optional[Dict[str, Any]]:
"""Get Microsoft connection from connection reference"""
try:
userConnection = self.serviceContainer.getUserConnectionFromConnectionReference(connectionReference)
if not userConnection or userConnection.authority != "msft" or userConnection.status != "active":
return None
# Get the corresponding token for this user and authority
token = self.serviceContainer.interfaceApp.getToken(userConnection.authority)
if not token:
logger.warning(f"No token found for user {userConnection.userId} and authority {userConnection.authority}")
return None
return {
"id": userConnection.id,
"accessToken": token.tokenAccess,
"refreshToken": token.tokenRefresh,
"scopes": ["Mail.ReadWrite", "User.Read"] # Default Microsoft scopes
}
except Exception as e:
logger.error(f"Error getting Microsoft connection: {str(e)}")
return None
async def readFile(self, fileId: str, connectionReference: str, sheetName: str = "Sheet1", range: str = None) -> Dict[str, Any]:
"""Read data from Excel file using Microsoft Graph API"""
try:
connection = self._getMicrosoftConnection(connectionReference)
if not connection:
return {
"error": "No valid Microsoft connection found for the provided connection reference",
"fileId": fileId,
"connectionReference": connectionReference
}
# Get file data from service container
file_data = self.serviceContainer.getFileData(fileId)
file_info = self.serviceContainer.getFileInfo(fileId)
if not file_data:
return {
"error": "File not found or empty",
"fileId": fileId
}
# For now, simulate Excel reading with AI analysis
# In a real implementation, you would use Microsoft Graph API
excel_prompt = f"""
Analyze this Excel file data and extract structured information.
File: {file_info.get('name', 'Unknown')}
Sheet: {sheetName}
Range: {range or 'All data'}
File content (first 5000 characters):
{file_data.decode('utf-8', errors='ignore')[:5000] if isinstance(file_data, bytes) else str(file_data)[:5000]}
Please extract:
1. All data from the specified sheet and range
2. Column headers and data types
3. Key metrics and calculations
4. Any charts or visualizations described
5. Summary statistics
Return the data in a structured JSON format.
"""
# Use AI to analyze Excel content
analysis_result = await self.serviceContainer.interfaceAiCalls.callAiTextAdvanced(excel_prompt)
return {
"fileId": fileId,
"sheetName": sheetName,
"range": range,
"data": analysis_result,
"fileInfo": file_info,
"connection": {
"id": connection["id"],
"authority": "microsoft",
"reference": connectionReference
}
}
except Exception as e:
logger.error(f"Error reading Excel file: {str(e)}")
return {
"error": str(e),
"fileId": fileId
}
async def writeFile(self, fileId: str, connectionReference: str, sheetName: str, data: Any, range: str = None) -> Dict[str, Any]:
"""Write data to Excel file using Microsoft Graph API"""
try:
connection = self._getMicrosoftConnection(connectionReference)
if not connection:
return {
"error": "No valid Microsoft connection found for the provided connection reference",
"fileId": fileId,
"connectionReference": connectionReference
}
# For now, simulate Excel writing
# In a real implementation, you would use Microsoft Graph API
write_prompt = f"""
Prepare data for writing to Excel file.
File: {fileId}
Sheet: {sheetName}
Range: {range or 'Auto-detect'}
Data to write:
{json.dumps(data, indent=2)}
Please format this data appropriately for Excel and provide:
1. Structured data ready for Excel
2. Column headers and formatting
3. Any formulas or calculations needed
4. Data validation rules if applicable
"""
# Use AI to prepare Excel data
prepared_data = await self.serviceContainer.interfaceAiCalls.callAiTextAdvanced(write_prompt)
return {
"fileId": fileId,
"sheetName": sheetName,
"range": range,
"data": prepared_data,
"status": "prepared",
"connection": {
"id": connection["id"],
"authority": "microsoft",
"reference": connectionReference
}
}
except Exception as e:
logger.error(f"Error writing to Excel file: {str(e)}")
return {
"error": str(e),
"fileId": fileId
}
async def createFile(self, fileName: str, connectionReference: str, template: str = None) -> Dict[str, Any]:
"""Create new Excel file using Microsoft Graph API"""
try:
connection = self._getMicrosoftConnection(connectionReference)
if not connection:
return {
"error": "No valid Microsoft connection found for the provided connection reference",
"connectionReference": connectionReference
}
# For now, simulate file creation
# In a real implementation, you would use Microsoft Graph API
create_prompt = f"""
Create a new Excel file structure.
File name: {fileName}
Template: {template or 'Standard'}
Please provide:
1. Initial sheet structure
2. Default column headers
3. Sample data if template specified
4. Formatting guidelines
"""
# Use AI to create Excel structure
file_structure = await self.serviceContainer.interfaceAiCalls.callAiTextAdvanced(create_prompt)
# Create file using service container
file_id = self.serviceContainer.createFile(
fileName=fileName,
mimeType="application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
content=file_structure,
base64encoded=False
)
return {
"fileId": file_id,
"fileName": fileName,
"template": template,
"structure": file_structure,
"connection": {
"id": connection["id"],
"authority": "microsoft",
"reference": connectionReference
}
}
except Exception as e:
logger.error(f"Error creating Excel file: {str(e)}")
return {
"error": str(e)
}
async def formatCells(self, fileId: str, connectionReference: str, sheetName: str, range: str, format: Dict[str, Any]) -> Dict[str, Any]:
"""Format Excel cells using Microsoft Graph API"""
try:
connection = self._getMicrosoftConnection(connectionReference)
if not connection:
return {
"error": "No valid Microsoft connection found for the provided connection reference",
"fileId": fileId,
"connectionReference": connectionReference
}
# For now, simulate formatting
# In a real implementation, you would use Microsoft Graph API
format_prompt = f"""
Apply formatting to Excel cells.
File: {fileId}
Sheet: {sheetName}
Range: {range}
Format: {json.dumps(format, indent=2)}
Please provide:
1. Applied formatting details
2. Visual representation of the formatting
3. Any conditional formatting rules
"""
# Use AI to describe formatting
formatting_result = await self.serviceContainer.interfaceAiCalls.callAiTextAdvanced(format_prompt)
return {
"fileId": fileId,
"sheetName": sheetName,
"range": range,
"format": format,
"result": formatting_result,
"connection": {
"id": connection["id"],
"authority": "microsoft",
"reference": connectionReference
}
}
except Exception as e:
logger.error(f"Error formatting Excel cells: {str(e)}")
return {
"error": str(e),
"fileId": fileId
}
class MethodExcel(MethodBase):
"""Excel method implementation for spreadsheet operations"""
def __init__(self, serviceContainer: Any):
"""Initialize the Excel method"""
super().__init__(serviceContainer)
self.name = "excel"
self.description = "Handle Excel spreadsheet operations like reading and writing data"
self.excelService = ExcelService(serviceContainer)
@action
async def read(self, parameters: Dict[str, Any]) -> ActionResult:
"""
Read data from Excel file
Parameters:
fileId (str): The ID of the Excel file to read
connectionReference (str): Reference to the Microsoft connection
sheetName (str, optional): Name of the sheet to read (default: "Sheet1")
range (str, optional): Excel range to read (e.g., "A1:D10")
includeHeaders (bool, optional): Whether to include column headers (default: True)
"""
try:
fileId = parameters.get("fileId")
connectionReference = parameters.get("connectionReference")
sheetName = parameters.get("sheetName", "Sheet1")
range = parameters.get("range")
includeHeaders = parameters.get("includeHeaders", True)
if not fileId or not connectionReference:
return self._createResult(
success=False,
data={},
error="File ID and connection reference are required"
)
# Read data from Excel
data = await self.excelService.readFile(
fileId=fileId,
connectionReference=connectionReference,
sheetName=sheetName,
range=range
)
return self._createResult(
success=True,
data=data
)
except Exception as e:
logger.error(f"Error reading Excel file: {str(e)}")
return self._createResult(
success=False,
data={},
error=str(e)
)
@action
async def write(self, parameters: Dict[str, Any]) -> ActionResult:
"""
Write data to Excel file
Parameters:
fileId (str): The ID of the Excel file to write to
connectionReference (str): Reference to the Microsoft connection
sheetName (str, optional): Name of the sheet to write to (default: "Sheet1")
data (Any): Data to write to the Excel file
range (str, optional): Excel range to write to (e.g., "A1:D10")
"""
try:
fileId = parameters.get("fileId")
connectionReference = parameters.get("connectionReference")
sheetName = parameters.get("sheetName", "Sheet1")
data = parameters.get("data")
range = parameters.get("range")
if not fileId or not connectionReference or not data:
return self._createResult(
success=False,
data={},
error="File ID, connection reference, and data are required"
)
# Write data to Excel
result = await self.excelService.writeFile(
fileId=fileId,
connectionReference=connectionReference,
sheetName=sheetName,
data=data,
range=range
)
return self._createResult(
success=True,
data=result
)
except Exception as e:
logger.error(f"Error writing to Excel file: {str(e)}")
return self._createResult(
success=False,
data={},
error=str(e)
)
@action
async def create(self, parameters: Dict[str, Any]) -> ActionResult:
"""
Create new Excel file
Parameters:
fileName (str): Name of the new Excel file
connectionReference (str): Reference to the Microsoft connection
template (str, optional): Template to use for the new file
"""
try:
fileName = parameters.get("fileName")
connectionReference = parameters.get("connectionReference")
template = parameters.get("template")
if not fileName or not connectionReference:
return self._createResult(
success=False,
data={},
error="File name and connection reference are required"
)
# Create Excel file
fileId = await self.excelService.createFile(
fileName=fileName,
connectionReference=connectionReference,
template=template
)
return self._createResult(
success=True,
data={"fileId": fileId}
)
except Exception as e:
logger.error(f"Error creating Excel file: {str(e)}")
return self._createResult(
success=False,
data={},
error=str(e)
)
@action
async def format(self, parameters: Dict[str, Any]) -> ActionResult:
"""
Format Excel cells
Parameters:
fileId (str): The ID of the Excel file to format
connectionReference (str): Reference to the Microsoft connection
sheetName (str, optional): Name of the sheet to format (default: "Sheet1")
range (str): Excel range to format (e.g., "A1:D10")
format (Dict[str, Any]): Formatting options (e.g., {"font": {"bold": True}})
"""
try:
fileId = parameters.get("fileId")
connectionReference = parameters.get("connectionReference")
sheetName = parameters.get("sheetName", "Sheet1")
range = parameters.get("range")
format = parameters.get("format")
if not fileId or not connectionReference or not range or not format:
return self._createResult(
success=False,
data={},
error="File ID, connection reference, range, and format are required"
)
# Apply formatting
result = await self.excelService.formatCells(
fileId=fileId,
connectionReference=connectionReference,
sheetName=sheetName,
range=range,
format=format
)
return self._createResult(
success=True,
data=result
)
except Exception as e:
logger.error(f"Error formatting Excel cells: {str(e)}")
return self._createResult(
success=False,
data={},
error=str(e)
)

443
modules/methods/methodOperator.py
View file

@ -3,184 +3,119 @@
from typing import Dict, List, Any, Optional
from datetime import datetime, UTC
import logging
import uuid
from modules.workflow.methodBase import MethodBase, ActionResult, action
logger = logging.getLogger(__name__)
class OperatorService:
"""Service for operator operations like forEach and AI calls"""
def __init__(self, serviceContainer: Any):
self.serviceContainer = serviceContainer
async def executeForEach(self, items: List[Any], action: Dict[str, Any]) -> List[Any]:
"""Execute an action for each item in a list"""
try:
results = []
for i, item in enumerate(items):
logger.info(f"Executing forEach action {i+1}/{len(items)}")
# Create context with current item
context = {
"item": item,
"index": i,
"total": len(items),
"isFirst": i == 0,
"isLast": i == len(items) - 1
}
# Execute the action using the service container
if "method" in action and "action" in action:
methodName = action["method"]
actionName = action["action"]
parameters = action.get("parameters", {})
# Add context to parameters
parameters["context"] = context
parameters["currentItem"] = item
# Execute the method action
result = await self.serviceContainer.executeAction(
methodName=methodName,
actionName=actionName,
parameters=parameters
)
# Return the exact result data, not wrapped
if result.success:
results.append(result.data)
else:
results.append({"error": result.error})
else:
# Simple action without method call
results.append({"error": "No method specified"})
return results
except Exception as e:
logger.error(f"Error executing forEach: {str(e)}")
return [{"error": str(e)}] * len(items) if items else []
async def executeAiCall(self, prompt: str, documents: List[Dict[str, Any]] = None) -> Dict[str, Any]:
"""Call AI service with document content"""
try:
# Prepare context from documents
context = ""
extractedDocuments = []
if documents:
for i, doc in enumerate(documents):
documentReference = doc.get('documentReference')
contentExtractionPrompt = doc.get('contentExtractionPrompt', 'Extract the main content from this document')
if documentReference:
# Get documents from reference
chatDocuments = self.serviceContainer.getChatDocumentsFromDocumentReference(documentReference)
if chatDocuments:
# Extract content from each document
for j, chatDoc in enumerate(chatDocuments):
try:
# Extract content using the document manager
extractedContent = await self.serviceContainer.documentManager.extractContentFromChatDocument(
chatDocument=chatDoc,
extractionPrompt=contentExtractionPrompt
)
extractedDocuments.append({
"documentReference": documentReference,
"documentId": chatDoc.id,
"extractionPrompt": contentExtractionPrompt,
"extractedContent": extractedContent
})
# Add to context
context += f"\n\nDocument {len(extractedDocuments)} (from {documentReference}):\n{extractedContent}"
except Exception as e:
logger.warning(f"Error extracting content from document {chatDoc.id}: {str(e)}")
extractedDocuments.append({
"documentReference": documentReference,
"documentId": chatDoc.id,
"extractionPrompt": contentExtractionPrompt,
"extractedContent": f"Error extracting content: {str(e)}"
})
else:
logger.warning(f"No documents found for reference: {documentReference}")
extractedDocuments.append({
"documentReference": documentReference,
"extractionPrompt": contentExtractionPrompt,
"extractedContent": f"No documents found for reference: {documentReference}"
})
# Create full prompt with context
fullPrompt = f"{prompt}\n\nContext:\n{context}" if context else prompt
# Call AI service
aiResponse = await self.serviceContainer.interfaceAiCalls.callAiTextAdvanced(fullPrompt)
return {
"prompt": prompt,
"documentsProcessed": len(extractedDocuments),
"extractedDocuments": extractedDocuments,
"response": aiResponse,
"timestamp": datetime.now(UTC).isoformat()
}
except Exception as e:
logger.error(f"Error executing AI call: {str(e)}")
return {
"error": str(e),
"prompt": prompt,
"documentsProcessed": 0,
"extractedDocuments": [],
"response": None
}
class MethodOperator(MethodBase):
"""Operator method implementation for handling collections and AI operations"""
"""Operator method implementation for data operations"""
def __init__(self, serviceContainer: Any):
super().__init__(serviceContainer)
self.name = "operator"
self.description = "Handle operations like forEach and AI calls"
self.operatorService = OperatorService(serviceContainer)
self.description = "Handle data operations like filtering, sorting, and transformation"
@action
async def forEach(self, parameters: Dict[str, Any]) -> ActionResult:
async def filter(self, parameters: Dict[str, Any]) -> ActionResult:
"""
Execute an action for each item in a list
Filter data based on criteria
Parameters:
items (List[Any]): List of items to process
action (Dict[str, Any]): Action to execute for each item (contains method, action, parameters)
documentList (str): Reference to the document list to filter
criteria (Dict[str, Any]): Filter criteria
field (str, optional): Field to filter on
"""
try:
items = parameters.get("items", [])
action = parameters.get("action", {})
documentList = parameters.get("documentList")
criteria = parameters.get("criteria")
field = parameters.get("field")
if not items or not action:
if not documentList or not criteria:
return self._createResult(
success=False,
data={},
error="Items and action are required"
error="Document list reference and criteria are required"
)
# Execute forEach operation
results = await self.operatorService.executeForEach(
items=items,
action=action
)
chatDocuments = self.serviceContainer.getChatDocumentsFromDocumentReference(documentList)
if not chatDocuments:
return self._createResult(
success=False,
data={},
error="No documents found for the provided reference"
)
# Extract content from all documents
all_document_content = []
for chatDocument in chatDocuments:
fileId = chatDocument.fileId
file_data = self.serviceContainer.getFileData(fileId)
file_info = self.serviceContainer.getFileInfo(fileId)
if not file_data:
logger.warning(f"File data not found for fileId: {fileId}")
continue
all_document_content.append({
"fileId": fileId,
"fileName": file_info.get('name', 'unknown'),
"content": file_data
})
if not all_document_content:
return self._createResult(
success=False,
data={},
error="No content could be extracted from any documents"
)
# Combine all document content for filtering
combined_content = "\n\n--- DOCUMENT SEPARATOR ---\n\n".join([
f"File: {doc['fileName']}\nContent: {doc['content']}"
for doc in all_document_content
])
filter_prompt = f"""
Filter the following data based on the specified criteria.
Data to filter:
{combined_content}
Filter criteria:
{criteria}
Field to filter on: {field or 'All fields'}
Please provide:
1. Filtered data that matches the criteria
2. Summary of filtering results
3. Number of items before and after filtering
4. Any data quality insights
"""
filtered_result = await self.serviceContainer.interfaceAiCalls.callAiTextAdvanced(filter_prompt)
result_data = {
"documentCount": len(chatDocuments),
"criteria": criteria,
"field": field,
"filteredData": filtered_result,
"originalCount": len(all_document_content),
"timestamp": datetime.now(UTC).isoformat()
}
return self._createResult(
success=True,
data=results
data={
"documentName": f"filtered_data_{datetime.now(UTC).strftime('%Y%m%d_%H%M%S')}.json",
"documentData": result_data
}
)
except Exception as e:
logger.error(f"Error in forEach execution: {str(e)}")
logger.error(f"Error filtering data: {str(e)}")
return self._createResult(
success=False,
data={},
@ -188,39 +123,215 @@ class MethodOperator(MethodBase):
)
@action
async def aiCall(self, parameters: Dict[str, Any]) -> ActionResult:
async def sort(self, parameters: Dict[str, Any]) -> ActionResult:
"""
Call AI service with document content
Sort data by specified field
Parameters:
prompt (str): The prompt to send to the AI service
documents (List[Dict[str, Any]], optional): List of documents to include in context
Each document should have: documentReference (str), contentExtractionPrompt (str, optional)
documentList (str): Reference to the document list to sort
field (str): Field to sort by
order (str, optional): Sort order (asc/desc, default: "asc")
"""
try:
prompt = parameters.get("prompt")
documents = parameters.get("documents", []) # List of {documentReference, contentExtractionPrompt}
documentList = parameters.get("documentList")
field = parameters.get("field")
order = parameters.get("order", "asc")
if not prompt:
if not documentList or not field:
return self._createResult(
success=False,
data={},
error="Prompt is required"
error="Document list reference and field are required"
)
# Execute AI call
result = await self.operatorService.executeAiCall(
prompt=prompt,
documents=documents
)
# Get documents from reference
chatDocuments = self.serviceContainer.getChatDocumentsFromDocumentReference(documentList)
if not chatDocuments:
return self._createResult(
success=False,
data={},
error="No documents found for the provided reference"
)
# Extract content from all documents
all_document_content = []
for chatDocument in chatDocuments:
fileId = chatDocument.fileId
file_data = self.serviceContainer.getFileData(fileId)
file_info = self.serviceContainer.getFileInfo(fileId)
if not file_data:
logger.warning(f"File data not found for fileId: {fileId}")
continue
all_document_content.append({
"fileId": fileId,
"fileName": file_info.get('name', 'unknown'),
"content": file_data
})
if not all_document_content:
return self._createResult(
success=False,
data={},
error="No content could be extracted from any documents"
)
# Combine all document content for sorting
combined_content = "\n\n--- DOCUMENT SEPARATOR ---\n\n".join([
f"File: {doc['fileName']}\nContent: {doc['content']}"
for doc in all_document_content
])
# Create sorting prompt
sort_prompt = f"""
Sort the following data by the specified field.
Data to sort:
{combined_content}
Sort field: {field}
Sort order: {order}
Please provide:
1. Sorted data in the specified order
2. Summary of sorting results
3. Any data insights from the sorting
4. Validation of sort field existence
"""
# Use AI to perform sorting
sorted_result = await self.serviceContainer.interfaceAiCalls.callAiTextAdvanced(sort_prompt)
# Create result data
result_data = {
"documentCount": len(chatDocuments),
"field": field,
"order": order,
"sortedData": sorted_result,
"timestamp": datetime.now(UTC).isoformat()
}
return self._createResult(
success=True,
data=result
data={
"documentName": f"sorted_data_{datetime.now(UTC).strftime('%Y%m%d_%H%M%S')}.json",
"documentData": result_data
}
)
except Exception as e:
logger.error(f"Error in AI call execution: {str(e)}")
logger.error(f"Error sorting data: {str(e)}")
return self._createResult(
success=False,
data={},
error=str(e)
)
@action
async def transform(self, parameters: Dict[str, Any]) -> ActionResult:
"""
Transform data structure or format
Parameters:
documentList (str): Reference to the document list to transform
transformation (Dict[str, Any]): Transformation rules
outputFormat (str, optional): Desired output format
"""
try:
documentList = parameters.get("documentList")
transformation = parameters.get("transformation")
outputFormat = parameters.get("outputFormat", "json")
if not documentList or not transformation:
return self._createResult(
success=False,
data={},
error="Document list reference and transformation rules are required"
)
# Get documents from reference
chatDocuments = self.serviceContainer.getChatDocumentsFromDocumentReference(documentList)
if not chatDocuments:
return self._createResult(
success=False,
data={},
error="No documents found for the provided reference"
)
# Extract content from all documents
all_document_content = []
for chatDocument in chatDocuments:
fileId = chatDocument.fileId
file_data = self.serviceContainer.getFileData(fileId)
file_info = self.serviceContainer.getFileInfo(fileId)
if not file_data:
logger.warning(f"File data not found for fileId: {fileId}")
continue
all_document_content.append({
"fileId": fileId,
"fileName": file_info.get('name', 'unknown'),
"content": file_data
})
if not all_document_content:
return self._createResult(
success=False,
data={},
error="No content could be extracted from any documents"
)
# Combine all document content for transformation
combined_content = "\n\n--- DOCUMENT SEPARATOR ---\n\n".join([
f"File: {doc['fileName']}\nContent: {doc['content']}"
for doc in all_document_content
])
# Create transformation prompt
transform_prompt = f"""
Transform the following data according to the specified rules.
Data to transform:
{combined_content}
Transformation rules:
{transformation}
Output format: {outputFormat}
Please provide:
1. Transformed data in the specified format
2. Summary of transformation results
3. Validation of transformation rules
4. Any data quality improvements
"""
# Use AI to perform transformation
transformed_result = await self.serviceContainer.interfaceAiCalls.callAiTextAdvanced(transform_prompt)
# Create result data
result_data = {
"documentCount": len(chatDocuments),
"transformation": transformation,
"outputFormat": outputFormat,
"transformedData": transformed_result,
"timestamp": datetime.now(UTC).isoformat()
}
return self._createResult(
success=True,
data={
"documentName": f"transformed_data_{datetime.now(UTC).strftime('%Y%m%d_%H%M%S')}.{outputFormat}",
"documentData": result_data
}
)
except Exception as e:
logger.error(f"Error transforming data: {str(e)}")
return self._createResult(
success=False,
data={},

480
modules/methods/methodOutlook.py
View file

@ -7,16 +7,20 @@ import logging
from typing import Dict, Any, List, Optional
from datetime import datetime, UTC
import json
import uuid
from modules.workflow.methodBase import MethodBase, ActionResult, action
logger = logging.getLogger(__name__)
class OutlookService:
"""Service for Microsoft Outlook operations using Graph API"""
class MethodOutlook(MethodBase):
"""Outlook method implementation for email operations"""
def __init__(self, serviceContainer: Any):
self.serviceContainer = serviceContainer
"""Initialize the Outlook method"""
super().__init__(serviceContainer)
self.name = "outlook"
self.description = "Handle Microsoft Outlook email operations"
def _getMicrosoftConnection(self, connectionReference: str) -> Optional[Dict[str, Any]]:
"""Get Microsoft connection from connection reference"""
@ -41,226 +45,22 @@ class OutlookService:
logger.error(f"Error getting Microsoft connection: {str(e)}")
return None
async def readMails(self, connectionReference: str, folder: str = "inbox", query: str = None, maxResults: int = 10, includeAttachments: bool = False) -> Dict[str, Any]:
"""Read emails from Outlook using Microsoft Graph API"""
try:
connection = self._getMicrosoftConnection(connectionReference)
if not connection:
return {
"error": "No valid Microsoft connection found for the provided connection reference",
"connectionReference": connectionReference
}
# For now, simulate email reading
# In a real implementation, you would use Microsoft Graph API
mail_prompt = f"""
Read emails from Outlook.
Folder: {folder}
Query: {query or 'All emails'}
Max Results: {maxResults}
Include Attachments: {includeAttachments}
Please provide:
1. Email messages with subject, sender, and content
2. Timestamps and priority levels
3. Attachment information if requested
4. Email threading and conversations
5. Categorization and flags
"""
# Use AI to simulate email data
mail_data = await self.serviceContainer.interfaceAiCalls.callAiTextAdvanced(mail_prompt)
return {
"folder": folder,
"query": query,
"maxResults": maxResults,
"includeAttachments": includeAttachments,
"messages": mail_data,
"connection": {
"id": connection["id"],
"authority": "microsoft",
"reference": connectionReference
}
}
except Exception as e:
logger.error(f"Error reading emails: {str(e)}")
return {
"error": str(e)
}
async def sendMail(self, connectionReference: str, to: List[str], subject: str, body: str, attachments: List[str] = None) -> Dict[str, Any]:
"""Send email using Outlook using Microsoft Graph API"""
try:
connection = self._getMicrosoftConnection(connectionReference)
if not connection:
return {
"error": "No valid Microsoft connection found for the provided connection reference",
"connectionReference": connectionReference
}
# For now, simulate email sending
# In a real implementation, you would use Microsoft Graph API
send_prompt = f"""
Send email using Outlook.
To: {', '.join(to)}
Subject: {subject}
Body: {body}
Attachments: {attachments or 'None'}
Please provide:
1. Email composition details
2. Recipient validation
3. Attachment processing
4. Send confirmation
5. Message tracking information
"""
# Use AI to simulate email sending
send_result = await self.serviceContainer.interfaceAiCalls.callAiTextAdvanced(send_prompt)
return {
"to": to,
"subject": subject,
"body": body,
"attachments": attachments,
"result": send_result,
"connection": {
"id": connection["id"],
"authority": "microsoft",
"reference": connectionReference
}
}
except Exception as e:
logger.error(f"Error sending email: {str(e)}")
return {
"error": str(e)
}
async def createFolder(self, connectionReference: str, name: str, parentFolderId: str = None) -> Dict[str, Any]:
"""Create folder in Outlook using Microsoft Graph API"""
try:
connection = self._getMicrosoftConnection(connectionReference)
if not connection:
return {
"error": "No valid Microsoft connection found for the provided connection reference",
"connectionReference": connectionReference
}
# For now, simulate folder creation
# In a real implementation, you would use Microsoft Graph API
folder_prompt = f"""
Create folder in Outlook.
Name: {name}
Parent Folder ID: {parentFolderId or 'Root'}
Please provide:
1. Folder creation details
2. Permission settings
3. Folder structure and hierarchy
4. Creation confirmation
5. Folder properties and metadata
"""
# Use AI to simulate folder creation
folder_result = await self.serviceContainer.interfaceAiCalls.callAiTextAdvanced(folder_prompt)
return {
"name": name,
"parentFolderId": parentFolderId,
"result": folder_result,
"connection": {
"id": connection["id"],
"authority": "microsoft",
"reference": connectionReference
}
}
except Exception as e:
logger.error(f"Error creating folder: {str(e)}")
return {
"error": str(e)
}
async def moveMail(self, connectionReference: str, messageId: str, targetFolderId: str) -> Dict[str, Any]:
"""Move email to different folder using Microsoft Graph API"""
try:
connection = self._getMicrosoftConnection(connectionReference)
if not connection:
return {
"error": "No valid Microsoft connection found for the provided connection reference",
"connectionReference": connectionReference
}
# For now, simulate mail moving
# In a real implementation, you would use Microsoft Graph API
move_prompt = f"""
Move email to different folder.
Message ID: {messageId}
Target Folder ID: {targetFolderId}
Please provide:
1. Move operation details
2. Source and destination folder information
3. Message preservation and metadata
4. Move confirmation
5. Updated folder structure
"""
# Use AI to simulate mail moving
move_result = await self.serviceContainer.interfaceAiCalls.callAiTextAdvanced(move_prompt)
return {
"messageId": messageId,
"targetFolderId": targetFolderId,
"result": move_result,
"connection": {
"id": connection["id"],
"authority": "microsoft",
"reference": connectionReference
}
}
except Exception as e:
logger.error(f"Error moving email: {str(e)}")
return {
"error": str(e)
}
class MethodOutlook(MethodBase):
"""Outlook method implementation for email operations"""
def __init__(self, serviceContainer: Any):
"""Initialize the Outlook method"""
super().__init__(serviceContainer)
self.name = "outlook"
self.description = "Handle Outlook email operations like reading and sending emails"
self.outlookService = OutlookService(serviceContainer)
@action
async def readMails(self, parameters: Dict[str, Any]) -> ActionResult:
async def readEmails(self, parameters: Dict[str, Any]) -> ActionResult:
"""
Read emails from Outlook
Parameters:
connectionReference (str): Reference to the Microsoft connection
folder (str, optional): Folder to read from (default: "inbox")
query (str, optional): Search query to filter emails
maxResults (int, optional): Maximum number of results (default: 10)
includeAttachments (bool, optional): Whether to include attachments (default: False)
folder (str, optional): Email folder to read from (default: "Inbox")
limit (int, optional): Maximum number of emails to read (default: 10)
filter (str, optional): Filter criteria for emails
"""
try:
connectionReference = parameters.get("connectionReference")
folder = parameters.get("folder", "inbox")
query = parameters.get("query")
maxResults = parameters.get("maxResults", 10)
includeAttachments = parameters.get("includeAttachments", False)
folder = parameters.get("folder", "Inbox")
limit = parameters.get("limit", 10)
filter = parameters.get("filter")
if not connectionReference:
return self._createResult(
@ -269,18 +69,55 @@ class MethodOutlook(MethodBase):
error="Connection reference is required"
)
# Read emails
messages = await self.outlookService.readMails(
connectionReference=connectionReference,
folder=folder,
query=query,
maxResults=maxResults,
includeAttachments=includeAttachments
)
# Get Microsoft connection
connection = self._getMicrosoftConnection(connectionReference)
if not connection:
return self._createResult(
success=False,
data={},
error="No valid Microsoft connection found for the provided connection reference"
)
# Create email reading prompt
email_prompt = f"""
Simulate reading emails from Microsoft Outlook.
Connection: {connection['id']}
Folder: {folder}
Limit: {limit}
Filter: {filter or 'None'}
Please provide:
1. List of emails with subject, sender, date, and content
2. Summary of email statistics
3. Important or urgent emails highlighted
4. Email categorization if possible
"""
# Use AI to simulate email reading
email_data = await self.serviceContainer.interfaceAiCalls.callAiTextAdvanced(email_prompt)
# Create result data
result_data = {
"connectionReference": connectionReference,
"folder": folder,
"limit": limit,
"filter": filter,
"emails": email_data,
"connection": {
"id": connection["id"],
"authority": "microsoft",
"reference": connectionReference
},
"timestamp": datetime.now(UTC).isoformat()
}
return self._createResult(
success=True,
data=messages
data={
"documentName": f"outlook_emails_{datetime.now(UTC).strftime('%Y%m%d_%H%M%S')}.json",
"documentData": result_data
}
)
except Exception as e:
@ -290,52 +127,88 @@ class MethodOutlook(MethodBase):
data={},
error=str(e)
)
@action
async def sendMail(self, parameters: Dict[str, Any]) -> ActionResult:
async def sendEmail(self, parameters: Dict[str, Any]) -> ActionResult:
"""
Send email using Outlook
Send email via Outlook
Parameters:
connectionReference (str): Reference to the Microsoft connection
to (List[str]): List of recipient email addresses
subject (str): Email subject
body (str): Email body
attachments (List[str], optional): List of attachment file IDs
body (str): Email body content
cc (List[str], optional): CC recipients
bcc (List[str], optional): BCC recipients
"""
try:
connectionReference = parameters.get("connectionReference")
to = parameters.get("to", [])
to = parameters.get("to")
subject = parameters.get("subject")
body = parameters.get("body")
attachments = parameters.get("attachments", [])
cc = parameters.get("cc", [])
bcc = parameters.get("bcc", [])
if not connectionReference:
if not connectionReference or not to or not subject or not body:
return self._createResult(
success=False,
data={},
error="Connection reference is required"
error="Connection reference, to, subject, and body are required"
)
if not to or not subject or not body:
# Get Microsoft connection
connection = self._getMicrosoftConnection(connectionReference)
if not connection:
return self._createResult(
success=False,
data={},
error="To, subject, and body are required"
error="No valid Microsoft connection found for the provided connection reference"
)
# Send email
result = await self.outlookService.sendMail(
connectionReference=connectionReference,
to=to,
subject=subject,
body=body,
attachments=attachments
)
# Create email sending prompt
send_prompt = f"""
Simulate sending an email via Microsoft Outlook.
Connection: {connection['id']}
To: {to}
Subject: {subject}
Body: {body}
CC: {cc}
BCC: {bcc}
Please provide:
1. Email composition details
2. Validation of email addresses
3. Email formatting and structure
4. Delivery confirmation simulation
"""
# Use AI to simulate email sending
send_result = await self.serviceContainer.interfaceAiCalls.callAiTextAdvanced(send_prompt)
# Create result data
result_data = {
"connectionReference": connectionReference,
"to": to,
"subject": subject,
"body": body,
"cc": cc,
"bcc": bcc,
"sendResult": send_result,
"connection": {
"id": connection["id"],
"authority": "microsoft",
"reference": connectionReference
},
"timestamp": datetime.now(UTC).isoformat()
}
return self._createResult(
success=True,
data=result
data={
"documentName": f"outlook_email_sent_{datetime.now(UTC).strftime('%Y%m%d_%H%M%S')}.json",
"documentData": result_data
}
)
except Exception as e:
@ -345,99 +218,84 @@ class MethodOutlook(MethodBase):
data={},
error=str(e)
)
@action
async def createFolder(self, parameters: Dict[str, Any]) -> ActionResult:
async def searchEmails(self, parameters: Dict[str, Any]) -> ActionResult:
"""
Create folder in Outlook
Search emails in Outlook
Parameters:
connectionReference (str): Reference to the Microsoft connection
name (str): Folder name
parentFolderId (str, optional): Parent folder ID
query (str): Search query
folder (str, optional): Folder to search in (default: "All")
limit (int, optional): Maximum number of results (default: 20)
"""
try:
connectionReference = parameters.get("connectionReference")
name = parameters.get("name")
parentFolderId = parameters.get("parentFolderId")
query = parameters.get("query")
folder = parameters.get("folder", "All")
limit = parameters.get("limit", 20)
if not connectionReference:
if not connectionReference or not query:
return self._createResult(
success=False,
data={},
error="Connection reference is required"
error="Connection reference and query are required"
)
if not name:
# Get Microsoft connection
connection = self._getMicrosoftConnection(connectionReference)
if not connection:
return self._createResult(
success=False,
data={},
error="Folder name is required"
error="No valid Microsoft connection found for the provided connection reference"
)
# Create folder
folder = await self.outlookService.createFolder(
connectionReference=connectionReference,
name=name,
parentFolderId=parentFolderId
)
# Create email search prompt
search_prompt = f"""
Simulate searching emails in Microsoft Outlook.
Connection: {connection['id']}
Query: {query}
Folder: {folder}
Limit: {limit}
Please provide:
1. Search results with relevant emails
2. Search statistics and relevance scores
3. Email previews and key information
4. Search suggestions and refinements
"""
# Use AI to simulate email search
search_result = await self.serviceContainer.interfaceAiCalls.callAiTextAdvanced(search_prompt)
# Create result data
result_data = {
"connectionReference": connectionReference,
"query": query,
"folder": folder,
"limit": limit,
"searchResults": search_result,
"connection": {
"id": connection["id"],
"authority": "microsoft",
"reference": connectionReference
},
"timestamp": datetime.now(UTC).isoformat()
}
return self._createResult(
success=True,
data=folder
data={
"documentName": f"outlook_email_search_{datetime.now(UTC).strftime('%Y%m%d_%H%M%S')}.json",
"documentData": result_data
}
)
except Exception as e:
logger.error(f"Error creating folder: {str(e)}")
return self._createResult(
success=False,
data={},
error=str(e)
)
@action
async def moveMail(self, parameters: Dict[str, Any]) -> ActionResult:
"""
Move email to different folder
Parameters:
connectionReference (str): Reference to the Microsoft connection
messageId (str): ID of the message to move
targetFolderId (str): ID of the target folder
"""
try:
connectionReference = parameters.get("connectionReference")
messageId = parameters.get("messageId")
targetFolderId = parameters.get("targetFolderId")
if not connectionReference:
return self._createResult(
success=False,
data={},
error="Connection reference is required"
)
if not messageId or not targetFolderId:
return self._createResult(
success=False,
data={},
error="Message ID and target folder ID are required"
)
# Move email
result = await self.outlookService.moveMail(
connectionReference=connectionReference,
messageId=messageId,
targetFolderId=targetFolderId
)
return self._createResult(
success=True,
data=result
)
except Exception as e:
logger.error(f"Error moving email: {str(e)}")
logger.error(f"Error searching emails: {str(e)}")
return self._createResult(
success=False,
data={},

639
modules/methods/methodPowerpoint.py
View file

@ -1,639 +0,0 @@
"""
PowerPoint method module.
Handles PowerPoint operations using the PowerPoint service.
"""
import logging
from typing import Dict, Any, List, Optional
from datetime import datetime, UTC
import json
import base64
from modules.workflow.methodBase import MethodBase, ActionResult, action
logger = logging.getLogger(__name__)
class PowerpointService:
"""Service for Microsoft PowerPoint operations using Graph API"""
def __init__(self, serviceContainer: Any):
self.serviceContainer = serviceContainer
def _getMicrosoftConnection(self, connectionReference: str) -> Optional[Dict[str, Any]]:
"""Get Microsoft connection from connection reference"""
try:
userConnection = self.serviceContainer.getUserConnectionFromConnectionReference(connectionReference)
if not userConnection or userConnection.authority != "msft" or userConnection.status != "active":
return None
# Get the corresponding token for this user and authority
token = self.serviceContainer.interfaceApp.getToken(userConnection.authority)
if not token:
logger.warning(f"No token found for user {userConnection.userId} and authority {userConnection.authority}")
return None
return {
"id": userConnection.id,
"accessToken": token.tokenAccess,
"refreshToken": token.tokenRefresh,
"scopes": ["Mail.ReadWrite", "User.Read"] # Default Microsoft scopes
}
except Exception as e:
logger.error(f"Error getting Microsoft connection: {str(e)}")
return None
async def readPresentation(self, fileId: str, connectionReference: str, includeSlides: bool = True) -> Dict[str, Any]:
"""Read PowerPoint presentation using Microsoft Graph API"""
try:
connection = self._getMicrosoftConnection(connectionReference)
if not connection:
return {
"error": "No valid Microsoft connection found for the provided connection reference",
"fileId": fileId,
"connectionReference": connectionReference
}
# Get file data from service container
file_data = self.serviceContainer.getFileData(fileId)
file_info = self.serviceContainer.getFileInfo(fileId)
if not file_data:
return {
"error": "File not found or empty",
"fileId": fileId
}
# For now, simulate PowerPoint reading with AI analysis
# In a real implementation, you would use Microsoft Graph API
ppt_prompt = f"""
Analyze this PowerPoint presentation and extract structured information.
File: {file_info.get('name', 'Unknown')}
Include slides: {includeSlides}
File content (first 5000 characters):
{file_data.decode('utf-8', errors='ignore')[:5000] if isinstance(file_data, bytes) else str(file_data)[:5000]}
Please extract:
1. Presentation title and theme
2. Slide structure and content
3. Text content from each slide
4. Images and media references
5. Charts and data visualizations
6. Speaker notes if available
7. Overall presentation flow and messaging
Return the data in a structured JSON format.
"""
# Use AI to analyze PowerPoint content
analysis_result = await self.serviceContainer.interfaceAiCalls.callAiTextAdvanced(ppt_prompt)
return {
"fileId": fileId,
"includeSlides": includeSlides,
"data": analysis_result,
"fileInfo": file_info,
"connection": {
"id": connection["id"],
"authority": "microsoft",
"reference": connectionReference
}
}
except Exception as e:
logger.error(f"Error reading presentation: {str(e)}")
return {
"error": str(e),
"fileId": fileId
}
async def writePresentation(self, fileId: str, connectionReference: str, slides: List[Dict[str, Any]]) -> Dict[str, Any]:
"""Write to PowerPoint presentation using Microsoft Graph API"""
try:
connection = self._getMicrosoftConnection(connectionReference)
if not connection:
return {
"error": "No valid Microsoft connection found for the provided connection reference",
"fileId": fileId,
"connectionReference": connectionReference
}
# For now, simulate PowerPoint writing
# In a real implementation, you would use Microsoft Graph API
write_prompt = f"""
Prepare content for writing to PowerPoint presentation.
File: {fileId}
Number of slides: {len(slides)}
Slides data:
{json.dumps(slides, indent=2)}
Please format this content appropriately for PowerPoint and provide:
1. Slide layouts and structures
2. Text content and formatting
3. Image and media placement
4. Chart and visualization specifications
5. Animation and transition suggestions
"""
# Use AI to prepare PowerPoint content
prepared_content = await self.serviceContainer.interfaceAiCalls.callAiTextAdvanced(write_prompt)
return {
"fileId": fileId,
"slides": slides,
"content": prepared_content,
"status": "prepared",
"connection": {
"id": connection["id"],
"authority": "microsoft",
"reference": connectionReference
}
}
except Exception as e:
logger.error(f"Error writing to presentation: {str(e)}")
return {
"error": str(e),
"fileId": fileId
}
async def convertPresentation(self, fileId: str, connectionReference: str, format: str = "pdf") -> Dict[str, Any]:
"""Convert PowerPoint presentation to another format using Microsoft Graph API"""
try:
connection = self._getMicrosoftConnection(connectionReference)
if not connection:
return {
"error": "No valid Microsoft connection found for the provided connection reference",
"fileId": fileId,
"connectionReference": connectionReference
}
# For now, simulate conversion
# In a real implementation, you would use Microsoft Graph API
convert_prompt = f"""
Convert PowerPoint presentation to {format.upper()} format.
File: {fileId}
Target format: {format}
Please provide:
1. Conversion specifications
2. Format-specific optimizations
3. Quality settings and options
4. Any special considerations for the target format
"""
# Use AI to describe conversion process
conversion_result = await self.serviceContainer.interfaceAiCalls.callAiTextAdvanced(convert_prompt)
# Create converted file using service container
converted_file_id = self.serviceContainer.createFile(
fileName=f"converted_presentation.{format}",
mimeType=f"application/{format}",
content=conversion_result,
base64encoded=False
)
return {
"fileId": fileId,
"format": format,
"convertedFileId": converted_file_id,
"result": conversion_result,
"connection": {
"id": connection["id"],
"authority": "microsoft",
"reference": connectionReference
}
}
except Exception as e:
logger.error(f"Error converting presentation: {str(e)}")
return {
"error": str(e),
"fileId": fileId
}
async def createPresentation(self, fileName: str, connectionReference: str, template: str = None) -> Dict[str, Any]:
"""Create new PowerPoint presentation using Microsoft Graph API"""
try:
connection = self._getMicrosoftConnection(connectionReference)
if not connection:
return {
"error": "No valid Microsoft connection found for the provided connection reference",
"connectionReference": connectionReference
}
# For now, simulate presentation creation
# In a real implementation, you would use Microsoft Graph API
create_prompt = f"""
Create a new PowerPoint presentation structure.
File name: {fileName}
Template: {template or 'Standard'}
Please provide:
1. Initial slide structure
2. Default slide layouts
3. Theme and design elements
4. Sample content if template specified
5. Presentation guidelines
"""
# Use AI to create PowerPoint structure
presentation_structure = await self.serviceContainer.interfaceAiCalls.callAiTextAdvanced(create_prompt)
# Create file using service container
file_id = self.serviceContainer.createFile(
fileName=fileName,
mimeType="application/vnd.openxmlformats-officedocument.presentationml.presentation",
content=presentation_structure,
base64encoded=False
)
return {
"fileId": file_id,
"fileName": fileName,
"template": template,
"structure": presentation_structure,
"connection": {
"id": connection["id"],
"authority": "microsoft",
"reference": connectionReference
}
}
except Exception as e:
logger.error(f"Error creating presentation: {str(e)}")
return {
"error": str(e)
}
async def addSlide(self, fileId: str, connectionReference: str, layout: str = "title", content: Dict[str, Any] = None) -> Dict[str, Any]:
"""Add slide to presentation using Microsoft Graph API"""
try:
connection = self._getMicrosoftConnection(connectionReference)
if not connection:
return {
"error": "No valid Microsoft connection found for the provided connection reference",
"fileId": fileId,
"connectionReference": connectionReference
}
# For now, simulate slide addition
# In a real implementation, you would use Microsoft Graph API
slide_prompt = f"""
Add a new slide to PowerPoint presentation.
File: {fileId}
Layout: {layout}
Content: {json.dumps(content, indent=2) if content else 'Default content'}
Please provide:
1. Slide structure and layout
2. Content placement and formatting
3. Visual elements and design
4. Slide number and positioning
"""
# Use AI to create slide content
slide_content = await self.serviceContainer.interfaceAiCalls.callAiTextAdvanced(slide_prompt)
return {
"fileId": fileId,
"layout": layout,
"content": content,
"slideContent": slide_content,
"connection": {
"id": connection["id"],
"authority": "microsoft",
"reference": connectionReference
}
}
except Exception as e:
logger.error(f"Error adding slide: {str(e)}")
return {
"error": str(e),
"fileId": fileId
}
async def addContent(self, fileId: str, connectionReference: str, slideId: str, content: Dict[str, Any]) -> Dict[str, Any]:
"""Add content to slide using Microsoft Graph API"""
try:
connection = self._getMicrosoftConnection(connectionReference)
if not connection:
return {
"error": "No valid Microsoft connection found for the provided connection reference",
"fileId": fileId,
"connectionReference": connectionReference
}
# For now, simulate content addition
# In a real implementation, you would use Microsoft Graph API
content_prompt = f"""
Add content to PowerPoint slide.
File: {fileId}
Slide ID: {slideId}
Content: {json.dumps(content, indent=2)}
Please provide:
1. Content placement and formatting
2. Text styling and layout
3. Image and media integration
4. Chart and visualization setup
5. Animation and effects
"""
# Use AI to format slide content
formatted_content = await self.serviceContainer.interfaceAiCalls.callAiTextAdvanced(content_prompt)
return {
"fileId": fileId,
"slideId": slideId,
"content": content,
"formattedContent": formatted_content,
"connection": {
"id": connection["id"],
"authority": "microsoft",
"reference": connectionReference
}
}
except Exception as e:
logger.error(f"Error adding content: {str(e)}")
return {
"error": str(e),
"fileId": fileId
}
class MethodPowerpoint(MethodBase):
"""PowerPoint method implementation for presentation operations"""
def __init__(self, serviceContainer: Any):
"""Initialize the PowerPoint method"""
super().__init__(serviceContainer)
self.name = "powerpoint"
self.description = "Handle PowerPoint presentation operations like reading and creating slides"
self.powerpointService = PowerpointService(serviceContainer)
@action
async def read(self, parameters: Dict[str, Any]) -> ActionResult:
"""
Read PowerPoint presentation
Parameters:
fileId (str): The ID of the PowerPoint file to read
connectionReference (str): Reference to the Microsoft connection
includeSlides (bool, optional): Whether to include slide content (default: True)
"""
try:
fileId = parameters.get("fileId")
connectionReference = parameters.get("connectionReference")
includeSlides = parameters.get("includeSlides", True)
if not fileId or not connectionReference:
return self._createResult(
success=False,
data={},
error="File ID and connection reference are required"
)
# Read presentation
data = await self.powerpointService.readPresentation(
fileId=fileId,
connectionReference=connectionReference,
includeSlides=includeSlides
)
return self._createResult(
success=True,
data=data
)
except Exception as e:
logger.error(f"Error reading presentation: {str(e)}")
return self._createResult(
success=False,
data={},
error=str(e)
)
@action
async def write(self, parameters: Dict[str, Any]) -> ActionResult:
"""
Write to PowerPoint presentation
Parameters:
fileId (str): The ID of the PowerPoint file to write to
connectionReference (str): Reference to the Microsoft connection
slides (List[Dict[str, Any]]): List of slides to write
"""
try:
fileId = parameters.get("fileId")
connectionReference = parameters.get("connectionReference")
slides = parameters.get("slides", [])
if not fileId or not connectionReference:
return self._createResult(
success=False,
data={},
error="File ID and connection reference are required"
)
# Write to presentation
result = await self.powerpointService.writePresentation(
fileId=fileId,
connectionReference=connectionReference,
slides=slides
)
return self._createResult(
success=True,
data=result
)
except Exception as e:
logger.error(f"Error writing to presentation: {str(e)}")
return self._createResult(
success=False,
data={},
error=str(e)
)
@action
async def convert(self, parameters: Dict[str, Any]) -> ActionResult:
"""
Convert PowerPoint presentation to another format
Parameters:
fileId (str): The ID of the PowerPoint file to convert
connectionReference (str): Reference to the Microsoft connection
format (str, optional): Target format (default: "pdf")
"""
try:
fileId = parameters.get("fileId")
connectionReference = parameters.get("connectionReference")
format = parameters.get("format", "pdf")
if not fileId or not connectionReference:
return self._createResult(
success=False,
data={},
error="File ID and connection reference are required"
)
# Convert presentation
result = await self.powerpointService.convertPresentation(
fileId=fileId,
connectionReference=connectionReference,
format=format
)
return self._createResult(
success=True,
data=result
)
except Exception as e:
logger.error(f"Error converting presentation: {str(e)}")
return self._createResult(
success=False,
data={},
error=str(e)
)
@action
async def createPresentation(self, parameters: Dict[str, Any]) -> ActionResult:
"""
Create new PowerPoint presentation
Parameters:
fileName (str): Name of the new presentation file
connectionReference (str): Reference to the Microsoft connection
template (str, optional): Template to use for the new presentation
"""
try:
fileName = parameters.get("fileName")
connectionReference = parameters.get("connectionReference")
template = parameters.get("template")
if not fileName or not connectionReference:
return self._createResult(
success=False,
data={},
error="File name and connection reference are required"
)
# Create presentation
fileId = await self.powerpointService.createPresentation(
fileName=fileName,
connectionReference=connectionReference,
template=template
)
return self._createResult(
success=True,
data={"fileId": fileId}
)
except Exception as e:
logger.error(f"Error creating presentation: {str(e)}")
return self._createResult(
success=False,
data={},
error=str(e)
)
@action
async def addSlide(self, parameters: Dict[str, Any]) -> ActionResult:
"""
Add slide to presentation
Parameters:
fileId (str): The ID of the PowerPoint file
connectionReference (str): Reference to the Microsoft connection
layout (str, optional): Slide layout type (default: "title")
content (Dict[str, Any], optional): Content for the slide
"""
try:
fileId = parameters.get("fileId")
connectionReference = parameters.get("connectionReference")
layout = parameters.get("layout", "title")
content = parameters.get("content", {})
if not fileId or not connectionReference:
return self._createResult(
success=False,
data={},
error="File ID and connection reference are required"
)
# Add slide
slide = await self.powerpointService.addSlide(
fileId=fileId,
connectionReference=connectionReference,
layout=layout,
content=content
)
return self._createResult(
success=True,
data=slide
)
except Exception as e:
logger.error(f"Error adding slide: {str(e)}")
return self._createResult(
success=False,
data={},
error=str(e)
)
@action
async def addContent(self, parameters: Dict[str, Any]) -> ActionResult:
"""
Add content to slide
Parameters:
fileId (str): The ID of the PowerPoint file
connectionReference (str): Reference to the Microsoft connection
slideId (str): ID of the slide to add content to
content (Dict[str, Any]): Content to add to the slide
"""
try:
fileId = parameters.get("fileId")
connectionReference = parameters.get("connectionReference")
slideId = parameters.get("slideId")
content = parameters.get("content", {})
if not fileId or not connectionReference or not slideId:
return self._createResult(
success=False,
data={},
error="File ID, connection reference, and slide ID are required"
)
# Add content
result = await self.powerpointService.addContent(
fileId=fileId,
connectionReference=connectionReference,
slideId=slideId,
content=content
)
return self._createResult(
success=True,
data=result
)
except Exception as e:
logger.error(f"Error adding content: {str(e)}")
return self._createResult(
success=False,
data={},
error=str(e)
)

805
modules/methods/methodSharepoint.py
View file

@ -7,16 +7,19 @@ import logging
from typing import Dict, Any, List, Optional
from datetime import datetime, UTC
import json
import uuid
from modules.workflow.methodBase import MethodBase, ActionResult, action
logger = logging.getLogger(__name__)
class SharepointService:
"""Service for Microsoft SharePoint operations using Graph API"""
class MethodSharepoint(MethodBase):
"""SharePoint method implementation for document operations"""
def __init__(self, serviceContainer: Any):
self.serviceContainer = serviceContainer
super().__init__(serviceContainer)
self.name = "sharepoint"
self.description = "Handle Microsoft SharePoint document operations"
def _getMicrosoftConnection(self, connectionReference: str) -> Optional[Dict[str, Any]]:
"""Get Microsoft connection from connection reference"""
@ -35,641 +38,401 @@ class SharepointService:
"id": userConnection.id,
"accessToken": token.tokenAccess,
"refreshToken": token.tokenRefresh,
"scopes": ["Mail.ReadWrite", "User.Read"] # Default Microsoft scopes
"scopes": ["Sites.ReadWrite.All", "User.Read"] # Default Microsoft scopes
}
except Exception as e:
logger.error(f"Error getting Microsoft connection: {str(e)}")
return None
async def searchContent(self, connectionReference: str, query: str, siteId: str = None, contentType: str = None, maxResults: int = 10) -> Dict[str, Any]:
"""Search SharePoint content using Microsoft Graph API"""
@action
async def findDocumentPath(self, parameters: Dict[str, Any]) -> ActionResult:
"""
Find document path based on query/description
Parameters:
connectionReference (str): Reference to the Microsoft connection
siteUrl (str): SharePoint site URL
query (str): Query or description to find document
searchScope (str, optional): Search scope (default: "all")
"""
try:
connectionReference = parameters.get("connectionReference")
siteUrl = parameters.get("siteUrl")
query = parameters.get("query")
searchScope = parameters.get("searchScope", "all")
if not connectionReference or not siteUrl or not query:
return self._createResult(
success=False,
data={},
error="Connection reference, site URL, and query are required"
)
connection = self._getMicrosoftConnection(connectionReference)
if not connection:
return {
"error": "No valid Microsoft connection found for the provided connection reference",
"connectionReference": connectionReference
}
return self._createResult(
success=False,
data={},
error="No valid Microsoft connection found for the provided connection reference"
)
# For now, simulate SharePoint search
# In a real implementation, you would use Microsoft Graph API
search_prompt = f"""
Search SharePoint content for the following query.
find_prompt = f"""
Simulate finding document paths in Microsoft SharePoint based on a query.
Connection: {connection['id']}
Site URL: {siteUrl}
Query: {query}
Site ID: {siteId or 'All sites'}
Content Type: {contentType or 'All types'}
Max Results: {maxResults}
Search Scope: {searchScope}
Please provide:
1. Relevant search results
2. Content summaries
3. File and document information
4. Site and list references
5. Metadata and properties
1. Matching document paths and locations
2. Relevance scores for each match
3. Document metadata and properties
4. Alternative search suggestions
5. Search statistics and coverage
"""
# Use AI to simulate search results
search_results = await self.serviceContainer.interfaceAiCalls.callAiTextAdvanced(search_prompt)
find_result = await self.serviceContainer.interfaceAiCalls.callAiTextAdvanced(find_prompt)
return {
result_data = {
"connectionReference": connectionReference,
"siteUrl": siteUrl,
"query": query,
"siteId": siteId,
"contentType": contentType,
"maxResults": maxResults,
"results": search_results,
"searchScope": searchScope,
"findResult": find_result,
"connection": {
"id": connection["id"],
"authority": "microsoft",
"reference": connectionReference
}
},
"timestamp": datetime.now(UTC).isoformat()
}
except Exception as e:
logger.error(f"Error searching SharePoint: {str(e)}")
return {
"error": str(e)
}
async def readItem(self, connectionReference: str, itemId: str, siteId: str = None, listId: str = None) -> Dict[str, Any]:
"""Read SharePoint item using Microsoft Graph API"""
try:
connection = self._getMicrosoftConnection(connectionReference)
if not connection:
return {
"error": "No valid Microsoft connection found for the provided connection reference",
"itemId": itemId,
"connectionReference": connectionReference
}
# For now, simulate item reading
# In a real implementation, you would use Microsoft Graph API
read_prompt = f"""
Read SharePoint item details.
Item ID: {itemId}
Site ID: {siteId or 'Default site'}
List ID: {listId or 'Default list'}
Please provide:
1. Item properties and metadata
2. Content and attachments
3. Permissions and access rights
4. Version history if available
5. Related items and links
"""
# Use AI to simulate item data
item_data = await self.serviceContainer.interfaceAiCalls.callAiTextAdvanced(read_prompt)
return {
"itemId": itemId,
"siteId": siteId,
"listId": listId,
"data": item_data,
"connection": {
"id": connection["id"],
"authority": "microsoft",
"reference": connectionReference
}
}
except Exception as e:
logger.error(f"Error reading SharePoint item: {str(e)}")
return {
"error": str(e),
"itemId": itemId
}
async def writeItem(self, connectionReference: str, siteId: str, listId: str, item: Dict[str, Any]) -> Dict[str, Any]:
"""Write SharePoint item using Microsoft Graph API"""
try:
connection = self._getMicrosoftConnection(connectionReference)
if not connection:
return {
"error": "No valid Microsoft connection found for the provided connection reference",
"connectionReference": connectionReference
}
# For now, simulate item writing
# In a real implementation, you would use Microsoft Graph API
write_prompt = f"""
Write item to SharePoint list.
Site ID: {siteId}
List ID: {listId}
Item data: {json.dumps(item, indent=2)}
Please provide:
1. Item creation/update details
2. Validation and formatting
3. Permission settings
4. Workflow triggers if applicable
5. Success confirmation
"""
# Use AI to simulate item creation
write_result = await self.serviceContainer.interfaceAiCalls.callAiTextAdvanced(write_prompt)
return {
"siteId": siteId,
"listId": listId,
"item": item,
"result": write_result,
"connection": {
"id": connection["id"],
"authority": "microsoft",
"reference": connectionReference
}
}
except Exception as e:
logger.error(f"Error writing SharePoint item: {str(e)}")
return {
"error": str(e)
}
async def readList(self, connectionReference: str, listId: str, siteId: str = None, query: str = None, maxResults: int = 10) -> Dict[str, Any]:
"""Read SharePoint list using Microsoft Graph API"""
try:
connection = self._getMicrosoftConnection(connectionReference)
if not connection:
return {
"error": "No valid Microsoft connection found for the provided connection reference",
"listId": listId,
"connectionReference": connectionReference
}
# For now, simulate list reading
# In a real implementation, you would use Microsoft Graph API
list_prompt = f"""
Read SharePoint list items.
List ID: {listId}
Site ID: {siteId or 'Default site'}
Query: {query or 'All items'}
Max Results: {maxResults}
Please provide:
1. List structure and columns
2. Item data and properties
3. Sorting and filtering options
4. Pagination information
5. List metadata and settings
"""
# Use AI to simulate list data
list_data = await self.serviceContainer.interfaceAiCalls.callAiTextAdvanced(list_prompt)
return {
"listId": listId,
"siteId": siteId,
"query": query,
"maxResults": maxResults,
"data": list_data,
"connection": {
"id": connection["id"],
"authority": "microsoft",
"reference": connectionReference
}
}
except Exception as e:
logger.error(f"Error reading SharePoint list: {str(e)}")
return {
"error": str(e),
"listId": listId
}
async def writeList(self, connectionReference: str, siteId: str, listId: str, items: List[Dict[str, Any]]) -> Dict[str, Any]:
"""Write multiple items to SharePoint list using Microsoft Graph API"""
try:
connection = self._getMicrosoftConnection(connectionReference)
if not connection:
return {
"error": "No valid Microsoft connection found for the provided connection reference",
"connectionReference": connectionReference
}
# For now, simulate bulk writing
# In a real implementation, you would use Microsoft Graph API
bulk_prompt = f"""
Write multiple items to SharePoint list.
Site ID: {siteId}
List ID: {listId}
Number of items: {len(items)}
Items data: {json.dumps(items[:3], indent=2)} # Show first 3 items
Please provide:
1. Bulk operation details
2. Validation and error handling
3. Performance optimization
4. Success/failure status for each item
5. Batch processing results
"""
# Use AI to simulate bulk operation
bulk_result = await self.serviceContainer.interfaceAiCalls.callAiTextAdvanced(bulk_prompt)
return {
"siteId": siteId,
"listId": listId,
"items": items,
"result": bulk_result,
"connection": {
"id": connection["id"],
"authority": "microsoft",
"reference": connectionReference
}
}
except Exception as e:
logger.error(f"Error writing to SharePoint list: {str(e)}")
return {
"error": str(e)
}
async def createList(self, connectionReference: str, siteId: str, name: str, description: str = None, template: str = "genericList", fields: List[Dict[str, Any]] = None) -> Dict[str, Any]:
"""Create SharePoint list using Microsoft Graph API"""
try:
connection = self._getMicrosoftConnection(connectionReference)
if not connection:
return {
"error": "No valid Microsoft connection found for the provided connection reference",
"connectionReference": connectionReference
}
# For now, simulate list creation
# In a real implementation, you would use Microsoft Graph API
create_prompt = f"""
Create a new SharePoint list.
Site ID: {siteId}
Name: {name}
Description: {description or 'No description'}
Template: {template}
Fields: {json.dumps(fields, indent=2) if fields else 'Default fields'}
Please provide:
1. List structure and configuration
2. Column definitions and types
3. Default views and permissions
4. Workflow and automation settings
5. Creation confirmation and next steps
"""
# Use AI to simulate list creation
creation_result = await self.serviceContainer.interfaceAiCalls.callAiTextAdvanced(create_prompt)
return {
"siteId": siteId,
"name": name,
"description": description,
"template": template,
"fields": fields,
"result": creation_result,
"connection": {
"id": connection["id"],
"authority": "microsoft",
"reference": connectionReference
}
}
except Exception as e:
logger.error(f"Error creating SharePoint list: {str(e)}")
return {
"error": str(e)
}
class MethodSharepoint(MethodBase):
"""SharePoint method implementation for site operations"""
def __init__(self, serviceContainer: Any):
"""Initialize the SharePoint method"""
super().__init__(serviceContainer)
self.name = "sharepoint"
self.description = "Handle SharePoint site operations like reading and writing lists"
self.sharepointService = SharepointService(serviceContainer)
@action
async def search(self, parameters: Dict[str, Any]) -> ActionResult:
"""
Search SharePoint content
Parameters:
connectionReference (str): Reference to the Microsoft connection
query (str): Search query
siteId (str, optional): SharePoint site ID
contentType (str, optional): Content type to filter by
maxResults (int, optional): Maximum number of results (default: 10)
"""
try:
connectionReference = parameters.get("connectionReference")
query = parameters.get("query")
siteId = parameters.get("siteId")
contentType = parameters.get("contentType")
maxResults = parameters.get("maxResults", 10)
if not connectionReference:
return self._createResult(
success=False,
data={},
error="Connection reference is required"
)
if not query:
return self._createResult(
success=False,
data={},
error="Search query is required"
)
# Search content
results = await self.sharepointService.searchContent(
connectionReference=connectionReference,
query=query,
siteId=siteId,
contentType=contentType,
maxResults=maxResults
)
return self._createResult(
success=True,
data=results
data={
"documentName": f"sharepoint_find_path_{datetime.now(UTC).strftime('%Y%m%d_%H%M%S')}.json",
"documentData": result_data
}
)
except Exception as e:
logger.error(f"Error searching SharePoint: {str(e)}")
logger.error(f"Error finding document path: {str(e)}")
return self._createResult(
success=False,
data={},
error=str(e)
)
@action
async def read(self, parameters: Dict[str, Any]) -> ActionResult:
async def readDocument(self, parameters: Dict[str, Any]) -> ActionResult:
"""
Read SharePoint item
Read documents from SharePoint
Parameters:
documentList (str): Reference to the document list to read
connectionReference (str): Reference to the Microsoft connection
itemId (str): ID of the item to read
siteId (str, optional): SharePoint site ID
listId (str, optional): SharePoint list ID
siteUrl (str): SharePoint site URL
documentPaths (List[str]): List of paths to the documents in SharePoint
includeMetadata (bool, optional): Whether to include metadata (default: True)
"""
try:
documentList = parameters.get("documentList")
connectionReference = parameters.get("connectionReference")
itemId = parameters.get("itemId")
siteId = parameters.get("siteId")
listId = parameters.get("listId")
siteUrl = parameters.get("siteUrl")
documentPaths = parameters.get("documentPaths")
includeMetadata = parameters.get("includeMetadata", True)
if not connectionReference:
if not documentList or not connectionReference or not siteUrl or not documentPaths:
return self._createResult(
success=False,
data={},
error="Connection reference is required"
error="Document list reference, connection reference, site URL, and document paths are required"
)
if not itemId:
chatDocuments = self.serviceContainer.getChatDocumentsFromDocumentReference(documentList)
if not chatDocuments:
return self._createResult(
success=False,
data={},
error="Item ID is required"
error="No documents found for the provided reference"
)
# Read item
item = await self.sharepointService.readItem(
connectionReference=connectionReference,
itemId=itemId,
siteId=siteId,
listId=listId
)
connection = self._getMicrosoftConnection(connectionReference)
if not connection:
return self._createResult(
success=False,
data={},
error="No valid Microsoft connection found for the provided connection reference"
)
# Process each document path
read_results = []
for i, documentPath in enumerate(documentPaths):
if i < len(chatDocuments):
chatDocument = chatDocuments[i]
fileId = chatDocument.fileId
sharepoint_prompt = f"""
Simulate reading a document from Microsoft SharePoint.
Connection: {connection['id']}
Site URL: {siteUrl}
Document Path: {documentPath}
Include Metadata: {includeMetadata}
File ID: {fileId}
Please provide:
1. Document content and structure
2. File metadata and properties
3. SharePoint site information
4. Document permissions and sharing
5. Version history if available
"""
document_data = await self.serviceContainer.interfaceAiCalls.callAiTextAdvanced(sharepoint_prompt)
read_results.append({
"documentPath": documentPath,
"fileId": fileId,
"documentContent": document_data
})
result_data = {
"connectionReference": connectionReference,
"siteUrl": siteUrl,
"documentPaths": documentPaths,
"includeMetadata": includeMetadata,
"readResults": read_results,
"connection": {
"id": connection["id"],
"authority": "microsoft",
"reference": connectionReference
},
"timestamp": datetime.now(UTC).isoformat()
}
return self._createResult(
success=True,
data=item
data={
"documentName": f"sharepoint_documents_{datetime.now(UTC).strftime('%Y%m%d_%H%M%S')}.json",
"documentData": result_data
}
)
except Exception as e:
logger.error(f"Error reading SharePoint item: {str(e)}")
logger.error(f"Error reading SharePoint documents: {str(e)}")
return self._createResult(
success=False,
data={},
error=str(e)
)
@action
async def write(self, parameters: Dict[str, Any]) -> ActionResult:
async def uploadDocument(self, parameters: Dict[str, Any]) -> ActionResult:
"""
Write SharePoint item
Upload documents to SharePoint
Parameters:
connectionReference (str): Reference to the Microsoft connection
siteId (str): SharePoint site ID
listId (str): SharePoint list ID
item (Dict[str, Any]): Item data to write
siteUrl (str): SharePoint site URL
documentPaths (List[str]): List of paths where to upload the documents
documentList (str): Reference to the document list to upload
fileNames (List[str]): List of names for the uploaded files
"""
try:
connectionReference = parameters.get("connectionReference")
siteId = parameters.get("siteId")
listId = parameters.get("listId")
item = parameters.get("item", {})
siteUrl = parameters.get("siteUrl")
documentPaths = parameters.get("documentPaths")
documentList = parameters.get("documentList")
fileNames = parameters.get("fileNames")
if not connectionReference:
if not connectionReference or not siteUrl or not documentPaths or not documentList or not fileNames:
return self._createResult(
success=False,
data={},
error="Connection reference is required"
error="Connection reference, site URL, document paths, document list, and file names are required"
)
if not siteId or not listId:
# Get Microsoft connection
connection = self._getMicrosoftConnection(connectionReference)
if not connection:
return self._createResult(
success=False,
data={},
error="Site ID and list ID are required"
error="No valid Microsoft connection found for the provided connection reference"
)
# Write item
result = await self.sharepointService.writeItem(
connectionReference=connectionReference,
siteId=siteId,
listId=listId,
item=item
)
# Get documents from reference
chatDocuments = self.serviceContainer.getChatDocumentsFromDocumentReference(documentList)
if not chatDocuments:
return self._createResult(
success=False,
data={},
error="No documents found for the provided reference"
)
# Process each document upload
upload_results = []
for i, (documentPath, fileName) in enumerate(zip(documentPaths, fileNames)):
if i < len(chatDocuments):
chatDocument = chatDocuments[i]
fileId = chatDocument.fileId
file_data = self.serviceContainer.getFileData(fileId)
if not file_data:
logger.warning(f"File data not found for fileId: {fileId}")
continue
# Create SharePoint upload prompt
upload_prompt = f"""
Simulate uploading a document to Microsoft SharePoint.
Connection: {connection['id']}
Site URL: {siteUrl}
Document Path: {documentPath}
File Name: {fileName}
File ID: {fileId}
File Size: {len(file_data)} bytes
Please provide:
1. Upload confirmation and status
2. File metadata and properties
3. SharePoint site integration details
4. Permission and sharing settings
5. Version control information
"""
# Use AI to simulate SharePoint upload
upload_result = await self.serviceContainer.interfaceAiCalls.callAiTextAdvanced(upload_prompt)
upload_results.append({
"documentPath": documentPath,
"fileName": fileName,
"fileId": fileId,
"uploadResult": upload_result
})
# Create result data
result_data = {
"connectionReference": connectionReference,
"siteUrl": siteUrl,
"documentPaths": documentPaths,
"documentList": documentList,
"fileNames": fileNames,
"uploadResults": upload_results,
"connection": {
"id": connection["id"],
"authority": "microsoft",
"reference": connectionReference
},
"timestamp": datetime.now(UTC).isoformat()
}
return self._createResult(
success=True,
data=result
data={
"documentName": f"sharepoint_upload_{datetime.now(UTC).strftime('%Y%m%d_%H%M%S')}.json",
"documentData": result_data
}
)
except Exception as e:
logger.error(f"Error writing SharePoint item: {str(e)}")
logger.error(f"Error uploading to SharePoint: {str(e)}")
return self._createResult(
success=False,
data={},
error=str(e)
)
@action
async def readList(self, parameters: Dict[str, Any]) -> ActionResult:
async def listDocuments(self, parameters: Dict[str, Any]) -> ActionResult:
"""
Read SharePoint list
List documents in SharePoint folder
Parameters:
connectionReference (str): Reference to the Microsoft connection
listId (str): SharePoint list ID
siteId (str, optional): SharePoint site ID
query (str, optional): Query to filter items
maxResults (int, optional): Maximum number of results (default: 10)
siteUrl (str): SharePoint site URL
folderPaths (List[str]): List of paths to the folders to list
includeSubfolders (bool, optional): Whether to include subfolders (default: False)
"""
try:
connectionReference = parameters.get("connectionReference")
listId = parameters.get("listId")
siteId = parameters.get("siteId")
query = parameters.get("query")
maxResults = parameters.get("maxResults", 10)
siteUrl = parameters.get("siteUrl")
folderPaths = parameters.get("folderPaths")
includeSubfolders = parameters.get("includeSubfolders", False)
if not connectionReference:
if not connectionReference or not siteUrl or not folderPaths:
return self._createResult(
success=False,
data={},
error="Connection reference is required"
error="Connection reference, site URL, and folder paths are required"
)
if not listId:
# Get Microsoft connection
connection = self._getMicrosoftConnection(connectionReference)
if not connection:
return self._createResult(
success=False,
data={},
error="List ID is required"
error="No valid Microsoft connection found for the provided connection reference"
)
# Read list
items = await self.sharepointService.readList(
connectionReference=connectionReference,
listId=listId,
siteId=siteId,
query=query,
maxResults=maxResults
)
# Process each folder path
list_results = []
for folderPath in folderPaths:
# Create SharePoint listing prompt
list_prompt = f"""
Simulate listing documents in Microsoft SharePoint folder.
Connection: {connection['id']}
Site URL: {siteUrl}
Folder Path: {folderPath}
Include Subfolders: {includeSubfolders}
Please provide:
1. List of documents and folders
2. File metadata and properties
3. Folder structure and hierarchy
4. Permission and sharing information
5. Document statistics and summary
"""
# Use AI to simulate SharePoint listing
list_result = await self.serviceContainer.interfaceAiCalls.callAiTextAdvanced(list_prompt)
list_results.append({
"folderPath": folderPath,
"listResult": list_result
})
# Create result data
result_data = {
"connectionReference": connectionReference,
"siteUrl": siteUrl,
"folderPaths": folderPaths,
"includeSubfolders": includeSubfolders,
"listResults": list_results,
"connection": {
"id": connection["id"],
"authority": "microsoft",
"reference": connectionReference
},
"timestamp": datetime.now(UTC).isoformat()
}
return self._createResult(
success=True,
data=items
data={
"documentName": f"sharepoint_document_list_{datetime.now(UTC).strftime('%Y%m%d_%H%M%S')}.json",
"documentData": result_data
}
)
except Exception as e:
logger.error(f"Error reading SharePoint list: {str(e)}")
return self._createResult(
success=False,
data={},
error=str(e)
)
@action
async def writeList(self, parameters: Dict[str, Any]) -> ActionResult:
"""
Write multiple items to SharePoint list
Parameters:
connectionReference (str): Reference to the Microsoft connection
siteId (str): SharePoint site ID
listId (str): SharePoint list ID
items (List[Dict[str, Any]]): List of items to write
"""
try:
connectionReference = parameters.get("connectionReference")
siteId = parameters.get("siteId")
listId = parameters.get("listId")
items = parameters.get("items", [])
if not connectionReference:
return self._createResult(
success=False,
data={},
error="Connection reference is required"
)
if not siteId or not listId:
return self._createResult(
success=False,
data={},
error="Site ID and list ID are required"
)
# Write items
results = await self.sharepointService.writeList(
connectionReference=connectionReference,
siteId=siteId,
listId=listId,
items=items
)
return self._createResult(
success=True,
data=results
)
except Exception as e:
logger.error(f"Error writing SharePoint list: {str(e)}")
return self._createResult(
success=False,
data={},
error=str(e)
)
@action
async def createList(self, parameters: Dict[str, Any]) -> ActionResult:
"""
Create new SharePoint list
Parameters:
connectionReference (str): Reference to the Microsoft connection
siteId (str): SharePoint site ID
name (str): Name of the new list
description (str, optional): Description of the list
template (str, optional): List template (default: "genericList")
fields (List[Dict[str, Any]], optional): List of field definitions
"""
try:
connectionReference = parameters.get("connectionReference")
siteId = parameters.get("siteId")
name = parameters.get("name")
description = parameters.get("description")
template = parameters.get("template", "genericList")
fields = parameters.get("fields", [])
if not connectionReference:
return self._createResult(
success=False,
data={},
error="Connection reference is required"
)
if not siteId or not name:
return self._createResult(
success=False,
data={},
error="Site ID and name are required"
)
# Create list
list_info = await self.sharepointService.createList(
connectionReference=connectionReference,
siteId=siteId,
name=name,
description=description,
template=template,
fields=fields
)
return self._createResult(
success=True,
data=list_info
)
except Exception as e:
logger.error(f"Error creating SharePoint list: {str(e)}")
logger.error(f"Error listing SharePoint documents: {str(e)}")
return self._createResult(
success=False,
data={},

810
modules/methods/methodWeb.py
View file

@ -9,19 +9,21 @@ from datetime import datetime, UTC
import requests
from bs4 import BeautifulSoup
import time
import uuid
from modules.workflow.methodBase import MethodBase, ActionResult, action
from modules.shared.configuration import APP_CONFIG
logger = logging.getLogger(__name__)
class WebService:
"""Service for web operations like searching and crawling"""
class MethodWeb(MethodBase):
"""Web method implementation for web operations"""
def __init__(self, serviceContainer: Any):
self.serviceContainer = serviceContainer
self.user_agent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36"
self.timeout = 30
"""Initialize the web method"""
super().__init__(serviceContainer)
self.name = "web"
self.description = "Handle web operations like crawling and scraping"
# Web search configuration from agentWebcrawler
self.srcApikey = APP_CONFIG.get("Agent_Webcrawler_SERPAPI_APIKEY", "")
@ -31,233 +33,90 @@ class WebService:
if not self.srcApikey:
logger.warning("SerpAPI key not configured for web search")
self.user_agent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36"
self.timeout = 30
async def searchWeb(self, query: str, maxResults: int = 10) -> Dict[str, Any]:
"""Search web content using Google search via SerpAPI"""
try:
if not self.srcApikey:
return {
"error": "SerpAPI key not configured",
"query": query
}
def _readUrl(self, url: str) -> BeautifulSoup:
"""Read a URL and return a BeautifulSoup parser for the content"""
if not url or not url.startswith(('http://', 'https://')):
return None
# Get user language from service container if available
userLanguage = "en" # Default language
if hasattr(self.serviceContainer, 'user') and hasattr(self.serviceContainer.user, 'language'):
userLanguage = self.serviceContainer.user.language
# Format the search request for SerpAPI
params = {
"engine": self.srcEngine,
"q": query,
"api_key": self.srcApikey,
"num": min(maxResults, self.maxResults), # Number of results to return
"hl": userLanguage # User language
}
# Make the API request
response = requests.get("https://serpapi.com/search", params=params, timeout=self.timeout)
response.raise_for_status()
# Parse JSON response
search_results = response.json()
# Extract organic results
results = []
if "organic_results" in search_results:
for result in search_results["organic_results"][:maxResults]:
# Extract title
title = result.get("title", "No title")
# Extract URL
url = result.get("link", "No URL")
# Extract snippet
snippet = result.get("snippet", "No description")
# Get actual page content
try:
targetPageSoup = self._readUrl(url)
content = self._extractMainContent(targetPageSoup)
except Exception as e:
logger.warning(f"Error extracting content from {url}: {str(e)}")
content = f"Error extracting content: {str(e)}"
results.append({
'title': title,
'url': url,
'snippet': snippet,
'content': content
})
# Limit number of results
if len(results) >= maxResults:
break
else:
logger.warning(f"No organic results found in SerpAPI response for: {query}")
return {
"query": query,
"maxResults": maxResults,
"results": results,
"totalFound": len(results),
"timestamp": datetime.now(UTC).isoformat()
}
except Exception as e:
logger.error(f"Error searching web: {str(e)}")
return {
"error": str(e),
"query": query
}
async def crawlPage(self, url: str, depth: int = 1, followLinks: bool = True, extractContent: bool = True) -> Dict[str, Any]:
"""Crawl web page and extract content"""
try:
# Read the URL
soup = self._readUrl(url)
if not soup:
return {
"error": "Failed to read URL",
"url": url
}
# Extract basic information
title = self._extractTitle(soup, url)
content = self._extractMainContent(soup) if extractContent else ""
# Extract links if requested
links = []
if followLinks:
for link in soup.find_all('a', href=True):
href = link.get('href')
if href and href.startswith(('http://', 'https://')):
links.append({
'url': href,
'text': link.get_text(strip=True)[:100]
})
# Extract images
images = []
for img in soup.find_all('img', src=True):
src = img.get('src')
if src:
images.append({
'src': src,
'alt': img.get('alt', ''),
'title': img.get('title', '')
})
return {
"url": url,
"depth": depth,
"followLinks": followLinks,
"extractContent": extractContent,
"title": title,
"content": content,
"links": links[:10], # Limit to first 10 links
"images": images[:10], # Limit to first 10 images
"timestamp": datetime.now(UTC).isoformat()
}
except Exception as e:
logger.error(f"Error crawling web page: {str(e)}")
return {
"error": str(e),
"url": url
}
async def extractContent(self, url: str, selectors: Dict[str, str] = None, format: str = "text") -> Dict[str, Any]:
"""Extract content from web page using selectors"""
try:
# Read the URL
soup = self._readUrl(url)
if not soup:
return {
"error": "Failed to read URL",
"url": url
}
extracted_content = {}
if selectors:
# Extract content using provided selectors
for selector_name, selector in selectors.items():
elements = soup.select(selector)
if elements:
if format == "text":
extracted_content[selector_name] = [elem.get_text(strip=True) for elem in elements]
elif format == "html":
extracted_content[selector_name] = [str(elem) for elem in elements]
else:
extracted_content[selector_name] = [elem.get_text(strip=True) for elem in elements]
else:
extracted_content[selector_name] = []
else:
# Auto-extract common elements
extracted_content = {
"title": self._extractTitle(soup, url),
"main_content": self._extractMainContent(soup),
"headings": [h.get_text(strip=True) for h in soup.find_all(['h1', 'h2', 'h3'])],
"links": [a.get('href') for a in soup.find_all('a', href=True) if a.get('href').startswith(('http://', 'https://'))],
"images": [img.get('src') for img in soup.find_all('img', src=True)]
}
return {
"url": url,
"selectors": selectors,
"format": format,
"content": extracted_content,
"timestamp": datetime.now(UTC).isoformat()
}
except Exception as e:
logger.error(f"Error extracting content: {str(e)}")
return {
"error": str(e),
"url": url
}
async def validatePage(self, url: str, checks: List[str] = None) -> Dict[str, Any]:
"""Validate web page for various criteria"""
if checks is None:
checks = ["accessibility", "seo", "performance"]
headers = {
'User-Agent': self.user_agent,
'Accept': 'text/html,application/xhtml+xml,application/xml',
'Accept-Language': 'en-US,en;q=0.9',
}
try:
# Read the URL
soup = self._readUrl(url)
if not soup:
return {
"error": "Failed to read URL",
"url": url
}
# Initial request
response = requests.get(url, headers=headers, timeout=self.timeout)
validation_results = {}
# Handling for status 202
if response.status_code == 202:
# Retry with backoff
backoff_times = [0.5, 1.0, 2.0, 5.0]
for wait_time in backoff_times:
time.sleep(wait_time)
response = requests.get(url, headers=headers, timeout=self.timeout)
if response.status_code != 202:
break
for check in checks:
if check == "accessibility":
validation_results["accessibility"] = self._checkAccessibility(soup)
elif check == "seo":
validation_results["seo"] = self._checkSEO(soup)
elif check == "performance":
validation_results["performance"] = self._checkPerformance(soup, url)
else:
validation_results[check] = {"status": "unknown", "message": f"Unknown check type: {check}"}
# Raise for error status codes
response.raise_for_status()
return {
"url": url,
"checks": checks,
"results": validation_results,
"timestamp": datetime.now(UTC).isoformat()
}
# Parse HTML
return BeautifulSoup(response.text, 'html.parser')
except Exception as e:
logger.error(f"Error validating web page: {str(e)}")
return {
"error": str(e),
"url": url
}
logger.error(f"Error reading URL {url}: {str(e)}")
return None
def _extractTitle(self, soup: BeautifulSoup, url: str) -> str:
"""Extract the title from a webpage"""
if not soup:
return f"Error with {url}"
# Extract title from title tag
title_tag = soup.find('title')
title = title_tag.text.strip() if title_tag else "No title"
# Alternative: Also look for h1 tags if title tag is missing
if title == "No title":
h1_tag = soup.find('h1')
if h1_tag:
title = h1_tag.text.strip()
return title
def _extractMainContent(self, soup: BeautifulSoup, max_chars: int = 10000) -> str:
"""Extract the main content from an HTML page"""
if not soup:
return ""
# Try to find main content elements in priority order
main_content = None
for selector in ['main', 'article', '#content', '.content', '#main', '.main']:
content = soup.select_one(selector)
if content:
main_content = content
break
# If no main content found, use the body
if not main_content:
main_content = soup.find('body') or soup
# Remove script, style, nav, footer elements that don't contribute to main content
for element in main_content.select('script, style, nav, footer, header, aside, .sidebar, #sidebar, .comments, #comments, .advertisement, .ads, iframe'):
element.extract()
# Extract text content
text_content = main_content.get_text(separator=' ', strip=True)
# Limit to max_chars
return text_content[:max_chars]
def _checkAccessibility(self, soup: BeautifulSoup) -> Dict[str, Any]:
"""Check basic accessibility features"""
@ -355,97 +214,204 @@ class WebService:
}
}
def _readUrl(self, url: str) -> BeautifulSoup:
"""Read a URL and return a BeautifulSoup parser for the content"""
if not url or not url.startswith(('http://', 'https://')):
return None
headers = {
'User-Agent': self.user_agent,
'Accept': 'text/html,application/xhtml+xml,application/xml',
'Accept-Language': 'en-US,en;q=0.9',
}
@action
async def crawl(self, parameters: Dict[str, Any]) -> ActionResult:
"""
Crawl web pages and extract content
Parameters:
urls (List[str]): List of URLs to crawl
maxDepth (int, optional): Maximum crawl depth (default: 2)
includeImages (bool, optional): Whether to include images (default: False)
followLinks (bool, optional): Whether to follow links (default: True)
"""
try:
# Initial request
response = requests.get(url, headers=headers, timeout=self.timeout)
urls = parameters.get("urls")
maxDepth = parameters.get("maxDepth", 2)
includeImages = parameters.get("includeImages", False)
followLinks = parameters.get("followLinks", True)
# Handling for status 202
if response.status_code == 202:
# Retry with backoff
backoff_times = [0.5, 1.0, 2.0, 5.0]
for wait_time in backoff_times:
time.sleep(wait_time)
response = requests.get(url, headers=headers, timeout=self.timeout)
if not urls:
return self._createResult(
success=False,
data={},
error="URLs are required"
)
# Crawl each URL
crawl_results = []
for url in urls:
try:
# Read the URL
soup = self._readUrl(url)
if not soup:
crawl_results.append({
"error": "Failed to read URL",
"url": url
})
continue
if response.status_code != 202:
break
# Extract basic information
title = self._extractTitle(soup, url)
content = self._extractMainContent(soup) if True else ""
# Extract links if requested
links = []
if followLinks:
for link in soup.find_all('a', href=True):
href = link.get('href')
if href and href.startswith(('http://', 'https://')):
links.append({
'url': href,
'text': link.get_text(strip=True)[:100]
})
# Extract images
images = []
for img in soup.find_all('img', src=True):
src = img.get('src')
if src:
images.append({
'src': src,
'alt': img.get('alt', ''),
'title': img.get('title', '')
})
crawl_results.append({
"url": url,
"depth": maxDepth,
"followLinks": followLinks,
"extractContent": True,
"title": title,
"content": content,
"links": links[:10], # Limit to first 10 links
"images": images[:10], # Limit to first 10 images
"timestamp": datetime.now(UTC).isoformat()
})
except Exception as e:
logger.error(f"Error crawling web page {url}: {str(e)}")
crawl_results.append({
"error": str(e),
"url": url
})
# Raise for error status codes
response.raise_for_status()
# Create result data
result_data = {
"urls": urls,
"maxDepth": maxDepth,
"includeImages": includeImages,
"followLinks": followLinks,
"crawlResults": crawl_results,
"timestamp": datetime.now(UTC).isoformat()
}
# Parse HTML
return BeautifulSoup(response.text, 'html.parser')
return self._createResult(
success=True,
data={
"documentName": f"web_crawl_{datetime.now(UTC).strftime('%Y%m%d_%H%M%S')}.json",
"documentData": result_data
}
)
except Exception as e:
logger.error(f"Error reading URL {url}: {str(e)}")
return None
logger.error(f"Error crawling web pages: {str(e)}")
return self._createResult(
success=False,
data={},
error=str(e)
)
def _extractTitle(self, soup: BeautifulSoup, url: str) -> str:
"""Extract the title from a webpage"""
if not soup:
return f"Error with {url}"
@action
async def scrape(self, parameters: Dict[str, Any]) -> ActionResult:
"""
Scrape specific data from web pages
# Extract title from title tag
title_tag = soup.find('title')
title = title_tag.text.strip() if title_tag else "No title"
# Alternative: Also look for h1 tags if title tag is missing
if title == "No title":
h1_tag = soup.find('h1')
if h1_tag:
title = h1_tag.text.strip()
return title
Parameters:
url (str): URL to scrape
selectors (Dict[str, str]): CSS selectors for data extraction
format (str, optional): Output format (default: "json")
"""
try:
url = parameters.get("url")
selectors = parameters.get("selectors")
format = parameters.get("format", "json")
if not url or not selectors:
return self._createResult(
success=False,
data={},
error="URL and selectors are required"
)
# Read the URL
soup = self._readUrl(url)
if not soup:
return self._createResult(
success=False,
data={},
error="Failed to read URL"
)
extracted_content = {}
if selectors:
# Extract content using provided selectors
for selector_name, selector in selectors.items():
elements = soup.select(selector)
if elements:
if format == "text":
extracted_content[selector_name] = [elem.get_text(strip=True) for elem in elements]
elif format == "html":
extracted_content[selector_name] = [str(elem) for elem in elements]
else:
extracted_content[selector_name] = [elem.get_text(strip=True) for elem in elements]
else:
extracted_content[selector_name] = []
else:
# Auto-extract common elements
extracted_content = {
"title": self._extractTitle(soup, url),
"main_content": self._extractMainContent(soup),
"headings": [h.get_text(strip=True) for h in soup.find_all(['h1', 'h2', 'h3'])],
"links": [a.get('href') for a in soup.find_all('a', href=True) if a.get('href').startswith(('http://', 'https://'))],
"images": [img.get('src') for img in soup.find_all('img', src=True)]
}
scrape_result = {
"url": url,
"selectors": selectors,
"format": format,
"content": extracted_content,
"timestamp": datetime.now(UTC).isoformat()
}
# Create result data
result_data = {
"url": url,
"selectors": selectors,
"format": format,
"scrapedData": scrape_result,
"timestamp": datetime.now(UTC).isoformat()
}
return self._createResult(
success=True,
data={
"documentName": f"web_scrape_{datetime.now(UTC).strftime('%Y%m%d_%H%M%S')}.{format}",
"documentData": result_data
}
)
except Exception as e:
logger.error(f"Error scraping web page: {str(e)}")
return self._createResult(
success=False,
data={},
error=str(e)
)
def _extractMainContent(self, soup: BeautifulSoup, max_chars: int = 10000) -> str:
"""Extract the main content from an HTML page"""
if not soup:
return ""
# Try to find main content elements in priority order
main_content = None
for selector in ['main', 'article', '#content', '.content', '#main', '.main']:
content = soup.select_one(selector)
if content:
main_content = content
break
# If no main content found, use the body
if not main_content:
main_content = soup.find('body') or soup
# Remove script, style, nav, footer elements that don't contribute to main content
for element in main_content.select('script, style, nav, footer, header, aside, .sidebar, #sidebar, .comments, #comments, .advertisement, .ads, iframe'):
element.extract()
# Extract text content
text_content = main_content.get_text(separator=' ', strip=True)
# Limit to max_chars
return text_content[:max_chars]
class MethodWeb(MethodBase):
"""Web method implementation for web operations"""
def __init__(self, serviceContainer: Any):
"""Initialize the web method"""
super().__init__(serviceContainer)
self.name = "web"
self.description = "Handle web operations like searching and crawling"
self.webService = WebService(serviceContainer)
@action
async def search(self, parameters: Dict[str, Any]) -> ActionResult:
"""
@ -453,11 +419,15 @@ class MethodWeb(MethodBase):
Parameters:
query (str): Search query
engine (str, optional): Search engine to use (default: "google")
maxResults (int, optional): Maximum number of results (default: 10)
filter (str, optional): Additional search filters
"""
try:
query = parameters.get("query")
engine = parameters.get("engine", "google")
maxResults = parameters.get("maxResults", 10)
filter = parameters.get("filter")
if not query:
return self._createResult(
@ -466,15 +436,101 @@ class MethodWeb(MethodBase):
error="Search query is required"
)
# Search web
results = await self.webService.searchWeb(
query=query,
maxResults=maxResults
)
# Search web content using Google search via SerpAPI
try:
if not self.srcApikey:
search_result = {
"error": "SerpAPI key not configured",
"query": query
}
else:
# Get user language from service container if available
userLanguage = "en" # Default language
if hasattr(self.serviceContainer, 'user') and hasattr(self.serviceContainer.user, 'language'):
userLanguage = self.serviceContainer.user.language
# Format the search request for SerpAPI
params = {
"engine": self.srcEngine,
"q": query,
"api_key": self.srcApikey,
"num": min(maxResults, self.maxResults), # Number of results to return
"hl": userLanguage # User language
}
# Make the API request
response = requests.get("https://serpapi.com/search", params=params, timeout=self.timeout)
response.raise_for_status()
# Parse JSON response
search_results = response.json()
# Extract organic results
results = []
if "organic_results" in search_results:
for result in search_results["organic_results"][:maxResults]:
# Extract title
title = result.get("title", "No title")
# Extract URL
url = result.get("link", "No URL")
# Extract snippet
snippet = result.get("snippet", "No description")
# Get actual page content
try:
targetPageSoup = self._readUrl(url)
content = self._extractMainContent(targetPageSoup)
except Exception as e:
logger.warning(f"Error extracting content from {url}: {str(e)}")
content = f"Error extracting content: {str(e)}"
results.append({
'title': title,
'url': url,
'snippet': snippet,
'content': content
})
# Limit number of results
if len(results) >= maxResults:
break
else:
logger.warning(f"No organic results found in SerpAPI response for: {query}")
search_result = {
"query": query,
"maxResults": maxResults,
"results": results,
"totalFound": len(results),
"timestamp": datetime.now(UTC).isoformat()
}
except Exception as e:
logger.error(f"Error searching web: {str(e)}")
search_result = {
"error": str(e),
"query": query
}
# Create result data
result_data = {
"query": query,
"engine": engine,
"maxResults": maxResults,
"filter": filter,
"searchResults": search_result,
"timestamp": datetime.now(UTC).isoformat()
}
return self._createResult(
success=True,
data=results
data={
"documentName": f"web_search_{datetime.now(UTC).strftime('%Y%m%d_%H%M%S')}.json",
"documentData": result_data
}
)
except Exception as e:
@ -484,98 +540,11 @@ class MethodWeb(MethodBase):
data={},
error=str(e)
)
@action
async def crawl(self, parameters: Dict[str, Any]) -> ActionResult:
"""
Crawl web page
Parameters:
url (str): URL to crawl
depth (int, optional): Crawl depth (default: 1)
followLinks (bool, optional): Whether to follow links (default: True)
extractContent (bool, optional): Whether to extract content (default: True)
"""
try:
url = parameters.get("url")
depth = parameters.get("depth", 1)
followLinks = parameters.get("followLinks", True)
extractContent = parameters.get("extractContent", True)
if not url:
return self._createResult(
success=False,
data={},
error="URL is required"
)
# Crawl page
results = await self.webService.crawlPage(
url=url,
depth=depth,
followLinks=followLinks,
extractContent=extractContent
)
return self._createResult(
success=True,
data=results
)
except Exception as e:
logger.error(f"Error crawling web page: {str(e)}")
return self._createResult(
success=False,
data={},
error=str(e)
)
@action
async def extract(self, parameters: Dict[str, Any]) -> ActionResult:
"""
Extract content from web page
Parameters:
url (str): URL to extract content from
selectors (Dict[str, str], optional): CSS selectors for specific content
format (str, optional): Output format (default: "text")
"""
try:
url = parameters.get("url")
selectors = parameters.get("selectors", {})
format = parameters.get("format", "text")
if not url:
return self._createResult(
success=False,
data={},
error="URL is required"
)
# Extract content
content = await self.webService.extractContent(
url=url,
selectors=selectors,
format=format
)
return self._createResult(
success=True,
data=content
)
except Exception as e:
logger.error(f"Error extracting content: {str(e)}")
return self._createResult(
success=False,
data={},
error=str(e)
)
@action
async def validate(self, parameters: Dict[str, Any]) -> ActionResult:
"""
Validate web page
Validate web pages for various criteria
Parameters:
url (str): URL to validate
@ -592,15 +561,48 @@ class MethodWeb(MethodBase):
error="URL is required"
)
# Validate page
results = await self.webService.validatePage(
url=url,
checks=checks
)
# Read the URL
soup = self._readUrl(url)
if not soup:
return self._createResult(
success=False,
data={},
error="Failed to read URL"
)
validation_results = {}
for check in checks:
if check == "accessibility":
validation_results["accessibility"] = self._checkAccessibility(soup)
elif check == "seo":
validation_results["seo"] = self._checkSEO(soup)
elif check == "performance":
validation_results["performance"] = self._checkPerformance(soup, url)
else:
validation_results[check] = {"status": "unknown", "message": f"Unknown check type: {check}"}
validation_result = {
"url": url,
"checks": checks,
"results": validation_results,
"timestamp": datetime.now(UTC).isoformat()
}
# Create result data
result_data = {
"url": url,
"checks": checks,
"validationResult": validation_result,
"timestamp": datetime.now(UTC).isoformat()
}
return self._createResult(
success=True,
data=results
data={
"documentName": f"web_validation_{datetime.now(UTC).strftime('%Y%m%d_%H%M%S')}.json",
"documentData": result_data
}
)
except Exception as e:

1316
modules/workflow/managerChat.py
View file

File diff suppressed because it is too large Load diff

498
modules/workflow/managerWorkflow.py
View file

@ -29,383 +29,8 @@ class WorkflowManager:
if workflow.status == "stopped":
raise WorkflowStoppedException("Workflow was stopped by user")
async def workflowProcess(self, userInput: UserInputRequest, workflow: ChatWorkflow) -> TaskItem:
"""Enhanced workflow process with proper task planning and handover review"""
try:
logger.info(f"Processing workflow: {workflow.id}")
# Phase 1: Create initial message with user request and documents
initial_message = await self._createInitialMessage(userInput, workflow)
if not initial_message:
raise Exception("Failed to create initial message")
# Phase 2: Generate task plan through AI analysis
task_plan = await self._generateTaskPlan(userInput, workflow, initial_message)
if not task_plan:
raise Exception("Failed to generate task plan")
# Phase 3: Execute tasks with handover review
task_result = await self._executeTaskPlan(task_plan, workflow, userInput)
return task_result
except Exception as e:
logger.error(f"Error in workflowProcess: {str(e)}")
raise
async def _createInitialMessage(self, userInput: UserInputRequest, workflow: ChatWorkflow) -> ChatMessage:
"""Create initial message with user request and processed documents"""
try:
# Initialize chat manager with workflow
await self.chatManager.initialize(workflow)
# Process file IDs into ChatDocument objects
documents = await self.chatManager.processFileIds(userInput.listFileId)
# Create message data
message_data = {
"id": f"msg_{uuid.uuid4()}",
"workflowId": workflow.id,
"role": "user",
"agentName": "",
"message": userInput.prompt,
"documents": documents,
"status": "step",
"publishedAt": self._getCurrentTimestamp()
}
# Create message in database
message = self.chatInterface.createWorkflowMessage(message_data)
if not message:
raise Exception("Failed to create workflow message")
logger.info(f"Created initial message: {message.id} with {len(documents)} documents")
return message
except Exception as e:
logger.error(f"Error creating initial message: {str(e)}")
return None
async def _generateTaskPlan(self, userInput: UserInputRequest, workflow: ChatWorkflow, initial_message: ChatMessage) -> Dict[str, Any]:
"""Generate task plan through AI analysis"""
try:
# Prepare context for AI analysis
context = {
"user_request": userInput.prompt,
"available_documents": [doc.filename for doc in initial_message.documents],
"workflow_id": workflow.id,
"message_id": initial_message.id
}
# Generate task plan using AI
task_plan = await self.chatManager.generateTaskPlan(context)
logger.info(f"Generated task plan with {len(task_plan.get('tasks', []))} tasks")
return task_plan
except Exception as e:
logger.error(f"Error generating task plan: {str(e)}")
return None
async def _executeTaskPlan(self, task_plan: Dict[str, Any], workflow: ChatWorkflow, userInput: UserInputRequest) -> TaskItem:
"""Execute task plan with handover review and enhanced error recovery"""
try:
tasks = task_plan.get('tasks', [])
if not tasks:
raise Exception("No tasks in task plan")
# Create main task item
task_data = {
"id": f"task_{uuid.uuid4()}",
"workflowId": workflow.id,
"userInput": userInput.prompt,
"status": TaskStatus.RUNNING,
"feedback": task_plan.get('overview', 'Executing task plan'),
"startedAt": self._getCurrentTimestamp(),
"actionList": [],
"taskPlan": task_plan
}
task = self.chatInterface.createTask(task_data)
if not task:
raise Exception("Failed to create task")
# Ensure task is saved to database
logger.info(f"Created task with ID: {task.id}")
# Execute each task with enhanced error recovery
for i, task_step in enumerate(tasks):
logger.info(f"Executing task {i+1}/{len(tasks)}: {task_step.get('description', 'Unknown')}")
# Execute task step with retry mechanism
step_result = await self._executeTaskStepWithRetry(task_step, workflow, task)
# Enhanced handover review
review_result = await self._performEnhancedHandoverReview(step_result, task_step, workflow, task)
if review_result['status'] == 'failed':
# Try alternative approach before giving up
alternative_result = await self._tryAlternativeApproach(task_step, workflow, task, review_result)
if alternative_result['status'] == 'failed':
# Update task status with detailed feedback
update_result = self.chatInterface.updateTask(task.id, {
"status": TaskStatus.FAILED,
"feedback": f"Task failed at step {i+1}: {review_result['reason']}. Alternative approach also failed.",
"errorDetails": {
"failedStep": task_step,
"originalError": review_result['reason'],
"suggestions": self._generateFailureSuggestions(task_step, review_result)
}
})
if not update_result:
logger.error(f"Failed to update task {task.id} status to FAILED")
return task
else:
step_result = alternative_result
review_result = {'status': 'success'}
elif review_result['status'] == 'retry':
# Retry with improved approach
logger.info(f"Retrying task step {i+1} with improved approach")
step_result = await self._executeTaskStepWithRetry(task_step, workflow, task, improvements=review_result.get('improvements'))
review_result = await self._performEnhancedHandoverReview(step_result, task_step, workflow, task)
if review_result['status'] == 'failed':
update_result = self.chatInterface.updateTask(task.id, {
"status": TaskStatus.FAILED,
"feedback": f"Task failed after retry at step {i+1}: {review_result['reason']}"
})
if not update_result:
logger.error(f"Failed to update task {task.id} status to FAILED after retry")
return task
# Add step result to task
if step_result and step_result.get('actions'):
for action in step_result['actions']:
# Convert action format to TaskAction format
task_action_data = {
"execMethod": action.get('method', 'unknown'),
"execAction": action.get('action', 'unknown'),
"execParameters": action.get('parameters', {}),
"execResultLabel": action.get('resultLabel', ''),
"status": TaskStatus.PENDING
}
task_action = self.chatInterface.createTaskAction(task_action_data)
if task_action:
task.actionList.append(task_action)
logger.info(f"Created task action: {task_action.execMethod}.{task_action.execAction}")
else:
logger.error(f"Failed to create task action: {action}")
# Update progress
self._updateTaskProgress(task, i + 1, len(tasks))
# Update task as completed
update_result = self.chatInterface.updateTask(task.id, {
"status": TaskStatus.COMPLETED,
"feedback": f"Successfully completed {len(tasks)} tasks with {len(task.actionList)} total actions",
"finishedAt": self._getCurrentTimestamp(),
"successMetrics": {
"totalTasks": len(tasks),
"totalActions": len(task.actionList),
"executionTime": self._calculateExecutionTime(task.startedAt)
}
})
if not update_result:
logger.error(f"Failed to update task {task.id} status to COMPLETED")
return task
except Exception as e:
logger.error(f"Error executing task plan: {str(e)}")
raise
async def _executeTaskStepWithRetry(self, task_step: Dict[str, Any], workflow: ChatWorkflow, task: TaskItem, max_retries: int = 3, improvements: str = None) -> Dict[str, Any]:
"""Execute task step with exponential backoff retry mechanism"""
last_error = None
for attempt in range(max_retries + 1):
try:
# Add exponential backoff delay for retries
if attempt > 0:
delay = min(2 ** attempt, 30) # Max 30 seconds
await asyncio.sleep(delay)
logger.info(f"Retry attempt {attempt} for task step: {task_step.get('description', 'Unknown')}")
# Execute task step
step_result = await self._executeTaskStep(task_step, workflow, task, improvements)
# Quick validation
if step_result.get('status') == 'completed':
return step_result
else:
last_error = step_result.get('error', 'Unknown error')
except Exception as e:
last_error = str(e)
logger.warning(f"Attempt {attempt + 1} failed for task step: {str(e)}")
# All retries exhausted
return {
'task_step': task_step,
'error': f"All {max_retries + 1} attempts failed. Last error: {last_error}",
'status': 'failed',
'retryAttempts': max_retries + 1
}
async def _performEnhancedHandoverReview(self, step_result: Dict[str, Any], task_step: Dict[str, Any], workflow: ChatWorkflow, task: TaskItem) -> Dict[str, Any]:
"""Enhanced handover review with quality assessment"""
try:
# Prepare enhanced review context
review_context = {
'task_step': task_step,
'step_result': step_result,
'workflow_id': workflow.id,
'task_id': task.id,
'previous_results': self._getPreviousResults(task)
}
# Use AI to review the results
review = await self.chatManager.reviewTaskStepResults(review_context)
# Add quality metrics
review['quality_metrics'] = await self._calculateQualityMetrics(step_result, task_step)
return review
except Exception as e:
logger.error(f"Error in enhanced handover review: {str(e)}")
return {
'status': 'failed',
'reason': f'Review failed: {str(e)}',
'quality_metrics': {'score': 0, 'confidence': 0}
}
async def _tryAlternativeApproach(self, task_step: Dict[str, Any], workflow: ChatWorkflow, task: TaskItem, original_review: Dict[str, Any]) -> Dict[str, Any]:
"""Try alternative approach when original method fails"""
try:
logger.info(f"Trying alternative approach for task step: {task_step.get('description', 'Unknown')}")
# Generate alternative approach based on failure analysis
alternative_prompt = self._createAlternativeApproachPrompt(task_step, original_review)
alternative_response = await self.chatManager._callAI(alternative_prompt, "alternative_approach")
# Parse alternative approach
alternative_approach = self._parseAlternativeApproach(alternative_response)
if alternative_approach:
# Execute alternative approach
step_result = await self._executeTaskStep(task_step, workflow, task, alternative_approach)
return step_result
else:
return {
'task_step': task_step,
'error': 'Could not generate alternative approach',
'status': 'failed'
}
except Exception as e:
logger.error(f"Error trying alternative approach: {str(e)}")
return {
'task_step': task_step,
'error': f'Alternative approach failed: {str(e)}',
'status': 'failed'
}
def _generateFailureSuggestions(self, task_step: Dict[str, Any], review_result: Dict[str, Any]) -> List[str]:
"""Generate helpful suggestions when tasks fail"""
suggestions = []
if 'missing_outputs' in review_result:
suggestions.append(f"Ensure all expected outputs are produced: {', '.join(review_result['missing_outputs'])}")
if 'unmet_criteria' in review_result:
suggestions.append(f"Address unmet success criteria: {', '.join(review_result['unmet_criteria'])}")
suggestions.append("Check if all required documents are available and accessible")
suggestions.append("Verify that the task step has all necessary dependencies completed")
return suggestions
async def _calculateQualityMetrics(self, step_result: Dict[str, Any], task_step: Dict[str, Any]) -> Dict[str, Any]:
"""Calculate quality metrics for task step results"""
try:
quality_score = 0
confidence = 0
if step_result.get('status') == 'completed':
quality_score = 8 # Base score for completion
# Check if all expected outputs were produced
expected_outputs = task_step.get('expected_outputs', [])
produced_outputs = step_result.get('outputs', [])
output_coverage = len(set(produced_outputs) & set(expected_outputs)) / len(expected_outputs) if expected_outputs else 1
quality_score += output_coverage * 2
confidence = min(quality_score / 10, 1.0)
return {
'score': min(quality_score, 10),
'confidence': confidence
}
except Exception as e:
logger.error(f"Error calculating quality metrics: {str(e)}")
return {'score': 0, 'confidence': 0}
def _updateTaskProgress(self, task: TaskItem, current_step: int, total_steps: int):
"""Update task progress information"""
progress = (current_step / total_steps) * 100
logger.info(f"Task progress: {progress:.1f}% ({current_step}/{total_steps})")
def _calculateExecutionTime(self, started_at: str) -> float:
"""Calculate execution time in seconds"""
try:
start_time = datetime.fromisoformat(started_at.replace('Z', '+00:00'))
end_time = datetime.now(UTC)
return (end_time - start_time).total_seconds()
except Exception:
return 0.0
def _getPreviousResults(self, task: TaskItem) -> List[str]:
"""Get list of previous results from completed actions"""
results = []
for action in task.actionList:
if action.execResultLabel:
results.append(action.execResultLabel)
return results
def _createAlternativeApproachPrompt(self, task_step: Dict[str, Any], original_review: Dict[str, Any]) -> str:
"""Create prompt for generating alternative approaches"""
return f"""The original approach for this task step failed. Please suggest an alternative approach.
TASK STEP: {task_step.get('description', 'Unknown')}
ORIGINAL FAILURE: {original_review.get('reason', 'Unknown error')}
MISSING OUTPUTS: {', '.join(original_review.get('missing_outputs', []))}
Please provide an alternative approach that addresses these issues."""
def _parseAlternativeApproach(self, response: str) -> Optional[str]:
"""Parse alternative approach from AI response"""
try:
# Simple parsing - extract the approach description
if "approach:" in response.lower():
lines = response.split('\n')
for line in lines:
if "approach:" in line.lower():
return line.split(":", 1)[1].strip()
return None
except Exception:
return None
def _getCurrentTimestamp(self) -> str:
"""Get current timestamp in ISO format"""
return datetime.now(UTC).isoformat()
async def workflowProcess_ORIGINAL_TEMPORARY_DEACTIVATED(self, userInput: UserInputRequest, workflow: ChatWorkflow) -> None:
"""Process a workflow with user input"""
async def workflowProcess(self, userInput: UserInputRequest, workflow: ChatWorkflow) -> None:
"""Process a workflow with user input using unified workflow phases"""
try:
# Initialize chat manager
await self.chatManager.initialize(workflow)
@ -416,31 +41,11 @@ Please provide an alternative approach that addresses these issues."""
# Send first message
message = await self._sendFirstMessage(userInput, workflow)
# Create initial task
task = await self.chatManager.createInitialTask(workflow, message)
# Execute unified workflow
workflow_result = await self.chatManager.executeUnifiedWorkflow(userInput.prompt, workflow)
# Process workflow
while True:
# Check if workflow is stopped
self._checkWorkflowStopped(workflow)
# Execute task
result = await self.chatManager.executeTask(task)
# Process result
await self.chatManager.parseTaskResult(workflow, result)
# Check if workflow should continue
if not await self.chatManager.shouldContinue(workflow):
break
# Identify next task
nextTaskResult = await self.chatManager.identifyNextTask(workflow)
# Create next task
task = await self.chatManager.createNextTask(workflow, nextTaskResult)
if not task:
break
# Process workflow results
await self._processWorkflowResults(workflow, workflow_result, message)
# Send last message
await self._sendLastMessage(workflow)
@ -507,29 +112,80 @@ Please provide an alternative approach that addresses these issues."""
logger.error(f"Error sending last message: {str(e)}")
raise
async def _executeTaskStep(self, task_step: Dict[str, Any], workflow: ChatWorkflow, task: TaskItem, improvements: str = None) -> Dict[str, Any]:
"""Execute a single task step and generate actions"""
async def _processWorkflowResults(self, workflow: ChatWorkflow, workflow_result: Dict[str, Any], initial_message: ChatMessage) -> None:
"""Process workflow results and create appropriate messages"""
try:
# Generate actions for this task step
actions = await self.chatManager.generateActionsForTask(task_step, workflow, task, improvements)
if workflow_result.get('status') == 'failed':
# Create error message
error_message = {
"workflowId": workflow.id,
"role": "assistant",
"message": f"Workflow failed: {workflow_result.get('error', 'Unknown error')}",
"status": "last",
"sequenceNr": len(workflow.messages) + 1,
"publishedAt": datetime.now(UTC).isoformat()
}
message = self.chatInterface.createWorkflowMessage(error_message)
if message:
workflow.messages.append(message)
return
# Execute actions
results = []
for action in actions:
action_result = await self.chatManager.executeAction(action, workflow)
results.append(action_result)
# Process successful workflow results
workflow_results = workflow_result.get('workflow_results', [])
return {
'task_step': task_step,
'actions': actions,
'results': results,
'status': 'completed'
for i, result in enumerate(workflow_results):
task_step = result['task_step']
action_results = result['action_results']
review_result = result['review_result']
# Create message for task step
step_message = {
"workflowId": workflow.id,
"role": "assistant",
"message": f"Completed task: {task_step.get('description', 'Unknown')}",
"status": "step",
"sequenceNr": len(workflow.messages) + 1,
"publishedAt": datetime.now(UTC).isoformat()
}
# Add action details if available
if action_results:
successful_actions = [r for r in action_results if r.get('status') == 'completed']
step_message["message"] += f"\nExecuted {len(successful_actions)}/{len(action_results)} actions successfully."
message = self.chatInterface.createWorkflowMessage(step_message)
if message:
workflow.messages.append(message)
# Create final summary message
successful_tasks = workflow_result.get('successful_tasks', 0)
total_tasks = workflow_result.get('total_tasks', 0)
summary_message = {
"workflowId": workflow.id,
"role": "assistant",
"message": f"Workflow completed successfully. Completed {successful_tasks}/{total_tasks} tasks.",
"status": "last",
"sequenceNr": len(workflow.messages) + 1,
"publishedAt": datetime.now(UTC).isoformat()
}
message = self.chatInterface.createWorkflowMessage(summary_message)
if message:
workflow.messages.append(message)
except Exception as e:
logger.error(f"Error executing task step: {str(e)}")
return {
'task_step': task_step,
'error': str(e),
'status': 'failed'
}
logger.error(f"Error processing workflow results: {str(e)}")
# Create error message
error_message = {
"workflowId": workflow.id,
"role": "assistant",
"message": f"Error processing workflow results: {str(e)}",
"status": "last",
"sequenceNr": len(workflow.messages) + 1,
"publishedAt": datetime.now(UTC).isoformat()
}
message = self.chatInterface.createWorkflowMessage(error_message)
if message:
workflow.messages.append(message)

86
modules/workflow/methodBase.py
View file

@ -5,7 +5,7 @@ from pydantic import BaseModel, Field
import logging
from modules.interfaces.interfaceChatModel import ActionResult
from functools import wraps
from inspect import signature
import inspect
logger = logging.getLogger(__name__)
@ -38,7 +38,7 @@ class MethodBase:
try:
attr = getattr(self, attr_name)
if callable(attr) and getattr(attr, 'is_action', False):
sig = signature(attr)
sig = inspect.signature(attr)
params = {}
for param_name, param in sig.parameters.items():
if param_name not in ['self', 'parameters', 'authData']:
@ -60,23 +60,91 @@ class MethodBase:
return actions
def getActionSignature(self, actionName: str) -> str:
"""Get formatted action signature for AI prompt generation"""
"""Get formatted action signature for AI prompt generation (detailed version)"""
if actionName not in self.actions:
return ""
action = self.actions[actionName]
paramList = []
for paramName, param in action['parameters'].items():
paramType = self._formatType(param['type'])
paramList.append(f"{paramName}:{paramType}")
# Extract detailed parameter information from docstring
docstring = action.get('description', '')
paramDescriptions, paramTypes = self._extractParameterDetails(docstring)
signature = f"{self.name}.{actionName}([{', '.join(paramList)}])"
for paramName in paramDescriptions:
paramType = paramTypes.get(paramName, 'Any')
paramDesc = paramDescriptions.get(paramName, '')
# Mark required parameters with * if possible (not available from docstring, so omit)
if paramDesc:
paramList.append(f"{paramName}:{paramType} # {paramDesc}")
else:
paramList.append(f"{paramName}:{paramType}")
if action.get('description'):
signature += f" # {action['description']}"
signature = f"{self.name}.{actionName}"
if paramList:
signature += f"({', '.join(paramList)})"
# Add return type and main description
returnType = "ActionResult"
mainDesc = self._extractMainDescription(docstring)
if mainDesc:
signature += f" -> {returnType} # {mainDesc}"
return signature
def _extractParameterDetails(self, docstring: str):
"""Extract parameter names, types, and descriptions from docstring"""
descriptions = {}
types = {}
if not docstring:
return descriptions, types
lines = docstring.split('\n')
inParameters = False
for line in lines:
line = line.strip()
if 'Parameters:' in line:
inParameters = True
continue
elif inParameters and (line.startswith('Returns:') or line.startswith('Raises:') or line.startswith('Args:')):
break
elif inParameters and line:
# Look for parameter descriptions like "paramName (type): description"
if ':' in line and '(' in line:
parts = line.split(':', 1)
if len(parts) == 2:
paramPart = parts[0].strip()
descPart = parts[1].strip()
# Extract parameter name and type
if '(' in paramPart:
paramName = paramPart.split('(')[0].strip()
paramType = paramPart[paramPart.find('(')+1:paramPart.find(')')].strip()
descriptions[paramName] = descPart
types[paramName] = paramType
# Also handle multi-line descriptions
elif line and not line.startswith('Each document') and not line.startswith('contains'):
if descriptions:
lastParam = list(descriptions.keys())[-1]
descriptions[lastParam] += " " + line
return descriptions, types
def _extractMainDescription(self, docstring: str) -> str:
"""Extract main description from docstring"""
if not docstring:
return ""
lines = docstring.split('\n')
mainDesc = ""
for line in lines:
line = line.strip()
if line and not line.startswith('Parameters:') and not line.startswith('Returns:') and not line.startswith('Raises:'):
mainDesc = line
break
return mainDesc
def _formatType(self, type_annotation) -> str:
"""Format type annotation for display"""

42
modules/workflow/serviceContainer.py
View file

@ -146,7 +146,8 @@ class ServiceContainer:
if signature:
methodList.append(signature)
return methodList
def getDocumentReferenceList(self) -> Dict[str, List[Dict[str, str]]]:
"""Get list of document references sorted by datetime, categorized by chat round"""
chat_refs = []
@ -212,7 +213,7 @@ class ServiceContainer:
def getDocumentReferenceFromChatDocument(self, document: ChatDocument) -> str:
"""Get document reference from ChatDocument"""
return f"document_{document.id}_{document.filename}"
return f"cdoc:{document.id}:{document.filename}"
def getDocumentReferenceFromMessage(self, message: ChatMessage) -> str:
"""Get document reference from ChatMessage with action context"""
@ -220,17 +221,17 @@ class ServiceContainer:
return None
# If documentsLabel already contains the full reference format, return it
if message.documentsLabel.startswith("documentList_"):
if message.documentsLabel.startswith("mdoc:"):
return message.documentsLabel
# Otherwise construct the reference
return f"documentList_{message.actionId}_{message.documentsLabel}"
return f"mdoc:{message.actionId}:{message.documentsLabel}"
def getChatDocumentsFromDocumentReference(self, documentReference: str) -> List[ChatDocument]:
"""Get ChatDocuments from document reference"""
try:
# Parse reference format
parts = documentReference.split('_', 2) # Split into max 3 parts
parts = documentReference.split(':', 2) # Split into max 3 parts
if len(parts) < 3:
return []
@ -238,8 +239,8 @@ class ServiceContainer:
ref_id = parts[1]
ref_label = parts[2] # Keep the full label
if ref_type == "document":
# Handle ChatDocument reference: document_<id>_<filename>
if ref_type == "cdoc":
# Handle ChatDocument reference: cdoc:<id>:<filename>
# Find document in workflow messages
for message in self.workflow.messages:
if message.documents:
@ -247,8 +248,8 @@ class ServiceContainer:
if doc.id == ref_id:
return [doc]
elif ref_type == "documentList":
# Handle document list reference: documentList_<action.id>_<label>
elif ref_type == "mdoc":
# Handle document list reference: mdoc:<action.id>:<label>
# Find message with matching action ID and documents label
for message in self.workflow.messages:
if (message.actionId == ref_id and
@ -262,34 +263,31 @@ class ServiceContainer:
logger.error(f"Error getting documents from reference {documentReference}: {str(e)}")
return []
def getConnectionReferenceList(self) -> List[Dict[str, str]]:
def getConnectionReferenceList(self) -> List[str]:
"""Get list of all UserConnection objects as references"""
connections = []
# Get user connections through AppObjects interface
user_connections = self.interfaceApp.getUserConnections(self.user.id)
for conn in user_connections:
connections.append({
"connectionReference": f"connection_{conn.id}_{conn.authority}_{conn.externalUsername}",
"authority": conn.authority
})
# Sort by authority
return sorted(connections, key=lambda x: x["authority"])
connections.append(self.getConnectionReferenceFromUserConnection(conn))
# Sort by connection reference
return sorted(connections)
def getConnectionReferenceFromUserConnection(self, connection: UserConnection) -> str:
"""Get connection reference from UserConnection"""
return f"connection_{connection.id}_{connection.authority}_{connection.externalUsername}"
return f"connection:{connection.authority}:{connection.externalUsername}:{connection.id}"
def getUserConnectionFromConnectionReference(self, connectionReference: str) -> Optional[UserConnection]:
"""Get UserConnection from reference string"""
try:
# Parse reference format: connection_{id}_{authority}_{username}
parts = connectionReference.split('_')
# Parse reference format: connection:{authority}:{username}:{id}
parts = connectionReference.split(':')
if len(parts) != 4 or parts[0] != "connection":
return None
conn_id = parts[1]
authority = parts[2]
username = parts[3]
authority = parts[1]
username = parts[2]
conn_id = parts[3]
# Get user connections through AppObjects interface
user_connections = self.interfaceApp.getUserConnections(self.user.id)

226
notes/WORKFLOW_ARCHITECTURE.md Normal file
View file

@ -0,0 +1,226 @@
# Workflow Architecture Documentation
## Overview
The workflow system has been refactored into a clear, structured approach with 5 distinct phases. This eliminates redundancies and provides better error handling, quality assessment, and maintainability.
## Architecture Principles
### 1. **Clear Phase Separation**
Each workflow phase has a specific responsibility and clear inputs/outputs.
### 2. **Unified Data Model**
Standardized on `TaskAction` objects throughout the system.
### 3. **Consistent Prompt Generation**
All AI interactions use dedicated prompt generation functions.
### 4. **Quality Assessment**
Each task is reviewed before proceeding to the next.
## Workflow Phases
### **Phase 1: High-Level Task Planning**
**Function:** `planHighLevelTasks()`
**Purpose:** Analyze user request and create a structured task plan
**Input:** User input, available documents
**Output:** Task plan with multiple task steps
**Prompt Function:** `_createTaskPlanningPrompt()`
```python
task_plan = await chatManager.planHighLevelTasks(userInput, workflow)
```
### **Phase 2: Task Definition and Action Generation**
**Function:** `defineTaskActions()`
**Purpose:** Define specific actions for each task step
**Input:** Task step, workflow context, previous results
**Output:** List of TaskAction objects
**Prompt Function:** `_createActionDefinitionPrompt()`
```python
task_actions = await chatManager.defineTaskActions(task_step, workflow, previous_results)
```
### **Phase 3: Action Execution**
**Function:** `executeTaskActions()`
**Purpose:** Execute all actions for a task step
**Input:** List of TaskAction objects
**Output:** List of action results
**Prompt Function:** `_createActionExecutionPrompt()`
```python
action_results = await chatManager.executeTaskActions(task_actions, workflow)
```
### **Phase 4: Task Review and Quality Assessment**
**Function:** `reviewTaskCompletion()`
**Purpose:** Review task completion and decide next steps
**Input:** Task step, actions, results
**Output:** Review result with quality metrics
**Prompt Function:** `_createResultReviewPrompt()`
```python
review_result = await chatManager.reviewTaskCompletion(task_step, task_actions, action_results, workflow)
```
### **Phase 5: Task Handover and State Management**
**Function:** `prepareTaskHandover()`
**Purpose:** Prepare results for next task or workflow completion
**Input:** Task step, actions, review result
**Output:** Handover data for next iteration
**Prompt Function:** None (data processing only)
```python
handover_data = await chatManager.prepareTaskHandover(task_step, task_actions, review_result, workflow)
```
## Unified Workflow Execution
### **Main Entry Point**
**Function:** `executeUnifiedWorkflow()`
**Purpose:** Orchestrate all phases in sequence
**Input:** User input, workflow
**Output:** Complete workflow results
```python
workflow_result = await chatManager.executeUnifiedWorkflow(userInput.prompt, workflow)
```
### **Workflow Flow**
```
1. planHighLevelTasks() → Task Plan
2. For each task step:
├── defineTaskActions() → Task Actions
├── executeTaskActions() → Action Results
├── reviewTaskCompletion() → Review Result
└── prepareTaskHandover() → Handover Data
3. Return workflow summary
```
## Prompt Generation Functions
| **Function** | **Used In** | **Purpose** |
|-------------|-------------|-------------|
| `_createTaskPlanningPrompt()` | `planHighLevelTasks()` | Generate high-level task plan |
| `_createActionDefinitionPrompt()` | `defineTaskActions()` | Generate specific actions for task |
| `_createActionExecutionPrompt()` | `executeTaskActions()` | Execute individual actions |
| `_createResultReviewPrompt()` | `reviewTaskCompletion()` | Review task completion |
## Data Models
### **TaskAction Object**
```python
class TaskAction:
id: str
execMethod: str
execAction: str
execParameters: Dict[str, Any]
execResultLabel: Optional[str]
status: TaskStatus
error: Optional[str]
result: Optional[str]
# ... other fields
```
### **Workflow Result Structure**
```python
{
'status': 'completed' | 'partial' | 'failed',
'successful_tasks': int,
'total_tasks': int,
'workflow_results': List[Dict],
'final_results': List[str]
}
```
## Error Handling
### **Phase-Level Error Handling**
Each phase has its own error handling:
- **Planning:** Fallback to basic task plan
- **Definition:** Skip task if no actions defined
- **Execution:** Stop on first action failure
- **Review:** Default to success to avoid blocking
- **Handover:** Provide empty results on error
### **Circuit Breaker Pattern**
AI calls use circuit breaker pattern to prevent cascading failures.
## Quality Metrics
### **Task Quality Assessment**
- Success rate of actions
- Completion of expected outputs
- Meeting of success criteria
- Confidence scores
### **Workflow Quality Metrics**
- Overall success rate
- Task completion percentage
- Error patterns and suggestions
## Benefits of Refactored Architecture
### **1. Clear Separation of Concerns**
Each phase has a single responsibility and clear interfaces.
### **2. Better Error Handling**
Granular error handling at each phase with appropriate fallbacks.
### **3. Quality Assessment**
Built-in review and quality metrics for each task.
### **4. Maintainability**
Consistent patterns and unified data models.
### **5. Extensibility**
Easy to add new phases or modify existing ones.
### **6. Debugging**
Clear logging and error reporting at each phase.
## Migration Path
### **Legacy Methods**
All legacy methods are preserved for backward compatibility:
- `createInitialTask()`
- `createNextTask()`
- `executeTask()`
- `executeAction()`
### **New Unified Approach**
Use `executeUnifiedWorkflow()` for new implementations.
## Usage Example
```python
# Initialize chat manager
await chatManager.initialize(workflow)
# Execute unified workflow
workflow_result = await chatManager.executeUnifiedWorkflow(userInput.prompt, workflow)
# Process results
if workflow_result['status'] == 'completed':
print(f"Workflow completed: {workflow_result['successful_tasks']}/{workflow_result['total_tasks']} tasks")
else:
print(f"Workflow failed: {workflow_result['error']}")
```
## Future Enhancements
### **1. Retry Logic**
Add exponential backoff retry for failed tasks.
### **2. Alternative Approaches**
When primary method fails, try alternative approaches.
### **3. Parallel Execution**
Execute independent tasks in parallel.
### **4. Progress Tracking**
Real-time progress updates during workflow execution.
### **5. Rollback Mechanisms**
Undo failed operations and restore previous state.

34
notes/changelog.txt
View file

@ -1,11 +1,43 @@
INIT
conda activate poweron
pip install -r requirements.txt
cd gateway
pip install -r requirements.txt
python app.py
LOGIC
1. HIGH-LEVEL TASK PLANNING
├── Analyze user request
├── Define major task steps
└── Create task plan with dependencies
2. FOR EACH TASK STEP:
├── TASK DEFINITION
│ ├── Define specific actions for this task
│ └── Set success criteria
│
├── ACTION EXECUTION
│ ├── Execute each action
│ └── Collect results
│
├── TASK REVIEW
│ ├── Evaluate task completion
│ ├── Check success criteria
│ └── Decide: Continue/Retry/Fail
│
└── HANDOVER
├── Prepare results for next task
└── Update workflow state
TODO methods:
- reultLabel not to generate in the function, but to be set according to the action definition
- align documentList objects: chat document to have prefix "cdoc", message document list to have prefix "mdoc" (globally: instead of document and documentList)
- after action execution to store the documents in db and in a message object
TODO
- neutralizer to put back placeholders to the returned data
- referenceHandling and authentication for connections in the method actions

170
test_document_handover.py
View file

@ -1,170 +0,0 @@
#!/usr/bin/env python3
"""
Test script to demonstrate the improved document handover mechanism.
This shows how documents are properly stored and retrieved between actions.
"""
import sys
import os
sys.path.append(os.path.dirname(os.path.abspath(__file__)))
from modules.workflow.serviceContainer import ServiceContainer
from modules.interfaces.interfaceAppModel import User
from modules.interfaces.interfaceChatModel import ChatWorkflow, ChatMessage, ChatDocument
from datetime import datetime, UTC
import json
def test_document_handover():
"""Test the document handover mechanism"""
# Create test user and workflow
user = User(
id="test_user",
username="testuser",
language="en",
mandateId="test_mandate"
)
workflow = ChatWorkflow(
id="test_workflow",
mandateId="test_mandate",
startedAt=datetime.now(UTC).isoformat(),
status="active",
currentRound=1,
lastActivity=datetime.now(UTC).isoformat(),
messages=[]
)
# Create service container
container = ServiceContainer(user, workflow)
print("=" * 80)
print("DOCUMENT HANDOVER MECHANISM TEST")
print("=" * 80)
# Simulate action execution and document creation
print("\n1. SIMULATING ACTION EXECUTION")
print("-" * 40)
# Simulate first action: SharePoint search
action1_result = {
"result": "Found 5 sales documents in SharePoint",
"resultLabel": "documentList_abc123_sales_documents",
"documents": [
"document_001_sales_report_q1.xlsx",
"document_002_sales_report_q2.xlsx",
"document_003_sales_report_q3.xlsx"
],
"error": None
}
print(f"Action 1 Result: {json.dumps(action1_result, indent=2)}")
# Simulate second action: Excel analysis
action2_result = {
"result": "Analyzed sales data and created summary report",
"resultLabel": "documentList_def456_analysis_results",
"documents": [
"document_004_sales_analysis_summary.xlsx",
"document_005_sales_trends_chart.png"
],
"error": None
}
print(f"\nAction 2 Result: {json.dumps(action2_result, indent=2)}")
# Simulate workflow messages with documents
print("\n2. SIMULATING WORKFLOW MESSAGES")
print("-" * 40)
# Create mock messages to simulate the workflow
messages = []
# Message 1: SharePoint search result
message1 = ChatMessage(
id="msg_001",
workflowId=workflow.id,
role="assistant",
message=action1_result["result"],
status="step",
sequenceNr=1,
publishedAt=datetime.now(UTC).isoformat(),
actionId="action_001",
actionMethod="sharepoint",
actionName="search",
documentsLabel=action1_result["resultLabel"],
documents=[
ChatDocument(id="001", fileId="file_001", filename="sales_report_q1.xlsx", fileSize=1024, mimeType="application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"),
ChatDocument(id="002", fileId="file_002", filename="sales_report_q2.xlsx", fileSize=2048, mimeType="application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"),
ChatDocument(id="003", fileId="file_003", filename="sales_report_q3.xlsx", fileSize=1536, mimeType="application/vnd.openxmlformats-officedocument.spreadsheetml.sheet")
]
)
messages.append(message1)
# Message 2: Excel analysis result
message2 = ChatMessage(
id="msg_002",
workflowId=workflow.id,
role="assistant",
message=action2_result["result"],
status="step",
sequenceNr=2,
publishedAt=datetime.now(UTC).isoformat(),
actionId="action_002",
actionMethod="excel",
actionName="analyze",
documentsLabel=action2_result["resultLabel"],
documents=[
ChatDocument(id="004", fileId="file_004", filename="sales_analysis_summary.xlsx", fileSize=3072, mimeType="application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"),
ChatDocument(id="005", fileId="file_005", filename="sales_trends_chart.png", fileSize=512, mimeType="image/png")
]
)
messages.append(message2)
# Add messages to workflow
workflow.messages = messages
print(f"Created {len(messages)} workflow messages with documents")
# Test document reference retrieval
print("\n3. TESTING DOCUMENT REFERENCE RETRIEVAL")
print("-" * 40)
doc_refs = container.getDocumentReferenceList()
print("Available Documents:")
for i, doc in enumerate(doc_refs.get('chat', []), 1):
print(f"{i}. {doc['documentReference']}")
print(f" Source: {doc['actionMethod']}.{doc['actionName']}")
print(f" Documents: {doc['documentCount']}")
print(f" Time: {doc['datetime']}")
print()
# Test document retrieval by reference
print("4. TESTING DOCUMENT RETRIEVAL BY REFERENCE")
print("-" * 40)
test_refs = [
"documentList_abc123_sales_documents",
"documentList_def456_analysis_results"
]
for ref in test_refs:
documents = container.getChatDocumentsFromDocumentReference(ref)
print(f"Reference: {ref}")
print(f"Found {len(documents)} documents:")
for doc in documents:
print(f" - {doc.filename} (ID: {doc.id}, Size: {doc.fileSize})")
print()
print("=" * 80)
print("HANDOVER MECHANISM SUMMARY")
print("=" * 80)
print("✅ Documents are properly stored in workflow messages")
print("✅ Result labels are correctly formatted")
print("✅ Document references are retrievable")
print("✅ Subsequent actions can find previous results")
print("✅ Clear handover chain between actions")
if __name__ == "__main__":
test_document_handover()

96
test_method_signatures.py
View file

@ -1,96 +0,0 @@
#!/usr/bin/env python3
"""
Test script to demonstrate the improved method signature format.
This shows how the AI will receive clear parameter information without automatic result labels.
"""
import sys
import os
sys.path.append(os.path.dirname(os.path.abspath(__file__)))
from modules.workflow.serviceContainer import ServiceContainer
from modules.interfaces.interfaceAppModel import User
from modules.interfaces.interfaceChatModel import ChatWorkflow
from datetime import datetime, UTC
def test_method_signatures():
"""Test the improved method signature format"""
# Create test user and workflow
user = User(
id="test_user",
username="testuser",
language="en",
mandateId="test_mandate"
)
workflow = ChatWorkflow(
id="test_workflow",
mandateId="test_mandate",
status="active",
currentRound=1,
lastActivity=datetime.now(UTC).isoformat(),
startedAt=datetime.now(UTC).isoformat(),
messages=[]
)
# Create service container
container = ServiceContainer(user, workflow)
# Get and display method list
print("=" * 80)
print("IMPROVED METHOD SIGNATURES")
print("=" * 80)
print("The AI will now receive clear parameter information without automatic result labels.")
print("The AI must set resultLabel according to the format: documentList_uuid_descriptive_label")
print()
method_list = container.getMethodsList()
print("AVAILABLE METHODS:")
print("-" * 40)
for i, method in enumerate(method_list, 1):
print(f"{i:2d}. {method}")
print()
print("=" * 80)
print("EXAMPLE OF HOW AI SHOULD SET RESULT LABELS:")
print("=" * 80)
print("""
{
"status": "pending",
"feedback": "I will analyze the Excel file and create a summary report.",
"actions": [
{
"method": "excel",
"action": "read",
"parameters": {
"fileId": "document_123_sales_data.xlsx",
"connectionReference": "connection_456_msft_user@example.com",
"sheetName": "Sheet1"
},
"resultLabel": "documentList_abc123_excel_data"
},
{
"method": "document",
"action": "analyze",
"parameters": {
"fileId": "documentList_abc123_excel_data"
},
"resultLabel": "documentList_def456_analysis_results"
}
]
}
""")
print("=" * 80)
print("KEY IMPROVEMENTS:")
print("=" * 80)
print("1. Clear parameter types and descriptions")
print("2. No automatic result labels - AI must set them")
print("3. Consistent format: documentList_uuid_descriptive_label")
print("4. Better parameter validation through type information")
print("5. Clear handover between actions using result labels")
if __name__ == "__main__":
test_method_signatures()

27
test_param_extraction.py Normal file
View file

@ -0,0 +1,27 @@
#!/usr/bin/env python3
from modules.workflow.methodBase import MethodBase
class TestMethod(MethodBase):
pass
def test_parameter_extraction():
test = TestMethod(None)
test.name = 'test'
docstring = """Call AI service with document content
Parameters:
prompt (str): The prompt to send to the AI service
documents (List[Dict[str, Any]], optional): List of documents to include in context
Each document should have: documentReference (str), contentExtractionPrompt (str, optional)"""
print("Docstring:")
print(docstring)
print("\nExtracted descriptions:")
descriptions = test._extractParameterDescriptions(docstring)
for param, desc in descriptions.items():
print(f" {param}: {desc}")
if __name__ == "__main__":
test_parameter_extraction()

23
test_signature.py Normal file
View file

@ -0,0 +1,23 @@
#!/usr/bin/env python3
from modules.workflow.serviceContainer import ServiceContainer
from modules.interfaces.interfaceAppObjects import User
from modules.interfaces.interfaceChatModel import ChatWorkflow
def test_signatures():
user = User(id='test', mandateId='test', username='test', email='test@test.com',
fullName='Test User', enabled=True, language='en', privilege='user',
authenticationAuthority='local')
workflow = ChatWorkflow(id='test', mandateId='test', status='running', name='Test',
currentRound=1, lastActivity='2025-01-01T00:00:00Z',
startedAt='2025-01-01T00:00:00Z', logs=[], messages=[],
stats=None, tasks=[])
service = ServiceContainer(user, workflow)
print("Method signatures:")
methodList = service.getMethodsList()
for sig in methodList[:5]: # Show first 5
print(f" {sig}")
if __name__ == "__main__":
test_signatures()

208
test_workflow.py
View file

@ -1,6 +1,6 @@
#!/usr/bin/env python3
"""
Test routine for WorkflowManager.workflowProcess()
Test routine for WorkflowManager.workflowProcess() with new unified workflow architecture
"""
import asyncio
@ -15,20 +15,26 @@ print("Starting test_workflow.py...")
# Configure logging FIRST, before any other imports
import logging
# Clear any existing handlers to avoid duplicate logs
for handler in logging.root.handlers[:]:
logging.root.removeHandler(handler)
logging.basicConfig(
level=logging.DEBUG,
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
format='%(asctime)s - %(levelname)s - %(name)s - %(message)s',
handlers=[
logging.StreamHandler(sys.stdout),
logging.FileHandler('test_workflow.log', encoding='utf-8')
logging.FileHandler('test_workflow.log', mode='w', encoding='utf-8') # 'w' mode clears the file
],
force=True # Force reconfiguration even if already configured
)
# logger = logging.getLogger(__name__)
# print("Logger level:", logger.level)
# logger.info("Logger is working!")
# print("Logger test done")
# Filter out httpcore messages
logging.getLogger('httpcore').setLevel(logging.WARNING)
logging.getLogger('httpx').setLevel(logging.WARNING)
logger = logging.getLogger(__name__)
# Set up test configuration
os.environ['POWERON_CONFIG_FILE'] = 'test_config.ini'
@ -47,6 +53,14 @@ except Exception as e:
traceback.print_exc()
sys.exit(1)
def log_workflow_debug(message: str, data: dict = None):
"""Log workflow debug data with JSON dumps"""
timestamp = datetime.now(UTC).isoformat()
if data:
logger.debug(f"[{timestamp}] {message}\n{json.dumps(data, indent=2, ensure_ascii=False)}")
else:
logger.debug(f"[{timestamp}] {message}")
def create_test_user() -> User:
"""Create a test user for the workflow"""
return User(
@ -276,9 +290,13 @@ EVALUATION WEIGHTS:
fileName=filename
)
test_files.append(file_item.id)
# logger.info(f"Created test file: {filename} (ID: {file_item.id})")
log_workflow_debug(f"Created test file: {filename}", {
"file_id": file_item.id,
"filename": filename,
"content_length": len(content)
})
except Exception as e:
# logger.error(f"Error creating test file {filename}: {str(e)}")
log_workflow_debug(f"Error creating test file {filename}: {str(e)}")
# Create a dummy file ID if creation fails
test_files.append(f"file_{filename.replace('.', '_')}")
@ -286,15 +304,22 @@ EVALUATION WEIGHTS:
async def test_workflow_process():
print("Inside test_workflow_process()")
"""Test the workflowProcess function"""
"""Test the workflowProcess function with new unified workflow architecture"""
try:
# logger.info("Starting workflow process test...")
logger.info("=== STARTING UNIFIED WORKFLOW PROCESS TEST ===")
# Create test data
test_user = create_test_user()
test_workflow = create_test_workflow()
test_user_input = create_test_user_input()
log_workflow_debug("Test data created", {
"user_id": test_user.id,
"workflow_id": test_workflow.id,
"user_input_prompt": test_user_input.prompt,
"file_ids": test_user_input.listFileId
})
# Create test user in database through AppObjects interface
from modules.interfaces.interfaceAppObjects import getRootInterface
from modules.interfaces.interfaceAppModel import AuthAuthority, ConnectionStatus, Token, UserPrivilege
@ -310,7 +335,11 @@ async def test_workflow_process():
privilege=UserPrivilege.USER,
authenticationAuthority=AuthAuthority.LOCAL
)
# logger.info(f"Created test user in database: {created_user.id}")
log_workflow_debug("Created test user in database", {
"user_id": created_user.id,
"username": created_user.username,
"email": created_user.email
})
# Create test connection through AppObjects interface
from modules.interfaces.interfaceAppObjects import getInterface as getAppObjects
@ -323,7 +352,11 @@ async def test_workflow_process():
externalEmail="testuser@example.com",
status=ConnectionStatus.ACTIVE
)
# logger.info(f"Created test connection: {test_connection.id}")
log_workflow_debug("Created test connection", {
"connection_id": test_connection.id,
"authority": test_connection.authority,
"external_username": test_connection.externalUsername
})
# Create test token for the connection
test_token = Token(
@ -336,23 +369,11 @@ async def test_workflow_process():
createdAt=datetime.now(UTC)
)
app_interface.saveToken(test_token)
# logger.info(f"Created test token for connection: {test_token.id}")
# logger.info(f"Test user: {created_user.username}")
# logger.info(f"Test workflow: {test_workflow.id}")
# Log the full prompt in JSON format
# logger.debug("=" * 60)
# logger.debug("USER INPUT PROMPT (JSON):")
# logger.debug("=" * 60)
prompt_data = {
"prompt": test_user_input.prompt,
"listFileId": test_user_input.listFileId,
"userLanguage": test_user_input.userLanguage
}
# logger.debug(json.dumps(prompt_data, indent=2, ensure_ascii=False))
# logger.debug("=" * 60)
# logger.debug(f"Test files: {test_user_input.listFileId}")
log_workflow_debug("Created test token", {
"token_id": test_token.id,
"authority": test_token.authority,
"expires_at": test_token.expiresAt
})
# Create test workflow in database through ChatObjects interface
from modules.interfaces.interfaceChatObjects import getInterface as getChatObjects
@ -367,85 +388,106 @@ async def test_workflow_process():
"lastActivity": test_workflow.lastActivity
}
created_workflow = chat_interface.createWorkflow(workflow_data)
# logger.info(f"Created test workflow: {created_workflow.id}")
log_workflow_debug("Created test workflow in database", {
"workflow_id": created_workflow.id,
"name": created_workflow.name,
"status": created_workflow.status
})
# Update the test_workflow object with the created workflow's ID
test_workflow.id = created_workflow.id
# Create test files in database
# logger.info("Creating test files for candidate evaluation...")
logger.info("Creating test files for candidate evaluation...")
test_file_ids = create_test_files(chat_interface)
# logger.info(f"Created {len(test_file_ids)} test files: {test_file_ids}")
log_workflow_debug("Test files created", {
"file_count": len(test_file_ids),
"file_ids": test_file_ids
})
# Update user input with real file IDs
test_user_input.listFileId = test_file_ids
# logger.info(f"Updated user input with file IDs: {test_user_input.listFileId}")
log_workflow_debug("Updated user input with file IDs", {
"file_ids": test_user_input.listFileId
})
# Initialize WorkflowManager
workflow_manager = WorkflowManager(chat_interface, created_user)
# logger.info("WorkflowManager initialized")
logger.info("WorkflowManager initialized")
# Test the workflowProcess function
# logger.info("Calling workflowProcess...")
task = await workflow_manager.workflowProcess(test_user_input, test_workflow)
logger.info("Calling workflowProcess with unified workflow architecture...")
# Log results
if task:
# logger.debug("Task created successfully!")
# logger.debug(f"Task ID: {task.id}")
# logger.debug(f"Task Status: {task.status}")
# logger.debug(f"Task Feedback: {task.feedback}")
# logger.info(f"Number of actions: {len(task.actionList) if task.actionList else 0}")
# logger.debug("=" * 60)
# logger.debug("TASK OBJECT (JSON):")
# logger.debug("=" * 60)
task_data = {
"id": task.id,
"status": task.status,
"feedback": task.feedback,
"actionList": [
try:
# Execute the unified workflow process
await workflow_manager.workflowProcess(test_user_input, test_workflow)
# Log workflow results
log_workflow_debug("Workflow process completed successfully", {
"workflow_id": test_workflow.id,
"workflow_status": test_workflow.status,
"message_count": len(test_workflow.messages),
"final_messages": [
{
"execMethod": action.execMethod,
"execAction": action.execAction,
"execParameters": action.execParameters,
"execResultLabel": action.execResultLabel
} for action in (task.actionList or [])
] if task.actionList else []
"role": msg.role,
"message": msg.message[:200] + "..." if len(msg.message) > 200 else msg.message,
"status": msg.status,
"sequence_nr": msg.sequenceNr
} for msg in test_workflow.messages[-3:] # Last 3 messages
]
})
# Log detailed workflow messages
for i, message in enumerate(test_workflow.messages):
log_workflow_debug(f"WORKFLOW MESSAGE {i+1}:", {
"role": message.role,
"message": message.message,
"status": message.status,
"sequence_nr": message.sequenceNr,
"published_at": message.publishedAt,
"document_count": len(message.documents) if hasattr(message, 'documents') else 0
})
return test_workflow
except Exception as e:
import traceback
error_details = {
"error_type": type(e).__name__,
"error_message": str(e),
"error_args": e.args if hasattr(e, 'args') else None,
"traceback": traceback.format_exc()
}
# logger.debug(json.dumps(task_data, indent=2, ensure_ascii=False))
# logger.debug("=" * 60)
if task.actionList:
for i, action in enumerate(task.actionList):
# logger.info(f"Action {i+1}: {action.execMethod}.{action.execAction}")
# logger.info(f" Parameters: {action.execParameters}")
pass
else:
# logger.warning("No task was created")
pass
# logger.info("Test completed successfully!")
return task
log_workflow_debug("WORKFLOW PROCESS EXCEPTION:", error_details)
raise
logger.info("=== UNIFIED WORKFLOW PROCESS TEST COMPLETED ===")
return test_workflow
except Exception as e:
# logger.error(f"❌ Test failed with error: {str(e)}")
# logger.exception("Full traceback:")
logger.error(f"❌ Test failed with error: {str(e)}")
log_workflow_debug("Full error details", {
"error_type": type(e).__name__,
"error_message": str(e)
})
raise
async def main():
print("Inside main()")
# logger.info("=" * 50)
# logger.info("CANDIDATE EVALUATION WORKFLOW TEST")
# logger.info("=" * 50)
logger.info("=" * 50)
logger.info("CANDIDATE EVALUATION UNIFIED WORKFLOW TEST")
logger.info("=" * 50)
try:
task = await test_workflow_process()
# logger.info("=" * 50)
# logger.info("TEST COMPLETED SUCCESSFULLY")
# logger.info("=" * 50)
return task
workflow = await test_workflow_process()
logger.info("=" * 50)
logger.info("TEST COMPLETED SUCCESSFULLY")
logger.info("=" * 50)
return workflow
except Exception as e:
# logger.error("=" * 50)
# logger.error("TEST FAILED")
# logger.error("=" * 50)
logger.error("=" * 50)
logger.error("TEST FAILED")
logger.error("=" * 50)
raise
if __name__ == "__main__":

926
tool_test01ui.py
View file

@ -1,926 +0,0 @@
"""
UI Test Procedure for PowerOn Frontend
Tests CRUD operations, user registration, authentication, and access control
"""
import os
import sys
import json
import logging
from datetime import datetime
from typing import Dict, List, Any
import requests
from dataclasses import dataclass
from enum import Enum
import time
# Configure logging
logging.basicConfig(
level=logging.INFO,
format='%(asctime)s - %(levelname)s - %(message)s',
handlers=[
logging.FileHandler('ui_test_report.log'),
logging.StreamHandler(sys.stdout)
]
)
logger = logging.getLogger(__name__)
# Test configuration
BASE_URL = "http://localhost:8080" # Adjust based on your frontend URL
API_URL = "http://localhost:8000" # Adjust based on your backend URL
class UserRole(Enum):
SYSADMIN = "sysadmin"
ADMIN = "admin"
USER = "user"
@dataclass
class TestResult:
test_name: str
success: bool
message: str
details: Dict[str, Any] = None
@dataclass
class TestReport:
timestamp: str
total_tests: int
passed_tests: int
failed_tests: int
results: List[TestResult]
bugs_found: List[Dict[str, str]]
required_adaptations: List[Dict[str, str]]
class UITestSuite:
def __init__(self):
self.session = requests.Session()
self.test_results = []
self.bugs_found = []
self.required_adaptations = []
self.current_user = None
self.current_role = None
def run_all_tests(self):
"""Run all test categories"""
logger.info("Starting UI Test Suite")
# Test user registration and authentication
self.test_user_registration()
self.test_user_authentication()
# Test CRUD operations for each module
self.test_files_module()
self.test_mandates_module()
self.test_prompts_module()
self.test_users_module()
# Generate and save report
self.generate_report()
def test_user_registration(self):
"""Test user registration for different roles"""
logger.info("Testing User Registration")
test_users = [
{"username": "test_sysadmin", "password": "Test123!", "role": UserRole.SYSADMIN},
{"username": "test_admin", "password": "Test123!", "role": UserRole.ADMIN},
{"username": "test_user", "password": "Test123!", "role": UserRole.USER}
]
for user in test_users:
try:
# Create userData object matching User model
user_data = {
"username": user["username"],
"email": f"{user['username']}@test.com",
"fullName": f"Test {user['role'].value}",
"language": "en",
"enabled": True,
"privilege": user["role"].value,
"authenticationAuthority": "local",
"mandateId": None, # Will be set by the backend
"connections": []
}
response = self.session.post(
f"{API_URL}/api/local/register",
json={
"userData": user_data,
"password": user["password"]
},
headers={
"X-CSRF-Token": "test-csrf-token",
"Content-Type": "application/json"
}
)
if response.status_code == 200:
logger.info(f"Successfully registered {user['role'].value} user")
self.test_results.append(TestResult(
f"Register {user['role'].value}",
True,
f"Successfully registered {user['role'].value} user"
))
else:
error_msg = f"Failed to register {user['role'].value} user: {response.status_code}"
if response.text:
error_msg += f" - {response.text}"
logger.error(error_msg)
self.test_results.append(TestResult(
f"Register {user['role'].value}",
False,
error_msg,
{"status_code": response.status_code, "response": response.text}
))
except Exception as e:
error_msg = f"Exception during registration: {str(e)}"
logger.error(error_msg)
self.test_results.append(TestResult(
f"Register {user['role'].value}",
False,
error_msg
))
def test_user_authentication(self):
"""Test login and logout for different roles"""
logger.info("Testing User Authentication")
test_users = [
{"username": "test_sysadmin", "password": "Test123!", "role": UserRole.SYSADMIN},
{"username": "test_admin", "password": "Test123!", "role": UserRole.ADMIN},
{"username": "test_user", "password": "Test123!", "role": UserRole.USER}
]
for user in test_users:
# Test login
try:
response = self.session.post(
f"{API_URL}/api/local/login",
data={
"username": user["username"],
"password": user["password"]
},
headers={
"X-CSRF-Token": "test-csrf-token" # Add CSRF token
}
)
if response.status_code == 200:
self.test_results.append(TestResult(
f"Login {user['role'].value}",
True,
f"Successfully logged in as {user['role'].value}"
))
# Test logout
logout_response = self.session.post(f"{API_URL}/api/security/local/logout")
if logout_response.status_code == 200:
self.test_results.append(TestResult(
f"Logout {user['role'].value}",
True,
f"Successfully logged out as {user['role'].value}"
))
else:
self.test_results.append(TestResult(
f"Logout {user['role'].value}",
False,
f"Failed to logout as {user['role'].value}"
))
else:
self.test_results.append(TestResult(
f"Login {user['role'].value}",
False,
f"Failed to login as {user['role'].value}"
))
except Exception as e:
self.test_results.append(TestResult(
f"Login {user['role'].value}",
False,
f"Exception during login: {str(e)}"
))
def test_files_module(self):
"""Test CRUD operations for files module"""
logger.info("Testing Files Module")
# Test for each role
for role in UserRole:
self.current_role = role
self._login_as_role(role)
# Test view files
self._test_view_files()
# Test create file
self._test_create_file()
# Test modify file
self._test_modify_file()
# Test delete file
self._test_delete_file()
self._logout()
def test_mandates_module(self):
"""Test CRUD operations for mandates module"""
logger.info("Testing Mandates Module")
for role in UserRole:
self.current_role = role
self._login_as_role(role)
# Test view mandates
self._test_view_mandates()
# Test create mandate
self._test_create_mandate()
# Test modify mandate
self._test_modify_mandate()
# Test delete mandate
self._test_delete_mandate()
self._logout()
def test_prompts_module(self):
"""Test CRUD operations for prompts module"""
logger.info("Testing Prompts Module")
for role in UserRole:
self.current_role = role
self._login_as_role(role)
# Test view prompts
self._test_view_prompts()
# Test create prompt
self._test_create_prompt()
# Test modify prompt
self._test_modify_prompt()
# Test delete prompt
self._test_delete_prompt()
self._logout()
def test_users_module(self):
"""Test CRUD operations for users module"""
logger.info("Testing Users Module")
for role in UserRole:
self.current_role = role
self._login_as_role(role)
# Test view users
self._test_view_users()
# Test create user
self._test_create_user()
# Test modify user
self._test_modify_user()
# Test delete user
self._test_delete_user()
self._logout()
def _login_as_role(self, role: UserRole):
"""Helper method to login as a specific role"""
username = f"test_{role.value}"
max_retries = 3
retry_delay = 12 # 12 seconds delay between retries (to stay under 5 per minute)
for attempt in range(max_retries):
try:
# Always wait before attempting login
if attempt == 0:
logger.info(f"Waiting {retry_delay}s before first login attempt for {role.value}")
else:
logger.info(f"Rate limit reached, waiting {retry_delay}s before retry {attempt + 1} for {role.value} login")
time.sleep(retry_delay)
response = self.session.post(
f"{API_URL}/api/local/login",
data={
"username": username,
"password": "Test123!"
},
headers={
"X-CSRF-Token": "test-csrf-token"
}
)
if response.status_code == 200:
self.current_user = response.json()
logger.info(f"Successfully logged in as {role.value}")
return
elif response.status_code == 429:
logger.info(f"Rate limit reached for {role.value} login: {response.text}")
if attempt < max_retries - 1:
continue
else:
logger.error(f"Max retries reached for {role.value} login after rate limits")
raise Exception(f"Max retries reached for {role.value} login after rate limits")
else:
error_msg = f"Failed to login as {role.value}: {response.status_code}"
if response.text:
error_msg += f" - {response.text}"
logger.error(error_msg)
raise Exception(error_msg)
except Exception as e:
if attempt == max_retries - 1:
logger.error(f"Login error for {role.value} after {max_retries} attempts: {str(e)}")
raise
continue
def _logout(self):
"""Helper method to logout"""
self.session.post(f"{API_URL}/api/security/local/logout")
self.current_user = None
def _test_view_files(self):
"""Test viewing files"""
try:
response = self.session.get(f"{API_URL}/api/files")
if response.status_code == 200:
self.test_results.append(TestResult(
f"View Files as {self.current_role.value}",
True,
"Successfully viewed files"
))
else:
self.test_results.append(TestResult(
f"View Files as {self.current_role.value}",
False,
f"Failed to view files: {response.status_code}"
))
except Exception as e:
self.test_results.append(TestResult(
f"View Files as {self.current_role.value}",
False,
f"Exception viewing files: {str(e)}"
))
def _test_create_file(self):
"""Test creating a file"""
try:
# Create a test file
test_file = {
"name": f"test_file_{self.current_role.value}",
"content": "Test content",
"type": "text/plain"
}
response = self.session.post(
f"{API_URL}/api/files",
json=test_file
)
if response.status_code == 200:
self.test_results.append(TestResult(
f"Create File as {self.current_role.value}",
True,
"Successfully created file"
))
else:
self.test_results.append(TestResult(
f"Create File as {self.current_role.value}",
False,
f"Failed to create file: {response.status_code}"
))
except Exception as e:
self.test_results.append(TestResult(
f"Create File as {self.current_role.value}",
False,
f"Exception creating file: {str(e)}"
))
def _test_modify_file(self):
"""Test modifying a file"""
try:
# First get a file to modify
files_response = self.session.get(f"{API_URL}/api/files")
if files_response.status_code == 200 and files_response.json():
file_id = files_response.json()[0]["id"]
# Modify the file
update_data = {
"name": f"modified_file_{self.current_role.value}",
"content": "Modified content"
}
response = self.session.put(
f"{API_URL}/api/files/{file_id}",
json=update_data
)
if response.status_code == 200:
self.test_results.append(TestResult(
f"Modify File as {self.current_role.value}",
True,
"Successfully modified file"
))
else:
self.test_results.append(TestResult(
f"Modify File as {self.current_role.value}",
False,
f"Failed to modify file: {response.status_code}"
))
else:
self.test_results.append(TestResult(
f"Modify File as {self.current_role.value}",
False,
"No files available to modify"
))
except Exception as e:
self.test_results.append(TestResult(
f"Modify File as {self.current_role.value}",
False,
f"Exception modifying file: {str(e)}"
))
def _test_delete_file(self):
"""Test deleting a file"""
try:
# First get a file to delete
files_response = self.session.get(f"{API_URL}/api/files")
if files_response.status_code == 200 and files_response.json():
file_id = files_response.json()[0]["id"]
response = self.session.delete(f"{API_URL}/api/files/{file_id}")
if response.status_code == 200:
self.test_results.append(TestResult(
f"Delete File as {self.current_role.value}",
True,
"Successfully deleted file"
))
else:
self.test_results.append(TestResult(
f"Delete File as {self.current_role.value}",
False,
f"Failed to delete file: {response.status_code}"
))
else:
self.test_results.append(TestResult(
f"Delete File as {self.current_role.value}",
False,
"No files available to delete"
))
except Exception as e:
self.test_results.append(TestResult(
f"Delete File as {self.current_role.value}",
False,
f"Exception deleting file: {str(e)}"
))
def _test_view_mandates(self):
"""Test viewing mandates"""
try:
response = self.session.get(f"{API_URL}/api/mandates")
if response.status_code == 200:
self.test_results.append(TestResult(
f"View Mandates as {self.current_role.value}",
True,
"Successfully viewed mandates"
))
else:
self.test_results.append(TestResult(
f"View Mandates as {self.current_role.value}",
False,
f"Failed to view mandates: {response.status_code}"
))
except Exception as e:
self.test_results.append(TestResult(
f"View Mandates as {self.current_role.value}",
False,
f"Exception viewing mandates: {str(e)}"
))
def _test_create_mandate(self):
"""Test creating a mandate"""
try:
test_mandate = {
"name": f"test_mandate_{self.current_role.value}",
"description": "Test mandate",
"enabled": True
}
response = self.session.post(
f"{API_URL}/api/mandates",
json=test_mandate
)
if response.status_code == 200:
self.test_results.append(TestResult(
f"Create Mandate as {self.current_role.value}",
True,
"Successfully created mandate"
))
else:
self.test_results.append(TestResult(
f"Create Mandate as {self.current_role.value}",
False,
f"Failed to create mandate: {response.status_code}"
))
except Exception as e:
self.test_results.append(TestResult(
f"Create Mandate as {self.current_role.value}",
False,
f"Exception creating mandate: {str(e)}"
))
def _test_modify_mandate(self):
"""Test modifying a mandate"""
try:
# First get a mandate to modify
mandates_response = self.session.get(f"{API_URL}/api/mandates")
if mandates_response.status_code == 200 and mandates_response.json():
mandate_id = mandates_response.json()[0]["id"]
update_data = {
"name": f"modified_mandate_{self.current_role.value}",
"description": "Modified mandate"
}
response = self.session.put(
f"{API_URL}/api/mandates/{mandate_id}",
json=update_data
)
if response.status_code == 200:
self.test_results.append(TestResult(
f"Modify Mandate as {self.current_role.value}",
True,
"Successfully modified mandate"
))
else:
self.test_results.append(TestResult(
f"Modify Mandate as {self.current_role.value}",
False,
f"Failed to modify mandate: {response.status_code}"
))
else:
self.test_results.append(TestResult(
f"Modify Mandate as {self.current_role.value}",
False,
"No mandates available to modify"
))
except Exception as e:
self.test_results.append(TestResult(
f"Modify Mandate as {self.current_role.value}",
False,
f"Exception modifying mandate: {str(e)}"
))
def _test_delete_mandate(self):
"""Test deleting a mandate"""
try:
# First get a mandate to delete
mandates_response = self.session.get(f"{API_URL}/api/mandates")
if mandates_response.status_code == 200 and mandates_response.json():
mandate_id = mandates_response.json()[0]["id"]
response = self.session.delete(f"{API_URL}/api/mandates/{mandate_id}")
if response.status_code == 200:
self.test_results.append(TestResult(
f"Delete Mandate as {self.current_role.value}",
True,
"Successfully deleted mandate"
))
else:
self.test_results.append(TestResult(
f"Delete Mandate as {self.current_role.value}",
False,
f"Failed to delete mandate: {response.status_code}"
))
else:
self.test_results.append(TestResult(
f"Delete Mandate as {self.current_role.value}",
False,
"No mandates available to delete"
))
except Exception as e:
self.test_results.append(TestResult(
f"Delete Mandate as {self.current_role.value}",
False,
f"Exception deleting mandate: {str(e)}"
))
def _test_view_prompts(self):
"""Test viewing prompts"""
try:
response = self.session.get(f"{API_URL}/api/prompts")
if response.status_code == 200:
self.test_results.append(TestResult(
f"View Prompts as {self.current_role.value}",
True,
"Successfully viewed prompts"
))
else:
self.test_results.append(TestResult(
f"View Prompts as {self.current_role.value}",
False,
f"Failed to view prompts: {response.status_code}"
))
except Exception as e:
self.test_results.append(TestResult(
f"View Prompts as {self.current_role.value}",
False,
f"Exception viewing prompts: {str(e)}"
))
def _test_create_prompt(self):
"""Test creating a prompt"""
try:
test_prompt = {
"name": f"test_prompt_{self.current_role.value}",
"content": "Test prompt content",
"category": "test"
}
response = self.session.post(
f"{API_URL}/api/prompts",
json=test_prompt
)
if response.status_code == 200:
self.test_results.append(TestResult(
f"Create Prompt as {self.current_role.value}",
True,
"Successfully created prompt"
))
else:
self.test_results.append(TestResult(
f"Create Prompt as {self.current_role.value}",
False,
f"Failed to create prompt: {response.status_code}"
))
except Exception as e:
self.test_results.append(TestResult(
f"Create Prompt as {self.current_role.value}",
False,
f"Exception creating prompt: {str(e)}"
))
def _test_modify_prompt(self):
"""Test modifying a prompt"""
try:
# First get a prompt to modify
prompts_response = self.session.get(f"{API_URL}/api/prompts")
if prompts_response.status_code == 200 and prompts_response.json():
prompt_id = prompts_response.json()[0]["id"]
update_data = {
"name": f"modified_prompt_{self.current_role.value}",
"content": "Modified prompt content"
}
response = self.session.put(
f"{API_URL}/api/prompts/{prompt_id}",
json=update_data
)
if response.status_code == 200:
self.test_results.append(TestResult(
f"Modify Prompt as {self.current_role.value}",
True,
"Successfully modified prompt"
))
else:
self.test_results.append(TestResult(
f"Modify Prompt as {self.current_role.value}",
False,
f"Failed to modify prompt: {response.status_code}"
))
else:
self.test_results.append(TestResult(
f"Modify Prompt as {self.current_role.value}",
False,
"No prompts available to modify"
))
except Exception as e:
self.test_results.append(TestResult(
f"Modify Prompt as {self.current_role.value}",
False,
f"Exception modifying prompt: {str(e)}"
))
def _test_delete_prompt(self):
"""Test deleting a prompt"""
try:
# First get a prompt to delete
prompts_response = self.session.get(f"{API_URL}/api/prompts")
if prompts_response.status_code == 200 and prompts_response.json():
prompt_id = prompts_response.json()[0]["id"]
response = self.session.delete(f"{API_URL}/api/prompts/{prompt_id}")
if response.status_code == 200:
self.test_results.append(TestResult(
f"Delete Prompt as {self.current_role.value}",
True,
"Successfully deleted prompt"
))
else:
self.test_results.append(TestResult(
f"Delete Prompt as {self.current_role.value}",
False,
f"Failed to delete prompt: {response.status_code}"
))
else:
self.test_results.append(TestResult(
f"Delete Prompt as {self.current_role.value}",
False,
"No prompts available to delete"
))
except Exception as e:
self.test_results.append(TestResult(
f"Delete Prompt as {self.current_role.value}",
False,
f"Exception deleting prompt: {str(e)}"
))
def _test_view_users(self):
"""Test viewing users"""
try:
response = self.session.get(f"{API_URL}/api/users")
if response.status_code == 200:
self.test_results.append(TestResult(
f"View Users as {self.current_role.value}",
True,
"Successfully viewed users"
))
else:
self.test_results.append(TestResult(
f"View Users as {self.current_role.value}",
False,
f"Failed to view users: {response.status_code}"
))
except Exception as e:
self.test_results.append(TestResult(
f"View Users as {self.current_role.value}",
False,
f"Exception viewing users: {str(e)}"
))
def _test_create_user(self):
"""Test creating a user"""
try:
test_user = {
"username": f"new_user_{self.current_role.value}",
"password": "Test123!",
"email": f"new_user_{self.current_role.value}@test.com",
"privilege": "user"
}
response = self.session.post(
f"{API_URL}/api/users",
json=test_user
)
if response.status_code == 200:
self.test_results.append(TestResult(
f"Create User as {self.current_role.value}",
True,
"Successfully created user"
))
else:
self.test_results.append(TestResult(
f"Create User as {self.current_role.value}",
False,
f"Failed to create user: {response.status_code}"
))
except Exception as e:
self.test_results.append(TestResult(
f"Create User as {self.current_role.value}",
False,
f"Exception creating user: {str(e)}"
))
def _test_modify_user(self):
"""Test modifying a user"""
try:
# First get a user to modify
users_response = self.session.get(f"{API_URL}/api/users")
if users_response.status_code == 200 and users_response.json():
user_id = users_response.json()[0]["id"]
update_data = {
"username": f"modified_user_{self.current_role.value}",
"email": f"modified_user_{self.current_role.value}@test.com"
}
response = self.session.put(
f"{API_URL}/api/users/{user_id}",
json=update_data
)
if response.status_code == 200:
self.test_results.append(TestResult(
f"Modify User as {self.current_role.value}",
True,
"Successfully modified user"
))
else:
self.test_results.append(TestResult(
f"Modify User as {self.current_role.value}",
False,
f"Failed to modify user: {response.status_code}"
))
else:
self.test_results.append(TestResult(
f"Modify User as {self.current_role.value}",
False,
"No users available to modify"
))
except Exception as e:
self.test_results.append(TestResult(
f"Modify User as {self.current_role.value}",
False,
f"Exception modifying user: {str(e)}"
))
def _test_delete_user(self):
"""Test deleting a user"""
try:
# First get a user to delete
users_response = self.session.get(f"{API_URL}/api/users")
if users_response.status_code == 200 and users_response.json():
user_id = users_response.json()[0]["id"]
response = self.session.delete(f"{API_URL}/api/users/{user_id}")
if response.status_code == 200:
self.test_results.append(TestResult(
f"Delete User as {self.current_role.value}",
True,
"Successfully deleted user"
))
else:
self.test_results.append(TestResult(
f"Delete User as {self.current_role.value}",
False,
f"Failed to delete user: {response.status_code}"
))
else:
self.test_results.append(TestResult(
f"Delete User as {self.current_role.value}",
False,
"No users available to delete"
))
except Exception as e:
self.test_results.append(TestResult(
f"Delete User as {self.current_role.value}",
False,
f"Exception deleting user: {str(e)}"
))
def generate_report(self):
"""Generate test report"""
# Convert TestResult objects to dictionaries
serialized_results = [
{
"test_name": r.test_name,
"success": r.success,
"message": r.message,
"details": r.details
}
for r in self.test_results
]
report = {
"timestamp": datetime.now().isoformat(),
"total_tests": len(self.test_results),
"passed_tests": sum(1 for r in self.test_results if r.success),
"failed_tests": sum(1 for r in self.test_results if not r.success),
"results": serialized_results,
"bugs_found": self.bugs_found,
"required_adaptations": self.required_adaptations
}
# Save report to file
with open('ui_test_report.json', 'w') as f:
json.dump(report, f, indent=2)
# Print summary
logger.info(f"""
Test Report Summary:
-------------------
Total Tests: {report['total_tests']}
Passed: {report['passed_tests']}
Failed: {report['failed_tests']}
Bugs Found: {len(report['bugs_found'])}
Required Adaptations: {len(report['required_adaptations'])}
""")
if __name__ == "__main__":
test_suite = UITestSuite()
test_suite.run_all_tests()

352
ui_test_report.json
View file

@ -1,352 +0,0 @@
{
"timestamp": "2025-06-02T20:18:49.451773",
"total_tests": 57,
"passed_tests": 6,
"failed_tests": 51,
"results": [
{
"test_name": "Register sysadmin",
"success": true,
"message": "Successfully registered sysadmin user",
"details": null
},
{
"test_name": "Register admin",
"success": true,
"message": "Successfully registered admin user",
"details": null
},
{
"test_name": "Register user",
"success": true,
"message": "Successfully registered user user",
"details": null
},
{
"test_name": "Login sysadmin",
"success": true,
"message": "Successfully logged in as sysadmin",
"details": null
},
{
"test_name": "Logout sysadmin",
"success": false,
"message": "Failed to logout as sysadmin",
"details": null
},
{
"test_name": "Login admin",
"success": true,
"message": "Successfully logged in as admin",
"details": null
},
{
"test_name": "Logout admin",
"success": false,
"message": "Failed to logout as admin",
"details": null
},
{
"test_name": "Login user",
"success": true,
"message": "Successfully logged in as user",
"details": null
},
{
"test_name": "Logout user",
"success": false,
"message": "Failed to logout as user",
"details": null
},
{
"test_name": "View Files as sysadmin",
"success": false,
"message": "Failed to view files: 405",
"details": null
},
{
"test_name": "Create File as sysadmin",
"success": false,
"message": "Failed to create file: 405",
"details": null
},
{
"test_name": "Modify File as sysadmin",
"success": false,
"message": "No files available to modify",
"details": null
},
{
"test_name": "Delete File as sysadmin",
"success": false,
"message": "No files available to delete",
"details": null
},
{
"test_name": "View Files as admin",
"success": false,
"message": "Failed to view files: 405",
"details": null
},
{
"test_name": "Create File as admin",
"success": false,
"message": "Failed to create file: 405",
"details": null
},
{
"test_name": "Modify File as admin",
"success": false,
"message": "No files available to modify",
"details": null
},
{
"test_name": "Delete File as admin",
"success": false,
"message": "No files available to delete",
"details": null
},
{
"test_name": "View Files as user",
"success": false,
"message": "Failed to view files: 405",
"details": null
},
{
"test_name": "Create File as user",
"success": false,
"message": "Failed to create file: 405",
"details": null
},
{
"test_name": "Modify File as user",
"success": false,
"message": "No files available to modify",
"details": null
},
{
"test_name": "Delete File as user",
"success": false,
"message": "No files available to delete",
"details": null
},
{
"test_name": "View Mandates as sysadmin",
"success": false,
"message": "Failed to view mandates: 405",
"details": null
},
{
"test_name": "Create Mandate as sysadmin",
"success": false,
"message": "Failed to create mandate: 405",
"details": null
},
{
"test_name": "Modify Mandate as sysadmin",
"success": false,
"message": "No mandates available to modify",
"details": null
},
{
"test_name": "Delete Mandate as sysadmin",
"success": false,
"message": "No mandates available to delete",
"details": null
},
{
"test_name": "View Mandates as admin",
"success": false,
"message": "Failed to view mandates: 405",
"details": null
},
{
"test_name": "Create Mandate as admin",
"success": false,
"message": "Failed to create mandate: 405",
"details": null
},
{
"test_name": "Modify Mandate as admin",
"success": false,
"message": "No mandates available to modify",
"details": null
},
{
"test_name": "Delete Mandate as admin",
"success": false,
"message": "No mandates available to delete",
"details": null
},
{
"test_name": "View Mandates as user",
"success": false,
"message": "Failed to view mandates: 405",
"details": null
},
{
"test_name": "Create Mandate as user",
"success": false,
"message": "Failed to create mandate: 405",
"details": null
},
{
"test_name": "Modify Mandate as user",
"success": false,
"message": "No mandates available to modify",
"details": null
},
{
"test_name": "Delete Mandate as user",
"success": false,
"message": "No mandates available to delete",
"details": null
},
{
"test_name": "View Prompts as sysadmin",
"success": false,
"message": "Failed to view prompts: 401",
"details": null
},
{
"test_name": "Create Prompt as sysadmin",
"success": false,
"message": "Failed to create prompt: 401",
"details": null
},
{
"test_name": "Modify Prompt as sysadmin",
"success": false,
"message": "No prompts available to modify",
"details": null
},
{
"test_name": "Delete Prompt as sysadmin",
"success": false,
"message": "No prompts available to delete",
"details": null
},
{
"test_name": "View Prompts as admin",
"success": false,
"message": "Failed to view prompts: 401",
"details": null
},
{
"test_name": "Create Prompt as admin",
"success": false,
"message": "Failed to create prompt: 401",
"details": null
},
{
"test_name": "Modify Prompt as admin",
"success": false,
"message": "No prompts available to modify",
"details": null
},
{
"test_name": "Delete Prompt as admin",
"success": false,
"message": "No prompts available to delete",
"details": null
},
{
"test_name": "View Prompts as user",
"success": false,
"message": "Failed to view prompts: 401",
"details": null
},
{
"test_name": "Create Prompt as user",
"success": false,
"message": "Failed to create prompt: 401",
"details": null
},
{
"test_name": "Modify Prompt as user",
"success": false,
"message": "No prompts available to modify",
"details": null
},
{
"test_name": "Delete Prompt as user",
"success": false,
"message": "No prompts available to delete",
"details": null
},
{
"test_name": "View Users as sysadmin",
"success": false,
"message": "Failed to view users: 405",
"details": null
},
{
"test_name": "Create User as sysadmin",
"success": false,
"message": "Failed to create user: 401",
"details": null
},
{
"test_name": "Modify User as sysadmin",
"success": false,
"message": "No users available to modify",
"details": null
},
{
"test_name": "Delete User as sysadmin",
"success": false,
"message": "No users available to delete",
"details": null
},
{
"test_name": "View Users as admin",
"success": false,
"message": "Failed to view users: 405",
"details": null
},
{
"test_name": "Create User as admin",
"success": false,
"message": "Failed to create user: 401",
"details": null
},
{
"test_name": "Modify User as admin",
"success": false,
"message": "No users available to modify",
"details": null
},
{
"test_name": "Delete User as admin",
"success": false,
"message": "No users available to delete",
"details": null
},
{
"test_name": "View Users as user",
"success": false,
"message": "Failed to view users: 405",
"details": null
},
{
"test_name": "Create User as user",
"success": false,
"message": "Failed to create user: 401",
"details": null
},
{
"test_name": "Modify User as user",
"success": false,
"message": "No users available to modify",
"details": null
},
{
"test_name": "Delete User as user",
"success": false,
"message": "No users available to delete",
"details": null
}
],
"bugs_found": [],
"required_adaptations": []
}