cleanup
This commit is contained in:
ValueOn AG
parent
9a355a628f
commit
1df7c6b591
2 changed files with 0 additions and 668 deletions
|
|
@ -1,239 +0,0 @@
|
|||
# Workflow Implementation Gaps Analysis
|
||||
|
||||
## Overview
|
||||
|
||||
After refactoring the AI process for fast track and full track, the workflow implementation does not match the architecture described in `ai_plan_architecture.md`. This document identifies the gaps and missing implementations.
|
||||
|
||||
---
|
||||
|
||||
## Critical Issues Found
|
||||
|
||||
### 1. ❌ Fast Path Implementation Missing
|
||||
|
||||
**Architecture Requirement** (`ai_plan_architecture.md` lines 377-395):
|
||||
- `detectComplexity()` function to determine "simple" | "moderate" | "complex"
|
||||
- `fastPathExecute()` function for simple requests (5-15s)
|
||||
- Fast path should skip full workflow and return directly
|
||||
|
||||
**Current Implementation**:
|
||||
- ❌ **NO** `detectComplexity()` function found
|
||||
- ❌ **NO** `fastPathExecute()` function found
|
||||
- ❌ **NO** complexity detection before workflow execution
|
||||
- ✅ User intention analysis exists but doesn't route to fast path
|
||||
|
||||
**Location**: Should be in `workflowManager.py` before `_planTasks()`
|
||||
|
||||
**Impact**: All requests go through full workflow, even simple ones that could be answered in 5-15s.
|
||||
|
||||
---
|
||||
|
||||
### 2. ❌ Task Numbering Always Shows "Task 0"
|
||||
|
||||
**Architecture Requirement** (`ai_plan_architecture.md` lines 117-152):
|
||||
- `ChatWorkflow.currentTask` should increment properly
|
||||
- `workflow.incrementTask()` should be called when starting new task
|
||||
- Task numbers should be 1-indexed for display
|
||||
|
||||
**Current Implementation**:
|
||||
- ✅ `ChatWorkflow.currentTask` field exists
|
||||
- ✅ `_updateWorkflowBeforeExecutingTask()` exists in `modeDynamic.py` (line 902)
|
||||
- ❌ **NOT CALLED** from `workflowManager._executeTasks()` (line 368)
|
||||
- ❌ Task index calculation uses `idx + 1` but never updates `workflow.currentTask`
|
||||
- ❌ Messages show `taskNumber: 0` because `currentTask` is never incremented
|
||||
|
||||
**Problem Location**: `workflowManager.py` line 368-396
|
||||
|
||||
**Current Code**:
|
||||
```python
|
||||
for idx, taskStep in enumerate(taskPlan.tasks):
|
||||
currentTaskIndex = idx + 1 # Calculated but never used to update workflow
|
||||
logger.info(f"Task {currentTaskIndex}/{totalTasks}: {taskStep.objective}")
|
||||
# ... task execution ...
|
||||
taskResult = await handling.executeTask(taskStep, workflow, taskContext)
|
||||
# ❌ Missing: workflow.currentTask = currentTaskIndex
|
||||
# ❌ Missing: workflow.incrementTask() or updateWorkflowBeforeExecutingTask()
|
||||
```
|
||||
|
||||
**Fix Required**:
|
||||
```python
|
||||
for idx, taskStep in enumerate(taskPlan.tasks):
|
||||
currentTaskIndex = idx + 1
|
||||
logger.info(f"Task {currentTaskIndex}/{totalTasks}: {taskStep.objective}")
|
||||
|
||||
# ✅ ADD: Update workflow before executing task
|
||||
handling.updateWorkflowBeforeExecutingTask(currentTaskIndex)
|
||||
|
||||
# ... task execution ...
|
||||
taskResult = await handling.executeTask(taskStep, workflow, taskContext)
|
||||
```
|
||||
|
||||
**Impact**: UI always shows "Task 0" instead of "Task 1", "Task 2", etc.
|
||||
|
||||
---
|
||||
|
||||
### 3. ❌ User Language Not in ChatMessages
|
||||
|
||||
**Architecture Requirement** (`ai_plan_architecture.md` lines 1300-1321):
|
||||
- ChatMessages should contain user-friendly text in **user's language**
|
||||
- User-facing workflows: Full `ChatMessage` with user language
|
||||
- Process automation: Minimal `ChatMessage` (system role)
|
||||
|
||||
**Current Implementation**:
|
||||
- ✅ User language detection exists (`workflowManager.py` line 256-277)
|
||||
- ✅ `userMessage` field exists in `TaskStep` and `TaskPlan`
|
||||
- ❌ **Messages use technical text** instead of user-friendly language
|
||||
- ❌ **No AI-generated user messages** in user's language
|
||||
- ❌ Messages show action names (`ai.process`, `outlook.readMails`) instead of user-friendly descriptions
|
||||
|
||||
**Problem Locations**:
|
||||
1. `messageCreator.py` line 136-145: Uses `action.userMessage` if available, but this is often missing
|
||||
2. `messageCreator.py` line 77: Task start message is just `"🚀 **Task {taskProgress}**"` - no user-friendly text
|
||||
3. `messageCreator.py` line 204: Task completion message is generic `"✅ Task completed successfully"` - not in user language
|
||||
|
||||
**Example Current Message**:
|
||||
```
|
||||
🚀 **Task 1**
|
||||
|
||||
💬 Analyze the provided documents and extract key information
|
||||
```
|
||||
|
||||
**Should Be** (in user's language, e.g., German):
|
||||
```
|
||||
🚀 **Aufgabe 1**
|
||||
|
||||
💬 Ich analysiere die bereitgestellten Dokumente und extrahiere die wichtigsten Informationen für Sie.
|
||||
```
|
||||
|
||||
**Fix Required**:
|
||||
1. Generate user-friendly messages in user's language during task planning
|
||||
2. Store user messages in `TaskStep.userMessage` (already exists but not populated)
|
||||
3. Use user messages in `messageCreator.py` instead of technical action names
|
||||
|
||||
**Impact**: Users see technical messages instead of friendly, localized messages.
|
||||
|
||||
---
|
||||
|
||||
### 4. ❌ Workflow-Level Architecture Not Implemented
|
||||
|
||||
**Architecture Requirement** (`ai_plan_architecture.md` lines 368-428):
|
||||
- `RequestContext` model for normalized request
|
||||
- `UnderstandingResult` model for initial understanding
|
||||
- `TaskDefinition` model with `extractionOptions`
|
||||
- `TaskResult` model for task results
|
||||
- `persistTaskResult()` function to create ChatMessages
|
||||
|
||||
**Current Implementation**:
|
||||
- ❌ **NO** `RequestContext` model
|
||||
- ❌ **NO** `UnderstandingResult` model
|
||||
- ❌ **NO** `TaskDefinition` model (using `TaskStep` instead)
|
||||
- ❌ **NO** `TaskResult` model (using `ActionResult` instead)
|
||||
- ❌ **NO** `persistTaskResult()` function
|
||||
- ✅ `TaskStep` exists but doesn't match `TaskDefinition` structure
|
||||
|
||||
**Impact**: Workflow-level architecture described in `ai_plan_architecture.md` is not implemented. The system uses old `TaskStep` model instead of new `TaskDefinition` model.
|
||||
|
||||
---
|
||||
|
||||
### 5. ❌ Initial Understanding Phase Missing
|
||||
|
||||
**Architecture Requirement** (`ai_plan_architecture.md` lines 411-414):
|
||||
- `initialUnderstanding(context)` → `UnderstandingResult`
|
||||
- Combined AI call: parameters + intention + context + tasks
|
||||
- Creates `TaskDefinition[]` from `UnderstandingResult`
|
||||
|
||||
**Current Implementation**:
|
||||
- ✅ User intention analysis exists (`workflowManager.py` line 221-289)
|
||||
- ❌ **NOT** structured as `initialUnderstanding()` function
|
||||
- ❌ **NOT** returning `UnderstandingResult` model
|
||||
- ❌ **NOT** creating `TaskDefinition[]` - creates `TaskStep[]` instead
|
||||
- ❌ **NO** combined AI call - separate calls for different purposes
|
||||
|
||||
**Impact**: Initial understanding phase doesn't match architecture. Uses old approach instead of new unified `UnderstandingResult` model.
|
||||
|
||||
---
|
||||
|
||||
## Summary of Missing Implementations
|
||||
|
||||
| Component | Status | Priority | Location |
|
||||
|-----------|--------|----------|----------|
|
||||
| Fast Path (`fastPathExecute`) | ❌ Missing | 🔴 Critical | `workflowManager.py` |
|
||||
| Complexity Detection (`detectComplexity`) | ❌ Missing | 🔴 Critical | `workflowManager.py` |
|
||||
| Task Numbering Update | ❌ Not Called | 🔴 Critical | `workflowManager._executeTasks()` |
|
||||
| User Language Messages | ⚠️ Partial | 🟡 High | `messageCreator.py` |
|
||||
| RequestContext Model | ❌ Missing | 🟡 High | `datamodelWorkflow.py` |
|
||||
| UnderstandingResult Model | ❌ Missing | 🟡 High | `datamodelWorkflow.py` |
|
||||
| TaskDefinition Model | ❌ Missing | 🟡 High | `datamodelWorkflow.py` |
|
||||
| TaskResult Model | ❌ Missing | 🟡 High | `datamodelWorkflow.py` |
|
||||
| persistTaskResult() | ❌ Missing | 🟡 High | `workflowProcessor.py` |
|
||||
| initialUnderstanding() | ❌ Missing | 🟡 High | `workflowProcessor.py` |
|
||||
|
||||
---
|
||||
|
||||
## Recommended Fix Order
|
||||
|
||||
### Phase 1: Critical Fixes (Immediate)
|
||||
1. **Fix Task Numbering** (30 min)
|
||||
- Call `updateWorkflowBeforeExecutingTask()` in `workflowManager._executeTasks()`
|
||||
- Verify `currentTask` increments properly
|
||||
|
||||
2. **Add Fast Path Detection** (2 hours)
|
||||
- Implement `detectComplexity()` function
|
||||
- Add routing logic to `fastPathExecute()` for simple requests
|
||||
- Test with simple vs. complex prompts
|
||||
|
||||
### Phase 2: User Experience (High Priority)
|
||||
3. **Generate User-Friendly Messages** (4 hours)
|
||||
- Add AI call to generate user messages in user's language
|
||||
- Update `messageCreator.py` to use user messages
|
||||
- Test with different languages
|
||||
|
||||
### Phase 3: Architecture Alignment (Medium Priority)
|
||||
4. **Implement Workflow-Level Models** (8 hours)
|
||||
- Create `RequestContext`, `UnderstandingResult`, `TaskDefinition`, `TaskResult` models
|
||||
- Migrate from `TaskStep` to `TaskDefinition`
|
||||
- Update all call sites
|
||||
|
||||
5. **Implement Initial Understanding** (4 hours)
|
||||
- Create `initialUnderstanding()` function
|
||||
- Return `UnderstandingResult` model
|
||||
- Create `TaskDefinition[]` from result
|
||||
|
||||
---
|
||||
|
||||
## Code References
|
||||
|
||||
### Current Task Execution Flow
|
||||
- **Entry Point**: `workflowManager.py` line 166 → `_executeTasks()`
|
||||
- **Task Loop**: `workflowManager.py` line 368-396
|
||||
- **Task Execution**: `workflowProcessor.py` → `executeTask()`
|
||||
- **Message Creation**: `messageCreator.py` → `createTaskStartMessage()`, `createActionMessage()`
|
||||
|
||||
### Missing Fast Path Flow
|
||||
- **Should Be**: `workflowManager.py` → `detectComplexity()` → `fastPathExecute()` OR `_planTasks()`
|
||||
- **Currently**: Always goes to `_planTasks()` → `_executeTasks()`
|
||||
|
||||
### Task Numbering Issue
|
||||
- **Problem**: `workflowManager._executeTasks()` line 369 calculates `currentTaskIndex` but never updates `workflow.currentTask`
|
||||
- **Solution**: Call `handling.updateWorkflowBeforeExecutingTask(currentTaskIndex)` before `executeTask()`
|
||||
|
||||
---
|
||||
|
||||
## Testing Checklist
|
||||
|
||||
After fixes, verify:
|
||||
- [ ] Simple requests use fast path (5-15s response time)
|
||||
- [ ] Complex requests use full workflow (30-120s response time)
|
||||
- [ ] Task numbers increment correctly (Task 1, Task 2, Task 3...)
|
||||
- [ ] ChatMessages contain user-friendly text in user's language
|
||||
- [ ] User messages are generated, not technical action names
|
||||
- [ ] Workflow state (`currentTask`, `currentRound`, `currentAction`) updates correctly
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. **Immediate**: Fix task numbering (30 min fix, high impact)
|
||||
2. **This Week**: Implement fast path detection and routing
|
||||
3. **Next Sprint**: Add user-friendly message generation
|
||||
4. **Future**: Migrate to workflow-level models (`TaskDefinition`, `UnderstandingResult`, etc.)
|
||||
|
||||
|
|
@ -1,429 +0,0 @@
|
|||
# ChatLog Erzeugung - Übersicht
|
||||
|
||||
Diese Übersicht zeigt alle Stellen im `modules` Verzeichnis, wo ChatLog-Datensätze erzeugt werden.
|
||||
|
||||
## Status der Dokumentation
|
||||
|
||||
- ✅ **Kapitel 6, 5, 4**: Vollständig und korrekt dokumentiert
|
||||
- 📋 **Kapitel 3**: Vollständig erweitert mit allen `progressLogStart/Update/Finish` Aufrufen
|
||||
- ✅ **Kapitel 2, 1**: Vollständig dokumentiert (Service-Layer und Datenbank-Methoden)
|
||||
|
||||
## Erzeugungswege
|
||||
|
||||
ChatLog-Datensätze werden über zwei Hauptwege erzeugt:
|
||||
|
||||
1. **Direkter Weg**: `interfaceDbChatObjects.createLog()` - erstellt direkt in der Datenbank
|
||||
2. **Service-Weg**: `mainServiceChat.storeLog()` → `interfaceDbChatObjects.createLog()` - erstellt über Service-Layer
|
||||
|
||||
## Übersicht nach Datei
|
||||
|
||||
### 1. `gateway/modules/interfaces/interfaceDbChatObjects.py`
|
||||
|
||||
**Hauptmethode für ChatLog-Erzeugung:**
|
||||
- **Zeile 1042-1091**: `createLog()` - Direkte Erzeugung von ChatLog-Datensätzen in der Datenbank
|
||||
- Wird von `storeLog()` aufgerufen
|
||||
- Validiert Daten gegen ChatLog-Modell
|
||||
- Erstellt Datensatz in normalisierter Tabelle
|
||||
|
||||
### 2. `gateway/modules/services/serviceChat/mainServiceChat.py`
|
||||
|
||||
**Service-Layer Methode:**
|
||||
- **Zeile 605-622**: `storeLog()` - Wrapper um `createLog()`
|
||||
- Fügt `workflowId` hinzu
|
||||
- Synchronisiert mit in-memory workflow.logs Liste
|
||||
- **Wird von vielen Stellen aufgerufen** (siehe unten)
|
||||
|
||||
### 3. `gateway/modules/shared/progressLogger.py`
|
||||
|
||||
**Progress-Logging System:**
|
||||
- **Zeile 102-132**: `_logProgress()` - Erstellt ChatLog für Progress-Updates
|
||||
- Wird von `startOperation()`, `updateOperation()`, `finishOperation()` aufgerufen
|
||||
- Erstellt Logs für alle Progress-Updates während Operationen
|
||||
- **Häufige Erzeugung**: Bei jedem Progress-Update wird ein ChatLog erstellt
|
||||
|
||||
**Alle Aufrufe von `progressLogStart()`, `progressLogUpdate()`, `progressLogFinish()`:**
|
||||
|
||||
#### 3.1. `gateway/modules/workflows/processing/workflowProcessor.py`
|
||||
|
||||
**Task Plan Generation:**
|
||||
- **Zeile 51**: `progressLogStart()` - Start Task Plan Generation
|
||||
```python
|
||||
self.services.chat.progressLogStart(
|
||||
operationId,
|
||||
"Workflow Planning",
|
||||
"Task Plan Generation",
|
||||
f"Mode: {workflow.workflowMode.value}"
|
||||
)
|
||||
```
|
||||
- **Zeile 68**: `progressLogUpdate(0.3)` - "Analyzing input"
|
||||
- **Zeile 74**: `progressLogUpdate(0.8)` - "Creating plan"
|
||||
- **Zeile 80**: `progressLogFinish(True)` - Erfolg
|
||||
- **Zeile 86**: `progressLogFinish(False)` - Fehler
|
||||
|
||||
**Task Execution:**
|
||||
- **Zeile 104**: `progressLogStart()` - Start Task Execution
|
||||
```python
|
||||
self.services.chat.progressLogStart(
|
||||
operationId,
|
||||
"Workflow Execution",
|
||||
"Task Execution",
|
||||
f"Task {taskIndex}"
|
||||
)
|
||||
```
|
||||
- **Zeile 117**: `progressLogUpdate(0.2)` - "Executing"
|
||||
- **Zeile 123**: `progressLogFinish(True)` - Erfolg
|
||||
- **Zeile 129**: `progressLogFinish(False)` - Fehler
|
||||
|
||||
#### 3.2. `gateway/modules/services/serviceAi/mainServiceAi.py`
|
||||
|
||||
**AI Call with Looping (_callAiWithLooping):**
|
||||
- **Zeile 198**: `progressLogUpdate(0.5)` - "Starting AI call iteration {iteration}" (erste Iteration)
|
||||
- **Zeile 202**: `progressLogUpdate(baseProgress)` - "Continuing generation (iteration {iteration})" (weitere Iterationen)
|
||||
- **Zeile 220**: `progressLogUpdate(0.51)` - "Calling AI model" (erste Iteration)
|
||||
- **Zeile 239**: `progressLogUpdate(0.6)` - "AI response received (iteration {iteration})" (erste Iteration)
|
||||
- **Zeile 242**: `progressLogUpdate(progress)` - "Processing response (iteration {iteration})" (weitere Iterationen)
|
||||
- **Zeile 284**: `progressLogUpdate(0.65 + ...)` - "Extracted {len} sections (iteration {iteration})"
|
||||
- **Zeile 304**: `progressLogUpdate(0.95)` - "Generation complete ({iteration} iterations, {len} sections)"
|
||||
|
||||
**AI Content Processing (callAiContent):**
|
||||
- **Zeile 568**: `progressLogStart()` - Start AI Content Processing
|
||||
```python
|
||||
self.services.chat.progressLogStart(
|
||||
aiOperationId,
|
||||
"AI content processing",
|
||||
"Content Processing",
|
||||
f"Format: {outputFormat or 'text'}"
|
||||
)
|
||||
```
|
||||
- **Zeile 593**: `progressLogUpdate(0.1)` - "Analyzing prompt parameters"
|
||||
- **Zeile 613**: `progressLogUpdate(0.4)` - "Calling AI for image generation" (IMAGE_GENERATE)
|
||||
- **Zeile 642**: `progressLogUpdate(0.9)` - "Image generated"
|
||||
- **Zeile 643**: `progressLogFinish(True)` - Erfolg (IMAGE_GENERATE)
|
||||
- **Zeile 653**: `progressLogFinish(False)` - Fehler (IMAGE_GENERATE)
|
||||
- **Zeile 658**: `progressLogUpdate(0.4)` - "Calling AI for {opType.name}" (WEB_SEARCH/WEB_CRAWL)
|
||||
- **Zeile 679**: `progressLogUpdate(0.9)` - "{opType.name} completed"
|
||||
- **Zeile 680**: `progressLogFinish(True)` - Erfolg (WEB_SEARCH/WEB_CRAWL)
|
||||
- **Zeile 689**: `progressLogFinish(False)` - Fehler (WEB_SEARCH/WEB_CRAWL)
|
||||
- **Zeile 705**: `progressLogUpdate(0.3)` - "Building generation prompt" (Document Generation)
|
||||
- **Zeile 719**: `progressLogUpdate(0.4)` - "Calling AI for content generation" (Document Generation)
|
||||
- **Zeile 729**: `progressLogUpdate(0.7)` - "Parsing generated JSON" (Document Generation)
|
||||
- **Zeile 736**: `progressLogFinish(False)` - Fehler JSON Parsing
|
||||
- **Zeile 758**: `progressLogUpdate(0.8)` - "Rendering to {outputFormat} format" (Document Generation)
|
||||
- **Zeile 796**: `progressLogFinish(True)` - Erfolg (Document Generation)
|
||||
- **Zeile 806**: `progressLogFinish(False)` - Fehler Rendering
|
||||
- **Zeile 810**: `progressLogUpdate(0.5)` - "Processing text call" (Text Processing)
|
||||
- **Zeile 830**: `progressLogFinish(True)` - Erfolg (Text Processing)
|
||||
- **Zeile 839**: `progressLogFinish(False)` - Fehler (Text Processing)
|
||||
|
||||
#### 3.3. `gateway/modules/services/serviceExtraction/mainServiceExtraction.py`
|
||||
|
||||
**Process Documents Per Chunk (processDocumentsPerChunk):**
|
||||
- **Zeile 424**: `progressLogStart()` - Start Document Processing
|
||||
```python
|
||||
self.services.chat.progressLogStart(
|
||||
operationId,
|
||||
"AI Text Extract",
|
||||
"Document Processing",
|
||||
f"Processing {len(documents)} documents"
|
||||
)
|
||||
```
|
||||
- **Zeile 451**: `progressLogUpdate(0.1)` - "Extracting content from {len} documents"
|
||||
- **Zeile 456**: `progressLogFinish(False)` - Fehler bei Extraction
|
||||
- **Zeile 461**: `progressLogUpdate(0.3)` - "Processing {len} extracted content parts"
|
||||
- **Zeile 466**: `progressLogUpdate(0.9)` - "Merging {len} part results"
|
||||
- **Zeile 473**: `progressLogFinish(True)` - Erfolg
|
||||
- **Zeile 479**: `progressLogFinish(False)` - Fehler
|
||||
- **Zeile 538**: `progressLogUpdate(progress)` - "Processing part {processedCount}/{totalParts}" (in Chunking Callback)
|
||||
|
||||
#### 3.4. `gateway/modules/workflows/methods/methodAi.py`
|
||||
|
||||
**AI Process (process):**
|
||||
- **Zeile 52**: `progressLogStart()` - Start AI Processing
|
||||
```python
|
||||
self.services.chat.progressLogStart(
|
||||
operationId,
|
||||
"Generate",
|
||||
"AI Processing",
|
||||
f"Format: {parameters.get('resultType', 'txt')}"
|
||||
)
|
||||
```
|
||||
- **Zeile 63**: `progressLogUpdate(0.2)` - "Preparing parameters"
|
||||
- **Zeile 111**: `progressLogUpdate(0.3)` - "Extracting content from documents"
|
||||
- **Zeile 145**: `progressLogUpdate(0.4)` - "Preparing AI call"
|
||||
- **Zeile 155**: `progressLogUpdate(0.6)` - "Calling AI"
|
||||
- **Zeile 166**: `progressLogUpdate(0.8)` - "Processing result"
|
||||
- **Zeile 211**: `progressLogFinish(True)` - Erfolg
|
||||
- **Zeile 220**: `progressLogFinish(False)` - Fehler
|
||||
|
||||
**Extract Content (extractContent):**
|
||||
- **Zeile 251**: `progressLogStart()` - Start Content Extraction
|
||||
```python
|
||||
self.services.chat.progressLogStart(
|
||||
operationId,
|
||||
"Extracting content from documents",
|
||||
"Content Extraction",
|
||||
f"Documents: {len(parameters.documentList.references)}"
|
||||
)
|
||||
```
|
||||
- **Zeile 259**: `progressLogUpdate(0.2)` - "Loading documents"
|
||||
- **Zeile 263**: `progressLogFinish(False)` - Fehler (keine Dokumente)
|
||||
- **Zeile 269**: `progressLogUpdate(0.3)` - "Preparing extraction options"
|
||||
- **Zeile 286**: `progressLogUpdate(0.5)` - "Extracting content from {len} documents"
|
||||
- **Zeile 290**: `progressLogUpdate(0.8)` - "Building result documents"
|
||||
- **Zeile 301**: `progressLogFinish(True)` - Erfolg
|
||||
- **Zeile 310**: `progressLogFinish(False)` - Fehler
|
||||
|
||||
**Generate Document (generateDocument):**
|
||||
- **Zeile 341**: `progressLogStart()` - Start Document Generation
|
||||
```python
|
||||
self.services.chat.progressLogStart(...)
|
||||
```
|
||||
- **Zeile 359**: `progressLogFinish(True)` - Erfolg
|
||||
- **Zeile 401**: `progressLogFinish(False)` - Fehler
|
||||
|
||||
#### 3.5. `gateway/modules/services/serviceWeb/mainServiceWeb.py`
|
||||
|
||||
**Research (research) - nur Updates, kein Start/Finish:**
|
||||
- **Zeile 50**: `progressLogUpdate(0.1)` - "Analyzing research intent"
|
||||
- **Zeile 75**: `progressLogUpdate(0.3)` - "Searching for URLs"
|
||||
- **Zeile 87**: `progressLogUpdate(0.5)` - "Found {len} total URLs"
|
||||
- **Zeile 102**: `progressLogUpdate(0.6)` - "Crawling {len} URLs"
|
||||
- **Zeile 110**: `progressLogUpdate(0.9)` - "Consolidating results"
|
||||
|
||||
**Hinweis**: `serviceWeb` verwendet nur `progressLogUpdate()`, aber kein `progressLogStart()` oder `progressLogFinish()`. Dies könnte ein Bug sein oder die Operation wird von einem anderen Service gestartet/beendet.
|
||||
|
||||
### 4. `gateway/modules/workflows/workflowManager.py`
|
||||
|
||||
**Workflow-Management - 11 ChatLog-Erzeugungen:**
|
||||
|
||||
- **Zeile 58**: Workflow gestoppt für neuen Prompt
|
||||
```python
|
||||
self.services.chat.storeLog(workflow, {
|
||||
"message": "Workflow stopped for new prompt",
|
||||
"type": "info",
|
||||
"status": "stopped",
|
||||
"progress": 1.0
|
||||
})
|
||||
```
|
||||
|
||||
- **Zeile 79**: Workflow fortgesetzt
|
||||
```python
|
||||
self.services.chat.storeLog(workflow, {
|
||||
"message": f"Workflow resumed (round {workflow.currentRound}) with mode: {workflowMode}",
|
||||
"type": "info",
|
||||
"status": "running",
|
||||
"progress": 0
|
||||
})
|
||||
```
|
||||
|
||||
- **Zeile 140**: Workflow gestoppt (workflowStop)
|
||||
```python
|
||||
self.services.chat.storeLog(workflow, {
|
||||
"message": "Workflow stopped",
|
||||
"type": "warning",
|
||||
"status": "stopped",
|
||||
"progress": 1.0
|
||||
})
|
||||
```
|
||||
|
||||
- **Zeile 635**: Workflow gestoppt durch Benutzer (in _processWorkflowResults)
|
||||
```python
|
||||
self.services.chat.storeLog(workflow, {
|
||||
"message": "Workflow stopped by user",
|
||||
"type": "warning",
|
||||
"status": "stopped",
|
||||
"progress": 1.0
|
||||
})
|
||||
```
|
||||
|
||||
- **Zeile 674**: Workflow fehlgeschlagen (in _processWorkflowResults)
|
||||
```python
|
||||
self.services.chat.storeLog(workflow, {
|
||||
"message": "Workflow failed: Unknown error",
|
||||
"type": "error",
|
||||
"status": "failed",
|
||||
"progress": 1.0
|
||||
})
|
||||
```
|
||||
|
||||
- **Zeile 762**: Workflow abgeschlossen (in _sendLastMessage)
|
||||
```python
|
||||
self.services.chat.storeLog(workflow, {
|
||||
"message": "Workflow completed",
|
||||
"type": "success",
|
||||
"status": "completed",
|
||||
"progress": 1.0
|
||||
})
|
||||
```
|
||||
|
||||
- **Zeile 837**: Workflow gestoppt (in _handleWorkflowStop)
|
||||
```python
|
||||
self.services.chat.storeLog(workflow, {
|
||||
"message": "Workflow stopped by user",
|
||||
"type": "warning",
|
||||
"status": "stopped",
|
||||
"progress": 1.0
|
||||
})
|
||||
```
|
||||
|
||||
- **Zeile 880**: Workflow-Fehler (in _handleWorkflowError)
|
||||
```python
|
||||
self.services.chat.storeLog(workflow, {
|
||||
"message": f"Workflow failed: {str(error)}",
|
||||
"type": "error",
|
||||
"status": "failed",
|
||||
"progress": 1.0
|
||||
})
|
||||
```
|
||||
|
||||
- **Zeile 931**: Binäre Datei Info (in _processFileIds - Neutralization)
|
||||
```python
|
||||
self.services.chat.storeLog(workflow, {
|
||||
"message": infoMsg,
|
||||
"type": "info",
|
||||
"status": "running",
|
||||
"progress": 50
|
||||
})
|
||||
```
|
||||
|
||||
- **Zeile 964**: Neutralization Fehler (in _processFileIds)
|
||||
```python
|
||||
self.services.chat.storeLog(workflow, {
|
||||
"message": errorMsg,
|
||||
"type": "error",
|
||||
"status": "error",
|
||||
"progress": -1
|
||||
})
|
||||
```
|
||||
|
||||
- **Zeile 974**: Neutralization Fehler (Exception) (in _processFileIds)
|
||||
```python
|
||||
self.services.chat.storeLog(workflow, {
|
||||
"message": errorMsg,
|
||||
"type": "error",
|
||||
"status": "error",
|
||||
"progress": -1
|
||||
})
|
||||
```
|
||||
|
||||
- **Zeile 999**: Datei-Verarbeitungsfehler (in _processFileIds)
|
||||
```python
|
||||
self.services.chat.storeLog(workflow, {
|
||||
"message": errorMsg,
|
||||
"type": "error",
|
||||
"status": "error",
|
||||
"progress": -1
|
||||
})
|
||||
```
|
||||
|
||||
### 5. `gateway/modules/workflows/processing/core/actionExecutor.py`
|
||||
|
||||
**Action Execution:**
|
||||
- **Zeile 148**: Action-Fehler
|
||||
```python
|
||||
self.services.chat.storeLog(workflow, {
|
||||
"message": f"❌ **Task {taskNum}**❌ **Action {actionNum}** failed: {result.error}",
|
||||
"type": "error",
|
||||
"progress": 1.0
|
||||
})
|
||||
```
|
||||
|
||||
### 6. `gateway/modules/services/serviceExtraction/mainServiceExtraction.py`
|
||||
|
||||
**Extraction Service:**
|
||||
- **Zeile 553**: Chunking Progress Callback
|
||||
```python
|
||||
self.services.chat.storeLog(workflow, logData)
|
||||
```
|
||||
- Wird während AI-Chunking aufgerufen
|
||||
- **Häufige Erzeugung**: Bei jedem Chunking-Progress-Update
|
||||
|
||||
## Zusammenfassung nach Kategorie
|
||||
|
||||
### ✅ Kapitel 6, 5, 4: Korrekt dokumentiert
|
||||
|
||||
Diese Kapitel sind vollständig und korrekt dokumentiert:
|
||||
- **Kapitel 6**: Extraction Service Chunking (`mainServiceExtraction.py` Zeile 553)
|
||||
- **Kapitel 5**: Action Executor (`actionExecutor.py` Zeile 148)
|
||||
- **Kapitel 4**: Workflow Manager (`workflowManager.py` - 11 Stellen)
|
||||
|
||||
### 📋 Kapitel 3: ProgressLogger Aufrufe - Vollständige Liste
|
||||
|
||||
**Alle Aufrufe von `progressLogStart()`, `progressLogUpdate()`, `progressLogFinish()`:**
|
||||
|
||||
**Gesamtanzahl Aufrufe:**
|
||||
- **startOperation**: 7 Aufrufe
|
||||
- workflowProcessor.py: 2x (Task Plan, Task Execution)
|
||||
- mainServiceAi.py: 1x (AI Content Processing)
|
||||
- mainServiceExtraction.py: 1x (Document Processing)
|
||||
- methodAi.py: 3x (AI Process, Extract Content, Generate Document)
|
||||
|
||||
- **updateOperation**: ~35+ Aufrufe
|
||||
- workflowProcessor.py: 2x
|
||||
- mainServiceAi.py: ~15x (AI Looping + Content Processing)
|
||||
- mainServiceExtraction.py: 4x (+ Chunking Callback)
|
||||
- methodAi.py: 8x
|
||||
- mainServiceWeb.py: 5x (ohne Start/Finish!)
|
||||
|
||||
- **finishOperation**: ~15 Aufrufe
|
||||
- workflowProcessor.py: 4x (2x Success, 2x Failure)
|
||||
- mainServiceAi.py: ~6x (verschiedene Operationen)
|
||||
- mainServiceExtraction.py: 2x
|
||||
- methodAi.py: 6x
|
||||
|
||||
**Problematische Stellen:**
|
||||
1. **Zu viele Update-Aufrufe**: Besonders in `_callAiWithLooping()` werden bei jeder Iteration mehrere Updates erstellt
|
||||
2. **Fehlende Start/Finish**: `mainServiceWeb.py` verwendet nur Updates ohne Start/Finish
|
||||
3. **Redundante Updates**: Viele Updates mit ähnlichen Progress-Werten (z.B. 0.4, 0.5, 0.6)
|
||||
|
||||
### 📋 Kapitel 2 und 1: Vollständig dokumentiert
|
||||
|
||||
- **Kapitel 2**: `mainServiceChat.storeLog()` - Service-Layer Methode
|
||||
- **Kapitel 1**: `interfaceDbChatObjects.createLog()` - Datenbank-Methode
|
||||
|
||||
## Empfehlungen zur Reduzierung
|
||||
|
||||
1. **ProgressLogger**:
|
||||
- Nur bei Start/Finish loggen, nicht bei jedem Update
|
||||
- Oder: Batch-Updates alle N Sekunden
|
||||
|
||||
2. **Chunking Progress**:
|
||||
- Nur bei wichtigen Meilensteinen (z.B. 25%, 50%, 75%, 100%)
|
||||
- Nicht bei jedem einzelnen Chunk
|
||||
|
||||
3. **Workflow Status**:
|
||||
- Behalten, aber Duplikate entfernen (z.B. mehrere "stopped" Logs)
|
||||
|
||||
4. **Fehler-Logging**:
|
||||
- Nur kritische Fehler loggen, nicht alle Warnungen/Infos
|
||||
|
||||
## Statistik
|
||||
|
||||
### ChatLog-Erzeugungen nach Quelle:
|
||||
|
||||
1. **ProgressLogger (Kapitel 3)**: ~57+ Aufrufe
|
||||
- 7x `startOperation()` → erzeugt ChatLog
|
||||
- ~35+ `updateOperation()` → erzeugt ChatLog bei jedem Update
|
||||
- ~15x `finishOperation()` → erzeugt ChatLog
|
||||
- **Häufigste Quelle** - kann bei langen Operationen sehr viele Logs erzeugen
|
||||
|
||||
2. **Extraction Service Chunking (Kapitel 6)**: Variable Häufigkeit
|
||||
- Wird bei jedem Chunking-Progress-Update aufgerufen
|
||||
- **Zweithäufigste Quelle** - abhängig von Anzahl der Chunks
|
||||
|
||||
3. **Workflow Manager (Kapitel 4)**: 11 feste Stellen
|
||||
- Workflow Status-Änderungen (einmalig pro Status-Übergang)
|
||||
- Fehler-Logging bei Datei-Verarbeitung
|
||||
|
||||
4. **Action Executor (Kapitel 5)**: 1 Stelle
|
||||
- Action-Fehler-Logging
|
||||
|
||||
### Gesamtübersicht:
|
||||
|
||||
- **Direkte `storeLog()` Aufrufe**: 12 Stellen (Kapitel 4, 5, 6)
|
||||
- **ProgressLogger Aufrufe**: ~57+ Stellen (Kapitel 3)
|
||||
- **Gesamtanzahl Code-Stellen**: ~70+ Stellen
|
||||
- **Potenzial für Reduzierung**:
|
||||
- ProgressLogger: ~35+ Update-Aufrufe könnten reduziert werden
|
||||
- Chunking: Variable, abhängig von Chunk-Anzahl
|
||||
|
||||
Loading…
Reference in a new issue