From 1df7c6b591e9defbc16dd4a393353ac68f6a3684 Mon Sep 17 00:00:00 2001 From: ValueOn AG Date: Wed, 19 Nov 2025 10:39:57 +0100 Subject: [PATCH] cleanup --- WORKFLOW_IMPLEMENTATION_GAPS.md | 239 --------------- modules/CHATLOG_CREATION_OVERVIEW.md | 429 --------------------------- 2 files changed, 668 deletions(-) delete mode 100644 WORKFLOW_IMPLEMENTATION_GAPS.md delete mode 100644 modules/CHATLOG_CREATION_OVERVIEW.md diff --git a/WORKFLOW_IMPLEMENTATION_GAPS.md b/WORKFLOW_IMPLEMENTATION_GAPS.md deleted file mode 100644 index 6eb68ed9..00000000 --- a/WORKFLOW_IMPLEMENTATION_GAPS.md +++ /dev/null @@ -1,239 +0,0 @@ -# Workflow Implementation Gaps Analysis - -## Overview - -After refactoring the AI process for fast track and full track, the workflow implementation does not match the architecture described in `ai_plan_architecture.md`. This document identifies the gaps and missing implementations. - ---- - -## Critical Issues Found - -### 1. ❌ Fast Path Implementation Missing - -**Architecture Requirement** (`ai_plan_architecture.md` lines 377-395): -- `detectComplexity()` function to determine "simple" | "moderate" | "complex" -- `fastPathExecute()` function for simple requests (5-15s) -- Fast path should skip full workflow and return directly - -**Current Implementation**: -- ❌ **NO** `detectComplexity()` function found -- ❌ **NO** `fastPathExecute()` function found -- ❌ **NO** complexity detection before workflow execution -- ✅ User intention analysis exists but doesn't route to fast path - -**Location**: Should be in `workflowManager.py` before `_planTasks()` - -**Impact**: All requests go through full workflow, even simple ones that could be answered in 5-15s. - ---- - -### 2. ❌ Task Numbering Always Shows "Task 0" - -**Architecture Requirement** (`ai_plan_architecture.md` lines 117-152): -- `ChatWorkflow.currentTask` should increment properly -- `workflow.incrementTask()` should be called when starting new task -- Task numbers should be 1-indexed for display - -**Current Implementation**: -- ✅ `ChatWorkflow.currentTask` field exists -- ✅ `_updateWorkflowBeforeExecutingTask()` exists in `modeDynamic.py` (line 902) -- ❌ **NOT CALLED** from `workflowManager._executeTasks()` (line 368) -- ❌ Task index calculation uses `idx + 1` but never updates `workflow.currentTask` -- ❌ Messages show `taskNumber: 0` because `currentTask` is never incremented - -**Problem Location**: `workflowManager.py` line 368-396 - -**Current Code**: -```python -for idx, taskStep in enumerate(taskPlan.tasks): - currentTaskIndex = idx + 1 # Calculated but never used to update workflow - logger.info(f"Task {currentTaskIndex}/{totalTasks}: {taskStep.objective}") - # ... task execution ... - taskResult = await handling.executeTask(taskStep, workflow, taskContext) - # ❌ Missing: workflow.currentTask = currentTaskIndex - # ❌ Missing: workflow.incrementTask() or updateWorkflowBeforeExecutingTask() -``` - -**Fix Required**: -```python -for idx, taskStep in enumerate(taskPlan.tasks): - currentTaskIndex = idx + 1 - logger.info(f"Task {currentTaskIndex}/{totalTasks}: {taskStep.objective}") - - # ✅ ADD: Update workflow before executing task - handling.updateWorkflowBeforeExecutingTask(currentTaskIndex) - - # ... task execution ... - taskResult = await handling.executeTask(taskStep, workflow, taskContext) -``` - -**Impact**: UI always shows "Task 0" instead of "Task 1", "Task 2", etc. - ---- - -### 3. ❌ User Language Not in ChatMessages - -**Architecture Requirement** (`ai_plan_architecture.md` lines 1300-1321): -- ChatMessages should contain user-friendly text in **user's language** -- User-facing workflows: Full `ChatMessage` with user language -- Process automation: Minimal `ChatMessage` (system role) - -**Current Implementation**: -- ✅ User language detection exists (`workflowManager.py` line 256-277) -- ✅ `userMessage` field exists in `TaskStep` and `TaskPlan` -- ❌ **Messages use technical text** instead of user-friendly language -- ❌ **No AI-generated user messages** in user's language -- ❌ Messages show action names (`ai.process`, `outlook.readMails`) instead of user-friendly descriptions - -**Problem Locations**: -1. `messageCreator.py` line 136-145: Uses `action.userMessage` if available, but this is often missing -2. `messageCreator.py` line 77: Task start message is just `"🚀 **Task {taskProgress}**"` - no user-friendly text -3. `messageCreator.py` line 204: Task completion message is generic `"✅ Task completed successfully"` - not in user language - -**Example Current Message**: -``` -🚀 **Task 1** - -💬 Analyze the provided documents and extract key information -``` - -**Should Be** (in user's language, e.g., German): -``` -🚀 **Aufgabe 1** - -💬 Ich analysiere die bereitgestellten Dokumente und extrahiere die wichtigsten Informationen für Sie. -``` - -**Fix Required**: -1. Generate user-friendly messages in user's language during task planning -2. Store user messages in `TaskStep.userMessage` (already exists but not populated) -3. Use user messages in `messageCreator.py` instead of technical action names - -**Impact**: Users see technical messages instead of friendly, localized messages. - ---- - -### 4. ❌ Workflow-Level Architecture Not Implemented - -**Architecture Requirement** (`ai_plan_architecture.md` lines 368-428): -- `RequestContext` model for normalized request -- `UnderstandingResult` model for initial understanding -- `TaskDefinition` model with `extractionOptions` -- `TaskResult` model for task results -- `persistTaskResult()` function to create ChatMessages - -**Current Implementation**: -- ❌ **NO** `RequestContext` model -- ❌ **NO** `UnderstandingResult` model -- ❌ **NO** `TaskDefinition` model (using `TaskStep` instead) -- ❌ **NO** `TaskResult` model (using `ActionResult` instead) -- ❌ **NO** `persistTaskResult()` function -- ✅ `TaskStep` exists but doesn't match `TaskDefinition` structure - -**Impact**: Workflow-level architecture described in `ai_plan_architecture.md` is not implemented. The system uses old `TaskStep` model instead of new `TaskDefinition` model. - ---- - -### 5. ❌ Initial Understanding Phase Missing - -**Architecture Requirement** (`ai_plan_architecture.md` lines 411-414): -- `initialUnderstanding(context)` → `UnderstandingResult` -- Combined AI call: parameters + intention + context + tasks -- Creates `TaskDefinition[]` from `UnderstandingResult` - -**Current Implementation**: -- ✅ User intention analysis exists (`workflowManager.py` line 221-289) -- ❌ **NOT** structured as `initialUnderstanding()` function -- ❌ **NOT** returning `UnderstandingResult` model -- ❌ **NOT** creating `TaskDefinition[]` - creates `TaskStep[]` instead -- ❌ **NO** combined AI call - separate calls for different purposes - -**Impact**: Initial understanding phase doesn't match architecture. Uses old approach instead of new unified `UnderstandingResult` model. - ---- - -## Summary of Missing Implementations - -| Component | Status | Priority | Location | -|-----------|--------|----------|----------| -| Fast Path (`fastPathExecute`) | ❌ Missing | 🔴 Critical | `workflowManager.py` | -| Complexity Detection (`detectComplexity`) | ❌ Missing | 🔴 Critical | `workflowManager.py` | -| Task Numbering Update | ❌ Not Called | 🔴 Critical | `workflowManager._executeTasks()` | -| User Language Messages | ⚠️ Partial | 🟡 High | `messageCreator.py` | -| RequestContext Model | ❌ Missing | 🟡 High | `datamodelWorkflow.py` | -| UnderstandingResult Model | ❌ Missing | 🟡 High | `datamodelWorkflow.py` | -| TaskDefinition Model | ❌ Missing | 🟡 High | `datamodelWorkflow.py` | -| TaskResult Model | ❌ Missing | 🟡 High | `datamodelWorkflow.py` | -| persistTaskResult() | ❌ Missing | 🟡 High | `workflowProcessor.py` | -| initialUnderstanding() | ❌ Missing | 🟡 High | `workflowProcessor.py` | - ---- - -## Recommended Fix Order - -### Phase 1: Critical Fixes (Immediate) -1. **Fix Task Numbering** (30 min) - - Call `updateWorkflowBeforeExecutingTask()` in `workflowManager._executeTasks()` - - Verify `currentTask` increments properly - -2. **Add Fast Path Detection** (2 hours) - - Implement `detectComplexity()` function - - Add routing logic to `fastPathExecute()` for simple requests - - Test with simple vs. complex prompts - -### Phase 2: User Experience (High Priority) -3. **Generate User-Friendly Messages** (4 hours) - - Add AI call to generate user messages in user's language - - Update `messageCreator.py` to use user messages - - Test with different languages - -### Phase 3: Architecture Alignment (Medium Priority) -4. **Implement Workflow-Level Models** (8 hours) - - Create `RequestContext`, `UnderstandingResult`, `TaskDefinition`, `TaskResult` models - - Migrate from `TaskStep` to `TaskDefinition` - - Update all call sites - -5. **Implement Initial Understanding** (4 hours) - - Create `initialUnderstanding()` function - - Return `UnderstandingResult` model - - Create `TaskDefinition[]` from result - ---- - -## Code References - -### Current Task Execution Flow -- **Entry Point**: `workflowManager.py` line 166 → `_executeTasks()` -- **Task Loop**: `workflowManager.py` line 368-396 -- **Task Execution**: `workflowProcessor.py` → `executeTask()` -- **Message Creation**: `messageCreator.py` → `createTaskStartMessage()`, `createActionMessage()` - -### Missing Fast Path Flow -- **Should Be**: `workflowManager.py` → `detectComplexity()` → `fastPathExecute()` OR `_planTasks()` -- **Currently**: Always goes to `_planTasks()` → `_executeTasks()` - -### Task Numbering Issue -- **Problem**: `workflowManager._executeTasks()` line 369 calculates `currentTaskIndex` but never updates `workflow.currentTask` -- **Solution**: Call `handling.updateWorkflowBeforeExecutingTask(currentTaskIndex)` before `executeTask()` - ---- - -## Testing Checklist - -After fixes, verify: -- [ ] Simple requests use fast path (5-15s response time) -- [ ] Complex requests use full workflow (30-120s response time) -- [ ] Task numbers increment correctly (Task 1, Task 2, Task 3...) -- [ ] ChatMessages contain user-friendly text in user's language -- [ ] User messages are generated, not technical action names -- [ ] Workflow state (`currentTask`, `currentRound`, `currentAction`) updates correctly - ---- - -## Next Steps - -1. **Immediate**: Fix task numbering (30 min fix, high impact) -2. **This Week**: Implement fast path detection and routing -3. **Next Sprint**: Add user-friendly message generation -4. **Future**: Migrate to workflow-level models (`TaskDefinition`, `UnderstandingResult`, etc.) - diff --git a/modules/CHATLOG_CREATION_OVERVIEW.md b/modules/CHATLOG_CREATION_OVERVIEW.md deleted file mode 100644 index 4c7b80b5..00000000 --- a/modules/CHATLOG_CREATION_OVERVIEW.md +++ /dev/null @@ -1,429 +0,0 @@ -# ChatLog Erzeugung - Übersicht - -Diese Übersicht zeigt alle Stellen im `modules` Verzeichnis, wo ChatLog-Datensätze erzeugt werden. - -## Status der Dokumentation - -- ✅ **Kapitel 6, 5, 4**: Vollständig und korrekt dokumentiert -- 📋 **Kapitel 3**: Vollständig erweitert mit allen `progressLogStart/Update/Finish` Aufrufen -- ✅ **Kapitel 2, 1**: Vollständig dokumentiert (Service-Layer und Datenbank-Methoden) - -## Erzeugungswege - -ChatLog-Datensätze werden über zwei Hauptwege erzeugt: - -1. **Direkter Weg**: `interfaceDbChatObjects.createLog()` - erstellt direkt in der Datenbank -2. **Service-Weg**: `mainServiceChat.storeLog()` → `interfaceDbChatObjects.createLog()` - erstellt über Service-Layer - -## Übersicht nach Datei - -### 1. `gateway/modules/interfaces/interfaceDbChatObjects.py` - -**Hauptmethode für ChatLog-Erzeugung:** -- **Zeile 1042-1091**: `createLog()` - Direkte Erzeugung von ChatLog-Datensätzen in der Datenbank - - Wird von `storeLog()` aufgerufen - - Validiert Daten gegen ChatLog-Modell - - Erstellt Datensatz in normalisierter Tabelle - -### 2. `gateway/modules/services/serviceChat/mainServiceChat.py` - -**Service-Layer Methode:** -- **Zeile 605-622**: `storeLog()` - Wrapper um `createLog()` - - Fügt `workflowId` hinzu - - Synchronisiert mit in-memory workflow.logs Liste - - **Wird von vielen Stellen aufgerufen** (siehe unten) - -### 3. `gateway/modules/shared/progressLogger.py` - -**Progress-Logging System:** -- **Zeile 102-132**: `_logProgress()` - Erstellt ChatLog für Progress-Updates - - Wird von `startOperation()`, `updateOperation()`, `finishOperation()` aufgerufen - - Erstellt Logs für alle Progress-Updates während Operationen - - **Häufige Erzeugung**: Bei jedem Progress-Update wird ein ChatLog erstellt - -**Alle Aufrufe von `progressLogStart()`, `progressLogUpdate()`, `progressLogFinish()`:** - -#### 3.1. `gateway/modules/workflows/processing/workflowProcessor.py` - -**Task Plan Generation:** -- **Zeile 51**: `progressLogStart()` - Start Task Plan Generation - ```python - self.services.chat.progressLogStart( - operationId, - "Workflow Planning", - "Task Plan Generation", - f"Mode: {workflow.workflowMode.value}" - ) - ``` -- **Zeile 68**: `progressLogUpdate(0.3)` - "Analyzing input" -- **Zeile 74**: `progressLogUpdate(0.8)` - "Creating plan" -- **Zeile 80**: `progressLogFinish(True)` - Erfolg -- **Zeile 86**: `progressLogFinish(False)` - Fehler - -**Task Execution:** -- **Zeile 104**: `progressLogStart()` - Start Task Execution - ```python - self.services.chat.progressLogStart( - operationId, - "Workflow Execution", - "Task Execution", - f"Task {taskIndex}" - ) - ``` -- **Zeile 117**: `progressLogUpdate(0.2)` - "Executing" -- **Zeile 123**: `progressLogFinish(True)` - Erfolg -- **Zeile 129**: `progressLogFinish(False)` - Fehler - -#### 3.2. `gateway/modules/services/serviceAi/mainServiceAi.py` - -**AI Call with Looping (_callAiWithLooping):** -- **Zeile 198**: `progressLogUpdate(0.5)` - "Starting AI call iteration {iteration}" (erste Iteration) -- **Zeile 202**: `progressLogUpdate(baseProgress)` - "Continuing generation (iteration {iteration})" (weitere Iterationen) -- **Zeile 220**: `progressLogUpdate(0.51)` - "Calling AI model" (erste Iteration) -- **Zeile 239**: `progressLogUpdate(0.6)` - "AI response received (iteration {iteration})" (erste Iteration) -- **Zeile 242**: `progressLogUpdate(progress)` - "Processing response (iteration {iteration})" (weitere Iterationen) -- **Zeile 284**: `progressLogUpdate(0.65 + ...)` - "Extracted {len} sections (iteration {iteration})" -- **Zeile 304**: `progressLogUpdate(0.95)` - "Generation complete ({iteration} iterations, {len} sections)" - -**AI Content Processing (callAiContent):** -- **Zeile 568**: `progressLogStart()` - Start AI Content Processing - ```python - self.services.chat.progressLogStart( - aiOperationId, - "AI content processing", - "Content Processing", - f"Format: {outputFormat or 'text'}" - ) - ``` -- **Zeile 593**: `progressLogUpdate(0.1)` - "Analyzing prompt parameters" -- **Zeile 613**: `progressLogUpdate(0.4)` - "Calling AI for image generation" (IMAGE_GENERATE) -- **Zeile 642**: `progressLogUpdate(0.9)` - "Image generated" -- **Zeile 643**: `progressLogFinish(True)` - Erfolg (IMAGE_GENERATE) -- **Zeile 653**: `progressLogFinish(False)` - Fehler (IMAGE_GENERATE) -- **Zeile 658**: `progressLogUpdate(0.4)` - "Calling AI for {opType.name}" (WEB_SEARCH/WEB_CRAWL) -- **Zeile 679**: `progressLogUpdate(0.9)` - "{opType.name} completed" -- **Zeile 680**: `progressLogFinish(True)` - Erfolg (WEB_SEARCH/WEB_CRAWL) -- **Zeile 689**: `progressLogFinish(False)` - Fehler (WEB_SEARCH/WEB_CRAWL) -- **Zeile 705**: `progressLogUpdate(0.3)` - "Building generation prompt" (Document Generation) -- **Zeile 719**: `progressLogUpdate(0.4)` - "Calling AI for content generation" (Document Generation) -- **Zeile 729**: `progressLogUpdate(0.7)` - "Parsing generated JSON" (Document Generation) -- **Zeile 736**: `progressLogFinish(False)` - Fehler JSON Parsing -- **Zeile 758**: `progressLogUpdate(0.8)` - "Rendering to {outputFormat} format" (Document Generation) -- **Zeile 796**: `progressLogFinish(True)` - Erfolg (Document Generation) -- **Zeile 806**: `progressLogFinish(False)` - Fehler Rendering -- **Zeile 810**: `progressLogUpdate(0.5)` - "Processing text call" (Text Processing) -- **Zeile 830**: `progressLogFinish(True)` - Erfolg (Text Processing) -- **Zeile 839**: `progressLogFinish(False)` - Fehler (Text Processing) - -#### 3.3. `gateway/modules/services/serviceExtraction/mainServiceExtraction.py` - -**Process Documents Per Chunk (processDocumentsPerChunk):** -- **Zeile 424**: `progressLogStart()` - Start Document Processing - ```python - self.services.chat.progressLogStart( - operationId, - "AI Text Extract", - "Document Processing", - f"Processing {len(documents)} documents" - ) - ``` -- **Zeile 451**: `progressLogUpdate(0.1)` - "Extracting content from {len} documents" -- **Zeile 456**: `progressLogFinish(False)` - Fehler bei Extraction -- **Zeile 461**: `progressLogUpdate(0.3)` - "Processing {len} extracted content parts" -- **Zeile 466**: `progressLogUpdate(0.9)` - "Merging {len} part results" -- **Zeile 473**: `progressLogFinish(True)` - Erfolg -- **Zeile 479**: `progressLogFinish(False)` - Fehler -- **Zeile 538**: `progressLogUpdate(progress)` - "Processing part {processedCount}/{totalParts}" (in Chunking Callback) - -#### 3.4. `gateway/modules/workflows/methods/methodAi.py` - -**AI Process (process):** -- **Zeile 52**: `progressLogStart()` - Start AI Processing - ```python - self.services.chat.progressLogStart( - operationId, - "Generate", - "AI Processing", - f"Format: {parameters.get('resultType', 'txt')}" - ) - ``` -- **Zeile 63**: `progressLogUpdate(0.2)` - "Preparing parameters" -- **Zeile 111**: `progressLogUpdate(0.3)` - "Extracting content from documents" -- **Zeile 145**: `progressLogUpdate(0.4)` - "Preparing AI call" -- **Zeile 155**: `progressLogUpdate(0.6)` - "Calling AI" -- **Zeile 166**: `progressLogUpdate(0.8)` - "Processing result" -- **Zeile 211**: `progressLogFinish(True)` - Erfolg -- **Zeile 220**: `progressLogFinish(False)` - Fehler - -**Extract Content (extractContent):** -- **Zeile 251**: `progressLogStart()` - Start Content Extraction - ```python - self.services.chat.progressLogStart( - operationId, - "Extracting content from documents", - "Content Extraction", - f"Documents: {len(parameters.documentList.references)}" - ) - ``` -- **Zeile 259**: `progressLogUpdate(0.2)` - "Loading documents" -- **Zeile 263**: `progressLogFinish(False)` - Fehler (keine Dokumente) -- **Zeile 269**: `progressLogUpdate(0.3)` - "Preparing extraction options" -- **Zeile 286**: `progressLogUpdate(0.5)` - "Extracting content from {len} documents" -- **Zeile 290**: `progressLogUpdate(0.8)` - "Building result documents" -- **Zeile 301**: `progressLogFinish(True)` - Erfolg -- **Zeile 310**: `progressLogFinish(False)` - Fehler - -**Generate Document (generateDocument):** -- **Zeile 341**: `progressLogStart()` - Start Document Generation - ```python - self.services.chat.progressLogStart(...) - ``` -- **Zeile 359**: `progressLogFinish(True)` - Erfolg -- **Zeile 401**: `progressLogFinish(False)` - Fehler - -#### 3.5. `gateway/modules/services/serviceWeb/mainServiceWeb.py` - -**Research (research) - nur Updates, kein Start/Finish:** -- **Zeile 50**: `progressLogUpdate(0.1)` - "Analyzing research intent" -- **Zeile 75**: `progressLogUpdate(0.3)` - "Searching for URLs" -- **Zeile 87**: `progressLogUpdate(0.5)` - "Found {len} total URLs" -- **Zeile 102**: `progressLogUpdate(0.6)` - "Crawling {len} URLs" -- **Zeile 110**: `progressLogUpdate(0.9)` - "Consolidating results" - -**Hinweis**: `serviceWeb` verwendet nur `progressLogUpdate()`, aber kein `progressLogStart()` oder `progressLogFinish()`. Dies könnte ein Bug sein oder die Operation wird von einem anderen Service gestartet/beendet. - -### 4. `gateway/modules/workflows/workflowManager.py` - -**Workflow-Management - 11 ChatLog-Erzeugungen:** - -- **Zeile 58**: Workflow gestoppt für neuen Prompt - ```python - self.services.chat.storeLog(workflow, { - "message": "Workflow stopped for new prompt", - "type": "info", - "status": "stopped", - "progress": 1.0 - }) - ``` - -- **Zeile 79**: Workflow fortgesetzt - ```python - self.services.chat.storeLog(workflow, { - "message": f"Workflow resumed (round {workflow.currentRound}) with mode: {workflowMode}", - "type": "info", - "status": "running", - "progress": 0 - }) - ``` - -- **Zeile 140**: Workflow gestoppt (workflowStop) - ```python - self.services.chat.storeLog(workflow, { - "message": "Workflow stopped", - "type": "warning", - "status": "stopped", - "progress": 1.0 - }) - ``` - -- **Zeile 635**: Workflow gestoppt durch Benutzer (in _processWorkflowResults) - ```python - self.services.chat.storeLog(workflow, { - "message": "Workflow stopped by user", - "type": "warning", - "status": "stopped", - "progress": 1.0 - }) - ``` - -- **Zeile 674**: Workflow fehlgeschlagen (in _processWorkflowResults) - ```python - self.services.chat.storeLog(workflow, { - "message": "Workflow failed: Unknown error", - "type": "error", - "status": "failed", - "progress": 1.0 - }) - ``` - -- **Zeile 762**: Workflow abgeschlossen (in _sendLastMessage) - ```python - self.services.chat.storeLog(workflow, { - "message": "Workflow completed", - "type": "success", - "status": "completed", - "progress": 1.0 - }) - ``` - -- **Zeile 837**: Workflow gestoppt (in _handleWorkflowStop) - ```python - self.services.chat.storeLog(workflow, { - "message": "Workflow stopped by user", - "type": "warning", - "status": "stopped", - "progress": 1.0 - }) - ``` - -- **Zeile 880**: Workflow-Fehler (in _handleWorkflowError) - ```python - self.services.chat.storeLog(workflow, { - "message": f"Workflow failed: {str(error)}", - "type": "error", - "status": "failed", - "progress": 1.0 - }) - ``` - -- **Zeile 931**: Binäre Datei Info (in _processFileIds - Neutralization) - ```python - self.services.chat.storeLog(workflow, { - "message": infoMsg, - "type": "info", - "status": "running", - "progress": 50 - }) - ``` - -- **Zeile 964**: Neutralization Fehler (in _processFileIds) - ```python - self.services.chat.storeLog(workflow, { - "message": errorMsg, - "type": "error", - "status": "error", - "progress": -1 - }) - ``` - -- **Zeile 974**: Neutralization Fehler (Exception) (in _processFileIds) - ```python - self.services.chat.storeLog(workflow, { - "message": errorMsg, - "type": "error", - "status": "error", - "progress": -1 - }) - ``` - -- **Zeile 999**: Datei-Verarbeitungsfehler (in _processFileIds) - ```python - self.services.chat.storeLog(workflow, { - "message": errorMsg, - "type": "error", - "status": "error", - "progress": -1 - }) - ``` - -### 5. `gateway/modules/workflows/processing/core/actionExecutor.py` - -**Action Execution:** -- **Zeile 148**: Action-Fehler - ```python - self.services.chat.storeLog(workflow, { - "message": f"❌ **Task {taskNum}**❌ **Action {actionNum}** failed: {result.error}", - "type": "error", - "progress": 1.0 - }) - ``` - -### 6. `gateway/modules/services/serviceExtraction/mainServiceExtraction.py` - -**Extraction Service:** -- **Zeile 553**: Chunking Progress Callback - ```python - self.services.chat.storeLog(workflow, logData) - ``` - - Wird während AI-Chunking aufgerufen - - **Häufige Erzeugung**: Bei jedem Chunking-Progress-Update - -## Zusammenfassung nach Kategorie - -### ✅ Kapitel 6, 5, 4: Korrekt dokumentiert - -Diese Kapitel sind vollständig und korrekt dokumentiert: -- **Kapitel 6**: Extraction Service Chunking (`mainServiceExtraction.py` Zeile 553) -- **Kapitel 5**: Action Executor (`actionExecutor.py` Zeile 148) -- **Kapitel 4**: Workflow Manager (`workflowManager.py` - 11 Stellen) - -### 📋 Kapitel 3: ProgressLogger Aufrufe - Vollständige Liste - -**Alle Aufrufe von `progressLogStart()`, `progressLogUpdate()`, `progressLogFinish()`:** - -**Gesamtanzahl Aufrufe:** -- **startOperation**: 7 Aufrufe - - workflowProcessor.py: 2x (Task Plan, Task Execution) - - mainServiceAi.py: 1x (AI Content Processing) - - mainServiceExtraction.py: 1x (Document Processing) - - methodAi.py: 3x (AI Process, Extract Content, Generate Document) - -- **updateOperation**: ~35+ Aufrufe - - workflowProcessor.py: 2x - - mainServiceAi.py: ~15x (AI Looping + Content Processing) - - mainServiceExtraction.py: 4x (+ Chunking Callback) - - methodAi.py: 8x - - mainServiceWeb.py: 5x (ohne Start/Finish!) - -- **finishOperation**: ~15 Aufrufe - - workflowProcessor.py: 4x (2x Success, 2x Failure) - - mainServiceAi.py: ~6x (verschiedene Operationen) - - mainServiceExtraction.py: 2x - - methodAi.py: 6x - -**Problematische Stellen:** -1. **Zu viele Update-Aufrufe**: Besonders in `_callAiWithLooping()` werden bei jeder Iteration mehrere Updates erstellt -2. **Fehlende Start/Finish**: `mainServiceWeb.py` verwendet nur Updates ohne Start/Finish -3. **Redundante Updates**: Viele Updates mit ähnlichen Progress-Werten (z.B. 0.4, 0.5, 0.6) - -### 📋 Kapitel 2 und 1: Vollständig dokumentiert - -- **Kapitel 2**: `mainServiceChat.storeLog()` - Service-Layer Methode -- **Kapitel 1**: `interfaceDbChatObjects.createLog()` - Datenbank-Methode - -## Empfehlungen zur Reduzierung - -1. **ProgressLogger**: - - Nur bei Start/Finish loggen, nicht bei jedem Update - - Oder: Batch-Updates alle N Sekunden - -2. **Chunking Progress**: - - Nur bei wichtigen Meilensteinen (z.B. 25%, 50%, 75%, 100%) - - Nicht bei jedem einzelnen Chunk - -3. **Workflow Status**: - - Behalten, aber Duplikate entfernen (z.B. mehrere "stopped" Logs) - -4. **Fehler-Logging**: - - Nur kritische Fehler loggen, nicht alle Warnungen/Infos - -## Statistik - -### ChatLog-Erzeugungen nach Quelle: - -1. **ProgressLogger (Kapitel 3)**: ~57+ Aufrufe - - 7x `startOperation()` → erzeugt ChatLog - - ~35+ `updateOperation()` → erzeugt ChatLog bei jedem Update - - ~15x `finishOperation()` → erzeugt ChatLog - - **Häufigste Quelle** - kann bei langen Operationen sehr viele Logs erzeugen - -2. **Extraction Service Chunking (Kapitel 6)**: Variable Häufigkeit - - Wird bei jedem Chunking-Progress-Update aufgerufen - - **Zweithäufigste Quelle** - abhängig von Anzahl der Chunks - -3. **Workflow Manager (Kapitel 4)**: 11 feste Stellen - - Workflow Status-Änderungen (einmalig pro Status-Übergang) - - Fehler-Logging bei Datei-Verarbeitung - -4. **Action Executor (Kapitel 5)**: 1 Stelle - - Action-Fehler-Logging - -### Gesamtübersicht: - -- **Direkte `storeLog()` Aufrufe**: 12 Stellen (Kapitel 4, 5, 6) -- **ProgressLogger Aufrufe**: ~57+ Stellen (Kapitel 3) -- **Gesamtanzahl Code-Stellen**: ~70+ Stellen -- **Potenzial für Reduzierung**: - - ProgressLogger: ~35+ Update-Aufrufe könnten reduziert werden - - Chunking: Variable, abhängig von Chunk-Anzahl -