146 lines
5 KiB
Markdown
146 lines
5 KiB
Markdown
# Progress Logging Implementation
|
|
|
|
## Overview
|
|
This implementation adds comprehensive progress tracking to the workflow system using standardized ChatLog messages with progress indicators. The system provides real-time progress updates for long-running operations.
|
|
|
|
## Key Components
|
|
|
|
### 1. ProgressLogger Utility (`gateway/modules/shared/progressLogger.py`)
|
|
- **Purpose**: Centralized progress management for workflow operations
|
|
- **Features**:
|
|
- Start/update/complete operations
|
|
- Automatic workflowId extraction from workflow object
|
|
- Standardized progress message format
|
|
- Error handling and cleanup
|
|
|
|
### 2. Enhanced WorkflowService (`gateway/modules/services/serviceWorkflow/mainServiceWorkflow.py`)
|
|
- **Added**: `createProgressLogger(workflow)` method
|
|
- **Purpose**: Factory method for creating ProgressLogger instances
|
|
|
|
### 3. Document Generation Integration (`gateway/modules/services/serviceAi/subDocumentGeneration.py`)
|
|
- **Enhanced Methods**:
|
|
- `_callAiWithSingleFileGeneration()` - Single-file document generation with progress
|
|
- `_callAiWithMultiFileGeneration()` - Multi-file document generation with progress
|
|
- **Progress Points**:
|
|
- Start: When generation begins
|
|
- 10%: Extraction prompt generated
|
|
- 20%: Document processing started
|
|
- 90%: Generation completed
|
|
- 100%: Operation completed
|
|
|
|
### 4. Workflow Processor Integration (`gateway/modules/workflows/processing/workflowProcessor.py`)
|
|
- **Enhanced Methods**:
|
|
- `generateTaskPlan()` - Task plan generation with progress
|
|
- `executeTask()` - Task execution with progress
|
|
- **Progress Points**:
|
|
- Start: When operation begins
|
|
- 30%: User input analyzed (task planning)
|
|
- 80%: Task plan message created
|
|
- 100%: Operation completed
|
|
|
|
### 5. Frontend Enhancements
|
|
- **WorkflowCoordination** (`frontend_agents/public/js/modules/workflowCoordination.js`):
|
|
- Added `handleProgressUpdate()` function for efficient progress updates
|
|
- Updates existing progress bars instead of creating new log entries
|
|
- **UI Renderer** (`frontend_agents/public/js/modules/workflowUiRenderer.js`):
|
|
- Enhanced progress bar rendering with status information
|
|
- Shows progress percentage and status text
|
|
|
|
## Standardized Progress Format
|
|
|
|
### Message Format
|
|
```
|
|
message = "Service Action"
|
|
status = "Context Details"
|
|
progress = 0.XX (0.0 to 1.0)
|
|
```
|
|
|
|
### Examples
|
|
- `message = "Service Extract", status="Document Filename.docx part 12", progress = 0.23`
|
|
- `message = "Service AI", status="Processing chunk 3 of 8", progress = 0.54`
|
|
- `message = "Service Generate", status="Creating document section 2", progress = 0.89`
|
|
|
|
## Usage Examples
|
|
|
|
### Basic Usage
|
|
```python
|
|
# Create progress logger
|
|
progressLogger = workflowService.createProgressLogger(workflow)
|
|
operationId = f"operation_{workflow.id}_{int(time.time())}"
|
|
|
|
# Start operation
|
|
progressLogger.startOperation(
|
|
operationId,
|
|
"ServiceName",
|
|
"ActionName",
|
|
"Context information"
|
|
)
|
|
|
|
# Update progress
|
|
progressLogger.updateProgress(operationId, 0.5, "Halfway done")
|
|
|
|
# Complete operation
|
|
progressLogger.completeOperation(operationId, True)
|
|
```
|
|
|
|
### Error Handling
|
|
```python
|
|
try:
|
|
# ... operation code ...
|
|
progressLogger.completeOperation(operationId, True)
|
|
except Exception as e:
|
|
progressLogger.completeOperation(operationId, False)
|
|
raise
|
|
```
|
|
|
|
## Frontend Integration
|
|
|
|
### Progress Update Handling
|
|
```javascript
|
|
// Import the function
|
|
import { handleProgressUpdate } from './workflowCoordination.js';
|
|
|
|
// Handle progress updates
|
|
handleProgressUpdate(logEntry);
|
|
```
|
|
|
|
### UI Rendering
|
|
The UI automatically renders progress bars when `progress` field is present in ChatLog entries:
|
|
- Progress bar with percentage
|
|
- Status text display
|
|
- Real-time updates
|
|
|
|
## Benefits
|
|
|
|
1. **Standardized Format**: Consistent progress reporting across all services
|
|
2. **Real-time Updates**: Users see progress as operations happen
|
|
3. **Efficient UI**: Progress bars update in place rather than creating new log entries
|
|
4. **Error Handling**: Failed operations show clear progress indication
|
|
5. **Concurrent Operations**: Multiple operations can show progress simultaneously
|
|
6. **CamelCase Naming**: Follows established coding conventions
|
|
|
|
## Implementation Status
|
|
|
|
✅ **Completed**:
|
|
- ProgressLogger utility class
|
|
- WorkflowService integration
|
|
- Document generation progress tracking
|
|
- Workflow processor progress tracking
|
|
- Frontend progress update handling
|
|
- UI progress bar enhancements
|
|
|
|
## Next Steps
|
|
|
|
1. **Test Integration**: Test the progress logging with actual workflow operations
|
|
2. **Performance Optimization**: Monitor performance impact of frequent progress updates
|
|
3. **Additional Services**: Add progress logging to other long-running services
|
|
4. **UI Polish**: Enhance progress bar styling and animations
|
|
|
|
## Files Modified
|
|
|
|
- `gateway/modules/shared/progressLogger.py` (new)
|
|
- `gateway/modules/services/serviceWorkflow/mainServiceWorkflow.py`
|
|
- `gateway/modules/services/serviceAi/subDocumentGeneration.py`
|
|
- `gateway/modules/workflows/processing/workflowProcessor.py`
|
|
- `frontend_agents/public/js/modules/workflowCoordination.js`
|
|
- `frontend_agents/public/js/modules/workflowUiRenderer.js`
|