PowerOn/wiki
3
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
wiki/z-archive/appdoc/ai_plan_implementation.md

30 KiB
Raw Permalink Blame History

AI Plan Implementation Guide

Overview

This document provides a structured implementation plan to adapt the current codebase to the enhanced architecture described in ai_plan_architecture.md.

Goal: Implement all architectural improvements while maintaining backward compatibility and clear tracking.

Reference Model: Implementation Phases

Phase 1: Foundation Models (Pydantic)
  ├─> New models: ActionDefinition, AiResponse, DocumentReference types
  └─> Enhanced models: ChatWorkflow, TaskContext
  ↓
Phase 2: State Management (ChatWorkflow)
  ├─> Add execution state fields
  ├─> Add helper methods (getRoundIndex, incrementRound, etc.)
  └─> Update all call sites
  ↓
Phase 3: TaskContext Enhancement
  ├─> Add Stage 2 fields
  ├─> Add updateFromSelection method
  └─> Remove SimpleNamespace workarounds
  ↓
Phase 4: Document References (Typed Models)
  ├─> Create DocumentReference classes
  ├─> Replace string references with typed models
  └─> Update document lookup logic
  ↓
Phase 5: JSON Parsing (Structured)
  ├─> Create parseJsonWithModel utility
  ├─> Replace manual JSON parsing
  └─> Update all AI response parsing
  ↓
Phase 6: AI Service Consolidation
  ├─> Consolidate callAiDocuments + callAiText → callAiContent
  ├─> Update all call sites
  └─> Ensure contentParts-only approach
  ↓
Phase 7: Document Extraction Separation
  ├─> Add extractContent action to methodAi
  ├─> Remove extraction from AI calls
  └─> Update action execution flow
  ↓
Phase 8: Remove Actionplan Mode
  ├─> Remove modeActionplan.py
  ├─> Remove promptGenerationActionsActionplan.py
  ├─> Remove WorkflowModeEnum.WORKFLOW_ACTIONPLAN
  └─> Update workflowProcessor to remove ActionplanMode references
  ↓
Phase 9: Testing & Validation
  ├─> Unit tests for new models
  ├─> Integration tests for workflow
  └─> End-to-end validation

Model Segregation Rules

Clear Separation of Concerns

datamodelChat.py (Chat/Workflow State Models):

  • ChatWorkflow (enhanced with execution state)
  • TaskContext (enhanced with Stage 2 fields)
  • ChatMessage (existing)
  • ChatDocument (existing)

datamodelWorkflow.py (Workflow Execution Models):

  • ActionDefinition (action selection and parameters)
  • AiResponse (unified AI response)
  • AiResponseMetadata (response metadata)
  • DocumentData (single document in response)
  • RequestContext (workflow-level request)
  • UnderstandingResult (workflow-level understanding)
  • TaskDefinition (workflow-level task)
  • TaskResult (workflow-level result)

datamodelDocref.py (Document Reference Models):

  • DocumentReference (base class)
  • DocumentListReference (docList: references)
  • DocumentItemReference (docItem: references)
  • DocumentReferenceList (list wrapper)

jsonUtils.py (JSON Utilities):

  • All JSON parsing utilities (including parseJsonWithModel)
  • No separate jsonModelUtils.py

Phase 1: Foundation Models (Pydantic)

1.1 Create New Pydantic Models

File: gateway/modules/datamodels/datamodelWorkflow.py (NEW)

Models to Create (Workflow execution models):

  • ActionDefinition (replaces ActionSelection + ActionParameters)
  • AiResponse (unified response model)
  • AiResponseMetadata (response metadata)
  • DocumentData (single document in response)
  • RequestContext (workflow-level)
  • UnderstandingResult (workflow-level)
  • TaskDefinition (workflow-level)
  • TaskResult (workflow-level)

Segregation Rule: datamodelWorkflow.py contains models related to workflow execution (actions, AI responses, task definitions).

Reference: ai_plan_architecture.md lines 216-335

Implementation Checklist:

  • Create datamodelWorkflow.py
  • Implement ActionDefinition with needsStage2() method
  • Implement AiResponse with toJson() method
  • Implement AiResponseMetadata with fromDict() classmethod
  • Implement workflow-level models (RequestContext, UnderstandingResult, etc.)
  • Add unit tests for model validation

1.2 Create Document Reference Models

File: gateway/modules/datamodels/datamodelDocref.py (NEW)

Models to Create (Document reference models):

  • DocumentReference (base class)
  • DocumentListReference (docList: references)
  • DocumentItemReference (docItem: references)
  • DocumentReferenceList (list wrapper with conversion methods)

Segregation Rule: datamodelDocref.py contains models related to document references (how documents are referenced in workflows).

Reference: ai_plan_architecture.md lines 174-212

Implementation Checklist:

  • Create datamodelDocref.py
  • Implement DocumentReference base class
  • Implement DocumentListReference with to_string() method
  • Implement DocumentItemReference with to_string() method
  • Implement DocumentReferenceList with to_string_list() and from_string_list() methods
  • Add parsing logic for string → typed conversion
  • Add unit tests for reference parsing

Phase 2: State Management (ChatWorkflow)

2.1 Enhance ChatWorkflow Model

File: gateway/modules/datamodels/datamodelChat.py

Segregation Rule: datamodelChat.py contains models related to chat/workflow state (ChatWorkflow, TaskContext, ChatMessage, ChatDocument).

Changes:

class ChatWorkflow(BaseModel):
    # ... existing fields ...
    
    # NEW: Execution state
    currentRound: int = 0
    currentTask: int = 0
    currentAction: int = 0
    
    # NEW: Helper methods
    def getRoundIndex(self) -> int:
        return self.currentRound
    
    def getTaskIndex(self) -> int:
        return self.currentTask
    
    def getActionIndex(self) -> int:
        return self.currentAction
    
    def incrementRound(self):
        self.currentRound += 1
        self.currentTask = 0
        self.currentAction = 0
    
    def incrementTask(self):
        self.currentTask += 1
        self.currentAction = 0
    
    def incrementAction(self):
        self.currentAction += 1

Reference: ai_plan_architecture.md lines 117-152

Implementation Checklist:

  • Add execution state fields to ChatWorkflow
  • Add helper methods (getRoundIndex, getTaskIndex, getActionIndex)
  • Add increment methods (incrementRound, incrementTask, incrementAction)
  • Update database migration if needed (add columns)
  • Add unit tests for state management

2.2 Update Call Sites (Remove Separate Index Parameters)

Files to Update:

  • gateway/modules/workflows/processing/workflowProcessor.py
  • gateway/modules/workflows/processing/modes/modeDynamic.py
  • gateway/modules/workflows/processing/actionExecutor.py

Changes:

# BEFORE
async def executeTask(taskStep, workflow, context, taskIndex: int, actionIndex: int):
    # Use taskIndex, actionIndex parameters

# AFTER
async def executeTask(taskStep, workflow, context):
    # Use workflow.getTaskIndex(), workflow.getActionIndex()
    taskIndex = workflow.getTaskIndex()
    actionIndex = workflow.getActionIndex()

Reference: ai_plan_architecture.md lines 435-568

Implementation Checklist:

  • Remove taskIndex, actionIndex parameters from workflowProcessor.executeTask()
  • Remove taskIndex, actionIndex parameters from modeDynamic.executeTask()
  • Remove taskIndex, actionIndex parameters from actionExecutor.executeSingleAction()
  • Update all call sites to use workflow.getTaskIndex(), workflow.getActionIndex()
  • Update increment calls to use workflow.incrementAction(), workflow.incrementTask()
  • Test workflow execution with state management

Phase 3: TaskContext Enhancement

3.1 Enhance TaskContext Model

File: gateway/modules/datamodels/datamodelChat.py

Segregation Rule: datamodelChat.py contains TaskContext (part of chat workflow context).

Changes:

class TaskContext(BaseModel):
    # ... existing fields ...
    
    # NEW: Stage 2 context fields
    actionObjective: Optional[str] = None
    parametersContext: Optional[str] = None
    learnings: List[str] = Field(default_factory=list)
    stage1Selection: Optional[Dict[str, Any]] = None
    
    def updateFromSelection(self, selection: ActionDefinition):
        """Update context from Stage 1 selection"""
        self.actionObjective = selection.actionObjective
        self.parametersContext = selection.parametersContext
        self.learnings = selection.learnings
        self.stage1Selection = selection.model_dump()

Reference: ai_plan_architecture.md lines 154-172

Implementation Checklist:

  • Add Stage 2 fields to TaskContext
  • Add updateFromSelection() method
  • Add unit tests for context updates

3.2 Remove SimpleNamespace Workarounds

Files to Update:

  • gateway/modules/workflows/processing/modes/modeDynamic.py

Changes:

# BEFORE
from types import SimpleNamespace
stage2Context = SimpleNamespace(
    actionObjective=selection.actionObjective,
    parametersContext=selection.parametersContext,
    learnings=selection.learnings
)

# AFTER
context.updateFromSelection(selection)
# Use context.actionObjective, context.parametersContext directly

Reference: ai_plan_architecture.md lines 529-530

Implementation Checklist:

  • Find all SimpleNamespace usages in modeDynamic.py
  • Replace with context.updateFromSelection(selection)
  • Update all references to use context.actionObjective, etc.
  • Remove SimpleNamespace imports
  • Test Stage 2 parameter generation

Phase 4: Document References (Typed Models)

4.1 Update Document Reference Usage

Files to Update:

  • gateway/modules/workflows/processing/modes/modeDynamic.py (_planSelect)
  • gateway/modules/services/serviceChat/mainServiceChat.py (getChatDocumentsFromDocumentList)

Changes:

# BEFORE
documentList: List[str] = ["docList:msg_123:label"]

# AFTER
from modules.datamodels.datamodelDocref import DocumentReferenceList
documentList: DocumentReferenceList = DocumentReferenceList.from_string_list(["docList:msg_123:label"])

Reference: ai_plan_architecture.md lines 500-507, 858-877

Implementation Checklist:

  • Update _planSelect() to convert string references to DocumentReferenceList
  • Update ActionDefinition.documentList to use DocumentReferenceList
  • Update getChatDocumentsFromDocumentList() to accept DocumentReferenceList
  • Update document lookup logic to use typed references
  • Test document reference parsing and lookup

Phase 5: JSON Parsing (Structured)

5.1 Create parseJsonWithModel Utility

File: gateway/modules/utils/jsonUtils.py (ADD to existing file)

Note: All JSON utilities are consolidated in jsonUtils.py (no separate jsonModelUtils.py).

Function:

def parseJsonWithModel(jsonString: str, modelClass: Type[BaseModel]) -> BaseModel:
    """
    Parse JSON string using Pydantic model with error handling.
    
    Uses existing jsonUtils methods:
    - tryParseJson() - Safe parsing
    - repairBrokenJson() - Repairs broken JSON
    - extractJsonString() - Extracts JSON from text
    
    Returns:
        Parsed Pydantic model instance
    
    Raises:
        ValueError: If JSON cannot be parsed or validated
    """

Reference: ai_plan_architecture.md lines 495-500, 538

Implementation Checklist:

  • Add parseJsonWithModel() function to existing jsonUtils.py
  • Integrate with existing jsonUtils methods (tryParseJson, repairBrokenJson, extractJsonString)
  • Add error handling and validation
  • Add unit tests for parsing edge cases

5.2 Replace Manual JSON Parsing

Files to Update:

  • gateway/modules/workflows/processing/modes/modeDynamic.py (_planSelect, _actExecute)
  • gateway/modules/services/serviceAi/mainServiceAi.py (AI response parsing)

Changes:

# BEFORE
responseJson = json.loads(response.content)
# Manual field access
action = responseJson.get("action")
parameters = responseJson.get("parameters")

# AFTER
from modules.utils.jsonUtils import parseJsonWithModel
from modules.datamodels.datamodelWorkflow import ActionDefinition

selection = parseJsonWithModel(response.content, ActionDefinition)
# Type-safe access
action = selection.action
parameters = selection.parameters

Reference: ai_plan_architecture.md lines 495-500, 538, 677

Implementation Checklist:

  • Replace manual JSON parsing in _planSelect() with parseJsonWithModel(..., ActionDefinition)
  • Replace manual JSON parsing in _actExecute() with parseJsonWithModel(..., ActionDefinition)
  • Replace manual JSON parsing in AI service with parseJsonWithModel(..., UnifiedJsonDocument)
  • Remove manual find(), rfind(), string manipulation for JSON
  • Test JSON parsing with various response formats

Phase 6: AI Service Consolidation

6.1 Consolidate AI Service Methods

File: gateway/modules/services/serviceAi/mainServiceAi.py

Changes:

# BEFORE
async def callAiDocuments(...) -> Dict[str, Any]:
    # Document generation logic

async def callAiText(...) -> str:
    # Text processing logic

# AFTER
async def callAiContent(
    prompt: str,
    contentParts: Optional[List[ContentPart]] = None,
    options: AiCallOptions,
    outputFormat: Optional[str] = None,
    title: Optional[str] = None
) -> AiResponse:
    # Unified logic for both text and documents
    # Returns AiResponse model

Reference: ai_plan_architecture.md lines 629-691

Implementation Checklist:

  • Create callAiContent() method (unified)
  • Move document generation logic from callAiDocuments() to callAiContent()
  • Move text processing logic from callAiText() to callAiContent()
  • Keep callAiPlanning() separate (unchanged)
  • Update return type to AiResponse
  • Mark callAiDocuments() and callAiText() as deprecated (or remove)

6.2 Update AI Service Call Sites

Files to Update:

  • gateway/modules/workflows/methods/methodAi.py (process action)
  • All other methods that call AI service

Changes:

# BEFORE
result = await self.services.ai.callAiDocuments(prompt, documents, options)
# or
result = await self.services.ai.callAiText(prompt, contentParts, options)

# AFTER
result = await self.services.ai.callAiContent(
    prompt=prompt,
    contentParts=contentParts,  # Already extracted
    options=options,
    outputFormat=outputFormat  # If document generation
)
# Returns AiResponse

Reference: ai_plan_architecture.md lines 615-625

Implementation Checklist:

  • Update methodAi.process() to use callAiContent()
  • Update all other method actions that call AI service
  • Ensure contentParts are extracted before calling (see Phase 7)
  • Update result handling to use AiResponse model
  • Test AI calls with both text and document generation

Phase 7: Document Extraction Separation

7.1 Create ExtractContent Action

File: gateway/modules/workflows/methods/methodAi.py (ADD to existing)

Action:

@action
async def extractContent(self, parameters: ExtractContentParameters) -> ActionResult:
    """
    Extract content from documents (separate from AI calls).
    
    Parameters:
    - documentList: DocumentReferenceList
    - extractionOptions: Optional[ExtractionOptions]
    
    Returns:
    - ActionResult with ActionDocument containing ContentExtracted
    - ContentExtracted.parts contains List[ContentPart]
    """
    # Call extractionService.extractContent()
    # Return ActionResult with ContentExtracted

Reference: ai_plan_architecture.md lines 721-786

Implementation Checklist:

  • Create ExtractContentParameters model (in same module as action, following action registry pattern)
  • Add extractContent action to methodAi class
  • Implement extraction logic (calls extractionService.extractContent())
  • Return ActionResult with ContentExtracted objects
  • Add unit tests for extraction action

7.2 Remove Extraction from AI Calls

File: gateway/modules/services/serviceAi/mainServiceAi.py

Changes:

# BEFORE
async def callAiContent(prompt, documents, options):
    # Extract documents here
    extractedContent = extractionService.extractContent(documents)
    # Process with AI

# AFTER
async def callAiContent(prompt, contentParts, options):
    # contentParts already extracted (required parameter)
    # Process with AI directly
    # NO extraction logic here

Reference: ai_plan_architecture.md lines 686-690

Implementation Checklist:

  • Remove extraction logic from callAiContent()
  • Make contentParts a required parameter (if documents need processing)
  • Update method signature to only accept contentParts (not documents)
  • Update all call sites to extract before calling
  • Test that extraction is done separately before AI calls

7.3 Update Action Execution Flow

Files to Update:

  • gateway/modules/workflows/processing/modes/modeDynamic.py (_actExecute)
  • gateway/modules/workflows/methods/methodAi.py (process)

Changes:

# BEFORE
# In methodAi.process()
documents = parameters.get("documentList", [])
# Extract and process in one step
result = await self.services.ai.callAiContent(prompt, documents, options)

# AFTER
# In _actExecute() or before calling methodAi.process()
if selection.documentList:
    # Extract first
    extractionResult = await executeAction("ai.extractContent", {
        "documentList": selection.documentList,
        "extractionOptions": task.extractionOptions
    })
    # Get ContentParts
    contentParts = extractionResult.documents[0].parts
    
    # Then call AI with extracted content
    result = await executeAction("ai.process", {
        "aiPrompt": prompt,
        "contentParts": contentParts,  # Already extracted
        "resultType": resultType
    })

Reference: ai_plan_architecture.md lines 622-625, 960-991

Implementation Checklist:

  • Update workflow to call ai.extractContent before ai.process
  • Update methodAi.process() to accept contentParts (not documentList)
  • Update AiProcessParameters model to use contentParts
  • Test extraction → AI call flow
  • Ensure ContentParts are reusable across multiple AI calls

Phase 8: Remove Actionplan Mode

8.1 Remove Actionplan Mode Files

Files to Delete:

  • gateway/modules/workflows/processing/modes/modeActionplan.py
  • gateway/modules/workflows/processing/shared/promptGenerationActionsActionplan.py

Rationale: Actionplan mode is no longer needed. Dynamic mode handles all workflow execution.

Implementation Checklist:

  • Delete modeActionplan.py
  • Delete promptGenerationActionsActionplan.py
  • Verify no other files import these modules

8.2 Remove Actionplan Mode References

File: gateway/modules/workflows/processing/workflowProcessor.py

Changes:

# BEFORE
from modules.workflows.processing.modes.modeActionplan import ActionplanMode

def _createMode(self, workflowMode: WorkflowModeEnum) -> BaseMode:
    if workflowMode == WorkflowModeEnum.WORKFLOW_DYNAMIC:
        return DynamicMode(self.services)
    elif workflowMode == WorkflowModeEnum.WORKFLOW_ACTIONPLAN:
        return ActionplanMode(self.services)
    elif workflowMode == WorkflowModeEnum.WORKFLOW_AUTOMATION:
        return AutomationMode(self.services)

# AFTER
# Remove ActionplanMode import

def _createMode(self, workflowMode: WorkflowModeEnum) -> BaseMode:
    if workflowMode == WorkflowModeEnum.WORKFLOW_DYNAMIC:
        return DynamicMode(self.services)
    elif workflowMode == WorkflowModeEnum.WORKFLOW_AUTOMATION:
        return AutomationMode(self.services)
    # Remove WORKFLOW_ACTIONPLAN case

Implementation Checklist:

  • Remove ActionplanMode import from workflowProcessor.py
  • Remove WORKFLOW_ACTIONPLAN case from _createMode() method
  • Update error handling if needed

8.3 Remove WorkflowModeEnum.WORKFLOW_ACTIONPLAN

File: gateway/modules/datamodels/datamodelChat.py

Changes:

# BEFORE
class WorkflowModeEnum(str, Enum):
    WORKFLOW_DYNAMIC = "Dynamic"
    WORKFLOW_ACTIONPLAN = "Actionplan"
    WORKFLOW_AUTOMATION = "Automation"

# AFTER
class WorkflowModeEnum(str, Enum):
    WORKFLOW_DYNAMIC = "Dynamic"
    WORKFLOW_AUTOMATION = "Automation"
    # Remove WORKFLOW_ACTIONPLAN

Also Remove:

  • Any UI labels/translations for WORKFLOW_ACTIONPLAN
  • Any default values or options that reference WORKFLOW_ACTIONPLAN

Implementation Checklist:

  • Remove WORKFLOW_ACTIONPLAN from WorkflowModeEnum
  • Remove UI labels/translations for Actionplan mode
  • Remove any default values referencing Actionplan mode
  • Update any database migrations if needed (existing workflows with ACTIONPLAN mode)

8.4 Clean Up Related References

Files to Check:

  • gateway/modules/workflows/processing/modes/modeDynamic.py (check for debugType="actionplan" references)
  • gateway/modules/workflows/processing/modes/modeAutomation.py (check for comments referencing ActionplanMode)

Changes:

# In modeDynamic.py
# BEFORE
debugType="actionplan"

# AFTER
debugType="dynamic"  # Or remove if not needed

Implementation Checklist:

  • Update debugType in modeDynamic.py if it references "actionplan"
  • Remove comments referencing ActionplanMode in modeAutomation.py
  • Search codebase for any remaining "actionplan" or "Actionplan" references
  • Update documentation if needed

8.5 Migration Strategy

Database Considerations:

  • Existing workflows with workflowMode = "Actionplan" need to be migrated
  • Options:
    1. Migrate to Dynamic: Convert all ACTIONPLAN workflows to DYNAMIC
    2. Error on Load: Fail gracefully if ACTIONPLAN mode is encountered
    3. Deprecation Period: Keep enum but mark as deprecated, migrate gradually

Recommended Approach: Migrate to Dynamic mode

Implementation Checklist:

  • Create migration script to convert ACTIONPLAN → DYNAMIC workflows
  • Test migration on sample workflows
  • Add validation to prevent creating new ACTIONPLAN workflows
  • Document migration process

Phase 9: Testing & Validation

9.1 Unit Tests

Test Files to Create/Update:

  • tests/unit/datamodels/test_workflow_models.py (NEW - for datamodelWorkflow.py)
  • tests/unit/datamodels/test_docref.py (NEW - for datamodelDocref.py)
  • tests/unit/services/test_ai_service.py (UPDATE)
  • tests/unit/workflows/test_state_management.py (NEW)
  • tests/unit/utils/test_json_utils.py (UPDATE - for parseJsonWithModel)

Test Coverage:

  • ActionDefinition.needsStage2() logic
  • DocumentReferenceList.from_string_list() parsing (in datamodelDocref.py)
  • ChatWorkflow state increment methods
  • TaskContext.updateFromSelection() method
  • parseJsonWithModel() with various JSON formats
  • callAiContent() with text and document generation

Test Files Created:

  • tests/unit/datamodels/test_workflow_models.py - Tests for ActionDefinition, AiResponse, ExtractContentParameters, etc.
  • tests/unit/datamodels/test_docref.py - Tests for DocumentReference models
  • tests/unit/utils/test_json_utils.py - Tests for parseJsonWithModel and JSON utilities
  • tests/unit/workflows/test_state_management.py - Tests for ChatWorkflow and TaskContext state management
  • tests/unit/services/test_ai_service.py - Tests for AI service methods

9.2 Integration Tests

Test Files to Create/Update:

  • tests/integration/workflows/test_workflow_execution.py (UPDATE)
  • tests/integration/workflows/test_action_execution.py (UPDATE)

Test Scenarios:

  • Full workflow execution with state management
  • Stage 1 → Stage 2 parameter generation
  • Document extraction → AI processing flow
  • Document reference lookup across tasks/rounds
  • JSON parsing with broken/incomplete JSON

Test Files Created:

  • tests/integration/workflows/test_workflow_execution.py - Integration tests for workflow execution

9.3 End-to-End Validation

Validation Checklist:

  • Simple request (fast path) works
  • Complex request (full workflow) works
  • Document extraction works separately
  • AI calls work with extracted ContentParts
  • State management (rounds, tasks, actions) works correctly
  • Document references work across tasks/rounds
  • Stage 2 parameter generation works
  • All existing workflows still function

Test Files Created:

  • tests/validation/test_architecture_validation.py - End-to-end validation tests

Implementation Tracking

Progress Tracking Table

Phase Component Status Assigned Notes
1.1 Foundation Models ⬜ Pending - -
1.2 Document References ⬜ Pending - -
2.1 ChatWorkflow Enhancement ⬜ Pending - -
2.2 Update Call Sites ⬜ Pending - -
3.1 TaskContext Enhancement ⬜ Pending - -
3.2 Remove SimpleNamespace ⬜ Pending - -
4.1 Update Document References ⬜ Pending - -
5.1 Create parseJsonWithModel ⬜ Pending - -
5.2 Replace Manual Parsing ⬜ Pending - -
6.1 Consolidate AI Service ⬜ Pending - -
6.2 Update Call Sites ⬜ Pending - -
7.1 Create ExtractContent Action ⬜ Pending - -
7.2 Remove Extraction from AI ⬜ Pending - -
7.3 Update Action Flow ⬜ Pending - -
8.1 Remove Actionplan Mode Files ⬜ Pending - -
8.2 Remove Actionplan References ⬜ Pending - -
8.3 Remove WorkflowModeEnum.ACTIONPLAN ⬜ Pending - -
8.4 Clean Up Related References ⬜ Pending - -
8.5 Migration Strategy ⬜ Pending - -
9.1 Unit Tests ⬜ Pending - -
9.2 Integration Tests ⬜ Pending - -
9.3 E2E Validation ⬜ Pending - -

Status Legend:

  • ⬜ Pending
  • 🟡 In Progress
  • ✅ Complete
  • ❌ Blocked

Dependencies & Order

Critical Path

Phase 1 (Foundation Models)
  ↓ (required for all other phases)
Phase 2 (State Management)
  ↓ (required for Phase 3)
Phase 3 (TaskContext)
  ↓ (required for Phase 4, 5)
Phase 4 (Document References) ─┐
Phase 5 (JSON Parsing)          ├─> (can be parallel)
  ↓                              │
Phase 6 (AI Service)            │
  ↓                              │
Phase 7 (Extraction Separation) ┘
  ↓
Phase 8 (Testing)

Parallel Work Opportunities

  • Phase 4 & 5: Can be done in parallel (independent)
  • Phase 6.1 & 6.2: Can be done in parallel with different files
  • Phase 7.1, 7.2, 7.3: Can be done in parallel (different components)

Risk Mitigation

High-Risk Areas

  1. State Management (Phase 2)

    • Risk: Breaking existing workflows
    • Mitigation: Feature flag, gradual rollout, extensive testing

  2. JSON Parsing (Phase 5)

    • Risk: Breaking AI response parsing
    • Mitigation: Keep old parsing as fallback, test with real AI responses

  3. Extraction Separation (Phase 7)

    • Risk: Breaking document processing
    • Mitigation: Test extraction → AI flow thoroughly, keep old path as fallback

Rollback Strategy

  • Feature Flags: Use flags to enable/disable new code paths
  • Backward Compatibility: Keep old code paths until new code is validated
  • Database Migration: Make state fields optional initially (default 0)

File Reference Map

New Files

File Purpose Phase
datamodelWorkflow.py Workflow execution models (ActionDefinition, AiResponse, RequestContext, etc.) 1.1
datamodelDocref.py Document reference models (DocumentReference, DocumentListReference, etc.) 1.2

Modified Files

File Changes Phase
datamodelChat.py ChatWorkflow, TaskContext enhancements, remove WorkflowModeEnum.WORKFLOW_ACTIONPLAN 2.1, 3.1, 8.3
jsonUtils.py Add parseJsonWithModel function 5.1
workflowProcessor.py Remove index parameters, remove ActionplanMode 2.2, 8.2
modeDynamic.py Remove SimpleNamespace, use typed models, update debugType 3.2, 4.1, 5.2, 8.4
modeAutomation.py Remove ActionplanMode comments 8.4
actionExecutor.py Remove index parameters 2.2
mainServiceAi.py Consolidate methods, remove extraction 6.1, 7.2
mainServiceChat.py Update document lookup 4.1
methodAi.py Add extractContent action, use contentParts, callAiContent 6.2, 7.1, 7.3

Deleted Files

File Reason Phase
modeActionplan.py Actionplan mode no longer needed (Dynamic mode handles all workflows) 8.1
promptGenerationActionsActionplan.py Only used by Actionplan mode 8.1

Quick Reference: Key Changes Summary

Model Changes

  • ✅ ChatWorkflow (in datamodelChat.py): Add currentRound, currentTask, currentAction + helper methods
  • ✅ TaskContext (in datamodelChat.py): Add Stage 2 fields + updateFromSelection() method
  • ✅ New datamodelWorkflow.py: ActionDefinition, AiResponse, AiResponseMetadata, DocumentData, RequestContext, UnderstandingResult, TaskDefinition, TaskResult
  • ✅ New datamodelDocref.py: DocumentReference, DocumentListReference, DocumentItemReference, DocumentReferenceList

Function Signature Changes

  • ✅ Remove taskIndex, actionIndex parameters (use workflow.getTaskIndex())
  • ✅ callAiContent() replaces callAiDocuments() + callAiText()
  • ✅ parseJsonWithModel() (in jsonUtils.py) replaces manual JSON parsing

Workflow Changes

  • ✅ Extract documents BEFORE AI calls (separate action)
  • ✅ AI calls receive contentParts (not documents)
  • ✅ Use typed DocumentReferenceList from datamodelDocref.py (not List[str])

Mode Changes

  • ✅ Remove Actionplan mode (no longer needed)
  • ✅ Dynamic mode handles all workflow execution
  • ✅ Remove WorkflowModeEnum.WORKFLOW_ACTIONPLAN from enum

Next Steps

  1. Review this plan with team
  2. Assign phases to developers
  3. Set up tracking (use progress table)
  4. Start Phase 1 (Foundation Models)
  5. Iterate through phases sequentially

Questions & Notes

Questions to Resolve:

  • Database migration strategy for ChatWorkflow state fields?
  • Feature flag approach for gradual rollout?
  • Backward compatibility requirements?
  • Testing environment setup?

Notes:

  • Keep old code paths until new code is validated
  • Test each phase before moving to next
  • Document any deviations from plan