184 lines
7.3 KiB
Markdown
184 lines
7.3 KiB
Markdown
# Analysis: `processDocumentsWithContinuation` and Subfunctions Usage
|
|
|
|
## Executive Summary
|
|
|
|
**FINDING**: The function `processDocumentsWithContinuation` in `subDocumentProcessing.py` is **NOT USED** anywhere in the active codebase. The continuation chain was only referenced by the deleted `subDocumentGeneration.py` module.
|
|
|
|
---
|
|
|
|
## Main Function: `processDocumentsWithContinuation`
|
|
|
|
**Location**: `gateway/modules/services/serviceAi/subDocumentProcessing.py:303`
|
|
|
|
**Status**: ❌ **NOT USED**
|
|
|
|
### Usage Search Results
|
|
|
|
- ❌ No actual code calls to `.processDocumentsWithContinuation(`
|
|
- ⚠️ Only mentioned in documentation files:
|
|
- `wiki/poweron/appdoc/doc_system_function_relationship_ai.md` (documentation)
|
|
- `gateway/callAiWithDocumentGeneration_usage_analysis.md` (previous analysis - noted it was called by deleted code)
|
|
|
|
### Why It's Not Used
|
|
|
|
The only caller was `subDocumentGeneration._processDocumentsUnified()` which we just deleted. The current active codebase uses `subCoreAi.callAiDocuments()` which has its own continuation logic via `_callAiWithLooping()`.
|
|
|
|
---
|
|
|
|
## Function Call Chain Analysis
|
|
|
|
```
|
|
processDocumentsWithContinuation (line 303) - ❌ NOT USED
|
|
├─> _buildContinuationPrompt (line 319, 324) - ❌ ONLY USED HERE
|
|
└─> _processWithContinuationLoop (line 322, 373) - ❌ ONLY USED HERE
|
|
├─> _buildContinuationIterationPrompt (line 393, 459) - ❌ ONLY USED HERE
|
|
└─> processDocumentsPerChunkJsonWithPrompt (line 402) - ✅ USED ELSEWHERE
|
|
```
|
|
|
|
---
|
|
|
|
## Subfunction Analysis
|
|
|
|
### 1. `_buildContinuationPrompt`
|
|
**Location**: Line 324-371
|
|
**Status**: ✅ **USED** (but only internally)
|
|
**Called by**: `processDocumentsWithContinuation` (line 319)
|
|
**Effectively**: ❌ **UNUSED** (because parent function is unused)
|
|
|
|
**Internal Usage**:
|
|
- Called from `processDocumentsWithContinuation` at line 319
|
|
|
|
**Functionality**:
|
|
- Builds a prompt with continuation instructions
|
|
- Adds JSON structure requirements with `"continue": true/false` flag
|
|
- Adds `continuation_context` field specification
|
|
|
|
**Note**: This uses a different continuation pattern than `SubCoreAi._callAiWithLooping()`:
|
|
- This uses `"continue": true/false + "continuation_context"` for document sections
|
|
- SubCoreAi uses `buildContinuationContext()` with `last_raw_json`
|
|
|
|
---
|
|
|
|
### 2. `_processWithContinuationLoop`
|
|
**Location**: Line 373-457
|
|
**Status**: ✅ **USED** (but only internally)
|
|
**Called by**: `processDocumentsWithContinuation` (line 322)
|
|
**Effectively**: ❌ **UNUSED** (because parent function is unused)
|
|
|
|
**Internal Usage**:
|
|
- Called from `processDocumentsWithContinuation` at line 322
|
|
|
|
**External Dependencies**:
|
|
- Calls `self._buildContinuationIterationPrompt()` (line 393)
|
|
- Calls `self.processDocumentsPerChunkJsonWithPrompt()` (line 402)
|
|
|
|
**Functionality**:
|
|
- Implements continuation loop (max 10 iterations)
|
|
- Accumulates sections across iterations
|
|
- Checks `continue` flag and `continuation_context` to determine if more iterations needed
|
|
- Builds final result with accumulated sections
|
|
|
|
---
|
|
|
|
### 3. `_buildContinuationIterationPrompt`
|
|
**Location**: Line 459-498
|
|
**Status**: ✅ **USED** (but only internally)
|
|
**Called by**: `_processWithContinuationLoop` (line 393)
|
|
**Effectively**: ❌ **UNUSED** (because parent chain is unused)
|
|
|
|
**Internal Usage**:
|
|
- Called from `_processWithContinuationLoop` at line 393 (in loop, conditionally)
|
|
|
|
**Functionality**:
|
|
- Builds a prompt for continuation iteration with context
|
|
- Includes summary of previously generated content (last 3 sections)
|
|
- Includes continuation instructions with last section ID, element index, remaining requirements
|
|
|
|
---
|
|
|
|
### 4. `processDocumentsPerChunkJsonWithPrompt`
|
|
**Location**: Line 219-301
|
|
**Status**: ✅ **USED ELSEWHERE**
|
|
**Called by**:
|
|
- `_processWithContinuationLoop` (line 402)
|
|
- Also referenced in backup files (not active code)
|
|
|
|
**Internal Usage**:
|
|
- Called from `_processWithContinuationLoop` at line 402
|
|
|
|
**External Usage Search**:
|
|
- ✅ Used internally by continuation loop
|
|
- ⚠️ Referenced in `local/backup/backup_mainServiceAi.py.txt` (backup file, not active)
|
|
- ❌ Not used by any other active code
|
|
|
|
**Functionality**:
|
|
- Processes documents with per-chunk AI calls
|
|
- Uses a custom prompt instead of default extraction prompt
|
|
- Returns merged JSON document
|
|
|
|
**Note**: This function itself is only used by the continuation loop. However, it's a more general function that could be useful, so it's not "dead code" - it's just currently only used by unused code.
|
|
|
|
---
|
|
|
|
## Summary Table
|
|
|
|
| Function | Line | Status | Called By | Effectively Used? |
|
|
|----------|------|--------|-----------|-------------------|
|
|
| `processDocumentsWithContinuation` | 303 | ❌ Not used | (external) | ❌ No |
|
|
| `_buildContinuationPrompt` | 324 | ✅ Used internally | `processDocumentsWithContinuation:319` | ❌ No |
|
|
| `_processWithContinuationLoop` | 373 | ✅ Used internally | `processDocumentsWithContinuation:322` | ❌ No |
|
|
| `_buildContinuationIterationPrompt` | 459 | ✅ Used internally | `_processWithContinuationLoop:393` | ❌ No |
|
|
| `processDocumentsPerChunkJsonWithPrompt` | 219 | ✅ Used internally | `_processWithContinuationLoop:402` | ⚠️ **ONLY USED BY UNUSED CODE** |
|
|
|
|
---
|
|
|
|
## Current Active Implementation
|
|
|
|
The active continuation logic is in `subCoreAi.callAiDocuments()` → `_callAiWithLooping()`:
|
|
- Uses `buildGenerationPrompt()` with `continuationContext` parameter
|
|
- Uses `buildContinuationContext()` to build context from sections
|
|
- Different continuation pattern (uses `last_raw_json` instead of `continuation_context`)
|
|
|
|
---
|
|
|
|
## Dead Code Identification
|
|
|
|
**Completely Unused Chain** (can be safely removed):
|
|
1. ✅ `processDocumentsWithContinuation` - entry point, not called
|
|
2. ✅ `_buildContinuationPrompt` - only used by #1
|
|
3. ✅ `_processWithContinuationLoop` - only used by #1
|
|
4. ✅ `_buildContinuationIterationPrompt` - only used by #3
|
|
|
|
**Potentially Unused** (only used by dead code):
|
|
- ⚠️ `processDocumentsPerChunkJsonWithPrompt` - only caller is dead code, but function is general-purpose
|
|
|
|
---
|
|
|
|
## Recommendations
|
|
|
|
1. **Remove Dead Code Chain**: All four functions (`processDocumentsWithContinuation`, `_buildContinuationPrompt`, `_processWithContinuationLoop`, `_buildContinuationIterationPrompt`) can be safely removed.
|
|
|
|
2. **For `processDocumentsPerChunkJsonWithPrompt`**:
|
|
- **Option A**: Remove if not needed (it's only used by the dead continuation chain)
|
|
- **Option B**: Keep if it might be useful for future custom prompt processing
|
|
- **Recommendation**: Since it's a general-purpose function that could be useful, keep it but note that it's currently unused.
|
|
|
|
3. **If Keeping**: Document why this continuation logic exists but is unused, or mark as deprecated/legacy alternative to `_callAiWithLooping()`.
|
|
|
|
---
|
|
|
|
## Verification Commands
|
|
|
|
To verify these findings:
|
|
|
|
```bash
|
|
# Search for actual function calls (should return no results for the main function)
|
|
grep -r "\.processDocumentsWithContinuation(" gateway/ --exclude-dir=wiki --exclude-dir=local --exclude-dir=backup
|
|
|
|
# Search for _buildContinuationPrompt usage (should only find the definition)
|
|
grep -r "_buildContinuationPrompt" gateway/ --exclude-dir=wiki --exclude-dir=local --exclude-dir=backup --exclude="*.md"
|
|
|
|
# Search for _processWithContinuationLoop usage (should only find the definition)
|
|
grep -r "_processWithContinuationLoop" gateway/ --exclude-dir=wiki --exclude-dir=local --exclude-dir=backup --exclude="*.md"
|
|
```
|
|
|