wiki/appdoc/doc_system_prompt_flow.md

# Prompt Building Flow - Summary

## ✅ Current State: CLEAN AND SIMPLE

**No placeholders** - Everything handled through function parameters.

---

## Complete Flow

```
1. mainServiceAi.callAiDocuments()
   └─> Delegates to subCoreAi.callAiDocuments()

2. subCoreAi.callAiDocuments()
   ├─> Builds FIRST prompt: buildGenerationPrompt(..., continuationContext=None)
   └─> Calls _callAiWithLooping(prompt, ..., promptBuilder, promptArgs)

3. _callAiWithLooping() - Loops until complete
   ├─> Iteration 1: Uses prompt from step 2
   └─> Iteration 2+: 
       ├─> Builds continuationContext = buildContinuationContext(allSections, lastRawResponse)
       └─> Calls promptBuilder(**promptArgs, continuationContext=continuationContext)

4. buildGenerationPrompt()
   ├─> If continuationContext has valid last_raw_json: Build CONTINUATION prompt
   └─> Else: Build INITIAL prompt

5. buildContinuationContext()
   └─> Returns: {section_count, last_raw_json}  [Simplified - only what's needed]
```

---

## Key Files & Responsibilities

### `mainServiceAi.py`
- **Role**: Entry point, delegation only
- **Responsibility**: Initialize AI objects, delegate to subCoreAi

### `subCoreAi.callAiDocuments()`
- **Role**: Document generation coordinator
- **Responsibility**: 
  - Build first prompt
  - Prepare promptArgs for continuation
  - Call looping function

### `subCoreAi._callAiWithLooping()`
- **Role**: Iteration manager
- **Responsibility**:
  - Manage iterations
  - Build continuation context for iterations 2+
  - Call prompt builder with context
  - Extract sections and detect completion

### `subPromptBuilderGeneration.buildGenerationPrompt()`
- **Role**: Prompt builder
- **Responsibility**:
  - Build initial prompt (if continuationContext=None or invalid)
  - Build continuation prompt (if continuationContext has valid last_raw_json)
  - Handle both cases in one function

### `jsonUtils.buildContinuationContext()`
- **Role**: Context builder
- **Responsibility**:
  - Extract section_count from accumulated sections
  - Extract and clean last_raw_json from last response
  - Return minimal context dict

---

## Data Flow

```
Initial Call:
  userPrompt, title, extracted_content
    ↓
  buildGenerationPrompt(..., continuationContext=None)
    ↓
  Initial Prompt (no continuation)

Continuation Call:
  allSections, lastRawResponse
    ↓
  buildContinuationContext(allSections, lastRawResponse)
    ↓
  {section_count: N, last_raw_json: "..."}
    ↓
  buildGenerationPrompt(..., continuationContext={...})
    ↓
  Continuation Prompt (with last JSON fragment)
```

---

## Improvements Made

1. ✅ **Simplified `buildContinuationContext()`**
   - Removed unused fields: `last_content_sample`, `last_content_type`, `continuation_guidance`
   - Only returns: `section_count`, `last_raw_json`

2. ✅ **Improved empty JSON handling**
   - Validates `last_raw_json` is not empty or `{}`
   - Falls back to initial prompt if invalid

3. ✅ **Clearer continuation check**
   - Single boolean `has_continuation` checks all conditions
   - Eliminates nested if/else confusion

---

## No Placeholders Needed!

The old placeholder-based approach (`<CONTINUATION_INSTRUCTION>`) has been completely removed. All logic is handled through:
- Function parameters (`continuationContext`)
- Direct conditional logic in `buildGenerationPrompt()`

This is **much simpler** and easier to understand and maintain.

---

## ✅ Generic for Any User Prompt

**Yes, this logic works for ANY user prompt!** 

### Why It's Generic:

1. **No Content Assumptions**
   - `userPrompt` is inserted directly into prompts without parsing or validation
   - No restrictions on length, format, or content type
   - No language detection required - AI handles any language

2. **Flexible JSON Template**
   - Supports multiple content types: `heading`, `paragraph`, `table`, `list`, `code`
   - AI chooses appropriate `content_type` based on the request
   - Can combine multiple content types in one document

3. **Generic Continuation**
   - Shows the exact last JSON fragment (works with any content)
   - No assumptions about what was generated
   - AI continues from wherever the fragment stopped

4. **No Special Handling Required**
   - Works for:
     - Lists (like "Generate 9000 prime numbers")
     - Text content (like "Write a story about...")
     - Structured data (like "Create a report with tables")
     - Code (like "Generate Python code for...")
     - Mixed content (like "Create a document with headings, paragraphs, and tables")

### Example Prompts That Work:

```python
# Numbers/Data
"Generate the first 9000 prime numbers"
"List all countries and their capitals"
"Create a multiplication table from 1 to 100"

# Text Content  
"Write a comprehensive guide about Python programming"
"Summarize the history of artificial intelligence"
"Create a document about project management best practices"

# Structured Data
"Create a report with sales data in tables"
"Generate a comparison chart of programming languages"
"Build a document with headings for each section"

# Code
"Generate Python code examples for common algorithms"
"Create documentation with code snippets"
"Write a tutorial with code blocks"

# Mixed Content
"Create a technical document with headings, paragraphs, lists, and code examples"
"Generate a report with introduction, tables, charts, and conclusion"
```

### Practical Considerations:

1. **Token Limits**: Very long prompts might hit model token limits, but this is handled by the continuation mechanism
2. **Special Characters**: Any characters in the prompt are passed through (AI handles them)
3. **Multi-line Prompts**: Fully supported - inserted into f-strings as-is
4. **Language**: Works with any language - AI detects and responds appropriately

### The Logic Adapts:

- **Initial Prompt**: Tells AI to generate content based on user request
- **Continuation Prompt**: Shows last fragment and asks to continue
- **Completion Detection**: AI sets `complete_response: true` when done

The AI model does the adaptation, not the code. The code just provides a consistent structure and continuation mechanism.