# JSON String Accumulation KPI Adaptation Plan

## Changes Required

### 1. Remove Current KPI Approach
- Remove KPI question from `buildContinuationContext()` in `jsonUtils.py`
- Remove KPI question display from `buildGenerationPrompt()` in `subPromptBuilderGeneration.py`
- Remove `extractKpiFromResponse()` and `validateKpiProgression()` from `subJsonResponseHandling.py` (or adapt them)

### 2. Add Separate AI Call for KPI Definition
**When:** After detecting incomplete JSON on iteration 1, BEFORE entering accumulation mode

**Input to AI:**
- User prompt
- Delivered data summary
- Last complete element
- Cut-off element
- Parsed JSON structure (if available)

**AI Task:** Analyze the prompt and delivered data to define:
- Which JSON items to track (e.g., "items in bullet_list", "rows in table", "elements in array")
- Target values for each tracked item
- JSON paths/selectors to extract values

**Output Format:**
```json
{
  "kpis": [
    {
      "id": "prime_numbers_count",
      "description": "Number of prime numbers in the list",
      "jsonPath": "sections[0].elements[0].items",
      "targetValue": 4000,
      "currentValue": 0
    },
    {
      "id": "table_rows",
      "description": "Rows in the data table",
      "jsonPath": "sections[1].elements[0].rows",
      "targetValue": 100,
      "currentValue": 0
    }
  ]
}
```

### 3. Update JsonAccumulationState
**Replace:**
- `lastKpi: Optional[int]` 

**With:**
- `kpiDefinitions: List[Dict[str, Any]]` - KPI definitions from AI call
- `lastKpiValues: Dict[str, int]` - Last values for each KPI (keyed by KPI id)

### 4. Add KPI Extraction Function
**Function:** `extractKpiValuesFromJson(parsedJson: Dict, kpiDefinitions: List[Dict]) -> Dict[str, int]`
- Extract current values from parsed JSON using JSON paths
- Return dict: `{kpi_id: current_value}`

### 5. Update KPI Validation Logic
**New Function:** `validateKpiProgression(accumulationState: JsonAccumulationState, currentKpiValues: Dict[str, int]) -> Tuple[bool, str]`
- Returns: `(shouldProceed, reason)`
- Logic:
  - Extract current values from parsed JSON
  - Compare with last values
  - **Proceed if:** At least ONE KPI increased
  - **Stop if:** Any KPI went backwards → return (False, "KPI went backwards")
  - **Stop if:** No KPIs progressed → return (False, "No progress")
  - **Finish if:** All KPIs completed OR JSON is complete → return (True, "Complete")

### 6. Update Flow in mainServiceAi.py

**Iteration 1:**
1. Try to parse JSON
2. If complete → done (no accumulation)
3. If incomplete:
   - Extract sections from partial JSON
   - Build continuation context
   - **NEW: Make separate AI call to define KPIs**
   - Create accumulationState with KPI definitions
   - Enter accumulation mode

**Subsequent Iterations:**
1. Accumulate JSON strings
2. Parse accumulated JSON
3. Extract current KPI values from parsed JSON
4. Validate KPI progression
5. Update accumulationState.lastKpiValues
6. Continue or stop based on validation

## Implementation Steps

1. Update `JsonAccumulationState` in `datamodelAi.py`
2. Remove KPI question from `jsonUtils.py` and `subPromptBuilderGeneration.py`
3. Create `defineKpisFromPrompt()` function in `subJsonResponseHandling.py` (or new module)
4. Create `extractKpiValuesFromJson()` function in `subJsonResponseHandling.py`
5. Update `validateKpiProgression()` in `subJsonResponseHandling.py`
6. Update `_extractSectionsFromResponse()` in `mainServiceAi.py` to call KPI definition AI
7. Update iteration loop to extract and validate KPIs