gateway/docs/frontend-documentation/dynamic-forms-and-pagination.md

# Dynamic Forms and Pagination - Generic Frontend Patterns

This document describes the generic, reusable patterns for dynamic form generation and pagination that apply across all entity types in the frontend. These patterns enable completely backend-driven UI generation with no hardcoding.

## Table of Contents

1. [Overview](#overview)
2. [Backend Metadata System](#backend-metadata-system)
3. [Dynamic Form Generation](#dynamic-form-generation)
4. [Pagination System](#pagination-system)
5. [Search and Filtering](#search-and-filtering)
6. [Sorting](#sorting)
7. [Localization](#localization)
8. [Implementation Patterns](#implementation-patterns)

---

## Overview

The frontend uses a **completely backend-driven approach** for generating forms, tables, filters, and pagination controls. All field definitions, labels, validation rules, and UI structure come from the backend through the attribute definition system.

### Key Principles

- **No Hardcoding**: Field names, labels, types, validation rules, and options are never hardcoded
- **Backend-Driven**: All UI components generated from backend metadata
- **Entity-Agnostic**: Same patterns work for workflows, users, mandates, files, prompts, automations, etc.
- **Automatic Updates**: When backend adds new fields, frontend automatically displays them without code changes
- **Localization**: All labels and options support multiple languages from backend

### Applicable Entity Types

These patterns apply to any entity type that has:
- An attributes endpoint: `GET /api/attributes/{EntityType}`
- A list endpoint supporting pagination: `GET /api/{entities}/?pagination=...`
- CRUD operations: Create, Read, Update, Delete

Examples: `ChatWorkflow`, `User`, `Mandate`, `FileItem`, `Prompt`, `AutomationDefinition`, `ChatMessage`, `ChatLog`, etc.

---

## Backend Metadata System

### Attribute Definition Endpoint

Every entity type exposes an attributes endpoint that provides complete metadata:

```
GET /api/attributes/{EntityType}
```

**Examples:**
- `GET /api/attributes/ChatWorkflow`
- `GET /api/attributes/User`
- `GET /api/attributes/Mandate`
- `GET /api/attributes/FileItem`
- `GET /api/attributes/Prompt`

### Attribute Definition Structure

Each attribute returned from the endpoint contains:

- **`name`** - Field name (e.g., "status", "name", "email", "createdAt")
- **`type`** - Field data type (e.g., "text", "select", "integer", "timestamp", "checkbox", "email", "url")
- **`label`** - Localized field label (object with language keys: `{"en": "English Label", "fr": "French Label"}`)
- **`description`** - Field description text (for help text/tooltips)
- **`required`** - Boolean indicating if field is required for validation
- **`readonly`** - Boolean indicating if field is read-only (cannot be edited)
- **`editable`** - Boolean indicating if field can be edited (inverse of readonly)
- **`visible`** - Boolean indicating if field should be displayed in UI
- **`options`** - Array of options for select fields (each option has `value` and localized `label`)

### Response Format

The attributes endpoint returns:

```json
{
  "entityType": "ChatWorkflow",
  "attributes": [
    {
      "name": "status",
      "type": "select",
      "label": {"en": "Status", "fr": "Statut"},
      "description": "Current status of the workflow",
      "required": false,
      "readonly": false,
      "editable": true,
      "visible": true,
      "options": [
        {"value": "running", "label": {"en": "Running", "fr": "En cours"}},
        {"value": "completed", "label": {"en": "Completed", "fr": "Terminé"}}
      ]
    },
    {
      "name": "name",
      "type": "text",
      "label": {"en": "Name", "fr": "Nom"},
      "description": "Name of the workflow",
      "required": true,
      "readonly": false,
      "editable": true,
      "visible": true
    }
    // ... more attributes
  ]
}
```

---

## Dynamic Form Generation

Dynamic forms are generated entirely from backend attribute definitions. The same pattern works for create forms, edit forms, and inline editing.

### Form Generation Process

1. **Fetch Attribute Definitions**
   - Call `GET /api/attributes/{EntityType}` when form component mounts
   - Store attribute definitions in component state

2. **Filter Attributes**
   - For edit forms: Filter where `visible: true` AND `editable: true`
   - For create forms: Filter where `visible: true` AND `editable: true` (may exclude readonly fields)
   - For display-only: Filter where `visible: true`

3. **Generate Form Fields**
   - Iterate through filtered attributes
   - Generate appropriate input component based on `type`
   - Use `label` for field labels (localized)
   - Use `description` for help text/tooltips
   - Use `required` to show required indicators

4. **Handle Form Submission**
   - Validate form using attribute metadata
   - Send form data to appropriate endpoint (POST for create, PUT for update)
   - Handle success/error responses

### Field Type to Input Component Mapping

Based on the `type` property, generate appropriate input components:

- **`text`** → Text input (single line)
- **`textarea`** → Multi-line text area
- **`select`** → Dropdown/select input with options from `options` array
- **`integer`** → Number input (integer only)
- **`float`** or **`number`** → Number input (decimal allowed)
- **`timestamp`** → Date/time picker
- **`date`** → Date picker (date only)
- **`time`** → Time picker (time only)
- **`checkbox`** → Checkbox input (boolean)
- **`email`** → Email input (with email validation)
- **`url`** → URL input (with URL validation)
- **`password`** → Password input (masked)
- **`file`** → File upload input

### Form Field Rendering Rules

For each form field:

1. **Field Label**
   - Use `label[userLanguage]` or `label.en` as fallback
   - Show required indicator (asterisk, etc.) if `required: true`

2. **Input Component**
   - Generate based on `type` property
   - For `select` fields: Populate options from `options` array, use localized labels
   - For `integer`/`number` fields: Set appropriate min/max constraints if available
   - For `timestamp`/`date`/`time` fields: Use appropriate date/time picker component

3. **Help Text**
   - Display `description` as help text or tooltip
   - Show below or next to input field

4. **Validation**
   - Mark field as required if `required: true`
   - Validate type (e.g., integer fields must be numbers)
   - Validate select fields (value must be in options array)
   - Validate email/url fields with appropriate regex patterns

5. **Disabled State**
   - Disable input if `readonly: true` or `editable: false`
   - Show read-only indicator if applicable

### Form Validation

Before form submission:

1. **Required Field Validation**
   - Check all fields where `required: true` have non-empty values
   - Show error messages for missing required fields

2. **Type Validation**
   - Validate integer fields contain valid integers
   - Validate number fields contain valid numbers
   - Validate email fields contain valid email addresses
   - Validate URL fields contain valid URLs
   - Validate timestamp/date fields contain valid dates

3. **Option Validation**
   - For select fields, ensure value exists in `options` array
   - Show error if invalid option selected

4. **Custom Validation**
   - Apply any additional validation rules from backend metadata if available

### Form Submission

On form submit:

1. **Collect Form Data**
   - Gather values from all form fields
   - Use attribute `name` as keys in form data object

2. **Prepare Request**
   - For create: `POST /api/{entities}` with form data
   - For update: `PUT /api/{entities}/{id}` with form data
   - Include only changed fields (for updates) or all form data (for creates)

3. **Handle Response**
   - On success: Show success message, navigate or refresh as appropriate
   - On error: Display validation errors from backend, highlight invalid fields

---

## Pagination System

Pagination is a unified system that works across all list endpoints. It supports pagination, sorting, filtering, and general search in a consistent way.

### Pagination Endpoint Pattern

All list endpoints support pagination through a query parameter:

```
GET /api/{entities}/?pagination={JSON_ENCODED_PAGINATION_PARAMS}
```

**Examples:**
- `GET /api/workflows/?pagination={"page":1,"pageSize":20,"sort":[],"filters":null}`
- `GET /api/users/?pagination={"page":1,"pageSize":20,"sort":[{"field":"name","direction":"asc"}],"filters":{"status":"active"}}`
- `GET /api/files/list?pagination={"page":2,"pageSize":50,"sort":[],"filters":{"search":"report"}}`

### Pagination Request Parameters

The `pagination` query parameter is a JSON-encoded object with the following structure:

**`PaginationParams` Structure:**
- **`page`** - Current page number (1-based, minimum 1)
- **`pageSize`** - Number of items per page (minimum 1, maximum 1000)
- **`sort`** - Array of sort field configurations
  - Each sort field contains:
    - `field` - Field name to sort by (must match an attribute name)
    - `direction` - Sort direction: "asc" or "desc"
- **`filters`** - Filter criteria dictionary (see [Search and Filtering](#search-and-filtering))

**Example:**
```json
{
  "page": 1,
  "pageSize": 20,
  "sort": [
    {"field": "lastActivity", "direction": "desc"},
    {"field": "name", "direction": "asc"}
  ],
  "filters": {
    "search": "invoice",
    "status": "running"
  }
}
```

### Pagination Response Structure

All paginated endpoints return data in this format:

```json
{
  "items": [
    // Array of entity objects
  ],
  "pagination": {
    "currentPage": 1,
    "pageSize": 20,
    "totalItems": 45,
    "totalPages": 3,
    "sort": [
      {"field": "lastActivity", "direction": "desc"}
    ],
    "filters": {
      "search": "invoice",
      "status": "running"
    }
  }
}
```

**`PaginationMetadata` Structure:**
- **`currentPage`** - Current page number (1-based)
- **`pageSize`** - Number of items per page
- **`totalItems`** - Total number of items across all pages (after filters applied)
- **`totalPages`** - Total number of pages (calculated from totalItems / pageSize)
- **`sort`** - Current sort configuration applied (array of SortField objects)
- **`filters`** - Current filters applied (mirrors request filters)

### Pagination UI Components

Generate pagination controls using the `pagination` metadata from response:

1. **Page Information Display**
   - Show "Page X of Y" (using `currentPage` and `totalPages`)
   - Show "Showing X-Y of Z items" (calculate from `currentPage`, `pageSize`, `totalItems`)

2. **Page Navigation Controls**
   - Previous page button (disabled if `currentPage === 1`)
   - Next page button (disabled if `currentPage === totalPages`)
   - Page number buttons (show current page and adjacent pages)
   - First page button (if not on first page)
   - Last page button (if not on last page)

3. **Page Size Selector**
   - Dropdown to change `pageSize` (common options: 10, 20, 50, 100)
   - When changed, reset to page 1 and refetch

4. **Page Change Handler**
   - When user clicks page number or navigation button:
     - Update `page` in pagination parameters
     - Preserve all filters and sort settings
     - Refetch data with updated pagination

### Non-Paginated Requests

If user wants all items without pagination:

- Omit `pagination` parameter entirely: `GET /api/{entities}/`
- Backend returns all items (still wrapped in `PaginatedResponse` with `pagination: null`)

---

## Search and Filtering

The pagination system supports both general search and field-specific filtering. Both are combined in the `filters` object.

### General Search

General search searches across all text fields in the entity.

**Implementation:**
1. Display a single search input box (not field-specific)
2. When user types, update `filters.search` in pagination parameters
3. Debounce search input (wait 300-500ms after user stops typing before sending request)
4. Reset to page 1 when search query changes
5. Combine search with field-specific filters (both are in the `filters` object)

**Filter Structure:**
```json
{
  "filters": {
    "search": "invoice"
  }
}
```

### Field-Specific Filtering

Field-specific filters allow filtering by individual fields with various operators.

**Filter Structure:**

Simple equals filter:
```json
{
  "filters": {
    "status": "running"
  }
}
```

Filter with operator:
```json
{
  "filters": {
    "name": {
      "operator": "contains",
      "value": "workflow"
    }
  }
}
```

**Supported Operators:**
- **`equals`** or **`eq`** - Exact match
- **`contains`** - Substring match (case-insensitive for strings)
- **`startsWith`** - String starts with (case-insensitive)
- **`endsWith`** - String ends with (case-insensitive)
- **`gt`** - Greater than (for numbers/dates)
- **`gte`** - Greater than or equal
- **`lt`** - Less than
- **`lte`** - Less than or equal
- **`in`** - Value in list (array of values)
- **`notIn`** - Value not in list (array of values)

**Multiple Filters:**
All filters are combined with AND logic:
```json
{
  "filters": {
    "search": "invoice",
    "status": "running",
    "currentRound": {
      "operator": "gt",
      "value": 0
    }
  }
}
```

### Filter Control Generation

Generate filter controls dynamically from attribute definitions:

1. **Fetch Attribute Definitions**
   - Call `GET /api/attributes/{EntityType}` when filter component mounts
   - Filter attributes where `visible: true` to determine which filters to show

2. **Generate Filter UI by Type**
   - **`text`** fields → Text input filter (supports "contains", "equals", "startsWith", "endsWith" operators)
   - **`select`** fields → Dropdown filter with options from `options` array (use localized labels)
   - **`integer`** or **`number`** fields → Number range filter (min/max inputs) or single number input with comparison operators (gt, gte, lt, lte)
   - **`timestamp`** or **`date`** fields → Date range picker or single date picker with comparison operators
   - **`checkbox`** fields → Boolean toggle filter (true/false/any)

3. **Filter Labels**
   - Use `label` property for filter labels (localized)
   - Display filter label next to filter control

4. **Apply Filters**
   - When user applies filter, update `filters[fieldName]` in pagination parameters
   - Support multiple filters simultaneously
   - Reset to page 1 when filters change
   - Refetch data with updated filters

5. **Active Filter Indicators**
   - Display active filters as chips/badges showing field label and value
   - Allow removing individual filters by removing them from `filters` object
   - Show "Clear all filters" button when filters are active

### Combined Search and Filters

Search and field filters work together:

- Both are stored in the `filters` object
- All filters are combined with AND logic
- When any filter changes, combine all and refetch
- Preserve all active filters when sorting or changing pages

---

## Sorting

Sorting allows users to order list items by one or more fields.

### Sort Configuration

Sorting is configured in the `sort` array of pagination parameters:

```json
{
  "sort": [
    {"field": "lastActivity", "direction": "desc"},
    {"field": "name", "direction": "asc"}
  ]
}
```

- Multiple sort fields create multi-level sorting (sorts by first field, then by second field for ties)
- Sort fields are applied in order (first field is primary sort, second is secondary sort, etc.)

### Sort UI Implementation

1. **Column Header Sorting**
   - Make column headers clickable for sortable columns
   - Show sort indicator (arrow up/down) next to column header when sorted
   - Clicking a column header:
     - If not currently sorted by that column: Add sort field with "asc" direction
     - If sorted ascending: Change to "desc"
     - If sorted descending: Remove sort field (or change to "asc" if keeping)
   - Update `sort` array in pagination parameters
   - Preserve all filters and search
   - Refetch data with updated sort

2. **Sort Indicator Display**
   - Show up arrow (↑) for ascending sort
   - Show down arrow (↓) for descending sort
   - Show both arrows when column is sortable but not currently sorted
   - Hide sort indicators for non-sortable columns

3. **Multi-Level Sort**
   - When user clicks a second column header, add it to the `sort` array
   - Display multiple sort indicators if multiple columns are sorted
   - Allow removing individual sort fields

### Sortable Columns

Determine which columns are sortable:

- All visible columns can potentially be sortable
- Backend handles sorting logic, so frontend can attempt to sort any field
- If backend doesn't support sorting a field, it will ignore that sort field
- Readonly fields may still be sortable (e.g., timestamps, IDs)

---

## Localization

All labels and options support multiple languages from the backend.

### Language Detection

1. **User Language Preference**
   - Get user's preferred language from user settings
   - Fall back to browser locale if user setting not available
   - Default to "en" (English) if no language detected

2. **Label Access Pattern**
   - Access localized labels from `label` object: `label[userLanguage]`
   - Fall back to `label.en` if label for current language is missing
   - For select options: `option.label[userLanguage]` or `option.label.en` as fallback

### Localized Components

Apply localization to:

1. **Form Fields**
   - Field labels: `attribute.label[userLanguage]`
   - Help text/descriptions: `attribute.description` (may be localized if backend provides)
   - Select options: `option.label[userLanguage]`
   - Validation error messages: Use localized messages

2. **Table Columns**
   - Column headers: `attribute.label[userLanguage]`
   - Cell values for select fields: `option.label[userLanguage]`

3. **Filter Controls**
   - Filter labels: `attribute.label[userLanguage]`
   - Filter options: `option.label[userLanguage]`

4. **Pagination Controls**
   - "Page X of Y" text: Use localized strings
   - "Showing X-Y of Z items" text: Use localized strings
   - Button labels (Previous, Next, etc.): Use localized strings

---

## Implementation Patterns

### Pattern 1: Generic List Page Component

A reusable list page component that works for any entity type:

**Component Structure:**
1. Accept `entityType` as prop (e.g., "ChatWorkflow", "User", "Mandate")
2. Fetch attributes: `GET /api/attributes/{entityType}`
3. Fetch list data: `GET /api/{entities}/?pagination=...`
4. Generate table columns from attributes
5. Generate filter controls from attributes
6. Handle pagination, search, filtering, and sorting
7. Handle row clicks (navigate to detail page)

**State Management:**
- Store attribute definitions
- Store pagination parameters (page, pageSize, sort, filters)
- Store pagination metadata from response
- Store list items
- Store loading/error states

**User Interactions:**
- Search input → Update `filters.search`
- Filter controls → Update `filters[fieldName]`
- Column header clicks → Update `sort` array
- Page navigation → Update `page`
- Page size change → Update `pageSize`, reset to page 1

### Pattern 2: Generic Form Component

A reusable form component that works for any entity type:

**Component Structure:**
1. Accept `entityType` and `mode` as props ("create" or "edit")
2. Accept `entityId` as prop (for edit mode)
3. Fetch attributes: `GET /api/attributes/{entityType}`
4. Fetch entity data (for edit mode): `GET /api/{entities}/{id}`
5. Generate form fields from attributes
6. Handle form submission
7. Handle validation

**Form Field Generation:**
- Filter attributes: `visible: true` AND `editable: true`
- Generate input components based on `type`
- Apply labels, descriptions, required indicators
- Handle validation

**Form Submission:**
- Validate form using attribute metadata
- For create: `POST /api/{entities}` with form data
- For update: `PUT /api/{entities}/{id}` with form data
- Handle success/error responses

### Pattern 3: Generic Table Component

A reusable table component that works for any entity type:

**Component Structure:**
1. Accept `attributes` and `items` as props
2. Generate columns from attributes (filter `visible: true`)
3. Render table rows with formatted cell values
4. Handle column header clicks for sorting
5. Handle row clicks for navigation

**Cell Value Formatting:**
- Use `type` property to determine formatting
- Format select fields using options array
- Format timestamps as relative/absolute time
- Format numbers with separators
- Handle null/undefined values

### Pattern 4: Generic Filter Component

A reusable filter component that works for any entity type:

**Component Structure:**
1. Accept `attributes` and `filters` as props
2. Generate filter controls from attributes (filter `visible: true`)
3. Handle filter changes
4. Display active filters as chips/badges
5. Allow removing individual filters

**Filter Control Generation:**
- Generate appropriate filter UI based on `type`
- Use localized labels
- Support multiple operators for text/number fields
- Update filters object and trigger refetch

### Key Implementation Guidelines

1. **Never Hardcode**
   - Never hardcode field names, labels, types, or validation rules
   - Always fetch attribute definitions from backend
   - Use attribute metadata for all UI generation

2. **Entity-Agnostic**
   - Same components work for all entity types
   - Pass `entityType` as parameter to determine endpoints
   - Use generic prop names (attributes, items, etc.)

3. **State Management**
   - Keep pagination state separate from entity data
   - Preserve filters and sort when changing pages
   - Reset to page 1 when filters/search change

4. **Error Handling**
   - Handle API errors gracefully
   - Display validation errors from backend
   - Show loading states during API calls

5. **Performance**
   - Debounce search input (300-500ms)
   - Cache attribute definitions (don't refetch on every render)
   - Use pagination to limit data fetched

6. **Accessibility**
   - Use semantic HTML for forms and tables
   - Provide ARIA labels for form fields
   - Ensure keyboard navigation works
   - Support screen readers

---

## Summary

This generic system enables:

- **Complete Backend-Driven UI**: All forms, tables, filters generated from backend metadata
- **Zero Hardcoding**: No field definitions, labels, or validation rules in frontend code
- **Automatic Updates**: New backend fields automatically appear in frontend
- **Consistent UX**: Same patterns across all entity types
- **Localization**: Multi-language support from backend metadata
- **Reusable Components**: Generic components work for all entity types

By following these patterns, the frontend becomes a thin presentation layer that dynamically renders UI based on backend metadata, ensuring consistency, maintainability, and automatic adaptation to backend changes.