981 lines
31 KiB
Markdown
981 lines
31 KiB
Markdown
# Features Component Documentation
|
|
|
|
Comprehensive documentation of the Features layer in the Gateway application, explaining the architecture, patterns, and implementation details of all feature modules and their relationship to connectors, services, and workflows.
|
|
|
|
## Table of Contents
|
|
|
|
1. [Overview](#overview)
|
|
2. [What is a Feature?](#what-is-a-feature)
|
|
3. [Features vs Services vs Workflows](#features-vs-services-vs-workflows)
|
|
4. [Feature Architecture](#feature-architecture)
|
|
5. [Feature Lifecycle Management](#feature-lifecycle-management)
|
|
6. [Connectors in the Architecture](#connectors-in-the-architecture)
|
|
7. [Individual Features](#individual-features)
|
|
8. [Feature Patterns and Best Practices](#feature-patterns-and-best-practices)
|
|
|
|
---
|
|
|
|
## Overview
|
|
|
|
The **Features Layer** is a domain-specific business logic layer that implements core functionality for specific use cases. Features serve as **temporary solutions** that bridge the gap between initial requirements and full service implementation or workflow integration. They provide rapid prototyping capabilities while maintaining clean architectural boundaries.
|
|
|
|
```mermaid
|
|
graph TB
|
|
subgraph "Application Layers"
|
|
Routes[Routes Layer<br/>API Endpoints]
|
|
Features[Features Layer<br/>Domain-Specific Logic]
|
|
Services[Services Layer<br/>Reusable Components]
|
|
Workflows[Workflows Layer<br/>Orchestration Engine]
|
|
Interfaces[Interfaces Layer<br/>Data Access]
|
|
Connectors[Connectors Layer<br/>External Systems]
|
|
end
|
|
|
|
Routes --> Features
|
|
Routes --> Services
|
|
Routes --> Workflows
|
|
Features --> Services
|
|
Features --> Interfaces
|
|
Workflows --> Services
|
|
Services --> Interfaces
|
|
Interfaces --> Connectors
|
|
|
|
style Features fill:#e8f5e9,stroke:#1b5e20,stroke-width:3px
|
|
style Connectors fill:#fff3e0,stroke:#e65100,stroke-width:2px
|
|
```
|
|
|
|
### Key Characteristics
|
|
|
|
- **Domain-Specific**: Each feature addresses a specific business domain or use case
|
|
- **Temporary by Design**: Features are intended to be migrated to services or workflows over time
|
|
- **Stateless**: Features operate without maintaining session state
|
|
- **Service-Dependent**: Features leverage services for cross-cutting functionality
|
|
- **Interface-Dependent**: Features use interfaces to access data through connectors
|
|
- **Lifecycle-Managed**: Background features are managed through the Features Lifecycle system
|
|
|
|
---
|
|
|
|
## What is a Feature?
|
|
|
|
A **Feature** is a domain-specific business logic module that implements functionality for a particular use case. Features are designed to:
|
|
|
|
1. **Rapid Prototyping**: Enable quick implementation of new functionality without full service architecture
|
|
2. **Domain Encapsulation**: Group related business logic for a specific domain (e.g., Real Estate, Chat, Data Synchronization)
|
|
3. **Temporary Solutions**: Serve as interim implementations before migration to services or workflows
|
|
4. **Orchestration**: Coordinate between services, interfaces, and external systems to fulfill business requirements
|
|
5. **Background Processing**: Support scheduled tasks, event-driven operations, and background managers
|
|
|
|
### Feature Lifecycle Philosophy
|
|
|
|
Features follow a natural evolution path:
|
|
|
|
```mermaid
|
|
graph LR
|
|
A[Initial Requirement] --> B[Feature Implementation]
|
|
B --> C{Stability & Usage}
|
|
C -->|Mature| D[Service Migration]
|
|
C -->|Complex Workflow| E[Workflow Integration]
|
|
C -->|Still Experimental| B
|
|
|
|
D --> F[Production Service]
|
|
E --> G[Workflow Component]
|
|
|
|
style B fill:#fff3e0,stroke:#e65100
|
|
style D fill:#e8f5e9,stroke:#1b5e20
|
|
style E fill:#e1f5ff,stroke:#01579b
|
|
```
|
|
|
|
**When to Use Features:**
|
|
- New functionality that needs rapid development
|
|
- Domain-specific logic that may not be reusable
|
|
- Experimental or proof-of-concept implementations
|
|
- Background tasks requiring scheduled execution
|
|
- Integrations that are still being refined
|
|
|
|
**When to Migrate to Services:**
|
|
- Functionality becomes reusable across multiple domains
|
|
- The feature is stable and well-tested
|
|
- Multiple features or routes need the same functionality
|
|
- The logic should be part of the core service layer
|
|
|
|
**When to Migrate to Workflows:**
|
|
- The feature involves complex multi-step user interactions
|
|
- Task planning and adaptive learning are required
|
|
- The feature needs workflow orchestration capabilities
|
|
- User interactions require state management and progress tracking
|
|
|
|
---
|
|
|
|
## Features vs Services vs Workflows
|
|
|
|
Understanding the distinction between Features, Services, and Workflows is crucial for architectural decisions.
|
|
|
|
```mermaid
|
|
graph TB
|
|
subgraph "Comparison Matrix"
|
|
A[Route Request] --> B{What Type?}
|
|
B -->|Domain-Specific<br/>Single Use Case| C[Feature]
|
|
B -->|Reusable<br/>Cross-Cutting| D[Service]
|
|
B -->|Complex Multi-Step<br/>User Interaction| E[Workflow]
|
|
|
|
C --> F[Uses Services]
|
|
C --> G[Uses Interfaces]
|
|
D --> H[Uses Other Services]
|
|
D --> G
|
|
E --> D
|
|
E --> I[Uses Methods]
|
|
|
|
G --> J[Uses Connectors]
|
|
end
|
|
|
|
style C fill:#fff3e0,stroke:#e65100
|
|
style D fill:#e8f5e9,stroke:#1b5e20
|
|
style E fill:#e1f5ff,stroke:#01579b
|
|
style J fill:#fce4ec,stroke:#880e4f
|
|
```
|
|
|
|
| Aspect | Feature | Service | Workflow |
|
|
|--------|---------|---------|----------|
|
|
| **Purpose** | Domain-specific business logic | Cross-cutting, reusable functionality | Complex multi-step orchestration |
|
|
| **Scope** | Single use case or domain | Multiple use cases | User interaction flows |
|
|
| **Reusability** | Low (domain-specific) | High (cross-domain) | Medium (workflow patterns) |
|
|
| **State Management** | Stateless | Stateless | Stateful (workflow state) |
|
|
| **Dependencies** | Uses services and interfaces | Uses other services and interfaces | Uses services and methods |
|
|
| **Lifecycle** | Temporary, may migrate | Permanent core component | Permanent orchestration engine |
|
|
| **Examples** | Real Estate queries, Chat Althaus scheduler | AI processing, Document extraction | Chat workflows, Task planning |
|
|
|
|
### Decision Flow
|
|
|
|
```mermaid
|
|
flowchart TD
|
|
Start[New Functionality Required] --> Q1{Is it reusable<br/>across domains?}
|
|
Q1 -->|Yes| Service[Implement as Service]
|
|
Q1 -->|No| Q2{Does it require<br/>complex multi-step<br/>user interaction?}
|
|
Q2 -->|Yes| Workflow[Implement as Workflow]
|
|
Q2 -->|No| Q3{Is it domain-specific<br/>or experimental?}
|
|
Q3 -->|Yes| Feature[Implement as Feature]
|
|
Q3 -->|No| Service
|
|
|
|
Feature --> Q4{Feature Matures}
|
|
Q4 -->|Stable & Reusable| Service
|
|
Q4 -->|Complex Interactions| Workflow
|
|
Q4 -->|Still Experimental| Feature
|
|
|
|
style Feature fill:#fff3e0,stroke:#e65100
|
|
style Service fill:#e8f5e9,stroke:#1b5e20
|
|
style Workflow fill:#e1f5ff,stroke:#01579b
|
|
```
|
|
|
|
---
|
|
|
|
## Feature Architecture
|
|
|
|
### High-Level Architecture
|
|
|
|
```mermaid
|
|
graph TB
|
|
subgraph "Entry Point"
|
|
App[app.py<br/>FastAPI Application]
|
|
Lifecycle[Features Lifecycle<br/>featuresLifecycle.py]
|
|
end
|
|
|
|
subgraph "API Layer"
|
|
Routes[Routes<br/>routeRealEstate.py<br/>routeChatPlayground.py<br/>routeDataNeutralization.py]
|
|
end
|
|
|
|
subgraph "Feature Layer"
|
|
RE[Real Estate Feature<br/>mainRealEstate.py]
|
|
CA[Chat Althaus Feature<br/>mainChatAlthaus.py]
|
|
SD[Sync Delta Feature<br/>mainSyncDelta.py]
|
|
CP[Chat Playground Feature<br/>mainChatPlayground.py]
|
|
NP[Neutralize Playground Feature<br/>mainNeutralizePlayground.py]
|
|
end
|
|
|
|
subgraph "Service Layer"
|
|
Services[Services Container<br/>AI, Chat, SharePoint, etc.]
|
|
end
|
|
|
|
subgraph "Interface Layer"
|
|
Interfaces[Interfaces<br/>Database, Ticket, etc.]
|
|
end
|
|
|
|
subgraph "Connector Layer"
|
|
DBConn[Database Connector<br/>connectorDbPostgre.py]
|
|
TicketConn[Ticket Connectors<br/>connectorTicketsJira.py<br/>connectorTicketsClickup.py]
|
|
VoiceConn[Voice Connector<br/>connectorVoiceGoogle.py]
|
|
JsonConn[JSON Connector<br/>connectorDbJson.py]
|
|
end
|
|
|
|
subgraph "External Systems"
|
|
DB[(PostgreSQL Database)]
|
|
Jira[Jira API]
|
|
ClickUp[ClickUp API]
|
|
SharePoint[SharePoint API]
|
|
GoogleVoice[Google Voice API]
|
|
end
|
|
|
|
App --> Lifecycle
|
|
App --> Routes
|
|
Lifecycle --> CA
|
|
Lifecycle --> SD
|
|
Routes --> RE
|
|
Routes --> CP
|
|
Routes --> NP
|
|
|
|
RE --> Services
|
|
CA --> Services
|
|
SD --> Services
|
|
CP --> Services
|
|
NP --> Services
|
|
|
|
Services --> Interfaces
|
|
Interfaces --> DBConn
|
|
Interfaces --> TicketConn
|
|
Interfaces --> VoiceConn
|
|
Interfaces --> JsonConn
|
|
|
|
DBConn --> DB
|
|
TicketConn --> Jira
|
|
TicketConn --> ClickUp
|
|
VoiceConn --> GoogleVoice
|
|
Services --> SharePoint
|
|
|
|
style Features fill:#fff3e0,stroke:#e65100
|
|
style Connectors fill:#fce4ec,stroke:#880e4f
|
|
```
|
|
|
|
### Feature Request Flow
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant Client
|
|
participant Route as Route<br/>routeRealEstate.py
|
|
participant Feature as Feature<br/>mainRealEstate.py
|
|
participant Service as Service<br/>Services Container
|
|
participant Interface as Interface<br/>interfaceDbRealEstateObjects.py
|
|
participant Connector as Connector<br/>connectorDbPostgre.py
|
|
participant DB as Database<br/>PostgreSQL
|
|
|
|
Client->>Route: HTTP Request
|
|
Route->>Route: Validate Request Data
|
|
Route->>Feature: Call Feature Function
|
|
Feature->>Service: Use Service (e.g., AI Service)
|
|
Service-->>Feature: Service Result
|
|
Feature->>Interface: Request Data Access
|
|
Interface->>Connector: Execute Query
|
|
Connector->>DB: SQL Query
|
|
DB-->>Connector: Raw Data
|
|
Connector-->>Interface: Raw Data
|
|
Interface->>Interface: Transform to Domain Objects
|
|
Interface-->>Feature: Domain Objects
|
|
Feature->>Feature: Process Business Logic
|
|
Feature-->>Route: Processed Result
|
|
Route->>Route: Serialize Response
|
|
Route-->>Client: HTTP Response
|
|
```
|
|
|
|
---
|
|
|
|
## Feature Lifecycle Management
|
|
|
|
Features that require background processing, scheduled tasks, or event-driven operations are managed through the **Features Lifecycle** system.
|
|
|
|
### Lifecycle Architecture
|
|
|
|
```mermaid
|
|
graph TB
|
|
subgraph "Application Startup"
|
|
App[app.py<br/>FastAPI Application]
|
|
Lifespan[Lifespan Context Manager]
|
|
end
|
|
|
|
subgraph "Features Lifecycle"
|
|
Lifecycle[featuresLifecycle.py]
|
|
Start[start function]
|
|
Stop[stop function]
|
|
end
|
|
|
|
subgraph "Background Features"
|
|
SyncDelta[SyncDelta Manager<br/>startSyncManager]
|
|
ChatAlthaus[ChatAlthaus Manager<br/>startDataScheduler]
|
|
AutomationEvents[Automation Events<br/>syncAutomationEvents]
|
|
end
|
|
|
|
App --> Lifespan
|
|
Lifespan -->|On Startup| Lifecycle
|
|
Lifecycle --> Start
|
|
Start --> SyncDelta
|
|
Start --> ChatAlthaus
|
|
Start --> AutomationEvents
|
|
|
|
Lifespan -->|On Shutdown| Lifecycle
|
|
Lifecycle --> Stop
|
|
Stop --> SyncDelta
|
|
Stop --> ChatAlthaus
|
|
|
|
style Lifecycle fill:#e8f5e9,stroke:#1b5e20
|
|
style SyncDelta fill:#fff3e0,stroke:#e65100
|
|
style ChatAlthaus fill:#fff3e0,stroke:#e65100
|
|
```
|
|
|
|
### Lifecycle Sequence
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant App as app.py
|
|
participant Lifespan as Lifespan Manager
|
|
participant Lifecycle as featuresLifecycle
|
|
participant EventUser as Event User
|
|
participant SyncDelta as SyncDelta Manager
|
|
participant ChatAlthaus as ChatAlthaus Manager
|
|
|
|
App->>Lifespan: Application Startup
|
|
Lifespan->>Lifecycle: start()
|
|
Lifecycle->>EventUser: getRootInterface().getUserByUsername("event")
|
|
EventUser-->>Lifecycle: Event User Object
|
|
|
|
Lifecycle->>ChatAlthaus: syncAutomationEvents()
|
|
ChatAlthaus-->>Lifecycle: Events Synced
|
|
|
|
Lifecycle->>SyncDelta: startSyncManager(eventUser)
|
|
SyncDelta->>SyncDelta: Initialize Background Thread
|
|
SyncDelta-->>Lifecycle: Manager Started
|
|
|
|
Lifecycle->>ChatAlthaus: startDataScheduler(eventUser)
|
|
ChatAlthaus->>ChatAlthaus: Initialize Scheduler
|
|
ChatAlthaus-->>Lifecycle: Scheduler Started
|
|
|
|
Lifecycle->>ChatAlthaus: performDataUpdate(eventUser)
|
|
ChatAlthaus-->>Lifecycle: Initial Update Complete
|
|
|
|
Lifecycle-->>Lifespan: Startup Complete
|
|
Lifespan-->>App: Application Ready
|
|
|
|
Note over App: Application Running...
|
|
|
|
App->>Lifespan: Application Shutdown
|
|
Lifespan->>Lifecycle: stop()
|
|
Lifecycle->>SyncDelta: Stop Manager
|
|
Lifecycle->>ChatAlthaus: Stop Scheduler
|
|
Lifecycle-->>Lifespan: Shutdown Complete
|
|
```
|
|
|
|
### Lifecycle-Managed Features
|
|
|
|
Features managed through the lifecycle system include:
|
|
|
|
1. **SyncDelta**: Background synchronization manager for ticket synchronization
|
|
2. **ChatAlthaus**: Scheduled data updates for Althaus preprocessing service
|
|
3. **Automation Events**: Event synchronization for chat automation
|
|
|
|
These features run continuously in the background and require proper initialization and cleanup during application startup and shutdown.
|
|
|
|
---
|
|
|
|
## Connectors in the Architecture
|
|
|
|
Connectors are the lowest-level abstraction for communicating with external systems. They provide concrete implementations for database connections, API integrations, and external service communication.
|
|
|
|
### Connector Architecture
|
|
|
|
```mermaid
|
|
graph TB
|
|
subgraph "Connector Types"
|
|
DBConn[Database Connectors<br/>connectorDbPostgre.py<br/>connectorDbJson.py]
|
|
TicketConn[Ticket Connectors<br/>connectorTicketsJira.py<br/>connectorTicketsClickup.py]
|
|
VoiceConn[Voice Connectors<br/>connectorVoiceGoogle.py]
|
|
end
|
|
|
|
subgraph "Connector Responsibilities"
|
|
Connection[Connection Management<br/>Establish & Maintain Connections]
|
|
Query[Query Execution<br/>Execute Queries & API Calls]
|
|
Transform[Data Transformation<br/>Raw Data ↔ Application Format]
|
|
Error[Error Handling<br/>Connection Errors & Retries]
|
|
end
|
|
|
|
subgraph "External Systems"
|
|
PostgreSQL[(PostgreSQL Database)]
|
|
JSONFile[JSON Files]
|
|
JiraAPI[Jira API]
|
|
ClickUpAPI[ClickUp API]
|
|
GoogleVoiceAPI[Google Voice API]
|
|
end
|
|
|
|
DBConn --> Connection
|
|
TicketConn --> Connection
|
|
VoiceConn --> Connection
|
|
|
|
Connection --> Query
|
|
Query --> Transform
|
|
Transform --> Error
|
|
|
|
DBConn --> PostgreSQL
|
|
DBConn --> JSONFile
|
|
TicketConn --> JiraAPI
|
|
TicketConn --> ClickUpAPI
|
|
VoiceConn --> GoogleVoiceAPI
|
|
|
|
style DBConn fill:#fce4ec,stroke:#880e4f
|
|
style TicketConn fill:#fce4ec,stroke:#880e4f
|
|
style VoiceConn fill:#fce4ec,stroke:#880e4f
|
|
```
|
|
|
|
### Connector Usage Flow
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant Feature as Feature
|
|
participant Service as Service
|
|
participant Interface as Interface
|
|
participant Connector as Connector
|
|
participant External as External System
|
|
|
|
Feature->>Service: Use Service
|
|
Service->>Interface: Request Data Access
|
|
Interface->>Connector: Initialize Connection
|
|
Connector->>External: Establish Connection
|
|
External-->>Connector: Connection Established
|
|
|
|
Interface->>Connector: Execute Query/API Call
|
|
Connector->>Connector: Format Request
|
|
Connector->>External: Send Request
|
|
External-->>Connector: Raw Response
|
|
Connector->>Connector: Parse Response
|
|
Connector-->>Interface: Formatted Data
|
|
Interface->>Interface: Transform to Domain Objects
|
|
Interface-->>Service: Domain Objects
|
|
Service-->>Feature: Processed Result
|
|
```
|
|
|
|
### Available Connectors
|
|
|
|
#### Database Connectors
|
|
|
|
**connectorDbPostgre.py** - PostgreSQL Database Connector
|
|
- Manages PostgreSQL database connections
|
|
- Executes SQL queries with parameterization
|
|
- Handles JSONB column types
|
|
- Provides transaction support
|
|
- Used by: Real Estate interfaces, Chat interfaces, Application interfaces
|
|
|
|
**connectorDbJson.py** - JSON File Database Connector
|
|
- Provides file-based data storage using JSON
|
|
- Useful for development and testing
|
|
- Lightweight alternative to PostgreSQL
|
|
- Used by: Development environments, Testing scenarios
|
|
|
|
#### Ticket Connectors
|
|
|
|
**connectorTicketsJira.py** - Jira Ticket Connector
|
|
- Integrates with Jira REST API
|
|
- Manages Jira tickets, issues, and projects
|
|
- Handles field mapping and synchronization
|
|
- Used by: SyncDelta feature, Ticket interfaces
|
|
|
|
**connectorTicketsClickup.py** - ClickUp Ticket Connector
|
|
- Integrates with ClickUp API
|
|
- Manages ClickUp tasks and lists
|
|
- Handles task synchronization
|
|
- Used by: Ticket interfaces
|
|
|
|
#### Voice Connectors
|
|
|
|
**connectorVoiceGoogle.py** - Google Voice Connector
|
|
- Integrates with Google Voice API
|
|
- Handles voice transcription and processing
|
|
- Manages voice data and audio files
|
|
- Used by: Voice-related services and features
|
|
|
|
### Connector Integration Pattern
|
|
|
|
Connectors are never directly accessed by features. Instead, they follow this integration pattern:
|
|
|
|
```mermaid
|
|
graph LR
|
|
A[Feature] --> B[Service]
|
|
B --> C[Interface]
|
|
C --> D[Connector]
|
|
D --> E[External System]
|
|
|
|
style A fill:#fff3e0,stroke:#e65100
|
|
style B fill:#e8f5e9,stroke:#1b5e20
|
|
style C fill:#e1f5ff,stroke:#01579b
|
|
style D fill:#fce4ec,stroke:#880e4f
|
|
style E fill:#f5f5f5,stroke:#424242
|
|
```
|
|
|
|
**Why This Pattern?**
|
|
- **Abstraction**: Interfaces hide connector implementation details
|
|
- **Flexibility**: Connectors can be swapped without affecting features
|
|
- **Testability**: Interfaces can be mocked for testing
|
|
- **Consistency**: All data access follows the same pattern
|
|
- **User Context**: Interfaces handle user context and access control
|
|
|
|
---
|
|
|
|
## Individual Features
|
|
|
|
### Real Estate Feature
|
|
|
|
**Location**: `modules/features/realEstate/mainRealEstate.py`
|
|
|
|
**Purpose**: Provides AI-powered natural language processing for Real Estate database operations. Enables users to interact with Real Estate data using natural language commands that are translated into CRUD operations.
|
|
|
|
**Architecture**:
|
|
|
|
```mermaid
|
|
graph TB
|
|
subgraph "Real Estate Feature"
|
|
Route[routeRealEstate.py]
|
|
Feature[mainRealEstate.py]
|
|
Intent[Intent Analysis<br/>analyzeUserIntent]
|
|
CRUD[CRUD Operations<br/>executeIntentBasedOperation]
|
|
Query[Direct Queries<br/>executeDirectQuery]
|
|
end
|
|
|
|
subgraph "Dependencies"
|
|
AIService[AI Service<br/>Intent Recognition]
|
|
REInterface[Real Estate Interface<br/>interfaceDbRealEstateObjects.py]
|
|
DBConnector[Database Connector<br/>connectorDbPostgre.py]
|
|
end
|
|
|
|
subgraph "Data Models"
|
|
REModels[Real Estate Models<br/>Projekt, Parzelle, etc.]
|
|
end
|
|
|
|
Route --> Feature
|
|
Feature --> Intent
|
|
Feature --> CRUD
|
|
Feature --> Query
|
|
|
|
Intent --> AIService
|
|
CRUD --> REInterface
|
|
Query --> REInterface
|
|
|
|
REInterface --> DBConnector
|
|
REInterface --> REModels
|
|
|
|
style Feature fill:#fff3e0,stroke:#e65100
|
|
```
|
|
|
|
**Key Functions**:
|
|
- `processNaturalLanguageCommand()`: Main entry point for natural language processing
|
|
- `analyzeUserIntent()`: Uses AI to analyze user input and extract intent, entity, and parameters
|
|
- `executeIntentBasedOperation()`: Executes CRUD operations based on analyzed intent
|
|
- `executeDirectQuery()`: Executes direct SQL queries without AI processing
|
|
|
|
**Connector Usage**:
|
|
- Uses **Database Connector** (`connectorDbPostgre.py`) through the Real Estate Interface
|
|
- Accesses PostgreSQL database for Real Estate data
|
|
- Handles CRUD operations on entities like Projekt, Parzelle, Dokument
|
|
|
|
**Service Integration**:
|
|
- Uses **AI Service** for intent recognition and natural language understanding
|
|
- Leverages AI planning capabilities to analyze user commands
|
|
|
|
**Migration Path**:
|
|
- May evolve into a **Service** if Real Estate operations become reusable across domains
|
|
- Could integrate with **Workflows** for complex multi-step Real Estate processes
|
|
|
|
---
|
|
|
|
### Chat Althaus Feature
|
|
|
|
**Location**: `modules/features/chatAlthaus/mainChatAlthaus.py`
|
|
|
|
**Purpose**: Manages scheduled data updates for the Althaus preprocessing service. Triggers daily updates to synchronize database configuration with external preprocessing service.
|
|
|
|
**Architecture**:
|
|
|
|
```mermaid
|
|
graph TB
|
|
subgraph "Chat Althaus Feature"
|
|
Lifecycle[featuresLifecycle.py]
|
|
Manager[ManagerChatAlthaus]
|
|
Scheduler[Data Scheduler]
|
|
Updater[Data Updater<br/>updateDatabaseWithConfig]
|
|
end
|
|
|
|
subgraph "Dependencies"
|
|
Services[Services Container]
|
|
HTTPClient[HTTP Client<br/>aiohttp]
|
|
Config[Configuration<br/>APP_CONFIG]
|
|
end
|
|
|
|
subgraph "External System"
|
|
AlthausAPI[Althaus Preprocessing API<br/>Azure Function]
|
|
end
|
|
|
|
Lifecycle --> Manager
|
|
Manager --> Scheduler
|
|
Manager --> Updater
|
|
|
|
Updater --> Services
|
|
Updater --> HTTPClient
|
|
Updater --> Config
|
|
|
|
Updater --> AlthausAPI
|
|
|
|
style Manager fill:#fff3e0,stroke:#e65100
|
|
```
|
|
|
|
**Key Functions**:
|
|
- `startDataScheduler()`: Initializes and starts the scheduled data update manager
|
|
- `performDataUpdate()`: Executes immediate data update
|
|
- `updateDatabaseWithConfig()`: Sends configuration to Althaus preprocessing service
|
|
|
|
**Scheduling**:
|
|
- Runs daily at 01:00 UTC
|
|
- Uses background scheduler for automated execution
|
|
- Managed through Features Lifecycle system
|
|
|
|
**Connector Usage**:
|
|
- Uses **HTTP Client** (aiohttp) for API communication
|
|
- No database connector (uses external API)
|
|
|
|
**Service Integration**:
|
|
- Uses **Services Container** for configuration access
|
|
- Leverages shared configuration utilities
|
|
|
|
**Migration Path**:
|
|
- Could become a **Service** if data synchronization becomes a core capability
|
|
- May integrate with **Workflows** for complex data processing pipelines
|
|
|
|
---
|
|
|
|
### Sync Delta Feature
|
|
|
|
**Location**: `modules/features/syncDelta/mainSyncDelta.py`
|
|
|
|
**Purpose**: Synchronizes tickets between Jira and SharePoint. Manages bidirectional synchronization of ticket data, supporting both CSV and Excel file formats.
|
|
|
|
**Architecture**:
|
|
|
|
```mermaid
|
|
graph TB
|
|
subgraph "Sync Delta Feature"
|
|
Lifecycle[featuresLifecycle.py]
|
|
Manager[ManagerSyncDelta]
|
|
Sync[syncTicketsOverSharepoint]
|
|
Merge[Data Merging Logic]
|
|
Audit[Audit Logging]
|
|
end
|
|
|
|
subgraph "Dependencies"
|
|
TicketService[Ticket Service]
|
|
SharePointService[SharePoint Service]
|
|
JiraConnector[Jira Connector<br/>connectorTicketsJira.py]
|
|
end
|
|
|
|
subgraph "External Systems"
|
|
Jira[Jira API]
|
|
SharePoint[SharePoint API]
|
|
end
|
|
|
|
Lifecycle --> Manager
|
|
Manager --> Sync
|
|
Sync --> Merge
|
|
Sync --> Audit
|
|
|
|
Sync --> TicketService
|
|
Sync --> SharePointService
|
|
|
|
TicketService --> JiraConnector
|
|
JiraConnector --> Jira
|
|
SharePointService --> SharePoint
|
|
|
|
style Manager fill:#fff3e0,stroke:#e65100
|
|
```
|
|
|
|
**Key Functions**:
|
|
- `startSyncManager()`: Initializes background synchronization manager
|
|
- `syncTicketsOverSharepoint()`: Performs synchronization between Jira and SharePoint
|
|
- `initializeInterface()`: Sets up connectors and validates connections
|
|
- `_logAuditEvent()`: Logs synchronization events for auditing
|
|
|
|
**Synchronization Modes**:
|
|
- **CSV Mode**: Uses CSV files for data exchange
|
|
- **Excel Mode**: Uses Excel (.xlsx) files for data exchange
|
|
|
|
**Connector Usage**:
|
|
- Uses **Jira Connector** (`connectorTicketsJira.py`) through Ticket Service
|
|
- Uses **SharePoint Service** for file operations
|
|
- Manages field mapping between Jira and SharePoint formats
|
|
|
|
**Service Integration**:
|
|
- Uses **Ticket Service** for ticket interface creation
|
|
- Uses **SharePoint Service** for file upload/download
|
|
- Leverages **Services Container** for configuration and utilities
|
|
|
|
**Migration Path**:
|
|
- Likely candidate for **Service** migration as ticket synchronization becomes core functionality
|
|
- Could integrate with **Workflows** for complex synchronization scenarios
|
|
|
|
---
|
|
|
|
### Chat Playground Feature
|
|
|
|
**Location**: `modules/features/chatPlayground/mainChatPlayground.py`
|
|
|
|
**Purpose**: Provides entry point for chat workflow functionality. Acts as a thin wrapper around the WorkflowManager for chat-based interactions.
|
|
|
|
**Architecture**:
|
|
|
|
```mermaid
|
|
graph TB
|
|
subgraph "Chat Playground Feature"
|
|
Route[routeChatPlayground.py]
|
|
Feature[mainChatPlayground.py]
|
|
Start[chatStart]
|
|
Stop[chatStop]
|
|
end
|
|
|
|
subgraph "Workflow System"
|
|
WorkflowManager[WorkflowManager]
|
|
WorkflowProcessor[WorkflowProcessor]
|
|
Methods[Workflow Methods]
|
|
end
|
|
|
|
subgraph "Dependencies"
|
|
Services[Services Container]
|
|
end
|
|
|
|
Route --> Feature
|
|
Feature --> Start
|
|
Feature --> Stop
|
|
|
|
Start --> WorkflowManager
|
|
Stop --> WorkflowManager
|
|
|
|
WorkflowManager --> WorkflowProcessor
|
|
WorkflowProcessor --> Methods
|
|
|
|
WorkflowManager --> Services
|
|
|
|
style Feature fill:#fff3e0,stroke:#e65100
|
|
style WorkflowManager fill:#e1f5ff,stroke:#01579b
|
|
```
|
|
|
|
**Key Functions**:
|
|
- `chatStart()`: Starts a new chat workflow or continues an existing one
|
|
- `chatStop()`: Stops a running chat workflow
|
|
|
|
**Relationship to Workflows**:
|
|
- This feature is a **bridge** between routes and the workflow system
|
|
- Delegates all processing to WorkflowManager
|
|
- Demonstrates how features can integrate with workflows
|
|
|
|
**Connector Usage**:
|
|
- No direct connector usage (delegates to workflows)
|
|
- Workflows use connectors through services and methods
|
|
|
|
**Service Integration**:
|
|
- Uses **Services Container** for workflow management
|
|
- Leverages workflow services for chat operations
|
|
|
|
**Migration Path**:
|
|
- Already integrated with **Workflows** system
|
|
- May be simplified or removed as workflows become the primary interface
|
|
|
|
---
|
|
|
|
### Neutralize Playground Feature
|
|
|
|
**Location**: `modules/features/neutralizePlayground/mainNeutralizePlayground.py`
|
|
|
|
**Purpose**: Provides a playground interface for data neutralization functionality. Wraps the Neutralization Service for testing and experimentation.
|
|
|
|
**Architecture**:
|
|
|
|
```mermaid
|
|
graph TB
|
|
subgraph "Neutralize Playground Feature"
|
|
Route[routeDataNeutralization.py]
|
|
Feature[NeutralizationPlayground]
|
|
ProcessText[processText]
|
|
ProcessFiles[processFiles]
|
|
CleanAttributes[cleanAttributes]
|
|
Stats[getStats]
|
|
Config[getConfig/saveConfig]
|
|
end
|
|
|
|
subgraph "Dependencies"
|
|
NeutralizationService[Neutralization Service]
|
|
end
|
|
|
|
Route --> Feature
|
|
Feature --> ProcessText
|
|
Feature --> ProcessFiles
|
|
Feature --> CleanAttributes
|
|
Feature --> Stats
|
|
Feature --> Config
|
|
|
|
ProcessText --> NeutralizationService
|
|
ProcessFiles --> NeutralizationService
|
|
CleanAttributes --> NeutralizationService
|
|
Stats --> NeutralizationService
|
|
Config --> NeutralizationService
|
|
|
|
style Feature fill:#fff3e0,stroke:#e65100
|
|
style NeutralizationService fill:#e8f5e9,stroke:#1b5e20
|
|
```
|
|
|
|
**Key Functions**:
|
|
- `processText()`: Processes text for data neutralization
|
|
- `processFiles()`: Processes files for data neutralization
|
|
- `cleanAttributes()`: Cleans neutralization attributes
|
|
- `getStats()`: Retrieves neutralization statistics
|
|
- `getConfig()` / `saveConfig()`: Manages neutralization configuration
|
|
|
|
**Purpose as Playground**:
|
|
- Provides testing interface for neutralization functionality
|
|
- Allows experimentation with neutralization patterns
|
|
- Demonstrates service usage patterns
|
|
|
|
**Connector Usage**:
|
|
- No direct connector usage (uses Neutralization Service)
|
|
- Service handles all data access internally
|
|
|
|
**Service Integration**:
|
|
- Wraps **Neutralization Service** for easy access
|
|
- Provides playground-specific functionality
|
|
|
|
**Migration Path**:
|
|
- May be removed once neutralization is fully integrated
|
|
- Functionality may move directly to routes using the service
|
|
|
|
---
|
|
|
|
## Feature Patterns and Best Practices
|
|
|
|
### Pattern 1: Stateless Feature Design
|
|
|
|
Features should be stateless and operate without session management. Each request should be independent and self-contained.
|
|
|
|
```mermaid
|
|
graph LR
|
|
A[Request] --> B[Feature Function]
|
|
B --> C[Process Request]
|
|
C --> D[Return Result]
|
|
|
|
style B fill:#fff3e0,stroke:#e65100
|
|
```
|
|
|
|
**Benefits**:
|
|
- Simpler implementation
|
|
- Better scalability
|
|
- Easier testing
|
|
- No state management overhead
|
|
|
|
### Pattern 2: Service Delegation
|
|
|
|
Features should delegate cross-cutting functionality to services rather than implementing it directly.
|
|
|
|
```mermaid
|
|
graph TB
|
|
A[Feature] --> B{Needs Functionality}
|
|
B -->|AI Processing| C[AI Service]
|
|
B -->|Data Access| D[Interface]
|
|
B -->|File Operations| E[File Service]
|
|
B -->|Other| F[Other Services]
|
|
|
|
style A fill:#fff3e0,stroke:#e65100
|
|
style C fill:#e8f5e9,stroke:#1b5e20
|
|
style D fill:#e1f5ff,stroke:#01579b
|
|
```
|
|
|
|
**Benefits**:
|
|
- Code reuse
|
|
- Consistent behavior
|
|
- Easier maintenance
|
|
- Better separation of concerns
|
|
|
|
### Pattern 3: Interface Abstraction
|
|
|
|
Features should never directly access connectors. All data access should go through interfaces.
|
|
|
|
```mermaid
|
|
graph LR
|
|
A[Feature] --> B[Interface]
|
|
B --> C[Connector]
|
|
C --> D[External System]
|
|
|
|
style A fill:#fff3e0,stroke:#e65100
|
|
style B fill:#e1f5ff,stroke:#01579b
|
|
style C fill:#fce4ec,stroke:#880e4f
|
|
```
|
|
|
|
**Benefits**:
|
|
- Abstraction of implementation details
|
|
- Flexibility to change connectors
|
|
- Consistent data access patterns
|
|
- User context handling
|
|
|
|
### Pattern 4: Background Processing
|
|
|
|
Features requiring background processing should use the Features Lifecycle system.
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant App as Application
|
|
participant Lifecycle as Features Lifecycle
|
|
participant Feature as Background Feature
|
|
participant Scheduler as Scheduler
|
|
|
|
App->>Lifecycle: Startup
|
|
Lifecycle->>Feature: Initialize Manager
|
|
Feature->>Scheduler: Start Background Task
|
|
Scheduler-->>Feature: Task Running
|
|
|
|
Note over Feature,Scheduler: Background Processing...
|
|
|
|
App->>Lifecycle: Shutdown
|
|
Lifecycle->>Feature: Stop Manager
|
|
Feature->>Scheduler: Stop Background Task
|
|
Scheduler-->>Feature: Task Stopped
|
|
```
|
|
|
|
**Benefits**:
|
|
- Proper lifecycle management
|
|
- Clean startup/shutdown
|
|
- Resource management
|
|
- Error handling
|
|
|
|
### Pattern 5: Migration Planning
|
|
|
|
Features should be designed with migration in mind. Consider future migration to services or workflows during design.
|
|
|
|
```mermaid
|
|
graph LR
|
|
A[Feature Design] --> B{Consider Migration}
|
|
B -->|Reusable Logic| C[Design as Service]
|
|
B -->|Complex Flow| D[Design for Workflow]
|
|
B -->|Temporary| E[Keep as Feature]
|
|
|
|
style A fill:#fff3e0,stroke:#e65100
|
|
style C fill:#e8f5e9,stroke:#1b5e20
|
|
style D fill:#e1f5ff,stroke:#01579b
|
|
```
|
|
|
|
**Best Practices**:
|
|
- Document migration path
|
|
- Keep dependencies minimal
|
|
- Use standard patterns
|
|
- Plan for refactoring
|
|
|
|
---
|
|
|
|
## Summary
|
|
|
|
The Features component provides a flexible, domain-specific business logic layer that enables rapid development while maintaining architectural boundaries. Features serve as temporary solutions that bridge the gap between initial requirements and full service or workflow implementation.
|
|
|
|
**Key Takeaways**:
|
|
|
|
1. **Features are Temporary**: Designed to be migrated to services or workflows as they mature
|
|
2. **Domain-Specific**: Each feature addresses a specific business domain or use case
|
|
3. **Service-Dependent**: Features leverage services for cross-cutting functionality
|
|
4. **Interface-Abstracted**: Features access data through interfaces, never directly through connectors
|
|
5. **Lifecycle-Managed**: Background features are managed through the Features Lifecycle system
|
|
6. **Connector Integration**: Connectors are accessed through interfaces, providing abstraction and flexibility
|
|
|
|
The architecture supports a natural evolution path from features to services or workflows, ensuring that the codebase remains maintainable and scalable as functionality matures.
|
|
|